brett.cannon
2008-11-25 19:19:17 UTC
Author: brett.cannon
Date: Tue Nov 25 20:19:17 2008
New Revision: 67380
Log:
dbm.gnu and dbm.ndbm accept both strings and bytes as keys and values. For the
former they are converted to bytes before being written to the DB.
Closes issue 3799. Reviewed by Skip Montanaro.
Modified:
python/branches/py3k/Doc/library/dbm.rst
python/branches/py3k/Lib/test/test_dbm_dumb.py
python/branches/py3k/Lib/test/test_dbm_gnu.py
python/branches/py3k/Lib/test/test_dbm_ndbm.py
python/branches/py3k/Misc/NEWS
python/branches/py3k/Modules/_dbmmodule.c
python/branches/py3k/Modules/_gdbmmodule.c
Modified: python/branches/py3k/Doc/library/dbm.rst
==============================================================================
--- python/branches/py3k/Doc/library/dbm.rst (original)
+++ python/branches/py3k/Doc/library/dbm.rst Tue Nov 25 20:19:17 2008
@@ -52,7 +52,9 @@
The object returned by :func:`open` supports most of the same functionality as
dictionaries; keys and their corresponding values can be stored, retrieved, and
deleted, and the :keyword:`in` operator and the :meth:`keys` method are
-available. Keys and values must always be strings.
+available. Key and values are always stored as bytes. This means that when
+strings are used they are implicitly converted to the default encoding before
+being stored.
The following example records some hostnames and a corresponding title, and
then prints out the contents of the database::
@@ -63,9 +65,15 @@
db = dbm.open('cache', 'c')
# Record some values
+ db[b'hello'] = b'there'
db['www.python.org'] = 'Python Website'
db['www.cnn.com'] = 'Cable News Network'
+ # Note that the keys are considered bytes now.
+ assert db[b'www.python.org'] == b'Python Website'
+ # Notice how the value is now in bytes.
+ assert db['www.cnn.com'] == b'Cable News Network'
+
# Loop through contents. Other dictionary methods
# such as .keys(), .values() also work.
for k, v in db.iteritems():
@@ -98,17 +106,18 @@
This module is quite similar to the :mod:`dbm` module, but uses the GNU library
``gdbm`` instead to provide some additional functionality. Please note that the
-file formats created by ``gdbm`` and ``dbm`` are incompatible.
+file formats created by :mod:`dbm.gnu` and :mod:`dbm.ndbm` are incompatible.
The :mod:`dbm.gnu` module provides an interface to the GNU DBM library.
-``gdbm`` objects behave like mappings (dictionaries), except that keys and
-values are always strings. Printing a :mod:`dbm.gnu` object doesn't print the
+``dbm.gnu.gdbm`` objects behave like mappings (dictionaries), except that keys and
+values are always converted to bytes before storing. Printing a ``gdbm``
+object doesn't print the
keys and values, and the :meth:`items` and :meth:`values` methods are not
supported.
.. exception:: error
- Raised on ``gdbm``\ -specific errors, such as I/O errors. :exc:`KeyError` is
+ Raised on :mod:`dbm.gnu`-specific errors, such as I/O errors. :exc:`KeyError` is
raised for general mapping errors like specifying an incorrect key.
@@ -183,7 +192,7 @@
If you have carried out a lot of deletions and would like to shrink the space
used by the ``gdbm`` file, this routine will reorganize the database. ``gdbm``
- will not shorten the length of a database file except by using this
+ objects will not shorten the length of a database file except by using this
reorganization; otherwise, deleted file space will be kept and reused as new
(key, value) pairs are added.
@@ -203,8 +212,8 @@
The :mod:`dbm.ndbm` module provides an interface to the Unix "(n)dbm" library.
Dbm objects behave like mappings (dictionaries), except that keys and values are
-always strings. Printing a dbm object doesn't print the keys and values, and the
-:meth:`items` and :meth:`values` methods are not supported.
+always stored as bytes. Printing a ``dbm`` object doesn't print the keys and
+values, and the :meth:`items` and :meth:`values` methods are not supported.
This module can be used with the "classic" ndbm interface, the BSD DB
compatibility interface, or the GNU GDBM compatibility interface. On Unix, the
@@ -213,7 +222,7 @@
.. exception:: error
- Raised on dbm-specific errors, such as I/O errors. :exc:`KeyError` is raised
+ Raised on :mod:`dbm.ndbm`-specific errors, such as I/O errors. :exc:`KeyError` is raised
for general mapping errors like specifying an incorrect key.
@@ -224,7 +233,7 @@
.. function:: open(filename[, flag[, mode]])
- Open a dbm database and return a dbm object. The *filename* argument is the
+ Open a dbm database and return a ``dbm`` object. The *filename* argument is the
name of the database file (without the :file:`.dir` or :file:`.pag` extensions;
note that the BSD DB implementation of the interface will append the extension
:file:`.db` and only create one file).
@@ -264,27 +273,27 @@
.. note::
The :mod:`dbm.dumb` module is intended as a last resort fallback for the
- :mod:`dbm` module when no more robust module is available. The :mod:`dbm.dumb`
+ :mod:`dbm` module when a more robust module is not available. The :mod:`dbm.dumb`
module is not written for speed and is not nearly as heavily used as the other
database modules.
The :mod:`dbm.dumb` module provides a persistent dictionary-like interface which
-is written entirely in Python. Unlike other modules such as :mod:`gdbm` no
+is written entirely in Python. Unlike other modules such as :mod:`dbm.gnu` no
external library is required. As with other persistent mappings, the keys and
-values must always be strings.
+values are always stored as bytes.
The module defines the following:
.. exception:: error
- Raised on dbm.dumb-specific errors, such as I/O errors. :exc:`KeyError` is
+ Raised on :mod:`dbm.dumb`-specific errors, such as I/O errors. :exc:`KeyError` is
raised for general mapping errors like specifying an incorrect key.
.. function:: open(filename[, flag[, mode]])
- Open a dumbdbm database and return a dumbdbm object. The *filename* argument is
+ Open a ``dumbdbm`` database and return a dumbdbm object. The *filename* argument is
the basename of the database file (without any specific extensions). When a
dumbdbm database is created, files with :file:`.dat` and :file:`.dir` extensions
are created.
Modified: python/branches/py3k/Lib/test/test_dbm_dumb.py
==============================================================================
--- python/branches/py3k/Lib/test/test_dbm_dumb.py (original)
+++ python/branches/py3k/Lib/test/test_dbm_dumb.py Tue Nov 25 20:19:17 2008
@@ -115,11 +115,13 @@
self.init_db()
f = dumbdbm.open(_fname)
f['\u00fc'] = b'!'
+ f['1'] = 'a'
f.close()
f = dumbdbm.open(_fname, 'r')
self.assert_('\u00fc' in f)
self.assertEqual(f['\u00fc'.encode('utf-8')],
self._dict['\u00fc'.encode('utf-8')])
+ self.assertEqual(f[b'1'], b'a')
def test_line_endings(self):
# test for bug #1172763: dumbdbm would die if the line endings
Modified: python/branches/py3k/Lib/test/test_dbm_gnu.py
==============================================================================
--- python/branches/py3k/Lib/test/test_dbm_gnu.py (original)
+++ python/branches/py3k/Lib/test/test_dbm_gnu.py Tue Nov 25 20:19:17 2008
@@ -20,9 +20,11 @@
self.assertEqual(self.g.keys(), [])
self.g['a'] = 'b'
self.g['12345678910'] = '019237410982340912840198242'
+ self.g[b'bytes'] = b'data'
key_set = set(self.g.keys())
self.assertEqual(key_set, set([b'a', b'12345678910']))
self.assert_(b'a' in self.g)
+ self.assertEqual(self.g[b'bytes'], b'data')
key = self.g.firstkey()
while key:
self.assert_(key in key_set)
Modified: python/branches/py3k/Lib/test/test_dbm_ndbm.py
==============================================================================
--- python/branches/py3k/Lib/test/test_dbm_ndbm.py (original)
+++ python/branches/py3k/Lib/test/test_dbm_ndbm.py Tue Nov 25 20:19:17 2008
@@ -20,9 +20,11 @@
self.d = dbm.ndbm.open(self.filename, 'c')
self.assert_(self.d.keys() == [])
self.d['a'] = 'b'
+ self.d[b'bytes'] = b'data'
self.d['12345678910'] = '019237410982340912840198242'
self.d.keys()
self.assert_(b'a' in self.d)
+ self.assertEqual(self.d[b'bytes'], b'data')
self.d.close()
def test_modes(self):
Modified: python/branches/py3k/Misc/NEWS
==============================================================================
--- python/branches/py3k/Misc/NEWS (original)
+++ python/branches/py3k/Misc/NEWS Tue Nov 25 20:19:17 2008
@@ -28,6 +28,12 @@
- Issue #4383: When IDLE cannot make the connection to its subprocess, it would
fail to properly display the error message.
+Docs
+----
+
+- Issue #3799: Document that dbm.gnu and dbm.ndbm will accept string arguments
+ for keys and values which will be converted to bytes before committal.
+
What's New in Python 3.0 release candidate 3?
=============================================
Modified: python/branches/py3k/Modules/_dbmmodule.c
==============================================================================
--- python/branches/py3k/Modules/_dbmmodule.c (original)
+++ python/branches/py3k/Modules/_dbmmodule.c Tue Nov 25 20:19:17 2008
@@ -122,7 +122,7 @@
if ( !PyArg_Parse(v, "s#", &krec.dptr, &tmp_size) ) {
PyErr_SetString(PyExc_TypeError,
- "dbm mappings have string keys only");
+ "dbm mappings have bytes or string keys only");
return -1;
}
krec.dsize = tmp_size;
@@ -140,7 +140,7 @@
} else {
if ( !PyArg_Parse(w, "s#", &drec.dptr, &tmp_size) ) {
PyErr_SetString(PyExc_TypeError,
- "dbm mappings have byte string elements only");
+ "dbm mappings have byte or string elements only");
return -1;
}
drec.dsize = tmp_size;
Modified: python/branches/py3k/Modules/_gdbmmodule.c
==============================================================================
--- python/branches/py3k/Modules/_gdbmmodule.c (original)
+++ python/branches/py3k/Modules/_gdbmmodule.c Tue Nov 25 20:19:17 2008
@@ -142,7 +142,7 @@
if (!PyArg_Parse(v, "s#", &krec.dptr, &krec.dsize) ) {
PyErr_SetString(PyExc_TypeError,
- "gdbm mappings have string indices only");
+ "gdbm mappings have bytes or string indices only");
return -1;
}
if (dp->di_dbm == NULL) {
@@ -160,7 +160,7 @@
else {
if (!PyArg_Parse(w, "s#", &drec.dptr, &drec.dsize)) {
PyErr_SetString(PyExc_TypeError,
- "gdbm mappings have byte string elements only");
+ "gdbm mappings have byte or string elements only");
return -1;
}
errno = 0;
Date: Tue Nov 25 20:19:17 2008
New Revision: 67380
Log:
dbm.gnu and dbm.ndbm accept both strings and bytes as keys and values. For the
former they are converted to bytes before being written to the DB.
Closes issue 3799. Reviewed by Skip Montanaro.
Modified:
python/branches/py3k/Doc/library/dbm.rst
python/branches/py3k/Lib/test/test_dbm_dumb.py
python/branches/py3k/Lib/test/test_dbm_gnu.py
python/branches/py3k/Lib/test/test_dbm_ndbm.py
python/branches/py3k/Misc/NEWS
python/branches/py3k/Modules/_dbmmodule.c
python/branches/py3k/Modules/_gdbmmodule.c
Modified: python/branches/py3k/Doc/library/dbm.rst
==============================================================================
--- python/branches/py3k/Doc/library/dbm.rst (original)
+++ python/branches/py3k/Doc/library/dbm.rst Tue Nov 25 20:19:17 2008
@@ -52,7 +52,9 @@
The object returned by :func:`open` supports most of the same functionality as
dictionaries; keys and their corresponding values can be stored, retrieved, and
deleted, and the :keyword:`in` operator and the :meth:`keys` method are
-available. Keys and values must always be strings.
+available. Key and values are always stored as bytes. This means that when
+strings are used they are implicitly converted to the default encoding before
+being stored.
The following example records some hostnames and a corresponding title, and
then prints out the contents of the database::
@@ -63,9 +65,15 @@
db = dbm.open('cache', 'c')
# Record some values
+ db[b'hello'] = b'there'
db['www.python.org'] = 'Python Website'
db['www.cnn.com'] = 'Cable News Network'
+ # Note that the keys are considered bytes now.
+ assert db[b'www.python.org'] == b'Python Website'
+ # Notice how the value is now in bytes.
+ assert db['www.cnn.com'] == b'Cable News Network'
+
# Loop through contents. Other dictionary methods
# such as .keys(), .values() also work.
for k, v in db.iteritems():
@@ -98,17 +106,18 @@
This module is quite similar to the :mod:`dbm` module, but uses the GNU library
``gdbm`` instead to provide some additional functionality. Please note that the
-file formats created by ``gdbm`` and ``dbm`` are incompatible.
+file formats created by :mod:`dbm.gnu` and :mod:`dbm.ndbm` are incompatible.
The :mod:`dbm.gnu` module provides an interface to the GNU DBM library.
-``gdbm`` objects behave like mappings (dictionaries), except that keys and
-values are always strings. Printing a :mod:`dbm.gnu` object doesn't print the
+``dbm.gnu.gdbm`` objects behave like mappings (dictionaries), except that keys and
+values are always converted to bytes before storing. Printing a ``gdbm``
+object doesn't print the
keys and values, and the :meth:`items` and :meth:`values` methods are not
supported.
.. exception:: error
- Raised on ``gdbm``\ -specific errors, such as I/O errors. :exc:`KeyError` is
+ Raised on :mod:`dbm.gnu`-specific errors, such as I/O errors. :exc:`KeyError` is
raised for general mapping errors like specifying an incorrect key.
@@ -183,7 +192,7 @@
If you have carried out a lot of deletions and would like to shrink the space
used by the ``gdbm`` file, this routine will reorganize the database. ``gdbm``
- will not shorten the length of a database file except by using this
+ objects will not shorten the length of a database file except by using this
reorganization; otherwise, deleted file space will be kept and reused as new
(key, value) pairs are added.
@@ -203,8 +212,8 @@
The :mod:`dbm.ndbm` module provides an interface to the Unix "(n)dbm" library.
Dbm objects behave like mappings (dictionaries), except that keys and values are
-always strings. Printing a dbm object doesn't print the keys and values, and the
-:meth:`items` and :meth:`values` methods are not supported.
+always stored as bytes. Printing a ``dbm`` object doesn't print the keys and
+values, and the :meth:`items` and :meth:`values` methods are not supported.
This module can be used with the "classic" ndbm interface, the BSD DB
compatibility interface, or the GNU GDBM compatibility interface. On Unix, the
@@ -213,7 +222,7 @@
.. exception:: error
- Raised on dbm-specific errors, such as I/O errors. :exc:`KeyError` is raised
+ Raised on :mod:`dbm.ndbm`-specific errors, such as I/O errors. :exc:`KeyError` is raised
for general mapping errors like specifying an incorrect key.
@@ -224,7 +233,7 @@
.. function:: open(filename[, flag[, mode]])
- Open a dbm database and return a dbm object. The *filename* argument is the
+ Open a dbm database and return a ``dbm`` object. The *filename* argument is the
name of the database file (without the :file:`.dir` or :file:`.pag` extensions;
note that the BSD DB implementation of the interface will append the extension
:file:`.db` and only create one file).
@@ -264,27 +273,27 @@
.. note::
The :mod:`dbm.dumb` module is intended as a last resort fallback for the
- :mod:`dbm` module when no more robust module is available. The :mod:`dbm.dumb`
+ :mod:`dbm` module when a more robust module is not available. The :mod:`dbm.dumb`
module is not written for speed and is not nearly as heavily used as the other
database modules.
The :mod:`dbm.dumb` module provides a persistent dictionary-like interface which
-is written entirely in Python. Unlike other modules such as :mod:`gdbm` no
+is written entirely in Python. Unlike other modules such as :mod:`dbm.gnu` no
external library is required. As with other persistent mappings, the keys and
-values must always be strings.
+values are always stored as bytes.
The module defines the following:
.. exception:: error
- Raised on dbm.dumb-specific errors, such as I/O errors. :exc:`KeyError` is
+ Raised on :mod:`dbm.dumb`-specific errors, such as I/O errors. :exc:`KeyError` is
raised for general mapping errors like specifying an incorrect key.
.. function:: open(filename[, flag[, mode]])
- Open a dumbdbm database and return a dumbdbm object. The *filename* argument is
+ Open a ``dumbdbm`` database and return a dumbdbm object. The *filename* argument is
the basename of the database file (without any specific extensions). When a
dumbdbm database is created, files with :file:`.dat` and :file:`.dir` extensions
are created.
Modified: python/branches/py3k/Lib/test/test_dbm_dumb.py
==============================================================================
--- python/branches/py3k/Lib/test/test_dbm_dumb.py (original)
+++ python/branches/py3k/Lib/test/test_dbm_dumb.py Tue Nov 25 20:19:17 2008
@@ -115,11 +115,13 @@
self.init_db()
f = dumbdbm.open(_fname)
f['\u00fc'] = b'!'
+ f['1'] = 'a'
f.close()
f = dumbdbm.open(_fname, 'r')
self.assert_('\u00fc' in f)
self.assertEqual(f['\u00fc'.encode('utf-8')],
self._dict['\u00fc'.encode('utf-8')])
+ self.assertEqual(f[b'1'], b'a')
def test_line_endings(self):
# test for bug #1172763: dumbdbm would die if the line endings
Modified: python/branches/py3k/Lib/test/test_dbm_gnu.py
==============================================================================
--- python/branches/py3k/Lib/test/test_dbm_gnu.py (original)
+++ python/branches/py3k/Lib/test/test_dbm_gnu.py Tue Nov 25 20:19:17 2008
@@ -20,9 +20,11 @@
self.assertEqual(self.g.keys(), [])
self.g['a'] = 'b'
self.g['12345678910'] = '019237410982340912840198242'
+ self.g[b'bytes'] = b'data'
key_set = set(self.g.keys())
self.assertEqual(key_set, set([b'a', b'12345678910']))
self.assert_(b'a' in self.g)
+ self.assertEqual(self.g[b'bytes'], b'data')
key = self.g.firstkey()
while key:
self.assert_(key in key_set)
Modified: python/branches/py3k/Lib/test/test_dbm_ndbm.py
==============================================================================
--- python/branches/py3k/Lib/test/test_dbm_ndbm.py (original)
+++ python/branches/py3k/Lib/test/test_dbm_ndbm.py Tue Nov 25 20:19:17 2008
@@ -20,9 +20,11 @@
self.d = dbm.ndbm.open(self.filename, 'c')
self.assert_(self.d.keys() == [])
self.d['a'] = 'b'
+ self.d[b'bytes'] = b'data'
self.d['12345678910'] = '019237410982340912840198242'
self.d.keys()
self.assert_(b'a' in self.d)
+ self.assertEqual(self.d[b'bytes'], b'data')
self.d.close()
def test_modes(self):
Modified: python/branches/py3k/Misc/NEWS
==============================================================================
--- python/branches/py3k/Misc/NEWS (original)
+++ python/branches/py3k/Misc/NEWS Tue Nov 25 20:19:17 2008
@@ -28,6 +28,12 @@
- Issue #4383: When IDLE cannot make the connection to its subprocess, it would
fail to properly display the error message.
+Docs
+----
+
+- Issue #3799: Document that dbm.gnu and dbm.ndbm will accept string arguments
+ for keys and values which will be converted to bytes before committal.
+
What's New in Python 3.0 release candidate 3?
=============================================
Modified: python/branches/py3k/Modules/_dbmmodule.c
==============================================================================
--- python/branches/py3k/Modules/_dbmmodule.c (original)
+++ python/branches/py3k/Modules/_dbmmodule.c Tue Nov 25 20:19:17 2008
@@ -122,7 +122,7 @@
if ( !PyArg_Parse(v, "s#", &krec.dptr, &tmp_size) ) {
PyErr_SetString(PyExc_TypeError,
- "dbm mappings have string keys only");
+ "dbm mappings have bytes or string keys only");
return -1;
}
krec.dsize = tmp_size;
@@ -140,7 +140,7 @@
} else {
if ( !PyArg_Parse(w, "s#", &drec.dptr, &tmp_size) ) {
PyErr_SetString(PyExc_TypeError,
- "dbm mappings have byte string elements only");
+ "dbm mappings have byte or string elements only");
return -1;
}
drec.dsize = tmp_size;
Modified: python/branches/py3k/Modules/_gdbmmodule.c
==============================================================================
--- python/branches/py3k/Modules/_gdbmmodule.c (original)
+++ python/branches/py3k/Modules/_gdbmmodule.c Tue Nov 25 20:19:17 2008
@@ -142,7 +142,7 @@
if (!PyArg_Parse(v, "s#", &krec.dptr, &krec.dsize) ) {
PyErr_SetString(PyExc_TypeError,
- "gdbm mappings have string indices only");
+ "gdbm mappings have bytes or string indices only");
return -1;
}
if (dp->di_dbm == NULL) {
@@ -160,7 +160,7 @@
else {
if (!PyArg_Parse(w, "s#", &drec.dptr, &drec.dsize)) {
PyErr_SetString(PyExc_TypeError,
- "gdbm mappings have byte string elements only");
+ "gdbm mappings have byte or string elements only");
return -1;
}
errno = 0;