Linux/Unix > Python Essential Reference, Second Edition > A. The Python Library > Data Management and Object Persistence | See All Titles

< BACK | Make Note | Bookmark | CONTINUE >

155117184014003188065099048180054212144238241179195140058238111161105080005016228139083140

### Data Management and Object Persistence

The modules in this section are used to store data in a variety of DBM-style database formats. These databases operate like a large disk-based hash table in which objects are stored and retrieved using unique keys represented by standard strings. Most of these modules are optional Python extensions that require third-party libraries and that must be enabled when Python is built. See Appendix B, “Extending and Embedding Python,” for details on enabling optional modules.

All the databases are opened using a variation of the `open()` function (defined in each database module):

    open(filename [, flag [, mode]])

Opens the database file `filename` and returns a database object. `flag` is `'r'` for read-only access, `'w'` for read-write access, `'c'` to create the database if it doesn’t exist, or `'n'` to force the creation of a new database. `mode` is the file-access mode (the default is `0666` on UNIX).

The object returned by the `open()` function supports the following dictionary-like operations:

Operation | Description
--- | ---
`d` `[``key` `] =` `value` | Inserts `value` into the database
`value` `=` `d` `[``key` `]` | Gets data from the database
`del` `d` `[``key` `]` | Removes a database entry
`d` `.close()` | Closes the database
`d` `.has_key(``key` `)` | Tests for a key
`d` `.keys()` | Returns a list of keys

In all cases, `key` must be a standard string. In addition, `value` must be a standard string for all the database modules except the `shelve` module. Unicode strings cannot be used for keys in any of the modules and cannot be used for values in any module except `shelve`.

#### Note

- Most of the database packages described rely on third-party libraries that must be installed in addition to Python.

#### `anydbm`

The `anydbm` module provides a generic interface that’s used to open a database without knowing which of the lower-level database packages are actually installed and available. When imported, it looks for one of the `bsddb`, `gdbm`, or `dbm` modules. If none are installed, the `dumbdbm` module is loaded.

A database object is created using the `open()` function:

    open(filename [, flag='r' [, mode]])

Opens the database file `filename` and returns a database object. If the database already exists, the `whichdb` module is used to determine its type and the corresponding database module to use. If the database doesn’t exist, an attempt is made to create it using the first installed module in the above list of database modules. `flags` and `mode` are as described in the introduction to this section,“Data Management and Object Persistence” (p. 149).

    error

A tuple containing the exceptions that can be raised by each of the supported database modules.

Programs wanting to catch errors should use this tuple as an argument to `except`. For example:

    try:
        d = anydbm.open('foo','r')
    except anydbm.error:
        # Handle error

##### Note

- If the `dumbdbm` module is the only installed database module, attempts to reopen a previously created database with `anydbm` might fail in certain older versions of Python. Use `dumbdbm.open()` instead.

? See Also dumbdbm (152), whichdb (154).

#### `bsddb`

The `bsddb` module provides an interface to the Berkeley DB library. Hash, btree, or record-based files can be created using the appropriate `open()` call:

    hashopen(filename [, flag='r' [, mode]])

Opens the hash format file named `filename` .

    btopen(filename [, flag='r' [, mode]])

Opens the btree format file named `filename` .

    rnopen( filename [, flag='r' [, mode ]])

Opens a DB record format file named `filename` .

Databases created by this module behave like dictionaries, as described in the introduction to this section (“Data Management and Object Persistence,” p. 149), and additionally provide methods for moving a “cursor” through records:

Method | Description
--- | ---
`d` `.set_location(``key` `)` | Sets the cursor to the item indicated by the key and returns it.
`d` `.first()` | Sets the cursor to the first item in the database and returns it.
`d` `.next()` | Sets the cursor to the next item in the database and returns it.
`d` `.previous()` | Sets the cursor to the previous item in the DB file and returns it. Not supported on hash table databases.
`d` `.last()` | Sets the cursor to the last item in the DB file and returns it. Not supported on hash table databases.
`d` `.sync()` | Synchronizes the database on disk.

##### Exception

    error

Exception raised on non–key-related database errors.

##### Notes

- This module uses the version 1.85 API of the Berkeley DB package available at http://www.sleepycat.com.

- All the `open()` functions accept additional optional arguments that are rarely used. Consult the online documentation for details.

- Consult New Riders’ Berkeley DB (2001, ISBN 0-7357-1064-3).

? See Also dbhash (151), http://www.python.org/doc/lib/module-bsddb.html.

#### `dbhash`

The `dbhash` module is used to open databases using the `bsddb` module, but with an interface that closely matches the interface of the other database modules.

    open(filename [, flag='r' [, mode])

Opens a DB database and returns the database object.

A database object `d` returned by `open()` behaves like a dictionary and also provides the following methods:

Method | Description
--- | ---
`d` `.first()` | Returns the first key in the database
`d` `.last()` | Returns the last key in a database traversal
`d` `.next(``key` `)` | Returns the next key following `key` in the database
`d` `.previous(``key` `)` | Returns the item that comes before `key` in a forward traversal of the database
`d` `.sync()` | Writes unsaved data to the disk

##### Exception

    error

Exception raised on database errors other than `KeyError`. Same as `bsddb.error`.

##### Note

- The `bsddb` module must be installed.

? See Also bsddb (151).

#### `dbm`

The `dbm` module provides an interface to the UNIX dbm library.

    open(filename [, flag='r' [, mode]])

Opens a dbm database and returns a dbm object. `filename` is the name of the database file (without the `.dir` or `.pag` extension). The returned object behaves like a dictionary, as described in the section introduction (“Data Management and Object Persistence,” p. 149).

##### Exception

    error

Exception raised for dbm-specific errors other than `KeyError`.

? See Also anydbm (150), gdbm (153).

#### `dumbdbm`

The `dumbdbm` module is a simple DBM-style database implemented in Python. It should only be used when no other DBM database modules are available.

    open(filename [, flag [, mode]])

Opens the database file `filename` . `filename` should not include any suffixes such as `.dat` or `.dir`. The returned database object behaves like a dictionary, as described in the section introduction (“Data Management and Object Persistence,” p. 149).

##### Exception

    error

Exception raised for database-related errors other than `KeyError`.

? See Also anydbm (150), whichdb (154).

#### `gdbm`

The `gdbm` module provides an interface to the GNU DBM library.

    open(filename [, flag='r' [, mode]])

Opens a `gdbm` database with filename `filename` . Appending `'f'` to the flag opens the database in fast mode. In this mode, altered data is not automatically written to disk after every change, resulting in better performance. If used, the `sync()` method should be used to force unwritten data to be written to disk on program termination.

A `gdbm` object `d` behaves like a dictionary as described in the section introduction (“Data Management and Object Persistence,” p. 149), but also supports the following methods:

Method | Description
--- | ---
`d` `.firstkey()` | Returns the starting key in the database.
`d` `.nextkey(``key` `)` | Returns the key that follows `key` in a traversal of the database.
`d` `.reorganize()` | Reorganizes the database and reclaims unused space. This can be used to shrink the size of the `gdbm` file after a lot of deletions have occurred.
`d` `.sync()` | Forces unwritten data to be written to disk.

##### Exception

    error

Exception raised for `gdbm` -specific errors.

##### Note

- The GNU DBM library is available at www.gnu.org/software/gdbm/gdbm.html.

? See Also anydbm (150), whichdb (154).

#### `shelve`

The `shelve` module provides support for persistent objects using a special “shelf” object. This object behaves like a dictionary except that all the objects it contains are stored on disk using a database such as `dbm` or `gdbm`. A shelf is created using the `shelve.open()` function.

    open(filename [, flag='c'])

Opens a shelf file. If the file doesn’t exist, it’s created. `filename` should be the database filename and should not include a suffix. `flag` has the same meaning as in the `open()` function and is one of `'r'`, `'w'`, `'c'`, or `'n'`. If the database file doesn’t exist, it is created. Returns a shelf object.

Once opened, the following dictionary operations can be performed on a shelf:

Operation | Description
--- | ---
`d` `[``key` `] =` `data` | Stores data at `key` . Overwrites existing data.
`data` `=` `d` `[``key` `]` | Retrieves data at `key` .
`del` `d` `[``key` `]` | Deletes data at `key` .
`d` `.has_key(``key` `)` | Tests for the existence of `key` .
`d` `.keys()` | Returns all keys.
`d` `.close()` | Closes the shelf.
`d` `.sync()` | Write unsaved data to disk.

The key values for a shelf must be strings. The objects stored in a shelf must be serializable using the `pickle` module.

? See Also `open` (103).

    Shelf (dict)

A class that implements the functionality of a shelf. `dict` is an existing dictionary object. This function can be used to create a shelf object that utilizes a preferred database engine. For example, `s = Shelf(dbm.open("foo","c"))`.

##### Note

- The `shelve` module differs from other database modules in that it allows almost any Python object to be stored.

? See Also pickle (113), Chapter 9.

#### `whichdb`

The `whichdb` module provides a function that attempts to guess which of the several simple database modules (`dbm`, `gdbm`, or `dbhash`) should be used to open a database file.

    whichdb(filename)

`filename` is a filename without any suffixes. Returns `None` if the file cannot be opened because it’s unreadable or doesn’t exist. Returns the empty string if the file format cannot be guessed. Otherwise, a string containing the required module name is returned, such as `'dbm'` or `'gdbm'`.

? See Also anydbm (150).

| Last updated on 3/28/2002 Python Essential Reference, Second Edition, © 2002 New Riders Publishing

< BACK | Make Note | Bookmark | CONTINUE >

© 2002, O'Reilly & Associates, Inc.