Chapter 4: ZODB Persistent Components

Most Zope components live in the Zope Object DataBase (ZODB). Components that are stored in ZODB are said to be persistent. Creating persistent components is, for the most part, a trivial exercise, but ZODB does impose a few rules that persistent components must obey in order to work properly. This chapter describes the persistence model and the interfaces that persistent objects can use to live inside the ZODB.

Anonymous User - July 9, 2002 8:38 am:
 In a SAMS book, "ZOPE Web Application Construction Kit" by Martina Brockmann etc, page 41, it says "... ZODB
 is not suitable for storing large amounts of data."
 Is it true? If it is, we are in trouble because Zope stores everything in it. This means that Zope is not
 suitable for large site. Can somebody please clarify this? Thanks.

Anonymous User - July 9, 2002 8:58 am:
Sure. This statement is actually completely false. ZODB is very capable of storing large amounts of data. But
 like any sort of database you need to be careful to put the right kinds of data structures in the right
 places for maximum efficiency. Also, you may want to consider a different storage than FileStorage if you
 have lots of data because FileStorage keeps all data in a single file, which can be problematic on systems
 which do not support large files. Additionally, ZODB recovery tools are currently not as robust as, say,
 Oracle recovery tools, so if your database does get corrupted due to hardware failure it may not be a
 recoverable error and you may need to restore from backup.
 That said, there is nothing stopping you from using Oracle or the filesystem to store all of your data just
like other application servers. Zope lets you do this. You don't need to use ZODB. The equation you use above
 is inaccurate.

Anonymous User - July 9, 2002 9:17 am:
 Thanks for your quick reply.

 How can I find more info about:

 (1) any guideline regarding "to put the right kinds of data structures in the right places for maximum
 efficiency;"
 (2) how Zope works with other application servers if I store static html pages and graphics files on those
 servers.
 Thanks again.

Anonymous User - July 9, 2002 9:57 am:
 Ask a detailed question including what kinds of data you want to store and which other servers you want to
 integrate with on the Zope maillist ([email protected]). See the Resources link of Zope.org for more maillist
 info.

Anonymous User - Jan. 16, 2003 1:40 am:
More importantly, how do you configure zope to use an Oracle database instead of ZODB? I can find information
 on DCOracleStorage stuffs, but nothing about configuring Zope to startup and point to an Oracle database.

axxackall - Apr. 5, 2003 11:09 am:
 I know several projects where developers had to re-write half of the system just because it was based on
 object persistence and POET (or other) ODBMS. It is a fact that enterprise users do not like ODBMS with
 consequently closed architecture. They want RDBMS with ability to integrate the web portal with other
 corporate infrmation sources. How to do that with ZODB? No way. I've tried to find any information about any
 attempts to get rid from ZODB on the backend of Zope and found that virtually nobody is working currently on
 it. If anyone is interesting in it, particularly in substituting ZODB by PostgreSQL, please send me email:
 [email protected] as such project is too much for one busy developer.

Anonymous User - Dec. 22, 2003 4:17 pm:
 "get rid from ZODB" is not going to happen. Zope is a ZODB application.
 What you should probably look at is APE, which (properly configured) will transparently
 store zodb data in the RDBMS of your choice. It is not yet stable but some people are already using it.

Anonymous User - June 27, 2005 7:53 pm:
 test

Persistent Objects

Persistent objects are Python objects that live for a long time. Most objects are created when a program is run and die when the program finishes. Persistent objects are not destroyed when the program ends, they are saved in a database.

Anonymous User - June 27, 2005 7:52 pm:
 test

A great benefit of persistent objects is their transparency. As a developer, you do not need to think about loading and unloading the state of the object from memory. Zope's persistent machinery handles all of that for you.

Anonymous User - Feb. 21, 2002 7:47 pm - A note on __module_aliases__. It almost never works. Everytime I've
wanted to do this, it has no effect. References to classes are to pervasive given the standard Zopish ways of
doing things.

mcdonc - Mar. 2, 2002 4:58 pm - Sorry to hear you're having difficulties with it, but it does work. I've used
it many times.

This is also a great benefit for application designers; you do not need to create your own kind of "data format" that gets saved to a file and reloaded again when your program stops and starts. Zope's persistence machinery works with any kind of Python objects (within the bounds of a few simple rules) and as your types of objects grow, your database simply grows transparently with it.

Anonymous User - Dec. 9, 2001 3:47 pm - The topic came up on the maillist of how a persistent object "finds"
its class. A persistent object has a reference to its class' fully-qualified name including at least one
module name. For example, an instance of the DTMLMethod class finds its class at
'Products.OFSP.DTMLMethod.DTMLMethod', where 'Products' and 'OFSP' are Python packages, the first
'DTMLMethod' refers to a module file, and the last 'DTMLMethod' refers to the classname. If you change the
location of a class within the filesystem (for example, if you moved the above Python class definition to
'Foo.Bar.DTMLMethod', instances of the original class that were stored in the ZODB will not be loadable. There
is a facility named __module_aliases__ in Zope for aliasing a module to more than one name, although there is
no such facility for use of ZODB outside Zope. *Need more docs on __module_aliases__* - chrism.

Anonymous User - Dec. 9, 2001 3:53 pm - Example of __module_aliases__ follows. It works by putting a tuple of
tuples in your Product's __init__.py module in a __module_aliases__ attribute at module scope. For example,
PythonScripts have the following __module_aliases__ attribute. from Shared.DC import Scripts __module_aliases__
= ( ('Products.PythonScripts.Script', Scripts.Script), ('Products.PythonScripts.Bindings', Scripts.Bindings),
('Products.PythonScripts.BindingsUI', Scripts.BindingsUI),) .. this maps the module that *used* to be at
Products.PythonScripts.Script to the module that is *now* at Scripts.Script, etc. This only works with modules
and not with classes or other types.

Anonymous User - Jan. 17, 2002 10:12 am - change/your types of objects grow/your number of objects grow/

Persistence Example

Here is a simple example of using ZODB outside of Zope. If all you plan on doing is using persistent objects with Zope, you can skip this section if you wish.

Anonymous User - Oct. 21, 2002 10:16 am:
well, it is really funny that the only example about persistence is not valid inside Zope. After reading this
 chapter, I still cannot use persistence. I think this chapter should explain how to use persistance inside
 Zope

Anonymous User - Nov. 8, 2002 10:27 pm:
 Maybe the ZDG should be split into a) Product developement inside ZODB and
 b) Advanced Developers Guide?

The first thing you need to do to start working with ZODB is to create a "root object". This process involves first opening a "storage" , which is the actual backend storage location for your data.

ZODB supports many pluggable storage back-ends, but for the purposes of this article we're going to show you how to use the FileStorage back-end storage, which stores your object data in a file. Other storages include storing objects in relational databases, Berkeley databases, and a client to server storage that stores objects on a remote storage server.

Anonymous User - Nov. 8, 2002 10:29 pm:
 Berkely db: URL?

Anonymous User - Nov. 16, 2002 3:40 am:
 A little more information or pointers to info on the different pluggable storage back-ends would go down a
 treat

To set up a ZODB, you must first install it. ZODB comes with Zope, so the easiest way to install ZODB is to install Zope and use the ZODB that comes with your Zope installation. For those of you who don't want all of Zope, but just ZODB, see the instructions for downloading ZODB from the ZODB web page.

After installing ZODB, you can start to experiment with it right from the Python command line interpreter. If you've installed Zope, before running this set of commands, shut down your Zope server, and "cd" to the "lib/python" directory of your Zope instance. If you're using a "standalone" version of ZODB, you likely don't need to do this, and you'll be able to use ZODB by importing it from a standard Python package directory. In either case, try the following set of commands:

chrism@saints:/opt/zope/lib/python$ python
Python 2.1.1 (#1, Aug  8 2001, 21:17:50) 
[GCC 2.95.2 20000220 (Debian GNU/Linux)] on linux2
Type "copyright", "credits" or "license" for more information. 
>>> from ZODB import FileStorage, DB
>>> storage = FileStorage.FileStorage('mydatabase.fs')
>>> db = DB( storage )
>>> connection = db.open()
>>> root = connection.root()

Here, you create storage and use the mydatabse.fs file to store the object information. Then, you create a database that uses that storage.

Anonymous User - July 18, 2002 1:36 pm:
 s/mydatabse\.fs/mydatabase\.fs/

Next, the database needs to be "opened" by calling the open() method. This will return a connection object to the database. The connection object then gives you access to the root of the database with the root() method.

The root object is the dictionary that holds all of your persistent objects. For example, you can store a simple list of strings in the root object:

root['employees'] = ['Bob', 'Mary', 'Jo']

Now, you have changed the persistent database by adding a new object, but this change is so far only temporary. In order to make the change permanent, you must commit the current transaction:

get_transaction().commit()

Transactions are ways to make a lot of changes in one atomic operation. In a later article, we'll show you how this is a very powerful feature. For now, you can think of committing transactions as "checkpoints" where you save the changes you've made to your objects so far. Later on, we'll show you how to abort those changes, and how to undo them after they are committed.

If you had used a relational database, you would have had to issue a SQL query to save even a simple python list like the above example. You would have also needed some code to convert a SQL query back into the list when you wanted to use it again. You don't have to do any of this work when using ZODB. Using ZODB is almost completely transparent, in fact, ZODB based programs often look suspiciously simple!

Working with simple python types is useful, but the real power of ZODB comes out when you store your own kinds of objects in the database. For example, consider a class that represents a employee:

from Persistence import Persistent

class Employee(Persistent):

    def setName(self, name):
        self.name = name

Anonymous User - Oct. 16, 2002 7:43 am:
 I get the message

 Error Value: exceptions.ImportError on import of "Persistent" from "Persistence" is unauthorized in '', at
 line 15, column 11
 I am the zope administrator and I am running it in a Win98, therefore shouldn't be problems with the
 authorization.
I am running it in a Python script that I call from a dtml page. Is there any problem with this? somebody can
 tell me why I get this problem?

Calling setName will set a name for the employee. Now, you can put Employee objects in your database:

for name in ['Bob', 'Mary', 'Joe']:
    employee = Employee()
    employee.setName(name)
    root['employees'].append(employee)

get_transaction().commit()

Don't forget to call commit(), so that the changes you have made so far are committed to the database, and a new transaction is begun.

Anonymous User - Apr. 10, 2003 5:03 pm:
This example seems to break the fourth rule as mentioned below. You are adding to the root's employees entry,
 a list, which is a mutable thing.
 At least, this is true when running the Python interpreter with a standalone ZODB.

Anonymous User - Nov. 22, 2003 11:44 pm:
No, it is saying that you must notify the database to persist each change after it has been made, because the
 default Python list type does not know about persistence. If you use the types provided with ZODB, then
 changes will be persisted automatically. The type is smart enough to update the database when the in-memory
 copy changes.

Anonymous User - Mar. 15, 2004 9:33 am:
 well said

Anonymous User - Oct. 12, 2004 5:31 pm:
 well...

Persistent Rules

There are a few rules that must be followed when your objects are persistent.

• Your objects, and their attributes, must be "pickleable".
• Your object cannot have any attributes that begin with _p_.
• Attributes of your object that begin with _v_ are "volatile" and are not saved to the database (see next section).
• You must explicitly signal any changes made to mutable attributes (such as instances, lists, and dictionaries) or use persistent versions of mutable objects, like ZODB.PersistentMapping (see below for more information on PersistentMapping.)

In this section, we'll look at each of these special rules one by one.

The first rules says that your objects must be pickleable. This means that they can be serialized into a data format with the "pickle" module. Most python data types (numbers, lists, dictionaries) can be pickled. Code objects (method, functions, classes) and file objects (files, sockets)

cannot

be pickled. Instances can be persistent objects if:

• They subclass Persistence.Persistent
• All of their attributes are pickleable

Anonymous User - Nov. 8, 2002 10:57 pm:
 The result of a ZSQL Method can not be pickled (thanks to a class r():pass)

Anonymous User - Apr. 10, 2003 5:03 pm:
 s/rules/rule/

The second rule is that none of your objects attributes can begin with _p_. For example, _p_b_and_j would be an illegal object attribute. This is because the persistence machinery reserves all of these names for its own purposes.

The third rule is that all object attributes that begin with _v_ are "volatile" and are not saved to the database. This means that as long as the persistent object is in Zope memory cache, volatile attributes can be used. When the object is deactivated (removed from memory) volatile attributes are thrown away.

Volatile attributes are useful for data that is good to cache for a while but can often be thrown away and easily recreated. File connections, cached calculations, rendered templates, all of these kinds of things are useful applications of volatile attributes. You must exercise care when using volatile attributes. Since you have little control over when your objects are moved in and out of memory, you never know when your volatile attributes may disappear.

poster - May 13, 2002 9:43 am:
 I assume that you can count on a volatile attribute remaining within the life of a method call that creates
it. What about within a transaction? In general, while I understand there will be a point at which you can no
 longer rely on the existence of a volatile attribute, when *can* you rely on it?

reiman - Aug. 16, 2002 12:13 pm:
 I also just learned that _v_ attributes are thread-specific. This too should be mentioned (and explained)
 here.

beyond - Oct. 5, 2002 10:44 am:
 Within one transaction you can rely on _v_ attributes (afaik). 
 Each thread gets its own transaction. So another thread -> another _v_ attribute 
 -> you can't rely on it in different transactions. Tansactions are kept in a pool
 and objects are cached so sometimes when accessing an object with a new 
 request (which gets the old transaction out of the pool) the object is still cached
 and it won't get loaded out of ZODB and then __setstate__ won't get called and 
 finally the _v_ attribute remains. But this is of course not relyable. I'm just 
 telling you this special case because recently I got some strange errors which were
 caused by this behavior.

The fourth rule is that you must signal changes to mutable types. This is because persistent objects can't detect when mutable types change, and therefore, doesn't know whether or not to save the persistent object or not.

For example, say you had a list of names as an attribute of your object called departments that you changed in a method called 'addDepartment':

class DepartmentManager(Persistent):

    def __init__(self):
        self.departments = []

    def addDepartment(self, department):
        self.departments.append(department)

When you call the addDepartment method you change a mutable type, departments but your persistent object will not save that change.

There are two solutions to this problem. First, you can assign a special flag, '_p_changed':

def addDepartment(self, department):
    self.department.append(department)
    self._p_changed = 1

Anonymous User - Mar. 24, 2003 4:23 pm:
 It says there are "two solutions", and then, "First...". What is the second solution?

Anonymous User - Nov. 8, 2002 11:05 pm:
 well, these are simple types. i had a (recursive) attribute 
 self.tree = [..., {'sub': <a subtree> ...} ...] 
 and sometimes (unreproducable) it didnt update in lieu of self._p_changed=1.
 Explain.

Anonymous User - Aug. 3, 2002 3:59 pm:
 Reassigning self.departments stores the entire list again. ZODB can know nothing about your intent. If you
 have an often-changing list like this, it'd likely be better to store it as a BTree (see
 lib/python/BTrees/Interfaces.py in the source).

Anonymous User - Aug. 3, 2002 3:45 pm:
 which technique is more efficient? When you tag a mutable object (i.e. list) with _p_changed = 1; what does
ZODB do to change the list? Does it commit the entire list all over again or does it just commit the new list
 element? Using a mutable as though it were immutable screams inefficiency to me... what if the list is HUGE
 (i.e. 1024 instances of a persistant class)? For the assignment: departments = self.departments, does ZODB
 pull the entire list out of storage or does it just do a shallow copy? I assume shallow... please advise.

bernddorn - Dec. 1, 2001 7:46 am - there are some mistakes in the example code: "self.department =
departments" should be "self.departments = departments" further above, the same mistake appears.

Transactions and Persistent Objects

When changes are saved to ZODB, they are saved in a

transaction

. This means that either all changes are saved, or none are saved. The reason for this is data consistency. Imagine the following scenario:

1. A user makes a credit card purchase at the sandwich.com website.
2. The bank debits their account.
3. An electronic payment is made to sandwich.com.

Now imagine that an error happens during the last step of this process, sending the payment to sandwich.com. Without transactions, this means that the account was debited, but the payment never went to sandwich.com! Obviously this is a bad situation. A better solution is to make all changes in a transaction:

1. A user makes a credit card purchase at the sandwich.com website.
2. The transaction begins
3. The bank debits their account.
4. An electronic payment is made to sandwich.com.
5. The transaction commits

Now, if an error is raised anywhere between steps 2 and 5, all changes made are thrown away, so if the payment fails to go to sandwich.com, the account won't be debited, and if debiting the account raises an error, the payment won't be made to sandwich.com, so your data is always consistent.

When using your persistent objects with Zope, Zope will automatically begin a transaction when a web request is made, and commit the transaction when the request is finished. If an error occurs at any time during that request, then the transaction is aborted, meaning all the changes made are thrown away.

If you want to intentionally abort a transaction in the middle of a request, then just raise an error at any time. For example, this snippet of Python will raise an error and cause the transaction to abort:

raise SandwichError('Not enough peanut butter.')

A more likely scenario is that your code will raise an exception when a problem arises. The great thing about transactions is that you don't have to include cleanup code to catch exceptions and undo everything you've done up to that point. Since the transaction is aborted the changes made in the transaction will not be saved.

Because Zope does transaction management for you, most of the time you do not need to explicitly begin, commit or abort your own transactions. For more information on doing transaction management manually, see the links at the end of this chapter that lead to more detailed tutorials of doing your own ZODB programming.

Anonymous User - Jan. 4, 2002 9:14 am - The text should probably mention that you have to let the exception
propagate "right out of Zope" for the "rollback" to occur in Zope (of course). Otherwise, it seems to be the
case that if the exception is to be handled within a Zope Product (so that a user of the application doesn't
see the standard error page), then an explicit transaction abort should be performed in the exception handler
in question.

peterb - Aug. 16, 2002 3:48 am:
It should be even more specific and mention that if the exception is caught, whether in a product, in DTML or
 in a script there is no automatic rollback.
 You need to call get_transaction.abort() in your exception handler, unless you rethrow the exception and you
 know it won't get caught.
 This is actually guesswork due to lacking docs, I just happen to have the problem right now.

Subtransactions

Zope waits until the transaction is committed to save all the changes to your objects. This means that the changes are saved in memory. If you try to change more objects than you have memory in your computer, your computer will begin to swap and thrash, and maybe even run you out of memory completely. This is bad. The easiest solution to this problem is to not change huge quantities of data in one transaction.

If you need to spread a transaction out of lots of data, however, you can use subtransactions. Subtransactions allow you to manage Zope's memory usage yourself, so as to avoid swapping during large transactions.

Anonymous User - Nov. 10, 2001 7:37 am - That first sentence doesn't make sense. How about "However, if you
need commit a transaction containing a lot of data you can use subtransactions." -- ChrisW

Subtransactions allow you to make huge transactions. Rather than being limited by available memory, you are limited by available disk space. Each subtransaction commit writes the current changes out to disk and frees memory to make room for more changes.

To commit a subtransaction, you first need to get a hold of a transaction object. Zope adds a function to get the transaction objects in your global namespace, get_transaction, and then call commit(1) on the transaction:

get_transaction().commit(1)

You must balance speed, memory, and temporary storage concerns when deciding how frequently to commit subtransactions. The more subtransactions, the less memory used, the slower the operation, and the more temporary space used. Here's and example of how you might use subtransactions in your Zope code:

tasks_per_subtransaction = 10
i = 0
for task in tasks:
    process(task)
    i = i + 1
    if i % tasks_per_subtransaction == 0:
        get_transaction().commit(1)

Anonymous User - Jan. 17, 2002 11:22 am - no "," before "and"

This example shows how to commit a subtransaction at regular intervals while processing a number of tasks.

Threads and Conflict Errors

Zope is a multi-threaded server. This means that many different clients may be executing your Python code in different threads. For most cases, this is not an issue and you don't need to worry about it, but there are a few cases you should look out for.

The first case involves threads making lots of changes to objects and writing to the database. The way ZODB and threading works is that each thread that uses the database gets its own connection to the database. Each connection gets its own copy of your object. All of the threads can read and change any of the objects. ZODB keeps all of these objects synchronized between the threads. The upshot is that you don't have to do any locking or thread synchronization yourself. Your code can act as through it is single threaded.

Anonymous User - Aug. 7, 2002 8:44 am:
 "through".replace("r", "")

However, synchronization problems can occur when objects are changed by two different threads at the same time.

Imagine that thread 1 gets its own copy of object A, as does thread 2. If thread 1 changes its copy of A, then thread 2 will not see those changes until thread 1 commits them. In cases where lots of objects are changing, this can cause thread 1 and 2 to try and commit changes to object 1 at the same time.

When this happens, ZODB lets one transaction do the commit (it "wins") and raises a ConflictError in the other thread (which "looses"). The looser can elect to try again, but this may raise yet another ConflictError if many threads are trying to change object A. Zope does all of its own transaction management and will retry a losing transaction three times before giving up and raising the ConflictError all the way up to the user.

mcdonc - Oct. 16, 2001 9:37 am - This is "loses" and "loser".  No pun intended.

Resolving Conflicts

If a conflict happens, you have two choices. The first choice is that you live with the error and you try again. Statistically, conflicts are going to happen, but only in situations where objects are "hot-spots". Most problems like this can be "designed away"; if you can redesign your application so that the changes get spread around to many different objects then you can usually get rid of the hot spot.

Anonymous User - Sep. 12, 2002 7:09 pm:
 This is totally retarded.  Why can't we get a mutex instead?

Anonymous User - Sep. 12, 2002 7:18 pm:
 Code talks.  Write code or shut the hell up.

Anonymous User - Sep. 13, 2002 11:20 am:
 Although the comment above was not very nice, this might be a good place
 to explain why a mutex is not appropriate.
 A mutex would have to span multiple ZEO servers and would
 serialize all Zope application threads.  Conflicts are one of the prices
 of scalability, but in practice conflicts are rare enough that
 few applications need to deal with them directly.

Your second choice is to try and resolve the conflict. In many situations, this can be done. For example, consider the following persistent object:

class Counter(Persistent):

    self.count = 0

    def hit(self):
        self.count = self.count + 1

This is a simple counter. If you hit this counter with a lot of requests though, it will cause conflict errors as different threads try to change the count attribute simultaneously.

But resolving the conflict between conflicting threads in this case is easy. Both threads want to increment the self.count attribute by a value, so the resolution is to increment the attribute by the sum of the two values and make both commits happy; no ConflictError is raised.

Anonymous User - Nov. 16, 2002 3:52 am:
 I'm with the 'code talks' aphorism - so how about a recoded example just here?

To resolve a conflict, a class should define an

_p_resolveConflict

method. This method takes three arguments.

oldState

The state of the object that the changes made by the current transaction were based on. The method is permitted to modify this value.

savedState

The state of the object that is currently stored in the database. This state was written after oldState and reflects changes made by a transaction that committed before the current transaction. The method is permitted to modify this value.

newState

The state after changes made by the current transaction. The method is not permitted to modify this value. This method should compute a new state by merging changes reflected in savedState and newState, relative to oldState.

Anonymous User - Nov. 16, 2002 3:55 am:
 Does it return anything useful - I'm assuming it does (the new state, perhaps) as newstate (below) is
 described as not valid for changing. But what's the actuality?

Jace - Nov. 15, 2002 3:20 pm:
 s/define an/define a/

Threadsafety of Non-Persistent Objects

ZODB takes care of threadsafety for persistent objects. However, you must handle threadsafey yourself for non-persistent objects which are shared between threads.

Anonymous User - Feb. 18, 2005 5:47 pm:
 s/safey/safety/

Mutable Default Arguments

One tricky type of non-persistent, shared objects are mutable default arguments to functions, and methods. Default arguments are useful because they are cached for speed, and do not need to be recreated every time the method is called. But if these cached default arguments are mutable, one thread may change (mutate) the object when another thread is using it, and that can be bad. So, code like:

def foo(bar=[]):
    bar.append('something')

Anonymous User - Sep. 6, 2002 9:49 am:
 need more explanation, please: what if foo is a method of class FooClass(Persistence.Persistent)? Does the
 persistence machinery guarantee threadsafety for argument 'bar' or we are in trouble all the same?
 Put another way, arguments of methods in persistent classes are persistent or not?

beyond - Oct. 5, 2002 10:55 am:
 (slightly off-topic)
 def foo(bar=[]) looks dangerous to me.
 When you use the default argument bar it will always be the same mutable 
 sequence. So calling 

 def myFoo(bar=[]):
    bar.append('bla')
    print bar

 two times will result in ['bla'] and ['bla','bla']

Could get in trouble if two threads execute this code because lists are mutable. There are two solutions to this problem:

• Don't use mutable default arguments. (Good)
• If you use them, you cannot change them. If you want to change them, you will need to implement your own locking. (Bad)

We recommend the first solution because mutable default arguments are confusing, generally a bad idea in the first place.

Shared Module Data

Objects stored in modules but not in the ZODB are not persistent and not-thread safe. In general it's not a good idea to store data (as opposed to functions, and class definitions) in modules when using ZODB.

reiman - Aug. 16, 2002 12:15 pm:
We should mention that module data is the easiest way to achive server-lifetime data store. This is where you
 would normally store external references (file handles or database connections or session data) that you
 cannot easily reconstruct.

If you decide to use module data which can change you'll need to protect it with a lock to ensure that only one thread at a time can make changes.

zigg - Jan. 16, 2002 9:31 am - See also http://www.zopelabs.com/cookbook/1006189713, which demos how to use
ThreadLock. Apparently this is the "official" way?

mcdonc - Jan. 17, 2002 8:27 am - chrism - ThreadLock provides recursive locks so that the thread that holds
the mutex can re-lock the lock (maybe by recursing or looping unintentionally or even intentionally) without
fear of deadlock. threading.Lock does not allow this to happen.

For example:

from threading import Lock
queue=[]
l=Lock()

def put(obj):
    l.acquire()
    try:
        queue.append(obj)
    finally:
        l.release()

def get():
    l.acquire()
    try:
        return queue.pop()
    finally:
        l.release()

Note, in most cases where you are tempted to use shared module data, you can likely achieve the same result with a single persistent object. For example, the above queue could be replaced with a single instance of this class:

class Queue(Persistent):

    def __init__(self):
        self.list=[]

    def put(self, obj):
        self.list=self.list + [obj]

    def get(self):
        obj=self.list[-1]
        self.list=self.list[0:-1]
        return obj

Notice how this class uses the mutable object self.list immutably. If this class used self.list.pop and self.list.append, then the persistence machinary would not notice that self.list had changed.

Shared External Resources

A final category of data for which you'll need to handle thread-safety is external resources such as files in the filesystem, and other processes. In practice, these concerns rarely come up.

Other ZODB Resources

This chapter has only covered the most important features of ZODB from a Zope developer's perspective. Check out some of these sources for more in depth information:

• Andrew Kuchling's ZODB pages include lots of information included a programmer's guide and links to ZODB mailing lists.
• ZODB Wiki has information about current ZODB projects.
• ZODB UML Model has the nitty gritty details on ZODB.
• Paper Introduction to the Zope Object Database by Jim Fulton, presented at the 8th Python Conference.

Summary

The ZODB is a complex and powerful system. However using persistent objects is almost completely painless. Seldom do you need to concern yourself with thread safety, transactions, conflicts, memory management, and database replication. ZODB takes care of these things for you. By following a few simple rules you can create persistent objects that just work.

Anonymous User - Aug. 21, 2002 12:42 pm:
 Andrew Kuchling's ZODB pages link is broken!!

Anonymous User - Aug. 21, 2002 12:53 pm:
 use http://www.amk.ca/zodb/ instead

Anonymous User - Oct. 17, 2003 6:47 am:
 The link above is broken as well!
 You can find Kuchling's guide at

 http://www.zope.org/Wikis/ZODB/guide/zodb.html