Recovering Corrupted Data.fs ZODB files

Created by . Last modified on 2003/08/05.

Warning: This can trash your Zope installation.  Only work on copies of your Data.fs, and make sure you have backups!

Every once in a while you'll corrupt your Zope Data.fs file, and Zope won't start. Since the Data.fs tacks transactions on to the end of the Data.fs file, recovering the Data.fs is most likely as easy as deleting the last transaction or three from the Data.fs.    

Zope 2.2 includes a command-line utility - lib/python/ZODB/fsrecover.py that will repair damaged Data.fs files. Back up your Data.fs, and then, in top level of your Zope directory:

python lib/python/ZODB/fsrecover.py var/Data.fs

If this doesn't work, or you're using 2.1, here's how to do it yourself, using Ty Sarna's cool tranalyzer:
 

1. Make a backup of the Data.fs file!!!
2. Download Ty Sarna's tranalyzer.py from http://www.zope.org/Members/tsarna/Tranalyzer
3. Tranalyzer prints out the transactions in the Data.fs and their location in the file.  So we'll run tranalyzer.py on the Data.fs and see what we get:


    [itamar@moriarity itamar]$ python tranalyzer.py Data.fs | tail -2000

Lets say the last result we got was:

TID: 331F256472BAFF7 @ 6191877 obs 1 len 7352 (status 'p') By  superuser
"/gl/harvest/balsamico//index.htm/manage_addProperty"
 OID: 10160 len 7234

TID: 331F2566736A2E6 @ 6199245 obs 1 len 8983 (status 'p') By  superuser
"/gl/harvest/balsamico//cooking.htm/manage_addProperty"
 OID: 10161 len 8863

This means the last transaction started at byte 6199245 and added 8983 bytes to the Data.fs.  So lets get rid of
it and see if Zope starts working again.  We'll truncate Data.fs so it's only 6199244 bytes long - that is, 6199245 - 1 bytes.  That's because the 6199245th byte is already the beginning of the new transaction.

4. We'll use Python, of course, to do this:
   
   Python 1.5.2 (#1, Feb  1 2000, 16:32:16)  [GCC egcs-2.91.66 19990314/Linux (egcs- on linux-i386

Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> f = open("Data.fs", "ab")
>>> # we'll truncate Data.fs so it's length is 6199244 - that's 1 byte less
... # then where the next transaction starts.
...
>>> f.truncate(6199244)
>>> f.close()

That's it!  If it still doesn't work, we'll get rid of the next to last transaction, till it starts working.