Disaster Recovery + Avoidance

Created by . Last modified on 2003/08/05.

Disaster Recovery

version 0.4

Here's Hoping to Save You 1/2 day Troubleshooting when you install Products or new versions of zope.

Revision History

0.3: added appendix A: Beta Zope upgrade on Debian
0.2: rewrote sections (2 warnings) based on better experience
0.1: description of my first recovery experience

This HOWTO is the result of my experience recovering from un-intended trouble caused by the ZLogger and ZPythonMethod Products. (nov 6,99) I also include an Appendix on setting up zope 2.1.0 on Debian) I realise now that this HOWTO can be summarized in two tips:

Warning: When installing a product into the zope library tree, it might change permissions of lib/python/lib/Products and thus cripple zope causing all sorts of bizarre errors

Warning: When installing a new product, you may need corresponding python modules

I realized this today (nov 4 99), and these should be just tips, However, before this insight, I wrote this HOWTO which might still be of some general use: (note: I speak from a Debian/GNU Linux perspective)

Here is the recipe for disaster prevention: Absolutely NO WARANTEE. You are on your own. Caveat Emptor. Before you install a new product and something goes awfully wrong in your zope website, then do these:

Make a backup of the database , for example:
cd /var/lib/zope/var

cp Data.fs Data.fs.oct19
Make a backup of /usr/lib/zope, for example:
cd /usr/lib

cp -v zope zope.oct19
NOW is the time to install your new Product, comforted by your recent backups!
There are several styles of Product installations:

Most of the ones I tried went something like this: (read the READMEs first of course)

cd /usr/lib/zope

tar -xzvof path/Product_name.tgz

(this is the type of Product I speak about in this HOWTO)
restart zope with: zopectl restart
now check your zope website: is everything OK? if yes, rejoice and forget about the rest of this HOWTO.
critical: Check the permissions of lib/python/lib/Products/ it should be 755 for the directories, 644 for files (i.e. world readable all along the tree)
zopectl restart ... does it work now?
Check if the relevant python modules are installed. (And no, I don't know of a standard way of doing this... but I provide a hint for Debian/GNU Linux users at the foot of this document). Install the missing python modules.
zopectl restart .... does it work now?
If not, then Delete the package from where it got installed, for example:
cd /usr/lib/zope

rm -Rf lib/python/Product/name_of_product
Restart zope: zopectl restart
Did it solve the problem? If yes, rejoice and forget about the rest of this HOWTO
If not, check your database with tranalyser.py (which is listed under Products on www.zope.org) and try:
python trananalyser.py Data.fs

Does it show any problems like truncations? if so,

cp Data.fs.backup Data.fs

chmod 0600 Data.fs

or truncate it at the suggested byte (tranalyser output) with
split -b #byte Data.fs

cp xaa Data.fs

and try again with zopectl restart
if not, then delete the entire zope library:
rm -Rf /usr/lib/zope

and reinstall zope and all the Products you use such as tinytables, squishdot, etc... up to but not including the latest one which caused the problems.
zopectl restart
For me that finally did it.

NOTES:

If (like me) you did not follow the correct order above, then you will have to turn your brain on and use good old common sense and logic to figure out your safest way out of the woods... the recipe above is useful.
I found tranalyser.py to be a very nice view of what is in Data.fs (the database) ... and I did fool around at length with truncating the database with "split -b # Data.fs" which worked but was never sufficient due to something outside the database having been somehow changed (as described above)
(For the Debian types only:) I used debsums zope|grep FAILED and found 38 files that failed the checksum test, but this seems to be the case even if the zope website is installed totally from scratch. Those files are all python compiled (probably as part of the installation process)
For Debian/GNU linux installations, these are python potato packages I found to be important:
- python-bobo
- python-bobodtml
- python-bobopos
- python-pygresql (for postgresql stuff)
Thanks to Amos who indicated publicly that one of the problems with that website when changing hardware was that he did not install all the requesite python modules and that caused bizarre errors ... makes me feel less foolish :-)

Appendix A: Beta Zope on Debian

Here are some notes based on my experience installing the latest zope 2.1.0 on a Debian/GNU Linux system:

The ideal is for a Debian maintainer to update the zope package so that this is all done automatically. Please note, that follwoing the zope instructions directly works but does not store the files in the same places as the current debian zope package; this is why I provide instructions for those who want to keep the same debian file structure:

Make Backup
unpackage the tarball in /tmp
./install -u www-data
cp -av lib/python/* /usr/lib/zope/lib/python
cp -av ZServer/* /usr/lib/zope/ZServer
cp -v z2* /usr/lib/zope
cp -v zpasswd* /usr/lib/zope
cd /usr/lib/zope; edit z2.py and change the settings appropriately: especially "here", "PCGI_FILE=/usr/lib/cgi-bin/zope" and "user=www-data" and see appendix B for fastCGI
edit /usr/sbin/zopectl and add "python" in front of the line that starts z2.py
it worked for me :-) (nov. 6, 99)

Appendix B: FastCGI and Debian

Install the package libapache-mod-fastcgi
change z2.py : line about FCGI: FCGI="/var/lib/zope/var/FCGI" or something like that

Log in