Update 05/25/01 - remove ln -s cPickle and ln -s ZServer, add using zeo_start and zeo_stop.
Update 03/22/01 - security aspects, and ZEO frontends
Update 06/10/00 - ZEO latest version - 0.4.1; INSTANCE HOME [pointed out by [email protected]]
This How-To covers the use of Zope Enterprise Objects(ZEO) to scale your Zope site.
Note: This is a note of what i did to scale my site, and is to be taken with a grain of salt. This may not be the best setup, but I'm using it and had no problems. Errors, pointers, experience sought!
Every small time web site dreams of getting big, but being big does not come without its own headaches and challenges.
However, Zope will let you scale your site so that your site can take the load and still give users good response time.
What does it do?
ZEO will help you scale your site and give redundancy/high availability to your site too. Every Zope installation will be a Client Storage (CS) that talks to the Zope Storage Server (ZSS). Every CS will have its own object cache and
will be able to work independently of other CS.
You can have the CS cache in-memory or on the file system. Every request will be taken from the cache, if available, and from the ZSS if not. Modifications and additions will be sync to the
ZSS. ZSS will take care of the object validity.
Basic setup
The most basic setup is to have your ZSS and CS in the same machine. However, I will use two physically different machine to fully use ZEO capability. However, you can increase the number of physical machine to the nth.
Here, we have a machine that's acting as a ZSS and CS(the main server). We have other machines
that will have a Zope instance and will be the other CSs(the client servers).
This may not be the only way to connect. This setup is quite flat, and you have a single point of failure, the ZSS.
Check out the ZEO FactSheet.
Security
Since there is no security measure taken when we serve ZEO server, we need to put it in a private network. We can achieve this by adding another NIC to each box, or create another private ip alias and serve ZEO server from the private ip alias.
Installing the main server
You will need Zope > 2.2.0.
Unpack and install Zope[Zope/path/]. Test your Zope installation.
Get ZEO.
Unpack ZEO-latestversion.tgz
cd ZEO-latestversion
mv or copy ZEO to Zope/path/lib/python
cd Zope/path/lib/python
ln -s ../../ZServer .
not needed anymore
cd ZEO
ln -s ../cPickle.so
not needed anymore
copy custom_zodb.py.dist to your Zope installation dir [Zope/path/] or INSTANCE HOME, if
you use INSTANCE HOME.
With release 0.4, a sample custom_zodb.py is included in misc folder/directory. I have yet to use this, so could not comment on it yet. If you use it,
please tell us what you did, changed, etc and I'll include it here.
edit custom_zodb.py. Add the appropriate host name and port number where you will run ZSS. The default is local host('') and port 8800.
rename custom_zodb.py.dist to custom_zodb.py
start ZSS
cd Zope/path
python lib/python/ZEO/start -p 8800
Note: To serve to private network, use
python lib/python/ZEO/start -p 8800 -h 192.168.1.1
This will serve up your Data.fs
A better way to control your ZEO Server is to use the included zeo_start and zeo_stop.
create a storage file in Zope/path/lib/python/ZEO. lets call it StorageConfig.py.
##StorageConfig.py
import ZODB.FileStorage
main_storage= ZODB.FileStorage.FileStorage('/Zope/path/var/Data.fs')
Now create a zeo_start in Zope/path/
###zeo_start
#!/bin/sh
reldir=`dirname $0`
exec python $reldir/lib/python/ZEO/start.py -S main=StorageConfig:main_storage -p 8800 -h 10.10.10.2\
ZEO_SERVER_PID=$reldir/var/ZEO_SERVER.pid \
STUPID_LOG_FILE=$reldir/var/ZEO_EVENTS.log \
"$@"
create zeo_stop in Zope/path/
###stop zeo
#!/bin/sh
reldir=`dirname $0`
kill `cat $reldir/var/ZEO_SERVER.pid`
zeo_start and zeo_stop is available in ZEO tar ball.
start Zope
If you start Zope without ZEO_CLIENT, the cache will be in-memory. To have the cache on your fs, start Zope with ZEO_CLIENT. Add ZEO_CLIENT=8800 in your start script
e.g.
#! /bin/sh
reldir=`dirname $0`
PYTHONHOME=`cd $reldir; pwd`
export PYTHONHOME
exec /usr/bin/python \
$PYTHONHOME/z2.py -X -w 80 STUPID_LOG_FILE='var/stupid.log' \
-D '' "$@" ZEO_CLIENT=8800
I think it is good practice to have STUPID_LOG_FILE since it will really help point out errors and help in debugging.
You will now have the cache in Zope/path/var
You can test your ZSS and first CS.
Installing the other CSs
Install Zope. You will need the exact copy of your Zope as installed in the main server
if you want to serve the same contents. This includes everything in Extensions (your external methods). However, you don't need to have Data.fs
Install ZEO
mv or copy ZEO to Zope/path/lib/python
cd Zope/path/lib/python
ln -s ../../ZServer .
cd ZEO
ln -s ../cPickle.so
copy custom_zodb.py.dist to your Zope installation dir [Zope/path/] or INSTANCE_HOME
edit custom_zodb.py. Add in your ZSS host name or ip number for ZSS_HOST and the ZSS port number
rename custom_zodb.py.dist to custom_zodb.py
start Zope
Repeat on other CS.
The front-end
Now that we a few machines that have the same data, how do we serve it? There are few methods that can be used to front-end ZEO.
Round robin DNS,
Eddie - an enhanced DNS and HTTP Gateway,
Ultra Monkey - a scalable server solution,
FastCgi,
if you got the dough, any commercial load balancing products, e.g Cisco Local Director
Balance - a simple load balance app. You will have better control compared to using DNS round robin.
Backhand - a module for apache that will load balance your apache cluster.
Also, look at a thread
from Zope mailing list.
I'm using a combination of DNS Round robin and balance. And things
couldn't have been better.
Advance setup
ZEO is not limited to the setup discussed above. ZEO can serve out many storages with ZSS.
This, I have looked at only briefly.
One setup that i could think of is using External Mount
for part of your site. For example, say we want to have polling available
to users. Every vote casted will increase the Data.fs size.
A solution, mount Noundo, packless Database (Berkeley storage) on a poll mount point
in the ZODB tree. Now, we don't need to worry about size increase.
I'll try and put something out soon. meanwhile, checkout ZEO Sessions How-to.
Gotchas
These are notes to what I encountered when using ZEO.
Get the same exact Zope installation, products, external methods, etc. Otherwise, some of the CS will spit out
errors
If you use SQL backend, give access to all CS machines. Use ip address or hostname when
you define the Z Database connection
ZEO < 0.3 has problems with multiple write access. I have used ZEO-0.3 for quite some time now, and it looks like the problem is resolved.
with Zope version >= 2.3.1b1, you can specify FORCE_PRODUCT_LOAD at one of your CS, and new installed products will show up
Notes
Check out ZEO page. Also, Zope book, a much more thorough and detailed explanation from a Zopista with much more Zen! and amk ZODB and ZEO intro.