You are not logged in Log in Join
You are here: Home » Members » adytumsolutions » How To Love ZODB and Forget RDBMS - Part III

Log in
Name

Password

 

How To Love ZODB and Forget RDBMS - Part III

 

Created by adytumsolutions . Last modified 2004-03-17 04:19:19.

Where Do I Put My Stuff?
  1. Introduction
  2. Creating/Opening a Stand-alone ZODB
  3. Adding Data to a ZODB
  4. Retrieving Data from a ZODB
  5. Using the ZODB Tools to inspect the ZODB
  6. Using the ZODB Tools to Backup and Recover the ZODB
  7. Using Stand-alone ZEO to Allow Many Connections
  8. Acknowledgements

RDBMS

If you have a background in Relational Database Management Systems, then you are familiar with databases, tables, schemas, etc. Let's summarize the things we need for an RDBMS:

  • A database server (like MySQL) installed somewhere on the file system
  • A database created (using SQL) on the database server
  • A connection to the database
  • A table schema for creating the table
  • A created table to put the data in

ZODB

So what do you do in ZODB's object oriented framework? Well, conceptually, it's pretty much the same thing; you just go about it differently. Let's talk about the things we have already done so far. We have:

  • A database server (the ZODB software and the /tmp/pymonitor.fs)
  • A connection to the database

These two were done in the previous section with the following code in interactive python:

 >>> import ZODB.config
 >>> db = ZODB.config.databaseFromURL('pymonitor.conf')
 >>> conn = db.open()
 >>> dbroot = conn.root()

We still need the following:

  • to create a database
  • to make a schema
  • to create something like a table to store our data

But right now, let's take care of the database. We have the database software installed and ready to go and we have a connection to it already.

We promised to explain the reason for using the name pymonitor.conf, and this seems like a good time to do that ;-) Let's imagine we have a large farm of web servers, all of which we are responsible for keeping up and running. We want to write an application that monitors them and keeps track of the state and related information for the servers; we're going to need to store that data in a database.

In SQL, we would create our database like this:

 CREATE DATABASE monitoring_db;

Just in like in the SQL example, for ZODB we'll need to give our database a name but we'll also have to decide what type of database it will be. Remember: object oriented databases are hierarchies, and there are many different ways to create hierarchies in Python. One type that scales particularly well is BTree (see the " Persistent-Aware Types" section in Advanced ZODB for Python Programmers for more details). Being the optimists we are, we think we might have millions of records (for our million servers and services we want to monitor!), so we decide to use BTree (in particular, we choose the OOBTree). Still using interactive python, we do the following:

 >>> from BTrees.OOBTree import OOBTree
 >>> dbname = 'monitoring_db'
 >>> dbroot[dbname] = OOBTree()
 >>> mondb = dbroot[dbname]

Code Breakdown : The first line should look familiar now: we are importing the python modules necessary to create our BTree database. Line two is a simple assignment of the database name to a variable, and the database is actually created on the third line. We've got our database, but calling it by the name dbroot[dbname] will get tiresome very quickly, so on the fourth line, we assign it to a variable so we can reference it more easily.

All we have left to do now is

  • make a schema
  • create a place to store our data

Schemas

In order to talk about table schemas, we should have a clear idea of what tables are and the components of a table:

  • Tables have fields
  • Fields contain data of all different types
  • There can be fields with unique values or index values
  • A record is an entry of data in a table

This is very basic and not very technical, but you get the idea.

Now, let's take a practical example. Let's say you want to have a table that keeps track of services you monitor on a server. Perhaps you want to monitor your Apache Web Server and your Zope Application Server. You want to know things about them like:

  • How many httpd and zope processes are running?
  • Are the httpd and zope services ok? should I be watching them closely? are they down?
  • When was the last time they went down?

These are just some of the things we could know about each service. We might also find the following interesting as well:

  • What is the name of the service? ("httpd", "zope")
  • What host (server) is it running on?
  • What is the friendly name? ("Apache Web Server")
  • What is the description? ("This is the production Zope instance servicing 16 low-volume clients")
  • What is the current state of the service? What was the previous state? ("OK", "WARN", "ERROR", "RECOVERING")
  • What was the last time it was checked?
  • What was the last time it was in the "OK" state? In the "WARN" state? In the "ERROR" state?

SQL Schemas

From all of these, we can build a table schema, and it might look something like this:

 +==================+
 | service_table    |
 +==================+
 |id                |
 |name              |
 |description       |
 |service           |
 |hostname          |
 |state_current     |
 |state_last        |
 |time_lastcheck    |
 |time_lastok       |
 |time_lastwarn     |
 |time_lasterr      |
 +==================+

Then, after connecting to the database, you might run something like this:

 CREATE TABLE service_table (
  id VARCHAR(15)
  name VARCHAR(30)
  ...
 );

creating all your fields in the table.

Python Schemas for ZODB

We are using python to interact with the ZODB, so if the ZODB doesn't use tables like SQL, what does it use? We will also use a schema, but that schema is more than just a symbol or drawing: it's actually useful. Instead of tables, we have objects; instead of fields in a table, we have object attributes. Our schema (class) will actually be instantiated as an object. Once it's instantiated, we can store data in it.

First, let's make our schema:

 >>> from Persistence import Persistent
 >>> class ServiceTable(Persistent):
 ...  def __init__(self):
 ...    self.id = ''
 ...    self.name = ''
 ...    self.description = ''
 ...    self.service = ''
 ...    self.hostname = ''
 ...    self.state_current = ''
 ...    self.state_last = ''
 ...    self.time_lastcheck = ''
 ...    self.time_lastok = ''
 ...    self.time_lastwarn = ''
 ...    self.time_lasterror = ''

Now, this maintains our analogy with relational tables... however, let's give this some further thought (as Kapil Thangavelu gave me in an email!): if we imagine having just entered some data into this, what we will have is a record set, or a row of data. The table is a collection of rows, so it would be more conceptually accurate to call this class ServiceRow :

 >>> from Persistence import Persistent
 >>> class ServiceRow(Persistent):
 ...  def __init__(self):
 ...    self.id = ''
 ...    self.name = ''
 ...    self.description = ''
 ...    self.service = ''
 ...    self.hostname = ''
 ...    self.state_current = ''
 ...    self.state_last = ''
 ...    self.time_lastcheck = ''
 ...    self.time_lastok = ''
 ...    self.time_lastwarn = ''
 ...    self.time_lasterror = ''

Code Breakdown : We won't go over the details of python classes here, but suffice it to say that we just created a special class called ServiceRow whose sole purpose is to have attributes assigned to it (i.e., enter rows of data; it has no methods other that the __init__ constructor). Our class "inherits" from another class called Persistent . It does this so that when we change an attribute (i.e., update a field in our table) the ZODB is aware that a change has been made. See Rules for Writing Persistent Classes and the Related Modules sections of the ZODB/ZEO Programming Guide as well as the last half of ZODB for Python Programmers . For more information on writing python classes, see the python tutorials.

With our schema setup, we can now instantiate it (like creating the SQL table, only we're doing it one row at a time):

 >>> newmon = ServiceRow()

That's it! For each new row of data we want to add (service whose data we want to store), we will do the same thing: instantiate the ServiceRow schema. Once that's done, we can start adding/updating data. We'll give it a unique id and add some test data:

 >>> newmon.id = 'myserver.hostingcompany.com-httpd' 
 >>> newmon.name = 'Apache Web Server'
 >>> newmon.description = 'This is the staging Web Server Service for 16 low-volume clients'
 >>> newmon.service = 'httpd'
 >>> newmon.hostname = 'myserver'
 >>> newmon.state_current = 'OK'
 >>> newmon.state_last = 'OK'
 >>> mondb[newmon.id] = newmon
 >>> get_transaction().commit()

Code Breakdown : Most of this should be pretty evident: we are simply giving string values for various fields (properties). For the id, we choose a unique value: the FQDN uniquely determines the server and there is only one httpd service we will be monitoring per machine, so these two give us a good id . (We won't worry about adding date/time data right now.) However, if all we did was make assignments and then quit, the data wouldn't save. In the second to last line, we give this data set a name (where the key is the unique id) in the database. The last line writes the data to the database by using the commit command.

Though you can't prove it yet, you've successfully added data to a ZODB database! But how do we look at it?

On to Part IV

Comment

Discussion icon incorrect sample

Posted by: xavier_g at 2006-03-25

from Persistence import Persistent

should be

from persistent import Persistent

at least in zodb 3.6

Comment

Discussion icon another bug

Posted by: xavier_g at 2006-03-25

get_transaction seems to have been depricated since 3.4. Now you write:

import transaction transaction.get()