You are not logged in Log in Join
You are here: Home » Members » jim » ZODB » BrewingIdeas

Log in
Name

Password

 
 
FrontPage » RelationalStorage »

BrewingIdeas

Version and Data Tables Should Be Combined
------------------------------------------
A previous version of the relational storage table structures dictates that
version data should be kept in a separate table.  Because the most common
queries join this table with the data table, and because it doesn't seem
to buy us anything in the way of speed, it makes sense to just
consolidate the data and versions tables.  Here is a diagram of the proposed
table structure:


(image last updated:

This table structure should be modified to include a pickle table.

Object Creation, Versioning, and Undo State Chart ------------------------------------------------- A record for an object needs the following properties:

serial (timestamp generated when record is written) nv (non-version serial number) pre (previous object serial number for undo) dataserial (a serial number which contains the pickle if it isn't in this record)

We need the following methods: LoadMethod (implicit) GetHistorical (implicit) StoreMethod FinishMethod UndoMethod TransactionExists TransactionIsNotUndoable MakeDummyUndoRecords UndoPopulatedRecords UndoUnpopulatedRecords ClearUndoStatus GetCurrentOids GetCurrent InsertObject InsertTrans SetStatus CommitVersion GetCurrentOidsInVersion InsertCommittedVersion ClearVersionStatus

One version of the Interbase tables left out the dataserial element. This was a mistake. It needs to be there.

We may have some record locking issues with the scheme outlined below. Actually, object locking issues. All writes to the DB other than pack and status-setting stuff is an append. There will be an issue if one thread stores a copy of the same object another thread is trying to store at the same time for whatever reason. I'm not sure I understand it quite fully, but Evan and Tres say that it can probably be co-opted at the Interbase level. If it can't, we'll need to forego the use of the z_status flag in the zodb_data table and instead use in-mem indexes.

On to the homegrown state chart...

The Tedious Life Of An Object in the InterbaseStorage:

--------------------------------------- I. A object gets created:

serial nv pre dataserial A None None None

Assertion: if the object is brand new, write it with no nv, no pre, and no dataserial.

Methods used:

StoreMethod --> GetCurrent --> InsertObject --> FinishMethod --> InsertTrans

Things that happened in StoreMethod:

GetCurrent (oid) returned None InsertObject (oid, newserial, "", "c", p, len(p), version, "", "")

Things that happened in FinishMethod:

InsertTrans (serial, u, d, e) --------------------------------------

-------------------------------------- II. It gets modified in a nonversion transaction and a new object takes its place:

serial nv pre dataserial B None A None

Assertion: if a nonversion object is not new, write it with nv=None, pre=oldserial and dataserial=None.

Methods used:

StoreMethod --> GetCurrent --> SetStatus --> InsertObject --> FinishMethod --> InsertTrans

Things that happened in StoreMethod:

GetCurrent (oid) returned A SetStatus (oid, A's serial, "h") InsertObject (oid, newserial, A's serial, "c", p, len(p), version, "", "")

Things that happened in FinishMethod:

InsertTrans (serial, u, d, e) --------------------------------------

-------------------------------------- III. Then the object is modified within a version for the first time:

serial nv pre dataserial C B None None

Assertion: if a version object is new within the version, write it with nv=nonversion serial, pre=None, dataserial=None.

Methods used:

StoreMethod --> GetCurrent --> SetStatus --> InsertObject --> FinishMethod --> InsertTrans

Things that happened in StoreMethod:

GetCurrent (oid) returned B SetStatus ("h", oid, B's serial) InsertObject (oid, newserial, "", "c", p, len(p), version, B's serial, "")

Things that happened in FinishMethod:

InsertTrans (newserial, u, d, e)

--------------------------------------

-------------------------------------- IV. The object is remodified within a version:

serial nv pre dataserial D B C None

Assertion: if a version object is not new within the version, write it with nv=nonversion serial, pre=old current serial within version, dataserial=None.

Methods used:

StoreMethod --> GetCurrent --> SetStatus --> InsertObject --> FinishMethod --> InsertTrans

Things that happened in StoreMethod:

GetCurrent (oid) returned C SetStatus ("h", oid, c's serial) InsertObject (oid, newserial, C's serial, "c", p, len(p), version, B's serial, "")

Things that happened in FinishMethod:

InsertTrans (newserial, u, d, e) --------------------------------------

-------------------------------------- IV. The version is committed, and a new record is created for the object:

serial nv pre dataserial E None D D

Assertion: If a version is committed, write a new object where nv=None, pre=most current serial for this oid from version, dataserial = most current serial for this oid from version

Methods used:

CommitVersion --> GetCurrentOidsInVersion --> InsertCommittedVersion --> ClearVersionStatus --> FinishMethod --> InsertTrans

Things that happened in CommitVersion:

GetCurrentOidsInVersion (src) InsertCommittedVersion (newserial, src) ClearVersionStatus (src)

Things tha happened in FinishMethod:

InsertTrans(newserial, u, d, e) --------------------------------------

-------------------------------------- V. The object is remodified outside a version:

serial nv pre dataserial F None E None

Assertion: if a nonversion object is not new, write it with nv=None, pre=old current serial outside any version and dataserial=None.

Methods used:

StoreMethod --> GetCurrent --> SetStatus --> InsertObject --> FinishMethod --> InsertTrans

Things that happened in StoreMethod:

GetCurrent (oid) returned E SetStatus (oid, E's serial, "h") InsertObject (oid, newserial, E's serial, "c", p, len(p), version, "", "")

Things that happened in FinishMethod:

InsertTrans (serial, u, d, e)

--------------------------------------

-------------------------------------- VI. Undo rolls back the modification, and a new record is created for the object:

serial nv pre dataserial G None D D

Assertion: Undo creates a new object, sets nv = the previous object's nv, pre = the previous object's pre (*), and dataserial = the previous object's dataserial or the previous object's serial if the previous object's dataserial is None.

(*) if pre is None, make a dummy record with a null pickle and null dataserial.

UndoMethod --> TransactionExists --> TransactionIsNotUndoable --> MakeDummyUndoRecords --> UndoPopulatedRecords --> UndoUnpopulatedRecords --> ClearUndoStatus

The majority of undo processing happens in the database. There is little Python logic in UndoMethod. --------------------------------------

-------------------------------------- VII. The object is modified within a version again (no one expects the version inquisition!):

serial nv pre dataserial H G None None

Assertion: if a version object is new within the version, write it with nv=nonversion serial, pre=None, dataserial=None. --------------------------------------

-------------------------------------- VIII. The object is remodified within a version:

serial nv pre dataserial I G H None

Assertion: if a version object is not new within the version, write it with nv=nonversion serial, pre=old versioned object serial, dataserial=None. --------------------------------------

-------------------------------------- VIIII. The remodification of the object is undone:

serial nv pre dataserial J G None H

Assertion: Undo creates a new object, sets nv = the previous object's nv, pre = the previous object's pre, and dataserial = the previous object's dataserial or the previous object's serial if the previous object's dataserial is None. --------------------------------------

-------------------------------------- X. The version that contains the object is aborted:

serial nv pre dataserial K None J D

Assertion: If a version is aborted, write a new object where nv=None, pre=most current serial for this oid from version, dataserial = either 1) the most current serial for this oid from nonversion index or 2) the most current serial's dataserial if a dataserial exists in its record. --------------------------------------

-------------------------------------- XI. The object is deleted outside any version:

containing object serial nv pre dataserial L None K None

Assertion: if a nonversion object is not new, write it with nv=None, pre=oldserial and dataserial=None. (same as II) -- this is actually really just a modification of the containing object that happens to make it not show up anymore. --------------------------------------