History for ApplicationLevelConflictResolution

??changed:
-
ZODB uses an optimistic concurrency control mechanism.  Objects are
not locked. Conflicting writes are checked when object changes are
committed. If a conflict is detected, then a 'ConflictError' is raised
and the request is retried. This approach works fine if conflicts are
rare. If conflicts occur frequently, then, at best, performance
degrades because requests must be resubmitted, or, at worst, requests
are rejected and raise errors due to conflicts.

Sometimes, there may be "hot spots" in an application
that get a lot of simultaneous writes. These hot spots can often be designed
away. An alternative is to provide application-level logic for sorting
out the changes made by conflicting writes.

A new interface is proposed to allow object authors to provide a
method for resolving conflicts. When a conflict is detected, then the
database checks to see if the class of the object being saved defines
the method, '_p_resolveConflict'. If the method is defined, then the
method is called on the object. If the method succeeds, then the object
change can be committed, otherwise a 'ConflictError' is raised as
usual.

*Conflict resolution, as proposed here, was added to ZODB in Zope 2.3.*

_p_resolveConflict(oldState, savedState, newState) -- 
   Return the state of the object after resolving different changes.

   Arguments:

   oldState -- The state of the object that the changes made by the
               current transaction were based on.

	       The method is permitted to modify this value.

   savedState -- The state of the object that is currently stored in
		 the database. This state was written after oldState and reflects
		 changes made by a transaction that committed before the current
		 transaction. 

  	         The method is permitted to modify this value.

   newState -- The state after changes made by the current transaction.

	       The method is *not* permitted to modify this value.

   This method should compute a new state by merging changes reflected
   in savedState and newState, relative to oldState.

   Consider an extremely simple example, a counter::

     class PCounter(Persistent):
       
       _value = 0

       def inc(self): self._value=self._value+1

       def _p_resolveConflict(self, oldState, savedState, newState):

           # Figure out how each state is different:
           savedDiff=savedState!['_value']-oldState!['_value']
           newDiff=newState!['_value']-oldState!['_value']
	   
	   # Apply both sets of changes to old state:
	   oldState!['_value'] = oldState!['_value'] + savedDiff + newDiff
	   
	   return oldState

   If the method cannot resolve the changes, then it should return
   None.

PJE -- It's an interesting idea, and would certainly address simple
    conflict cases.  But what if one used the value of the counter to
    do something else?

    Jim -- It would depend if the "something else" caused any other
      conflicts. For conflict resolution to prevent a conflict error,
      all conflicting updates must be resolvable.  This is not a
      problem, since, generally, only a few (typically one) of the
      changed in a transaction should conflict.

    In that case, the conflict cannot be "backed
    out" by only considering the state of one object. 

    Jim -- The state's of all conflicting objects will be
    considered. If all of the conflicts can be resolved, the
    transaction will not have to be retried.

    Adding objects to a ZCatalog for example, cannot be de-conflicted
    in this way.

    Jim -- Sure it can, in most cases.  We will update the indexing
    machinery used by the catalog to leverage conflict resolution. If
    a change doesn't cause a bucket to split, then a conflict will be
    avoided, even if changes affect multiple indexes.

    It seems to me that most of the really interesting (to me, anyway)
    conflict cases are the ones that this scheme cannot address,
    because the "state" that you are interested in is spread out
    across more than one Persistent instance.  What are your thoughts
    on this type of scenario?

    Jim -- My thought are that changes affecting multiple objects are
    not a problem, as long as all of the onflicts can be resolved.

Toby Dickenson -- Whats the difference between 'newState' and 'self'?
     What happens to changes to 'self'. 

    Jim -- The conflict resolution method should not modify self.
       Effects of changes to self are undefined. The 'self' argument
       is really there to hand the method on. :)

Toby Dickenson -- How does this interact with subtransactions?

  Jim -- It doesn't really.  Note, however, that conflicts are
    detected when the entire transaction is committed, so conflict 
    resolution will only be performed when the outer transaction commits.

Toby Dickenson -- I assume changes to savedState and oldState are
  discarded (unless that object is the one that gets returned, as in the
  example.)

  Jim -- Right.

John Heintz -- When thinking through this issue I see that many conflicts would occur in PersistentMapping and PersistentList objects, not my own domain objects.  This "one object at a time" approach would really discourage using any persistent collection helpers.  Here is my idea for an alternative mechanism: DualCacheConflictResolution