I just merged, within the
ZODB trunk (3.7), a branch
implementing an after commit hook support on transaction objects. This
should be available with the 3.7a1 release.
In the meanwhile, you can grab a svn checkout of the ZODB trunk, if you
want to try out, including this feature over there :
http://svn.zope.org/ZODB/trunk/
Or, if you are a lucky
CPS
developer,
CPSCompat
already applies a patch on ZODB 3.6 allowing you to take advantage of this
using Zope-2.9. You will even find some API extensions for hook execution
ordering support within
CPSCore.
Motivations
Sometimes, applications want to execute some code after a transaction is
committed. For example, one might want to launch non transactional code
after a successful, or aborted, commit. Or still someone might want to
launch asynchronous code after a commit. A post-commit hook is now
available for such use cases.
At Nuxeo, we needed this for a while for various reasons :
- CPS and non transactional RDF db setup (such as redland)
- Zope3 and non transactional lucene setup (FSDirectory backend)
(Note, I'll post about
Zope3 and lucene integration using PyLucene pretty soon)
- Eclipse / CPS application specifics for a customer.
- CPS asynchronous indexation
- User Notification when a sensitive commit succeed or abort.
- etc...
I'm sure this will be useful in various use cases in the future as we are
considering more and more Zope, and especially Zope3, as an integration
platform taking advantages of various technologies
around. It's another topic but I guess it was worth mentioning it quickly
here.
Implementation details
Here is the method exposed by the ITransaction interface with the associated
comment :
>>> def addAfterCommitHook(hook, args=(), kws=None):
... """Register a hook to call after a transaction commit attempt.
...
... he specified hook function will be called after the transaction
... commit succeeds or aborts. The first argument passed to the hook
... is a Boolean value, true if the commit succeeded, or false if the
... commit aborted. `args` specifies additional positional, and `kws`
... keyword, arguments to pass to the hook. `args` is a sequence of
... positional arguments to be passed, defaulting to an empty tuple
... (only the true/false success argument is passed). `kws` is a
... dictionary of keyword argumet names and values to be passed, or
... the default None (no keyword arguments are passed).
...
... Multiple hooks can be registered and will be called in the order they
... were registered (first registered,first called). This method can
... also be called from a hook an executing hook can register more
... hooks. Applications should take care to avoid creating infinite loops
... by recursively registering hooks.
...
... Hooks are called only for a top-level commit. A subtransaction
... commit or savepoint creation does not call any hooks. Calling a
... hook "consumes" its registration: hook registrations do not
... persist across transactions. If it's desired to call the same
... hook on every transaction commit, then addAfterCommitHook() must be
... called with that hook during every transaction; in such a case
... consider registering a synchronizer object via a TransactionManager's
... registerSynch() method instead.
... """
>>>
Examples
Here is the tutorial doctest available within the ZODB transaction tests
:
Let's define a hook to call, and a way to see that it was called.
>>> log = []
>>> def reset_log():
... del log[:]
>>> def hook(status, arg='no_arg', kw1='no_kw1', kw2='no_kw2'):
... log.append("%r arg %r kw1 %r kw2 %r" (status, arg, kw1, kw2))
Now register the hook with a transaction.
>>> import transaction
>>> t = transaction.begin()
>>> t.addAfterCommitHook(hook, '1')
We can see that the hook is indeed registered.
>>> [(hook.func_name, args, kws)
... for hook, args, kws in t.getAfterCommitHooks()]
[('hook', ('1',), {})]
When transaction commit is done, the hook is called, with its arguments.
>>> log
[]
>>> t.commit()
>>> log
["True arg '1' kw1 'no_kw1' kw2 'no_kw2'"]
>>> reset_log()
A hook's registration is consumed whenever the hook is called.
Since the hook above was called, it's no longer registered:
>>> len(list(t.getAfterCommitHooks()))
0
>>> transaction.commit()
>>> log
[]
The hook is only called after a full commit, not for a savepoint or subtransaction.
>>> t = transaction.begin()
>>> t.addAfterCommitHook(hook, 'A', dict(kw1='B'))
>>> dummy = t.savepoint()
>>> log
[]
>>> t.commit(subtransaction=True)
>>> log
[]
>>> t.commit()
>>> log
["True arg 'A' kw1 'B' kw2 'no_kw2'"]
>>> reset_log()
If a transaction is aborted, no hook is called.
>>> t = transaction.begin()
>>> t.addAfterCommitHook(hook, ["OOPS!"])
>>> transaction.abort()
>>> log
[]
>>> transaction.commit()
>>> log
[]
The hook is called after the commit is done, so even if the
commit fails the hook will have been called. To provoke failures in
commit, we'll add failing resource manager to the transaction.
>>> class CommitFailure(Exception):
... pass
>>> class FailingDataManager:
... def tpc_begin(self, txn, sub=False):
... raise
... def abort(self, txn):
... pass
>>> t = transaction.begin()
>>> t.join(FailingDataManager())
>>> t.addAfterCommitHook(hook, '2')
>>> t.commit()
Traceback (most recent call last):
...
CommitFailure
>>> log
["False arg '2' kw1 'no_kw1' kw2 'no_kw2'"]
>>> reset_log()
Let's register several hooks.
>>> t = transaction.begin()
>>> t.addAfterCommitHook(hook, '4', dict(kw1='4.1'))
>>> t.addAfterCommitHook(hook, '5', dict(kw2='5.2'))
They are returned in the same order by getAfterCommitHooks.
>>> [(hook.func_name, args, kws) #doctest: +NORMALIZE_WHITESPACE
... for hook, args, kws in t.getAfterCommitHooks()]
[('hook', ('4',), {'kw1': '4.1'}),
('hook', ('5',), {'kw2': '5.2'})]
And commit also calls them in this order.
>>> t.commit()
>>> len(log)
2
>>> log #doctest: +NORMALIZE_WHITESPACE
["True arg '4' kw1 '4.1' kw2 'no_kw2'",
"True arg '5' kw1 'no_kw1' kw2 '5.2'"]
>>> reset_log()
While executing, a hook can itself add more hooks, and they will all
be called before the real commit starts.
>>> def recurse(status, txn, arg):
... log.append('rec' + str(arg))
... if arg:
... txn.addAfterCommitHook(hook, '-')
... txn.addAfterCommitHook(recurse, (txn, arg-1))
>>> t = transaction.begin()
>>> t.addAfterCommitHook(recurse, (t, 3))
>>> transaction.commit()
>>> log #doctest: +NORMALIZE_WHITESPACE
['rec3',
"True arg '-' kw1 'no_kw1' kw2 'no_kw2'",
'rec2',
"True arg '-' kw1 'no_kw1' kw2 'no_kw2'",
'rec1',
"True arg '-' kw1 'no_kw1' kw2 'no_kw2'",
'rec0']
>>> reset_log()
If an after commit hook is raising an exception then it will log a
message at error level so that if other hooks are registered they
can be executed. We don't support execution dependencies at this
level.
>>> mgr = transaction.TransactionManager()
>>> do = DataObject(mgr)
>>> def hookRaise(status, arg='no_arg', kw1='no_kw1', kw2='no_kw2'):
... raise TypeError("Fake raise")
>>> t = transaction.begin()
>>> t.addAfterCommitHook(hook, ('-', 1))
>>> t.addAfterCommitHook(hookRaise, ('-', 2))
>>> t.addAfterCommitHook(hook, ('-', 3))
>>> transaction.commit()
>>> log
["True arg '-' kw1 1 kw2 'no_kw2'", "True arg '-' kw1 3 kw2 'no_kw2'"]
>>> reset_log()
Test that the associated transaction manager has been cleanup when
after commit hooks are registered
>>> mgr = transaction.TransactionManager()
>>> do = DataObject(mgr)
>>> t = transaction.begin()
>>> len(t._manager._txns)
1
>>> t.addAfterCommitHook(hook, ('-', 1))
>>> transaction.commit()
>>> log
["True arg '-' kw1 1 kw2 'no_kw2'"]
>>> len(t._manager._txns)
0
>>> reset_log()
The transaction is already committed when the after commit hooks
will be executed. Executing the hooks must not have further
effects on persistent objects.
Start a new transaction
>>> t = transaction.begin()
Create a DB instance and add a IOBTree within
>>> from ZODB.tests.util import DB
>>> from ZODB.tests.util import P
>>> db = DB()
>>> con = db.open()
>>> root = con.root()
>>> root['p'] = P('julien')
>>> p = root['p']
>>> p.name
'julien'
This hook will get the object from the `DB` instance and change the flag attribute.
>>> def badhook(status, arg=None, kw1='no_kw1', kw2='no_kw2'):
... p.name = 'jul'
Now register this hook and commit.
>>> t.addAfterCommitHook(badhook, (p, 1))
>>> transaction.commit()
Nothing should have changed since it should have been aborted.
>>> p.name
'julien'
>>> db.close()
I let you check the code for further implementation details.
(Post originally written by Julien Anguenot on the old Nuxeo blogs.)
Subscribe to Feed
Follow us on Twitter
Recent Comments
Our tweets