ZAsync for asynchronous Zope actions part #1

I'm running into a problem for CPSMailAccess (the next CPS webmail) It has some enhanced features that needs to keeps things on zodb side. So a synchro process is called to update things on zodb side when the imap server changes.

When users synchronizes their mailbox, it can take a few seconds, sometimes more, depending on the size of the box. The first synchro takes a long of time too, because a lot of things are done, like indexings. That makes a real problem for large scales intranets when 3000 people come to the office and synchronizes their webmail. The 4 little available zope threads cannot handle that.

Eric Barrocca pointed me out the existence of ZAsync product from Zope Corp. extract from the readme :

ZAsync is a Zope 2 product that allows Twisted to asynchronously perform tasks in a separate process over ZEO on behalf of Zope requests. Intended use cases (only the first two are realized) include performing large numbers of long-running external database queries without tying up Zope; allowing "fire and forget" Zope transactions that return control to the app server while scheduling long-running Zope transactions to be performed in the background and potentially on another physical machine; connecting over a long-running socket with other asynchronous processes such as instant message servers; and performing intelligent agent types of tasks.

This product seems to fulfill the need since it can launch process on the background, thus letting the 4 "frontal" threads available for "regular" user queries. The only care would be to protect the synchronize methods into a critical sectionto avoid conflicts. This product brings some dependencies though (twisted, ZEO, etc...) but it worth a shot for big scale intranets, as I can probably set the ZEO node in charge of background process to a high number of threads.

part #2 will be the results of my tries, stay tuned :-)