I'm running into a problem for CPSMailAccess (the next CPS webmail) It has some enhanced features that needs to keeps things on zodb side. So a synchro process is called to update things on zodb side when the imap server changes.

When users synchronizes their mailbox, it can take a few seconds, sometimes more, depending on the size of the box. The first synchro takes a long of time too, because a lot of things are done, like indexings. That makes a real problem for large scales intranets when 3000 people come to the office and synchronizes their webmail. The 4 little available zope threads cannot handle that.

Eric Barrocca pointed me out the existence of ZAsync product from Zope Corp. extract from the readme :

ZAsync is a Zope 2 product that allows Twisted to asynchronously perform tasks in a separate process over ZEO on behalf of Zope requests. Intended use cases (only the first two are realized) include performing large numbers of long-running external database queries without tying up Zope; allowing "fire and forget" Zope transactions that return control to the app server while scheduling long-running Zope transactions to be performed in the background and potentially on another physical machine; connecting over a long-running socket with other asynchronous processes such as instant message servers; and performing intelligent agent types of tasks.

This product seems to fulfill the need since it can launch process on the background, thus letting the 4 "frontal" threads available for "regular" user queries. The only care would be to protect the synchronize methods into a critical sectionto avoid conflicts. This product brings some dependencies though (twisted, ZEO, etc...) but it worth a shot for big scale intranets, as I can probably set the ZEO node in charge of background process to a high number of threads.

part #2 will be the results of my tries, stay tuned :-)

Here's an interview of GvR I have made, gathering questions from some of my friends from AFPY. This is the english version I am dropping in my blog.
French version here :
http://afpy.org/Members/tarek/afpynews.2005-04-10.2791031007


- Why did you create Python in the first place ?


I was working (with a bunch of other folks) on a new distributed operating system named Amoeba (it's quite famous, Google for its paper legacy).

We wanted Amoeba to be as useful as Unix for our daily work, but it was lacking a scripting language. So I set out to design my own. The explicit goal was for it to be somewhere in the middle between Shell programming (too high-level) and C programming (too low-level), and I wanted it to be cross platform.
(Maybe I wasn't too optimistic of Amoeba's success.  :-)

- What are the strengths of Python ?


I'm not really the right person to ask, you know... I'm so immersed in it that it's hard to give you an objective list of advantages. Just look it up on python.org. My own view is that it makes my job as a programmer much easier than most other languages because of all the stuff that Python takes care of: memory management, platform dependencies, etc.


- My little brother starts to learn programming, how can I convince
  him to learn with Python ?

Download PyGame, a cool toolkit for creating graphics games, and give him some simplified APIs that let him create simple graphics objects and animate them.

There are several English Python boosk for beginners (see
http://www.python.org/moin/IntroductoryBooks) and even a French one
(http://www.python.org/moin/FrenchPythonBooks).


- If you could change some things that have been done in the
  langage, what would it be ?

Drop lambda, filter, map and reduce. Make range() behave like xrange(). Make keys() and many other methods and built-ins return iterators rather than lists.


- What are the new features, or the upcoming ones, that are the most
  exciting ?

In Python 2.4, decorators and generator expressions are the coolest
new things. For more, see http://python.org/doc/2.4/whatsnew/whatsnew24.html.


- You're back from Pycon. What were the most interesting things,
  events you have seen over there?

Hard to choose between IronPython (the Python port to .NET, now supported by Microsoft) or the keynote about Google's extensive use of Python.

- What do you think about the actual python community activity ?

I think the community is thriving.

- What's the goal of the PSF ?

In my opinion its primary goal is to exist as a safe neutral non-profit organization that "owns" Python, to make sure that it always remains free and never ever gets gobbled up by some big corporation.

In the opinion of other members it is now also important for the PSF
to promote Python and Python-related activities, for example
education, and sponsoring coding projects.

I think it's very important that the PSF has enough money to be able
to continue to run PyCon -- even if PyCon has made a moderate profit
each year, it is an enormous financial responsibility that could not
be done without the stable financial situation of the PSF.

- Does Python is as popular overseas as in Europe ?

It's very popular everywhere. There are huge user groups being
established in Brazil and Argentina. I heard that a few years ago
there 700 people showed up on a Python event in South Korea. Etc.

- Does a specification (ISO, ECMA, ..) is planned for Python and when ?

No, never. I don't see the point.

- Is their such a big gap today between a dynamic languages like
  Python and compiled ones as it used to be ?

Java is closing some of the gap by adding automatic memory management
(they call it Garbage Collection) to the repertoire of compiled
languages. It is also pioneering techniques such as JIT that will
eventually benefit dynamic languages; the PyPy project in Europe is
attempting to do this for Python (in some sense).

- Which external libraries that brings new features to Python (PEAK,
  ...) do you think are the most interesting ?


Since you mention it, I think PEAK is way too advancedfor most Python users.

I think Twisted is one of the most interesting third party libraries,
and also wxPython (too bad it's based on C++).

- Which python open source projects are the most do you think are
  the most interesting at this time ?

Twisted, Zope. I'm probably missing the *really* important ones
because I'm not using much 3rd party Python code myself (I live in my
own self-contained "Not Invented Here world").

- What do you think about Zope evolution ?

I think Zope 3 is a big step forward.

- What do you think about Pyrex ? Should'nt it be used to rewrite
  some parts of Python ?

Perhaps. Rewriting anything as big as Python from scratch is always a
big risk, it usually takes longer than expected and has unexpected
problems. I'd rather rewrite some parts of Python in Python itself.
The PyPy folks think so too.  :-)

- Which one of those projects created to speed Python do you think
  are the most interesting : Pyrex, Psyco,    Pypy,...

Definitely PyPy. It is rolling up Psyco.

OTOH, Pyrex is probably a lot more practical today -- PyPy has a long
way to go (currently they are 1000x *slower* than CPython  :-) .

- Talking about speed, wich way Python will head towards : less CPU
  consuming or memory managment   optimisation ?

I expect to see a trend towards speeding things up at the cost of a
little more memory usage. But we've also reduced memory wastage in
certain cases while at the same time speeding things up (e.g.
list.append() in 2.4).

- How come IronPython is doing better than CPython ?

Probably a choice of benchmark. There are some areas (like exception
handling) where it's actually a lot slower.

- How much are you involved in the community nowadays ?

Mostly as a token figure -- I don't make most of the decisions any
more, although I end up breaking ties when the rest of the community
can't agree. I am mostly still interested in evolution of the language
(see my blogs) rather than the library or platform support.

- What are your actual professionnal projects ?

Since 2003 I am working for Elemental Security, a start-up company
founded by Dan Farmer to create a new type of enterprise security
application. The application is structured as a server and agents, and
the agent is mostly written in Python. I've also designed a
special-purpose language named Fuel which is well suited for writing
security checking scripts. It looks a bit like a statically-typed
subset of Python.

- What code editor are you using ? your OS, Desktop ?

XEmacs and vi. Red Hat. I don't recall what my desktop software is
running (KDE or Gnome); the applications visible are mostly Firefox,
Xemacs, and lots of xterm windows.

- Are you still having fun with Python ?

Oh yeah.

- When will you be candidate for US presidentials ?

I can't, since I wasn't born here. But my son can be.  :-)

- Thank you Guido !