An IMAP proxy server to speed up IMAP side full text search

It's been two days that i am googling to try to find an IMAP Proxy program that would let me speed up this kind of search:

find me all mails that contains the word "cps" in their body.

in my 30k mails mailbox, it lasts over to 30 seconds, depending on server charge.

Why mail bodies are not indexed on Zope side

I have decided a while ago to stop indexing mail bodies on zope side, as this is not reliable for big intranets, big account, etc... It is not a problem of speedness, as a primary indexing of a big mailbox was taking ages, because zasync could be use there, but it's more a problem of making the ZODB growing very, very fast and big.

A filesystem storage linked to the ZODB ?

This would bring a lot of hard work on setting up everything to work fine together. (one more player, you need a chaire and more room around he table)

What could be done on IMAP side: a Python Proxy

Back to my pure IMAP search: I find this too slow, and I don't really understand why they are no options in IMAP servers to index the messages to speed it up things.... I can't find anything related on such tools like Courier IMAP.

I don't want to use ZOE either, I want a server side tool that can work with all mail clients.

I tried to find some proxies that would do this kind of work, to be placed between client requests and the real IMAP server, but I have just found some IMAP Proxy that would make statefull connections for webmail apps that reconnects everytime something is done. (Connection are statefull in CPSMailAccess)

The code of such a tool is quite simple and short but has to be very robust:

A SocketServer.TCPServer that would listen to IMAP ports (both SSL and not SSL) can:

  • Catch all body search queries and search in a system file that contains index of message bodies instead of calling the real IMAP server
  • Index incoming message before passing them to the IMAP server.

This would speed up all full text searches and bring them as fast as classical header searches, even if it's coded in Python.

Now, this sounds easy said this way, but this is a first thaught so I guess I do not see all problems involved.

Important announcement: Join the Nuxeo team and contribute to the Nuxeo project! We have open positions in France and the UK for open source Java EE developers and sales engineers, both junior and senior.

Like this post? Share it:


Trackback Pings

Trackback URL for this entry:
http://blogs.nuxeo.com/sections/blogs/tarek_ziade/2005_06_10_an_imap_proxy_server_to/tbping
Posted by Tarek Ziadé @ 06/10/2005 12:15 PM. - Categories: cps, linux, web, zope, zope3 -  0 comments

Nuxeo Bloggers: Log in!
Nuxeo - Indesko - Nuxeo 5 Project
All content is copyrighted by their author.
CPSSkins is Copyright © 2003-2006 by Jean-Marc Orliaguet. | CPS is Copyright © 2002-2006 by Nuxeo SAS.