Nuxeo Developers Blog

[GramOOo] Bilan

J'arrive bientôt à la fin de mon stage chez  Nuxeo/Indesko.

Le projet du correcteur grammatical n'est évidemment pas fini, mais nous arrivons à quelques résultats encourageants pour la suite.

D'ici à la fin du mois, sera disponible mon rapport de stage. Celui-ci présentera tout le travail effectué, les résultats, les limites du système et les perspectives futures.

Il servira de guide à quiconque continuera le projet.

Ce stage marquant la fin de mes études, j'aurai moi-même la possibilité d'y contribuer encore un peu...

(Post originally written by Myriam Lechelt on the old Nuxeo blogs.)

Zope 2.8.0b2 out

I have spent some time this week trying out Zope 2.8 in it's second beta incarnation. There are a lot of basic groundwork changes in Zope 2.8, which is evident from the deluge of deprecation warnings you get when using it with applications like CPS. But other than that, there was only a couple of few things that didn't work straight away, and these have now been fixed.

Zope 2.8 it also includes Five, to get people started on the path to Zope 3 development. This is amazingly cool. The version shipping with Zope 2.8b2 is usable unless you want to internationalize your ZPTs so users can change the interface language. If you want that you need to wait for Five 1.1. Luckily, Zope allows you to easily install a new version of Five in your products directory, and it will use that version instead, so keeping Zope 2.8 up with Five developments will be easy.

(Post originally written by Lennart Regebro on the old Nuxeo blogs.)

Why people don't use webmails

No matter how terrific a webmail application can be, people will almost never drop their regular mail client.
 
You can make a webmail way more efficient, featured, than a heavy mail client, users will just think it will be useful to get to their mailbox when they are out of the office. But they will never drop Thunderbird.

This is too bad since one of the goal of CPSMailAccess, the upcoming CPS webmail, is to fill the gap between datas contained in groupwares and intranets, making mails behave like any other portal document. And keeping Thunderbird as the master mail application won't help on this.

After looking at users behavior, I found out that the only reason they act that way is the lack of desktop integration of a webmail app.

A web application sleeps on your desktop, and this is so annoying to users that they just close the darn' firefox window.

More desktop integration and interaction are the keys.

If you provide a small desktop application that pops whenever a mail is dropped in your mailbox, and let you quickly get to your webmail inbox, or let you quickly get to the mail editor, people will drop their mail client.

Thanks to Python this can be done quite easily to support most plateforms, and I will probably start to code later a small desktop app to be released within CPSMailAccess.

That reminds me a thread in Z3lab Mailing list we had, involving people from the Z3 ECM project: I keep on thinking that Zope can be the central part of intranet and desktop apps and that a lot of small of desktop apps interacting with zope servers will start to come up in the next years.

(Post originally written by Tarek Ziadé on the old Nuxeo blogs.)

Zope3 / ECM : Project launched !

That's official ! The new Zope3 / ECM project is launched !

See the annoucement on the project portal.

The main goal of this project is to build the next-generation Open Source ECM Platform on top of Zope 3 able to compete with the major ECM solutions on the market such as Documentum, OpenText, etc.

What is Z3Lab ?

Z3Lab aims at building a strong community leveraging the experiences of both individuals and companies that are involved in the Zope ECM/ CMS market nowadays to create the best ECM platform on the market and offer high-value professional services to customers.

ECM Platform

The project's main goals are :

• to unify the whole Zope/CMS-involved community to drastically reduce waste of resources (doing twice or more equivalent components / features).
• to spread Zope 3 technology on the market by providing a world class ECM Platform with companies providing consulting worldwide and backing the project.

The project's goals are really ambitious but considerint the amount of excellent Zope hackers all around the world it is more than an acceptable mission :)

The project's just starting and the first iteration will be to write specifications and to come up with the architecture of the framework. If you are interested in ECM architecture join the mailing list !

Couple of documents are already published on the portal such as a concept map and an overview of the ECM Plateform.

Hope to see you all on the mailing list soon :)

(Post originally written by Julien Anguenot on the old Nuxeo blogs.)

[GramOOo] [Languagetool] Principes

Languagetool est un correcteur de style et de grammaire pour l'anglais créé par Daniel Naber. Il est écrit en Python.
C'est un système « rule-based », c'est-à-dire basé sur des règles syntaxiques écrites à la main.
Il procède à plusieurs étapes d'analyse grammaticale.
Il faut découper le texte en phrases, puis à chaque espace pour obtenir une liste de mots. Il étiquette ces mots (phase de tagging), puis il découpe en syntagmes (chunks) c'est-à-dire des groupes nominaux, verbaux etc. Ensuite, il applique des règles écrites en XML.
Le tagging est basé sur des probabilités et des règles. Il créé deux tableaux associatifs. Le premier expose le mot, son étiquette (c'est-à-dire sa catégorie grammaticale et des informations morphologiques) et la probabilité qui correspond au tag. Le deuxième montre une suite de deux tags avec la probabilité correspondant à cette séquence (par exemple sequence [(DET, AJ0)] = 0,1] signifie que la probabilité qu'un adjectif suive un déterminant est de 0,1).
Plusieurs mots peuvent être ambigus, on prend alors une suite une séquence de trois mots et on regarde les probabilités que ces trois mots se suivent (par exemple qu'un déterminant soit suivit d'un nom et d'un adjectif) et on sélectionne la plus haute probabilité. On avance d'un mot et on continue le traitement.
Daniel Naber ajoute une phase à son tagger probabiliste. Si le résultat du tagger n'est pas correct et des mots nécessites d'être désambiguïsés, des règles écrites manuellement sont appliquées.
Ensuite vient la phase de chunking qui est basée sur des règles. Par exemple, un NP (noun phrase c'est-à-dire un syntagme nominal) se compose d'un déterminant et d'un nom.
Enfin vient la phase de grammaire basée sur de règles XML. Ces règles sont dépendantes des étiquettes. Elles se composent de plusieurs éléments :
- id et name qui sont l'identifiant et le nom de la règle
- pattern qui décrit ce qui est incorrect,
- message qui est l'énonciation de la règle et qui est destiné à l'utilisateur, et
- example qui est l'exemple de l'erreur.
On a par exemple une règle d'accord sujet verbe.

(Post originally written by Myriam Lechelt on the old Nuxeo blogs.)

[GramOOo] [Grac] principes

Maxime Biais dévelloppe actuellement Grac, un correcteur grammatical indépendant d'une langue particulière et programmé en Python.

Grac nécessite une phase d'apprentissage, afin de créer une base de connaissances probabiliste et une base de règles de grammaire. Il faut donc un corpus étiqueté et un corpus ne contenant aucune erreur grammaticale.

On découpe le texte en phrases et en mots. On traite le texte avec un tagger afin d'étiqueter les mots non ambigus. Puis on utilise un système de probabilités basé sur des enchaînements de trois tags possibles pour étiqueter les mots ambigus. Les enchaînements possibles sont contenus dans une base.

Ensuite, on passe par deux phases de détection d'erreurs grammaticales (GDE). La première est la phase d'apprentissage qui se base sur un corpus étiqueté, sans erreur de grammaire pour créér des règles qui sont une liste de TAG probables. La deuxième phase se base sur ces règles pour déterminer les erreurs. Quand on applique ces règles au texte à traiter, si aucune ne correspond, c'est qu'il y a une faute.

Grac ne possède pas encore de documentation très détaillée car il est encore en cours de dévelloppement.

Les parties GED ne sont à l'heure actuelle pas encore développées.

(Post originally written by Myriam Lechelt on the old Nuxeo blogs.)

Wiki Scripting

The main problem with wiki syntaxes is that they have almost always fixed features. Some wikis are extensible, like Trac, who let you create macros.

CPSWiki is going to have its own system of macros, that will give the user the ability to execute scripts.

Those scripts will be able to interact with the portal through an execution context that will make the portal accessible to the code.

Zwiki has a similar approach in http://zwiki.org/ZwikiCookbook by letting the user execute dtml.

The more interesting way to do it is to create a wiki namespace and to keep all scripts in the wiki folder. This namespace will allow scripts to interact and
 users to construct complex features. (dynamic graph creation, etc.)

This has to come with a library of base scripts that the user can start to manipulate.

Let's get rid of all thoses heavy CMS features and get back to a good o'wiki to create all our sites ;)

(Post originally written by Tarek Ziadé on the old Nuxeo blogs.)

CSS: no more round box headaches

Everytime I am starting to design round boxes, I'm having a big headache.

It's just too complicated for me. Whenever i get close to a good looking box, wich content does not run away on the sides, looking good on my Firefox, and behaving well in my CPS, I get depressed when I switch into IE.

Now I think I will always use this recipe for now one :

gecko engine comes with its own border-radius style, not yet validated in CSS3 but it should be sometimes,

   #mailBody {

     background-color: white;

     margin-top: 10px;

     padding:8px;

     -moz-border-radius-topleft: 12px;

     -moz-border-radius-topright: 12px;

     -moz-border-radius-bottomleft: 12px;

     -moz-border-radius-bottomright: 12px;

   }

This is clean, looks good, and cut off all those small pictures I was trying to stick together.

See the attached file for a screenschot

(Post originally written by Tarek Ziadé on the old Nuxeo blogs.)

All quiet on the Bugtracker front

So far in my life I have used and tried five different bugtracking systems: Bugzilla, Mantis, Zope orgs "Collector", one I wrote myself in Lotus Notes ten years ago, and now lately Trac.

Trac is cool, because it has a Wiki. Other than that, these systems are pretty much all crap, although Trac is quite likely less crap than the others. So this blog started as a complaint over the bad state of bug trackers...

...but, it quickly turned into something less whiney and more constructive; a feature requirement list of bug tracking systems. This are some features I can come up with that I want in a bug tracker. You are welcome to add your own requirements as comments, and if I get a lot of comments, I'll merge this into another blog entry later.

You are also welcome to come with more recommendations of bug trackers. I don't have time to test bug trackers in detail, so if you say bug tracker X has feature Y, I will just believe you. Hype your own product!

1. Workflow

You need to be able to assign tickets to different persons, add comments, reassign and move around, and of course, see what is assigned to you.

2. Flexible schema

Every company/open source community has it's own requirements of what the bug report should include. Lets see some examples of this:

Organizing Projects/Products/Modules/Components.

A company that makes commercial products might want to have products that consist of several component products. For example, Norton SystemWorks include Norton AntiVirus, and Norton DiskDoctor and so on.

Consulting companies need projects that are orthogonal to components.

Therefore there need to be some sort of top-level attribute to each bugreport/ticket (different names, same thing) that selects this product/project, and there must be an attribute to select the component, and these should be orthogonal. A component that is used in several projects/products should not need redefinition in each project/product.

Also, this top level attribute needs to be renamable between project and product (and maybe something else).

Extra candy would be nice an ability to select which modules are used in a project, and a non-orthogonal component collections, or even a component hierarchy. If you have collections of components, these might be called "products", but then again, if your other top hierarchy is "products", they should not. So yet again, the name needs to be configurable. And so should the name "modules" too.

Lets see how the products I tried fulfull this:

Mozilla: Yes! Has a product and a component. These are orthogonal. There is no easy renaming that I know of, but I guess you can change the HTML-forms in worst case. There is no selection of which components are valid for which products, as far as I can see. No non-orthogonal component collections, either, so non of the nice extras are there.

Mantis: No. Has projects and categories but these are not orthogonal. Projects are the all-encompassing major division, and everything else is unique to that project.

Trac: No. Has only components.

Version numbers

Any bug report needs to have a version number field. In a big company, it should be a requiered drop-down box, to make it easier for support to know if this is a known/solved bug that just got reported again. But the numbers in this drop-down box needs to be maintained, and they need to be unique to each product. We all know a small company is not gonna maintain those lists, so it should also be possible to just have a text-field.

Again we see that some configurability here is needed.

Status

What the possible status of a bug report should be causes a lot of discussion when it comes to Zope.orgs collector. This is clearly a matter of different needs. Should the reason for closing be a status or a separate field? depends on what reasons you want to allow, and so on.

I'm sure I could come up with more examples of how the schema needs differs if I want to, but this should be enough.

3. Project management

a. Milestones

Each bug report should be able to be assigned a target milestone and a blocker status (only by the project managers of course). You should be able to get a list of what bugs are blocking, and who the guy is that is supposed to fix it.

Mozilla: Yes!

Mantis: No.

Trac: Yes!

b. XP support

Each bugreport should also be an XP card, and it should be possible to do all the things you do with XP cards, i e set a projected time for how long it probably will take to fix it, as well as set in the actual number of hours it took to fix it when it was fixed. Of course, with the simple addition (whic most bugtrackers already have) to mark something as a feature request, and not a bug, this turns the bugtracker into an XP-project management software.

I know of no software that supports this.

4. Security

The ability to assign different people different security depending on what product/project you are on, and also restricting access to bug reports considering security is a requirement.

5. Good queries with simple access

If you look at one bug ticket, and you want to see the rest for one component, all you should need to do is click the component name. Going to the query page and making a query each time is way to many steps.

(Post originally written by Lennart Regebro on the old Nuxeo blogs.)

ZAsync for asynchronous Zope actions part #2

 I have successfully set up and used ZAsync for webmail folder synchronisations, here's a small summary on what I have done.

Setting up ZAsync with Zeo

The first step was to set up a zeo installation with a few zope nodes.This is quite easy and usual, thanks to Zope smart architecture.

Then I have installed zasync product and created a singleton object called "Asynchronous Call Manager" in the root of the Zope.

This object provides a few methods that can be called to delegate some work to zasync.

The next step was to launch the ZAsync client that will actually run asynchronous calls on the ZODB.

ZAsync comes with a nice conf file, similar to the zope one, where you can provide informations like the adress and port of the ZEO server and set up things like logging or path.

path configuration is very important for instance, when you run a Zope2/Five/Zope3 configuration.

When everything is  set up, the client is launched and listen to incoming calls, sent by the Asynchronous Call Manager

How does it work ?

Zasync maintains a custom BTree called BForest, that holds commands to be run. Commands to be run are sent to an object called Asyncrhonous Call Manager. A client is launched besides Zope. The client looks upon the tree to see if there's some tasks to do and run them.

Why webmail needs more features

Now using one asynchronous call manager with one client can't be enough for a webmail that's supposed to handle something like 3000 people on a intranet.
I need to run several zasync clients on several servers to speed up background processes. I was starting to think about it when Gary Poster told me how to do it :
Setting up several asynchronous call manager, each one with a dedicated client. and make some kind of top function to send calls to call managers, with load balancing.

This is easy to set up since we can specify  in the client conf file the target call manager.

ZAsyncLoadBalancer Product

Since this mechanism will propably be needed in other projects, I have decided to create a small Zope product in charge of all zasync clients, that will keep a collection of asynchronous call manager and distribute calls to them according to their load.

This product will probably be callable in a high level way and fully integrated to our CMS.

Stay tuned for part #3, with a first version of ZAsyncLoadBalancer hopefully :)

(Post originally written by Tarek Ziadé on the old Nuxeo blogs.)