Nuxeo Developers Blog

OOo2 standalone from rpm files

After the command line for debian archive set, here is the one for rpm to have a standalone runable OOo 2.0 version "out of the box", running with other already existing OOo2 versions

for aFile in *.rpm;do rpm2cpio $aFile|cpio -ivdum ;done

This will create a directory (/opt) in the current directory containing the runable version
Remember to handle userEnv correctly to avoid installation mixing

(Post originally written by Laurent Godard on the old Nuxeo blogs.)

Indexing events

I have lately been looking into increasing the performance of CPSSharedCalendar when you have many events in the database (some tens of thousands or more). There are a couple of things that quickly gets tricky when doing this:

1. No matter what you index on, the result set from each index will be very big.
2. Indexing recurring events is slightly tricky.
   

Big result sets

The main thing to filter on for events is obviously start and end time. You create one index for start time and one for end time. When searching you then have one result set with anything that falls before one of these dates and another result set with everything that falls after one of the other dates. And then you need to make a union of these result sets.

Sound simple enough? Well, it is simple, but it is also very slow, because the result sets will be huge, and making a union will take a long time. Creating more indexes does not help, because the only thing you typically search on is the dates and who goes to the meeting, which of course just creates a third huge result set.

The way around this is to only use one index, for the data that is likely to create the smallest result set. This is the index where you compare with start time, as most searches will be on "today" or "this week" and there are typically more events in the past than in the future.

The other data you want to compare with are kept in a sort of "meta data" index, that is, instead of mapping from value to event, you map from event to value. Then you loop through the result set, and fetch the end date value from this "meta data index" and compare with that.

This is faster than having two index result sets and doing a union, because both result sets will be so large that it is faster to get the data event by event, as the amount of comparisons that is needed gets much smaller.

More hints on how to do efficient searching with large results sets are welcome.

Indexing recurring events

The biggest problem of indexing recurring events is that if you set something up to happen every week, forever, you have no end date to index.

I solved this by indexing events with no end date with the value "None" and handling them separately. This causes some extra work. In principle, you need to check every single event that recurs forver in every request. You also need to check every occurence of every recurring event where the time searched falls on the time span of the occurrences.

This can be handled in several ways.

• You could index a recurring event with the start date of the first occurrence, and the end date of the last. This is the simplest and slowest.
  
• Or, you could simply index every recurrence separately (except for never ending recurrences, of course).
  
• Or you can mix it. You can index every recurrence for the next year separately, and then index the test as one very long occurrence. You could even index the occurences during the search, using the indexing as a sort of "occurence caching".

But I don't expect recurring events to be that common, so after briefly looking into the faster and more complex ways of recurring this, I went with the simplest and slowest, according to the "You Ain't Gonna Need It" principle. If your calendar becomes slow and this is because you have very many recurring events, then these methods might be of interest.

The results

Well, I'm happy to say that the time to search out a months worth of data amongst a 100.000 events has dropped from 3.5 second to 0.17 seconds thanks to the performance improvements I made last week. I think that should be enough for quite a while.

(Post originally written by Lennart Regebro on the old Nuxeo blogs.)

Perspectives for web applications and Z3ECM

In my last post, I was writing that Perspectives are, for me, maybe the most interesting feature (or, at least, one of them) of Eclipse RCP when building applications for users. Jean-Marc Orliaguet read it and got a new idea : might this concept work for web applications ?

Since it's Jean-Marc and that he likes to have ideas but prefer put them at work, he started to work on this concept and see how it could fit in the Z3ECM project (next-generation ECM platform, buit on top of Zope 3).

What for ?

Imagine that when writing a new component, you also can easily define perspective. Let's take an example : a blog application. Well you can define a "Blog Perspective" that would be activated when accessing to a blog and that would arrange the portal to offer a "blog view" putting portlets in right places. WIth this and the whole CPSSkins machinery, it would be very easy to define interfaces that can adapt to user's activity.
The same approach would also work for webmail, calendar, collaborative work, personal portal dashboard, etc. The application would then only define portlets and perspectives (no more pages, view, whatever :-).
From the user point of view, it would really improve the usability and how it the portal can adapt itself to his need. The user would also be albe to define its own perspectives (like it's dashboard) and switch between them.

It would be new approach in the design of web applications, that would allow to think them as user-oriented applications and not as a chains of html pages.

Can I see it ?

Of course you can ! :-) Jean-Marc already implemented a prototype then made a post on his blog and a flash demo.

"Jean-Marc rocks !"(tm) :-)

Technorati Tags: cps, eclipse, ecm, nuxeo, rcp, rich client, zope, zope3

(Post originally written by Eric Barroca on the old Nuxeo blogs.)

Coming soon : CPS Rich Client based on Eclipse RCP

Introduction

Here, at Nuxeo, we are thinking for a long time at desktop applications for CPS. Why in this new full web, ajaxified, world ? :-) Well, I think that for many applications and features, a desktop application is still needed.

Here is some examples of features / requirements :

• Disconnected client to work in the train, in the plane, on the beach :-)
  • read, annotate, search in a document repository
  • use electronics forms when working outside the office
• Deep integration with legacy desktop applications (what about having OpenOffice running inside a dedicated client to improve cooperative work)
• Strong security : document signing, workflow action signing, etc.
• Advanced text edition : interactive diff / merge
• Advanced editors (photos, rich media, rich documents, etc.)
• Reduce server loads doing many computation on the client
• many, many more ideas :-)

We started to work, as some customers asked, on a desktop application. One point was clear: limit the burden of deployment!

Rich Client ?

We studied deeply Flash, Mozilla and Eclipse Rich Client Platform (RCP).

Flash was not standalone enough and too tied to graphics.

Mozilla seemed fine, but the current framework is too young (we were told by a proeminent Mozilla developer to wait for Mozilla 2.0 before investing on the technology) and not enough adapted to generic application, plus it needed us to write in C++ and had some fear about cross-platform compilation.

Eclipse RCP came then and we quickly discovered that it proposes an amazingly "sexy" framework, extensible, component-oriented and very well designed. But, besides those technical considerations, we mainly found that Eclipse RCP proposes a new approach and a new paradigm for desktop applications... very user-centric!

Eclipse RCP offers new concepts (at least, for me) for desktop applications, maybe inspired by webapp : actions, views, portlets, workspaces, perspectives, etc. Eclipse RCP comes, of course, also with a lot of interesting points for deployment and development: automatic on-line software update, lot of plug-ins, real cross-platform compatibility, SWT (a very fast graphic toolkit for Java), etc.

Perspectives ?

I think the most noticeable concept of Eclipse RCP to create applications for user is the concept of Perspective. A perspective is a view of your work space: portlet arrangement, toolbar, menu. It allows to offer several different ways to present the application, depending on the usage context. For example, you can have a perspective to edit a document, where the work space is document-centric, and a perspective to browse the document repository, where the work space is centered on navigation and searching. Moreover, users can define very easily new perspectives, dragging and dropping or adding portlets and then ask the application to save this perspective.

In short, it allows to adapt the application to user activity and working habits. It's the first time I am aware of a desktop framework offering this feature. Perspectives are designed to build user-centric / activity-centric applications and that is what users like, when you place them in the heart of the application usage paradigm. ;-)

Real-life projects

We now are working two projects based on CPS and Eclipse RCP. CPS is used as the document / content manager and Eclipse RCP as the framework to build a dedicated desktop application (let's call it Rich Client to CPS). The Rich Client connect offers UI and local features. CPS offers document management core. Best of both worlds ! ;-)

The first project is for AFP (French Press Agency) and aims at building the new news items management back-office. In this project, the Rich Client is the only CPS UI the user sees. The Rich Client is used by journalists to read internal news feeds, then write new documents, add / edit photos, manage the workflow and the output feed (rubrics, etc.). CPS, stores documents, manage permissions, serve the search, etc.

The second project is for the French Institute of Statutory Auditors (Compagnie Nationale des Commissaires aux Comptes, CNCC) and aims at building a system to manage documents and books authored and published by CNCC. CPS, here, is used to manage documents (authored with OpenOffice.org) and build books (collection of documents organized in a hierarchy / table of content). The Rich Client is the consultation interface of books : it synchronizes its local repository of books with the server, indexes it (in an RDF Database - Sesame  - for metadata and in Lucene for full-text) and offers to the user the reading/browsing/searching interface.

In both projects, the Rich Client proposes several perspectives, depending on user activity (users can also create their own perspectives), is auto-upgraded and communicates with CPS using HTTP and XML-RPC, RSS or ATOM.

First feedbacks from customers are very interesting and encouraging (users seem to really appreciate the interface). Feedback from our side is that the framework is amazing and allows very fast development thanks to the library of plugins and the quality of the software infrastructure offered. More details about those projects will be available soon (as soon as I find time to write on those :-)

The big picture for CPS Rich Client : the Personal Portal

Those projects, CNCC then AFP, are allowing us to create Eclipse plugins to communicate with CPS. We took then some time to plus those plugins together and offer a basic CPS client for offline work. It will be the base for next projects.

Our goal is to create a "pluggable" Rich Client that can be the Unified Rich Client for CPS so that users can work on CPS using this client or using a browser depending on their activity. We would like to be able to share as many features as we can between both client type: forms, portlets sources, actions, etc. That's where XForms, ATOM, XML Schema and stuff are taking place...

Imagine that, with this Rich Client framework for CPS, you would be able to quickly add a new feature and deploy it to all clients (for example a new application for users). Maybe even you can put this plugin to CPS and when clients connects to CPS plugins are installed, configured and available for the users. Users install the Rich Client (that can't do anything), then connect to the CPS instance and then get all components needed (an electronic form application, a collaborative work application, etc.).

We think this Rich Client as the personal portal of users, where they can do emails, calendaring, document sharing, specific corporate applications, all backed and stored into the corporate Intranet and also available with their browser. So fun ! :-)

Of course, all this code will be available as soon as we have some generic and useful codebase as Open Source software under the ZPL or EPL.

I will write more soon and often about those topics that can open a new world to document management, desktop applications and corporate Intranets.

To finish, I would like to thank IBM, the Eclipse Foundation and all Eclipse developers / contributors for creating and developing this amazing framework... Thanks to all ! :-)

Technorati Tags: zope, rich client, cps, eclipse, nuxeo, rcp

(Post originally written by Eric Barroca on the old Nuxeo blogs.)

AtomAPI fully implemented into CPSBlog

I am going further with AtomAPI support in CPSBlog. I have just added :

• DELETE support
• categories management as described in AtomSyndication

The AtomAPI support has been refactored to be more generic and split into 4 classes : AtomMixin, AtomAware, AtomAwareEntry and AtomAwareCollection. I still don't know if it's the best architecture for Atom support, and I know there is a lot of improvements to do, but it's better than yesterday ;-) This is the first step to a full Atom support in CPS. I think it should be possible to apply this Atom support to any document and folder so. And that would be the first step to a fully RESTful implementation of content syndication, cross-site publication and remote content management (let's talk now about native XMPP support, Atom extension, JEI-60, etc.). We will have to go further in this way very soon... The code is in CPSBlog trunk and should work with any AtomAPI client. I would be very delighted to get some reports with other clients.

Technorati Tags: atom, atomapi, cps, nuxeo, rest

(Post originally written by Eric Barroca on the old Nuxeo blogs.)

CPSBlog now supports AtomAPI

I have implemented a partial AtomAPI support into CPSBlog, using specification and methods described by Blogger and Typepad AtomAPI.

This mainly allow blogging from desktop and mobile application.

Right now, CPSBlog AtomAPI implementation supports:

• adding entries
• editing entries
• retrieving and using categories (though some bugs seems to exists depending on the client)
• reading feed / entries

From a technical point of view, I've used the excellent lxml python component to parse incoming request and manage to also deal with SOAP enveloppe request (sent by some client, even if the server never says it accept it ! :-).

This blog will be posted using ecto on MacOS X (at least I hope it will work since it's the first post using AtomAPI on this site ;-). BTW, I was really disappointed to find out that there is very few AtomAPI clients (and almost none open source, but PyQLogger which seems to not work at the moment :-/).

How to get it working (taking this site as example):

• Protocol : Atom / AtomAPI
• Access / Entry Point : your blog URL/atomBlogServicesDiscovery (ex: http://blogs.nuxeo.com/sections/blogs/eric_barroca/atomBlogServicesDiscovery)
• Login / Pass : your usual login / password on the CPS site !

Then your client should get the required information and download the feed to allow you to post ! :-)

I would be pleased to get some feedback of using the API from other clients (Windows / Linux).

TODO:

• HTTP DELETE support to delete an entry
• pictures / files upload support (if I can find a client that supports it)
• test, test, test, test and more test !

Hope this help !

Technorati Tags: atom, atomapi, cps, nuxeo, rest

(Post originally written by Eric Barroca on the old Nuxeo blogs.)

XML Schema support on Zope3 (Concret example)

I added a demo package to illustrate the zope3 / xml schema integration.
You can grab de code over there :
http://svn.nuxeo.org/trac/pub/browser/z3lab/zope/xmlschema/trunk/demo/

The goal of the demo is to get a new content object registred within Zope3 with an "add "and "edit" form driven by an XML Schema definition.

This example is pretty simple but illustrate perfectly the goal of the XML integration and the power of Zope3 ;)

Let's take a look at the xsd we will use in this demo  :

<?xml version="1.0" encoding="UTF-8"?>

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">

  <xs:attribute name="title" use="required" type="xs:normalizedString" />

  <xs:attribute name="description" use="required" type="xs:string"/>

</xs:schema>

(demo.foo.xs)

It defines a schema having 2 attributs : title and description.
title is required and is a normalizedString data type
description is required as well and is a string data type.

Let's define an interface IFoo and set the "foo" schema above. Note I'm using directly an id in here for the schema (i.e : foo). This is because at this stage the schema will be registred within Zope3 under this name. We will see how just after.

import zope.interface

import zope.xmlschema



class IFoo(zope.interface.Interface):

    zope.xmlschema.set('foo')

(demo.interfaces.py)

Then define the Foo content object implementing IFoo. It inherits from presistent so that the content object will be persistent but it's not compulsory.

import persistent

import zope.interface

from zope.xmlschema.demo.interfaces import IFoo

class Foo(persistent.Persistent):

    zope.interface.implements(IFoo)

    def __init__(self, title='', description=''):

        self.title = title

        self.description = description

(demo.foo.py)

Now the zcml configuration (the glue) (see demo.configure.zcml) to tell Zope what to do whith the definitions above.

Let's take a look at the configure.zcml file for the package as explained below :

<xmlschema

      id = "foo"

      document ="foo.xs"

      >

  </xmlschema>

This directive registers the xsd foo.xs under the name of 'foo'

Then the one below defines the new content object (notice the set_schema directive.) You can forget about the Zope permission stuffs if you're not familiar with Zope3

 <content class=".foo.Foo">

    <implements

  interface="zope.app.annotation.IAttributeAnnotatable" />

    <require permission="zope.View"

            interface=".interfaces.IFoo" />

    <require permission="zope.ManageContent"

       set_schema=".interfaces.IFoo" />

  </content>

Then, we specify the add and edit form for the Foo content object.

<browser:addform

      schema=".interfaces.IFoo"

      label="Add a Foo content object"

      content_factory=".foo.Foo"   

      arguments="title description"

      name="addFoo.html"

      permission="zope.ManageContent"

      />



  <browser:editform

      schema=".interfaces.IFoo"

      label="Edit Foo"

      name="edit.html"

      menu="zmi_views" title="Edit"

      permission="zope.ManageContent"

      />

Then we need to tell Zope to add an entry in ZMI (Zope Management Interface) menu for us to be able to create a Foo content object through the ZMI.

<browser:addMenuItem

      class=".foo.Foo"

      title="Foo"

      permission="zope.ManageContent"

      view="addFoo.html"

      />

Now start Zope3 go in the ZMI and click the Add menu entry and ask for the creation of a Foo content object.

You will get the form below :



Then you can try to submit with empty values and since we specified required values on the xml schema the submission is not possible :




Here it is for today.

The 'magic' for the rendering against the schema is done by Zope3 itself. It's not the job of the zope.xmlschema component.

Again, the next step will be to have the rendering done with XForms :) Currently, looking at it. I played today with the mozilla xforms plugin. I'll dig into this whike coming back from holidays at the beginning of September.

Time to sleep...

(Post originally written by Julien Anguenot on the old Nuxeo blogs.)

Extending products Zope3-style.

Today we had a need to extend the CPSSharedCalendar a little bit in a separate project. The functionality needed was not seen as generic enough to go into the CPSSharedCalendar itself. Luckily, since it's written with Five, this can be done easily, and without touching the original code, or monkey patching, and with none of the standard problems of using portal_skins. So I thought it might be interesting to show how this is done, as a way to convert more people to Zope3 and Five.

Basically, the project needed to display a list of events from the users calendar, but only the events from today, and with a custom formatting. Here is the rough steps used, as an example to show to others how to do it:

• Create a new product. A folder under Products and an empty __init__.py is all that is needed.
  
• Create a view class that has a method that performs the actions you need. Lets call the module customeventview.py, and the class CustomEventsView.

class CustomEventsView:
    "View for returning a list of all events of today"

    def getEvents(self):
        events = self.context.getTheEventsWhichIwant()
        return events

• Create a template that calls the view class, and return the list of events as you want it, lets call it customevents.pt.

<html metal:use-macro="context/@@standard_macros/page">
  <metal:block fill-slot="body">
    <ul tal:define="events view/getEvents">
      <li tal:contents="event/title" />
    </ul>
  </metal:block>
</html>

• Create a configure.zcml that connects the template to the view class.

<configure xmlns="http://namespaces.zope.org/zope"
           xmlns:browser="http://namespaces.zope.org/browser">

  <browser:page
    for="calcore.interfaces.ICalendar"
    name="customevents.html"
    template="customevents.pt"
    class=".customeventsview.CustomEventsView"
    permission="calendar.ViewCalendar"
    />

</configure>

This example, as you see, provides no great functionality. Only another, custom view. But the point here is that this customization is completely local, it does not require any modification of the original software, nor does it monkeypatch it. At the same time, should you want to, the customization is easy to move into the original product.

But, you say, this can be done by making a new skin in portal_skins! Sure, but then you run into problems of the restricted code of python scripts. This is all disk based, and has no such problems. Also, normally you would have to set up a custom skin folder for these overrides, and they have to be set up in the Zope site, while this type of Zope3/Five extension only needs you to copy the product into the Products folder and restart Zope. You don't need to run any installation scripts, or add any new skin folder to the list of folders in all the skins.

That's the power of Zope3 and Five.

(Post originally written by Lennart Regebro on the old Nuxeo blogs.)

XML Schema support on Zope 3

I started yesterday an XML Schema support for the ZopeInterface.  I would like to be able to define my schema with XML Schemas definitions instead of having to define them in Python (which is boring and Python specific...).

The code is within the z3ecm repository: http://svn.nuxeo.org/trac/pub/browser/z3lab/zope/xmlschema/trunk/

See an example :

filepath is a path to an XML Schema definition as defined below :

<?xml version="1.0" encoding="UTF-8"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<xsd:element name="title" type="xsd:string"/>
<xsd:element name="description" type="xsd:string"/>
</xsd:schema>

 >>> import zope.interface
 >>> import zope.xmlschema
 >>> class ITest(zope.interface.Interface):
 ...  zope.xmlschema.set(filepath)


Now introspect the fields defined within the xsd. We will see two new
fields : title and description.

 >>> names = list(ITest)
 >>> names.sort()
 >>> names
 ['description', 'title']

Now define a class implementing this interface.

 >>> class Test(object):
 ...  zope.interface.implements(ITest)
 ...  title = ''
 ...  description = ''
 >>>

ITest is implemented by Test

 >>> ITest.implementedBy(Test)
 True

Let's create an instance of Test

 >>> test = Test()

Defines potential values for the two fields

 >>> title = u"Title field"
 >>> description = u"Description field"

Now we get the fields from the interface:

 >>> title_field = ITest.get('title')
 >>> description_field = ITest.get('description')

Next we have to bind these fields to the context, so that instance-specific
information can be used for validation:

 >>> title_bound = title_field.bind(test)
 >>> description_bound = description_field.bind(test)

Now try to validate them :

 >>> title_bound.validate(title)
 >>> description_bound.validate(description)

You can see that once the schema is set on the interface the interface behaves like if it had been defined in Python. Now let's dig into the XML Schema specification and implements at least all the zope.schema features so that zope.schema and zope.xmlschema will provide the same declarative possibility.

Notice, this component uses lxml as core XML parser.

It's the first step toward XForms support on z3ecm :)

(Post originally written by Julien Anguenot on the old Nuxeo blogs.)

Interfaces versus ABC within Python

Guido wrote a blog entry about interfaces versus ABC in Python.
http://www.artima.com/weblogs/viewpost.jsp?thread=92662

Nice to see the discussion about this point occurring. Note he's taking into account the fact that Zope3 and Twisted do have an actual interface implementation (which is the ZopeInterface BTW)

As a Zope3 developer I just hope he will go for core Python interfaces ;)

(Post originally written by Julien Anguenot on the old Nuxeo blogs.)