Florent Guillaume

JavaOne 2007 Slides

fguillaume — 2007-05-12

JavaOne is now finished. We had a very interesting few days there, meeting lots of people interested in Nuxeo and wanting to know more about the technology and the business around it. My talk itself was well received, and I hope it gave people a good sense of what is possible and desirable in the Open Source ECM problem space.

You can view the slides of the talk here.

I also attended some very interesting sessions regarding subjects that are important when writing an ECM framework: persistence, distribution, scalability, validation, object-relational modeling, etc. A number of efforts are ongoing to standardize as JSRs some important aspects related to this. In the coming days I'll be blogging in more detail about these JSRs that we'll be tracking closely.

For our first participation, this was a very fruitful experience, and we'll certainly be going again next year!

Discover Nuxeo 5 at JavaOne 2007

fguillaume — 2007-05-03

JavaOne 2007 is in a few days now, and we'll be giving a talk about Nuxeo 5.

The talk has ID TS-4532 (in the Java EE track), and takes place on Thursday the 10th at 5:30 pm in Hall E - 134. The title of the session is Building an Embeddable Enterprise Content Management Core with the Latest Java Technologies.

This is a great opportunity if you want to discover more about how Nuxeo 5 is architected, see demos of the product, or talk to us about technical or business-related questions.

As a teaser, here's the overview of the talk:

Learn about the design and use of Nuxeo 5, an embeddable, extensible Enterprise Content Management framework for Java EE and other platforms

Agenda:

What is Enterprise Content Management
What do we want to achieve
Core Framework
Core and High-Level Services
UI Layers
Extensible and Pluggable
Using the Framework

See you in San Francisco!

What's the point of JCR?

fguillaume — 2006-10-18

Nuxeo is switching its ECM to Java, and we're using JCR for our document storage. JCR (Java Content Repository, standardized by JSR-170 and the upcoming JSR-283) is a young specification with a promising future — but what's its point, you may ask, as all existing content management systems are already storing content very well without it? Its goal is interoperability between vendors, which will make it possible for people who write applications needing to store content to have a unified API for such manipulations. All major content repository vendors are active in the JSR-283 expert group, and all are working on JCR bindings for their various proprietary repositories.

Of course a standardized and wildly successful way of manipulating content already existed before JCR: SQL. But SQL and JCR have a different focus:

SQL is a language; it manipulates rows and is geared toward generic relation manipulation,
JCR is a Java API; it manipulates nodes and is geared toward hierarchical manipulation (parent-children).

SQL and JCR have quite different underlying data models:

SQL's model is that of tables with fixed schemas, and relations between tables,
JCR's model is that of a tree of nodes with flexible schemas and with parent-children relations as the main focus — although other types of relations exist.

JCR also offers higher level features than SQL, notably workspace and version management.

For many kinds of applications, there is a focus on being able to arrange documents in folder hierarchies, and to have a wild variety of structure for these documents. In this case, a storage model based on JCR is much more suited than something based on SQL.

It should be noted that many things in the computing world already are based on the notion of folder hierarchies storing arbitrary documents:

filesystems are a tree of (unstructured) documents,
Revision control systems are based on filesystem concepts but add a lot of structure on top of them,
WebDAV (along with DAV and DELTAV) is a protocol that addresses documents using a path, and where documents have a flexible set of properties,
most proprietary content management systems are based on (or offer the notion of) organizing the content in a hierarchy,
Nuxeo's CPS is itself based on this classic model.

This is why JCR has emerged as a common and useful storage API for all these use cases.

For an ECM framework like Nuxeo 5, JCR interoperability can be seen from two different directions:

JCR provides an API that we can use to store our content, to make us vendor-independent and flexible regarding storage,
JCR provides an API through which we can expose our content, which makes our platform usable by any external system that understands it — we're ourselves a vendor providing JCR bindings.

For its initial release, Nuxeo 5 is focusing on the use of JCR as its main storage implementation and uses Jackrabbit to store most of our content. In the future, we'll also provide JCR bindings so that the high-level content we provide can be directly accessed from external applications using JCR too.

Nuxeo 5: Unifying the content APIs

fguillaume — 2006-10-18

In a content management system, the actual data that the system or the users manipulate comes from many kinds of sources. Content can come from a JCR repository, or from a relational database, or from an LDAP directory, or from a semantic storage engine like Jena, or from any other kind of open or proprietary storage engine.

But fundamentally all these kinds of content, which I'll call "records", aren't very different:

a record can be created, viewed, modified, deleted,
a record can often be copied or moved,
a record obeys a schema that can be known to the system, this means that its individual fields are strictly typed,
when being viewed or modified, a record has a user interface that is based on forms, labels, widgets, depending on the schema,
records can be searched and a result set returned,
records can be listed in a compact form (search results, folder contents, user dashboard, workflow workitems, RDB table listing, user information browsing, etc.),
records have an identity (like a unique id) or a location (like a path), sometimes both.

One of the strengths of CPS is to use a common abstraction for many of these concepts, embodied in the CPSSchemas component. In Nuxeo 5 we want to go further and provide even more integration for all these, the base components for these abstractions are NXCore and NXTypeManager.

The reasons to strive for convergence are numerous:

this merges into unique concepts things that had been previously separated because of different implementation choices. For instance, an LDAP schema is not fundamentally different from an SQL schema (or from an XML Schema if one is interested in the relevant subset).
this gives the programmers a common API for all data-related operations, which means more reusability. For instance changing an attribute in an LDAP entry doesn't have to be different from changing the title of a document or changing a value in an RDB row; processing a list of search results to display them in a table doesn't have to be different from processing the children of a folder to display the folder's contents.
this gives the framework developers a way to optimize some operations because of commonalities in the underlying implementations. For instance you don't need three kinds of events dispatching for "LDAP entry modified", "RDB row modified" or "document modified".
this gives the users a unified way of manipulating different kinds of data when there's really no need to have a different UI for them. When a user fills a form it's really the same process whether he's modifying his personal preferences, adding a keyword to a document, or changing a quantity in an RDB row.
this allows very simple migrations between storage technologies, when these are felt necessary. A customer could start with an LDAP database for its user base and later have the need to move them to an RDB table. User entries in an RDB table may need to be versioned and moved to a JCR storage. An application should survive all these with only configuration changes and no code to rewrite.

It should be noted that this means that JCR is in no way the primary storage model for Nuxeo 5, it's only the first one to be implemented. In the future, it will be possible to store documents in LDAP or an RDB. When a suitable storage model is devised and implemented, you'll be able to apply workflow or versioning to RDB-based documents for instance.

This convergence is quite exciting to us, and our goal is to allow people to build complex applications with Nuxeo 5 in a more straightforward manner.

GenericSetup for CPS, CMF and Zope

fguillaume — 2006-01-24

GenericSetup is a framework to describe the configuration of a Zope site as a set of XML files (and sometimes other associated files). It can import profiles, which may create objects or change their configuration, and export profiles, which makes a snapshot of the configuration and writes it to a set of XML files.

GenericSetup provides a tool that can store snapshots of a configuration in the ZODB itself, where it can be examined and even modified. It can also do diffs between two snapshots, which is very useful to find out what changed in a configuration (it's a good idea to take a full snapshot anytime some significant changes are made to the configuration).

GenericSetup differentiates between Base and Extension profiles. A Base profile is a profile that describes "everything". When it is imported, it removes and overwrites any previous configuration. An Extension profile is a profile designed to be incrementally added on top of a previously existing configuration. It may of course overwrite some settings, or even in some case remove objects, but its goal is generally to add optional configuration on top of a main one

GenericSetup is based on a small number of concepts: the toolset, and some import and export steps. GenericSetup provides a framework where import and export steps can be written simply using Zope 3 adapters.

GenericSetup was born as "CMFSetup" but was later made generic and can be used by any Zope 2 application. It is now available at http://svn.zope.org/GenericSetup/trunk/. A subclass with additional features is used in CPS.

Setup tool

The setup tool is called setup_tool by default, and portal_setup in CPS (if missing, it can be instantiated by selecting "CPS Tools" from the ZMI Add menu).

At a given time, the setup tool knows about a few things:

the currently selected profile, which may point to an area in the filesystem where a profile has been registered, or to a path in the ZODB where a snapshot was taken,
the current toolset,
the current import steps,
the current export steps.

When a new profile it selected, its toolset, import steps and export steps XML files are loaded and merged with the tool's current ones.

Any import or export is based on the full toolset or import/export steps (even if the currently selected profile is an extension profile), but the source of XML configuration files depends solely on the currently selected profile.

Toolset

The file toolset.xml describes the Toolset, which is the set of tools needed in a given configuration. (While the name "tool" would suggest CMF, it's just a set of objects that can be instantiated at the root of the configured site.)

Beyond being based on data that is updated when an extension profile is selected, the toolset is a normal import/export step.

When the toolset is imported, all the objects in the toolset are instantiated if they're missing or if their class doesn't match with what it's supposed to be.

An excerpt of toolset.xml for the CPS Tree Tool would be:

<?xml version="1.0"?>
<tool-setup>
  ...
  <required tool_id="portal_trees"
            class="Products.CPSCore.TreeTool.TreeTool"/>
  ...
</tool-setup>

Import steps

Import steps (import_steps.xml) describe a set of configuration steps available for import, and their dependencies. A step is just the dotted name of a function, and the dependencies is simply the steps that have to be run before this one can be done (most steps depend on the toolset, because they need a base tool to be instantiated before it can be configured).

While an import step can do anything it likes, GenericSetup provides a framework based on Zope 3 adapters to simply describe how a single object is imported from XML, and to recurse among objects to create or configure them one by one.

Purge

During import, there are two possible behaviors, corresponding to the two kinds of profiles. For a Base profile, the import happens in "purge" mode, while for an extension profile the import doesn't purge. The functions and adapters doing the import have to take that into account when they read a profile.

In purge mode, every previous configuration has to be removed, so that the result is indistinguishable from an install from scratch. In non-purge mode, care must be taken to not overwrite or remove settings or objects which are not explicitely specified in the imported XML file.

An excerpt of import_steps.xml for the CPS Tree Tool would be:

<?xml version="1.0"?>
<import-steps>
  ...
  <import-step id="trees" version="20051230-01"
               handler="Products.CPSCore.exportimport.importTreeTool"
               title="Tree Tool">
    <dependency step="toolset"/>
    Import tree tool and tree caches.
  </import-step>
  ...
</import-steps>

Export steps

Export steps (export_steps.xml) describe the list of steps available for export. An export step is used in exactly the same way than an import step, except for the fact that there are no dependencies between export steps, and that instead of being read, XML files are written.

One important thing to note is that export steps cannot do the "incremental exports" that many people expect. When an extension profile is read by the import steps, only the available XML files for that profile are read. However when writing, there's no way to choose which XML files are relevant, so the whole profile for that step is written (and recursion is done in all subobjects).

It's possible that future versions of GenericSetup will have some capabilities to do incremental exports, but this is not possible for now.

An excerpt of export_steps.xml for the CPS Tree Tool would be:

<?xml version="1.0"?>
<export-steps>
  ...
  <export-step id="trees"
               handler="Products.CPSCore.exportimport.exportTreeTool"
               title="Tree Tool">
    Export tree tool and tree caches.
  </export-step>
  ...
</export-steps>

Adapters

The standard work done in an import or export step is to find the base tool, and call importObjects or exportObjects on it; these are recursive functions that take each object, find an adapter for them describing how the XML import or export is done, and call it.

For the CPS Tree Tool, the import steps and export steps call the following functions:

def exportTreeTool(context):
    """Export Tree tool and tree caches as a set of XML files.
    """
    site = context.getSite()
    tool = getToolByName(site, 'portal_trees', None)
    if tool is None:
        logger = context.getLogger('trees')
        logger.info("Nothing to export.")
        return
    exportObjects(tool, '', context)

def importTreeTool(context):
    """Import Tree tool and tree caches as a set of XML files.
    """
    site = context.getSite()
    tool = getToolByName(site, 'portal_trees')
    importObjects(tool, '', context)

This is pretty boileplate and could even be simplified in the future through ZCML declarations. Above, '' simply refers to the root of the profile directory.

The adapters are multi-adapters, adapting both an object and an import context (called "environ" in GenericSetup), to the IBody interface that basically describes a file body. They can be registered through ZCML using the standard statement:

<adapter
    factory=".exportimport.TreeToolXMLAdapter"
    provides="Products.GenericSetup.interfaces.IBody"
    for=".interfaces.ITreeTool
         Products.GenericSetup.interfaces.ISetupEnviron"
    />

This assumes of course that the exported object is described through an interface, for instance here declared as:

from zope.interface import Interface

class ITreeTool(Interface):
    """Tree Tool.
    """

The interface is implemented by the object's class with:

from zope.interface import implements

class TreeTool(UniqueObject, Folder):
    """Tree Tool that caches information about the site's hierarchies.
    """
    implements(ITreeTool)

    id = 'portal_trees'
    meta_type = 'CPS Tree Tool'
    ...

When doing an export, the adapters build a DOM tree for the configuration. When doing an import, they read the DOM tree and create or modify properties as needed. The standard adapters don't create subobjects or recurse in them, this is left to the importObjects function.

These adapters can be written easily because GenericSetup provides helpers for the common cases of objects configured only through standard Zope 2 properties (PropertyManager), or having subobjects (ObjectManager).

Of course all this can be changed for specific cases. Many older CMF tools are configured through things that are not standard properties, and for instance CPS needs to do recursion into more than simple ObjectManager subobjects. It is also possible to read and write files that are not XML files, CPS does this for the images included in portlet objects, where it writes a real image file.

Additional CPS feature: upgrades

CPS has extended the setup tool to provide a basic Upgrade feature, that is related to the configuration of the site but cannot be expressed by the standard GenericSetup profiles.

An upgrade step is registered through ZCML with something like:

<cps:upgradeStep
    title="Replace CMF URL tool with a CPS-specific one"
    source="3.3.4" destination="3.3.5"
    handler=".upgrade.upgradeURLTool"
    checker=".upgrade.check_upgradeURLTool"
    />

This describes between what CPS versions this upgrade is needed, how to do it, and how to check if it's already been done.

The setup tool lists which steps have not yet been done, and provides a way to run them one by one or all at once. At a given time, a CPS site "knows" (through a site-global property last_upgraded_version) up to which version it's been upgraded.

The checker is optional; if it would be too costly to check for the conversion, it can be omitted. This is the case typically for an upgrade step that would recurse in the content object to do some fixups; to check if they've already been done it would have to recurse in the content objects too, and that's too much work for a simple check. The setup tool won't list a step without a checker if the portal has already been upgraded to a later version than the step's destination version.

Object event dispatching in Zope

fguillaume — 2005-11-10

Here are some explanations about what happens in Zope 3.2 (and Zope 2.9 when using Five) when an event notification is sent by some code, up to a specific subscriber. It focuses more specifically on object events, which go through some additional hoops. All this is complex because there are many simple components that are linked together.

Let's start with some framework code that sends an event after an object has been added (similar to what zope.app.container.contained actually does):

event = ObjectAddedEvent(ob, container, name)
zope.event.notify(event)

In zope.event we have the definition for this function:

subscribers = [] # registered subscribers
def notify(event):
    for subscriber in subscribers:
        subscriber(event)

During initialization, zope.app.event.dispatching has registered a subscriber:

def dispatch(*event):
    # Iterating over subscribers assures they get executed.
    for ignored in zope.component.subscribers(event, None):
        pass
zope.event.subscribers.append(dispatch)

The function zope.component.subscribers will then call all matching subscribers.

During initialization, zope/app/event/configure.zcml has registered a subscriber for zope.app.event.interfaces.IObjectEvent with the handler zope.app.event.objectevent.objectEventNotify:

<subscriber
    for="zope.app.event.interfaces.IObjectEvent"
    handler="zope.app.event.objectevent.objectEventNotify"
    />

This handler does:

def objectEventNotify(event):
    adapters = zope.component.subscribers((event.object, event), None)
    for adapter in adapters:
        pass # Getting them does the work.

This means that the event will be redispatched, but this time a subscriber can match using multi-adaptation on both the object and the event interfaces, which gives much more flexibility and filtering possibilities.

During initialization, zope/app/container/configure.zcml has registered a multi-subscriber:

<subscriber
    for="zope.app.location.interfaces.ILocation
         zope.app.container.interfaces.IObjectMovedEvent"
    handler="zope.app.container.contained.dispatchToSublocations"
    />

Note that IContained, a base interface for most content objects, derives from ILocation, so this subscriber will match most content objects. Also, the IObjectAddedEvent sent initially derives from IObjectMovedEvent so it will be matched.

When using Five, in Five/event.zcml some similar subscribers are registered to react on IObjectManager instead of ILocation, and to dispatch using the same handler.

The dispatchToSublocations handler is:

def dispatchToSublocations(object, event):
    subs = ISublocations(object, None)
    if subs is not None:
        for sub in subs.sublocations():
            for ignored in zapi.subscribers((sub, event), None):
                pass # They do work in the adapter fetch

This redispatches the same event to all subobjects, where "subobjects" is defined using the ISublocations adapter. Now, zope/app/container/configure.zcml has an adapter:

<adapter
    for="zope.app.container.interfaces.IReadContainer"
    provides="zope.app.location.interfaces.ISublocations"
    factory="zope.app.container.contained.ContainerSublocations"
    />

Most Zope 3 container objects are also IReadContainer. The ContainerSublocations handler does simply get the sublocations using:

class ContainerSublocations(object):
    def __init__(self, container):
        self.container = container
    def sublocations(self):
        container = self.container
        for key in container:
            yield container[key]

When using Five, a similar adapter is registered:

<adapter
    for="OFS.interfaces.IObjectManager"
    provides="zope.app.location.interfaces.ISublocations"
    factory="OFS.subscribers.ObjectManagerSublocations"
    />

And the Five adapter does something comparable to the Zope 3 one:

class ObjectManagerSublocations(object):
    def __init__(self, container):
        self.container = container
    def sublocations(self):
        for ob in self.container.objectValues():
            yield ob

All in all, our initial event will be redispatched to the sublocations of the original object, and the process will be done recursively to all the sublocations.

Now some user code's configure.zcml can registered a multi-subscriber:

<subscriber
    for=".interfaces.IFoo
         zope.app.container.interfaces.IObjectAddedEvent"
    handler=".foo.reactOnAdd"
    />

Which ties to the handler:

def reactOnAdd(ob, event):
    """Does something."""

In the handler, which will be called for all sublocations, ob is any sublocation of the original object (including the object itself), and event is the original event (which means that event.object is the original object).

This concludes our dive into through Zope 3 events. You can read in http://blogs.nuxeo.com/sections/blogs/florent_guillaume/2005_11_08_events-in-zope-2-9 how object events can be used in Zope 2.9 to do what used to be done using manage_afterAdd.

Events in Zope 2.9

fguillaume — 2005-11-08

Zope 2.9 (and Zope 2.8 when using Five 1.2) introduces a big change: Zope 3 style container events.

With container events, you finally have the ability to react to things happening to objects without have to subclass manage_afterAdd, manage_beforeDelete or manage_afterClone. Instead, you just have to register a subscriber for the appropriate event, for instance IObjectAddedEvent, and make it do the work.

Indeed, the old methods like manage_afterAdd are now deprecated, you shouldn't use them anymore.

Let's see how to migrate your products.

Old product

Suppose that in an old product you have code that needs to register through a central tool whenever a document is created. Or it could be indexing itself. Or it could initialize an attribute according to its current path. Code like:

class CoolDocument(...):
    ...
    def manage_afterAdd(self, item, container):
        self.mangled_path = mangle('/'.join(self.getPhysicalPath()))
        getToolByName(self, 'portal_cool').registerCool(self)
        super(CoolDocument, self).manage_afterAdd(item, container)

    def manage_afterClone(self, item):
        self.mangled_path = mangle('/'.join(self.getPhysicalPath()))
        getToolByName(self, 'portal_cool').registerCool(self)
        super(CoolDocument, self).manage_afterClone(item)

    def manage_beforeDelete(self, item, container):
        super(CoolDocument, self).manage_beforeDelete(item, container)
        getToolByName(self, 'portal_cool').unregisterCool(self)

This would be the best practice in Zope 2.8. Note the use of super() to call the base class, which is often omitted because people "know" that SimpleItem for instance doesn't do anything in these methods.

If you run this code in Zope 2.9, you will get deprecation warnings, telling you that:

Calling Products.CoolProduct.CoolDocument.CoolDocument.manage_afterAdd
is deprecated when using Five, instead use event subscribers or mark
the class with <five:deprecatedManageAddDelete/>

Using five:deprecatedManageAddDelete

The simplest thing you can do to deal with the deprecation warnings, and have correct behavior, is to add in your products a configure.zcml file containing:

<configure
    xmlns="http://namespaces.zope.org/zope"
    xmlns:five="http://namespaces.zope.org/five">

  <five:deprecatedManageAddDelete
      class="Products.CoolProduct.CoolDocument.CoolDocument"/>

</configure>

This tells Zope that you acknowledge that your class contains deprecated methods, and ask it to still call them in the proper manner. So Zope will be sending events when an object is added, for instance, and in addition call your old manage_afterAdd method.

One subtlety here is that you may have to modify you methods to just do their work, and not call their super class. This is necessary because proper events are already dispatched to all relevant classes, and the work of the super class will be done trough events, you must not redo it by hand. If you call the super class, you will get a warning, saying for instance:

CoolDocument.manage_afterAdd is deprecated and will be removed in
Zope 2.11, you should use an IObjectAddedEvent subscriber instead.

The fact that you must "just do your work" is especially important for the rare cases where people subclass the manage_afterAdd of object managers like folders, and decided to reimplement recursion into the children themselves. If you do that, then there will be two recursions going on in parallel, the one done by events, and the one done by your code. This would be bad.

Using subscribers

In the long run, and before Zope 2.11 where manage_afterAdd and friends will be removed, you will want to use proper subscribers.

First, you'll have to write a subscriber that "does the work", for instance:

def addedCoolDocument(ob, event):
    """A Cool Document was added to a container."""
    self.mangled_path = mangle('/'.join(self.getPhysicalPath()))

Note that we're not calling the portal_cool tool anymore, because presumably this tool will also be modified to do its work through events, and will have a similar subscriber doing the necessary registerCool. Note also that here we don't care about the event, but in more complex cases we would.

Now we have to register our subscriber for our object. To do that, we need to "mark" our object through an interface. We can define in our product's interfaces.py:

from zope.interface import Interface, Attribute

class ICoolDocument(Interface):
    """Cool Document."""
    mangled_path = Attribute("Our mangled path.")
    ...

Then the class CoolDocument is marked with this interface:

from zope.interface import implements
from Products.CoolProduct.interfaces import ICoolDocument
class CoolDocument(...):
    implements(ICoolDocument)
    ...

Finally we must link the event and the interface to the subscriber using zcml, so in configure.zcml we'll add:

...
  <subscriber
      for="Products.CoolProduct.interfaces.ICoolDocument
           zope.app.container.interfaces.IObjectAddedEvent"
      handler="Products.CoolProduct.CoolDocument.addedCoolDocument"
      />
...

And that's it, everything is plugged. Note that IObjectAddedEvent takes care of both manage_afterAdd and manage_afterClone, as it's sent whenever a new object is placed into a container. However this won't take care of moves and renamings, we'll see below how to do that.

Event dispatching

When an IObjectEvent (from which all the events we're talking here derive) is initially sent, it concerns one object. For instance, a specific object is removed. The event.object attribute is this object.

To be able to know about removals, we could just subscribe to the appropriate event using a standard event subscriber. In that case, we'd have to filter "by hand" to check if the object removed is of the type we're interested in, which would be a chore. In addition, any subobjects of the removed object wouldn't know what happens to them, and for instance they wouldn't have any way of doing some cleanup before they disappear.

To solve these two problems, Zope 3 has an additional mechanism by which any IObjectEvent is redispatched using multi-adapters of the form (ob, event), so that a subscriber can be specific about the type of object it's interested in. Furthermore, this is done recursively for all sublocations ob of the initial object. The event won't change though, and event.object will still be the original object for which the event was initially sent (this corresponds to self and item in the manage_afterAdd method -- self is ob, and item is event.object).

Understanding the hierarchy of events is important to see how to subscribe to them.

IObjectEvent is the most general. Any event focused on an object derives from this.

IObjectMovedEvent is sent when an object changes location or is renamed. It is quite general, as it also encompasses the case where there's no old location (addition) or no new location (removal).

IObjectAddedEvent and IObjectRemovedEvent both derive from IObjectMovedEvent.

IObjectCopiedEvent is sent just after an object copy is made, but this doesn't mean the object has been put into its new container yet, so it doesn't have a location.

There are only a few basic use cases about what one wants to do with respect to events (but you might want to read the full story in Five/tests/event.txt).

The first use case is the one where the object has to be aware of its path, like in the CoolDocument example above. That's strictly a Zope 2 concern, as Zope 3 has others ways to deal with this.

In Zope 2 an object has a new path through creation, copy or move (rename is a kind of move). The events sent during these three operations are varied: creation sends IObjectAddedEvent, copy sends IObjectCopiedEvent then IObjectAddedEvent, and move sends IObjectMovedEvent.

So to react to new paths, we have to subscribe to IObjectMovedEvent, but this will also get us any IObjectRemovedEvent, which we'll have to filter out by hand (this is unfortunate, and due to the way the Zope 3 interface hierarchy is organized). So to fix the CoolDocument configuration we have to add:

def movedCoolDocument(ob, event):
    """A Cool Document was moved."""
    if not IObjectRemovedEvent.providedBy(event):
        addedCoolDocument(ob, event)

And replace the subscriber with:

...
  <subscriber
      for="Products.CoolProduct.interfaces.ICoolDocument
           zope.app.container.interfaces.IObjectMovedEvent"
      handler="Products.CoolProduct.CoolDocument.movedCoolDocument"
      />
...

The second use case is when the object has to do some cleanup when it is removed from its parent. This used to be in manage_beforeDelete, now we can do the work in a removedCoolDocument method and just subscribe to IObjectRemovedEvent. But wait, this won't take into account moves... So in the same vein as above, we would have to write:

def movedCoolDocument(ob, event):
    """A Cool Document was moved."""
    if not IObjectRemovedEvent.providedBy(event):
        addedCoolDocument(ob, event)
    if not IObjectAddedEvent.providedBy(event):
        removedCoolDocument(ob, event)

The third use case is when your object has to stay registered with some tool, for instance indexed in a catalog, or as above registered with portal_cool. Here we have to know the old object's path to unregister it, so we have to be called before it is removed. We'll use IObjectWillBe... events, that are sent before the actual operations take place:

from OFS.interfaces import IObjectWillBeAddedEvent
def beforeMoveCoolDocument(ob, event):
    """A Cool Document will be moved."""
    if not IObjectWillBeAddedEvent.providedBy(event):
        getToolByName(ob, 'portal_cool').unregisterCool(ob)

def movedCoolDocument(ob, event):
    """A Cool Document was moved."""
    if not IObjectRemovedEvent.providedBy(event):
        getToolByName(ob, 'portal_cool').registerCool(ob)
    ...

And use an additional subscriber:

...
  <subscriber
      for="Products.CoolProduct.interfaces.ICoolDocument
           OFS.interfaces.IObjectWillBeMovedEvent"
      handler="Products.CoolProduct.CoolDocument.beforeMoveCoolDocument"
      />
...

This has to be done if the tool cannot react by itself to objects being added and removed, which obviously would be better as it's ultimately the tool's responsibility and not the object's.

Note that if having tests like:

if not IObjectWillBeAddedEvent.providedBy(event):
if not IObjectRemovedEvent.providedBy(event):

seems cumbersome (and backwards), it is also possible to check what kind of event you're dealing with using:

if event.oldParent is not None:
if event.newParent is not None:

(However be careful, the oldParent and newParent are the old and new parents of the original object for which the event was sent, not of the one to which the event was redispatched using the multi-subscribers we have registered.)

The IObjectWillBe... events are specific to Zope 2 (and imported from OFS.interfaces). Zope 3 doesn't really need them, as object identity is often enough.

CPS Second Bug Day Wrapup

fguillaume — 2005-10-28

This bug day was announced on short notice, we we still managed to kill 26 bugs (while 9 new were opened). That's good work!

Because there are still 84 active tickets for CPS 3.4 (some are only details of course), we've decided to release a CPS 3.3.7 in the meantime, probably early next week. And we'll keep doing bug days, I propose to have them every Thursday until CPS 3.4 is finally released.

CPS 3.3.6 released

fguillaume — 2005-09-21

CPS 3.3.6 has just been released.

You'll find download links in the "Quick Download" box of http://www.cps-project.org.

This release contains mainly bugfixes, and some speed improvements. It also bundles for the first time CPSSharedCalendar (successor to CPSCalendar) and CPSMailAccess (sucessor to CPSWebMail), two components heavily based on Five.

CPS Bug Day Wrapup

fguillaume — 2005-09-16

The CPS bug day this Wednesday was quite productive. A number of actual bugs were fixed, but more importantly a lot of bug triage and sorting happened, to determine what was urgent (going into CPS 3.3.6), what was necessary for the next stable CPS 3.4 release, and what could be postponed to a later development cycle (CPS 3.5). The Timeline for Wednesday show 65 bugs resolved (fixed or closed). In addition to these, several hundreds have been retargetted or cleaned up.

I'm now confident that CPS 3.3.6 will get released within a week. Before CPS 3.4 we still have a number of important things to do though, so there may be a need for a CPS 3.3.7 which would serve as some kind of prerelease, we'll really need "in the wild" validating of some important features such as the CMFSetup-based installation and import/export tools.

The schedule for the remaining work before CPS 3.4 is really conditionned by the availability of the developers. We still expect CPS 3.4 to be released around mid-october.