Nuxeo DM 5.3 release notes

Nuxeo EP and DM 5.3 have recently been announced. Here are some additional technical notes about this release.

If you can't wait, go download the packages now.

Highlights

Standards for Social Business: Support for OpenSocial allows the creation of gadgets to build web mashups within enterprise applications. Nuxeo DM 5.3 can serve as both publisher of gadgets as well as be an OpenSocial container, allowing the Nuxeo DM repository to host and participate in enterprise mashups.

Windows Sharepoint Services support: Access Nuxeo DM 5.3 via Microsoft Sharepoint for basic library services. Native integration with Windows Explorer ensures information workers can use familiar browsing habits to access Nuxeo DM content and perform common file operations. Microsoft Office integration allows files to be opened or saved directly to/from Nuxeo DM and lets users see information about their content directly from the Document Panel in Microsoft Office.

Federated publishing: Centrally control and publish content to remote Nuxeo applications, file systems, HTTP servers, web portals, and more thanks to a pluggable and unified Publishing Service. It is suited to deployments with a distributed information architecture, balancing ease of publication with requirements to for content control.

Tagging Services: Enhanced metadata capabilities ensures information workers can categorize their content in ways that make sense for average users. Tagging recommendations and tag cloud support enrich an organization's ability to describe and retrieve their information assets.

New Performance Record for Large Deployments: Faster search and retrieval benchmarks for Nuxeo DM repositories holding several million items for hundreds of concurrent users using inexpensive commodity hardware specifications.

Email Capture Enhancements for MS Exchange and Gmail: Improved support for email folders. Fetch email from Microsoft Exchange or Gmail to/from the Nuxeo DM 5.3 email archive folders to ensure contextual management of business communication.

CMIS Support: Ensure true interoperability across multiple ECM, ERP and search systems, Nuxeo DM 5.3 offers a CMIS Server via an add-on component, based on the CMIS draft 0.62.

Search/Indexing Improvements: Granular metadata search and indexing allows users to find and retrieve content based on one or more fields.

Improved Developer Productivity: Several enhancements have been made to make the experience of developing on top of the Nuxeo platform more productive and enjoyable.

From 5.2.1 to 5.3

Our initial plan was to release a 5.2.1 at the beginning of the summer.

But, for good or for bad, we were very busy working on customer projects and missed the windows of opportunity to make a release at the time.

So when we finally could focus on making a new public release of Nuxeo DM, we realized that we had done too much work to just call it a simple maintenance release.

That's why this present release is numbered 5.3.

In parallel, critical fixes have been back-ported in 5.2 as a service pack (Nuxeo 5.2 SP1).

Full backward compatibility

This release introduces new services and APIs, but we did not break any existing API.

This means that developments done against 5.2 should run without major problems against a 5.3.

Unlike the 5.1->5.2 migration that required compatibility packages, the 5.2->5.3 migration should be completely pain-less (see below).

Additional features in Nuxeo DM

As always, we tried to include in Nuxeo DM as many useful features as possible.

Nevertheless, all the new features provided by Nuxeo EP are not directly visible in Nuxeo DM 5.3.

There are several reasons for this:

• the Nuxeo DM distribution is already pretty big

• some of the technical infrastructure improvements can not be simply demonstrated

This means that new features in Nuxeo EP:

• have been included by default in DM when it makes sense

• are available as add-ons in the other cases

For the features released as add-ons, all necessary new APIs in Nuxeo DM are already part of 5.3.

What's new in 5.3?

Web features

During the last months we've worked with our partners and customers on several intranet and portal projects.

This means a lot of small improvements have been made to WebEngine and the Theme engine.

Tag Service

You can now add tags to Nuxeo Documents and:

• search documents based on their tags

• navigate the document database via a tag cloud

The tag service is only available when using VCS storage.

In the middle term, it could become a feature of Nuxeo Core.

Here is a demo:

Blogs

We finally released a simple Blog implementation using WebEngine framework.

This blog is a simple example of how Documents can be managed in the back-office (in the default JSF WebApp) and accessed via a public Web interface.

WebWidgets

The Theme engine of 5.3 comes with support for WebWidgets.

It allows to include UWA JavaScript widgets containers in the Themes.

WebWidgets are very close to OpenSocial gadgets (see below). In the future we will make converge these 2 javascript portlets models.

Because OpenSocial and WebWidgets are providing very similar features, WebWidgets are not part of the default packaging.

OpenSocial

Contributed by the community

If you were at last year's Nuxeo DevDay conference, you probably know that Damien Metzler [video proof] and his team from Leroy Merlin have been working on Nuxeo WebEngine and OpenSocial for some months now. You can find more informations about their work on [Damien's blog].

They contributed a lot of their work on the integration of OpenSocial in Nuxeo:

• Apache Shindig (OpenSocial server) as a Nuxeo Service

• GWT based OpenSocial gadget container

• WebEngine based OpenSocial portal

• Integration of Shindig with Nuxeo's authentication and user management

• Gadget persistence API based on Nuxeo's DocumentManager

Some slides about Nuxeo and OpenSocial:

Nuxeo 5.3 integration

Based on the work contributed by Damien's team, we rebuilt the Dashboard using the GWT gadget container and Nuxeo's REST API.

Basically, all most previously available portlets are still here but now:

• users can customize their dashboard

  • add/remove new portlets (gadgets)
  • change layout

• it's very easy to add specific gadgets (HTML + JavaScript, instead of JSF + Nuxeo Themes)

Here is a video showing their work in action:

Currently, OpenSocial integration into DM is limited to Gadgets. It does not expose all the infrastructure work already done, nor does it exposes all advantages we can gain from the OpenSocial standard.

In the next months, we expect to also take advantage of OpenSocial social features.

Multi-instance management

Remote Publishing

The publishing service has been replaced by a completely new Publisher Service.

The new API lets you publish a Nuxeo Document to an abstract tree.

This tree could be:

• a local Nuxeo Sections tree (as before)

• a Section tree hosted on a remote Nuxeo server

• a filesystem tree

• a custom tree pointing to an external application

This service was designed to support decentralized publishing in Nuxeo, but can also be used to publish Nuxeo document to a existing portal or web site.

Nuxeo Replication

This new add-on provides an application-level replication service.

It is used to replicate changes (Documents, Audit, Directories...) from one Nuxeo instance to another.

Because this replication is managed at the application level, you can decide which part of your data you want to replicate. This replication scope is typically defined by a NXQL request.

This service has already been used to:

• provide an offline client based on Jetty DM bundle (replicate only the documents accessible by the user).

• provide staging between several Nuxeo instances (push a whole tree to staging).

Windows integration

Browser and Office helpers

Since we are not .NET and MSOffice experts, LiveEdit and MSIE plugin have always been a pain to package.

The good news is that we found someone to help us with this, and as a first step, we did a big code cleanup, some bug fixes and all .Net builds were now integrated in the CI chain.

As a consequence, it will be easier to maintain and improve these plugins.

Any feedback on these plugins will be helpful: we don't have many MSIE and MSOffice users on site.

WSS Extensions

Windows SharePoint Services (WSS) is a set of protocols published by Microsoft that describe how SharePoint communicates with the rest of the Microsoft world.

WSS has a broad scope and contains different technologies (FrontPage extensions, WebDAV, CAML, WebServices...).

For implementing WSS extensions for Nuxeo, we focussed of the interfaces exposed by SharePoint to client applications like MS-Office and Windows Explorer.

The goal is to let MS-Office and Explorer talk to a Nuxeo server as if it was a SharePoint server.

Since part of the work is boring protocol implementation (like implementing Front Page extensions), the Nuxeo WSS extensions are implemented by two separated module:

• a generic handler that does not rely on Nuxeo framework code and provides a SPI (Service Provider Interface)

• the Nuxeo WSS backend which implements the SPI on top of the Nuxeo EP services (Repository, Relations, Workflow, UserManager)

In order to use WSS you will need MS-Office 2003 or 2007.

See the screencast below for these features in action:

VCS improvements

Nuxeo 5.2 was the first version to ship with VCS, or "Visible Content Store", our SQL-based backend for Nuxeo.

Nuxeo 5.3 comes with a lot of improvements of VCS.

Performances

We did a lot of performances testing on VCS, using FunkLoad (our open source functional and load testing toolkit) and the importer (see below).

Based on our results and on the feedbacks from support, we did some performance improvements.

Security checks: low-level security checks have been optimized so that even filtering several millions of documents can be very quick.

Path-based queries: VCS now manages a new "ancestors" table that allows quick queries on the path.

Proxies optimizations: proxies search now avoids costly joins.

Thanks to these optimizations, browsing and searching on a repository with several millions of documents is not an issue, even with a cheap server.

Indexing

In addition of the CMIS Join support (see below), VCS now supports multiple fulltext indexes.

Database support

VCS has now been tested "in real life" with several database vendors.

We fixed a lot of small issues related to specific vendors.

Nuxeo DM is now completely CI tested (unit tests and functional testing) against target DBs.

New APIs

Some new APIs have been introduced to manage import and replication.

Nuxeo DocumentManager now support CMIS queries (including JOINS).

CMIS

We are actively working on Chemistry and CMIS, the public review of CMIS 1.0 has just started.

We will publish a demo server based on Chemistry + Nuxeo in a few weeks.

Please see this wiki page for more informations.

Follow the blog entries tagged "CMIS" on the Nuxeo blogs.

Tools

Importer framework

A lot of projects using Nuxeo need at some point to inject documents into a Nuxeo repository:

• initial data migration (from a filesystem or a legacy document management application)

• fetch and inject mail, feeds, records...

• load testing

The importer add-on provides the infrastructure for doing such tasks easily and efficiently:

• thread pool for multi-threaded import

• pluggable threading policy

• pluggable DocumentModel factory

• pluggable crawler

• configurable batch transaction management

• bulk mode management for events and associated handlers

• JAX-RS bindings

It includes ready to use implementation for:

• a simple file system importer

• a file system + meta-data importer

• a random text injector based on dictionaries

This package is available in mercurial.

JCR2VCS migration tool

This tool provides a way to migrate JCR based repositories to VCS.

This add-ons contains 2 parts:

• one exporter (JCR on 5.1.6 and 5.2)

• one importer (VCS on 5.3)

Documentation about data migration is available here.

Improving the developer experience

Jetty / Tomcat support for DM

Nuxeo distribution now supports deploying Nuxeo DM on Jetty and Tomcat.

These distributions does not include any Java EE supports (Transactions, EJB3, JCA...), but provide the same ECM features.

These packagings are mainly targeted to development environment since startup is significantly faster than a complete JEE container (less than 40s for full startup).

In the near future (one month), we will add a Transaction Manager to the tomcat package, this will make the tomcat package completly ready for production.

JBoss speed-up

The JBoss deployer has been optimized to start faster. A full Nuxeo DM startup on a laptop with JBoss takes now 1m45 instead of 2m30.

NB: this speed improvement should be very significant on windows boxes having an virus scanner intercepting all file system accesses.

Seam Hot reload support

Seam components hot reload is now supported.

This can significantly improve development speed when using the JSF framework.

See: http://doc.nuxeo.org/xwiki/bin/view/FAQ/SeamJSFHotReload for more info.

GWT

GWT integration in Nuxeo has been improved so that you can now run GWT Application in hosted most with a bundled Nuxeo server deployed inside Eclipse as a Jetty server.

It is now easier to build a GWT app that uses Nuxeo Platform:

• in development mode, eveything is integrated into Eclipse

• we provide a single API jar for all Nuxeo services

Documentation

We improved the available documentation:

• Single entry point for learning Nuxeo www.nuxeo.org

• "Getting started" entry point

• Documents about DB tuning and settings

• Documents about data migration

Translations

Translations files have been externalized so that it will be easy for the people interested in submitting translations to do so.

Your turn now!

Trying it out

To try it out, grab the packages:

• http://www.nuxeo.com/en/products/dm/try-it

Joining the community

• Read the doc.

• Get the source code.

• Join the conversation: mailing list, forum.