Nuxeo Developers Blog

ElementTree, serialization and namespace prefixes

The way ElementTree outputs namespaces in serialized output can be a problem with some applications.

Here is an example of such an ouput :

  >>> import cElementTree as etree

  >>> stream = """<?xml version="1.0" encoding="UTF-8" ?>

  ... <doc xmlns="http://bar"

  ...      xmlns:foo="http://foo/">

  ...   <foo:sub/>

  ... </doc>""

  >>>

  >>> doc = etree.XML(stream)

  >>> print etree.tostring(doc, encoding="UTF-8")

  <?xml version="1.0" encoding="UTF-8" ?>

  <ns0:doc xmlns:ns0="http://bar">

    <ns1:sub xmlns:ns1="http://foo" />

  </ns0:doc>

  >>>

We can see that the declared namespaces are now given an alias and all prefixes are now changed using those defined aliases. This is absolutley correct in a XML point of view but you can be in trouble sometimes with some applications for which you are outputing XML from elementtree based Python programs because they do not support this properly on their side.

Here is a workaround I found but I don't know if others exist :

  >>> import cElementTree

  >>> import elementtree.ElementTree

  >>>

  >>> my_namespaces = {'http://foo' : 'foo',

  ...                  'http://bar/' : bar}

  >>> elementtree.ElementTree._namespace_map.update(my_namespaces)

  >>>

  >>> stream = """<?xml version="1.0" encoding="UTF-8" ?>

  ... <doc xmlns="http://bar"

  ...      xmlns:foo="http://foo">

  ... <foo:sub/>

  ... </doc>"""

  >>>

  >>> doc = cElementTree.XML(stream)

  >>> print cElementTree.tostring(doc)

  <bar:doc xmlns="http://bar">

    <foo:sub xmlns:foo="http://foo" />

  </bar:doc>

Here, this has been serialized without replacing the prefixes within qualifed names.

The idea is that we are adding well known namespace prefixes to elementtree default ones.

The default elementtre ones are defined within elementtree/ElementTree.py like below :

  _namespace_map = {

      # "well-known" namespace prefixes

      "http://www.w3.org/XML/1998/namespace": "xml",

      "http://www.w3.org/1999/xhtml": "html",

      "http://www.w3.org/1999/02/22-rdf-syntax-ns#": "rdf",

      "http://schemas.xmlsoap.org/wsdl/": "wsdl",

  }

This is not the best way I would have hope to find. Please let me know if you know any others.

The problem I had recently was with OpenOffice.org 1.1.x.  (I don't know about the version2 though).

I could parse and serialize OpenOffice.org content XML documents and read them from OpenOffice.org at first. But as soon as I was modifiying the document from OpenOffice.org then it wasn't taking the namespace prefix aliases into consideration while inserting new elements. I used this trick and now OpenOffice.org is happy. I'm gonna report this issue to Laurent to see if the OpenOffice.org guys are aware about this issue.

I fixed the issue as shown below. I used the nmspace.mod from the OOo dtd to find out the relevant OOo namespaces.

OOo_NS = "http://openoffice.org/2000/"



OFFICE_NS = "%soffice" % OOo_NS

TABLE_NS  = "%stable" % OOo_NS

STYLE_NS  = "%sstyle" % OOo_NS

TEXT_NS   = "%stext" % OOo_NS

META_NS   = "%smeta" % OOo_NS

SCRIPT_NS = "%sscript" % OOo_NS

DRAWING_NS = "%sdrawing" % OOo_NS

CHART_NS = "%schart" % OOo_NS

NUMBER_NS = "%snumber" % OOo_NS

DATASTYLE_NS = "%sdatastyle" % OOo_NS

DR3D_NS = "%sdr3d" % OOo_NS

FORM_NS = "%sform" % OOo_NS

CONFIG_NS = "%sconfig" % OOo_NS



FO_NS = "http://www.w3.org/1999/XSL/Format"

XLINK_NS = "http://www.w3.org/1999/xlink"

SVG_NS =  "http://www.w3.org/2000/svg"

MATH_NS = "http://www.w3.org/1998/Math/MathML"

# This will be used for the XML serialization and elementtree.

NAMESPACE_MAP = {

    OFFICE_NS    : 'office',

    TABLE_NS     : 'table',

    STYLE_NS     : 'style',

    TEXT_NS      : 'text',

    META_NS      : 'meta',

    SCRIPT_NS    : 'script',

    DRAWING_NS   : 'drawing',

    CHART_NS     : 'chard',

    NUMBER_NS    : 'number',

    DATASTYLE_NS : 'datastyle',

    DR3D_NS      : 'dr3d',

    FORM_NS      : 'form',

    CONFIG_NS    : 'config',

    MATH_NS      : 'math',

    SVG_NS       : 'svg',

    XLINK_NS     : 'xlink',

    FO_NS        : 'fo',

   }



import elementtree.ElementTree as etree

etree._namespace_map.update(NAMESPACE_MAP)

(Post originally written by Julien Anguenot on the old Nuxeo blogs.)

Choosing a Javascript library for Zope

Choosing a Javascript library for Zope

Background

What's AJAX ?

(If you know it, you should skip this section)

Wikipedia says:

Asynchronous JavaScript and XML, or Ajax, is a web development technique for
creating interactive web applications using a combination of:

  - XHTML (or HTML) and CSS for marking up and styling information
  - The Document Object Model manipulated through JavaScript to dynamically
    display and interact with the information presented
  - The XMLHttpRequest object to exchange data asynchronously with the web
    server. (XML is commonly used, although any format will work, including
    preformatted HTML, plain text, JSON and even EBML)

Like DHTML, LAMP, or SPA, Ajax is not a technology in itself, but a term
that refers to the use of a group of technologies together. In fact,
derivative/composite technologies based substantially upon Ajax, such as
AFLAX, are already appearing.

(full definition here: What's AJAX ?)

In other words, AJAX allows a developer to make calls to the server from the loaded page with Javascript, and change the page based on the server's answer without having to do a complete page reload.

Since the publishing machinery is not involved, a asynchronous round-trip is very fast and allow big ergonomics improvments. This is due to the fact that asyncrhonous calls are made over server methods that just quickly renders the needed datas, in opposition to a regular call that calculates and renders a full page.

Why should we care of AJAX in Zope applications ?

AJAX isn't just the latest buzz word out there you can show up in your applications (the come-here-we-have-some effect), AJAX is not this latest and coolest technological thing all hype developers should know about.

AJAX is not the web UI silver bullet either.

AJAX is just a tool that let you focus on something that often get lost, when creating a web applications: people.

I believe this is the main reason of AJAX success: developers are able to greatly enhance their users experience with a few drops of Javascript.

Examples of use cases:

• A select box value is changed, the values of a second one are reloaded.
• Direct edition of page parts (see CPSWiki for instance)
• Content panel that changes on user actions
• Forms are checked before actually sent (see CPSMailAccess email editor)
• etc..

People can argue there are a lot of caveats on this (for example, using the 'back' button of the navigator can brake it) but it improves so much the flexibility of a web application and make web app as reactive as desktop apps.

A recent Ajax Survey shows us that AJAX is used in real applications everywhere. Even if the amount of developers that have answered this survey is not big, it also brings a good idea on what tools are actually used to do AJAX.

Zope is not apart from the main web developpers stream, and can use any AJAX library out there. AJAX libraries have different approaches, explained in the next section.

Different libraries approaches

There are several kind of libraries available to work with AJAX:

• Low-level client-side frameworks (LLF)
• Client-side Application frameworks (CSF)
• Server-side Javascript generation frameworks (SSF)
• XForm approach framework (XFF)
• Other frameworks

Low-level client-side frameworks

Low-level Javascript libraries provide a simple piping to access the server by wrapping XMLHTTPRequest objects and let the developper perfom DOM manipulation with the server's answer.

Most of these libraries are a thin layer that provides a portable API for these actions, and other utilies like:

• XPath query APIs
• XSLT APIs
• etc.

Example of such libraries are: Sarissa, XHConn, LibXMLHttpRequest (non GPL), etc.

Here's a small example of a simple AJAX call in Sarissa:

Sarissa.updateContentFromURI = function(sFromUrl, oTargetElement, xsltproc) {
    try{
        oTargetElement.style.cursor = "wait";
        var xmlhttp = new XMLHttpRequest();
        xmlhttp.open("GET", sFromUrl);
        function sarissa_dhtml_loadHandler() {
            if (xmlhttp.readyState == 4) {
                oTargetElement.style.cursor = "auto";
                Sarissa.updateContentFromNode(xmlhttp.responseXML, oTargetElement, xsltproc);
            };
        };
        xmlhttp.onreadystatechange = sarissa_dhtml_loadHandler;
        xmlhttp.send(null);
        oTargetElement.style.cursor = "auto";
    }
    catch(e){
        oTargetElement.style.cursor = "auto";
        throw e;
    };
};

Sarissa takes care of all portability aspects to allow this code to work under most navigators that implements Javascript. These simple libraries provide a quick and simple way to use asynchronous calls on a web view.

Google has also released its XSLT/XPath JavaScript engine as an open source project: AJAXSLT

Client-side application frameworks

Most of the time, client-side application frameworks provide the same features than Low-level client-side frameworks, but also brings a higher level, component-oriented, set of APIs.

These APIs let the developer work on client-side like he or she would do in a desktop app with a classical GUI toolkit. The style and the amount of javascript code produced are very dependent on the library, and the developer is quite driven by the toolkit.

For example, toolkits like Prototype provides a object-oriented way to work in Javascript, and therefore changes a lot how Javascript is used in a web application and its importance in the architecture.

This example show how to create a class hierarchy with Prototype:

function Manager () {
    this.reports = [];
}
Manager.prototype = new Employee;

function WorkerBee () {
    this.projects = [];
}
WorkerBee.prototype = new Employee;

function SalesPerson () {
  this.dept = "sales";
  this.quota = 100;
}
SalesPerson.prototype = new WorkerBee;

etc...

Some toolkits also provide very high level functionnalities that let the developper implement common AJAX use cases in a few lines. OpenRico, which is built on the top of Prototype, is one of those.

For example, adding a nice fade effect in OpenRico is done with a single command:

new Effect.FadeTo('fadeMe',
                  .2, // 20% opacity
                  500, // 500ms (1/2 second)
                  10, // 10 steps
                  {complete:function() {setStatus('done fading element.',
                   1500);}} );

Complex use cases, like live grids, can be provided as well. See OpenRico live grid demo

Client-side application frameworks also provide other developper tools like:

• Unit-testing framework integration, like ECMAUnit or Test.Simple
• Debuggers
• Different protocol for client-server dialogs (XML, JSON)

The learning curve involved by these frameworks worth it, as they bring to Javascript what we have in Python.

• Javascript? huh! I don't want to use it
• What if you can actually use object oriented programming in it ?
• I always get thousands of errors in a few lines of Javascript
• What if you can debug it ?
• Is there any pop corn left ?

Server-side Javascript generation frameworks

Server-side Javascript generation is generally based on the no-javascript-skills-needed paradigm: the developper creates code on server side, using the server's language using specific APIs, or a specific model. The framework then automatically generates the right HTML and Javascript elements and sends them to the client. Sometime this generation is made on client sied with a javascript engine.

CrackAjax is one of those for Python, and let the developper create AJAX views:

import crackajax
import cherrypy
import ituneslib

class iTunesAjaxPage(crackajax.AjaxPage):

    @crackajax.clientside
    def do_search():
        update_list(search_songs(document.getElementById("searchbox").value))

    @crackajax.serverside
    def get_all_songs(self):
        return [self.SONGS]

    @crackajax.serverside
    def search_songs(self, query):
        query = query.lower()
        return [filter(lambda s: self.does_song_match(query, s),
                self.SONGS)]

crackajax.init("jsolait")
cherrypy.root = iTunesAjaxPage(ituneslib.Library("Library.xml"), "")
cherrypy.server.start()

The serverside decorator tags the methods to become an XML-RPC call, and the clientside automatically convert Python code into client-side javascript, using Python-to-JScript .NET compiler.

There are also less radical approaches, where the framework provides APIs to describe the javascript that needs to be created, using for example a descriptive langage that covers explicit behaviors. (see Azax approach).

XForm approach frameworks

Section to be completed (see FormFaces)

Other frameworks

They are many other hybrid approaches that mixes server-side and client-side, or approaches that provides even lower-level mecanisms.

They are not detailed here because they are often tighted to a particular web framework or don't bring more features that what we would have in other types.

Choosing a library

For CPS , and moreover for Zope-based applications, an AJAX library has to be taken in CSF category or SSF, since all functionalities available in LLF exists in CSF. The next sections provide a very quick review on existing libraries.

SSF Libraries

CrackAjax

Pros:

• straight-forward: a server side approach, like CrackAjax or Azax, provides a straight-forward tool to add Ajax features in a web application, by hiding the Javascript layers from the developer with techniques that can be compared to Template MetaProgramming, and providing a meta langage or a set of APIs in the web application native language.

Cons:

• not so hidden: as you can see in this CrackAjax example, the code written in Python is very similar to what you would have done in Javascript. Furthermore, it's a bit of magic since the given Python code cannot be unit tested howsoever (the 'document' variable does not exists on Python side)

Azax

Pros:

• clean architecture: Sometime the SSF approach is also meant to drive the developper to cleary separate the Controller and the Interface. For example Azax xml templates provides a way to separate the behavior of a page and the UI, example of separation: Azax page behavior / Azax page template
• includes a link to mochikit, allowing hybrid behaviors

Cons:

• one more layer: This clean separation can be done straight forward in Javascript, without adding a layer, but describing behaviors in XML can also be seen as more portable and readable, because the code could work without Javascript, wich is just the last layer This approach is very similar to what XUL has in XBL Events, but this is a bit of Not Invented Here.

CSF Libraries

A quite complete survey at OSA foundation is available and EDevil's blog has a nice library list.

A library for CPS

In the next blurg, I will try to see wich library fits the best for CPS by:

• Suming up the use cases that already need to be covered, in both Zope 2 and Zope 3.
• Selecting the 10 best libs out there
• Trying out all use cases with each toolkit to compare.

And please don't hesitate to react on this blurg, if you want to give more precisions or corrections on some toolkits, or if you have opinions, comments. These will be integrated on the page.

Thanks to Paul Everitt, for for pointing out some mistakes on this entry, and for pointing out the XForm approach, that will be completed soon.

(Post originally written by Tarek Ziadé on the old Nuxeo blogs.)

CPS Ajaxification, round #1

In my last blurg I said I would select some toolkit out there to compare the result, but after some mails exchanged on Z3 mailing list ( Ajax in Zope 3 Thread), I agreed with what Benji York and Gary Poster said in this subject:

"I'm very interested in coming up with a one or more relatively simple server-side proposals to make Ajax stuff easier, but I think the Ajax stuff (and even the whole JavaScript world) is too immature to settle on one client-side implementation. Therefore we should concentrate on doing things that help most or all of the JS toolkits be more usable with Z3."

So I have started the work over CPS with Scriptaculous, focusing on uses cases and when doable, on the server-side infrastructure.

CPS Use case #1: CPSDocument AJAX form controller with graceful degradation

The first use case in CPS is to embed existing forms used to change document fields. The Javascript needs to let the user validate a form, that is sent asynchronously through an XMLHttpRequest object, to get direct feedback.

The feedback is then used on client-side to either:

• Update the form on errors, to point out the fields that are not correct;
• tell the user changes where made, optionnaly changes the page title when the user has change the title, and eventually redirect the user.

Since this is an ajaxification of existing server side Z2 infrastructure, it was very hard, even overkill to start doing server-side work, so for this use case I have just added a bit of code on server side to make it work, without trying to do a clean infrastructure.

The constraints are:

• Graceful degradation: if JavaScript is not available, the form has to work as usual
• If the user tries to send a file in the form, the classic form will be used
• Use Object Oriented approach (OO)
• Use Test-driven development approach (TDD)

Graceful degradation

This is one of the most interesting thing to work on: make the pages still work when Javascript is unavailable, so CPS stays compliant with all accessibility levels.

This is simply done by an unobtrusive way of doing thing: when initialized, the Javascript code hook itslef to the form, degrading its action and its submit buttons.

TDD and OO

The OO approach is very easy under Scriptaculous, wich is built on the top of Prototype. So this is as easy as Mochikit, and I would say, as easy a coding some classes in Python !

The TDD approach is very simple too: Scriptaculous comes with a unittest framework that provides usual tools you would find under Python (assertXXX methods, test fixtures, etc.). Doing TDD in Javascript come in two parts: browser-side testing and server-side testing when AJAx is involved.

browser-side testing

browser-side testing is done by creating a simple HTML file that contains all use cases to be tested, wich is the HTML environnement where the JS has to evolve, and the test object itself with all tests. When the HTML file is called, all test are called and you can even display feedback with green divs and red divs a la selenium.

Try CPSDocument Js Test here, it should fail on some testunder Firefox, maybe more on Safari, and even more under Konqueror :) (old version).

server-side testing

I won't discuss this part in that blurg, but to cover the whole code, we need to validate the code that generates AJAX answers as well.

Conclusion

CPS now has a nice AJAX form validation that will be available in the next releases, and wich is still beeing worked out in a branch at this time.

This was quite simple to do, thanks to the nice, smart js toolkit, and thanks to the scriptaculous community out there, that helped me out on some small problems

I didn't talk about two things yet:

• Server-side infrastructure
• Test automation

Server-side infrastructure

The next use case fits better for presenting how AJAX helpers can be done on Zope side: drag'n'drop in folderish objects.

Test automation

In my next blurg I will explain how to automate this tests and add them in a Zope environment, to be able to call them in a Python test suite, try out several browser types, and get feedback about it.

-- footnotes

tries to send a file: You cannot send file with Javascript (security hole) unless you use some iframe artifact.

(Post originally written by Tarek Ziadé on the old Nuxeo blogs.)

Guido and Webframeworks

Guido van Rossum has started to look for web frameworks. Now, he doesn't seem to enthusiastic about using Zope. And at first it looks like he is making the right choice, because his stated requirement list is too small:

• Independence from web server technology.
• Templating with reuse.
• Cookie handling.
• Query parsing.
• URL dispatch.

Note especially the lack of any kind of data storage, authentication/access control, forms and session handling. Well, with these requirements, he shouldn't use Zope, that's for sure. But does he really not want any storage, or access control?

Wel....in his second post he extends the requirements quite lot, and suddenly it seems like persistence, acces control and session is in the game again:

"Maybe we need more standardization efforts like WSGI, that let you plug in different animals, or roll your own, for various pieces of useful web functionality: for example URL dispatch, templates, persistence, authentication, sessions, forms, style sheets, i18n, and client-side scripting (AJAX or not)."

Right, for a really good, generic web framework, we surely do need all this. And Zope 3 does it. But Guidos comment about Zope is: "From this perspective, Zope and Twisted are off the scale: they support the mix-and-match approach, offering several alternative solutions for many of the important issues (templating, persistence, authentication, etc.). But they only work if you drink lavish quantities of their particular flavor of kool-aid, and that's not good enough for me. I don't want to depend on any particular flavor of interfaces, adaptation, serialization, discovery, etc."

Yes. They do require that you use their "particular flavor of interfaces" and so on. But how are you to do it otherwise? If you want the persistance to be pluggable, then everything that uses the persistance has to conform to the same interface. If you want to have pluggable URL dispatch, then everything that uses this URL dispatch must conform to the same interface. If you want to have pluggable authentication, then all the plugins must be written for the same plugin system.

Because that's what standardization *is*, conforming to a standard.

What Zope 3 does is not only to standardize all the mentioned things, but to standardize the way to make these standards, that is by defining interfaces, making objects and adapters, and connecting it all together in ZCML. Now, if you don't like it, thats fine, but any effort to do the same things will necessarily need to create a framwork of similar complexity. It might be possible to show that Zope 3s approach of how to do it is wrong, but Zope 3 does do it.

I suspect there are two issues here:

1. Zope is often seen as being unpythonic. Zope 2 certainly was, but Zope 3 is trying really hard, and in my opinion succeeding, in being pythonic. There is no more PYTHONPATH magic, there are no more half-assed "pythonscripts".
   In Zope 3, you create pure Python products, and you the create a user interface for these by creating "views", which will typically be a page template. That's really the whole gist of it. Now, if you want to do override some default behavious, like the way URL handling is done, then yes, you will have to write the necessary adapters for doing that according to the Zope 3 standard of doing it. But isn't that unavoidable?
   
2. Guido does not like Zope Page Templates, and I can understand why. But, as noted, the templating system in Zope 3 is pluggable, and you can for example use ClearSilver thanks to Martijn Faassens Clarity product.
   

I'm all for reinventing the wheel if the new wheel gets better. But really, all you Python people who shun Zope because it's rumoured to be unpythonic or force you to do things certain ways, take a long hard look at Zope 3. You'll be happy you did.

"I'm looking for solutions that depend only on the Python standard library, and use accepted Python idioms and patterns. Any takers?"

Yes, Zope 3. :-)

(Post originally written by Lennart Regebro on the old Nuxeo blogs.)

Reminder : OOo Developpers Article Contest

The first session of the Developpers Article Contest is running and will end at the end of the month.

As announced, this contest is to generate new documentation to developpers and everybody can participate. Developpers and technical writers are welcomed to propose articles. The subject has to cover a technical aspect of OOo such as building, patching, addon, extensions or OOo scripting aso.

Submissions have to be sent as attachments in the OASIS OpenDocument file format to the mailing list contest@development.openoffice.org

The contest rules can be found on the OOoWiki. Each month, an article will be elected and the winner author will earn 750 euros. He will also be member of the comitee for three months if he wants The remaining articles will stay in the contest for the next month so you have many chances to win :-)

So, write your articles and submit them. We are all waiting reading them ...

(Post originally written by Laurent Godard on the old Nuxeo blogs.)

OOoCon2006 : Results ... After Koper, let's go to ...

Lyon has been choosen by votes

The official annoucement by Jacqueline, marketing lead
http://www.openoffice.org/servlets/ReadMsg?list=announce&msgNo=272

After the good work of Cédric at INSA, lets help him now organizing this event.
All the energies are welcomed to make this event as successfull as the previous years

You"ll surely be called for help - Stay tuned !!

(Post originally written by Laurent Godard on the old Nuxeo blogs.)

Apogée project website goes live !

We just put online the community portal for the Apogée project : apogee.nuxeo.org.

It includes a public website and a collaboration workspaces where all people interested in the project can collaborate (document sharing and reviewing with versionning, openoffice and docbook support, etc.).

We also published some design documents of the project (on the XForms engine and global architecture). This site is really expected to live and be the center of the community.

We invite all people interested to join the site and start working on documents (the mailing-list is also available for discussions).

I hope to see you there ! :-)

(Post originally written by Eric Barroca on the old Nuxeo blogs.)

A few rules to keep in mind when using Ajax

In this entry, I try to sum up some rules that should be looked over when adding AJAX features in a CMS.

keep accessibility

A CMS is used by a wide variety of people and WAI rules should be kept in mind. All functions and screens should be usable even though javascript is not enabled.

This means that AJAX features have to be implemented with methods like graceful degradation to avoid the CMS to be dependent on Javascript features.

Most of the time, this is quite easy to implement a Javascript enhanced version over a regular version, but sometimes two version has to be developed in parallel, like gmail for example does.

There are few exceptions for this rule:

• it's ok to develop managment screens or backoffice screens in javascript

• same things for some products like webmails, calendars: people that really need accessibility for these very specific products will use specialized pieces of softwares.

• the product is very specific, and ajax is used for example to reduce network bandwidth. example: MacRumorsLive

one state, one url

This is probably one of the most important point. When developing ajax interfaces, it's very tempting to develop advanced screens that let the user play around without doing direct calls.

The javascript retrieves datas asynchronously from the server and refresh the interface, the user makes changes, and the javascript display these changes directly.

One of the common approach is to use a "model-driven" infrastructure that let the different interface elements get synchronized with a local pool of data, using xml transformation or, like in some recent developments, using things like CTAL. In other words, if the user changes the data through one interface, like an input text, a button, or anything else, the local data pool is changed and other part of the screens are refreshed consequently.

It's a great approach as long as the rendered screen looks exactly as it would look if another user would load the same url in her browser.  This means that the server state should always be synchronized with what the user is doing.

So having a local pool of data (what people call "model-driven") is more likely to be an optimisation to minimize exchanges by having a local cache, and should be done with this rule in mind.

Of course the benefit of this approach also let the client-side calculate the interface itself. In this case the javascript becomes a template engine and push the server to a single role wich is: data provider; but this has to be done with the first rule in mind: a javascript template engine won't work under lynx, and having two mecanisms to do the same thing can be hard to keep growing together.

Back to the one state, one url rule.

If this rule is not respected, third party softwares, like search engines, or xml-rpc callers, won't be able to use the web application because they use urls as single state elements. Moreover, users most of the time expect a given url to give them the same result (that's why the "favorite" button exists)

use test-driven approach, and automate tests

Most of the time, a piece of javascript is not well tested. Developers should work in a test-driven development (TDD) way like they do in their main language.

All actual Javascript toolkit now has everything needed to set up test automation, and let the developpers benefit from TDD.

Moreover, Javascript pieces should be tested over all possible execution contexts. A tool like Selenium can be used to automate functional test over all kind of browsers.

References:

• Why Ajax Sucks, by Chris McEvoy, http://www.usabilityviews.com/ajaxsucks.html
• one state, one url not respected example: http://weblogs.asp.net/mschwarz/archive/2005/08/06/421761.aspx
• MacRumorsLive: http://www.digg.com/apple/macrumorslive.com_using_AJAX_to_update_their_MacWorld_news
• WAI: http://www.w3.org/WAI/
  

(Post originally written by Tarek Ziadé on the old Nuxeo blogs.)

CPS Ajaxification, round #6 a dynamic tree and an event oserver

Features

The treeview that we wanted is pretty similar to what Zope 3 has (xmltree), except that it one is implemented with scriptaculous. The principles are quite simple: branches of the tree are asynchronously loaded through Javascript whenever we need it. Scriptaculous provides the nice fade-in effect, which make the whole thing looks good.

Now the biggest deal was to be able to hook the treeview with what happens in the center of the page.

Since some drag'and'drop features were previously added, the tree needs to be refreshed whenever a drop which changes the structure of the portal is made. In our case, it happens when folders are reordered or a folder is dropped over the treeview. A event mechanism has been created, in order to be able to refresh the treeview whenever a 'drop event' happens.

See 'implementation details' section for some details.

Screencast

(if you read this entry through a aggregator, you should open your flash enabled browser to see the screencast)

Implementation

The treeview is displayed in a portlet, which is, thanks to the briliant people that have thaught it, a standalone document. In other words, a simple Five view was the only thing to add to be able to query the server for an fresh version of the portlet

The treeview acts as an observer over the page, registering itself on all 'drop' events. This is done with a simple javascript class that gathers for each event name, a list of functions to call when the event is triggered. The drag'n'drop features triggers the so-called event, and the function asynchronously gets the new rendered portlet

All the code is viewable here: http://svn.nuxeo.org/trac/pub/browser/CPSPortlets/trunk/browser/
and here is the class that register event observers and trigger them : http://svn.nuxeo.org/trac/pub/file/CPSDefault/trunk/skins/cps_javascript/cpsdefault.js.dtml (see at end of file)

(Post originally written by Tarek Ziadé on the old Nuxeo blogs.)

Looking for fast and memory friendly Python XML processing ?

You don't how to optimize your Python based XML application anymore ? Are you tired of running out of RAM ? You got memory leaks all around ?

I've been in this situation until last week for one year.

I spent the last week rewriting a customer application written in cDomlette using cElementTree (note when I wrote it at this time cELementTree didn't exist).

My cDomlette experience has been a real pain last year with this project for the reasons I described above.

Don't get me wrong about cDomlette. This is a library really well documented and much more better than the available DOM libraries in the standard Python distribution but you can't use it for applications such as the one I've been working on. Simply it's not enough.

The application I'm talking about is an application for financial auditors. This is based on CPS and make an heavy use of OpenOffice.org calc documents. Lots and lots of OOo calc documents to process, modify according to complex financial rules every transaction.

So two weeks ago I had problems with my application heavily requestedby several users. Too slow, the servers were overloaded processing XML documents. For those who know what I'm talking about note the bad performances weren't only related to the XML processing. Not only...

I couldn't optimize the XML processing part anymore then I decided to recode from scratch all the XML processing modules I wrote with cDomlette using cElementTree this time. And oh dude, it rocks ! It really really rocks ! It works amazingly well now. The code is much more readable and maintainable,
because of the elementtree API, it's fast and it's not consuming lots of memory. The same transactions are now completed ten times quicker than before !

Thank you Fredrik Lundh. You're the man !

I'm more than happy for several reasons here :

• my application is working and my customer is happy.

• having cElementTree working this way (meaning working so well)
  makes Python a first candidate language for XML processing.Java,
  for instance, doesn't have such module available it seems.

• elementtree will be soon available within the Python standard library.

What about lxml ?

I considered using lxml for production but this library is too young right now. These are the problems I met trying to use it in production :

• The dependencies are far too high. (libxml2 and libxslt bleeding
  edge revisions) This makes it hard to use everywhere and it seems
  to be the reason why Zope3 can't makes it a dependency for instance.
  

• Not yet memory leaks free... (see the mailing list)

• missing iterparse() method of elementtree for SAX-like processing (this one is a real killing in elementtre)

lxml should be pretty close to cElementTree performances though.

Of course, lxml has some nice features that elementtree does not provide yet because lxml is based on libxml2 and libxslt. But for 90% of the applications, I might need to write, elementtree does the job. And it does it really well.

(Post originally written by Julien Anguenot on the old Nuxeo blogs.)