Ruslan Spivak

Here we go - Java Content Repository Ice Connector

rspivak — 2006-04-28

This is kind of announcement what i was working on recently :)

And recently i began to work on Jackrabbit JCR connector that is Java implementation of a facade to JCR utilising Ice communications engine that provides bidirectional cross-language integration service, high-scalable communication platform and this
way Ice Connector allows to interact transparently with JCR from Java, C#, VB, C++, Python, PHP.

Ice, the Internet Communications Engine, is a free high-performance middleware which includes wealth of services and plug-ins:
messaging service, grid compuring and much more.
You can get additional information at http://www.zeroc.com

Currently JCR Ice Connector provides small set of read methods, but more stuff is on the way. And, yes simple Python client is included :)
You can check the code at:
http://svn.nuxeo.org/trac/pub/browser/YellowCake/org.nuxeo.jackrabbit.ice/trunk
More information is at:
http://svn.nuxeo.org/trac/pub/file/YellowCake/org.nuxeo.jackrabbit.ice/trunk/README.txt
This will be part of our ongoing effort towards CPS4, so stay tuned :)

Nautilus + ps2pdf script

rspivak — 2005-11-20

Recently i was producing many funkload's bench results. Firefox lets you easily create files in PostScript format by printing page to file. I wanted easy way to convert produced ps files to pdf from Nautilus without going to terminal. Thanks to Nautilus, it supports custom scripts(more info here), that after putting to ~./gnome2/nautilus-scripts directory, appear in popup context menu.

  #!/bin/bash

  cd $NAUTILUS_SCRIPT_CURRENT_URI
  for arg in $@
  do
      ps2pdf $arg
  done

After saving that in ps2pdf.sh and making script executable you can convert one or multiple selected ps files to pdf by right clicking on selected file(s) -> Scripts -> ps2pdf
Very nice :)

AdapterLookup C optimizations: status

rspivak — 2005-09-06

I was talking to Jim Fulton some time ago regarding C optimizations to adapter lookup. Jim currently makes experiments regarding new architecture for that stuff. If he decides to left old architecture then my branch will be considered. There is a lot of stuff to clean and make better in my branch, but currently i'm waiting for Jim's decision.

As a byproduct of my work on adapter lookup C optimization i wrote 2 patches for Python, one was fixing possible memory leak(that possibility is though very rare) in builtin filter function(Python/bltinmodule.c) and was accepted nearly at once(thanks to developers not beating me for sending it to dev list, one should post it into sf.net patches section for python :), second was just to remove trailing whitespaces in builtin module, both were actually one patch, but i was told(and right) to divide cosmetics into separate patch and that was recently accepted, so both now are in Python2.5 trunk.
Well, at least i'm glad i was able to make something useful(?:) for best programming language in the world! :)

BTrees and pop()

rspivak — 2005-09-03

Yesterday Tim Peters merged my branch for pop() method in BTree and Bucket into main trunk, so ZODB3.6 now has this feature and is closer to what python's mapping protocol provides.

Of course Tim made modifications to C code to make it more simple and added more tests :) Thanks Tim!

Now we have both setdefault() and pop() methods, hope people will find it useful in their ZODB applications.

BTrees and setdefault

rspivak — 2005-09-02

Working on Z3ECM code base i met following code construction:

try:
    return self.__applications[key]
except KeyError:
    self.__applications[key] = PersistentList()
    return self.__applications[key]

where __applications is OOBTree.

My immediate reaction was to use standard python's mapping protocol method – setdefault as �Readability counts� (C) and rewrite above long code to following:

return self.__applications.setdefault(key, PersistentList())

nice, isn't it? and i was a bit surprised to get

AttributeError: 'BTrees._OOBTree.OOBTree' object has no attribute 'setdefault'

Of course setdefault can be easily emulated, but i want to have short code :) You need to be aware that default argument instance will be created each time you call setdefault, even if it won't be used.

So i raised question in zodb-dev mailing list about adding setdefault method to ZODB and was kindly pointed by Dmitry Vasiliev that this question was raised nearly a year ago - http://mail.zope.org/pipermail/zodb-dev/2004-October/008040.html and as a result it was frozen and no implementation was provided.

Currently some people came up that it's a good idea to have this method for btrees, some had objections. Tim Peters showed how setdefault could be emulated:
This one is obvious:

result = self.__applications.get(key, None)
if result is None:
    result = self.__applications[key] = PersistentList()
return result

While this one uses special btree's method insert:

self.__applications.insert(key, PersistentList())
return self.__applications[key]

This time Jim Fulton gave +1 and as Tim noted: - �Jim's +1 wipes out any number of -0 votes :) .�
So i made my hands dirty, created separate branch in ZODB and implemented setdefault (of course in C :) Bit lately when reviewing my branch Tim noted the behaviour(specific) of python's setdefault doesn't fit well with BTree:

>>> d = {}
>>> d.setdefault(666)
>>> d
{666: None}

You see what's the point?
It will work only with OOBTree and IO, but not with OI, II, IF.
Tim proposed to have 2 required parameters to setdefault and that's how it's now – you should explicitely pass default to method and this was raised on python-dev too and seems like this behaviour will go into standard python, because usage of setdefault without explicit default is confusing and useless.

So, Tim mercilesly refactored my code, of course to make it better :), thanks Tim.
As a result branch is now merged and we have setdefault method in ZODB3.5 final, so you can use it with Zope3 now (currently with 3.1 branch and soon with trunk) and write short, readable code, though, of course, someone may argue :)

More to come...

AdapterLookup C optimizations

rspivak — 2005-08-05

Recently i have created branch in Zope3 repository for checkin in AdapterLookup optimizations, ie rewriting it in C for speed up as many applications will be doing lots of lookup requests, so this part is quite crucial.
You can take a look at it here:
http://svn.zope.org/Zope3/branches/alienoid-adapter_lookup_coptimizations/

For the moment all methods(namely lookup, lookup1, adapter_hook, queryAdapter, subscriptions, subscribers, queryMultiAdapter) are implemented and pass tests, of course code is far from ideal, but i'm trying and waiting for
Jim Fulton to come back from holidays and take a look at my attempt to make world better or better to say faster :)

Here are benchmarks with 'timeit' i have on my P4 2.4Gh 1Gb RAM, run 3 times for Python and C versions. All developments and tests under linux of course :)

Python version:
==========
2.189   registry.lookup([IR1], IP2, '')
2.199   registry.lookup([IR2], IP2, '')
2.067   registry.lookup([zope.interface.implementedBy(C2)], IP2, '')
4.418   registry.lookup([IR1, IQ], IP2, '')
2.377   registry.lookup1(IR2, IP1)
3.528   registry.queryAdapter(x, IP1)
3.271   registry.adapter_hook(IP1, x)
6.128   registry.queryMultiAdapter((x, q), IM)
2.467   registry.subscriptions([IR1], IP2)
2.472   registry.subscriptions([IR2], IP1)
3.445   registry.subscriptions([IR1, IQ], IP2)
5.385   registry.subscribers((x, q), IM)
201.965163946 - total elapsed
--------------------------------------------------
2.194   registry.lookup([IR1], IP2, '')
2.197   registry.lookup([IR2], IP2, '')
2.071   registry.lookup([zope.interface.implementedBy(C2)], IP2, '')
4.389   registry.lookup([IR1, IQ], IP2, '')
2.414   registry.lookup1(IR2, IP1)
3.563   registry.queryAdapter(x, IP1)
3.295   registry.adapter_hook(IP1, x)
6.148   registry.queryMultiAdapter((x, q), IM)
2.475   registry.subscriptions([IR1], IP2)
2.467   registry.subscriptions([IR2], IP1)
3.440   registry.subscriptions([IR1, IQ], IP2)
5.375   registry.subscribers((x, q), IM)
201.607703924 - total elapsed
------------------------------------------------
2.230   registry.lookup([IR1], IP2, '')
2.220   registry.lookup([IR2], IP2, '')
2.091   registry.lookup([zope.interface.implementedBy(C2)], IP2, '')
4.453   registry.lookup([IR1, IQ], IP2, '')
2.400   registry.lookup1(IR2, IP1)
3.517   registry.queryAdapter(x, IP1)
3.300   registry.adapter_hook(IP1, x)
6.198   registry.queryMultiAdapter((x, q), IM)
2.505   registry.subscriptions([IR1], IP2)
2.490   registry.subscriptions([IR2], IP1)
3.464   registry.subscriptions([IR1, IQ], IP2)
5.404   registry.subscribers((x, q), IM)
202.900254011 - total elapsed

C Version
==========
1.655   registry.lookup([IR1], IP2, '')
1.658   registry.lookup([IR2], IP2, '')
1.706   registry.lookup([zope.interface.implementedBy(C2)], IP2, '')
3.078   registry.lookup([IR1, IQ], IP2, '')
1.798   registry.lookup1(IR2, IP1)
2.646   registry.queryAdapter(x, IP1)
2.542   registry.adapter_hook(IP1, x)
4.676   registry.queryMultiAdapter((x, q), IM)
1.792   registry.subscriptions([IR1], IP2)
1.785   registry.subscriptions([IR2], IP1)
2.777   registry.subscriptions([IR1, IQ], IP2)
3.948   registry.subscribers((x, q), IM)
152.926780939 - total elapsed
----------------------------------------------
1.673   registry.lookup([IR1], IP2, '')
1.674   registry.lookup([IR2], IP2, '')
1.683   registry.lookup([zope.interface.implementedBy(C2)], IP2, '')
3.101   registry.lookup([IR1, IQ], IP2, '')
1.778   registry.lookup1(IR2, IP1)
2.633   registry.queryAdapter(x, IP1)
2.522   registry.adapter_hook(IP1, x)
4.625   registry.queryMultiAdapter((x, q), IM)
1.808   registry.subscriptions([IR1], IP2)
1.799   registry.subscriptions([IR2], IP1)
2.786   registry.subscriptions([IR1, IQ], IP2)
3.861   registry.subscribers((x, q), IM)
151.658464193 - total elapsed
----------------------------------------------
1.690   registry.lookup([IR1], IP2, '')
1.690   registry.lookup([IR2], IP2, '')
1.709   registry.lookup([zope.interface.implementedBy(C2)], IP2, '')
3.114   registry.lookup([IR1, IQ], IP2, '')
1.806   registry.lookup1(IR2, IP1)
2.641   registry.queryAdapter(x, IP1)
2.567   registry.adapter_hook(IP1, x)
4.700   registry.queryMultiAdapter((x, q), IM)
1.821   registry.subscriptions([IR1], IP2)
1.821   registry.subscriptions([IR2], IP1)
2.812   registry.subscriptions([IR1, IQ], IP2)
3.953   registry.subscribers((x, q), IM)
153.785687923 - total elapsed

So, speedup by my estimation is ~20-26% for different methods.
It may vary in future but this is how it is currently at my side.

Benchmark script is attached.

tal:condition

rspivak — 2005-03-03

This is mainly reminder for myself.
Couple of times i was beaten by something like following code:

<h2
    tal:define="my_var1 here/title_or_id|nothing; my_var2 python: []"
    tal:condition="my_var2|my_var1">
Here goes text that won't be displayed because above condition is False,
though one can assume that '|' is equivalent to python's 'or' and my_var1
should be used(let's imagine my_var1 is true in this context) and it really would
be the case if my_var2 was path expression, and that path didn't exist or was
set to 'nothing'.
</h2>

To make it work, above could be changed to:
<h2
    tal:define="my_var1 here/title_or_id|nothing; my_var2 python: []"
    tal:condition="python:my_var2 or my_var1">
         TRUE
</h2>

or using at first place before '|' path expression.

nxml-mode + tabs

rspivak — 2005-02-24

Yesterday Anahide noted that my .pt files contain TABS, oh my.
After careful consideration, yep, i saw that they really contain tabs. I was confused because i was naive to believe that '(indent-tabs-mode nil) in my dot emacs will be working by default with nxml-mode. Explicit is better then implicit, so i began to explore how to force nxml-mode to use expliciltly spaces instead of tabs being in nxml mode. Thanks for Marc-Aur�le, he was kind to make suggestions about my case and i set hook in .emacs similar to one for xml mode:

(add-hook 'nxml-mode-hook (lambda()
(setq indent-tabs-mode nil)
))

This works just fine, but note that if you use desktop saving feature in emacs you need first to delete .emacs.desktop file otherwise above hook won't work with files that are saved by desktop, this how it worked for me.
All is left just to run C-x h M-x untabify RET on my *.pt files to remove tabs.

Well, lesson learned.

What is Trackback

rspivak — 2005-02-23

As you probably noted there is such feature on this site as Trackbacks. Here is small excerpt about trackbacks, more information could be found at http://www.movabletype.org/trackback/beginners/

In a nutshell, TrackBack was designed to provide a method of notification between websites: it is a method of person A saying to person B, "This is something you may be interested in." To do that, person A sends a TrackBack ping to person B.

TrackBack ping: a ping in this context means a small message sent from one webserver to another.

And why would person B be interested in what person A has to say?

Person A has written a post on his own weblog that comments on a post in Person B's weblog. This is a form of remote comments--rather than posting the comment directly on Person B's weblog, Person A posts it on his own weblog, then sends a TrackBack ping to notify Person B.
Person A has written a post on a topic that a group of people are interested in. This is a form of content aggregation--by sending a TrackBack ping to a central server, visitors can read all posts about that topic. For example, imagine a site which collects weblog posts about Justin Timberlake. Anyone interested in reading about JT could look at this site to keep updated on what other webloggers were saying about his new album, a photo shoot in a magazine, etc.

One more thought about using trackbacks instead of comments on blog are taken from http://ln.hixie.ch/?start=1033171507&count=1

"Personally I love finding that other people have commented about my post (every pingback, trackback and new referrer I receive automatically sends me an e-mail). I don't like having comments on my Web log because I believe if you want to comment then you should do so on your own Web log. If I'd wanted to host a discussion forum, I'd have installed discussion forum software, not Web logging software!"

Blog for Zope3

rspivak — 2005-02-14

I have begun to work in my "copious spare" time at Z3 version of blog. Hope CPSBlog and zblog will benefit from each other.
For those who is interested initial code is located at:
http://codespeak.net/svn/z3/zblog/