Meta Feed http://meta/exportrss Interesting energy this evening. The Baha'is were all off at feast, so we secular humanists (and a Pentacostal Christian) spent the first few hours discussing God, evolution, and dating. We actually spent the whole evening in my bedroom (which conver... http://blog.vrplumber.com/1099 mcfletch Mike Fletcher: A pleasant evening in with friends Over the years, there have been many, many failed attempts to create alternative VMs for Python, in the hopes of increasing program performance. Even if we ignore the many half-finished Python-to-Parrot translator projects still lurching erratically onward like a half-decayed zombie army, the road to better VM performance is lined on both sides by the gravestones of colorfully-named projects like Mamba, Rattlesnake, and Vyper, all lying untended and forgotten.> >Meanwhile, newer projects like pycore and ShedSkin are announced all the time, with a hopeful optimism all too similar to that of their predecessors. (Announced a little over a year ago with much fanfare in the blogosphere, pycore is already missing in action, without a single release yet.)> >Making Python run fast, it seems, is a lot harder than it looks. You don't have to be a compiler or VM design expert to look at CPython's implementation and say, "Doing X is wasteful. I'll bet you could make that faster by doing Y." The problem is that at least 9 times out of 10, somebody already tried doing Y, and got maybe 80% of Python to work with their design before they hit a wall.> >That wall basically amounts to this: 80% of Python is not Python, because everybody uses some part of that remaining 20%. The only reason ShedSkin isn't already in the land of the dead with all the other projects is that it neatly sidesteps this issue by not pretending to be anything but a "Python-like" language. However, that's sort of like the Black Knight in Monty Python and The Holy Grail, insisting that his lack of arms and legs is "only a flesh wound". True, in other words, but not very useful.> >On the other side we see the alternative VMs that actually implement the Python language, but don't (for the most part) try to outdo CPython for speed. Jython and IronPython actually implement reasonably complete forms of the Python language, but from a practical perspective they are different platforms. Trying to target an application to work across CPython, Jython, and IronPython would be rather pointless, so only pure-Python libraries are portable across the implementations in any case.> >But it's the impure libraries that give (C/J/Iron)Python most of its current value! Be it database access, number crunching, interfaces to GUI toolkits, or any of a thousand other uses, it's the C, Java, or CLR libraries that make Python useful. CPython is basically a glue language for assembling programs from C libraries, and to the extent that Jython and IronPython are successful, it's because they're glue languages for assembling Java or CLR components. What's more, since their value equation lies elsewhere, Jython and IronPython don't have to fully implement CPython's semantics, although they do try to come fairly close.> >And IronPython actually manages to improve on some Python performance microbenchmarks, although I'd say the jury is still out on whether IronPython programs perform better in general. Of course, it's difficult to measure this well because IronPython is a different platform. A heavy number-crunching program using NumPy isn't going to run on IronPython, for example, so how would you compare them?> >And that leads us to the very heart of the issue with CPython. If the value of CPython comes from all the things that work with it today, then CPython is very close to being at a dead-end for further performance improvement. Most proposed performance enhancements these days get rejected because they change the Python C API in backwards-incompatible ways. If a change requires that everybody rewrite their C code, the language might as well not be Python any more. In short, CPython isn't just a language implementation, it's a platform API, not unlike the Java VM and libraries.> >It used to be that we held out a hope for Python 3000 - Guido's bold vision of a Python rethought from the ground up, unburdened by the need for backward compatibility. Here we could break with the C API of the past, and explore new territory - or so we thought.> >But more recently, Guido has pulled back from the original plan, citing the ongoing vaporware status of Perl 6, and Joel Spolsky's arguments against rewriting your flagship product. Python 3000 has become Python 3.0, instead. Not a complete rewrite, but a still somewhat vague plan for tuning-up the existing language, and tossing out a few things Guido considers mistakes in retrospect. Backwards incompatibility will be allowed, but Guido has pronounced that there will be no from-scratch rewrite of the CPython implementation. It's not yet clear whether that means we can refactor in ways that would require third-party extensions to be rewritten. Perhaps this will be decided on a case-by-case basis.> >But arguably the single biggest mistake in the CPython platform as it exists today is the lack of a foreign function interface, defined by the language and expressable by Python code. Instead, CPython has always relied on a fixed C API to express foreign interfaces. For its original intended purpose - an embedded scripting language for the Amoeba OS - that was probably okay. But the lack of a C FFI has meant that tools like SWIG, Pyrex, ctypes, Boost::Python, etc. had to spring up to fill the gap, but none of them are "standard" to Python, so a given CPython extension could be written in any of them, or none of the above. Thus, today's backward-compatibility ball-and-chain: the Python/C API.> >What's more, few of these tools are designed to be independent of the existing CPython implementation. All but ctypes tend to have quirks that are a function of their intended code-generation target. But a Python language-defined FFI would have allowed the CPython API to be a mere implementation detail, able to be changed with little consequence. Indeed, such an FFI could conceivably have been usable even with Jython and IronPython, allowing even greater portability.> >But, it's too late to fix all that now. Or is it?> >Enter PyPy. Two months ago, PyPy 0.7 was released. A major milestone, PyPy 0.7 is the first self-hosting Python implementation. That is, an implementation of Python, written in Python, that can interpret itself. What's more, part of PyPy is a translation system that allows Python code to be translated to other languages, and it includes a kind of foreign function interface, although not a standardized one blessed by Guido. The PyPy developers have now done the work of rewriting all but a minimum of platform-specific C code as high-level Python code. In short, PyPy has already taken the most important step for us to escape from the CPython "gravity well" of needing a backward-compatible C API.> >It's hard to overstress how important this is. The current CPython implementation is locked into a host of design decisions that PyPy is not. As a simple example, PyPy can generate threads-supporting and non-threads-supporting versions of itself, refcounting and garbage collection versions of itself, and so on. Essentially, PyPy is completely virtual with respect to the underlying VM, even though it uses CPython bytecode. So, in the next few years it will be possible to experiment with radical redesigns of the VM, without getting bogged down in the "last 20%" issues experienced by projects of the past. Heck, it should be possible to use custom-tuned VMs on an application-by-application basis!> >Further, because PyPy is implemented in Python, hacking on it to change the actual Python language or its semantics will be easier than hacking CPython. In short, we are almost on the doorstep of a renaissance in the development of the Python language, and on the way out of the alternative-implementations graveyard.> >But what about speed? PyPy is currently described as 200-300 times slower than CPython, depending on what you're doing, and what VM you translate it to. This sounds ludicrously bad, until you look at the fact that the untranslated PyPy, running on top of CPython, runs 2000 times slower. Which means -- if you're paying attention -- that PyPy's translator is already able to turn Python code into C that runs 10 times faster!> >That is one heck of an improvement, folks. Granted, the code in question is technically "RPython" - a restricted subset of Python that eschews the use of certain more-dynamic features. But it doesn't need type declarations in order to get speed, like Pyrex does. And this technology could be available for practical use soon, if Stackless guru Christian Tismer has his way, by creating an RPython-to-CPython extension module translator.> >So, if it's possible to create efficient C from a subset of Python, does that now mean that PyPy is finished? Can't we just take that translation process and go on our way? Unfortunately, no. Although we could certainly take those fast modules back to the CPython platform, the translation process is still quite slow, and needs some accelerating of its own. Also, it still doesn't really make CPython any faster - it just means that we can compile some individual modules and make them faster.> >To reach the promised land, then, PyPy has to first get close to CPython speed. As it gets closer and closer to this goal, more and more people with an idea or two about speeding things up will say to themselves, "I wonder if I can get PyPy to do Y instead of X?" And, unlike the situation with CPython now, they won't need to be both a Python guru and a CPython VM expert to have a prayer of implementing it.> >So, instead of entirely new VM's springing up and dying incomplete, it may be that we will soon see the opposite trend: existing VMs fading away, consolidated and replaced by an ever-more flexible PyPy. With any luck, we may yet see PyPy become the One Python to Rule Them All, replacing CPython, Jython, and IronPython with C, Java, and C# translator backends respectively.> > Update: Just after I posted this, I found a message that appears to be saying that as of September, PyPy is now only 20 times slower than CPython. If that's the case, things are moving quickly indeed. 2000, 200, 20... How much longer till 2, and 0.2 (five times faster than CPython)? Unfortunately, each new order of magnitude from this point on will probably be more difficult than the last. Too bad they can't just feed the output back to the input and make it ten times faster as many times as they want. :) http://dirtsimple.org/2005/10/children-of-lesser-python.html PJE Dirt Simple: Children of a Lesser Python The last few days I've been setting up my wife's brand-new blog. For some reason, I went with Blogger as a technology. Because her site (like this one) is static HTML generated by XSLT on my laptop, I wanted to generate the Blogger template the same way. That way, the template would have the same look and feel as the rest of the site, and when the overall look changes, the template would change too. (more..) http://www.nedbatchelder.com/blog/200510.html#e20051015T181052 Ned Batchelder: Blogger via XSLT I insisted on doing the grocery shopping this afternoon, even though Rosey had volunteered (she was trying to get out of writing a cover letter). Nothing exciting, just getting a few odds and sods for having a few people over for coffee this evening.... http://blog.vrplumber.com/1098 mcfletch Mike Fletcher: Meeting on a bus aggdraw 1.1 final is 1.1b3 plus some minor fixes. Enjoy! http://online.effbot.org#aggdraw-11 online.effbot.org - Fredrik Lundh: aggdraw 1.1 final I'm surprised that this one hasn't made the rounds more. John Siracusa at Ars Technica has written a piece called Avoiding Copland 2010 (there's now a Part 2 and Part 3), where he claims that the lack of a memory managed language/API set will create a Copland style crisis for Apple in the next several years. Longhorn/Windows Vista is supposed to provide its new functionality via CLR/C# API's, so I imagine that Siracusa is looking at that and getting nervous. It has taken Microsoft longer than expected to pull this off, and depending on which rumors you believe, some of the more ambitious attempts to use managed code had to be scaled back in order to ship a product. If it is any consolation, the Linux folks have exactly the same problem that Apple does. Lots of people have commented on Siracusa's post, and a number of folks have pointed back to languages like Lisp, Smalltalk, and Dylan. Rainer Joswig, a long time Lisp hacker has written his opinion on what this means for the Lisp community. I found it amusing that around the same time that I saw Joswig's post, MIT announced that they were open sourcing the CADR Lisp Machine operating system, and the Gwydion Dylan developers announced the availability of OpenDylan 1.0b1. You can say what you like, but Lisp and Smalltalk machines were both systems where everything was build in a managed language. Some people think that automatic memory management only helps developers, but I think that you can get some improvements in reliability and security as well. Of course, as we've discovered with Java and C# code, it's also possible to create very inefficient code. Much has been written about the arrival of dual core processors, and rumor sites have produced screenshots of quad processor Intel Macs. I think that it's more likely that the needs of highly concurrent hardware will push Apple (and everybody else) to a Copland style breaking point. AnandTech has confirmed what many have suspected, that there are some fairly serious problems with threading in OS X. If applications need to become more concurrent in order to take advantage of new hardware, then that is going to be a bigger deal than memory management issues. We know that techniques like garbage collection can help to deal with the reliability and modularity problems associated with manual memory management. But we don't really have any good solutions for making concurrent programming less error prone. Maybe I'd better dig up all that Actor language stuff that initially led me into graduate school. There's one more wildcard, which is the whole question of whether or not desktop operating systems and applications are where the future of computing is going. In today's environment, you'd think that all innovation in computing is happening in the Web or Web 2.0 space, and that if we could just get a local storage mechanism and a decent drawing model into web browers, we could just swear off desktop computing all together. Given the number of times that I've had to kill a wedged Safari or Firefox, I'd have to say that I think that's a bit premature. Still, one never knows. In some ways, I'd be glad if Apple, or even better, the whole desktop computing world, had a Copland moment. Maybe that would give people a chance to take a deep breath and wonder whether what we have is the best that we can do. http://www.sauria.com/blog/2005/10/15#1397 Ted Leung (twl@sauria.com) Ted Leung on the Air: Copland 20xx? I came across this collaberation, text editing website which looks promising for future collaberation. But I don’t know anyone whom I want to collaberate with at the moment, so I was thinking … Let’s all try collaberating on this short screen shot script I posted a few days ago. Just ... http://www.blendedtechnologies.com/social-code-editing-experiment/40 Blended Technologies: Social Code Editing Experiment Instead of writing my OSDC paper last night, I watched Grave of the Fireflies. I picked it up a few days ago to expand my collection of Studio Ghibli films. It's about two Japanese children during WWII. It's a great movie, as I have come to expect from anything out of Studio Ghibli. The animation is fantastic, but you can see it come out in a different way in this one as opposed to the other greats like Princess Mononoke and Spirited Away; there's no magic or fantasy, so a lot of the mundane things get more attention. For example, the way a 4-year old girl cries about the death of loved ones. It's literally the saddest movie I've ever seen. Similar to JML's hesitance to recommend Requiem for a Dream, I hesitate to recommend it to many: not because it's full of foul imagery and themes (although it does have its share of morbid stuff), but simply because the story is incredibly depressing. I was pretty well incapacitated after watching it, so I didn't finish up my OSDC paper. Even remembering some of its scenes is still now bringing a lump to my throat. A good thing about its sadness is that it isn't purely about how some external force (e.g. drugs, war) causes strife for the protagonists, but also about their sometimes silly pride, bad decisions, or even bad timing. Oh well, I don't want to give anything away. I guess I think everyone in the world should see it, but just make sure you're expecting some soul-crushing despair. (This is not the saddest thing, but it is pretty sad) http://radix.twistedmatrix.com/archives/000118.html Tales of a Programming Hobo - Christopher Armstrong: Liebot, what is the saddest thing? After spending a couple of weeks with my newborn baby I have returned to work and have found a couple of engagements where I have to correct bugs and create new functionality for some pretty big Zope based applications.> >I have been working with Zope for years and I like it, but I am now used to what we could call Zope's best practices for web development, which include creating Python products, working exclusively on the file system, testing, using version control, keeping too much logic out of page templates, documenting the system and more (check the bottom of this post for links).> >The problem with these applications I'm working on is that they are old school: hundreds of Python scripts and page templates all stored in the ZODB. Lots of the scripts call each other freely and there are also a bunch of javascript generating scripts which sometimes include very important pieces of business logic in them, making debugging and following the flow of the application very difficult for people unfamiliar with the code (myself!).> >One of the systems is fairly well laid out and to be fair was developed at a time where information about Zope best practices and Python product development was scarce. The other one, however, seems to be an adaptation of some older system for similar but not altogether equal requirements and the scripts and templates are full of commented-out code and unused logic which make finding out what's going on even harder.> >At least they do not include DTML, which I hate, and the scripts and templates are all organized into folders, so it's not the worst case of Zope hell I have seen, but making even simple changes becomes very difficult in systems like these.> >After spending some time with these systems, I can see why some people loathe Zope and call it unpythonic (well, they call it other things too but they go against my self-imposed editorial policy). It doesn't have to be like this, though. Here are a couple of links for those who are interested:> /plone.org/Members/pupq/best-practices.pdf/view">/plone.org/Members/pupq/best-practices.pdf/view">/plone.org/Members/pupq/best-practices.pdf/view">/plone.org/Members/pupq/best-practices.pdf/view">/plone.org/Members/pupq/best-practices.pdf/view">Best practices for Plone development. A tutorial by Joel Burton. A theory on collaborative development using Zope. By Chris McDonough. Team development with Zope. By Kapil Thangavelu. Team development with Plone/Zope/ZEO/Subversion/iPython. By Michael Thornhill.> http://cguardia.blogspot.com/2005/10/visit-to-infamous-zope-hell.html Carlos de la Guardia Carlos de la Guardia: A visit to the infamous Zope hell Have now patched the nvidia-kernel drivers with two different x86-64 patches that are supposed to fix various problems, yet no change in the symptom. Takes a very long time for the change-test-reboot cycle on this. Oh well, for now I still have to ru... http://blog.vrplumber.com/1097 mcfletch Mike Fletcher: Another couple of hours down its gullet No, I don't know what's up with Termite. I like the idea of Scheme being an UNCOL. But what I really want is an Erlang environment up and running for me. So I decided not to wait for Termite to mature or to rewrite Termite using Gambit's new mailbox mechanism. We are now in the postmodern, dare I say, Web 2.0, era. I'm back to using Erlang after several years and loving it. It has so much good stuff packed in, from Mnesia to simple distibuted messages. It's not my favorite sequential language, but it seems like my favorite concurrent language. http://patricklogan.blogspot.com/2005/10/termite.html Patrick Logan Making It Stick (Patrick Logan): Termite There's always a danger, when you start into describing the operation of the human mind in order to describe how to get something done (e.g. design a building). It's tempting to reduce the entirety of human perception down to a simple metaphor or pro... http://blog.vrplumber.com/1096 mcfletch Mike Fletcher: The Path-and-Place metaphor I left a comment on James Robertson's blog re: his item on CORBA and WS. I agree with his point about port 80. (Although CORBA was addressing that just as the SOAP community accelerated.) Still, as far as port 80 and people working hard on the basic interop capabilities go... the result I think is still essentially that SOAP and WS-* interop is about at the level XML-RPC was five years ago. Is it really much beyond that today? The data passed via SOAP is arguably more expressive, or at least more widely recognizable as XML, than the XML-RPC syntax. Once you get off of port 80 though and stop using HTTP as a transport, then WS-* seems to quickly devolve into the same old proprietary vendor race to lock in customers. JBI is an attempt to avoid this but has little momentum that I know of. The ESB is still promoted as the key to deep success of integration within the enterprise. We had this five years ago in proprietary form. Today we are no further from the vendor lock-in that it takes to move in this direction. Outside the enterprise there's still no advantage over REST in sight. This is still interop-in-the-small... I think even CORBA was beyond this point five years ago. http://patricklogan.blogspot.com/2005/10/ws-interop-in-small.html Patrick Logan Making It Stick (Patrick Logan): WS-Interop-In-The-Small The first book I picked it up this afternoon sounded fascinating, it's approach sounding very similar to my own, and I figured maybe I had found a kindred spirit. I decided that, rather than read it at the library, I'd keep looking for other books to... http://blog.vrplumber.com/1095 mcfletch Mike Fletcher: Existence, Space and Architecture Jon Udell points out that the cathedral and the bazaar have much in common, perhaps most of all would be their respective ongoing evolution. They are equally bizarre. To the point about Sed and Awk... I am not sure the cathedral or today's bazaar is more coherent. I am not sure they need to be. We've been postmodern for some time and we should not expect that to change. Embracing the chaos is almost the goal itself. Innovation implies chaos. Sometimes the cathedral can be too limiting for innovation, or at least timely innovation. Look at the back and forth on schedules and dependencies among the new work coming out of Microsoft's dotnet and longhorn team. When was WinFS avalable again? What platforms? How does that line up with the LINQ query capability again? In the bazaar there is much less control over the chaos. I'm fine with that because generally people find or make ways to get from one place to another within the bazaar. Here's some postmodern glue I have been working with lately... I wanted to use Erlang with the Berekeley XML database as well as with the Clips rules engine. After a little consideration of my options for integration, for my purposes the easy answer was Python. Running SWIG for Python on Erlang's C-based erl_interface creates the glue that gets me into Python from Erlang (with all the desired Erlang node management capabilities). From there Python already has Berkeley XML DB integration and the PyClips interface to Clips. Plus I can send Python code from Erlang over the distributed process connection and have it executed dynamically in those Python "agents". On the front end, Erlang sends JavaScript, XML, JSON, and CSS to the browser. These combinations make Sed, Awk, and various shells look downright homogenous. Bizarre combinations gathered from the bazaar. But when you begin to piece together all the different MSFT components you get something equally as bizarre. http://patricklogan.blogspot.com/2005/10/cathedral-is-bizarre-too.html Patrick Logan Making It Stick (Patrick Logan): The Cathedral is the Bizarre (Too) Spent the day at the library reading again. Seem to have hit that point where continuing to read is not going to help. I need to start writing some things to get a better feel for where I need more material. I also spent some time reviewing my bachel... http://blog.vrplumber.com/1094 mcfletch Mike Fletcher: Quiet day reading New release of TurboGears with lots of goodies. See the TurboGears 0.8a1 Changelog.Also see the announcement by Kevin TurboGears 0.8 Released! and his status update: TurboGears going like gangbusters. The growth is impressive with more than 350 now on the mailing list. As Kevin has been going through the release of 0.8 the list has been actively testing the release. It seems to me that this is the biggest test of the whole python egg and setup tool set, there have been some minor issues but they seem to have been worked through. That is good news for this project and also for the whole Python community. Also progress was documented on the list for getting all this to work without root access which is good for hosting with TextDrive and the like. In other news more supporting tools are being created such as  MochiKit DOM helper script (panela.blog-city.com) documented at  HTMLtoMochiDOM HTML to MochiDOM script - Spike Developer Zone. Also Django is moving forward, this time with more documentation Design philosophies, see the weblog post at Design philosophies documented. http://42.blogs.warnock.me.uk/2005/10/keeping_up_with.html Dave David Warnock: Keeping up with TurboGears et al (quick link) http://biz.yahoo.com/prnews/051013/ukth010.html markpasc Mark Paschal: Habbo Islands for Nokia N-Gage TurboGears 0.8a1 is available now! What’s New This is a brief summary. The complete information about what’s new can be found here: http://www.turbogears.org/about/changelog.html API improvements based on feedback and patches from the first public release. Seven people contributed patches to TurboGears directly, and there’s been quite a bit going on in the other projects. Easier production of XML output from controller methods Static file directories are set up in new quickstarted projects Updates to all of the main included projects IPython is used as the shell, if IPython is available Bonjour support on the Mac New getting started guide, and command line tool and configuration references and a new site template by Sebastian Jansson. Several bug fixes http://www.blueskyonmars.com/2005/10/14/turbogears-08-released/ tazzzzz Blue Sky On Mars: TurboGears 0.8 Released! After the slashdotting, Ian Landsman wondered what the extended effect of the slashdotting would be. Certainly, after serving more than 13,000 views of the screencast on Monday, things have tapered off. The 20 Minute Wiki is still getting 1,000 views a day! (That’s 80GB per day, for those watching their waistline bandwidth usage). I haven’t had a chance to do a total tally, but I’m guessing that the screencast is up to 25,000 views. The main TurboGears discussion Google Group has added about 100 people since Monday, crossing 350 earlier today. The announcements list crossed 100 today. Mailing list traffic is increasing, and people are coming up with some cool stuff. Interesting things are taking shape for 0.9! Thanks to all of the people putting their energy into TurboGears! http://www.blueskyonmars.com/2005/10/14/turbogears-going-like-gangbusters/ tazzzzz Blue Sky On Mars: TurboGears going like gangbusters So, I've been trying out MochiKit. The stuff I've done so far seems very straightforward and also functional (as in lispy). Hopefully I'll have something to show for it soon. In the meantime, I've created a python script for converting well fo http://panela.blog-city.com/mochikit_dom_helper_script.htm Matt Harrison: MochiKit DOM helper script Python Parsers May I need to find a parser again, here's a compilation (no pun intended) of parsers. http://awkly.org/blog/python-parsers sidnei Sidnei da Silva: Python Parsers We've added a new piece of documentation, Design philosophies, that lays out the fundamental philosophies we've used in creating the Django framework. Its goal is to explain the past and guide the future. http://www.djangoproject.com/weblog/2005/oct/14/design_philosophies/ Django Weblog: Design philosophies documented Peter and Kevin will be in Amsterdam next week at the O'Reilly Open Source Convention, and are looking forward to a wide range of presentations on both technical and business tracks.> >They'll keep you updated from these pages, hopefully post a few photos as well.> >Are you going to EuroOSCON as well? Fancy meeting us there? Let us know and let's see if we can make a plan. http://logicalware.blogspot.com/2005/10/logicalware-at-eurooscon-2005.html Logicalware Team Open Sauce: Logicalware at EuroOSCON 2005 So we were having a ‘discussion’ on the work mailing list about a change I’d made to the UI code for MailManager. Given the following page template: <?xml version="1.0" encoding="utf-8"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en" i18n:attributes="lang language; xml:lang language" xmlns:tal="http://xml.zope.org/namespaces/tal" xmlns:metal="http://xml.zope.org/namespaces/metal" xmlns:i18n="http://xml.zope.org/namespaces/i18n"> <head> <title tal:content="template/title">The title</title> </head> <body> <p> <option tal:attributes="selected python:1==1">xxx</option> <option tal:attributes="selected python:1==0">yyy</option> </p> </body> </html> both Andy and Kev asserted that it would render to: <?xml version="1.0" encoding="utf-8"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"> <head> <title></title> </head> <body> <p> <option selected="selected">xxx</option> <option>yyy</option> </p> </body> </html> while I was asserting that it rendered to: <?xml version="1.0" encoding="utf-8"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"> <head> <title></title> </head> <body> <p> <option selected="True">xxx</option> <option selected="False">yyy</option> </p> </body> </html> (note the difference in the rendering of the ’selected’ attribute of the <option> tag.) I had distilled mine down from the MailManager code, as a test instance, whereas both Kev & Andy had created a fresh template in the ZMI. And both of us appeared to be right. We wondered if it was a difference in platform — I’m working on my laptop, with python 2.4 & Zope 2.7.7. But no. The difference is that I’m rendering the files as content-type text/xml (which Zope kindly defaults to if it encounters a file starting <?xml ... whereas they were forcing the content-type to text/html (the default if you create a template through the ZMI). If you force the content-type to text/xml in the ZMI you’ll see the same results as I get. So it turns out that exactly the same Zope Page Templates will render slightly differently, depending on whether or not they are marked as being HTML or XML. Cool. (I can understand why they are pulled apart by different parsers, since it’s so much nicer to use a proper XML parser if you’re expecting valid XML — and leave the grotty excuses for HTML that one sometimes finds to another lame, hacky thing — but to not mangle them into the same AST and use a unified generator at the other end?) When I’ve got more time, I should go file bugs. But I’ve got an alpha of MailManager 2.1 to release this afternoon, so I’d better get on with that or there’ll be no getting to go to the pub for me! tags: bugs, html, tal, xhtml, xml, zope page templates, zope http://woss.name/2005/10/14/zope-page-template-weirdness/ mathie Graeme Mathieson: Zope Page Template weirdness Daniel Krech, recently updated the Redfoot homepage with some additional documentation on what Redfoot is. It's a very interesting concept for leveraging Python (or any other scriptable language) and RDF as a distributed framework for applications. Beyond the known advantages of modelling distributed components on an RDF Graph with well defined semantics for how you retrieve programs and execute them it also relies on a hybrid of XML and Python called Kid to facilitate templating of HTML. The advantages of using a flexible programming language (such as Python) for manipulating XML is well written about (sift through the Copia archives, you'll find plenty). Couple that with a well modelled framework for including and executing remote modules as well as a programmatic access (using a similar idiom) to an underlying RDF Graph and you have yourself a very flexible foundation. For example. Below is the Kid template used to render the contributers page on rdflib.net: <div xmlns="http://www.w3.org/1999/xhtml" xmlns:kid="http://purl.org/kid/ns#"> <?python FOAF = redfoot.namespace("http://xmlns.com/foaf/0.1/") DOAP = redfoot.namespace("http://usefulinc.com/ns/doap#") project = URIRef("%s#" % request.host) people = [] seen = set() for property in [DOAP.maintainer, DOAP.developer, DOAP.documenter, DOAP.translator, DOAP.tester,DOAP.helper]: for person in redfoot.objects(project, property): if person not in seen: seen.add(person) label = redfoot.label(person) or person relationships = set() for relationship in redfoot.predicates(project, person): relationships.add(redfoot.label(relationship)) people.append((label, person, relationships)) people.sort() ?> <ul> <li kid:for="label, person, relationships in people"> <a href="${redfoot.value(person, FOAF.homepage)}">${label}</a>, ${redfoot.value(person, FOAF.nick)}, (${", ".join(relationships)}) </li> </ul> </div> Redfoot feels like a hybrid of Narval and the 4Suite repository and represents what is common between the tangential goals of those two projects. rdflib.net and redfoot.net (as well as some other sites) are examples of applications that run on a Redfoot instance. http://copia.ogbuji.net/blog/2005-10-14/Redfoot__U Uche and Chimezie Ogbuji: Redfoot: Updated Documentation Sorry for the boring title. I'll just keep you up to date on a few of my projects (which is probably more relevant than my last post). ... [145 words] http://www.voidspace.org.uk/python/weblog/arch_d7_2005_10_08.shtml#e114 Voidspace: Python Projects I've just made seven [1] new entries in my personal blog. I don't want to repeat them all here (even the relevant ones) - so I'll give you a summary : Are you my friend ? ... [111 words] http://www.voidspace.org.uk/python/weblog/arch_d7_2005_10_08.shtml#e113 Voidspace: Lots of Stuff My friend Justin has just released his mini-app X-Stream TV. It's basically a utility that connects you to lots of the free internet TV and radio channels out there. ... [99 words] http://www.voidspace.org.uk/python/weblog/arch_d7_2005_10_08.shtml#e112 Voidspace: Pretty X-Stream Generally recruiter spam is not a good thing (if you are employeed and like your job), but one looking for "Python/Open source" developer still sounds kind of cool! http://feeds.feedburner.com/PythonAndTheWeb?m=73 Max Khesin: python recruiter spam sighting It all started a couple days ago, when Ian Bicking posted about his attempt at using generic functions for a simple JSON-ification task.> >Then, Rene Dudfield posted comments to the effect that generic functions were a poor fit for the task, and slower to boot. He included a benchmark that was supposed to show that generic functions were 30 times slower than a hand-optimized version of the same operation, although the numbers he posted actually showed only a 23.4 times slowdown.> >Well, I didn't think the benchmark was a very good one, but what the heck. I tried it out for myself, made a couple of minor tweaks, and spent 30 minutes or so writing a C version of one part of RuleDispatch that I'd been meaning to get around to anyway, and got the benchmark down to only a 1.37 times slowdown - a mere 37% slower than the hand-tuned version.> >But since it's still not fair to compare a function you're supposed to extend by adding new methods, with a hand-tuned version that has all the methods it will ever have, I devised a slightly fairer benchmark. Since Rene proposed that monkeypatching - that is, replacing the original function with a new version - was a better way to implement extensibility, I added a couple of types to his version with monkeypatching, and added a couple of types to the generic function version as well.> >And then the worm turned: the generic function version was now 35% faster than the monkeypatched version of the hand-tuned function. I was a bit surprised by that, I thought it would've taken more layers of monkeypatching first. But no, just one extra layer in the typical case made it the same speed as the generic function, and two layers made it slower. (Presumably, additional layers would continue to degrade performance at a linear rate.)> >Now, before anybody gets the wrong idea, I don't promote monkeypatching in the general case, and in the specific case of a framework function like Ian's jsonify(), it would be crazy to recommend that people monkeypatch it. Monkeypatching as a recommended extension technique is little short of lunacy - it's trivial to accidentally break it or change the semantics due to a change in import order, you can't import the function normally (e.g from jsonify import jsonify), and as Rene's own benchmark shows, it doesn't even come close to being scalable from a performance perspective.> >But thanks to that benchmark, RuleDispatch users everywhere can now benefit from my speeding up of isinstance() checks, without making any changes to their code. Perhaps other people can now design and post other bogus benchmarks, so that I then can go ahead and spend a few minutes making those cases faster too. Ah, the wonders of blogging and open source. ;)> >Seriously, though, I do want to thank Rene for his comments, despite the fact that I think he's still quite thoroughly missing the point, which is that generic functions are for people creating extensible libraries and application platforms, not writing one-off scripts or applications. Nonetheless, if he hadn't taken the time to write his comments, I still wouldn't have gotten around to writing that bit of C code, and RuleDispatch wouldn't now be so much faster for isinstance() dispatching. And I wouldn't be feeling quite as smug right now, about how little monkeypatching overhead is required before RuleDispatch kicks some serious ass! http://dirtsimple.org/2005/10/ruledispatch-mojo-kicks.html PJE Dirt Simple: RuleDispatch Mojo Kicks Monkeypatching's Ass As I am about to take off for vacation, here is some things to watch:Since you've probably seen the 20-minute Wiki (from TurboGears) you should probably check out the 40-minute blog in Java, just to keep things in balance. (probably 15 mins in TurboGears ;)?
 If you are a search geek and want to see what GoogleGuy Sergei Brin looks like in person or just want to know if that guy who wrote the big fat AI book actually speaks human, here is a link for you.
 And finally, the third act is new Apple product pr0n. Itself in 3 acts.
 Should give you plenty to do unil I come back!
 :) http://feeds.feedburner.com/PythonAndTheWeb?m=72 Max Khesin: Geek pr0n update python-nicefloat is a Python module implementing an algorithm based on the paper "Printing Floating-Point Numbers Quickly and Accurately", by Robert G. Burger and R. Kent Dybvig.The implemented algorithm will find the shortest, correctly rounded output string representing a decimal number that converts to the same internal binary floating-point number when read.Update: the download link is now fixed.Have fun! http://www.livejournal.com/users/gniemeyer/12204.html Gustavo Niemeyer: Hey, nice float! Python Web Framework Niches an excellent post exploring some of the differences between Django and Turbogears in particular. Good points made, well worth reading! http://42.blogs.warnock.me.uk/2005/10/python_web_fram.html Dave David Warnock: Python Web Framework Niches As a (mostly ex-) Java programmer myself who prefers Python to Ruby, I'm puzzled by what seems like a rush of Java programmers to embrace Ruby as though it were the only dynamic language on the planet. I understand that it's mostly because of the success of Rails, which definitely came at the right time with the right marketing. But Ruby really doesn't seem like a good philosophical match with Java to me. Java, to a large degree, tried to be "C++ done right." That is, C++ without all the misfeatures that seemed good at the time but whose benefit turned out to not be worth the cost in complexity for developers. Java is a very orthogonally designed language; there is usually one obvious way to accomplish something. Python shares this. Ruby, OTOH, takes the C++ and Perl philosophy of "there's more than one way to do it," with predictible effects on maintainability. ("Perl," said a former co-worker to me yesterday, "is the drunken frat boy of languages.") I could point at other areas where Python seemed like a better fit to me, like real threading support, etc., but the core issue seems to be: what is the language's philosophy? Is it trying to help my team write maintainable, readable code, or is it more interested in being "clever?" Java and Python were designed for readability. C++ and Ruby were designed to be clever. About the only thing Ruby has going for it, from this Java developer's perspective, is braces. Ahh, comforting braces. Are people that afraid of syntactical change? I guess now I know how Lisp developers feel, a little. http://spyced.blogspot.com/2005/10/why-do-java-programmers-like-ruby.html Jonathan Ellis Spyced: Why do Java programmers like Ruby? I’ve come to the belief lately that the web frameworks available in Python are increasingly fine-tuned to specific application requirements. Of course, anyone reading the ‘About’ sections for these frameworks should realize this as well. I wonder how many people actually read that section as I’ve seen people latch onto web frameworks without knowing the task it was originally made for. Without knowing the reason the framework was created, its common for many people to leap to the conclusion that its another Rails wanna-be just because its a ‘full-stack’ web framework. I was playing around with a nice full-stack framework called WebObjects years ago which made it easy to setup database objects, generate CRUD, etc. Zope’s been doing the same stuff for what now seems like eons as well, yet I don’t see people declaring RoR a Zope clone (It obviously isn’t). In light of that, I’m inclined to agree with Ian Bicking’s response about the lessons Python web people did learn from RoR. The concept I want to focus on is that people create these new frameworks because they make their task easier than any of the other frameworks already out there. While they might pick up features from other frameworks, most of them aren’t aspiring to be “Python on Rails”. Sometimes this task is easier when other tools can be integrated to avoid code replication, as is the case in one framework I cover here. Many people have declared the amount of Python web frameworks a “problem” that should be “solved” somehow, perhaps a Highlander fight with swords to the death (There can be only one!). I’d like to suggest the opposite, there’s a lot of Python programmers and I think there’s room for even more web frameworks. The variety is a strength because they make it easier to get specific web applications done. TurboGears The TurboGears site has a nice about page describing its purpose, though I feel it doesn’t completely explain the rationale for its creation. There’s some interesting and unique decisions made in TurboGears, like using Kid instead of Cheetah or Myghty for templating. Then there’s the inclusion of Mochikit and the TurboGears decorators for returning output as JSON for use with Mochikit. So what kind of applications is this web framework geared for? (Please excuse the pun) The best way to answer this is to look at the application this framework was created for, Zesty News, and the abilities of some of the tools being used. Zesty News is in a rather interesting category of web applications in that the end-users themselves will be installing it, quite likely on their home computer rather than a server. Being able to package it up and easily distribute/upgrade it becomes a key issue along with database portability and code thats database agnostic. Two tools assist here, setuptools for distribution/packaging and SQLObject for portable database code. Zesty News deals extensively with RSS and XML, so it makes sense that the templating language chosen was actually created for dealing with web services. These design decisions behind TurboGears should make it fairly obvious when to consider it for your next project. The cohesive toolset you get when you choose TurboGears is ideal for developing portable, easily deployable AJAX-enabled web applications that likely deal with XML frequently and need to stay database agnostic. Even if your web application doesn’t deal with XML frequently, the decisions TurboGears makes for AJAX integration will make it easy to add heavy dynamic interaction to a TG webapp. Django Django was created to deal with the requirements of working in the web development department of a news publisher. As such, the framework was created specifically to deal with the requirements placed upon the author. What’s rather interesting is the lack of re-use in Django when it comes to doing things that have been done before in other projects (Database mapper, form validation, etc). The tools and parts of Django were specifically built to work as one package, and using Django makes that very obvious. One of the things most common when in a newsroom or publishing environment is dealing with CRUD. That is, there is a lot of content and ways to get content into the system and administrate the content is a high priority. As a web framework built for dealing with Content, many of the design decisions reflect the common tasks present in CMS’s (Content Management System). To start with, you get a slick administration interface for your conntent, that’s miles beyond any of the generated CRUD type stuff in other web frameworks. This differs from the philosophy of other web frameworks that give you basic CRUD (Scaffolding in Rails-speak) in that Django’s admin interface is aimed directly at being production-ready with no modification at all. Django also makes it fairly easy to make a Django ‘application’ like a Forum or Blog, then slot it into other Django application environments. Again, this makes a lot of sense given the original requirements placed on the creator of Django. If a company has 4 websites, and wants them all to have the new Forum/Classified ability it makes a lot of sense for this task to be optimized. So what web applications are you going to want to use Django for? Quite a few, as it turns out dealing with Content is a very common task. If you’re writing a web application heavy on content, that needs a full featured web interface for managing the content it’d be really hard not to recommend Django. It’s easy to get started, and in almost no time you have very powerful functionality running that gives you a lot of usability. Don’t be Everything for Everyone Part of the reason I picked these two projects to talk about is that they’re both extracted from a working project (as Rails was). I also haven’t seen many people mention the fact that frameworks developed in such a manner are also inherently going to be optimized for the use-cases that brought them into existence. Open-sourcing the project lets them grow to an extent, but their design is largely baked in and a useful limitation. Too much expansion past the initial design requirement will make them generic, and with that comes a lot of complexity (sometimes worth it though for the extra re-usability). Note that the specific things Django gives you don’t help that much if you’re trying to write a Zesty News style application. The same goes in reverse as well, since building an Admin interface of your own isn’t fun and can be time consuming. While it’s possible to make web applications that do this in either framework, compensating for framework design will require extra time when you try to use one framework for everything. If you’re using one framework for everything, maybe its time to take a look around. http://www.groovie.org/articles/2005/10/13/python-web-framework-niches Groovie: Python Web Framework Niches ? http://agiletesting.blogspot.com/2005/10/article-on-selenium-in-october-issue.html Grig Gheorghiu Grig Gheorghiu: Article on Selenium in October issue of "Better Software" ? http://agiletesting.blogspot.com/2005/10/cheesecake-how-tasty-is-your-code.html Grig Gheorghiu Grig Gheorghiu: Cheesecake: how tasty is your code? Micah put my del.icio.us daily links tool to immediate use on his blog. He uncovered a bug in the character handling, which is now fixed in the posted amara_delicious.py file. I usually invoke the script from cron, but Micah asked if there was an alternative. I've been meaning to hack up a poor man's cron for PyBlosxom and this gave me an additional push. The result is task_control.py. A sort of poor man's cron for PyBlosxom, this plug-in allows you to specify tasks (as Python scripts) to be run only at certain intervals Each time the plug-in is invoked it checks a set of tasks and the last time they were run. It runs only those that have not been run within the specified interval. To run the Amara del.icio.us daily links script once a day, you would add the following to your config file: py["tasks"] = {"/usr/local/bin/amara_delicious.py": 24*60*60} py["task_control_file"] = py['datadir'] + "/task_control.dat" You could of course have multiple script/interval mappings in the "tasks" dict. The scripts are run with variables request and config set, so, for example, if running from task_control.py, you could change the line of amara_delicious.py from BASEDIR = '/srv/www/ogbuji.net/copia/pyblosxom/datadir' to BASEDIR = config['datadir'] http://copia.ogbuji.net/blog/2005-10-13/More_on_th Uche and Chimezie Ogbuji: More on the PyBlosxom del.icio.us plug-in, and introducing task_control.py, a a pseudo cron plug-in for PyBlosxom Hot on the heels of some interesting developments in terms of the adoption of Open Document Format comes intriguing stuff surrounding $100 laptops for kids. The article says that the state of Massachusetts has committed to providing every shoolkid with a http://seanmcgrath.blogspot.com/archives/2005_10_09_seanmcgrath_archive.html#112921895943014427 Sean Sean McGrath: Its all happening these days in Massachusetts If you wonder why it has been so long between 1.0 and 1.3 of the CalCore/CalZope/CPSSharedCalendar trio, then the answer is not only vacation, and a whole bunch of big new features, but also that every time I have been close to releasing a new version, I have first released it to some of our trusty customers for testing. And they have promptly found bugs, which I then fixed, and made a new release, and so on. After the release of 1.0, several additional customers have installed the new calendar, and we also have people using it outside of CPS altogether. At this point I dare to say that the new calendar is both prettier, faster and more stable than the old CPSCalendar. The type of bugs that has been popping up lately are mostly translation and usability problems, where the calendar doesn't behave exactly as you would expect it to. An upgrade is warmly recommended. The major new features in 1.3 are: Major speed increases for big installations. Event coloring according to type. Calendar "stacking": You can create calendars which display the events of several calendars as if they were one. Migration from the older CPSCalendar. Support for Zope 2.7 and 2.8, CMF 1.4 and 1.5, CPS 3.2 and 3.3. Localization of date format. Of course, there are also many bugfixes, most having to do with translations, something that always takes a long time. Separate releases per target There are now several separate releases. This is mostly because the install instructions just got way to complicated, and it's also an effect of the desire to release a non-zope release of the CalCore library that does all the heavy calendar lifting, so the rest of teh Python world can use it. CalCore requires some libraries from Zope 3,  so I made a CalCore-bundle release that includes these packages, and which is easily installable into any non-zope python installation with the standard distutils. There is also a CalZope bundle that includes all you need to install it under Zope or CMF (except for Five and Zope 3), which you need if you run Zope 2.7. And of course, we also have the CPSSharedCalendar bundle, which includes all the products you need to install the calendar on CPS (again not including Five and Zope 3). You can find the latest version of these bundles on cps-project.com: http://www.cps-project.org/static/misc/ http://blogs.nuxeo.com/sections/blogs/lennart_regebro/2005_10_13_1-3-calendar-released lregebro Nuxeo: 1.3 of the Calendar released! so this is where the time machine is these days... http://online.effbot.org#pinter online.effbot.org - Fredrik Lundh: observations Since the chance of getting any significant amount of work done before needing to leave for the meeting seems to be pretty close to nil, I've been working on the video-card problem. I rebuilt with the ~amd64 nvidia-glx and nvidia-kernel stuff from so... http://blog.vrplumber.com/1093 mcfletch Mike Fletcher: Fumbling towards KDE Every skyscrapered city should have some of these. It would add an extra dimension to gazing out of a hotel window in downtown Manhattan or Chicago. Heck what am I thinking. Las Vegas is the place for this sort of thing. Would you bet on two pixels climbi http://seanmcgrath.blogspot.com/archives/2005_10_09_seanmcgrath_archive.html#112919661954976223 Sean Sean McGrath: Skyscrapers = rectangular blocks of windows = pixel display unit Burnt off some of the excess energy last night trying to get the new video card working (there were some posts to the gentoo amd64 list that suggested the problem might be solved by updating to the ~amd64 drivers). No such luck. Still, got the deskto... http://blog.vrplumber.com/1092 mcfletch Mike Fletcher: Limping nVidia card Philipp von Weitershausen, the author of"Web Component Development with Zope 3" has given a list ofentry level articles here:http://www.worldcookery.com/AppetizersMy article is too listed. http://www.artima.com/forums/flat.jsp?forum=122&thread=132129 Zope3 articles collection by philipp von New release of TurboGears with lots of goodies. See the TurboGears 0.8a1 Changelog.... http://www.artima.com/forums/flat.jsp?forum=122&thread=132087 Keeping up with TurboGears et al Python Parsers May I need to find a parser again, here's a compilation (no pun intended) of parsers. http://www.artima.com/forums/flat.jsp?forum=122&thread=132071 Python Parsers Python Parsers May I need to find a parser again, here's a compilation (no pun intended) of parsers. http://www.artima.com/forums/flat.jsp?forum=122&thread=132063 Python Parsers Python and XUL: The Screenshot Mark Hammond just shown me a screenshot of Python and XUL. That's right, Python used as scripting language on our most beloved browser to create XUL interfaces. I am so excited about it that I couldn't keep it to myself and thought it would be nice to share with you. http://www.artima.com/forums/flat.jsp?forum=122&thread=132049 Python and XUL: The Screenshot Python Parsers May I need to find a parser again, here's a compilation (no pun intended) of parsers. http://www.artima.com/forums/flat.jsp?forum=122&thread=132039 Python Parsers Python and XUL: The Screenshot Mark Hammond just shown me a screenshot of Python and XUL. That's right, Python used as scripting language on our most beloved browser to create XUL interfaces. I am so excited about it that I couldn't keep it to myself and thought it would be nice to share with you. http://www.artima.com/forums/flat.jsp?forum=122&thread=132032 Python and XUL: The Screenshot Zope.org: Permanent Failure? Rumor has it that some brave folks are trying to come up with a zope3.org website, so that Zope 3 related information is found separate from the current zope.org website. I have lost track of how many different attempts have happened to restore sanity at the zope.org website. This one will be the second try at a... http://www.artima.com/forums/flat.jsp?forum=122&thread=132022 Zope.org: Permanent Failure? Python and XUL: The Screenshot Mark Hammond just shown me a screenshot of Python and XUL. That's right, Python used as scripting language on our most beloved browser to create XUL interfaces. I am so excited about it that I couldn't keep it to myself and thought it would be nice to share with you. http://www.artima.com/forums/flat.jsp?forum=122&thread=132012 Python and XUL: The Screenshot Zope.org: Permanent Failure? Rumor has it that some brave folks are trying to come up with a zope3.org website, so that Zope 3 related information is found separate from the current zope.org website. I have lost track of how many different attempts have happened to restore sanity at the zope.org website. This one will be the second try at a... http://www.artima.com/forums/flat.jsp?forum=122&thread=132002 Zope.org: Permanent Failure? memcached Cache Manager for Zope 2 Made some good progress on my memcached Cache Manager for Zope 2. It is available from the CacheFu project in the collective. While staring at the code for the memcached python wrapper, I wondered if there was anything that could be done to make it faster, but sadly anything that I tried just made it slower... http://www.artima.com/forums/flat.jsp?forum=122&thread=131992 memcached Cache Manager for Zope 2 Generally recruiter spam is not a good thing (if you are employeed and like your job), but one looking for "Python/Open source" developer still sounds kind of cool! http://www.artima.com/forums/flat.jsp?forum=122&thread=131910 python recruiter spam sighting As I am about to take off for vacation, here is some things to watch:Since you've probably seen the 20-minute Wiki (from TurboGears) you should probably check out the 40-minute blog in Java, just to keep things in balance. (probably 15 mins in TurboGears ;)?
 If you are a search geek and want to see what GoogleGuy Sergei Brin looks like in... http://www.artima.com/forums/flat.jsp?forum=122&thread=131884 Geek pr0n update Python Web Framework Niches an excellent post exploring some of the differences between Django and Turbogears in particular. Good points made, well worth reading!... http://www.artima.com/forums/flat.jsp?forum=122&thread=131870 Python Web Framework Niches I've been thinking about events for the 0.8 series of SQLObject. I'm thinking about using PyDispatcher, but there's still some open questions. Jim Fulton actually got me thinking about events a while ago, in lieu of "hooks"... PyCon 2003, I guess. He made a presentation at PyCon 2005 too, but it didn't resonate in the same way, it just seemed... http://www.artima.com/forums/flat.jsp?forum=122&thread=131634 Events and SQLObject You are in a maze of twisty little config files, all alike I'm currently preparing to create a non-Zope bundle of CalCore and all it's dependencies. Since these dependencies are mostly Zope 3 packages (interface and schema) it seems natural to use Zope Corps tool for this: zpkg. Unfortunately zpkg is a complex tool with cryptic documentation. I have made packages with zpkg before, and it was hard. This time around, it was equally hard, because I had forgotten everything. I'm sorry to say this, but when it's hard to do something, and hard to remember how you did it, this is a sign of a bad design. Configuring a product to use zpkg is far from easy or logical, and there are many confusingly similar files, all named SOMETHING.cfg. Getting started First you need to check it out:   svn co svn://svn.zope.org/repos/main/zpkgtools/tags/zpkg-1.0.0 zpkgtools or, for the more adventurous:   svn co svn://svn.zope.org/repos/main/zpkgtools/trunk zpkgtools The main application is in bin/zpkg. It takes many parameters, and after a while I found that the best thing to do is to create a configuration file, and call that with -C. I called mine BUILD.cfg. Zope corp likes to call them the name of the package, such as Zope.cfg, but for me that is confusing, it sounds way to similar to the zope.conf file. The format of the configuration files the same format as the zope.conf file, that is you use # for comments, and write keyword value pairs, and create groups with <tag> </tag>. The first keyword I needed was:     collect-dependencies  yes This tells zpkg to include the dependencies of the product that I'm packaging and include them in the package. That's what the whole thing is about, if you don't need that you don't need zpkg at all. Also, I'd like to not have to write the name of the resource I'm packaging every time I'm packaging, so I include the line     default-collection calcore However, this will create a package named calcore-x.x.x.tgz. This is confusingly similar to the CalCore-x.x.x.tgz packages created by Nuxeos packaging tool, which includes CalCore only and not dependencies, so I also add     release-name CalCore-bundle to make the file be called CalCore-bundle-x.x.x.tgz instead. Now I can run zpkg -C BUILD.cfg, and it will try to build the release. However, it tries to package the calcore resource by default, and has absolutely no idea how to do that, so you get an error. Defining resources Resources are defined in a resource map. This is a list of resource names, and a location where you can find them. You can either put the list of resources into the configuration file directly like this:   <resources>     calcore src/calcore   </resources> or you can put it in a separate file, lets call it RESOURCES.cfg, like this:     calcore src/calcore and just include it in the main file with     resource-map RESOURCES.cfg The resource definition for calcore points to a subdirectory in the current hierarchy. In most cases you have the product directly in the same directory as the configuration file you are now creating, and then you would define it up with a dot:     calcore . Including the dependencies At this point, you can now actually make a package. It will however not include the dependencies, which of course is the whole point of this excercise. For this, we need to tell zpkg which dependencies the resource has. This is done with a DEPENDENCIES.cfg file, which simply lists the resources that this resource needs. The calcore resource was defined up as being in src/calcore, so that's where the DEPENDENCIES.cfg needs to be. It looks like this:   zope.interface   zope.schema   zope.i18nmessageid   iCalendar Of course, zpkg has absolutely no idea where to find these resources, so we have to extend the RESOURCES.cfg file:   zope.interface svn://svn.zope.org/repos/main/Zope3/tags/ZopeX3-3.0.0/src/zope/interface   zope.schema svn://svn.zope.org/repos/main/Zope3/tags/ZopeX3-3.0.0/src/zope/schema   zope.testing svn://svn.zope.org/repos/main/Zope3/tags/ZopeX3-3.0.0/src/zope/testing   zope.i18nmessageid svn://svn.zope.org/repos/main/Zope3/tags/ZopeX3-3.0.0/src/zope/i18nmessageid   iCalendar svn:http://codespeak.net/svn/iCalendar/tag/iCalendar-0.10 Note here that these resources are defined up as svn tags! zpkg will fetch the files directly from the svn of each product. Very practical. You can also have a * instead of the tag-name, in which case it will use some kind of logic to figure out which tag to use (what logic I don't know). Also note that I included zope.testing above. That's because several of the resources above define up their own DEPENDENCIES.cfg, and of course, those dependencies must be defined as well, if they are to be included. zope.testing is really not needed unless you want to run the unittests, but I include it anyway for good measure. For some reason zpkg will complain that the resource "zope" is not defined and skip it. That's a good thing, because it's would mean that all of Zope3 was included, which is quite silly. Why it wants to include it is beyond me, none of the resources have it in their DEPENDENCIES.cfg file. Adding files to the distribution root I want to have some files in the distribution root. A README.txt that tells you what the package is and how to install it should be in every package. I also want the HISTORY file to be included (but named HISTORY.txt), the GPL license, with the name COPYING.txt and the whole doc directory. To add them to the root I need to create a PACKAGE.cfg in the resource directory, that includes the follwing lines: <load>   HISTORY.txt svn:http://svn.nuxeo.org/pub/CalCore/tags/*/HISTORY   COPYING.txt svn:http://svn.nuxeo.org/pub/CalCore/tags/*/COPYING.txt   README.txt  svn:http://svn.nuxeo.org/pub/CalCore/tags/*/README.txt   doc         svn:http://svn.nuxeo.org/pub/CalCore/tags/*/doc </load> <distribution>   HISTORY.txt   COPYING.txt   README.txt   doc </distribution> The first part adds the files into the resource directory. It isn't possible in zpkg to include files that are above the resource root, so I instead load them from svn. This can also be used to have the latest version of the licensing in a separate package and thereby automatically include the correct version every time you package, and other cool things. The distribution tag also tells zpkg to include these files in the distribution root. (They will end up both there and in the resource directory). Nice extras The PUBLICATION.cfg is not needed, but it's a place to put in meta data about the product, which is a good thing:   Name: calcore   Summary: Python calendaring   Home-page: http://www.nuxeo.com/   Author: Martijn Faassen, Lennart Regebro, Nuxeo SARL.   Author-email: lregebro@nuxeo.com   Licence: GPL 2   Description: A python package for making personal and group calendars. Note that PUBLICATION.cfg uses an RFC-822 type format, that is, it uses keyword-value pairs with a colon after the keyword, while all the other config files does not have a colon. Thats it! There is a lot of cfg files involved and it can be very confusing, and I'm in no way near understanding all the details. But at least, after some trial and error, I succeeded in making a tgz that includes what I want. http://blogs.nuxeo.com/sections/blogs/lennart_regebro/2005_10_12_using-zpkg lregebro zpkg tool: A quick intro Jeff Shell has just posted a reminder for several introductory documents and tutorials about Zope 3. Nice thing about it is that he even mentions z3lab.org, a project dear to us at Nuxeo: This site has blogs, documentation, proposals, movies, prototypes, and more, for building a high class content management platform on top of the Zope 3 framework. There is a lot of information and ideas floating around here, and the animations are very impressive. http://blogs.nuxeo.com/sections/blogs/fermigier/2005_10_11_some-zope-3-quick-starts sfermigier "Some Zope 3 Quick Starts and Resources" I went to the Neckar Sprint last week in Tubingen, Germany. A lot of cool things were done/started there for Zope 3: Twisted integration Http publisher puggable sub-protocol Viewlet/portlet framework A static version of apidoc Some Zope3.org work Customizable container views, from the proposal here XML-RPC introspection integration, from the proposal here I guess I will decorate these points with some links when people that have worke on each tasks start to publish about it. The wiki page is definitly the place where all info will be added i guess. I have worked on two tasks with Andreas Jung, the puggable publisher and the introspection stuff. Http publisher pluggable sub-protocol The main idea of this task was to change the publisher so anyone who wants to add a new publication class (to handle json for example) can do it smoothly by adding some configuration. You needed to modify the publisher code to do so. So we did a change based on a study of the incoming request, deciding  which publication was best fitting, with registered utilities that would link a publication class to a couple of interfaces that represented: The mimetype of the request The method So we had our list of Interfaces a developer could gather to provide a publication configuration. This was done here. I spent a part of the evening finishing up the work, as Andreas went home. Friday morning when he came back he came up with a better idea: "why don't we just create a new directive that would let the user configure the publication picker, without having to add all those IGet, IPut, etc.. interfaces" Yup, brilliant idea ! So we refactored everything to let the user extend the publisher by simply giving a zcml "publisher" directives and a publication class. Look how the default publication is, it is very intuitive to extend this way.   XML-RPC Introspection The introspection stuff was very easy to code, based on some code greping from zope.app.apidoc. So we implemented it here. The fact that Python does not (yet) have static typing (i am saying yet because I've seen some proposal for Python 3000) was a problem for the methodSignature API, so after a discussion with the others, we came up with the conclusion that a decorator would be the best pick for this static typing. All is explained here. Performance regression tester On the last day, we had a very interesting discussion with Stephan Richter about a performance regression tester. I had a first shot on this a while ago here. He came up with the conclusion that this would be perfect if it was based on Pystone so it can work the same for all boxes. Simple and Brilliant. :) It's still visible here, but I have reverted this commit, as we need to discuss more on this and it relies on deep structural changes, so it is more likely to become a proposal. Conclusions I had a great experience there, met a lot of incredible people. I came back home with this conclusion in mind:  Zope 3 is absolutely terrific. I could literaly feel the power of it. You can do anything you want with it, as long as the design is well thaught. But most important point: I had terrific chocolate pretzels at the Hotel, need to try to find some in Paris. http://blogs.nuxeo.com/sections/blogs/tarek_ziade/2005_10_11_neckar-zope-3-sprint tziade Neckar Zope 3 sprint - summary During my Summer I was selected in the Google Summer of Code to produce a Eclipse plugin that will help OpenOffice.org developpers in their task. This project is very wide and could be expanded with a lot of new features. The initial goal is to provide a tool for new OOo developpers, with code generation, UNOIDL syntax highlighting, OpenOffice.org or URE and SDK configuration and of course many creation wizards. At this point, there are a few basic features added, however there is still many work to do. Where to find it ? You can find the sources on the OpenOffice.org api project CVS, module ooeclipseintegration. I am trying to build an update site on my own website... but it is still under construction. The plugin archive can be found in the build directory. Let me know if your feelings about this piece of work, but do never forget that it has to be continued. http://blogs.nuxeo.com/sections/blogs/cedric_bosdonnat/2005_10_10_very-nice-summer-for cbosdonnat A very nice summer for OpenOffice.org How to create a new URE based application ? The documentation is quite hard to find, but the main important thing is: create a UNO component implementing the com.sun.star.lang.XMain interface. This one will contain a method run() I'll explain how to create a "Hello URE World" application. The project will be organized in the following way: idl diretory containing the application UNOIDL specifications source directory containing the implementation of the UNOIDL types program directory which will contain the application registries and libraries MANIFEST.MF file which will describe the registration class of the component Makefile file to make the build much easier unotest file to launch the newly created application UNOIDL file definition The file idl/org/unotest/Main.idl will describe a simple service org.unotest.Main exporting the XMain interface this gives the following code: module org { module unotest {     service Main: com::sun::star::lang::XMain {       create ();     } }; }; Note that this service is a new style one: defining constructors and exporting a single interface. Java implementation Now just create a org.unotest.comp.MyMain Java class in the source/org/unotest/comp/MyMain.java file. This class will implement the following three methods:    public int run (String[] arguments) {       // Printing Hello URE World ;) and exit       System.out.println("Hello URE World ;)");       return 0;    }   __getServiceFactory(...) and __writeRegistryServiceInfo(...) are used for the component registration. They implementation is fully described in the chapters 4.5.3 and 4.5.4 of the OpenOffice.org Developer Guide Generation of the component The component generation works like the OpenOffice.org components generation, however there are some tricky things to know: use the URE regmerge and regcomp tools. Otherwise you will get some Loader exceptions when trying to use regcomp on your component. do not put your class files directly in your current directory: this is a new issue that will be corrected. If you do not respect this advice, the regcomp tool will use the class files in your current directory before the one in your jar file. In our unotest case, the build instructions are written in the Makefile. It should use the following tools: idlc -Ourd -I$(SDK_HOME}/idl idl/org/unotest/Main.idl $(URE_HOME)/bin/regmerge program/types.rdb UCR urd/Main.urd javamaker -Torg.unotest.Main -BUCR program/types.rdb -X$(URE_HOME)/share/misc/services.rdb -X$(URE_HOME)/share/misc/types.rdb javac -d . -cp $(CLASSPATH) source/org/unotest/comp/MyMain.java jar cfm program/unotest.uno.jar MANIFEST.MF org/unotest/compMyMain.class org/unotest/Main.class rm -Rf org $(URE_HOME)/bin/regcomp -register -br $(URE_HOME)/share/misc/types.rdb -br $(URE_HOME)/share/misc/services.rdb -r program/services.rdb -c file://$(PWD)/program/unotest.uno.jar -classpath $(CLASSPATH) What will these commands do ? Compile your UNOIDL specification to create the urd/Main.urd registry Merge the generated urd file in the program/types.rdb registry Create the org/unotest/Main.class file Compile your java implementation of the service Create the jar file of the component Delete the unused class files to avoid problems with regcomp Register the component in the URE Launching the application In the unotest file, you just have to put a call to the uno tool contained in the URE: $URE_HOME/bin/uno -c org.unotest.comp.MyMain \                                  -l file://`pwd`/program/unotest.uno.jar \                                  -ro program/services.rdb \                                  -ro program/types.rdb \                                  -ro $URE_HOME/share/misc/services.rdb \                                  -ro $URE_HOME/share/misc/types.rdb Thus when launching the unotest file, you will launch the URE with you org.unotest.comp.MyMain as the main class to load at the beginning. This will print you the expected "Hello URE World ;)" The next step is to do the same with a C++ component... but I'm still working on it. http://blogs.nuxeo.com/sections/blogs/cedric_bosdonnat/2005_10_10_my-first-ure-component cbosdonnat My first URE Component in Java OOoCon at Koper was great Nice meeting, professional level media coverage (with free tools) Interressant discussions around Addons and Scripting. This subject was deeply covered by numerous presentations ( Paolo Montavani, Ian Laurenson, Mathias Bauer ...). We see OOo2 is coming : it's scripting capabilities will boost OOo development Following my presentation, it is now time to work :) The slides have been transmitted to the organization team and will be soon available at http://marketing.openoffice.org/ooocon2005/ The video is also available, showing interressant questions and start of discussions I posted a first mail trying to start the discussion You can join the dev@scripting.openoffice.org mailing list that will host us before the creation of the new incubator project Kazunari Hirano, our great japanese blogger already relayed this announcement to the japanese community As requested, i post my slides here ... And now, lets go building the next OOo development booster http://blogs.nuxeo.com/sections/blogs/laurent_godard/2005_10_06_ooocon2005-slides lgodard OOoCon2005 - Slides I see  many questions on the differences between Zope 2 and Zope 3 on mailing lists. Here is a short attempt to answer some of the most common ones. Should I use Zope 2 or Zope 3? Short answer: If you need a product that runs on Zope 2, like for example CPS, Plone or Silva, you should use Zope 2. Otherwise you should use Zope3. Long answer: Zope2 and Zope 3 has slightly different target audiences. Zope 2 was ment as a webserver, where you could create dynamically generated websites and manage them through the web. However, Zope 2 was so powerful and the combination of true OO database and Python so developer friendly, that many people soon started to use it as a web application development platform, creating web applications that had nothing to do with the orginal concept of creating websites. Zope 3 is directly and originally aimed at that type of application development. That means that is you are looking for a web development platform, you should use Zope 3. If your aim it to make a website, Zope 3 is currently lacking in that department. Mainly, a complete enterprise CMS is not yet available for Zope 3 (at least not as open source), so there is no substitute for CPS or Plone yet. What about Five? Five is a technology bridging the gap between Zope 2 and Zope 3. It is included in Zope 2.8, and it enables you to use many of the Zope 3 features in Zope 2. It is therefore mainly of use if you have to use Zope 2 for the above reasons, but you want to start using the nice Zope 3 programming patterns, such as views and adapters. Currently, products written for Five will NOT run under Zope 3, and products written for Zope 3 will NOT run under Five. However, with every new release of Five and of Zope 2, it is intended to bring Five products and Zope 3 products closer and closer, thereby having a situation where only small modifications, or maybe even no modifications at all, are needed. Hints for Zope 2 developers As a Zope 2 developer, it's easy to get into the wrong programming patterns for Zope 3. Many of the techniques used in Zope 2 are not necessary for Zope 3. Don't use stock objects! A common pattern in Zope 2 is to monkey-patch the stock objects to provide more functionality. Typical examples of this is that you want to make all folders ordered, so you can change the sorting of the objects, or override all applications manage_afterAdd method to reindex them, and so on. But Zope 3 is not a way to make websites. It's an application development platform. The application you create are not expected to use any "stock objects". There are not very many of them anyway. You are supposed to create your own objects and use them. This also means you don't have to monkey patch anything. In addition, events and adapters mean that have have better ways of changing and overriding functionality without any monkey patches. Don't monkey patch! See above: You shouldn't need to monkey patch in Zope 3. If you do, you are most likely doing something wrong. There is of course the possibility that you need to override something that should be overridable via some sort of configuration, but isn't, in which case this could be seen as a Zope 3 bug. Fixing Zope 3 bugs by monkey patching is OK, provided you also get the bug fix into the next release of Zope 3, of course. Create your own ZCML statements! ZCML is the Zope 3 way of configuring. It's only global configuration (configuration done on a site or folder basis needs to be done through the web at the moment) but all that global configuration should go into ZCML. It should absolutely not go into Python code. Why? Because ZCML is overridable. If you have in your product a default ZCML statement to configure your product, the people who use your products do not need to change your code to modify the begaviour of your product. Instead, all they need to do is create a new product with an overrides.zcml, and it will override your statements. Making ZCML statement may be a bit confusing the first time, but once you got a hang of it, it's quite simple. http://blogs.nuxeo.com/sections/blogs/lennart_regebro/2005_10_04_zope2-vs-zope3-faq lregebro Zope2 vs Zope3 FAQ Après avoir annoncé son passage à OpenOffice.org en début d'année, la Gendarmerie Française s'investit dans le scripting. Ce sont près de 1000 livres qui vont être commandés afin de permettre aux gendarmes d'exploiter encore plus la puissance d'OpenOffice.org au travers de son API. Gageons que celà débouchera vers des contributions au projet OpenOffice.org en terme d'addons ... http://blogs.nuxeo.com/sections/blogs/laurent_godard/2005_10_03_scripting-d-ooo-s-etend lgodard Le scripting d'OOo s'étend There are some subjects in the Python mailing list that can fire up people interest. The way Python implements the object model is one of those, and the history and use of leading underscore in the langage and in Zope too.. http://www.forbiddenweb.org/viewtopic.php?id=51090 Gosh, (+100 answers) :-) http://blogs.nuxeo.com/sections/blogs/tarek_ziade/2005_10_03_one-longest-python tziade Looooooong Python thread A recipe version of the SimpleAsyncHTTPServer with a few additional options not previously available. http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440665 Asynchronous HTTP server Given a message encoded with a shift/rotation cipher, such as rot13, this recipe recovers the most probable plain text for the message. It does this by using statistics of bigram (2-character sequence) counts from a sample of text. http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/442000 Decoding a Shift (or Rotation, or Caesar) Cipher (or Code) This JSON parser works well with stringified Python list or dictionary. It is from json.org javacript json parser with small modification. http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440637 Simple AJAX with javascript JSON parser A simple but useful class that emulates a function to gracefully permit latent assignment. In other words, I can use the emulating class as a valid function in assignments with the ability to later associate a function to perfrom the actual operations. http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/441240 Function emulation using __class__ The Voicent Python Simple Interface class contains the following functions. callText callAudio callStatus callRemove callTillConfirm These functions are used to invoke telephone calls from your Python program. For example, callText is used to call a specified number and automatically play your text message using text-to-speech engine. In order for this class to work, you'll need to have Voicent Gateway installed somewhere in your network. This class simply sends HTTP request for telephone calls to the gateway. Voicent has a free edition for the gateway. You can download it from http://www.voicent.com More information can be found at: http://www.voicent.com/devnet/docs/pyapi.htm http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/441239 Making telephone calls from your python program using Voicent Gateway This recipe applies the "once and only once" principle to function default values. Often two or more callables specify the same default values for one or more arguments. This is especially typical when overriding a method. Using the defaultsfrom(func) decorator, a method may 'inherit' the default values from the super method. More generally, any function may inherit the default values from another one. http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440702 Reusing default function arguments This recipe explains how to check that a given function does not run slower than a given pystone rate. It first calculates the pystone ratio on your box. http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440700 performance testing with a pystone measurement decorator Makes it easier to execute async calls or deal with external systems calls to which can block forever or occassionally take long time to complete. http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440569 Decorator to make a decorated call in a separate thread with timeout Using string templates to separate views from models and controllers is fine, but passing data from controllers to views is often tiresome. Using frame inspection can make things a lot more straightforward, saving you the hassle of explicitely passing each and every bit of data the template needs through boring lines of code like {'name':name}. Here is a sample with a fake templating system. http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440685 Use frame inspection to simplify template usage This function accepts a string and returns a string with whitespace(one space) inserted between words with leading capitalized letters. http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440698 Split string on capitalized/uppercase char This recipe is Python implementation of few lines of C-code that get useful information about current working console on Windows. It may be useful for console application to proper formatting output. Recipe need ctypes package to be installed. http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440694 Determine size of console window on Windows This is my implementation of Rendering Arbitrary Objects with Nevow (http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/286260) using CherryPy 2.1. CherryPy doesn't care about adapters or stuff like that, but since it is written in Python it is easy to add that sort of functionality. http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440696 Rendering Arbitrary Objects with CherryPy Generates an infinite character pasword http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440693 Infinite Character Password Generator This little class starts up an SMTP server which acts as an email sink, collecting all received emails destined for any address. All emails are routed to a Portable Unix Mailbox file. This is very handy for testing applications that send email. It runs in its own thread, so you can easily use it from a test fixture to collect your emails and verify them for correctness. http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440690 SMTP Mailsink A list of bits, compacted in memory so that 8 elements use 1 byte of storage. http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440658 Memory compacted list of bits Use ListMixin to create custom list classes from a small subset of list methods. http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440656 List mixin The Heap class is an improvement over a previous recipe (http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/437116) that also wraps the heapq module. There are two main differences from the other recipe: - All methods on this heap preserve the heap property invariant; therefore there is no need for is_heap(). - When creating a new heap, an optional 'key' argument can be specified to determine the comparison key for the items to be pushed into the heap. http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440673 Safe heap queue class Memoize, but clear cache when last function call returns. Suitable for dynamic programming. http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440678 Memoization with cache cleared on return of last function call A small, powerful templating language using template strings embedded in standard Python syntax. Templates are valid Python source, compiled directly to bytecode. Variable substitution is performed using 'string.Template'. http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440629 Good enough templating