GenericSetup is a framework to describe the configuration of a Zope site as a set of XML files (and sometimes other associated files). It can import profiles, which may create objects or change their configuration, and export profiles, which makes a snapshot of the configuration and writes it to a set of XML files.
GenericSetup provides a tool that can store snapshots of a configuration in the ZODB itself, where it can be examined and even modified. It can also do diffs between two snapshots, which is very useful to find out what changed in a configuration (it's a good idea to take a full snapshot anytime some significant changes are made to the configuration).
GenericSetup differentiates between Base and Extension profiles. A Base profile is a profile that describes "everything". When it is imported, it removes and overwrites any previous configuration. An Extension profile is a profile designed to be incrementally added on top of a previously existing configuration. It may of course overwrite some settings, or even in some case remove objects, but its goal is generally to add optional configuration on top of a main one
GenericSetup is based on a small number of concepts: the toolset, and some import and export steps. GenericSetup provides a framework where import and export steps can be written simply using Zope 3 adapters.
GenericSetup was born as "CMFSetup" but was later made generic and can be used by any Zope 2 application. It is now available at http://svn.zope.org/GenericSetup/trunk/. A subclass with additional features is used in CPS.
The setup tool is called setup_tool by default, and portal_setup in CPS (if missing, it can be instantiated by selecting "CPS Tools" from the ZMI Add menu).
At a given time, the setup tool knows about a few things:
- the currently selected profile, which may point to an area in the filesystem where a profile has been registered, or to a path in the ZODB where a snapshot was taken,
- the current toolset,
- the current import steps,
- the current export steps.
When a new profile it selected, its toolset, import steps and export steps XML files are loaded and merged with the tool's current ones.
Any import or export is based on the full toolset or import/export steps (even if the currently selected profile is an extension profile), but the source of XML configuration files depends solely on the currently selected profile.
The file toolset.xml describes the Toolset, which is the set of tools needed in a given configuration. (While the name "tool" would suggest CMF, it's just a set of objects that can be instantiated at the root of the configured site.)
Beyond being based on data that is updated when an extension profile is selected, the toolset is a normal import/export step.
When the toolset is imported, all the objects in the toolset are instantiated if they're missing or if their class doesn't match with what it's supposed to be.
An excerpt of toolset.xml for the CPS Tree Tool would be:
<?xml version="1.0"?>
<tool-setup>
...
<required tool_id="portal_trees"
class="Products.CPSCore.TreeTool.TreeTool"/>
...
</tool-setup>
Import steps (import_steps.xml) describe a set of configuration steps available for import, and their dependencies. A step is just the dotted name of a function, and the dependencies is simply the steps that have to be run before this one can be done (most steps depend on the toolset, because they need a base tool to be instantiated before it can be configured).
While an import step can do anything it likes, GenericSetup provides a framework based on Zope 3 adapters to simply describe how a single object is imported from XML, and to recurse among objects to create or configure them one by one.
During import, there are two possible behaviors, corresponding to the two kinds of profiles. For a Base profile, the import happens in "purge" mode, while for an extension profile the import doesn't purge. The functions and adapters doing the import have to take that into account when they read a profile.
In purge mode, every previous configuration has to be removed, so that the result is indistinguishable from an install from scratch. In non-purge mode, care must be taken to not overwrite or remove settings or objects which are not explicitely specified in the imported XML file.
An excerpt of import_steps.xml for the CPS Tree Tool would be:
<?xml version="1.0"?>
<import-steps>
...
<import-step id="trees" version="20051230-01"
handler="Products.CPSCore.exportimport.importTreeTool"
title="Tree Tool">
<dependency step="toolset"/>
Import tree tool and tree caches.
</import-step>
...
</import-steps>
Export steps (export_steps.xml) describe the list of steps available for export. An export step is used in exactly the same way than an import step, except for the fact that there are no dependencies between export steps, and that instead of being read, XML files are written.
One important thing to note is that export steps cannot do the "incremental exports" that many people expect. When an extension profile is read by the import steps, only the available XML files for that profile are read. However when writing, there's no way to choose which XML files are relevant, so the whole profile for that step is written (and recursion is done in all subobjects).
It's possible that future versions of GenericSetup will have some capabilities to do incremental exports, but this is not possible for now.
An excerpt of export_steps.xml for the CPS Tree Tool would be:
<?xml version="1.0"?>
<export-steps>
...
<export-step id="trees"
handler="Products.CPSCore.exportimport.exportTreeTool"
title="Tree Tool">
Export tree tool and tree caches.
</export-step>
...
</export-steps>
The standard work done in an import or export step is to find the base tool, and call importObjects or exportObjects on it; these are recursive functions that take each object, find an adapter for them describing how the XML import or export is done, and call it.
For the CPS Tree Tool, the import steps and export steps call the following functions:
def exportTreeTool(context):
"""Export Tree tool and tree caches as a set of XML files.
"""
site = context.getSite()
tool = getToolByName(site, 'portal_trees', None)
if tool is None:
logger = context.getLogger('trees')
logger.info("Nothing to export.")
return
exportObjects(tool, '', context)
def importTreeTool(context):
"""Import Tree tool and tree caches as a set of XML files.
"""
site = context.getSite()
tool = getToolByName(site, 'portal_trees')
importObjects(tool, '', context)
This is pretty boileplate and could even be simplified in the future through ZCML declarations. Above, '' simply refers to the root of the profile directory.
The adapters are multi-adapters, adapting both an object and an import context (called "environ" in GenericSetup), to the IBody interface that basically describes a file body. They can be registered through ZCML using the standard statement:
<adapter
factory=".exportimport.TreeToolXMLAdapter"
provides="Products.GenericSetup.interfaces.IBody"
for=".interfaces.ITreeTool
Products.GenericSetup.interfaces.ISetupEnviron"
/>
This assumes of course that the exported object is described through an interface, for instance here declared as:
from zope.interface import Interface
class ITreeTool(Interface):
"""Tree Tool.
"""
The interface is implemented by the object's class with:
from zope.interface import implements
class TreeTool(UniqueObject, Folder):
"""Tree Tool that caches information about the site's hierarchies.
"""
implements(ITreeTool)
id = 'portal_trees'
meta_type = 'CPS Tree Tool'
...
When doing an export, the adapters build a DOM tree for the configuration. When doing an import, they read the DOM tree and create or modify properties as needed. The standard adapters don't create subobjects or recurse in them, this is left to the importObjects function.
These adapters can be written easily because GenericSetup provides helpers for the common cases of objects configured only through standard Zope 2 properties (PropertyManager), or having subobjects (ObjectManager).
Of course all this can be changed for specific cases. Many older CMF tools are configured through things that are not standard properties, and for instance CPS needs to do recursion into more than simple ObjectManager subobjects. It is also possible to read and write files that are not XML files, CPS does this for the images included in portlet objects, where it writes a real image file.
CPS has extended the setup tool to provide a basic Upgrade feature, that is related to the configuration of the site but cannot be expressed by the standard GenericSetup profiles.
An upgrade step is registered through ZCML with something like:
<cps:upgradeStep
title="Replace CMF URL tool with a CPS-specific one"
source="3.3.4" destination="3.3.5"
handler=".upgrade.upgradeURLTool"
checker=".upgrade.check_upgradeURLTool"
/>
This describes between what CPS versions this upgrade is needed, how to do it, and how to check if it's already been done.
The setup tool lists which steps have not yet been done, and provides a way to run them one by one or all at once. At a given time, a CPS site "knows" (through a site-global property last_upgraded_version) up to which version it's been upgraded.
The checker is optional; if it would be too costly to check for the conversion, it can be omitted. This is the case typically for an upgrade step that would recurse in the content object to do some fixups; to check if they've already been done it would have to recurse in the content objects too, and that's too much work for a simple check. The setup tool won't list a step without a checker if the portal has already been upgraded to a later version than the step's destination version.
