Itinerant Source

The Evils of GenericSetup

Summary

brain dump on generic setup and product bundles, ie. separately configuration management and package management. there are a couple of different issues here, configuration and state management of an instance, and distribution of software.

Use Cases

i think its important to preface discussion of these with a consideration of user stories

a user wants to

- install an addon from a remote repository, preferably installed without restart.

- try out an addon, decide against it, and remove it safely from the system, with zero application footprint. likely will need a restart.

a developer/integrator wants to have

- easy upgrading

an additional noteworthy case of upgrading, are
developers want to upgrade and deploy against scm
revisions.

regarding configuration and state management

generic setup is about managing configuration state for easy snapshotting and restoration of a system configuration. its good about managing and extending the whole system state, but its ability to deal with removal of partial states, and state differences states requires human intervention dealing with tool specific file configuration diffs.

modeling configuration as reversible changesets can accomplish these use cases. an installation/changeset would need to be composed of tasks, and each either associated with date or preferably revision, conflict discriminators (ala zcml), associated task dependencies, and install and uninstall methods. a library of common reversible tasks and a common configuration mechanism (zcml/yaml/etc) for usage and definition, ala tools like cpsinstaller, ploneinstaller, pymake.

an interesting issue is customization, its a hard issue and its an area where plone could use improvement. generic setup makes available raw file diffs, but with the ability to address task sets, we can compose much more user/integrator friendly with task conflicts noted by task, and optional installation of partial changesets (easy customized tool overides). conflicting configurations utilize the task discriminator, and can be presented to the user.

another interesting issue is working against svn with independent development, developers often want to use the latest and greatest features from svn while developing systems, but the confidence to know that going into production. an installation and configuration mechanism for plone should be able to intelligently update checkouts via always doing secondary sorting (post dependency sorting) on a task’s revision/date metadata, so we can track updates by order of application, instead of version number.

one doubt i have on this is that it continues the maintenance of configuration as instance data in the zodb, in addition to tool state the use of z3 components and achieving these benefits, relies on increased reliance on persistent component registries, when the move in z3 has been towards global configuration, i think at some level the growth of persistent data is inevitable for the degree of application instance specific customization of plone typically wants to offer. its unclear, though it would be interesting to see if stephan’s z3c.baseregistry is capable of dealing well with persistent utilities, and adapte registrations.

any solution needs to be at least interim compatible with quickinstaller to facilitate installation on existing versions of plone pre 3.0, which are likely to see widespread use for at least another year, a extensions install stub that fires off the task set installation would suffice.

distribution

distribution is mostly orthogonal to configuration, but the parts should play nice together in the ui to offer a good end user experience, good product distribution basic building blocks should also start to address the needs of micro distribution within the community, such as non profit distributions, intranet/collaboration distributions, in addition to plone’s adherence to being an easy to use cms product.

eggs are a good distribution means and are well supported in the python community.

we need packaging/development with a tool thats accomplished at doing source buildouts and construction of eggs, with examples/documentation on using it for plone. zc.buildout looks like a good tool though lacking in documentation or examples for plone at the moment.

but easy_install isn’t nesc. the appropriate management interface for plone eggs, part of being a being web app product, should also include the optional installation of components through the web. for a plone package manager, its not clear if reutilizing the easy install library is the best way to facilitate it, or if just dealing with raw eggs and using pkg_resources is. the easy_install internals are filled with tons of abstractions, that may or may not be easily used for such a manager construction. regardless, the use of eggs as a standard format for distribution is still a clear win.

for a global repository, there’s been talk of pypi enabling the software center, it not clear whats the best approach, outside of that it should be easy to upload globally such that software is discoverable, and the approach should be able to facilitate running various mini-distribution repositories.