[omelette]
recipe = collective.recipe.omelette
eggs = ${instance:eggs}
products = ${instance:products}
packages = ${zope2:location}/lib/python ./
Don't forget the ./ at the end. Note: the README.txt of the recipe mentions modules instead of packages but that is a typo. Fixed in subversion.
I especially like it that with this configuration you can look in parts/omelette/Products and have links to the exact same Products that your zope instance will be using, whether a Product is in an egg, in the productsdistros/checkouts directory, in parts/plone or in the Zope 2 Products dir. That is sweet.
Thanks a lot David!
]]>Grokking is triggered from zcml, with the grok directive. This should be the only zcml that is needed. It scans modules in your code for classes that subclass grok's base classes. Grok then registers these components with the Zope Component Architecture (ZCA).
If you follow standard patterns, you can leave out grok directives (grok.name, grok.require, etc) from your code, but if you need more control, you can use them.
grok.Model is usually your base class. Very light weight. It should be contained in a class with grok.Container as base class. There is no ordered container class yet; we will add it to the todo list of the sprint.
Traversal in Grok is very easy. There is of course the default traversal that just works. But if you need more control, you add a traverse method to your model. This can instantiate virtual objects on the fly.
You can also use grok.Traverser to implement custom behaviour for someone else's context.
Sites is Zope 3 is a place to store local utilities and tools. Grok has grok.Site for this. As todo we may want to add locally defined views and adapters. And having non-persistent local utilities would be useful as well. And we want to be able to install local utilities in an existing site, instead of only in a new site, without hairy upgrade code.
grok.Application is a special kind of grok.Site.
Todo: look at GenericSetup to make upgrades easier.
grok.Indexes makes working with the catalog easier. Todo: upgrading indexes in an existing site should be possible.
grokcore.component has several components you can use, also in plain Zope 3:
Todo: set up caching for views.
Using persistance:
Security is all view related.
zope.formlib based forms.
That concludes the component tour.
Grok is more than just the API. It is a sort of language with base classes, decorators, methods. It is an attitude to be friendly to beginners.
Web Services Gateway Integration. We have code for this, but we need to make this easier; possibly put it in grokproject by default.
Rule: a sprint project is not finished without documentation!
I want the Caps Lock key to behave as an extra Control key. This worked before and it still works, but now when I press the Caps Lock key the light indicating that Caps Lock is on starts or stops shining, even though I only really use it as a Control key. See this bug report, which also mentions some remedies. I ended up adding some lines to my ~/.bashrc file:
xmodmap | grep lock | grep Caps_Lock > /dev/null
if test $? -eq 0; then
xmodmap -e "remove Lock = Caps_Lock" -e "keysym Caps_Lock = Control_L" -e "add Control = Control_L"
fi
Firefox 3.0b5 is used. There are not yet versions of Firebug and Live HTTP Headers that are compatible with this. That is not good when you are doing web development. I expect that this will get fixed soon.
In light of that last issue I think I will keep the laptop from my work at the previous Ubuntu for a while until those extensions are available.
Update: Both Wichert Akkerman and Jeroen Vloothuis tell me that there is a Firebug 1.1 beta that works with the new Firefox. This was not automatically found when Firefox looked for updates. This version indeed works, though for me it only works when opening Firebug in a new Window.
All in all a big thank you to everyone who made another Ubuntu version happen!
]]>It turns out that when you do an svn export and then a python2.4 setup.py bdist_egg to create your egg, you are missing some files. At least I was. When you do an svn checkout and then make your egg, all files are there.
In neither case was any zcml file mentioned in the SOURCES.txt file in the automatically created egg-info directory.
For the tar ball created by a python2.4 setup.py sdist I did not notice any difference.
Does anyone know what is going on here and how to solve this so it works from an svn export too?
Cooking lesson for today: check that egg!
Oh, and sorry for spamming http://planet.plone.org today. I changed the language of a lot of weblog entries from neutral into English. Now they all needlessly show up on that home page.
Daniel Nouri pointed me to the setuptools docs that explain what is going on; plus the docs for disutils, especially the part about adding a manifest file. And Andreas Zeidler pointed to a thread on the Plone developers list. And Florian Schulze pointed me to both. :-) I have the feeling there are a few people reading this weblog. :-) Thanks everyone!
In my case the tar ball problems pointed out by Florian did not affect me. In fact, when I uploaded just the tar ball and tried to easy_install it, this went fine: an egg was automatically created from the tar ball. But it is good to know that there can be problems in that area too.
Conclusion for me: easiest is just to create and upload an egg from a subversion checkout, not from an export. And if you want more control, read those setuptools docs.
It now has an own Product page on plone.org too. Not much is there yet. For downloads just use the cheese shop. If you are using it, drop me a note: it would be nice to know.
Let me show some code. We subscribe to an event in configure.zcml:
<subscriber
for="*
zope.app.publication.interfaces.IBeforeTraverseEvent"
handler=".traversal.switch_skin"
/>
And in traversal.py you then have this code:
def switch_skin(object, event):
"""Switch to the Plone Default skin when we are editing.
"""
context = object
request = event.request
portal_props = getToolByName(context, 'portal_properties')
editskin_props = portal_props.get('editskin_switcher')
edit_skin = editskin_props.getProperty('edit_skin', '')
...
context.changeSkin(edit_skin, request)
Well, the code is slightly longer, but this should give you an idea. Except for details this is the same code as was in the External Method, which is now gone.
All in all, for this change one file was added: traversal.py. And various files could be removed, mainly: Extensions/* and setuphandlers.py.
More importantly: if you now remove this package from your zope instance without properly reinstalling it, you are not left with an ExternalMethod in your Plone Site that gives errors because the file is not available anymore.
BTW, if you are using this package already, it is best to uninstall version 0.2 and then install version 0.3; that way the quick installer will take care of removing that external method with the access rule for you.
]]>You could already get a different skin (presumably for editors) when you are visiting your Plone Site with an edit url, like edit.yourdomain.com.
Now you can also get that edit skin when you are logged in. More precisely: when you have the __ac cookie. This does not work when you are logged in with basic authentication, in other words: via the ZMI.
This is controlled by two new properties in the portal_properties/editskin_switcher.
You can combine the two behaviours if you want. If they are both True, then you need to have the right url and you need to be logged in.
When both are False, nothing happens: then you might as well simply uninstall this product as it is not useful.
]]>For a customer of Zest Software I created a package called collective.editskinswitcher. I gladly took some code from colleague Mark van Lent who did something similar for a different web site. The package is on the Cheese Shop so it can be easy installed. And the code is in the Plone collective.
Let's say you have a Plone Site. I tested this with Plone 3. I see no reason why it should fail on Plone 2.5. Maybe it even works on a CMF site. Anyway, whatever site you have is available on two urls: www.yourdomain.com and edit.yourdomain.com. Some day you should ask your local Apache guru how he did that.
With collective.editskinswitcher installed (with the portal quick installer), visitors that go to the website with the url edit.yourdomain.com will see the Plone Default skin, which is meant for content editors. Visitors to www.yourdomain.com will see whatever skin you have set as the default skin in portal_skins. Can be pretty handy.
Developer types probably like the fact that you also get the default skin when visiting localhost and the edit skin when you go to 127.0.0.1.
But maybe you want to turn it around: your visitors should see Plone Default and your editors should see your brilliant Monty Python Skin! Ni! Just go to the portal_properties, then editskin_switcher and change the edit_skin property to your dashing theme.
I looked at CMFUrlSkinSwitcher first but it had not been touched in two years. One import error (CMFCorePermissions) could easily be fixed as that import was not even used. But after that tests were failing all over the place. Theoretically always fixable of course, but rolling an own package seemed easier, cleaner and faster.
Also, CMFUrlSkinSwitcher does some more things. At least it messes around with some methods like absolute_url. It could be that I find out later that this is necessary in collective.editskinswitcher too, but currently it does not look like that will be the case.
The easiest way to test this package in a default plone site (apart from running the tests of course), is:
On Linux you can edit /etc/hosts and add a line like:
127.0.0.1 edit.yourdomain.com www.yourdomain.com
Now visiting edit.yourdomain.com should give you default Plone and www.yourdomain.com should give you the customized Plone.
You can also let the edit urls begin with cms or manage. As long as the url is something like:
...//(edit|cms|manage).something.something....
you end up in the edit skin.
Have fun!
Maurits van Rees
I first wrote a rather difficult guide to getting a stable, repeatable buildout. After some helpful comments from Martijn Faassen I could make this much simpler. I decided to keep the old weblog entry around: it could be helpful for people googling for strange errors in their buildout.
I assume you know what buildout is, otherwise you would probably not be reading this. Martin Aspeli has written a tutorial on Managing projects with zc.buildout. This should be your first stop for learning more about buildout, at least if you are a Plone developer.
My brother and colleague Reinout has written a weblog entry about the Buildout development/production strategy that we are starting to use at Zest Software. The goal is to use the same buildout for both development and production deployment.
As the buildout should be repeatable, you may want to read the repeatable test file from the zc.buildout package.
Now I am going to write about creating a really stable and repeatable buildout. In Reinout's story that would fit in with creating a stable.cfg. If you are not concerning yourself with development or if you have a different buildout for production deployment, this would just be the buildout.cfg file.
Let's begin with some perhaps arbitrary definitions.
With stable I mean: you run the buildout script in a directory; one year later you run buildout for the second time; you get the same result as one year earlier. Use case: by accident you removed the entire parts directory and you need to rebuild it.
With repeatable I mean: you run the buildout script in a directory; one year later you run the buildout on a completely different server; both directories are exactly the same. Use case: you want to move the zope instance of a customer to a new server.
What I write in this entry should be applicable to any buildout with or without Plone. My main focus and example is a Plone buildout though. You can fix up a current buildout of course, but here let's start with a clean default buildout for Plone 3, which you can create with paster:
paster create -t plone3_buildout test cd test python2.4 bootstrap.py
When running bin/buildout by default it picks the newest package it can find for any dependency. We can tell buildout to first look if some version of that package is already available:
[buildout] newest = false
This way, a second call of bin/buildout will not try to get any new packages. So it will finish quickly and just keep the current packages. This already makes the buildout stable in the sense that a new run of bin/buildout in that same directory will give you no new packages.
But if you want to recreate that buildout in a different directory or even a different server, you can get other versions. For starters, currently this buildout config will give you Plone 3.0.5, but a few months from now you will probably get Plone 3.1. So this is not yet a repeatable buildout.
As an aside, while we are in the buildout section, let's quickly specify a simpler (read: faster) index than the default CheeseShop:
[buildout] newest = false index = http://download.zope.org/ppix
See my brother's weblog entry on ppix instead of pypi from now on.
By the way, this has nothing to do with making your buildout more stable or repeatable.
Let's go get some more stability in our buildout. For some recipes and packages there are more ways to pin versions. But none of them work in all cases, except one. So we will just use that one way:
[buildout] ... versions = versions [versions] ...
So in your [buildout] section you add an option versions in which you point to another section that contains the versions that packages should be pinned to.
First things first: we pick a Plone version. We simply add a line to the [versions] section that we added above:
[versions] plone.recipe.plone = 3.0.5
This recipe has strict versions for the Plone products and packages that make up a Plone release, so for example Products.CMFPlone, Products.CMFCore, the various plone.* packages, etcetera.
And this has a zope2-url property that tells the plone.recipe.zope2install recipe that it should use Zope 2.10.5.
So the most important versions have already been pinned now.
Wichert Akkerman came up with the following one-liner to get a list of versions that are not pinned by you, but picked by buildout:
bin/buildout -Novvvvv |sed -ne 's/^Picked: //p' | sort | uniq
Currently this returns this list:
elementtree = 1.2.7-20070827-preview plone.recipe.distros = 1.3 plone.recipe.zope2install = 1.2 plone.recipe.zope2instance = 1.3 python-openid = 2.0.1 setuptools = 0.6c7 zc.recipe.egg = 1.0.0
This basically means that the mentioned packages are not pinned by our buildout config, but chosen (picked) by buildout. This means that they are not stable yet: rerunning this script in a few months time will likely give you different versions.
So what should you do? Very simple: you literally add those lines to the [versions] section. Now all your packages are pinned.
You can of course also pin any extra products that you want to use in your buildout. Best is to use an official release, such as a tar ball:
[productdistros]
recipe = plone.recipe.distros
urls =
http://plone.org/products/poi/releases/1.1/poi_1.1.tgz
or a checkout of a subversion tag:
[productcheckouts] recipe = infrae.subversion urls = http://svn.plone.org/svn/collective/eXtremeManagement/tags/1.5.2/ eXtremeManagement
If for some reason there is no tag you can use, you can still specify a revision in the url with the @ sign. Thanks to Guido Wesdorp for pointing this out:
[productcheckouts]
recipe = infrae.subversion
urls =
http://getpaid.googlecode.com/svn/trunk/products/PloneGetPaid@1132 PloneGetPaid
For now let's keep the example simple here and remove these products again from our buildout.cfg.
A buildout can be made very stable: just put newest = false in your buildout section. If one year later you accidentally remove the parts directory, you can rerun buildout and get your original directory back. If you remove your eggs though, you are in trouble.
Making a buildout repeatable is not that difficult either after all. Put this in your buildout.cfg:
[buildout] ... versions = versions [versions]
Right below [versions] add the result of this one-liner:
bin/buildout -Novvvvv |sed -ne 's/^Picked: //p' | sort | uniq
Contrary to some earlier conclusions that I drew, this is everything that is needed to pin all packages.
For pinning old-style Products, see the section on Pinning extra products.
plone.recipe.plone rigourously depends on specific packages and products. So what do you do if you want to use a newer version of just one or two packages? For instance, I did some fixes for plone.locking which are still not in Plone 3.0.5. And for multilingual sites you really want to use a newer version with an important bug fix. You might at first think that this would be enough:
[versions] plone.locking = 1.0.5 plone.app.i18n = 1.0.2
But running bin/buildout then gives an error:
The version, 1.0.2, is not consistent with the requirement, 'plone.app.i18n==1.0.1'. While: Installing plone. Error: Bad version 1.0.2
So you need to add two lines to the [plone] section:
[plone]
recipe = plone.recipe.plone
eggs =
plone.locking
plone.app.i18n
These lines basically erase the version pinning of those two packages that is in the recipe. After that, the pinning in the [versions] section can take effect.
Let's end with showing what our buildout.cfg now looks like. Or actually, let's let's show a buildout that uses some earlier versions: Plone 3.0.2, older recipes, and older setuptools and elementtree. And instead of plone.app.i18n 1.0 that is required by Plone 3.0.2 we pin this package to a slightly newer version 1.0.1, which is not the latest version in the CheeseShop. This older buildout should show nicely that you can have a stable, repeatable buildout months later. Try it! So here is the file itself:
[buildout]
newest = false
index = http://download.zope.org/ppix
parts =
instance
zope2
plone
zopepy
productdistros
find-links =
http://dist.plone.org
http://download.zope.org/ppix/
http://download.zope.org/distribution/
http://effbot.org/downloads
# Add additional eggs here
# elementtree is required by Plone
eggs =
elementtree
versions = versions
[versions]
setuptools = 0.6c6
zc.recipe.egg = 1.0.0b6
plone.recipe.plone = 3.0.2
elementtree = 1.2.6-20050316
plone.recipe.distros = 0.3
plone.recipe.zope2install = 1.0
plone.recipe.zope2instance = 1.0
python-openid = 2.0.1
plone.app.i18n = 1.0.1
[plone]
recipe = plone.recipe.plone
eggs =
plone.app.i18n
[zope2]
recipe = plone.recipe.zope2install
url = ${plone:zope2-url}
[productdistros]
recipe = plone.recipe.distros
urls =
nested-packages =
version-suffix-packages =
[instance]
recipe = plone.recipe.zope2instance
zope2-location = ${zope2:location}
user = admin:
http-address = 8080
#debug-mode = on
#verbose-security = on
# If you want Zope to know about any additional eggs, list them here.
# This should include any development eggs you listed in develop-eggs above,
# e.g. eggs = ${buildout:eggs} ${plone:eggs} my.package
eggs =
${buildout:eggs}
${plone:eggs}
# If you want to register ZCML slugs for any packages, list them here.
# e.g. zcml = my.package my.other.package
zcml =
products =
${buildout:directory}/products
${productdistros:location}
${plone:products}
[zopepy]
recipe = zc.recipe.egg
eggs = ${instance:eggs}
interpreter = zopepy
extra-paths = ${zope2:location}/lib/python
scripts = zopepy
When you create a buildout for a production website, you want to be able to recreate that exact buildout one year from now on a different server. How do you do that?
I assume you know what buildout is, otherwise you would probably not be reading this. Martin Aspeli has written a tutorial on Managing projects with zc.buildout. This should be your first stop for learning more about buildout, at least if you are a Plone developer.
My brother and colleague Reinout has written a weblog entry about the Buildout development/production strategy that we are starting to use at Zest Software. The goal is to use the same buildout for both development and production deployment.
As the buildout should be repeatable, you may want to read the repeatable test file from the zc.buildout package.
Now I am going to write about creating a really stable and repeatable buildout. In Reinout's story that would fit in with creating a stable.cfg. If you are not concerning yourself with development or if you have a different buildout for production deployment, this would just be the buildout.cfg file.
Let's begin with some perhaps arbitrary definitions.
With stable I mean: you run the buildout script in a directory; one year later you run buildout for the second time; you get the same result as one year earlier. Use case: by accident you removed the entire parts directory and you need to rebuild it.
With repeatable I mean: you run the buildout script in a directory; one year later you run the buildout on a completely different server; both directories are exactly the same. Use case: you want to move the zope instance of a customer to a new server.
What I write in this entry should be applicable to any buildout with or without Plone. My main focus and example is a Plone buildout though. You can fix up a current buildout of course, but here let's start with a clean default buildout for Plone 3, which you can create with paster:
paster create -t plone3_buildout test cd test python2.4 bootstrap.py
When running bin/buildout by default it picks the newest package it can find for any dependency. We can tell buildout to first look if some version of that package is already available:
[buildout] newest = false
This way, a second call of bin/buildout will not try to get any new packages. So it will finish quickly and just keep the current packages. This already makes the buildout stable in the sense that a new run of bin/buildout in that same directory will give you no new packages.
But if you want to recreate that buildout in a different directory or even a different server, you can get other versions. For starters, currently this buildout config will give you Plone 3.0.5, but a few months from now you will probably get Plone 3.1. So this is not yet a repeatable buildout.
As an aside, while we are in the buildout section, let's quickly specify a simpler (read: faster) index than the default CheeseShop:
[buildout] newest = false index = http://download.zope.org/ppix
See my brother's weblog entry on ppix instead of pypi from now on.
By the way, this has nothing to do with making your buildout more stable or repeatable.
Let's go get some more stability in our buildout. First things first: we pick a Plone version. Change this:
recipe = plone.recipe.plone
into this:
recipe = plone.recipe.plone == 3.0.5
This recipe has strict versions for the Plone products and packages that make up a Plone release, so for example Products.CMFPlone, Products.CMFCore, the various plone.* packages, etcetera.
And this has a zope2-url property that tells the plone.recipe.zope2install recipe that it should use Zope 2.10.5.
So the most important versions have been pinned now.
You can of course also pin any extra products that you want to use in your buildout. Best is to use an official release, such as a tar ball:
[productdistros]
recipe = plone.recipe.distros
urls =
http://plone.org/products/poi/releases/1.1/poi_1.1.tgz
or a checkout of a subversion tag:
[productcheckouts] recipe = infrae.subversion urls = http://svn.plone.org/svn/collective/eXtremeManagement/tags/1.5.2/ eXtremeManagement
If for some reason there is no tag you can use, you can still specify a revision in the url with the @ sign. Thanks to Guido Wesdorp for pointing this out:
[productcheckouts]
recipe = infrae.subversion
urls =
http://getpaid.googlecode.com/svn/trunk/products/PloneGetPaid@1132 PloneGetPaid
You can of course also pin the eggs of packages that you need, which we will see below. For now let's keep the example simple here and remove these products again from our buildout.cfg.
Wichert Akkerman came up with the following one-liner to get a list of versions that are not pinned by you, but picked by buildout:
bin/buildout -Novvvvv |sed -ne 's/^Picked: //p' | sort | uniq
Currently this returns this list:
elementtree = 1.2.7-20070827-preview plone.recipe.distros = 1.3 plone.recipe.zope2install = 1.2 plone.recipe.zope2instance = 1.3 python-openid = 2.0.1 setuptools = 0.6c7 zc.recipe.egg = 1.0.0
This basically means that the mentioned packages are not pinned by our buildout config, but chosen (picked) by buildout. This means that they are not stable yet: rerunning this script in a few months time will likely give you different versions.
We start easily enough by pin elementtree. Instead of this:
eggs =
elementtree
we write this:
eggs =
elementtree == 1.2.7-20070827-preview
As an aside, note that the top directive eggs has one equals sign after it and the option elementtree below it has a double equals sign. The point is that in the options you write a test, so you could also choose greater than (>=) or smaller than (<=). But we want stability, so we keep the double equals sign.
If you now run bin/buildout and then run that one-liner from Wichert again you will now see that the elementtree line has disappeared from the output: buildout no longer picks that version for us as we have pinned it. We are getting closer to stability!
At this point it is good to say that you could stop here: all Zope and Plone products and packages are pinned, which is the most important. Does it really matter if one year from now plone.recipe.zope2instance is at version 2.0 instead of 1.3? Probably not. But there is still room for more stability and repeatability. So if you are still interested, let's continue our quest.
Now we start pinning the other recipes that are in the output. Find these four lines in your buildout.cfg (they will not be right below each other):
recipe = plone.recipe.zope2install recipe = plone.recipe.distros recipe = plone.recipe.zope2instance recipe = zc.recipe.egg
and pin them to specific versions:
recipe = plone.recipe.zope2install == 1.2 recipe = plone.recipe.distros == 1.3 recipe = plone.recipe.zope2instance == 1.3 recipe = zc.recipe.egg == 1.0.0
Now we run the one-liner again and get:
plone.recipe.distros = 1.3 python-openid = 2.0.1 setuptools = 0.6c7 zc.recipe.egg = 1.0.0
This may not be what you had expected. We have pinned plone.recipe.distros and zc.recipe.egg right? That is correct, but look at this partial output of bin/buildout with the verbose option:
$ bin/buildout -v Installing 'plone.recipe.plone == 3.0.5'. We have the distribution that satisfies 'plone.recipe.plone==3.0.5'. Getting required 'plone.recipe.distros' required by plone.recipe.plone 3.0.5. Picked: plone.recipe.distros = 1.3 Getting required 'zc.recipe.egg' required by plone.recipe.plone 3.0.5. Picked: zc.recipe.egg = 1.0.0
What happens here is that plone.recipe.plone depends on two other recipes. They are in the install_requires option of the setup.py file of the recipe. So buildout itself picks a version of those dependencies. It does not look at our pinnings at it is handling the [plone] part of buildout.cfg now and not the [productdistros] or [zopepy] parts that try to pin versions for those recipes.
In fact, you can get a conflict with this. Temporarily we try to pin plone.recipe.distros to an earlier version:
[productdistros] recipe = plone.recipe.distros == 0.3
We run bin/buildout:
While:
Installing.
Getting section productdistros.
Initializing section productdistros.
Loading recipe 'plone.recipe.distros == 0.3'.
An internal error occured due to a bug in either zc.buildout or in a
recipe being used:
VersionConflict:
(plone.recipe.distros 1.3
(.../plone.recipe.distros-1.3-py2.4.egg),
Requirement.parse('plone.recipe.distros==0.3'))
In this case buildout already picked version 1.3 and now we tell it that we require 0.3 so this conflicts. How do we solve that? We change the order in which the buildout parts are executed. Currently it is this:
[buildout]
parts =
plone
zope2
productdistros
instance
zopepy
Now we make sure that productdistros and zopepy are above plone:
[buildout]
parts =
productdistros
zopepy
plone
zope2
instance
We run bin/buildout -v again:
Uninstalling productdistros.
While:
Installing.
Uninstalling productdistros.
Loading recipe 'plone.recipe.distros == 1.3'.
An internal error occured due to a bug in either zc.buildout or in a
recipe being used:
VersionConflict:
(plone.recipe.distros 0.3
(.../plone.recipe.distros-0.3-py2.4.egg),
Requirement.parse('plone.recipe.distros==1.3'))
Oops. Here we are uninstalling version 1.3 as that was installed by the previous buildout run and at the same time we want to install 0.3. buildout does not like this. Only solution I know: remove the .installed.cfg file that is automatically created by buildout. The function of that file is to keep track of what buildout has previously installed, so it knows if it should do any uninstalling or reinstalling or if it can just do nothing and finish within one second. In other words: removing that file should be okay. Anyway, you are keeping backups, right?
Five minutes later...
Right, we are keeping backups. We remove that .installed.cfg file and run bin/buildout again. This will take a bit longer now, as among others it compiles Zope again. At least it completes without conflicts now, and plone.recipe.distros is not picked by buildout anymore, but pinned by us.
For some reason zc.recipe.egg is still picked though. Ah, the [zopepy] part that pins this recipe contains this line:
extra-paths = ${zope2:location}/lib/python
So this part depends on the [zope2] part, which in turn depends on the [plone] part, which in turn depends on the zc.recipe.egg package, which is therefore picked by buildout, ignoring our pinning.
Wonderful.
Okay, seems like we have to resort to trickery. We introduce a new section [dummy-for-pinning]:
[dummy-for-pinning] recipe = zc.recipe.egg == 1.0.0 eggs = zc.recipe.egg == 1.0.0
Yes, I tried this and you need the version number in both lines. Now we simply put that section at the top of the buildout parts. And we move the [zopepy] section down, as it depends on those other parts anyway. So our parts now looks like this:
parts =
dummy-for-pinning
productdistros
plone
zope2
instance
zopepy
What does the one-liner tell us?:
$ bin/buildout -Novvvvv |sed -ne 's/^Picked: //p' | sort | uniq python-openid = 2.0.1 setuptools = 0.6c7 zc.buildout = 1.0.0
zc.buildout is a new one. This is a requirement of zc.recipe.egg, though I could not say why it did not end up in that list earlier. We can add all three remaining picked packages to our [dummy-for-pinning] section so it now looks like this:
[dummy-for-pinning]
recipe = zc.recipe.egg == 1.0.0
eggs =
zc.recipe.egg == 1.0.0
zc.buildout == 1.0.0
setuptools == 0.6c7
python-openid == 2.0.1
After this we hit the end of the road. zc.buildout is pinned, but setuptools and python-openid are still picked for us by buildout. The last possibility I can think of is adding those two to a [versions] section, like this:
[versions] setuptools = 0.6c7 python-openid = 2.0.1
But this has no noticeable effect. In fact, when I change the setuptools version to 0.6c6 (so 6 instead of 7) in both cases, I get this output from buildout:
Installing dummy-for-pinning. Installing 'zc.recipe.egg == 1.0.0', 'zc.buildout == 1.0.0', 'setuptools == 0.6c6', 'python-openid == 2.0.1'. We have the distribution that satisfies 'zc.recipe.egg==1.0.0'. We have the distribution that satisfies 'zc.buildout==1.0.0'. We have the distribution that satisfies 'setuptools==0.6c6'. We have the distribution that satisfies 'python-openid==2.0.1'. ... Getting required 'setuptools' required by five.customerize 0.2. ... required by plone.app.kss 1.2.5. Picked: setuptools = 0.6c7
So our pinning of setuptools is ignored. The same is true for python-openid:
Getting required 'python-openid>=2.0.0,<2.0.999' required by plone.openid 1.0.1. Picked: python-openid = 2.0.1
One last resort: introduce a dummy package that requires a specific version of those two packages in its setup.py. We add it to our buildout and see that it gets used correctly:
We have the distribution that satisfies 'zest.recipe.dummy==0.3'. Getting required 'python-openid==2.0.1' required by zest.recipe.dummy 0.3. We have the distribution that satisfies 'python-openid==2.0.1'. Getting required 'setuptools==0.6c6' required by zest.recipe.dummy 0.3. We have the distribution that satisfies 'setuptools==0.6c6'.
But further on in the buildout nothing has changed and buildout still picks its own versions.
One final twist though: I removed both versions of setuptools from the eggs directory. I then ran buildout again. It again fetched version 6 and then claimed to have picked version 7, but the only package that I could actually find, was version 6. And the second time I ran buildout, it already had version 6, so it no longer picked version 7, but version 6.
Are you still with me? :-) Then maybe just a few more random notes before we get to the conclusions.
A buildout can be made very stable: just put newest = false in your buildout section. If one year later you accidentally remove the parts directory, you can rerun buildout and get your original directory back. If you remove your eggs though, you are in trouble.
Making a buildout repeatable is more difficult. The first easy step is pinning the plone.recipe.plone package. Any extra packages or products can be pinned quite easily as well.
What is difficult is pinning dependencies. If you do not do this correctly, your pinning will cause a conflict. The order in which the buildout parts or sections get executed is important here.
You will have to resort to trickery to get the last few packages pinned down. And even then it does not seem possible to pin really everything. But we are very close and I think there is a good chance that the remaining issues will get solved in buildout, setuptools or easy_install.
plone.recipe.plone rigourously depends on specific packages and products. So what do you do if you want to use a newer version of just one or two packages? For instance, I did some fixes for plone.locking which are still not in Plone 3.0.5. And for multilingual sites you really want to use a newer version with an important bug fix. You might at first think that this would be enough:
[plone]
recipe = plone.recipe.plone == 3.0.5
eggs =
plone.locking == 1.0.5
plone.app.i18n == 1.0.2
But running bin/buildout then gives an error:
ValueError:
('Missing distribution spec', '==')
This is how you accomplish that:
[versions]
plone.locking = 1.0.5
plone.app.i18n = 1.0.2
[plone]
recipe = plone.recipe.plone
eggs =
plone.locking
plone.app.i18n
To start at the bottom: these lines basically erase the version pinning of those two packages that is in the recipe. After that, the pinning in the [versions] section can take effect.
But in fact, nothing happens just yet. This is because we are running buildout in non-newest mode (newest = false in the [buildout] section). But here we actually do not want stability: we want newer versions! So we run buildout in the newest mode, either by temporarily setting newest = true or by calling bin/buildout -n once. Now we get the new packages that we want. Since we have pinned almost everything else, this should be quite safe.
Let's end with showing what our buildout.cfg now looks like. Note here that the mention of setuptools, zc.buildout and python-openid in [versions] or [dummy-for-pinning] may not be too useful. And I will not fault you if you skip that [dummy-for-pinning] section entirely. The reported picked versions are now:
plone.app.i18n = 1.0.2 plone.locking = 1.0.5 python-openid = 2.0.1 setuptools = 0.6c7
And here is the file itself:
[buildout]
newest = false
index = http://download.zope.org/ppix
parts =
dummy-for-pinning
productdistros
plone
zope2
instance
zopepy
find-links =
http://dist.plone.org
http://download.zope.org/ppix/
http://download.zope.org/distribution/
http://effbot.org/downloads
# Add additional eggs here
# elementtree is required by Plone
eggs =
elementtree == 1.2.7-20070827-preview
[dummy-for-pinning]
recipe = zc.recipe.egg == 1.0.0
eggs =
python-openid == 2.0.1
setuptools == 0.6c7
zc.buildout == 1.0.0
zc.recipe.egg == 1.0.0
[versions]
setuptools = 0.6c7
zc.buildout = 1.0.0
plone.locking = 1.0.5
plone.app.i18n = 1.0.2
[plone]
recipe = plone.recipe.plone == 3.0.5
eggs =
plone.locking
plone.app.i18n
[zope2]
recipe = plone.recipe.zope2install == 1.2
url = ${plone:zope2-url}
[productdistros]
recipe = plone.recipe.distros == 1.3
urls =
nested-packages =
version-suffix-packages =
[instance]
recipe = plone.recipe.zope2instance == 1.3
zope2-location = ${zope2:location}
user = admin:
http-address = 8080
#debug-mode = on
#verbose-security = on
# If you want Zope to know about any additional eggs, list them here.
# This should include any development eggs you listed in develop-eggs above,
# e.g. eggs = ${buildout:eggs} ${plone:eggs} my.package
eggs =
${buildout:eggs}
${plone:eggs}
# If you want to register ZCML slugs for any packages, list them here.
# e.g. zcml = my.package my.other.package
zcml =
products =
${buildout:directory}/products
${productdistros:location}
${plone:products}
[zopepy]
recipe = zc.recipe.egg == 1.0.0
eggs = ${instance:eggs}
interpreter = zopepy
extra-paths = ${zope2:location}/lib/python
scripts = zopepy
Some details are left out, like explaining how files and classes are called and where they are located (most are in the browser/ directory), or not showing some methods whose goal and code are hopefully clear enough from their name. The idea is not to make this too long and I am failing at that already. :-) If I omitted too much for your taste and you have a specific question, just ask.
We start with a browser view. On the top we need some imports:
from zope import schema from zope.interface import Interface from zope.interface import implements from zope.formlib import form from Products.Five.formlib import formbase
With our search form we want to search for some searchable text and for Products of a specific Brand. Brand is a content type we (Zest Software) made for a customer. A Product is another content type, which is added to a Brand. We made a vocabulary for this which maps the title of a Brand (shown in the form) to the path where this Brand is found.
We define an Interface for our search form. Very important: let the parameters match the names of the catalog indices that you want to search on:
class ISearchForm(Interface):
"""These are the fields of our search form.
"""
path = schema.Choice(title = _(u'Brand'),
vocabulary=u"customer.brand",
description = _(u'Brand, mark, formula.'),
required = False,
)
SearchableText = schema.TextLine(title = _(u'Search word'),
description = _(u'Search for specific words.'),
required = False,
)
For schema.Choice the default widget is a SelectWidget or RadioWidget. As empty value it displays "(no value)" in the drop down box. At least for a Dutch web site this is not very nice, as there is no translation. You can provide an own version of that message by doing something like this:
from customer.product import customerMessageFactory as _
from zope.app.form.browser.itemswidgets import SelectWidget
SelectWidget._messageNoValue = _("vocabulary-missing-single-value-for-edit",
"Selecteer indien gewenst een waarde.")
Or do this in English and use i18ndude of course.
Now write a class that inherits from:
In our case the search form will be only part of a page, so we choose the second option:
class Search(formbase.SubPageForm):
implements(ISearchForm)
form_fields = form.FormFields(ISearchForm)
label = _(u"Working independently")
description = _(u"You are looking for a product for:")
Add an action button to that class:
@form.action(_(u"Search"))
def action_search(self, action, data):
"""Perform search.
"""
On a SubFormPage this can actually be empty (pass): the surrounding form tag that you have to supply yourself takes care of redirecting. For a complete FormPage you would at least need something like this:
self.request.response.redirect(target)
where target should be a page with search results.
Hook up your browser view in configure.zcml. We only register this view for the Site Root in this case:
<page
name="product-search"
class=".search.Search"
for="Products.CMFCore.interfaces.ISiteRoot"
permission="zope.Public"
allowed_interface=".search.ISearchForm"
/>
Same for a results page:
<page
name="results"
class=".results.Results"
template="products.pt"
for="*"
permission="zope.Public"
allowed_interface=".results.IResults"
/>
And the main page showing search plus results:
<page
name="main-search"
template="searchpage.pt"
for="Products.CMFCore.interfaces.ISiteRoot"
permission="zope.Public"
allowed_interface=".search.ISearchForm"
/>
Use something like CMFPlone/skins/plone_content/document_view.pt as base and add the structure of the search and results subpages:
<html ...
metal:use-macro="here/main_template/macros/master">
<body>
<metal:main fill-slot="main">
...
<div tal:replace="structure provider:plone.belowcontenttitle" />
<form action="" method="get">
<div tal:replace="structure context/product-search" />
</form>
<div tal:replace="structure context/results" />
<div tal:replace="structure provider:plone.belowcontentbody" />
...
</metal:main>
</body>
</html>
If the default template from zope.formlib does not give you what you want, you overwrite it. Configure some named adapters in zcml:
<adapter
factory="zope.formlib.form.default_subpage_template"
name="default.subform" />
<adapter
factory=".search.subpage_template"
name="customer.search" />
The factory for the first adapter is in zope.formlib. Our own alternative factory is this:
from zope.formlib import interfaces, namedtemplate
from zope.app.pagetemplate import ViewPageTemplateFile
subpage_template = namedtemplate.NamedTemplateImplementation(
ViewPageTemplateFile('searchform.pt'), interfaces.ISubPageForm)
Let your class know it should use the new template:
class Search(formbase.SubPageForm):
...
original_template = namedtemplate.NamedTemplate('default.subform')
template = namedtemplate.NamedTemplate('customer.search')
Now write the template, using the original template via the original_template attribute from the view. In this case we add the form description in a slot:
<metal:form use-macro="view/original_template/macros/form">
<div metal:fill-slot="extra_info">
<h2 tal:content="view/description">A nice description</h2>
</div>
</metal:form>
The form now submits to itself. Currently its values are forgotten. So we fix that:
from zope.formlib.form import setUpWidgets
class Search(formbase.SubPageForm):
...
# We set an empty prefix, otherwise we would end up with
# 'form.SearchableText', etc, in the request while the code here and in
# results.py expects 'SearchableText'.
prefix = ''
def setUpWidgets(self, ignore_request=False):
"""From zope.formlib.form.Formbase.
Difference: pass extra data=self.request.form.
"""
self.adapters = {}
self.widgets = setUpWidgets(
self.form_fields, self.prefix, self.context, self.request,
form=self, adapters=self.adapters, ignore_request=ignore_request,
data=data)
While we are here, let's fix a possible unicode error. There is a difference between getting a value from the request or from request.form when unicode is involved:
>>> request.form.get('SearchableText')
u'\xff'
>>> request.get('SearchableText')
'\xc3\xbf'
In the method zope.app.form.browser.textwidgets.TextWidget._getFormInput() the value is fetched from the request and not request.form. So you get a UnicodeDecodeError. We need to guard against that:
def setUpWidgets(self, ignore_request=False):
"""From zope.formlib.form.Formbase.
"""
self.adapters = {}
fieldnames = [x.__name__ for x in self.form_fields]
data = {}
for key in fieldnames:
value = self.request.form.get(key)
if value is not None and not value == u'':
data[key] = value
self.request[key] = value
self.widgets = setUpWidgets(
self.form_fields, self.prefix, self.context, self.request,
form=self, adapters=self.adapters, ignore_request=ignore_request,
data=data)
The search form is working fine now. So we move on the the search results. The results template can be quite simple: a ul with an li for every search result if that is enough for you. The only interesting part is batching:
<div
tal:condition="view/has_results">
<table class="listing">
...
</table>
<tal:batch replace="structure view/batching" />
</div>
So we need a view with attributes has_results and batching. This is the base of the browser view:
from plone.app.content.batching import Batch
class IResults(Interface):
"""List of Products.
"""
has_results = Attribute("The search has matching results.")
items = Attribute("List of Products")
batch = Attribute("Batch of brains")
url = Attribute("URL for this context")
class Results(BrowserView):
implements(IResults)
# We want to use the batching.pt file from plone.app.content, but
# that has a few drawbacks so we copied it ourselves.
batching = ViewPageTemplateFile('batching.pt')
We overwrite it because it has one flaw. It has a question mark where we want an ampersand, so we can add more options in the url. It was:
tal:attributes="href string:${view/url}?pagenumber=${batch/previouspage}&sort_on=${request/sort_on|string:getObjPositionInParent}">
and we change that to:
tal:attributes="href string:${view/url}&pagenumber=${batch/previouspage}&sort_on=${request/sort_on|string:getObjPositionInParent}">
This change means we can add our filled in search terms (path, SearchableText) to the url method of the view:
@property
def url(self):
"""Base url, needed by the batching template."""
url = self.context.absolute_url()
terms = ["%s=%s" % (key, value) for key, value in self.search_filter.items()]
query = '&'.join(terms)
return url + '?' + query
We add a search_filter to our view that gets the filled in values from our search form:
@property
@memoize
def search_filter(self):
"""Construct search filter.
Only add valid search terms from the request.
"""
context = aq_inner(self.context)
form = self.request.form
search_filter = {}
for key in ['path', 'SearchableText']:
value = form.get(key)
if value is not None and not value == u'':
search_filter[key] = value
# When viewing a Brand, add its path to the filter.
if search_filter.get('path') is None:
if IBrand.providedBy(context):
search_filter['path'] = '/'.join(context.getPhysicalPath())
return search_filter
Incidentally, our search form and results will also be used on the view of a Brand, which is why the last few lines were added.
Our view needs a batch property:
@property
def batch(self):
"""Batch of Products (brains).
"""
context = aq_inner(self.context)
# We only search for Products. We need to copy the
# search_filter as we need it original contents somewhere
# else, without the portal_type.
search_filter = self.search_filter.copy()
search_filter['portal_type'] = 'Product'
catalog = getToolByName(context, 'portal_catalog')
brains = catalog.searchResults(search_filter)
batch = Batch(items=brains, pagesize=10,
pagenumber=self.pagenumber, navlistsize=5)
return batch
We use that batch in the shown batching.pt template, but also in our own view, as we do not want to pass brains to our template, but a nice list of dictionaries:
def items(self):
"""List of Products.
Here we get some info for all the Products that are in the
current page of the batch.
"""
batch = self.batch
items = []
for brain in batch:
product = brain.getObject()
info = dict(
url=product.absolute_url(),
title=product.Title(),
isbn=product.getIsbn(),
)
items.append(info)
return items
There you have it: a search form with batched results on the same page. And all thanks to the authors of zope.formlib and plone.app.content.batching!
The U.S. slurps up to a quarter of the world's oil—about three gallons (11 liters) a person every day—even though it has just 5 percent of the world's population.
Pretty bizarre numbers...
First with the official Plone 3.0 tar file:
zope_version = '2.10.4'
archivebundle_sources = [
dict(url='http://plone.googlecode.com/files/Plone-3.0.tar.gz',
internalBundles=['Products', 'lib']),
]
internalBundles is the key here. It tells instancemanager that the tar ball has two directories that have products in them. Not those directories but only their contents should be put in the correct directories in the instance. Since one of those directories is called lib instancemanager knows that it has to put its contents in the lib directory of the instance instead of in Products.
And here is a version using svn checkouts, also showing an alternative way to specifying the dictionaries:
zope_version = '2.10.4'
symlinkbundle_sources = [
{'url': 'https://svn.plone.org/svn/plone/bundles/3.0'},
{'url': 'https://svn.plone.org/svn/plone/bundles/3.0-lib',
'pylib': True},
]
Here pylib tells instancemanager to link the contents into the lib/python directory.
]]>(with many thanks to Kapil Thangavelu (who helped prepare but could not give this presentation) and Chris Johnson)
Show of hands: about half of the people attending are integrators and the other half developers.
E-commerce is hard. There is astounding complexity: payment processing, shipping, discounts, taxes, etcetera. Every one of those items is essential to somebody. So if you want to please everyone you need to have all of that in your system.
Getpaid does not do that. It tries to be a minimal framework. It uses zope 3 style python packages on one side and plone content integration and UI on the other. It has no content types of its own, so you are not forced to use say GetPaidBook or GetPaidDVD because they do not exist. You can use the zope 3 adapters of getpaid on your own content types or the core Plone content types. This means it is pretty small and clean.
Getpaid is currently in production on a few sites. There is:
5 examples of best technologies that we use:
Work in progress:
Wishlist: pure zope 3 UI, as that is the only part that is Plone dependent now.
We have a sprint here this weekend, so join us to finish version 1.0.
Everything is pluggable, as it is pure zope 3, which helps a lot in customizing.
Is there SQL database integration? No, but you could use sqlalchemy.
What do you mean with a product catalog? I mean that you can query for all products in a category. Could be done by querying the portal_catalog, but that would tie it to Plone, which we want to avoid.
Is there a demosite? Yes: http://dev.plonegetpaid.com
"I know I should write tests, but I will just let my customer or the community do that for me. They should just report bugs." Not a good idea, especially if your app only fails in some code that only gets executed by your boss. He will not be happy if it fails.
So first write a test, then start coding. That is Test Driven Development (TDD). Core developers should know that and should not need to follow this talk. :)
With TDD you catch design mistakes early on. It can keep your code simple: if the tests all pass, why would you add more code because you think you might need it in the future, complicating the code?
Tests should exercise API. Demonstrate how to use them. Other developers are happy with this kind of tested documentation. You do this with doctests. They look like a python interpreter session with some paragraphs in between. We use restructured text, so it can be turned into html and pdf and is still readable on its own.
We also have Documentation Driven Development (DDD). You write doctests first. You write a science fiction story. You tell it to an imaginary user. Just use the words "we" and "you". You can put that story on the product homepage (plone.org, PyPI). It helps you keep in mind who you are actually talking to, instead of making documentation that only you or some core developers can understand.
Now Martin Aspeli gave a demo. He just wrote to the attendees list giving some links. I am sure he will not mind if I quote him here so others can read it too:I've checked in the examples to my two talks in the collective:
- http://svn.plone.org/svn/collective/examples/policy-buildout/ contains the sample code for the "Extending and customising Plone 3" talk. The slides are there too, in PDF format.
- http://svn.plone.org/svn/collective/examples/example.tests/ contains boilerplate examples for the different types of tests covered in the "Untested code is broken code" talk, which I gave with Philipp von Weitershausen this morning. This may be useful as it has well-commented test setup examples. I always copy and paste this stuff between my products, so now hopefully we can keep this in one place.