Weblog
Missing files from eggs
That egg you just build might be missing some files. Check that egg! Update: I received some pointers for more info.
Today I discovered that an egg I just made for a package was missing some files. Most noticeable was the missing of some zcml files.
It turns out that when you do an svn export and then a python2.4 setup.py bdist_egg to create your egg, you are missing some files. At least I was. When you do an svn checkout and then make your egg, all files are there.
In neither case was any zcml file mentioned in the SOURCES.txt file in the automatically created egg-info directory.
For the tar ball created by a python2.4 setup.py sdist I did not notice any difference.
Does anyone know what is going on here and how to solve this so it works from an svn export too?
Cooking lesson for today: check that egg!
Oh, and sorry for spamming http://planet.plone.org today. I changed the language of a lot of weblog entries from neutral into English. Now they all needlessly show up on that home page.
Update
Daniel Nouri pointed me to the setuptools docs that explain what is going on; plus the docs for disutils, especially the part about adding a manifest file. And Andreas Zeidler pointed to a thread on the Plone developers list. And Florian Schulze pointed me to both. :-) I have the feeling there are a few people reading this weblog. :-) Thanks everyone!
In my case the tar ball problems pointed out by Florian did not affect me. In fact, when I uploaded just the tar ball and tried to easy_install it, this went fine: an egg was automatically created from the tar ball. But it is good to know that there can be problems in that area too.
Conclusion for me: easiest is just to create and upload an egg from a subversion checkout, not from an export. And if you want more control, read those setuptools docs.
Skin switching with pre-traversal
collective.skinswitcher uses a pre-traversal hook now instead of an access rule.
For collective.editskinswitcher David Convent gave me the idea to get rid of the Access Rule and use a pre-traversal hook instead. That turned out to be pretty easy to do. This looks far cleaner to me. So collective.editskinswitcher is now available as version 0.3. Thanks for the idea David!
It now has an own Product page on plone.org too. Not much is there yet. For downloads just use the cheese shop. If you are using it, drop me a note: it would be nice to know.
Let me show some code. We subscribe to an event in configure.zcml:
And in traversal.py you then have this code:
def switch_skin(object, event):
"""Switch to the Plone Default skin when we are editing.
"""
context = object
request = event.request
portal_props = getToolByName(context, 'portal_properties')
editskin_props = portal_props.get('editskin_switcher')
edit_skin = editskin_props.getProperty('edit_skin', '')
...
context.changeSkin(edit_skin, request)
Well, the code is slightly longer, but this should give you an idea. Except for details this is the same code as was in the External Method, which is now gone.
All in all, for this change one file was added: traversal.py. And various files could be removed, mainly: Extensions/* and setuphandlers.py.
More importantly: if you now remove this package from your zope instance without properly uninstalling it, you are not left with an ExternalMethod in your Plone Site that gives errors because the file is not available anymore.
BTW, if you are using this package already, it is best to uninstall version 0.2 and then install version 0.3; that way the quick installer will take care of removing that external method with the access rule for you.
Switch your skin if you are logged in
With collective.editskinswitcher 0.2 you can now also switch your skin if you are logged in.
I improved collective.editskinswitcher again. For explanation on what it is, see my first blog entry about it.
You could already get a different skin (presumably for editors) when you are visiting your Plone Site with an edit url, like edit.yourdomain.com.
Now you can also get that edit skin when you are logged in. More precisely: when you have the __ac cookie. This does not work when you are logged in with basic authentication, in other words: via the ZMI.
This is controlled by two new properties in the portal_properties/editskin_switcher.
- based_on_url: when True (the default) you get the behaviour described earlier.
- need_authentication: when True you need to be logged in before your skin is switched. By default this is set to False. This looks for the __ac cookie that Plone gives you when logged in. Note: logging in via the Zope Management Interface is handled without cookies, so the editskin switcher regards you as anonymous then.
You can combine the two behaviours if you want. If they are both True, then you need to have the right url and you need to be logged in.
When both are False, nothing happens: then you might as well simply uninstall this product as it is not useful.
Switch your skin based on the url
Announcing the new collective.editskinswitcher package
Collective Edit Skin Switcher
For a customer of Zest Software I created a package called collective.editskinswitcher. I gladly took some code from colleague Mark van Lent who did something similar for a different web site. The package is on the Cheese Shop so it can be easy installed. And the code is in the Plone collective.
What does it do?
Let's say you have a Plone Site. I tested this with Plone 3. I see no reason why it should fail on Plone 2.5. Maybe it even works on a CMF site. Anyway, whatever site you have is available on two urls: www.yourdomain.com and edit.yourdomain.com. Some day you should ask your local Apache guru how he did that.
With collective.editskinswitcher installed (with the portal quick installer), visitors that go to the website with the url edit.yourdomain.com will see the Plone Default skin, which is meant for content editors. Visitors to www.yourdomain.com will see whatever skin you have set as the default skin in portal_skins. Can be pretty handy.
Developer types probably like the fact that you also get the default skin when visiting localhost and the edit skin when you go to 127.0.0.1.
But maybe you want to turn it around: your visitors should see Plone Default and your editors should see your brilliant Monty Python Skin! Ni! Just go to the portal_properties, then editskin_switcher and change the edit_skin property to your dashing theme.
Why not CMFUrlSkinSwitcher?
I looked at CMFUrlSkinSwitcher first but it had not been touched in two years. One import error (CMFCorePermissions) could easily be fixed as that import was not even used. But after that tests were failing all over the place. Theoretically always fixable of course, but rolling an own package seemed easier, cleaner and faster.
Also, CMFUrlSkinSwitcher does some more things. At least it messes around with some methods like absolute_url. It could be that I find out later that this is necessary in collective.editskinswitcher too, but currently it does not look like that will be the case.
How do I know this is working?
The easiest way to test this package in a default plone site (apart from running the tests of course), is:
- Install collective.editskinswitcher.
- Go to portal_skins in the ZMI.
- Create a new skin selection based on Plone Default. In the tests I call this Monty Python Skin, so I will use that term here as well.
- Make Monty Python Skin the default skin.
- Remove the custom skin layer from Plone Default.
- Customize the main template or the logo or something else that is easy to spot.
- Visit 127.0.0.1:8080/plonesite and you will see default Plone.
- Visit localhost:8080/plonesite and you will see Plone with your customization.
On Linux you can edit /etc/hosts and add a line like:
127.0.0.1 edit.yourdomain.com www.yourdomain.com
Now visiting edit.yourdomain.com should give you default Plone and www.yourdomain.com should give you the customized Plone.
You can also let the edit urls begin with cms or manage. As long as the url is something like:
...//(edit|cms|manage).something.something....
you end up in the edit skin.
Have fun!
Maurits van Rees
Easily creating repeatable buildouts
When you create a buildout for a production website, you want to be able to recreate that exact buildout one year from now on a different server. How do you do that? This is a more simple and better working version of an earlier post.
Background
I first wrote a rather difficult guide to getting a stable, repeatable buildout. After some helpful comments from Martijn Faassen I could make this much simpler. I decided to keep the old weblog entry around: it could be helpful for people googling for strange errors in their buildout.
I assume you know what buildout is, otherwise you would probably not be reading this. Martin Aspeli has written a tutorial on Managing projects with zc.buildout. This should be your first stop for learning more about buildout, at least if you are a Plone developer. [Edit: I cannot find this tutorial anymore. Instead see for example buildout.org or an old manual for Plone.]
My brother and colleague Reinout has written a weblog entry about the Buildout development/production strategy that we are starting to use at Zest Software. The goal is to use the same buildout for both development and production deployment.
As the buildout should be repeatable, you may want to read the repeatable test file from the zc.buildout package.
Definitions
Now I am going to write about creating a really stable and repeatable buildout. In Reinout's story that would fit in with creating a stable.cfg. If you are not concerning yourself with development or if you have a different buildout for production deployment, this would just be the buildout.cfg file.
Let's begin with some perhaps arbitrary definitions.
With stable I mean: you run the buildout script in a directory; one year later you run buildout for the second time; you get the same result as one year earlier. Use case: by accident you removed the entire parts directory and you need to rebuild it.
With repeatable I mean: you run the buildout script in a directory; one year later you run the buildout on a completely different server; both directories are exactly the same. Use case: you want to move the zope instance of a customer to a new server.
Creating the buildout
What I write in this entry should be applicable to any buildout with or without Plone. My main focus and example is a Plone buildout though. You can fix up a current buildout of course, but here let's start with a clean default buildout for Plone 3, which you can create with paster:
paster create -t plone3_buildout test cd test python2.4 bootstrap.py
No newer packages
When running bin/buildout by default it picks the newest package it can find for any dependency. We can tell buildout to first look if some version of that package is already available:
[buildout] newest = false
This way, a second call of bin/buildout will not try to get any new packages. So it will finish quickly and just keep the current packages. This already makes the buildout stable in the sense that a new run of bin/buildout in that same directory will give you no new packages.
But if you want to recreate that buildout in a different directory or even a different server, you can get other versions. For starters, currently this buildout config will give you Plone 3.0.5, but a few months from now you will probably get Plone 3.1. So this is not yet a repeatable buildout.
Simple index
As an aside, while we are in the buildout section, let's quickly specify a simpler (read: faster) index than the default CheeseShop:
[buildout] newest = false index = http://download.zope.org/ppix
See my brother's weblog entry on ppix instead of pypi from now on.
By the way, this has nothing to do with making your buildout more stable or repeatable.
Pinning versions
Let's go get some more stability in our buildout. For some recipes and packages there are more ways to pin versions. But none of them work in all cases, except one. So we will just use that one way:
[buildout] ... versions = versions [versions] ...
So in your [buildout] section you add an option versions in which you point to another section that contains the versions that packages should be pinned to.
Pinning the plone version
First things first: we pick a Plone version. We simply add a line to the [versions] section that we added above:
[versions] plone.recipe.plone = 3.0.5
This recipe has strict versions for the Plone products and packages that make up a Plone release, so for example Products.CMFPlone, Products.CMFCore, the various plone.* packages, etcetera.
And this has a zope2-url property that tells the plone.recipe.zope2install recipe that it should use Zope 2.10.5.
So the most important versions have already been pinned now.
Knowing which versions are not pinned
Wichert Akkerman came up with the following one-liner to get a list of versions that are not pinned by you, but picked by buildout:
bin/buildout -Novvvvv |sed -ne 's/^Picked: //p' | sort | uniq
Currently this returns this list:
elementtree = 1.2.7-20070827-preview plone.recipe.distros = 1.3 plone.recipe.zope2install = 1.2 plone.recipe.zope2instance = 1.3 python-openid = 2.0.1 setuptools = 0.6c7 zc.recipe.egg = 1.0.0
This basically means that the mentioned packages are not pinned by our buildout config, but chosen (picked) by buildout. This means that they are not stable yet: rerunning this script in a few months time will likely give you different versions.
So what should you do? Very simple: you literally add those lines to the [versions] section. Now all your packages are pinned.
Pinning extra products
You can of course also pin any extra products that you want to use in your buildout. Best is to use an official release, such as a tar ball:
[productdistros]
recipe = plone.recipe.distros
urls =
http://plone.org/products/poi/releases/1.1/poi_1.1.tgz
or a checkout of a subversion tag:
[productcheckouts] recipe = infrae.subversion urls = http://svn.plone.org/svn/collective/eXtremeManagement/tags/1.5.2/ eXtremeManagement
If for some reason there is no tag you can use, you can still specify a revision in the url with the @ sign. Thanks to Guido Wesdorp for pointing this out:
[productcheckouts]
recipe = infrae.subversion
urls =
http://getpaid.googlecode.com/svn/trunk/products/PloneGetPaid@1132 PloneGetPaid
For now let's keep the example simple here and remove these products again from our buildout.cfg.
Conclusions
A buildout can be made very stable: just put newest = false in your buildout section. If one year later you accidentally remove the parts directory, you can rerun buildout and get your original directory back. If you remove your eggs though, you are in trouble.
Making a buildout repeatable is not that difficult either after all. Put this in your buildout.cfg:
[buildout] ... versions = versions [versions]
Right below [versions] add the result of this one-liner:
bin/buildout -Novvvvv |sed -ne 's/^Picked: //p' | sort | uniq
Contrary to some earlier conclusions that I drew, this is everything that is needed to pin all packages.
For pinning old-style Products, see the section on Pinning extra products.
Bonus
plone.recipe.plone rigourously depends on specific packages and products. So what do you do if you want to use a newer version of just one or two packages? For instance, I did some fixes for plone.locking which are still not in Plone 3.0.5. And for multilingual sites you really want to use a newer version with an important bug fix. You might at first think that this would be enough:
[versions] plone.locking = 1.0.5 plone.app.i18n = 1.0.2
But running bin/buildout then gives an error:
The version, 1.0.2, is not consistent with the requirement, 'plone.app.i18n==1.0.1'. While: Installing plone. Error: Bad version 1.0.2
So you need to add two lines to the [plone] section:
[plone]
recipe = plone.recipe.plone
eggs =
plone.locking
plone.app.i18n
The versions for these two packages are already specified in the recipe, but with different versions than we want. By listing those two eggs here, we tell the recipe to forget its version pinnings and be happy with any version of those two packages. After this, the pinnings for these two eggs in the [versions] section can finally take effect.
The final version
Let's end with showing what our buildout.cfg now looks like. Or actually, let's let's show a buildout that uses some earlier versions: Plone 3.0.2, older recipes, and older setuptools and elementtree. And instead of plone.app.i18n 1.0 that is required by Plone 3.0.2 we pin this package to a slightly newer version 1.0.1, which is not the latest version in the CheeseShop. This older buildout should show nicely that you can have a stable, repeatable buildout months later. Try it! So here is the file itself:
[buildout]
newest = false
index = http://download.zope.org/ppix
parts =
instance
zope2
plone
zopepy
productdistros
find-links =
http://dist.plone.org
http://download.zope.org/ppix/
http://download.zope.org/distribution/
http://effbot.org/downloads
# Add additional eggs here
# elementtree is required by Plone
eggs =
elementtree
versions = versions
[versions]
setuptools = 0.6c6
zc.recipe.egg = 1.0.0b6
plone.recipe.plone = 3.0.2
elementtree = 1.2.6-20050316
plone.recipe.distros = 0.3
plone.recipe.zope2install = 1.0
plone.recipe.zope2instance = 1.0
python-openid = 2.0.1
plone.app.i18n = 1.0.1
[plone]
recipe = plone.recipe.plone
eggs =
plone.app.i18n
[zope2]
recipe = plone.recipe.zope2install
url = ${plone:zope2-url}
[productdistros]
recipe = plone.recipe.distros
urls =
nested-packages =
version-suffix-packages =
[instance]
recipe = plone.recipe.zope2instance
zope2-location = ${zope2:location}
user = admin:
http-address = 8080
#debug-mode = on
#verbose-security = on
# If you want Zope to know about any additional eggs, list them here.
# This should include any development eggs you listed in develop-eggs above,
# e.g. eggs = ${buildout:eggs} ${plone:eggs} my.package
eggs =
${buildout:eggs}
${plone:eggs}
# If you want to register ZCML slugs for any packages, list them here.
# e.g. zcml = my.package my.other.package
zcml =
products =
${productdistros:location}
${plone:products}
[zopepy]
recipe = zc.recipe.egg
eggs = ${instance:eggs}
interpreter = zopepy
extra-paths = ${zope2:location}/lib/python
scripts = zopepy