How to avoid getting a too new Plone version

Start

You have a website that is still running fine on Plone 3.0.6. (The story is the same or similar for Plone 2.x, 3.0.x and 3.1.x.) You have a buildout.cfg like this:

[buildout]
parts = instance
versions = versions
find-links =
    http://dist.plone.org
    http://dist.plone.org/thirdparty
eggs =
    elementtree

[versions]
plone.recipe.plone = 3.0.6
plone.recipe.zope2instance = 3.6

[plone]
recipe = plone.recipe.plone

[zope2]
recipe = plone.recipe.zope2install
fake-zope-eggs = true
url = ${plone:zope2-url}

[instance]
recipe = plone.recipe.zope2instance
zope2-location = ${zope2:location}
user = admin:secret
http-address = 8080
eggs =
    ${buildout:eggs}
    ${plone:eggs}
zcml =
products = ${plone:products}

Error

Now you want to add an issue tracker product to this website. Okay, you add Products.Poi to the eggs:

[buildout]
...
eggs =
    elementtree
    Products.Poi

You run bin/buildout again and the result is an error:

Uninstalling instance.
Updating plone.
Updating zope2.
Updating fake eggs
Installing instance.
While:
  Installing instance.
Error: There is a version conflict.
We already have: plone.app.layout 1.0.5
but Plone 4.0b1 requires 'plone.app.layout>=1.1.7dev-r23744'.

This is not the result that you hope to get. Seeing this error you might think you need to do something about that plone.app.layout version. But the real error here is that bin/buildout tries to install Plone version 4, where the buildout.cfg has Plone version 3.0.6. It does this, because the added Products.Poi package specifies a dependency on Plone 4, specifically Plone>=4.0a1.

So why does bin/buildout not quit with a more informative error, like:

Error: There is a version conflict. We already have: Plone 3.0.6 but Products.Poi requires 'Plone>=4.0a1'.

The thing is, bin/buildout has no way of knowing that you have version 3.0.6, because this Plone version is not "eggified". Simply speaking, what this means is that there is no link to Plone 3.0.6 on this page: http://pypi.python.org/simple/Plone. There you will only find Plone 3.2 and higher, as that is the first eggified Plone version. For Plone 3.0.x and 3.1.x, the plone.recipe.plone buildout recipe is used to pull Plone and the other not-eggified products into your buildout. But this does not help when a third party package has a dependency on Plone: bin/buildout will see this and pull in the latest Plone egg it can find, resulting in errors like above.

Fake eggs

So how do we solve this? We could try to fake having a Plone egg. plone.recipe.zope2install already fakes a Zope2 egg, and lots of zope.*, zope.app.*, and other eggs, next to its main task of actually installing Zope2. You could say that plone.recipe.plone is a workaround for the not-eggified Plone and ``plone.recipe.zope2install is a workaround for the not-eggified Zope2. So, how could you fake a Plone egg? You add Plone to the additional-fake-eggs option of the recipe:

[zope2]
recipe = plone.recipe.zope2install
fake-zope-eggs = true
url = ${plone:zope2-url}
additional-fake-eggs =
    Plone

If you comment out the Products.Poi egg in buildout.cfg for a moment and rerun bin/buildout you will now have a file fake-eggs/Plone/Plone.egg-info with these contents:

Metadata-Version: 1.0
Name: Plone
Version: 0.0

If you do not like that version string 0.0, you could specify a different version in the recipe config:

additional-fake-eggs =
    Plone = 3.0.6

But of course later you will decide to update your buildout.cfg to use plone.recipe.plone=3.1.7 and will forget to update the additional fake egg version, leading to confusion, so you might as will accept the default 0.0 version.

Anyway, we have a fake egg now, but does it actually help us? Errr... no. Make sure Products.Poi is listed in the eggs again and rerun buildout:

Error: There is a version conflict.
We already have: plone.app.layout 1.0.5
but Plone 4.0b1 requires 'plone.app.layout>=1.1.7dev-r23744'.

That is the same error we got earlier. So why did it not work? That is easy: the picked Products.Poi version (2.0a1 at the time of writing) depends on Plone>=4.0a1, like we mentioned before. Plone 0.0 is available in the buildout due to our additional fake egg, but this version does not meet the requirements of Products.Poi, so bin/buildout gets the latest available Plone version instead, leading to problems.

Our previous solution would have worked if the picked Products.Poi version would have specified a dependency on any Plone version (Plone) or for example any Plone version lower than 4 (Plone<=4.0dev). So it might have its merits, but it is not enough in our case. In fact, I can imagine that some add-on products specify a dependency on Plone expecting to get at least Plone version 3.2, as that is the first eggified Plone version. Having a fake Plone egg would go against that expectation and might cause problems. So if you use this, use it with care.

Pin Plone

Our problems is still not solved. But the solution is only one line away. We add a version pin for Plone:

[buildout]
...
versions = versions
...

[versions]
plone.recipe.plone = 3.0.6
plone.recipe.zope2instance = 3.6
Plone = 0.0

When we now run bin/buildout again with Products.Poi in the eggs, we get this error:

The version, 0.0, is not consistent with the requirement, 'Plone>=4.0a1'.
While:
  Installing instance.
Error: Bad version 0.0

This error message is still not ideal, but at least this does not lead us on a wild goose chase after plone.app.layout, but clearly points out that some package depends on a newer Plone version than we want.

Debugging

If you now run bin/buildout in verbose mode by adding one or more v options, you get more info, which hopefully helps in debugging this (bin/buildout -v):

Getting required 'collective.watcherlist>=0.2'
  required by Products.Poi 2.0a1.
We have the best distribution that satisfies 'collective.watcherlist>=0.2'.
Picked: collective.watcherlist = 0.2
Getting required 'collective.autopermission'
  required by Products.Poi 2.0a1.
We have the best distribution that satisfies 'collective.autopermission'.
Picked: collective.autopermission = 1.0b1
Getting required 'Products.DataGridField>=1.8a1'
  required by Products.Poi 2.0a1.
We have the best distribution that satisfies 'Products.DataGridField>=1.8a1'.
Picked: Products.DataGridField = 1.8a1
Getting required 'Products.AddRemoveWidget>=1.4.2'
  required by Products.Poi 2.0a1.
We have the best distribution that satisfies 'Products.AddRemoveWidget>=1.4.2'.
Picked: Products.AddRemoveWidget = 1.4.2
The version, 0.0, is not consistent with the requirement, 'Plone>=4.0a1'.
While:
  Installing instance.
Error: Bad version 0.0

At this point, without further information, I would probably work my way up from the bottom and start investigating if Products.AddRemoveWidget 1.4.2 specifies a dependency on Plone (which it does not), and then checking Products.Poi 2.0a1 (bingo). Of course in this case we do know that the buildout worked fine before and is throwing errors after adding Products.Poi to the eggs. So starting the investigation with Products.Poi would make the most sense here. But knowing where to start looking is a matter of knowledge, experience and luck. Also: do not change too many things at the same time: do not update to a new Plone and Zope version and at the same time add ten new products in your buildout.cfg; it will only confuse you when things start going wrong.

In this case, at http://pypi.python.org/pypi/Products.Poi you currently see a list pointing to versions 2.0a1 and 1.2.8. Version 1.2.8 does not specify a dependency on Plone (speaking as Poi maintainer: nor will future 1.2.x versions), so you pin that version and things should work again:

[versions]
plone.recipe.plone = 3.0.6
plone.recipe.zope2instance = 3.6
Plone = 0.0
Products.Poi = 1.2.8

Dump all picked versions

Indeed, running bin/buildout now works. This might change though when one of the other not-pinned packages is updated and has unwanted changes or starts depending on Plone 4. So to top this overly long example off, we will add the buildout.dumppickedversions buildout extension, run bin/buildout and copy the reported versions to our buildout.cfg, resulting in the following file:

[buildout]
extensions = buildout.dumppickedversions
parts = instance
versions = versions
find-links =
    http://dist.plone.org
    http://dist.plone.org/thirdparty
eggs =
    elementtree
    Products.Poi

[versions]
Plone = 0.0
Products.AddRemoveWidget = 1.4.2
Products.DataGridField = 1.6.1
Products.Poi = 1.2.8
buildout.dumppickedversions = 0.4
distribute = 0.6.10
elementtree = 1.2.7-20070827-preview
plone.recipe.distros = 1.5
plone.recipe.plone = 3.0.6
plone.recipe.zope2install = 3.2
plone.recipe.zope2instance = 3.6
python-openid = 2.0.1
zc.buildout = 1.4.3
zc.recipe.egg = 1.2.2

[plone]
recipe = plone.recipe.plone

[zope2]
recipe = plone.recipe.zope2install
fake-zope-eggs = true
url = ${plone:zope2-url}

[instance]
recipe = plone.recipe.zope2instance
zope2-location = ${zope2:location}
user = admin:secret
http-address = 8080
eggs =
    ${buildout:eggs}
    ${plone:eggs}
zcml =
products = ${plone:products}

Fake it after all?

Incidentally, if you have an add-on product that simply depends on Plone without a minimum version, you would now get an error like this when running bin/buildout:

Getting distribution for 'Plone==0.0'.
While:
  Installing instance.
  Getting distribution for 'Plone==0.0'.
Error: Couldn't find a distribution for 'Plone==0.0'.

If you understand what you are doing you can still use the fake egg trick from earlier to fix this. But probably you should instead figure out which add-on is pulling in Plone, and see if an earlier version of that add-on is available that works with your Plone version without having to resort to faking eggs.

Note that for proper fake eggs, you will want to use at least version 3.0 of plone.recipe.zope2install, and probably the latest 3.2.

Summary

To avoid new Plone versions from being pulled in by add-on products when you are on a not-eggified Plone version (2.x, 3.0.x, 3.1.x), just add a version pin for Plone:

[buildout]
...
versions = versions
...

[versions]
Plone = 0.0

This will not magically solve your buildout problems. Basically, this tells bin/buildout that it should quit with an error when a package is depending on a Plone version. Hopefully the lines before the error message have a hint about which package this is; otherwise try bin/buildout -v to get more verbose information.