Scrambled eggs

published Sep 20, 2008

Broken python eggs won't hatch. [Update: there are patches.]

As Philipp von Weitershausen says, there is mostly no point to distributing .egg files. Indeed I have been creating source distributions (still called eggs usually) using python setup.py sdist for some months now. But those eggs can have their problems too, though some of those problems are shared with the .egg files.

Broken tar files on python 2.4

Python 2.4 has a tarfile module that breaks in some cases. If you unpack a tar file that contains a directory path with exactly 100 characters the unpacking will fail. Marius Gedminas helpfully fished up some bug reports when I asked about this on the distutils-sig mailing list a few weeks ago. Read that complete thread for a bit more info.

The result is that an sdist of my.package in tar.gz format can work fine in version 1.2, fail in 1.2.3 and work again in 1.2.3.4: the path that has a slash at position 100 in 1.2.3 will have that slash at position 98 in version 1.2 and at 102 in 1.2.3.4.

Solution: use the zip format when creating an sdist. You can do that with python2.4 setup.py sdist --formats=zip or create a file setup.cfg next to setup.py with these contents:

[sdist]
# guard against tarfile bug by using the zip format
formats = zip

Until now I am still using the tar.gz format that is the default (on Linux) and only create .zip files when it is needed. But I am not sure if there is a really a good reason to stick to tar.gz.

Setuptools and subversion 1.5

When the server that contains your subversion code is using subversion 1.4 and you have subversion 1.5 on your computer, setuptools fails with a missing python import. See this thread on distutils-sig and the blog from Mr. Topf. Both have some solutions too.

You can hit that problem while easy_installing something or when running buildout. There may be other version combinations or actions that trigger the error. At least I think I saw this with subversion 1.4 too, but could not reproduce it later.

Actually, that bug is only important when installing an existing egg. But creating an egg can also fail; and that also has to do with subversion 1.5 too. This calls for a new section.

Missing files with subversion 1.5

I just tried creating an sdist of the new xm.tracker package (an optional package for Products.eXtremeManagement, adding a time tracker).

With subversion 1.4 a perfect tar ball is created. With subversion 1.5 about half the files are missing, for example the version.txt file. Creating a .zip file instead does not help, so it is not the tar file bug mentioned earlier. And in fact, creating a .egg file does not help either. Using python 2.5 instead of 2.4 has no effect.

I have not heard reports of this yet, so there may be something weird in this particular package. If you want to try creating an sdist yourself, do an svn checkout of http://svn.plone.org/svn/collective/xm.tracker/tags/0.2

In this case I fixed it by logging into a server that still had subversion 1.4 and creating the sdist there.

Update: as colleague Mark van Lent writes there is a patched version of setuptools available that fixes this problem. Or use easy_install setuptools==dev06.

Conclusion

Entirely too much can go wrong when creating an sdist. And with .eggs you are not safe from harm either. Like I already said in a previous log entry, you need to check that egg!

So after you create it, try easy_installing it. Or for Zope/Plone packages: create a buildout configuration that uses this released egg and run the automated tests and try to start Zope with it. I just started doing that last week with the released.cfg file in the plonehrm buildout.

Keywords
plone xm