Weblog
When your correctly configured portal tool is not working
Case in point: portal_transforms has a pdf_to_text transform but when indexing a pdf the transform is not found so SearchableText returns no content from the pdf file.
For a customer at Zest Software I am migrating a site from Plone 2.5 to Plone 3.1. In the migrated site I uploaded a pdf file. None of its contents ended up in the SearchableText index. In a fresh Plone Site in the same Zope instance this did work. In the portal_transforms tool the pdf_to_text transform was correctly registered. The mimetypes_registry looked okay. The pdftotext binary was available on the system. So everything looked fine, but did not actually work. What is going on?
Well, it turned out that the portal_transforms in the ZMI was not actually used. A getToolByName call was made which did not give back this tool but a utility. And the utility did not have the pdf_to_text transform. So I went to the Components tab in the ZMI of the Plone Site root. I removed the portal_transforms utility from the xml listed there and applied the changes. This made the pdf_to_text transform available again. Problem solved.
Note that this is the first time I edited the xml on the Components tab, so be careful if you do this: it may have adverse effects that I have not noticed yet; and I can imagine that typos are dangerous here.
So how did this go wrong? I did not explore this further, so I can only guess. I think during the migration from 2.5 to 3.1 the pdf_to_text binary was not available. Or there was some other reason why the transform did not work. During the migration the utility got added so it missed this transform. I removed the portal_transforms tool and added it again to get the missing transforms. At that point the utility and the tool were not linked to each other anymore. Again, it is a guess.
So, the conclusion of all this: if your portal tool does not work like you think it should, check the Components tab and see if a utility is registered under the same name. Removing it there may help. Note: keep a backup of your Data.fs when you do this and do not try it out on a production website but try it locally first.
Scrambled eggs
Broken python eggs won't hatch. [Update: there are patches.]
As Philipp von Weitershausen says, there is mostly no point to distributing .egg files. Indeed I have been creating source distributions (still called eggs usually) using python setup.py sdist for some months now. But those eggs can have their problems too, though some of those problems are shared with the .egg files.
Broken tar files on python 2.4
Python 2.4 has a tarfile module that breaks in some cases. If you unpack a tar file that contains a directory path with exactly 100 characters the unpacking will fail. Marius Gedminas helpfully fished up some bug reports when I asked about this on the distutils-sig mailing list a few weeks ago. Read that complete thread for a bit more info.
The result is that an sdist of my.package in tar.gz format can work fine in version 1.2, fail in 1.2.3 and work again in 1.2.3.4: the path that has a slash at position 100 in 1.2.3 will have that slash at position 98 in version 1.2 and at 102 in 1.2.3.4.
Solution: use the zip format when creating an sdist. You can do that with python2.4 setup.py sdist --formats=zip or create a file setup.cfg next to setup.py with these contents:
[sdist] # guard against tarfile bug by using the zip format formats = zip
Until now I am still using the tar.gz format that is the default (on Linux) and only create .zip files when it is needed. But I am not sure if there is a really a good reason to stick to tar.gz.
Setuptools and subversion 1.5
When the server that contains your subversion code is using subversion 1.4 and you have subversion 1.5 on your computer, setuptools fails with a missing python import. See this thread on distutils-sig and the blog from Mr. Topf. Both have some solutions too.
You can hit that problem while easy_installing something or when running buildout. There may be other version combinations or actions that trigger the error. At least I think I saw this with subversion 1.4 too, but could not reproduce it later.
Actually, that bug is only important when installing an existing egg. But creating an egg can also fail; and that also has to do with subversion 1.5 too. This calls for a new section.
Missing files with subversion 1.5
I just tried creating an sdist of the new xm.tracker package (an optional package for Products.eXtremeManagement, adding a time tracker).
With subversion 1.4 a perfect tar ball is created. With subversion 1.5 about half the files are missing, for example the version.txt file. Creating a .zip file instead does not help, so it is not the tar file bug mentioned earlier. And in fact, creating a .egg file does not help either. Using python 2.5 instead of 2.4 has no effect.
I have not heard reports of this yet, so there may be something weird in this particular package. If you want to try creating an sdist yourself, do an svn checkout of http://svn.plone.org/svn/collective/xm.tracker/tags/0.2
In this case I fixed it by logging into a server that still had subversion 1.4 and creating the sdist there.
Update: as colleague Mark van Lent writes there is a patched version of setuptools available that fixes this problem. Or use easy_install setuptools==dev06.
Conclusion
Entirely too much can go wrong when creating an sdist. And with .eggs you are not safe from harm either. Like I already said in a previous log entry, you need to check that egg!
So after you create it, try easy_installing it. Or for Zope/Plone packages: create a buildout configuration that uses this released egg and run the automated tests and try to start Zope with it. I just started doing that last week with the released.cfg file in the plonehrm buildout.
Translations of portal status messages in KSS
What do you do when KSS does not translate your portal status messages? And what if Dutch translations are shown when you expect English?
You have a package that has some translations. One of those translations is used in a portal status message that gets shown by a KSS action. Something like this, taken from the xm.tracker package:
from kss.core import kssaction
from plone.app.kss.plonekssview import PloneKSSView
from zope.i18nmessageid import MessageFactory
_ = MessageFactory('tracker')
class KSSStart(PloneKSSView):
"""kss view for starting the timer"""
@kssaction
def start_timer(self):
message = _(u'msg_started_timer',
default=u'Started the timer')
plone = self.getCommandSet("plone")
plone.issuePortalMessage(message)
But -- tragedy of tragedies! -- it gets shown in English even though you have a fine Dutch translation and your Plone Site has Dutch as default language. What can possibly be wrong? Two things really.
plone.app.kss
For translations of portal messages you need plone.app.kss 1.4.3 or later. Check your Plone version. If it is Plone 3.1.5 or later you are fine. If you have 3.1.4 or earlier you need to upgrade Plone or update the plone.app.kss package.
Products/*/i18n
Are translations still not showing up? Then your package is probably in the Products name space and your translations are in an i18n folder. That directory is the domain of the PlacelessTranslationService from Plone. Those translations are not found by the zope.i18n translation machinery, which is in effect in plone.app.kss. You may encounter the same problem when you are using pure Zope 3 page templates.
To fix it, you need to add a few lines to your product. For example, I just added this in Products/eXtremeManagement/__init__.py:
from Products.PlacelessTranslationService.utility import PTSTranslationDomain
xmdomain = PTSTranslationDomain('eXtremeManagement')
Then register that as a utility in your configure.zcml:
In the next major Plone release (4.0) this will not be needed anymore, but that is still quite a way off. You will probably be using locales then anyway.
I want English!
While I am writing about translations, here is something else I bumped into. Take a Plone Site with Dutch as default language and English as an allowed language. Take a package with a registered locales directory. It has Dutch translations in the locales/nl/LC_MESSAGES directory. Now visit that site with English as the preferred (or only) language of your browser. With the proper settings in the portal_languages tool of your Plone Site, this should present the user interface mostly in English. But: the translations for your package will be shown in Dutch! What is wrong here?
The problem is that the translation machinery does not know that the default strings (message ids) of your package are in English. It might as well be in Chinese and it refuses to guess. So what you do is: create a locales/en/LC_MESSAGES directory, copy your mydomain.pot file there and change its name to mydomain.po. You probably want to change some of the headers of that file slightly. Restart Zope and you should be able to enjoy the beautiful English language again. Tally ho!
Refreshing content with KSS
Give your content a refreshing KiSS!
I just uploaded a how-to on kssproject.org about refreshing content with KSS. Read that if you are interested in the goal of that how-to:
You are looking at a page on your Plone Site. You change the review state of the current object. Without a complete page reload you want some bits of the page to change automatically. Plone itself is doing that already. You want to have that kind of fun too!
Omelette: now even tastier!
collective.recipe.omelette is a must.
David Glick has just released collective.recipe.omelette version 0.5. You have got to try it. As you may know, it makes a parts/omelette/ directory with links to all the various eggs that you have in your buildout. With version 0.5 it can do the same for Products. And you can point it to some more packages. In a standard Plone buildout you would have this part:
[omelette]
recipe = collective.recipe.omelette
eggs = ${instance:eggs}
products = ${instance:products}
packages = ${zope2:location}/lib/python ./
Don't forget the ./ at the end. Note: the README.txt of the recipe mentions modules instead of packages but that is a typo. Fixed in subversion.
I especially like it that with this configuration you can look in parts/omelette/Products and have links to the exact same Products that your zope instance will be using, whether a Product is in an egg, in the productsdistros/checkouts directory, in parts/plone or in the Zope 2 Products dir. That is sweet.
Thanks a lot David!