Plone
This is here to serve as contents for the atom/rss feed for Plone, also read by planet.plone.org.
Summaries Python Users Netherlands meeting in Utrecht
Meeting of the Python Users Netherlands group. This evening was organised by Nelen & Schuurmans in Utrecht on 16 February 2011.
My brother Reinout (who was the main organiser) also has summaries.
Christo Butcher (Fox IT): Trac with python
Sorry, I came in late and missed the first part of this talk and was eating during the second part. :-)
Maurits van Rees (Zest Software): Theming with xdv, Paste and fanstatic
Since a few days I use xdv, Paste and fanstatic to put a new theme around my old Plone 2.5 website. The code is available on github and can be instructive as an example of these technologies: https://github.com/mauritsvanrees/maurits-site-xdv
See the slides.
Jan-Wijbrand Kolman (The Health Agency): Publish your changes
I do a lot of releases, especially for Grok and the Zope Toolkit. So I look a lot at the Python Package Index (PyPI) to see which changes have been made in packages.
Quote: "Zope Community and Friends: I love you so much for putting changelogs in your PyPI info." (jshell)
Compare e.g. Sphinx and grokcore.component on PyPI. Sphinx hardly says anything on its PyPI page. grokcore.component is much smaller but publishes a lot more info right there on PyPI, especially its changelog. Add that to your packages and give them a meaningful package description. It is probably the first page someone sees for most packages, so you had better leave a good first impression.
Jasper Spaans (Fox IT): pypy rocks, even in the real world
(Note: pypy is not PyPI.) Pypy puts a python in your pythons. It is a python interpreter implemented in RPython, restricted python. You can do memory management also in RPython, so you don't need to handle, understand and write C code if you need your own specific memory management. It should be a drop-in replacement for normal python (CPython) in most cases. It offers a possibility for a JIT optimizer, which can make your code faster.
We tried pypy a few weeks ago. We encountered some SyntaxErrors, as version 1.4.1 only supports python 2.5). Class decorators are not supported yet (so we decorated manually). The with-statement needs to be imported from __future__. Memory usage exploded (fixed by changing to the hybrid garbage collection). SQLAlchemy was too hairy; for that part we put in a bit of standard python to handle the SQLAlchemy integration, together with protobuf.
Several of our performance tests showed improvements of 7 to 13 percent when using pypy. The memory usage was 20 percent higher though.
Reinout van Rees (Nelen & Schuurmans): Geographic information websites for water management
Our customers are mostly government agencies working in water management. They have a lot of information about water. We make that info available in a format that is easier for them, on a website. Customers can click through maps in our application to get info. What technology do we use? Lots of python!
- Mapnik: uses the rendering engine from Openstreetmap. You have points, lines and grids. It understands the WMS standard for web mapping. It gives a .png file as output.
- Gdal: used behind the scenes mostly, for grids.
- Matplotlib: a graph library. It can do everything. Others are easier, but wit limitations. Several customes have too many peculiar wishes that are difficult to do with other packages, if it is possible at all. (Remark from the audience: use protovis.)
- Pyproj. You have various measurements for where on the globe you are: rijksdriehoek (in the Netherlands), degrees, Google mercator. Pyproj translates between these.
- On the client side we use jquery, openlayers, and the blueprint css framework.
We have a layered software structure:
- core: splitting everything up.
- lizard-ui: defines the html page structure.
- lizard-map: this is the big python/django application that we have, the core of the map. Map visualisation; basic graph and search handling; basic popups; plugin mechanism for adapters (using extra attributes in html5) with which you can search, get a layer and show the corresponding html in a popup.
Organisation:
- We have collective code ownership. Everyone can touch all the code. For most of the code everyone has a basic understanding of what is going on. We want code that conforms to the pep8 and pyflakes standards. For automatically running the tests we use Jenkins (formerly known as Hudson). We borrow each others brains: sit with each other to check ideas that you have for sanity.
- Documentation is generated by Sphinx.
Our clients are mostly governmental and they like it when they use open source, so we have open sourced our code, also because it uses a lot of open source itself. This also attracts me to the company.
See http://doc.lizardsystem.nl/ and http://reinout.vanrees.org/weblog/
We want you! Business is booming! See http://www.nelen-schuurmans.nl/
Coen Nengerman (Nelen & Schuurmans): ArcGIS
I studied hydrology; I am more a python user, not a developer. ArcGIS is a professional tool to manage your geographical information. It makes lots of info and tools available in a nice UI. Catalog, tools, python scripts using e.g. scipy and numpy, including an integrated python command line. It can find maps online as well, like buienradar or maps of the flooding in Pakistan.
Jan-Jaap Driessen (The Health Agency): fanstatic
Fanstatic is a python package for resource management: css and javascript files. I am using zope and grok. Getting resources from zope can be slow, because lots of checks are done, for example authentication and authorization, which is not always needed. Usually you include too much resources on all pages, because you do not know for sure which of them you need for a specific page. Fanstatic fixes this for you. The fanstatic middleware will figure out which resources you really need and serve them to you.
You could download a jquery package from somewhere and put it in your own code, but we want to create python packages for this so you can easy_install them (js.jquery, js.yui). There is no real management of dependencies in the javascript world currently like there is for python, so for now we create python packages that define these dependencies.
Fanstatic minimizes http requests by doing rollup, bundling, using a cdn (content delivery network that hosts common javascript resources) if available, and versioning a resource so it can be cached for ten years in a browser without getting stale: when you change the resource a version with a new id is requested in the html. It takes hints from Steve Souders at http://developer.yahoo.com/performance/rules.html
On the roadmap (some already on branches): bundling, cdn, compile for minification (google closure compiler), lazy loading (google loader, requirejs), commonjs.
Nico de Groot (Tilburg School of Theology): web2py
web2py is easy to learn, stable, secure, light weight, faster than zope ;-), can use different kinds of databases, can run inside Google Apps, some mercurial integration.
At the top you have applications consisting of plugins. In the middle there is gluon (old name of web2py) and general libraries like simplejson, and at the bottom the rocket server.
Jean-Paul Ladage (Zest Software): Prettig Personeel
Prettig Personeel is an online human resource management system created by Zest Software using Plone. You can generate contracts, get reminders that contracts are ending or for birthdays or for people who have been ill for a long time.
When you view an employee in this web application there are lots of boxes with info in which you can view, add and edit contracts, etc. We developed jquery.pyproxy to handle this. With that, you can use jquery within python (works in Plone and Django). You can use all manipulation and effects of the jQuery API. The result is returned as an xml response for an AJAX call so it gets processed on the client side in the browser.
It's on pypi and github:
Quickly change nginx configs
Shell script to change nginx configs to ignore logins and cookies.
See these pages for info on why you may want this: http://plone.org/products/plone/security/advisories/cve-2011-0720 and http://plone.org/documentation/kb/disable-logins-for-a-plone-site
If you want to change lots of nginx config files to temporarily switch off login (authentication) and cookies, you can use this bash script at your own risk:
#! /bin/bash
# Note: /bin/sh would be better, but at least when that points to
# /bin/dash it complains about some of my usage of 'test'.
cat < $CONFFILE
fi
fi
fi
done
echo "Do not forget to reload or restart nginx after changes."
Permission problems for objects that have a non-existing owner
And show how to change ownership in Plone.
I had problems at a client site using Plone 3.3 and quintagroup.plonecomments 4.1.2. I do not think the problem is in that last package, but that is the spot where strange behaviour surfaced. The client reported that on some pages the comments were visible but the buttons to Remove or Edit them was not visible, not even for a Manager.
The comments and their buttons were shown by the plone.comments viewlet, which is customized by quintagroup.plonecomments. The Remove button was guarded by a permission 'Moderate Discussion'. The page template claimed that as Manager I did not have this permission. A simple Script Python added in the ZMI showed that this was non-sense: I did have that permission:
return context.portal_membership.checkPermission(
'Moderate Discussion', context)
In the end, what turned out to be the problem was that the object was owned by a user that no longer existed. Somehow that seems to have tricked Plone/Zope into thinking (at least in the template of this viewlet) that no one had the Moderate Discussion permission anymore. Other permissions seemed unaffected; for example you could still view the comments and reply to them.
Giving someone the local role Owner in the ZMI, or changing the Creator in the Ownership tab of the edit form did not have any effect. I had to add 'ownership_form' at the end of the url of the object to give ownership of this item (and its subobjects) to someone else. This fixed it, even when I myself was not the new owner.
So it looks like somewhere some code path was triggered that did not like the fact that the current owner did not exist anymore.
Now, maybe something weird was happening because the plone.comments viewlet was customized in the portal_view_customizations, but this worked without problems on a local older copy of the Data.fs. I still do not understand what the difference is between the live site and my local copy where I could remove comments everywhere just fine, even though the owner was gone there as well. Well, stranger things have happened in this particular Data.fs and I have come up with stranger solutions to tackle them. This one was quite clean. :-)
Python Users Netherlands meeting
Summary of the Python Users Nederland meeting in Arnhem, 3 november 2010.
Rob Gietema: Deco
I will talk about Deco and some more of Plone 4. I work for Four Digits, who organized this PUN. Originally four people, now seven. Since a few years we do only Plone. I myself do mostly front-end development, like integration of the TinyMCE visual editor in Plone.
Plone is a CMS. One of the current problems is creating composite pages. There are add-ons for that, but it is still crufty. Deco is meant to improve that a lot. In 2008 several people came with a proposal for that. Deco as front-end, Blocks as back-end.
Deco is a grid-based system. So you create multiple columns that you can style with CSS. The columns can be filled with tiles. We have structure tiles, like rich text. Also field tiles, like a title, description, list of tags. And application tiles, which can be anything: image, discussion forum, table of contents, multi-file upload.
The idea was good, but there were not a lot of people who wanted to write the javascript needed for that. But I do like to do that. :-)
We sprinted a few times on this. Four Digits hosted the Living Statues Sprint, there was a sprint at the Plone Conference in Bristol last week.
We created a demo website using Deco: http://decobrewery.com/. I will demo that now.
This is mostly for Plone 5, but parts will be available in upcoming Plone 4.x versions.
The parts involved in Deco are currently split over about 14 small python packages.
Create your own tile: an interface class with a schema, a class with a __call__ method that returns some html (can be in an html template), and some zcml to register this and glue everything together.
To do: create more templates for content types, fix bugs.
Want to code on it? Use https://svn.plone.org/svn/plone/plone.app.deco/buildouts/dev
It actually works in IE6 as well!
Reinout van Rees: checkoutmanager
checkoutmanager is a small tool to manage all your checkouts on your whole file system. You have 20 personal projects, 30 client projects, some dotfiles in your home dir. Just manage them all. I created checkoutmanager when I switched to a new laptop. Use it to checkout git, subversion, bazaard, mercurial. Every morning I do a checkoutmanager up to update all checkouts. I do checkoutmanager st to see which files in all my checkouts I forgot to commit. Also checkoutmanager out to see which local commits from git or mercurial I have not pushed to the central server yet. (To do: mention this in the docs.)
Roel Bruggink: Subversion pre-commit hooks for pyflakes, pep8
I installed pre-commit hooks on our subversion to disallow commits that go against pyflakes and pep8. I wrote some code that made it possible to add a comment in a file to ignore one or more PEP8 style checks. Hooked it up to TextMate to prevent you from saving a file with PEP8 problems, except when you explicitly specify it with that comment. Handy for those times where you really can't help it. If you want the code, mail me.
Lars van de Kerkhof: Hudson and Fabric
Hudson is a Java tool for continuous integration. It basically just runs a shell script for you and reports the results back. Usually those are test results. We use it for deployment as well, but of course only when the test requirements are met. We let it use Fabric for that. It is tricky to get Fabric to use the correct virtualenv, but we got that working. I also made a way to print a shell script that looks like what Fabric would do for real.
Rob Gietema: XDV/Diaza/Deliverance for theming
XDV uses XSLT to get html from for example Plone and transform and push it into an html template that your designer has created. Nice alternative way to do theming. You can also combine multiple sources, not just from Plone, but from other systems as well at the same time.
Varundev Kashimath: Why do certain products succeed?
Working at TU Delft. We did some research into how long it takes to do a project with Rails and with Django. Rails took longer in this case. Some discussion followed, hard to summarize.
Vincent Pretre (Zest Software): jquery.pyproxy
Lightning talk during the Dutch Plone User Day 2010 in the Euromast, Rotterdam. Also, cheap way of pointing Dutch readers to the Dutch summary of the whole day.
Vincent Pretre is lead developer at Zest Software for http://prettigpersoneel.nl, a hosted service for Human Resource Management for small to medium sized companies. The site uses Products.plonehrm, collective.sendaspdf and: jquery.pyproxy.
The idea is to replace KSS with jQuery for Ajax requests. Modify the DOM with python. We wanted it to be easy to use on client (browser) and server side. Keep jQuery syntax as close as possible. It should also be as extensible as jQuery, so existing plugins should work. It can be used as jQuery plugin.
pyproxy binds a call to an event. There is also pyproxy_call. In python: @jquery defines a view as callable by jquery.pyproxy. JQueryProxy: an object used to modify the DOM. extend_grammar: defines plugins syntax.
Current limitations: you cannot do a chained call, cannot save a query, cannot have functions as parameters (so no callbacks).
It is stable: used on production for months. Works with Plone and Django (can be extended to other frameworks), works with Firefox, Chrome, IE, WebKit.
The product is hosted on github. See the README there. Releases are available on PyPI (0.2 released on 22 October 2010).
See the slides.
More Dutch
But wait, there is more... if you are Dutch! I have a Dutch overview of all the lightning talks and a Dutch summary of all the talks.