Python User group Netherlands meeting

published Sep 25, 2009

The three-monthly meeting of the Dutch python user group was held Thursday 24 September 2009 in Amsterdam, in the Beurs van Berlage, organised by Go2People. First a half-hour presentation about Phatch, then some lightning talks, another half-hour presentation on MonetDB and moving from python 2 to python 3, then a few more lightning talks.

My brother Reinout was also there and of course he wrote a summary too.

Stani: Phatch - Batch photo processing for everyone

Lots of photo data, including metadata. You have ImageMagick. Phatch allows you to resize images, do colour profiling. Combine actions. Prefabricated recipes. You don't need to know how to do graphics manipulation. You can remove metadata so you can upload photos anonymously to sites. You can have a droplet on your desktop; just drop an image on it and it will do the actions you specified. Safe and unsafe mode. Safe restricts you; in unsafe mode you have the full power of python. It's a command line tool, but also a UI. Uses open source fonts. Phatch can do anything Gimp and ImageMagick does and more as they are the back ends, including PIL and blender. Its goal is to unite all open source graphics tools. Anyone can submit new objects. Works on Linux and mostly on Windows and Mac, but we need more developers there. About six developers now. Someone has plans to work on Django integration. Link: http://photobatch.stani.be/

Jasper Spaans, Fox-IT: Pycuda - massively parallel computing

Calculating with the GPU (Graphical Processing Units). GPU's are incredibly fast for parallel computations compared to their price. Cuda is only nor Nvidia. OpenCL does that too, and that can also use ATI/AMD. Use it for parallel algorithms. PyCuda is a python interface for cuda. Slides in Dutch: http://jasper.es/talks/pun2009-09

Wim Feijen, Go2People: Bazaar (and mercurial and git)

Bazaar: you can have one or more repositories. You can work and commit locally. It's simple. It tracks movements. You can branch projects. Look at http://github.com (not only for git users) and http://bitbucket.org, for free project hosting.

Remco Wendt, Maykin Media: libraries that you love

Does anyone know of a library that is cool to use? Speak up about it!

Jan Jaap Driessen (Health Agency) and Sylvain Viollon (Infrae): Grok

Web framework, based on Zope. Even cavemen can use Zope. Just had a sprint about Grok. Takes the pain out of working with Zope. Zope has lots of layers, configuration, zcml, adapters, whatever. With Grok you do not register your stuff in zcml. Set of default settings. Grok "groks" your code, understands your code without needing a separate configuration language. Grok 1.0 is due for release this or next week. We have been using it for three years already. http://grok.zope.org

Gijs Molenaar, UvA, CWI: MonetDB and python 3

DBMS, developed by CWI, open source, BSD, platform for scientific research, not only SQL, http://www.monetdb.nl. Good for big databases; SkyServer is 6TB. For query intensive apps that are mostly read-only. For complex data models beyond SQL. Sometimes outperforms other databases by a hundred times for big tables.

It is a column based storage database. All columns are shown separately. Flexible kernel, which is easily extensible. SQL front end, but not limited to SQL; can use Xquery.

You can store xml documents in your database. Monet takes the document apart and stores it in several tables.

There are APIs for Ruby, PHP, python, perl (needs a rewrite but nobody wants to touch it), C, java. The CWI at the University of Amsterdam wanted me to rewrite the python API. MonetDB API is the wire protocol. Basic functions: set up connection, execute commands, parse the response, chop queries and responses in blocks. python.sql is the implementation of all DBAPI 2.0 functionality. Bridge between this and MonetAPI (mapi).

There were tests available for the MySQL python API, so I borrowed those for Monet. We looked at supporting python 2 and 3. We did not want to use two trees. We decided to do a 2+3 hybrid, though Guido van Rossum tells us not to do that. In python 3 there is a new IO layer with bytes and strings. New IO layer is used in the mapi layer. We made two versions here, mapi2.py and mapi3.py as it was too hard to combine, with exceptions that were no longer there in python 3. We do a python 3 version check in only two places, like conditionally importing mapi2 or mapi3, so not too bad. Tip: don't use print, use logging.

The hybrid python2-3 model works quite well in our case as the API is quite small. We minimised code duplication. Not advised for all projects.

Future work: bug fixes, port the xquery api, currently I am rewriting the C api (mapilite), unit tests in python using ctypes, having problem with structs.

Something completely different. http://pythonic.nl non-profit amateur web hosting provider for python web development. Focus on django.

Question: How mature is MonetDB? Well, there have been cases of data loss, but it has gotten better the last few years and it has been used in various projects.

Slides: http://gijs.pythonic.nl/static/pun-090924.pdf

Tim Molendijk: debugging

I usually just hunt for a bug, kill it and forget how I did the debugging and not improve it next time. But I am learning. I should probably use the python debugger, but usually I start with adding a few print statements. But with WSGI applications that normally does not work. So yes, next step is (i)pdb. For Django: Django debug toolbar.

Suggestions from the crowd: post mortem debugger, Twill, zc.testbrowser, mechanize, Selenium, winpdb, pydev for eclipse (you can still use vim for writing the code if you have to), werkzeug (for django).

Maurits van Rees: international emails

Short version of a presentation I gave at my employer Zest Software and that I blogged about recently. Suggestion from Jasper Spaans: use lamson (mentioned above), an smtp server framework in python.

Keywords
grok plone zest