Python Users Netherlands meeting 20 October 2011

Jan-Jaap Driessen: Connecting the bots

Now for something more technical. We use buildbot to test our software. We use Selenium (using gocept.selenium) to test in a real web browser, including javascript. Instead of checking the end result of clicking with Selenium by going to a page, you can also check the database directly. We have a local browser farm, so with several versions of several browsers on several operating systems. You can try setting up something yourself or use the saucelabs paid service for this. So we have this setup:

zc.buildout + buildbot + selenium = nightly testing on all browsers

When this is running on saucelabs you can actually see the screen of the browser that is being tested at that moment! You can also see a video of it later and share that with your client to show that your software is working or that this particular bug has been fixed. The saucelabs service is pretty cheap actually, cheaper than setting up some hosts in the cloud yourself and keeping them running.

Boaz Leskes: PyCounters

PyCounters is a light weight library to monitor performance and events in production systems. We started this project because we had a performance problem. We had lots of layers in the system, so it was challenging to see where the bottleneck is. PyCounters gives us that. It is a combination of light weight events, counters to collect needed statistics, and reporters to regularly output statistics. Short example to see how many times per second a function is being called:

from pycounters.shortcuts import frequency

@frequency("req_per_sec")
def serve_a_request():
    """do something interesting and serve a request"""
    pass

Some possibilities:

• We can calculate frequencies, average values per time, simply counting.
• There is JSON file output.
• A helper class for Munin plugins.
• Support for counting what happens in a single thread or for using multiple processes or servers.

Future plans:

• Using context managers (the with statement) is now planned, as decorators are not always fine grained enough.
• Django specific counters.
• More output formats (suggestions welcome).

See documentation at http://pycounters.readthedocs.org/ and source code at https://bitbucket.org/bleskes/pycounters

Note from Maurits: do not confuse this with the pycounters module of François Beausoleil which does something different, though it seems to have overlapping functionality.

Steve Alexander: Celebration, Motivation, Distribution

"Now we're talking." I was a python programmer, then I worked for Launchpad, then Canonical (making Ubuntu), then I did acting classes and stuff. Then I started a company to talk about how we work. We have really complicated brains. I am using a metaphor: the hero's journey. George Lucas used that same story to create the Star Wars movies. When we go to work we also go on a journey and we should enjoy it.

Some of us work alone, some in a community or team. Who is my team? What is my value to the team? We need to achieve a goal, for example fixing a bug or other issue somewhere in a tracker. You fix or create something and that means you add value to the team. That is the outer journey. There is also an inner journey, some kind of transformation that goes on. (Insert some Star Wars and Indiana Jones jokes here.) You have some personal 'Aha!' moments, where you discover amazing things, or that you can do amazing things. What those moments tell you is your team's value to you.

I wrote a blog article about celebrating the hero's journey. See http://blog.nowtalking.nl/archives/55. Lots of different reactions are possible from colleagues or clients when you tell them "Hey, I fixed a bug."

Frank Niessink: Ran 160242 tests in 13.716s

Task Coach: your friendly task manager. This is a project that has been going on since 2005. Done in Python of course, and some Objective C. It is open source. Runs on Windows, Mac OS X, Linux, BSD, iPhone. We try to release often; last year about twice a month. We have quite some downloads, in total at least 2 million downloads. More than 4000 users use it daily. Over 4500 unit tests and with 40 languages over 160,000 language tests. Translatable strings can contain keyboard shortcuts; translating this wrongly (e.g. translating Ctrl into German as Strg instead of leaving it at Ctrl) can actually crash the program. We have generated a unit test for each translation, like checking whether a translator forgot to add ... when necessary or indeed checking for program crashes.

Vincent Pretre: collective.autosaveform

collective.autosaveform is an open source project for Plone. A user has entered a lot of data in a web form, his internet connection quits and he gets mad because he has lost everything. With collective.autosaveform we save the form every five seconds or every click. We use localstorage for this in the browser, supported by IE8 and higher and normal browsers. After ten local saves we also send the data to the server so in case the browser crashes or something like that the data can still be made available later. When something bad has happened and the user is loading the form again, we compare the local and remote database versions and use the most recent. You need to register the form for autosaving in Plone with a few lines, and use one jQuery line in your javascript. It's configurable, friendly on the server and for the user, supported on multiple browsers, supports all HTML inputs except file. It should be pretty doable to make it work in Django.

See the slides.

Sylvain Viollon: context managers and generators to work on lots of objects

This is code that is hard to summarize, but you can do interesting things with context managers and generators, at least with Python 2.6 and higher. You can save some memory but more importantly you can save a lot of lines of code with good use of these utilities.

See his code at http://pastebin.com/hgqiSTVz. The long stretches of white space between the code parts should be seen as separating different versions of the code.

Remco Wendt: showing two python tools

vcprompt: show in the command line prompt which version control system (vcs) you are using, which branch, etc. There is a binary one, but you should use the Python one.

dotfiles: define a repository of dotfiles that you have in a vcs, show what dotfiles you have managed already and allow you to sync or override them, especially handy when on a new system.