Alessandro Molina - High Performance Web Applications with Python and TurboGears

See the PyGrunn website for more info about this one-day Python conference in Groningen, The Netherlands.

I am @__amol__ on Twitter. I am a member of the TurboGears team. I will talk about general rules which apply to any framework, some quick wins for TurboGears, some real cases, my personal experiences and preferences, feel free to disagree.

People seem obsessed with the raw speed of web servers. But you are not going to serve a "Hello world" page. My personal stack has nginx plus mod_wsgi or nginx plus Circus-Chaussette-gevent.

Try to avoid making everything an asynchronous (AJAX) request. Browsers have limited concurrency. HTPP has overhead. You will actually slow things down if you have too much of them. Your page may start fast but complete slow. Learn your framework and how you can use it the best way for your use case.

TurboGears is a framework for rapid development, encouraging flexibility. It was created in 2005, with 2.0 a major rewrite in 2009 to embrace the WSGI standard. It is based on object dispatching. You can use regular expressions for url matching, but they can get messy, so write them only when you must. By default an XML template engine with error detection. Declarative models with a transactional unit of work. It has built-in validation, authentication, authorization, caching, sessions, migration, etcetera.

Features versus speed: TurboGears is a full-stack framework. That makes it quite slow by default. You can switch things off that you do not need. The team invested effort to constantly speed it up since the 2.1 release. Keeping all the features around has its price, obviously. To cope with this, a minimal mode got introduced, that switches several things off that have the biggest influence on performance. You go from about 900 to 2100 requests per second.

Avoid serving static files in your framework. Let some other part of your stack handle this. This can take a lot of load from your application server.

Use caching. Caching means preorganizing your data the way you are going to use it. For example with a NoSQL database you can load the comments directly when accessing the page, so you don't need to load them separately. Frameworks usually provide various types of caching. Get to know them and use them.

Use HTML5 and Javascript. Invalidating your whole cache just to add a message "Welcome back, mister X" is not a good idea. Cache the result and use Javascript to do minor changes. If you are using varnish, nginx or any other frontend cache, consider using Javascript plus localstorage instead of cookies for trivial customizations, because cookies disrupt the cache.

Cache the result of rendering a template, with a cache key.

Entity caching: cache parts of your page, for example the html of one comment or notification.

Proactively update the page in your cache: when you edit the page, update the cache before your visitors ask for it.

If you are struggling too much with improving performance, you are probably doing something your application is not meant to do. Also, football fans are really eager for updates.

Offload work. Update the core cache to provide the author with an immediate feedback. Let some other process, program or thread handle the related background changes. For example, use something like Celery.

New Relic App Speed Index reports an average of 5.0 seconds of response time for accepted experience: http://newrelic.com/ASI If response time is less than 200 milliseconds, this is seen as 'right now'.

Sorry, there is no silver bullet for speeding up your application.