Weblog
Python web meetup Netherlands
PUN web meetup with presentations.
On Wednesday 4 July, there was a PUN web meetup, formerly known as Dutch Python Django Meeting.
Building Single-page web-applications with Django, Twisted and TxWebsocket
Talk by Jeroen van Veen (Goldmund, Wyldebeast & Wunderliebe, GWW).
Websockets give you a persistent TCP connection from a browser to the server. So: you don't need to do a request every time. The connection is there. You need to decide what to send over that connection. This does not need to be HTTP.
With javascript you can upgrade to a persistent connection. The protocol for the handshake used here is changing but getting more stable. Google for Hybi-16. Both utf-8 encoded data and binary data are supported. You have low latency: no headers need to be sent every time, the connection does not need to be setup for each request. So it has a very small foot print. The preferred way of serialization, for me at least, is with JSON.
There is support for this in Python: TxWebsocket (my preferred library, based on Twisted), Autobahn WS, Gevent-websocket. Supported in browsers: Chrome, Firefox, Opera, Safari, IE 10. As browser fallback you could use Flash Websocket.
When buildout a web application, you have to decide whether to build this multi-page or single-page. Multi-page is of course what most websites are using. For a single-page approach, (almost) everything is done on one page. You bootstrap a page, that sets up the websocket connection using the handshake. In this handshake the sessionid cookie from Django can be reused so you can authenticate.
I want to mimic Django URL routing on the server side. And on the client side too, really, using javascript. I use XRegexp for named group support, which helps here. When clicking on a link, you do not want to end up in a fresh page, but want to stay on the same page. But you can change the URL of the navigation bar anyway with javascript. It starts acting more like a multi-page website then, but can still remain single-page. This is usable with the HTML5 history API, better than for example adding hashes to the url. Using this, you can build a website where you can edit the same document with multiple persons simultaneously.
Django WS arranges the authentication during the handshake with the session cookie. It handles a client transport pool, event subscription, websocket URL-routing protocol; Twisted MultiFile, runws, autoreload; BSD license. It is in development. Modules: core, misc, blog, wiki. It is used in a project now for narrow casting, letting messages appear on screens, possibly for a longer period.
There are some challenges. Cleanup the design and architecture. A usable pluggable reference CMS. Proper documentation. Community. SEO: a web crawler will not index much, as only the contents of the bootstrap page are indexed.
The Django WS Code is on github: https://github.com/phrearch/django-ws-core Find me on Twitter: @jeroen_van_veen.
Audience: look at Tornado, it does something similar.
Document automation (Office) using Django
Talk by Henk Vos: http://www.rapasso.eu
You want to generate for example a Word document by pushing a button on your website and have this document contain some information that comes out of your database. You could install MS Office on your server. But we have Linux, and it is not available there. You could use a different box. Or maybe Google Docs, or the Pyuno bridge to OpenOffice. There is also python-docx.
But really, such a Word document is just an archive with some xml files. So we can use the Django template engine. Put some normal {{Django variables}} literally in your Word document. Extract the important file from the archive, run the Django template engine over it, save it, zip it up again, and let your web server serve it in a request.
DjangoCon Europe
Talk by Reinout van Rees (http://reinout.vanrees.org/)
This year's DjangoCon Europe was in Zürich, inside a stadium, a few weeks ago.
There were several talks about the real-time web. Tricky is that you can end up with two parallel MVC stacks, one in the Django backend and one in the Javascript frontend. Putting a caching server in front to speed things up will not work with web sockets. Idea is at least to always use an API.
Databases. Not everything fits in one kind of database. Files are better handled on the file system. There were tips for Postgres. You could use a big box and store everything. The NoSQL support in Django core is not done yet.
A few talks about diversity. Involving women in the community. Hard discussion. Stereotypes everywhere.
Healthy web apps. dogslow: for tracebacks.
CSS. http://gluecss.com for sprite compression. Preprocessors: hurray!
Security. Login should be done using SSL (so https). Try a restore of your backups some time, otherwise you might as well not make any backups, as you do not know if it actually helps you. django-admin-sso for giving easier access to admins for several servers.
Heroku. It's a 'cloud unix'. Focused apps, one thing. You can only use an API to communicate with the outside world. Plus documentation. They have 2-3 person teams. That was the ideal size for them. Quality comes from solid engineering, so you need time, not deadlines.
Various bits. You need to work with timezone-aware datetimes. Flask micro framework. Django core moved to github, which turned out very well.
See summaries on my website: http://reinout.vanrees.org/ There are videos online too. Check the Heroku talk and the keynote about internationalization.
Rethinking the Django-CMS landscape
Talk by Diederik van der Boor (Edoburu).
Why does everyone create there own CMS (in Django)?
Do one thing and do it well. Tight focus versus feature creep. Don;t be afraid of multiple apps in your website, separate them out. Write for flexibility and distribution.
There are various Django CMSes. You've got Django CMS, FeinCMS, Fiber, Ella, Merengue, Philo and more. Big outside contenders are Wordpress (easy to setup, but plugin hell) and Umbraco (customized UI layouts, customised page node types).
For page contents and a page tree you can use the django-fluent-pages package. That might be the only CMS-like functionality you need in your application, instead of a full blown CMS. You can use plugins, like code highlighting, MarkDown, or create your own.
If you are going to create a blog you could do it as one application, but it should really be several separate applications. Do one thing and do it well: the django-fluent-comments package.
How about user authentication, social media auth, RSS feeds for comments, e-mail subscriptions, spam protection. Those should also be separate packages. Currently, most blog applications have all or some of these thrown together.
So mostly: separate your packages.
There is a balance to strike: a too monolithic one-package approach, versus lots of packages that need lots of glue code.
See the code at https://github.com/edoburu Find me on Twitter: @vdboor and @edoburu.
Audience: a danger is that your application might work today and is broken tomorrow when someone uploads a new release on PyPI that has backwards incompatible changes. You need to keep an eye on that. Freeze your dependencies in production.
TND Dataview and Metaclass Magic in Python
Talk by Maarten ter Horst from Top Notch Development.
Python metaclasses are ideal for frameworks and abstract classes, code that you want to reuse. A metaclass is a class that handles the creation of other classes. So an instance of a metaclass is a class. Normal classes inherit from object but meta classes inherit from type. It creates a new class based on your class. So basically you can use this to have a different base class instead of object and override some basic behaviour. You can for example check if a newly created class conforms to an interface.
We use it to inspect the class definition and add a urls method that lists all other methods that return an HTTP object.
Google 'IBM Python metaclass' for a good tutorial on metaclass programming. The presentation is available soon on Top Notch Development.
Michael Bayer: SQLAlchemy
Michael Bayer delivers the keynote on SQLAlchemy at PyGrunn.
See the PyGrunn website for more info about this one-day Python conference in Groningen, The Netherlands.
"Space Invaders! Relational Modeling! And Domain Models! A mashup."
Computer life
My computer life started in the 1980s with a TRS-80. At school I used that to print Snoopy in ascii art. Then there was the Atari 800. It's still at my parent's house. I started programming on that one, which meant using assembly code. I got as far as creating a cool title screen before giving up.
In the nineties I used Rational Rose for some time to do architectural design. So real waterfall development. But I switch to iterative development to create the real code.
Domain Models
Looking a WordPress code you have request handling, sql statements, e-mailing and html all in one file. This is the water slide model, you just go down the file, without real control.
Compare that to the domain model: a conceptual model of a domain of interest. You model the problem agnostic of implementation details. Create explicit adaptations between the outside world and the model. Keep these adaptations as distinct from each other as possible. You have a request, security concerns, services, data handling, persistence and decide how that interacts with your model. When you paint a picture of this, you get the feeling of a pipeline: you are more in control of how it flows.
Relational Modeling
The relational model represents information as collections of rows, consisting of columns. It allows fluid manipulation of rows organized into sets. It encourages normalized form, to minimize repetition and dependency. It provides declarative query languages such as SQL to access and manipulate data. ACID guarantees.
- Joins: intersection of rows.
- Unions: combine sets of rows.
- Grouping: divide rows into sub groups, apply aggregations.
Imperative programming is what you do in day to day python: "Mr. Computer, open this file -> okay."
SQL is generally declarative. We state our intent. "Mr. Database, give me all the rows in table A that starts with 'Q' -> okay."
How do we reconcile the relational model with the domain model? Represent discrete objects as elements in relational sets, that is: object-relational mapping. Expose "relational geometry" within the object relational mapping system. Automate database tasks, don't hide the database. Allow efficient questions to the database. Consider object instances as table rows. Map compositions to relational structure: an object may have child objects, which in the database is modeled with foreign keys.
Keeping the 'R' in 'ORM'. You should expose relational concepts, otherwise you cannot easily represent some SQL statements. So you should allow nested SELECT statements; subqueries should be possible. It is not good to abstract this away. You should also allow efficient 'questions'. Queries should be combined when possible. Do not do one query and they for each row in the result do an extra query. This is what JOINs are for.
Mashup
Now for the fun part. Warning: Armin thought this was rediculous. We are going to build a space invaders game in Python. It will store the state of the game in a SQLite database in memory.
We have a domain model. Every object you paint on the screen is called a glyph. There are enemy gliphs, player gliphs, message glyphs, etc.
The code uses @hybrid_property so a method can get data back from an instance or get a SQL statement back from a class. Checking whether a missile hits an enemy or an enemy reaches the bottom of the screen, is done using SQL JOINs.
Of course this is all using SQLAlchemy, a Python SQL Toolkit and Object Relational Mapper.
He played a demo game and won, with lots and lots of SQL statements scrolling by. Seriously cool. He will put the code on his blog.
Alexandros Kanterakis: PyPedia
Alexandros Kanterakis talks about PyPedia, at PyGrunn.
See the PyGrunn website for more info about this one-day Python conference in Groningen, The Netherlands.
PyPedia is the free programming environment that everyone can edit. It uses the Google App Engine (GAE). You can execute Python code on that website. The GAE is a sandboxed, restricted environment, so: no, you cannot delete all files on the system.
Usually, when you search for the solution for some programming problem, you either find a page that executes the code, or you can see the source code but have to download it and run it yourself, or you find documentation. On PyPedia you get all three.
On the home page you see some code, for example this:
Hanoi_towers(5, True)
You can run it right there. Hanoi_towers is an article on the PyPedia wiki. The source code of that article gets executed.
You can create an account, login and create your own articles.
With the pypedia package you can on your own computer import code from the PyPedia website. You can also easily download the code of an article and run it.
You can give other users permission to edit your articles. You can fork articles, just like in github.com.
All content is under the Simplified BSD License.
Good content with unit test can be put in the validated methods section.
You can use PyPedia for open science. A complete analysis can be hosted so your findings can be reproduced. PyPedia offers a REST interface:
www.pypedia.com/index.php?b_timestamp=&get_code=python_code
The python code here must be urlencoded. Other option: run_code.
Future plans: attract communities, add content. Some presentations are planned at conferences.
The code is here: http://github.com/kantale/pypedia
Douwe van der Meij: AOP in Python API design
Douwe van der Meij talks about Aspect Oriented Programming in Python API design at PyGrunn.
See the PyGrunn website for more info about this one-day Python conference in Groningen, The Netherlands.
Aspect oriented programming is more often used in Java than Python. I have used Java before, so I tried using this concept in Python now.
The use case was BioBench, which is one of the sponsors of this conference. It is an application for biogas installations. A farmer enters measurements and gets info from this.
One aspect you need for your API is security. Also statistics/logging, serialization. Probably more. The danger is that those aspects are scattered all over your code base and are tangled, interweaved with other aspects.
Aspect Oriented Programming means: separating your concerns. Put each aspect in a separate module. How can you implement this in pure python? Decorators!
What is an aspect:
- pointcuts, a signature of your code, a regular expression (though I don't use it in my Python code)
- join points: point in your code where the pointcut expression is true, so where the advices should be run
- advices: code to run before or after the join point, or around it.
The security code is an 'around' advice. If a condition is true, we execute the main code, otherwise we skip it and raise an exception.
So for example you end up with:
@secure
@serialize
@statistics
@dispatch
def api_call(...):
...
It is good to apply this for aspects that are orthogonal to your main code, like (mostly) security.
Audience: in Zope you can use zcml for aspect oriented programming.
Miguel Araujo: Django crispy-forms
Miguel Araujo talks about Django crispy-forms at PyGrunn.
See the PyGrunn website for more info about this one-day Python conference in Groningen, The Netherlands.
In Django forms you can display a form in html in different ways. But it can be hard to customize or to target it using css. You can reorder fields using keyOrder, as fields is a SortedDict. ModelForms are different. You could override templates, but it gets hacky fast.
django-crispy-forms was formerly known as django-uni-forms, created by Daniel Greenfeld. I joined the project in 2010 and became the lead developer.
It has a filter |crispy and a tag {% crispy %}. It adds divs and ids to your form.
With the FormHelper class you control the global form rendering behaviour, using attributes, for example form_method='post', form_action, form_id, form_class, form_tag=False. You can define your own attributes, with the code behind it, to for example have help text as placeholder.
The layout power comes from layout objects. They are Python classes.
Every layout object has an associated class. For example, Div,
Fieldset, HTML. Div(Div('name', 'value'), HTML(" hello
Layouts can be decoupled, and composed into another Layout.
crispy-forms templates can be easily overridden at various levels. There are layout templates and global templates.
There is more power available. Documentation is at http://django-crispy-forms.rtfd.org