Weblog
Armin Ronacher: I am doing HTTP wrong
Armin Ronacher has a fresh look at HTTP from Python, at PyGrunn.
See the PyGrunn website for more info about this one-day Python conference in Groningen, The Netherlands.
Most Python web frameworks approach HTTP in the same way. For a project I started looking at it in a more basic way, wondering if the lower TCP level would be enough.
Usually you have a request object, you return a response object, and some other layers (like in WSGI) may change some things along the way. Why don't we abstract that more? What do we like about HTTP?
- It is plain text.
- You can use the REST concept.
- Content negotiation (returning html or json).
- Incredibly well supported: every browser supports it.
- It works where TCP does not. In some cases you cannot use TCP, but can use HTTP, like Windows mobile before version 7.
- It is somewhat simple to implement.
- You can upgrade to custom protocols when needed.
You can stream HTTP or buffer it up. In Python the headers will always be buffered up in WSGI. Same for headers, forms, files. When all has been gathered and is ready, the HTTP is streamed back to the client.
When your server has started reading the data of a request, it can no longer tell the client to stop sending. You could reject too large requests. But what is large? A form of 16 MB that needs to be loaded in memory may be too large. A big half gigabyte file that is directly stored on disk, hardly using any memory in the process, may be fine. So lots of Python web frameworks are vulnerable to attacks by sending it big requests.
A new approach
Dynamic typing has made us lazy. We treat HTTP as an implementation detail.
For a web app we have defined schemas so we can push around data in our application between client and server, without needing keys in request parameters.
Be strict in what you send, but generous in what your receive; variant of Postel's law. When you do that, you need to know what you recieve and know if you can trust it.
A GET request has no body, so parameters have to be URL encoded. So there is inconsistency with JSON POST requests.
How do we handle streaming data? We don't. For things that need actual streaming, like file upload or send, we have separate end points. Streaming is really very different, but we can stream until we need buffering.
We can discard useless stuff, keys that users send us but that we will not handle.
Modern web apps are APIs. Separate from that you have the front-end. Do you need to support 'dumb' clients? Move the client logic to the server, for example handle javascript on the server.
We are currently using this for an iOS game.
Oh hai, we are hiring: http://fireteam.net/careers
Rick Oost: Generalized traversals
Rick Oost talks about Generalized traversals at PyGrunn.
See the PyGrunn website for more info about this one-day Python conference in Groningen, The Netherlands.
Inspired by the Haskell programming language he wrote a package for traversing. It does various things with mapping (applying a function to all items in a list) and reducing (combining a list into one return value). There are ways to reduce starting with the left-most value or the right-most value. So there is more flexibility than the standard Python list comprehension. It generalizes generators. Next to the normal yields of a generator you can have a return value at the end.
The theory and definitions went very quick. Then he showed code examples that needed too much formatting and too much explanation to write down here properly, sorry.
He needs to write some documentation and then will put it on PyPI (as traverse.tree or some other name).
PyGrunn
PyGrunn is the Python conference in Groningen, held on Friday 11 May 2012.
See the PyGrunn website for more info about this one-day Python conference in Groningen, The Netherlands.
I will be blogging today about this conference. You may want to check blogs for this day by my brother Reinout too.
Berco Beute, CEO of PayLogic welcomes us, also on behalf of fellow organizers Goldmund Wyldebeast & Wunderliebe. They do a lot with open source, mostly Python. The PyGrunn conference is even more popular now than last year; we are sold out.
Summary of Plone Open Garden 2012
Summary of the PLOG 2012 in Sorrento.
More info about PLOG 2012: http://www.abstract.it/abstract-en/initiative/plone-open-garden-2012/
- Maurizio: activity for http://plone.com has started. We should be able to do more this month. Should be live before the next conference. We also added some info to http://plone.it. Talked about creating some bundles of often used products.
- Guido: I started out here with a working prototype of plonesocial.activitystream, based on plone.app.discussion. But there were concerns about performance. We started with the lower end. We experimented with plone.app.async, which should help to avoid write lock contention when there are lots of updates on the sites. In the code we can now override the plone.app.discussion storage, to use our own data structures if needed. The async stuff breaks on a low level though. So we now have a bug reported upstream. We also discussed following/liking/favoriting people, users, pages, other content. So currently we are building on native Plone functionality. We need to be more strict on security, partly just checking that this works as expected. More info: http://pypi.python.org/pypi/plonesocial.activitystream and http://github.com/cosent/plonesocial.activitystream
- Simone: working on collective.geo. We worked on settings in the control panel. We made it more user friendly. We wrote more documentation, using Sphinx. Final result: currently broken, so a bit of patience please. Also, see collective.geo.usersmap on github. More info: http://pypi.python.org/pypi/collective.geo.bundle and http://plone.org/products/collective.geo
- Giampiero: worked on a comparative list between Plone and WordPress. Writing about how to migrate from WordPress to Plone, good for people really new to Plone. It is basically a map into the Plone world. Can be good for helping people move to Plone, even if it is just because their boss forces them. Help them make connections, give them some links so they can find their way. More info: https://docs.google.com/spreadsheet/ccc?key=0AjjG4VVlsN-IdG5rRmNnbWJmTjdzZDFaNklKOWoyd0E
- Matt: we brain stormed a feature list of Plone. I am working on distilling this into a more structured form.
- David Siedband: we discovered that when migrating to Plone 4, you loose versioning information for files and images during the blob migration. Looking at if this information is then still available somewhere so it can be fixed.
Rob Gietema: Plone conference 2012
Rob Gietema introduces the Ploneconf at the Plone Open Garden.
More info about PLOG 2012: http://www.abstract.it/abstract-en/initiative/plone-open-garden-2012/
The Plone conference 2012 will be helt in Arnhem, The Netherlands. This is a city of about 150,000 people, so everything is basically within walking distance.
There is enough space for everyone, we have several rooms available for various audience sizes. The party will also be helt there. There is food and coffee near all rooms.
Wyn Williams will setup the wireless internet. We are actually getting a new fiber connection on the conference venue.
Airports: Amsterdam (Schiphol) is at about an hour by train from Arnhem. Rotterdam and Düsseldorf are also near.
We will have two days of training, three days of conference, two days of sprints.
Most people in The Netherlands can speak English fairly well.
Rooms are available, but there are is also a big concert stadium nearby, which can lead to more people in hotels, so do book soon. If no rooms are available anymore, contact us and we are glad to help. We can also hire a large boat for this occasion where you can sleep.
If you want to do a presentation like this one on for example your local python group or university, contact us so we can help.