Plone
This is here to serve as contents for the atom/rss feed for Plone, also read by planet.plone.org.
Kenneth Reitz - Python for humans
Kenneth Reitz talks about Python for humans, at PyGrunn.
See the PyGrunn website for more info about this one-day Python conference in Groningen, The Netherlands.
Follow me on twitter: @kennethreitz_. I work at Heroku. I am a member of the Python software foundation. I created the requests library that makes is easier for humans to retrieve a web page in Python.
I want everything I build to be open source. I build for open source, pretend it is open source even when it will never be released to the public because it is too client specific. Add documentation, please, also to your internal tools.
Open your Python prompt and see the Zen of Python:
>>> import this
Beautiful is better than ugly: you don't need lots of curly braces in Python. Explicit is better than implicit.
Most importantly for today: there should be one - and preferably only one - obvious way to do it.
Welcome to paradise. You have found Python, a language with a beautiful zen philosophy and all is going to be fine. Lies!
Look at some Ruby code to download the contents of an https web page. Perhaps a bit too many lines, but it is pretty straight forward. Now you switch to Python. Which library? urllib, urllib2, something else? Okay, you use urllib2 and then you have to add some password manager and lots more lines that are unclear. You will leave and never come back.
This is a serious problem. I think HTTP should be as simple as a print statement. It is used so often! Our world is connected over HTTP right now so we need this to be simpler.
Python needs more Pragmatic Packages. Deal with things sensibly and realistically, in a way that is based on practical rather than theoretical considerations. Make it practical for humans, for the common case.
The requests library is HTTP for humans. A small set of methods with consistent parameters. Do this for more modules! Fit the ninety percent use case. Features, efficiency, performance, etcetera are important, but you should ignore them for such a practical package. Go for the common case. Write the README the way you think it is supposed to work.
"Cool story, bro, but why should I do with this?" It's worth your time and everyone else's time as a developer.
Some things that could be improved:
- File and system operations. We have the modules, sys, shutils, os, os.path, io. Which should you use for a task? It is really difficult to run external commands and this blocks dev+ops folks from using Python.
- For installing Python there are various ways. Use the Python that came with the system or compile your own? Python 2 or 3?
- XML hell. etree annoys people. lxml is awesome, but can be difficult to install.
- Packaging and dependencies. Pip or easy_install? How about an easy_uninstall? Distribute or setuptools? [Holger Krekel will talk about this topic later today in the keynote.]
- Dates, datetimes. Which module: datetime, date, time, calendar, dateutil, version 1.5? Timezones, they are ridiculous.
- Unicode. We'll just skip that one.
- Testing. Lots of different ways to create tests. Doctests and unit tests. Various test runner libraries, like nose and tox.
- Installing dependencies. Python-mysql, if you remember the exact name. (My solution: just use Postgres.) Python Imaging Library needs several system packages before you can install it. mod_wsgi: if you install this, which Python version are you using, the one from the system?
Hitchhiker's guide to Python: http://python-guide.org The goal of this project is to write down the tribal logic, documenting best practices. A guide book for newcomers. A reference manual for seasoned pros. It tries to document the one - and preferably only one - obvious way to do it. Please contribute documentation here. This lets of practice what we preach.
Lets fix our APIs and improve our documentation.
requests might get into core at some point.
requests started as a wrapper around urllib2. It was a nightmare. It now uses http to handle the lower level parts.
If I would have said 'yes' to all feature proposals for requests, it would have been too much. People can write a library around requests. Saying 'no' made it possible to create and maintain a good architecture.
Dutch Python Web Meetup 21 March 2013
Hosted in the Travelbird office, Amsterdam
Remco Wendt starts this meeting off with some bad news.
Malcolm Tredinnick passed away. He was a really major contributor to Django. He will be a big loss to the community.
A UK company is claiming a trademark over Python. Ah: it has been resolved.
The Django sprint in Utrecht was a big success. It has just been decided at the general Dutch Django meeting to repeat this next year: 22 and 23 February 2014.
Orne Brocaar - Job-Runner
Job-Runner is a product we built inside Spil. I am a freelance Python/Django web application developer. I am currently working for Spil Games, a gaming company in Hilversum, about 70 people working there, about 50 gaming portals worldwide.
I am working within the data warehouse team. We gather data from internal and external sources, like analytics data, game related data. We convert this into a Data Vault and Data Mart model, where we can analyze it so we can understand our users. We create reports to inform the business. The data is imported into the staging area (database), then the data warehouse, then we go to reporting, analysis, recommendations. These are all jobs that need to be performed in order.
Before Job-Runner this was all scheduled by cron. If one job failed, the follow-up jobs would fail too. There was no good overview of running, completed and failed jobs, hard to quickly see what (if anything) went wrong. Cron could start a new job if the previous long-running job was still running. Not good.
Job-Runner is a centralized dashboard, an admin interface, a broadcaster for jobs, a RESTful interface. Then there are decentralized workers (job-runner-worker), and a centralized WebSocket server (job-runner-ws-service). Those two communicate with Job-Runner.
The dashboard is HTTP, the queue broadcaster is ZeroMQ.
A project has a job template, for example a python snippet with some variables that are filled in. A job is created from this template. The job is executed in a worker pool. The job may run multiple times, for example once every day. You can also start it manually from the dashboard.
Job-Runner is a Django project, with TastyPie for the RESTful interface. There is a long running management command (the publisher), called broadcast_queue. The worker subscribes to this publisher. It publishes events to ZeroMQ. gevent handles the event queue. The WS Server is a simple proxy between ZeroMQ and the WebSocket server.
We started in December last year. We looked around for other solutions, but they did not fit our needs. The authentication system of Django worked nicely for us. You can setup notification e-mail addresses for a job or pool. Via the admin interface you can schedule jobs at a specific time. If a previous job run has not finished yet, the new job instance will not yet be started.
We have used this for hundreds or even tens of thousands of jobs. We only have one normal cronjob left.
The documentation is at http://job-runner.readthedocs.org
Personal website: http://www.brocaar.com
Jan-Jaap Driessen - Sentry
I work at Mind District. We set up Sentry in our web stack, because we don't want to hunt around for stuff in lots of logs on several servers.
Tip: use UDP for sending the logs to the central server.
We have used this in Python, but are now also using it in javascript, as simple as this:
MD.logging.warn('bar');
Inspired by Strophe. We have a few lines of code for that. Ask me. We also use log4javascript, which is 128 kB and that is minified. In development you want to use the console, but in deployment you want to use Sentry. The code uses Raven to capture the logged message and send the message to the Sentry server.
The WSGI integration is in Zope3, let me know if you are interested.
Diederik van der Boor - 10 things to take care of when Open Sourcing your package
For example django-fluent-pages, developed by me.
- License: do not use the GPL, but the BSD or the Apache license, otherwise you cut your project off from other Python community modules. [For the record: I do not agree with this. Maurits]
- Please provide a README with a reason for using your project. Maybe a screenshot.
- A changelog please. I don't want to look at github commits to check if my bug got fixed in a specific version.
- Documentation. For example on http://readthedocs.org. Look into sphinx.ext.autodoc for automatically generating documentation from your code. Also graphviz and intersphinx.
- Add tests. A runtests.py that is enough to run your tests without needing to setup anything is a good idea. Look into Travis for automatically running tests on a server. tox is good for running tests locally.
- Use the communication channels. Keep people updated. Have a website, with documentation.
- Please provide an example project, so people can easily see how it works.
- Create a good setup.py.
See my github account: https://github.com/edoburu
I count eight things, not ten, so I apparently missed two. Well, two points from the audience follow.
Audience:
- If you start losing interest in your project, look for someone who can take over, preferably someone who has already created some pull requests. Also give people commit access to your repository if they have made some good pull requests.
- Upload your distributions to PyPI, not your own small server. And leave all versions online, otherwise some people will be very annoyed as they are using it on their live site.
Eric - Two Scoops of Django
Daniel Greenfeld and Audrey Roy are writing a book on best practices for Django development. It is in beta, but you can already buy it. It is very good, so please do. The beta is 14 dollars and then you get the final version for free.
Get it at https://django.2scoops.org
Jan Murre - Webapp2
I am active with Python, Zope, Plone, Django, since 1999. I am currently working for a company in Utrecht and before that Pareto.
Webapp2 is a lightweight Python web application framework. It is compatible with webapp, its predecessor, which is the original Google App Engine framework. Webapp2 has several improvements, for example better URI routing and WebOb. See the webapp2_extras package for more stuff. Google has adopted it since GAE SDK 1.6.0.
It has WSGI integration. You can choose your templating language: Jinja2, Mako, etcetera. Forms: wtform, formish, etcetera. Persistence: for AppEngine there is BigTable. Stand-alone you can again use whatever you like: SQLAlchemy, some noSQL, etc. Use a WSGI container, like uWSGI, mod_wsgi, paste. So you pick and choose your own stuff on top of the micro framework.
Google App Engine is Platform As A Service (PAAS). You upload your application there. Comparable in that respect with Heroku. Easy to build, easy to maintain, scales and load-balances automatically, you pay for what you are using. There are several quotas and limits. For some applications we only have to pay the minimum of two dollars a week for the professional version. There is also a free version, with low limits for bandwidth, memory. Languages supported are Python, Java, Go (experimental). BigTable is a noSQL storage, with queries, sorting, transaction, caching built in. URL fetching. E-mailing. There are task queues (think: Celery). Scheduled tasks (think: cron).
You start the application with a simple, plain text yaml configuration. Of course I use Buildout to setup my local development environment, where you have some local storage. The recipe to use is rod.recipe.appengine.
There are other Python (micro) frameworks that you can look at, for example Flask, Pyramid. You can use those on the Google App Engine too, but it takes a bit more setup.
Conclusions for Webapp2: small, but nice. Quite some batteries included. Good routing engine. Plays nice with Google App Engine.
Together with someone else I created http://mijnvoetbaltrainer.nl that uses this, in combination with Amazon for hosting the video material.
More info on webapp2: http://webapp-improved.appspot.com
Pepijn Vos - Pygments, ctags
Pygments: code colorizer. Ctags: allows you to jump to a definition of some code. You can combine that. I did that in spelunking. You can use it to browse github repositories and click around in it. It downloads the repository and runs Pygments and ctags on it and makes it available.
Code: https://github.com/pepijndevos/spelunking
Demo server: http://raspi.pepijndevos.nl
Jan-Jaap Driessen - video conferencing
We have built some cool stuff for health care professionals who visit other people. It can be handy to do this via video communication. We did not base this on Skype, but on Google video conferencing. http://videmo.minddistrict.com There was a SOAP api that we had to interact with. If you have to do that, you should use suds. Using bosh, jabber, xmpp on the server. You can use it simply with an Android phone, no need to buy special hardware.
Wil jij mijn collega worden?
Mijn werkgever Zest Software is op zoek naar een Python programmeur.
Mijn Franse collega Vincent Pretre gaat weg bij Zest Software. Ik vind het jammer om hem te zien gaan, maar uiteindelijk was de aantrekkingskracht van familie, bergen en kaas die ook goed smaakt buiten een tosti te groot. Wij zoeken dus naar iemand om hem te vervangen, anders hebben we teveel werk te doen.
We willen iemand met ongeveer de vaardigheden van hem of mij. Het belangrijkste is dat je competent bent in het programmeren in Python en dat ook graag doet. We bouwen websites gebaseerd op Plone of Django, dus als je één of beide beheerst, is dat handig. Als je geen ervaring hebt met deze twee frameworks, maar je houdt van een uitdaging, neem dan toch contact met ons op. Leren hoort bij het vak.
Ik werk al zeven jaar voor Zest en dat bevalt me nog steeds goed. Ten eerste hebben we een klein, maar goed team. Jean-Paul Ladage is de baas, met vele jaren ervaring in Zope en Plone. Hij heeft er een handje van uit te vogelen wat een klant nodig heeft in plaats van wat hij wil, en hij kan dat uitleggen aan het ontwikkelteam. Fred van Dijk houdt het hoofd koel bij het helpen van klanten met veel websites en servers. Hij zoekt uit of een wens of bug aangepakt kan worden met standaard Plone of met add-ons of dat Vincent of ik als programmeurs erop los mogen. Thom van Ledden heeft veel styling voor ons gedaan, maar hij zal weinig tijd hebben, want hij gaat de komende tijd aan een afstudeeropdracht zitten. Daniel Ferreira doet marketing voor ons. Elke dag lunchen we samen.
Ik vind het leuk dat ik de hele week veel kan programmeren. Ik kan me concentreren op Plone en wat Django en ook een beetje Grok, terwijl Jean-Paul en Fred de meeste klanten opvangen voor ze me af kunnen leiden. Klantcontact is prima, maar niet teveel op één dag. :-) Ik vermaak mezelf het beste als ik aan een project werk en daarbij verbeteringen kan doen aan de kern van Plone of aan add-ons, en dat komt geregeld voor.
Dus: als je op zoek bent naar een Pythonbaan met een goed team in Rotterdam (Hoogvliet), neem dan contact met ons op.
Want to become my colleague?
My employer Zest Software is looking for a Python programmer.
My French colleague Vincent Pretre is leaving Zest Software. I am sad to see him go, but finally the pull of family, mountains and cheese that also tastes good outside of a tosti was just too strong. So we are looking for someone to replace him, otherwise we will have too much work to do.
Basically we want someone with about the same skill set as him or me. The most important thing is that you need to be competent and happy when programming in Python. We build websites based on Plone or Django, so being skilled in one or both of those is good. If you do not have experience with those two frameworks, but you like a challenge, do contact us. Learning is definitely part of the job.
I have worked for Zest for seven years now and am still enjoying it. First of all, we have a small, but good team. Jean-Paul Ladage is the boss, with lots of years of Zope and Plone experience, and a knack for figuring out what the customer needs instead of what he wants, and explaining that to the development team. Fred van Dijk keeps a clear head when helping clients with lots of websites and servers, and figuring out if a feature or bug can be dealt with using standard Plone and add-ons or if Vincent or I as programmers should have a go at it. Thom van Ledden did lots of styling for us, though he won't have much time anymore as he will be busy finishing his studies. Daniel Ferreira is doing marketing for us. Every day we lunch together.
I like that I get to do lots of programming the whole week. I can concentrate on Plone, and some Django, and also a bit of Grok come to think of it, while Jean-Paul and Fred keep most customers out of my hair -- and the other way around. Contact with customers is fine, just not too many on one day. :-) I enjoy myself the most when working on a project means improving core Plone or add-ons, and that regularly happens.
So: if you are looking for a Python job with a good team in Rotterdam (Hoogvliet) contact us.
Mountain Lion
Upgrading to OS X Mountain Lion.
IANAME (I Am Not A Mac Expert), but I have recently upgraded my MacBook Pro from Snow Leopard (10.6.8) to Mountain Lion (10.8.2). Some pointers follow.
Get the software
Just open the App Store application, search for Mount Lion (it may be on the front page as popular download), and buy it. The download is about 4 GB, so you will want to start with this step. Or buy it on CD somewhere.
Cleanup
I do not think you need much extra hard disk space for the new Mac OS X version, but this is still a good time to make some extra room. Disk Inventory can be a good assistent here, showing you which files or directories take up the most space.
Your Downloads directory may also contain old installers that can be deleted. Maybe you have some backups from 2004 that you finally realize you will not need anymore.
Be sure to go to the Trash folder and empty it.
Backup
Make sure you have a backup, probably with the Time Machine. If you are a developer, make sure all your source code changes have been committed and pushed.
Upgrade
Close your programs and run the Mountain Lion installer. I think this took about half an hour.
TRIM Enabler
If you have a Solid State Disk, you are probably using TRIM Enabler. After the first reboot, you will need to start that application and enable it again. Then you probably need to restart again, but the application will tell you this.
XCode
If you are a developer, you will want to install the latest XCode from Apple. Open the App Store and download it. It is about 1.6 GB so this will again take a while.
When you install it, make sure to install the command line tools too. Open XCode, open the Preferences, go to the Downloads section and download and install the command line tools.
Status
At this point, as I am a Plone developer, I tried starting an existing zope instance. This worked fine. But then I tried rebuilding the Python buildout, specifically installing the 3.3 one, which failed. If I recall correctly, no good C compiler could be found. I have that in Mac ports, so I needed to update those.
Mac ports
You will need to completely reinstall the Mac ports package and your active ports. Follow instructions at https://trac.macports.org/wiki/Migration
For step 1 under 'Reinstall ports' you may want to gather only the active ports:
port -qv installed | grep active > myports.txt
Then you can open that file and remove the lines for ports that you think are not needed anymore. Maybe keep a copy of the original file in case you later find things are missing.
In step 3 you execute this:
sudo port clean all
This might fail with an error like this:
Error: Unable to open port: can't read "build.pre_args": can't read "build.cmd": Failed to locate 'gnumake' in path: '/opt/local/bin:/opt/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin' or at its MacPorts configuration time location, did you move it?
If that is the case, then most likely you have not installed the command line tools of XCode. See http://guide.macports.org/chunked/installing.xcode.html on how to do this. Or see above.
Note that freshly installing all you favorite mac ports will take a while. You may want to grab a drink or some sleep at this point.
Python buildout
If you are developing Plone, you may already be using the Python buildout. This is on github now. It seems best to rebuild your pythons at this point. Use the bootstrap-1.4.4.py file, otherwise you will run into problems.
Initial impressions
Wow, it feels fast! My feeling is that starting an application is now twice as fast. Maybe I would have had the same result with a fresh install of Snow Leopard, but it sure is nice. Note that I was already using a Solid State Disk, so it was already fast.
I have switched the natural scrolling off (System Preferences, Hardware, Mouse). Maybe I should just try it, but I am used to the old behaviour.
For a while, iTunes started up and began playing music without me doing anything. It did this lots of times. Today I did not see this anymore.
Mail looks rather differently. I started it up and did not see my mail accounts and folders, just some messages. So I quit Mail and started ThunderBird, which is still my favorite mail reader because I know its keyboard short cuts. The Mail interface can be easily tweaked though, nothing wrong with it, just a bit surprising for me the first time. I think the layout will work fine, at least on wide screens.