Weblog

published Nov 03, 2021, last modified Nov 04, 2021

Brandon Rhodes - Python Web Technologies

published Oct 21, 2016

Brandon Rhodes gives the third keynote talk at the Plone Conference 2016 in Boston.

When I met the Plone world around 2008, I saw there were lots of small companies and individual consultants. It inspired me to quit my job and university. Then the economy collapsed. Coincidence?

Movie The Social Network, 2010. When I heard about this movie, I thought it was going to be horrible. A movie about Facebook, really? But it was really good writing. One line struck me especially: "We don't even know what it is yet." Often we will get a new technology, and in retrospect have no idea what it was yet. We invent something, but we don't even know what it is yet.

The web in Python, end of the nineties. It was not much, it did not look like clean code today. The web was young. The blink tag, anyone? These two not yet mature technologies met, and mingled.

What does Python offer? What sets it apart?

  • Reflection. Your program can look at itself, like in a mirror. Things like: list all functions of a class, ask the type, get an attribute by name. Not a lot of languages did that.
  • Object oriented. Not just object based, allowing you to split functionality over several objects. But sub classing, multiple inheritance. You get invited to write new methods for existing classes. (And it eats a whole level of indentation, but that is another story.)
  • Dynamic. You can wind up with code and objects that don't even exist in your raw source code. You can generate code on the fly, insert variables dynamically/magically. Such things might not make your code more readable though: code that does not even exist on the file system?
  • Simple. Einstein: "Things should be as simple as possible, but no simpler." Dan McKinley, book Choose Boring Technology: "Every company gets about three innovation tokens." You can do about three new things without falling over. Are you going to use MongoDB? You have just used up one of your innovation tokens. Only choose new exiting technology in strategic places. For the rest: just use MySQL, Postgres, Python. Boring, but stable.

Some Python web technologies: Django, Flask, Bottle, Pyramid, Morepath. Flask and Django are an order of magnitude more popular. I created this list, found the same list somewhere else. But some techs fitting in the middle were forgotten, like web.py, web2py, CherryPy.

Flask and Django:

  • Views are plain functions
  • explicit registration, not reflection
  • your app can start small and simple

Today, Django stands as Python's default first framework. Flask actually has more github 'stars' for people who follow it. Yet there are far more Django conferences or meetings world wide. Django gives you a good start. Years later, you may get more opinionated, with good reasons, and choose your own ORM and a different, smaller framework. As a beginner, Django just helps you a lot.

And Django has a forms library. Wait, that may involve classes. You may need inheritance and introspection. But good programmers don't avoid complexity when it helps them. Then again, a form library is a bit like backing up a truck with multiple trailers. You have knobs to control how they are going, but can you do that without failure? And you are probably missing all kinds of knobs that the forms library has not thought about. But a forms library does automatic validation, refills forms for retry, does cross site scripting, which a new web developer probably does not even think about. So it is good for them to use it.

Flask has become the go-to second framework, for when you know what you are doing.

But will we keep writing Python for the web? We have competition. Javascript is in every browser. With Node.js you even have it on the server.

Javascript is a much cleaner story from the outside. Python has 2 versus 3, C Python versus PyPy. For Javascript on the server, everyone uses Node.js, which is the PyPy of Javascript. And for ES6 there is a cross compiler to ES5.

But, you say: Python has reflection, generators, iterators, classes, modules, and Javascript has not. Wrong. ES6 adds this. You turn your back, and suddenly a language has involved. Javascript has Python features! Vi has syntax highlighting! "Everything that rises must converge." Languages grow similar.

But what about the fact that Javascript is just dumb?

40 - '30' -> 10
40 + '30' -> '4030'

In Python you get a helpful traceback when you try this, instead of running into undebuggable errors later on. Ah, but in TypeScript you get the same. They fight back successfully against the broken type system. And you can cross compile it to ES5.

So Javascript is a contender, is gaining features, and becoming safer.

There has always been another language that was more popular than Python. There is nothing new. What are Python's advantages then?

It is becoming the world's default language. Science is moving to Python. Data is moving to Python. At weather conferences there is now a Python track. Simple syntax is perfect for the occasional programmer, who has other stuff to do, like science. You do your science, and one every three weeks you write a small program. Django Girls organizes lots of weekends around the world where women use Python to write their first website. The new programmer needs Python, because it is simple.

The web and Python met when both were immature. In the years, both learned the patterns to make the web possible and sometimes even simple to program for. And even after all this years, we don't even know what it is yet.

Questions?

First language for a child? Scratch is good for young children. Python is a good language to get used to. In the US it has become the default in universities for courses.

Where at the language syntax level has Python done anything really new? When you open a file, once PyPy got popular, it got clear that files would remain open. We had to have a way to run cleanup code, even when exceptions rose. So we did try/finally, like in the Java world. It solves the problem... and it is ugly because it is Java. So we decided we needed to do one better. I believe we innovated with the with statement. It was not intuitive for me at first, but it is very helpful. It is marvellous for any kind of recursive context management, changing directories being sure that you will return to the previous directory.

In Javascript you have classes now, finally, but the Python community is far more used to it, right, which helps? Yes. Node and Javascript are still in a period of change, features have not been around for a decade. Writing in NodeJS you have spent one of your innovation tokens, writing Python you still have that one.

Find me on Twitter: @brandon_rhodes.

Lightning talks Thursday

published Oct 20, 2016

Lightning talks on Thursday at the Plone Conference 2016 in Boston.

Hector Velarde: News sites with Plone

Several add-ons:

  • collective.fingerpointing: select which events to log, like who logged in, who created content, etc.
  • collective.lazysizes: lazy loading of images, iframes and tweets, while you scroll down the page. We reduced initial load times with 80 percent.
  • collective.liveblog: content type for micro updates, bit like twitter. Automatically refresh using ajax calls.

Gil Forcada: Python 3 report

Unofficial Zope on Python 3 status: http://zope3.pov.lt/py3

Today I worked on the same for Plone packages. So:

Unofficial Plone on Python 3 status: http://gil.badall.net/py3

Gil Forcada: Teams

Everybody was sure that Somebody would do a job. Nobody realised that Anybody could do it. Etcetera.

That is decision paralysis, waiting for anybody else to do the job. That is why we have teams in Plone. Maybe we can use the teams more on github. If you start looking for things to do, there are hundreds of issues for Plone. Split by team, it is less daunting.

Crissy Wainwright: Sprints

There will be sprints this weekend, in this room. It is when a group of people get together for a project. Timo will be leading and assisting the sprints. Find him if you have questions.

There is a list of things to work on, you can add more. Please put your name in as a leader if you are up for that.

Mohammad Tareq Alam: Big sale on Plone!

Buy one, get one free!

Thank you, Plone community, for all the documentation.

At http://themeforest.net you can find and submit themes as plugins for various systems.

http://themes.quintagroup.com is already doing this for Plone.

Why doesn't http://plone.com have something to sell? That was always my question. The Plone Foundation could get a share of the sales.

Money is not everything. You get to involve developers from around the world.

Gabrielle Hendryx Parker: Plone wants you

How to grow the community with little time and no money.

We have a small percentage of the market. Don't wait for marketing to rescue Plone. What made you come to Plone? Most of you know and like Python. You tinker with code.

IndyPy User Group started very small, now hundreds of people, with at least forty people meeting each month. In a Python group like that you are going to find Pythoneers, thinkers, tinkerers, prime targets for becoming a Plone person. Sponsoring is cheap: buy pizza. There are 700 Python meetups in the world. We can spread the message about Plone there.

How about creating a demo leader board? A competition for who gives the most demos for Plone.

Philip Bauer: State of Plone trainings

I gave my first Plone training in the Arnhem conference. In Bristol there were four trainings, Bucharest three, lots of people were there. In Boston we had 19 trainings, 22 trainers, over half the people of the conference attended the trainings.

Trainings are hosted on http://training.plone.org. For various audience and topics. If you use the trainings and spot an error in the code or in the language, please report it.

Bill Blanchard: deploying Flask in five minutes

I use cookiecutter to create a new project. I copy some settings over. Then I use zappa to deploy it to lambda from Amazon, or update it.

Works with Django, Flash and Pyramid. May work with Plone, but I have not tried it. Only Python 2.7, not Python 3 currently.

See https://pypi.python.org/pypi/zappa

Nejc Zupan: Dragonsprint

This is a Pyramid sprint in my home town of Ljubjana, Slovenia. 5 to 9 December this year. It would be great to have Plone people there too.

Mikko Ohtamaa: IRC and other great Finnish inventions

IRC comes from Finland, just like Linux, SSH, and Angry Birds.

Read this: http://www.fizerkhan.com/blog/posts/Every-Programmer-should-use-IRC.html

On IRC you invest in the community, and the community invests in you. Useful IRC channels are #plone, #pyramid, #python-friendly.

But IRC is dying. Other mediums are Stack Overflow chat.

Fred van Dijk: Teach Plone in four hours

Some thoughts on training. I moved from end user to consultant and trainer.

I tried to teach configuring Plone in four hours.

We forgot to buy coffee. The guy made us a blend of coffee after a few questions. I did the same during the training. The people I taught were older, more experienced than I am. Who was the newbie?

Structure the information that you are teaching. That gives them a map.

Progressive exposure, teach them more and deeper as you go along. Don't start too deep. But: when the people already know enough, then you can dive deep quickly.

You guide people on their learning path.

You can be a trainer too!

Sven Strack: mr.docs

mr.docs helps you write documentation for you packages. From the source it creates nice html, without you needing to setup Sphinx yourself. It checks for typos. It knows names of PyPI packages and adds it to the dictionary, which helps. You can let it create a fresh config for you to edit.

See http://mrdocs.readthedocs.io/

Alin Voinea: Plone and Docker

Since last week, we have an official Docker image for Plone.

See also his talk earlier today: http://maurits.vanrees.org/weblog/archive/2016/10/alin-voinea-docker-and-plone

Jens Klein - Modern LDAP User and Group Authentication

published Oct 20, 2016

Talk by Jens Klein at the Plone Conference 2016 in Boston.

This is an update on pas.plugins.ldap.

LDAP is Lightweight Directory Address Protocol. This protocol for storing and querying information on users goes back to the eighties. There are various solutions, like OpenLDAP, ActiveDirectory.

There has been LDAP support in Plone for a long time. Products.LDAPUserFolder has been available as basis, replacing the standard user folder. Products.LDAPMultiPlugins wraps it to the PAS API, difficult to install. Products.PloneLDAP integrates is more into Plone, but also difficult to install. plone.app.ldap is a wrapper around this, making it easy to install in Plone. But everything is stacked on top of each other, very difficult to maintain or understand.

We created a complete rewrite: pas.plugins.ldap. We use an abstraction of the LDAP features, in``node.ext.ldap``, Python 2.7 only, using python-ldap, pure Python.

You can get groups and users from LDAP. It maps LDAP attributes to PAS property sheets. Support for caching, for many users, 98 percent test coverage.

Limitations: you cannot add users or groups, only edit them. The underlying library does implement this, so this is a TODO item, although our sites don't actually need it: users are added or deleted in a different system. No multiple (fallback) servers possible. It patches the user portraits, so this may conflict with other patches that you may have.

We have a test layer that uses an OpenLDAP instance.

node.ext.ugm is used, which is a set of interfaces for User and Group Management.

pas.plugins.ldap uses those base packages, configures it using GenericSetup, provides ZMI and control panel forms to configure LDAP.

Supporting 'many' users is tricky, this was solved using money from a fund raiser campaign. Thank you!

David Glick - Greater Than the Sum of the Parts

published Oct 20, 2016

Talk by David Glick at the Plone Conference 2016 in Boston.

Lessons from integrating Pyramid, React, and Plone (and elasticsearch, and Salesforce, and a payment gateway, and...) Quoting from the Plone conference site: "The Volunteer Management System (VMS) is a tool that Jazkarta built for Washington Trails Association to manage volunteer sign-ups for helping maintain hiking trails in the state of Washington. It allows staff to schedule work parties, volunteers to find and register for them, crew leaders to access information about their crew, and land managers to report on work that was done in their region."

The tools you use matter. So use the correct tools.

So at the start, we had the wta.org site, and Salesforce, and vols.wta.org, which was basically a bunch of ten year old scripts.

We could have built everything in Salesforce. It has all kinds of options for doing this, but there are all kinds of restrictions.

We considered Plone. All user accounts were already in here. But we did not need a lot of content management.

On the front-end we used ReactJS. This communicates with Plone for content and with Pyramid for the volunteer management. Pyramid interfaces with Redis, ElasticSearch and Salesforce.

With Pyramid, the view configuration is very close to the actual code, so you don't need a separate zcml file like in Plone. There are no globals, which makes it easier to write isolated tests. Views are easier to test because they return a dictionary instead of a complete html response.

Pyramid is a framework that is good for writing frameworks. We needed to write some infrastructure ourselves, which is nice to do in Pyramid. For example, with config.add_request_method we made a method available on the request to access Salesforce. We registered an exception view to gracefully handle exceptions for either html or json.

With Pyramid, batteries are available for cheap:

  • pyramid_chameleon
  • pyramid_layout
  • pyramid_cachebust
  • pyramid_mailer

As special deployment bonus: we were not using a ZODB, so no ZODB cache was needed, so memory requirements were low, so we could use lots of threads to handle moments of high load.

It is not perfect, for example there is no automatic csrf protection like in Plone. Some assembly is required: Plone has lots built-in and in Pyramid you need to do more yourself, for example setting up logging.

We used the py.test framework, which makes it really easy to define fixtures. There is some magic, which not everyone likes. Assertions are easier to debug, giving you more feedback. We used factory_boy to provide fixtures. [See a talk at PyGrunn earlier this year, by Oleg Pidsanyi. Maurits]

We used http://locust.io for load testing.

React was our chosen Javascript framework. I like that it is declarative. You can focus on what is displayed, not how. A lot of changes, like toggling a checkbox, happen automatically.

We have shared navigation between Plone and Pyramid. The navigation is done in Plone, and we created a json endpoint in Plone that Pyramid could call.

We needed to think about Cross Origin Resource Sharing (CORS), with all kinds of headers. These days I would look at plone.rest and how it does this.

Single sign-on. Pyramid redirects you to Plone, where you can login, and with a token you are then logged in to Pyramid.

Érico Andrei - The Headless CMS

published Oct 20, 2016

Talk by Érico Andrei at the Plone Conference 2016 in Boston.

I am in Berlin via a live video feed. Hello!

In several recent projects with Plone I had a common challenge. The CMS was just one of many parts of the stack. People may view our content on a large computer screen, a mobile phone, or virtual reality glasses, which all need to be handled differently. There are several solutions, like contentful. Some common traits:

  • Everything is through the web.
  • APIs for various languages

Why was I using Plone for that?

  • No more custom PHP development please.
  • Lots of things included.
  • I am still quite productive with Plone.

Biggest reason:

  • I did not look for hosted solutions.

If I build it in Plone, I may be the only one at the company who can maintain it. Not so with a hosted solution.

But: I still love Plone, and want to use it, for good reasons:

  • One API endpoint.
  • Better control over performance issues.
  • Integration with front-end development is essential.

And now I actually have four Plone developers in the team, so that is great. The team is happy with Plone. We use as building blocks:

  • plone.restapi. This saved a lot of time.
  • plone.app.multilingual
  • a policy package
  • PyCharm IDE for writing Python
  • AWS/Kubernetes/Deis
  • Thumbor. Image server over Tornado. Cropping, filters, transforms. We have integrated this in Plone using event subscribers.

Lessons for Plone:

  • documentation, documentation, documentation. Everything I did was possible due to documentation.
  • More documentation, more documentation, more documentation.
  • We need more projects like plone.client.
  • We need more commercial offering of headless hosted Plone.