Weblog
Nathan van Gheem and Ramon Navarro Bosch: guillotina and asyncio
Talk by Nathan van Gheem and Ramon Navarro Bosch at the Plone Conference 2017 in Barcelona.
At first we had plone.server as name, but it was not really Plone anymore, so we were looking for a new name and come up with guillotine, and then Ramon said we should call it guillotina, the Catalan name for it.
Some background. There are web frameworks like Angular and React. Server rendering frameworks are dying out: the front end just wants to talk to an API. We love Plone, but wanted to be able to use it in high performant situations with modern front ends.
History:
- 18 years ago, Zope and the ZODB were created, an object oriented application and database.
- 16 years ago Plone got built on top of it.
- 7 years ago Pyramid created.
- 2 years ago Plone REST API created
- a bit more than one year ago plone.server/guillotina was born.
Guillotina is an evolution, done in the spirit of Plone. It is not necessarily a replacement for Plone. We are taking lessons learned. We are okay with forking some code to change it to our needs. We are not going to provide everything out of the box. Zope and Plone are inspirations, they are in our heads, like the hierarchical data model. It is not a re-implementation of Plone, it is not plone.restapi compatible. It is an asynchronous REST API service.
We have transactions, with conflict resolution policies that reduce conflict errors and gives better performance. We want to use the best database systems available, we support PostgreSQL and CockroachDB.
Information is organised in trees of objects. We love that in Plone and decided to keep it. Objects are resources, with schema attributes and annotations, OO inheritance and static or dynamic behaviors. The data is serialised as json.
Other features:
- Security: similar to how this is in Zope and Plone, but simplified.
- All code is based on asyncio for network integrations with external indexers, database, caching, services. It is based on aioHTTP.
- We want the installation to be simple. You can do pip install guillotina or docker run guillotina/guillotina.
- You can configure how CORS should be handled.
- It is easy to use web sockets.
- It is perfect for micro services, like handled in lots of containers with kubernetis.
- It is extensible, uses zope.interface and based on zope.component, with utilities and adapters.
- You can use Cookie Cutter to create a new project.
- We have a persistent configuration registry for each container. A container is comparable with a Plone Site inside a Zope database.
- You can point to a static directory for static files and javascript apps.
- You can mount multiple databases at the same time.
- We have open sourced our S3 and GCloud storage packages, and ElasticSearch indexing.
- Automatic API documentation generation using Swagger. With guillotina-swagger you get clickable documentation where you can try everything out.
- Give a container 200 MB of RAM and it will handle hundreds of requests per second.
- Urls are like: /db/container-name/obj1/obj2
- zope.interface is the only package that we kept from Zope/Plone.
- Only Python 3.6 or higher.
- Hopefully we get some reusable components that are also used for the Plone REST API.
Links:
- code: https://github.com/plone/guillotina
- docs: http://guillotina.readthedocs.io/en/latest/
- docker: https://hub.docker.com/r/guillotina/guillotina/
- modules: https://github.com/guillotinaweb
- our Onna company: https://github.com/onna
Devon Bernard: How to design a great API using Flask
Talk by Devon Bernard at the Plone Conference 2017 in Barcelona.
At Enlitic we study lots of medical data using computers. I hope to teach you how to create an api that developers and users love. If you are a backend developer, you can see the front end developers as your user. It should be intuitive, durable, flexible.
Intuitive
Your API should not surprise your users.
Hard coded variables are bad. Keep runtime variables in one central place, so users can find them. Make it so that you do not need to juggle with git to avoid accidentally committing a password. Search github for 'removed password' and you find over 300,000 commits with that commit message... Good: create a Config class, and DevelopmentConfig, and ProductionConfig and handle the differences there.
Use object relational mappers (ORM), instead of writing SQL, as you constantly worry about SQL injections.
How to standardise database migrations? A tool like alembic helps, to create a way to upgrade and downgrade. But please change the alembic file template to start with a date instead of a random hash.
Have a standard database seed, that users can use to get a database with standard content.
Use a requirements.txt file with all the requirements that you need. You can get packages from a git repository too. And you can install packages from the local file system, which is especially handy if you develop two packages at the same time, so you don't need to push changes all the time for package A when you yet know if they actually work with package B.
Is the setup code in your README working? If you can replace this by a single command, that would be better.
Create unit tests. You may hear:
- "But it wastes time." No, you still have to verify your code before you deploy it to production. And you do not just need to verify the new code, also the older code, as it may be affected by your change.
- "But writing it is boring." Manually clicking to verify that it works, is even more boring.
Database flush versus commit. A flush reserves a placeholder spot in the database. A commit saves records in the database. If you explicitly commit, and later something goes wrong, your commit is still in the database; usually you do not want this: either all changes should be written, or none at all. Also: use flush instead of commit when you need to get an auto increment id back: what is the id of the row that will be committed to the database.
Flexible
Create an app factory, so there is one way to instantiate your app. This may also help for avoiding circular imports, like when your API depends on your database code, and the other way around.
Create blueprints. For example create one decorator for registering an admin view, and another for anonymous views. That makes it very clear which view is for what kind of user.
Reliable
How to maximize uptime? Prevent downtime by not shipping bugs: have automated testing, have staging servers
Versioning: have a way to know what the used version is. Keep backwards compatibility. Communicate when you will drop a feature.
Get some analytics about API usage. If one of your front end developers is still calling a deprecated part of the API, detect this and send them an email.
First rule of endpoint design is: be consistent.
Document your API, or it does not exist.
A tool like Postman can be used to show the end points, and is helpful for debugging: paste the exact payload that gave an error on production.
Use Python profiling: what lines of your code take the most time?
Caching: store common responses in memory for quick retrieval.
Find me on Twitter: @devonwbernard.
Eric Steele: State of Plone
Keynote talk by Eric Steele at the Plone Conference 2017 in Barcelona.
Welcome to Barcelona! Great to be here. The Plone Barceloneta theme was made here, guillotina started here.
Plone 5.1 release candidate 1 is out.
Some of the features:
- collective.indexing integrated in core, to speed up indexing operations
- new portal actions control panel, so you can manage these in the Plone UI
- we support Retina (high definition) scales
- auto rotation for images
When: soon.
The stories that preserve our past. I want to talk about stories today. People leave projects, other people replace them, if you are doing it right. Contributors start of doing small things now and then, and then get hooked and start to do more and more, and then it usually goes down, and they may go to another job and another project than Plone. When all is well, others persons then stand up, and the project still continues.
Debian: developer half life of about 7.5 years: after that time, half of the developers are no longer involved. Others step up. Continuous renewal is needed, keeping the project fresh. You run into problems when a new generation does not step up quickly enough. If knowledge isn't shared, it is lost.
Alex Limi: we can have heated discussions in the community online, but usually you have seen the other person on a conference or sprint, and this helps in keeping the discussion healthy.
Antoine de Saint-Exupéry: if you want to build a ship, you don't tell people to get hammers and nails and wood and sail, but you teach people to yearn for the sea.
Blog post Martin Aspeli years ago: Pete and Andy try Plone 4, where he highlights some things that they like and that they miss. Several of these ideas are in current Plone.
Cal Doval was a designer who created a mockup of how the add-ons control panel could work. People saw this and knew that they wanted it. It is now in Plone.
It's time for new stories. For example: the Pastanaga UI, which Timo Stollenwerk will now talk about.
Timo
Pastanaga UI was created by Albert Casada. Pastanaga means carrot, and the logo is a stylised carrot. Albert worked on this in his free time, creating hundreds of icons, etc. I was impressed before I even saw it. Main ideas:
- Simplify. Do not show what the user does not need to see.
- Adaptive user interface. More people are viewing web sites on mobile devices than on desk tops now. You UI should work good on all kinds of devices.
- Focus. I was writing a blog post in my code editor. [Note from Maurits: me too.] We want users to do that in Plone. Most current CMSes use TinyMCE or a similar editor, like Plone does, and we no longer stand out. The most time of our users is probably spent in TinyMCE, so we should make this better again.
Pastanaga is a carrot. Albert said he wanted a carrot on a stick, to entice people into using Plone again.
Another story. Ramon did all the stuff on plone.rest, I saw he wast smart, so I just nodded and created a few tests. I did not have much time to work on it afterwards. Others came and made it work with Archetypes, which we did not initially want, but they improved our initial code to make it flexible, so why not. Then Eric Bréhault came along for plone.restapi. Then people worked on an angular client. So: we told a story, did some work, and others jumped in.
It is hard to maintain stuff on our own. We must become consumers of libraries that are already there, especially javascript libraries. Do not reinvent, but reuse.
For us developers, it is cool to create a version of Plone that works on Zope 4, or Python 3, but clients mostly do not care. They need something else, like the Pastanaga UI.
Make Plone outstanding again.
Eric
It's time for new stories. Start writing them. Thank you.
Genesis
Column Klokgelui voor Overschiese Krant.
U snapt wel waar deze column over gaat: de tentoonstelling Genesis, in het Fotomuseum. Ik had als titel eerst 'blote borsten' gekozen. Die zag ik namelijk op de tentoonstelling, en zo'n titel trekt meer de aandacht, als ik mijn eigen gedachtengang tenminste mag doortrekken naar de gedachten van de gemiddelde lezer van deze krant. Maar om borsten nu ook al als lekkermakertje bij een column namens de kerken te gebruiken, ging me te ver. Niet getreurd: in de inhoud komt u ze zeker tegen.
De fototentoonstelling bestaat uit prachtige zwart-witfoto's uit de natuur. Daarbij zitten ook foto's van mensen die in deze natuur leven. En dat ziet er anders uit dan de meeste familiekiekjes van een gezin uit Overschie. Je ziet blote borsten, beschilderingen, vaak met modder, weelderige pruiken, peniskokers, schotels door lippen of oorlelletjes, pinnen door een neus of zelfs staven door de kin heen. Ze zouden flink opvallen bij een wandeling over de Burgemeester Baumannlaan.
Maar ook zij zouden hun ogen uitkijken als ze ons zagen. Geen beschilderingen met modder, maar wel tattoos. Geen pruiken, hoewel, maar wel hanenkammen. Of gewoon lang, blond haar: ook ik vind dat exotisch. Geen lange peniskokers, wel grote auto's of dure mobieltjes, met wellicht hetzelfde doel. Geen schotels of pinnen door lippen en neuzen, maar wel overal piercings en oorbellen. En blote borsten zie je niet snel op de openbare weg, maar daar hebben we dan weer TV, internet en snapchat voor uitgevonden.
En zo zien wij dit soort volken misschien op het eerste gezicht als primitief, maar eigenlijk zijn ze heel hip en niet zo verschillend van ons.
Ik lees nu een boek, Het recht van de Radch, waarin iedereen wordt aangeduid als 'zij' of 'haar', ook als het over een man gaat. Er staat 'zij gaf haar een hand', en je hebt geen idee of het over twee vrouwen, twee mannen, of een man en een vrouw gaat. Aan kleding of haardracht kan je het ook niet zien, want alle combinaties daarvan vind je bij mannen en vrouwen. Dus waar in de tentoonstelling het verschil tussen man en vrouw duidelijk is, is dat in dit boek totaal anders.
In een ander boek, de bijbel, zegt God: 'Wat je van buiten ziet, is niet belangrijk. Daar kijken mensen altijd het eerst naar, maar ik let erop hoe een mens van binnen is.' Dat is een goede blik om mee door de fototentoonstelling te lopen, of door Overschie.
Dat ene woord
Column Klokgelui voor Overschiese Krant.
Ik loop de laatste tijd in mijn hoofd met een melodie: 'Niets is sterker dan dat ene woord.' Misschien neuriet u dat ook wel. Welk woord was het ook al weer? Niets is sterker dan dat ene woord: agenda! WhatsApp! Facebook! Succes!
Is het niet zo? Je bent aan het werk, of aan het praten met een vriend of met je kind, en dan komt er een appje tussendoor en je bent afgeleid. Letterlijk afgeleid: je laat je leiden door je telefoon. Je telefoon of je agenda is geen handig hulpje meer, maar is je baas.
Het leven is een nooit eindigend spel. Bij voetbal win en verlies je wedstrijden, en aan het eind van het seizoen is Feyenoord de winnaar. In het leven heb je succesvolle periodes, en op andere momenten zit het tegen of is het zelfs een groot drama, en aan het eind ga je dood. Ik geloof dat er leven is na de dood, maar dat is nu niet mijn punt. Je wordt aan het eind van je leven niet gehuldigd op de Coolsingel. Je hoeft dus niet de eerste te zijn, de beste, de snelste, de mooiste, de slimste. Je hoeft niet de meeste appjes gestuurd te hebben om tot winnaar gekroond te worden in het leven. Er is geen roze trui voor wie het snelste reageert op berichten, of voor wie de meeste volgers op Facebook of Twitter heeft.
Als je bang bent om iets te missen, dan mis je het belangrijkste. Dan volg je aandachtstrekkers en mis je dat wat kalm en bescheiden wacht tot je tijd hebt.
Dus zoek eens de stilte. Geen stilte voor de Twitterstorm, maar stilte voor de stilte. Als stilte saai is, verveling, moet je meer oefenen, het langzaam opbouwen.
Toen Jezus op aarde rondliep, nam hij na de drukte van de dag tijd om met zijn vader te praten. Hij liet zich niet beïnvloeden door politieke of sociale dagkoersen. Hij was sterk doordat hij bouwde op het fundament van zijn relatie met God de vader en belangrijk vond wat de maker van het leven belangrijk vond.
Ja, niets is sterker dan dat ene Woord: Jezus.