Weblog
Paul Roeland: An omnivore's perspective
Talk by Paul Roeland at the Plone Conference 2017 in Barcelona.
This is an opinionated view on where we can learn, what we should stop doing, and how choices play out.
What do I mean with an omnivore's perspective? I work for a non-profit organisation. I have used Plone from version 0.99. I build sites, maintain them, renovate them, revive them, keep them, put them to sleep. So I do not build sites and then let someone else handle it. Some sites have content of over twelve years old. Some survive for three years even though the client promised me it would only be there for a year.
What systems do I use? Plone, Wordpress, Drupal, Wagtail, Mezzanine, Ghost, Pagekit, Sulu, Grav, Hugo. Those were the CMSes, well, sort of. I also use CiviVRM, Mailchimp, Odoo. These are ones that I have used in the last twelve months alone.
Some are very good. It helps that some are very new, so they have no balast.
Some good trends that I like:
- design is getting simpler and simpler
- designers got into their heads that there are mobile users
- testing is done better
- staging and production are known words for most systems
There is also the bad:
- developer centric consultantware, where for any change you need to ask a developer, instead of having a lot of power already in your hands like in Plone
- Some are so simple that it is basically brochureware. It works and looks fine as long as you have at most ten pages. So for a larger NGO: maybe not. 'Does it scale?' is really a good question here. Does it survive three years with 300 new pages a year, including several restructurings.
And of course the ugly:
- The worst ideas resurface, like the inline editing that we had in Plone.
- Once you publish it, you cannot change the url ever... Do you work in a real organisation?
- One way street, also known as data grave. Please give me a way to export the data without resorting to crawling the site.
- Security as an afterthought.
What Plone has that is really awesome and mostly not anywhere else:
- placeful content: folders with stuff in them. This means you can find it. You can move a folder somewhere else including all its stuff. Again, a flat structure is fine for ten pages, but not for more.
- collections: show a list of content somewhere else. Awesome, I want more and better of those.
- workflow: just unrivalled. Others may have a bit, but not good.
- content rules: if you do not use them, you are missing out.
Plone's happy place, where it shines, according to me:
- long term content, that you are supposed to keep for years and years. Overkill for a site that is just there for a short term action.
- skilled editors. They do not need to be full time editors, but they should be happy to at least spend half an hour to learn how the system works.
- commitment from the organisation
- where you can trust and empower your power users. They are your ambassadors.
Where can Plone approve? In the way that we approach content management as a whole. We often get bogged down in details in our daily work. But you are a captain and should act like it: Captain Janeway said 'there is coffee in that nebula, go for it,' instead of 'point the ship at direction 180.53 and warp 4.623.' Get the big picture. Tell developers (or Plone) what you want, not how exactly you want it.
Content life:
- 20 percent is spent writing the content outside of the CMS. Say Word or an email or Google docs.
- 5 percent of time is spent getting it into the CMS.
- 75 percent is spent to re-arrange it, re-use, refer, tag, archive. In other words: content management. It is annoying if it takes twenty clicks to get anything done.
Content types:
- Forget dexterity versus Archetypes for a moment.
- You have text and images.
- Embed: youtube, twitter, etc. It is there on basically every site.
- Snippets and results. Extra things like 'people you bought this article, also bought these other three, so please by them as well'.
- Office and PDF. People expect to see it this in the site. People do not see what the difference is between a Word file and an html page. Why is one opening in Word and another in the site? Make it easy for them with some integration. There are ways.
- Composite pages. This is a hard problem. We had at least seven attempts, the last one Mosaic, which is on its way, but good be better. Remember: be a captain. Make it possible to say: item A is the most important, show items B and C if you have room, drop item D and E when you are on a phone. What if we all get extremely large screens or extremely small screens in your glasses? Do you really need to go through all your pages then?
Sub sites:
- Folder, Composite and Theme.
- limited navigation
- handmade is good enough
- If the sub site is too big, with too many non standard options, make it a separate site.
WYSIWYG:
- What You See Is What You Get?
- Actually: What I See Is Most Likely Not What You Get
- Instead, Markdown is fine for most sites. It is limited, but that is a strength. The editors are less tempted to paste a Word doc into it. A preview is nice.
Stop assuming that something will appear in the left column, because on mobile there may be only one column, or it will not appear at all.
Forms:
- Basically all form frameworks are sh*t.
- PloneFormGen is Archetypes and no one is going to keep fixing the fields, so use collective.easyform.
- A quick edit drag and drop page is nice, but form creation is for power editors, so it is fine if the interface is geeky. So focus on the end user, that the end result looks good.
System setup:
- should be repeatable
- containers are here to stay, whether Docker or something else. If you want to do it in a different way, we can say in docs that you are on your own. We should stop supporting all the different deployment strategies in the world.
Configuration:
- Readability counts. It does not need to be shiny: I don't configure a site every day for hours.
- TTW: allow theming override. A different color should be possible. Allow overriding the translation strings through the web, let the marketing people do that.
- I don't want to do Javascript TTW, just on the command line with webpack or whatever. I was always in favour of TTW, but here: no.
Don't reinvent wheels. For example, one of the biggest problems in computer science: writing responsive emails. Integrate something like mosaico that has solved this.
My practical wishlist for users:
- TinyMCE: use Markdown and raw
- tiles and layout recipes
- Make 'embed' easy.
Give me more:
- smart media handling, without letting me handle it manually
- consistency in UI
- clevererer collections: can we have something that says: these three items are relevant for you?
Team player:
- Good import and export should be in core. JSON dump is fine.
- REST and GraphQL: good if we can have both.
- Office integration when needed for intranet sites
- although security is hard
System:
- For configuration can we move from XML to YAML? No, not JSON, as that is unreadable and unwritable too.
- One Plone per Zope would be much easier.
- A command line interface to do basic maintenance would be great.
- roundtrip configuration: export config and import in another container. We sort of have this, but it is hard to find.
- For power users, function is more important than form.
- Warn me when updates are needed. Should be configurable. Probably not web downloadable updates, for security reasons.
Make it so!
Maybe you totally disagree, that is fine. Come talk to me and we can start a discussion. I am deeply partial to Plone and let's make it better.
Nathan van Gheem and Ramon Navarro Bosch: guillotina and asyncio
Talk by Nathan van Gheem and Ramon Navarro Bosch at the Plone Conference 2017 in Barcelona.
At first we had plone.server as name, but it was not really Plone anymore, so we were looking for a new name and come up with guillotine, and then Ramon said we should call it guillotina, the Catalan name for it.
Some background. There are web frameworks like Angular and React. Server rendering frameworks are dying out: the front end just wants to talk to an API. We love Plone, but wanted to be able to use it in high performant situations with modern front ends.
History:
- 18 years ago, Zope and the ZODB were created, an object oriented application and database.
- 16 years ago Plone got built on top of it.
- 7 years ago Pyramid created.
- 2 years ago Plone REST API created
- a bit more than one year ago plone.server/guillotina was born.
Guillotina is an evolution, done in the spirit of Plone. It is not necessarily a replacement for Plone. We are taking lessons learned. We are okay with forking some code to change it to our needs. We are not going to provide everything out of the box. Zope and Plone are inspirations, they are in our heads, like the hierarchical data model. It is not a re-implementation of Plone, it is not plone.restapi compatible. It is an asynchronous REST API service.
We have transactions, with conflict resolution policies that reduce conflict errors and gives better performance. We want to use the best database systems available, we support PostgreSQL and CockroachDB.
Information is organised in trees of objects. We love that in Plone and decided to keep it. Objects are resources, with schema attributes and annotations, OO inheritance and static or dynamic behaviors. The data is serialised as json.
Other features:
- Security: similar to how this is in Zope and Plone, but simplified.
- All code is based on asyncio for network integrations with external indexers, database, caching, services. It is based on aioHTTP.
- We want the installation to be simple. You can do pip install guillotina or docker run guillotina/guillotina.
- You can configure how CORS should be handled.
- It is easy to use web sockets.
- It is perfect for micro services, like handled in lots of containers with kubernetis.
- It is extensible, uses zope.interface and based on zope.component, with utilities and adapters.
- You can use Cookie Cutter to create a new project.
- We have a persistent configuration registry for each container. A container is comparable with a Plone Site inside a Zope database.
- You can point to a static directory for static files and javascript apps.
- You can mount multiple databases at the same time.
- We have open sourced our S3 and GCloud storage packages, and ElasticSearch indexing.
- Automatic API documentation generation using Swagger. With guillotina-swagger you get clickable documentation where you can try everything out.
- Give a container 200 MB of RAM and it will handle hundreds of requests per second.
- Urls are like: /db/container-name/obj1/obj2
- zope.interface is the only package that we kept from Zope/Plone.
- Only Python 3.6 or higher.
- Hopefully we get some reusable components that are also used for the Plone REST API.
Links:
- code: https://github.com/plone/guillotina
- docs: http://guillotina.readthedocs.io/en/latest/
- docker: https://hub.docker.com/r/guillotina/guillotina/
- modules: https://github.com/guillotinaweb
- our Onna company: https://github.com/onna
Devon Bernard: How to design a great API using Flask
Talk by Devon Bernard at the Plone Conference 2017 in Barcelona.
At Enlitic we study lots of medical data using computers. I hope to teach you how to create an api that developers and users love. If you are a backend developer, you can see the front end developers as your user. It should be intuitive, durable, flexible.
Intuitive
Your API should not surprise your users.
Hard coded variables are bad. Keep runtime variables in one central place, so users can find them. Make it so that you do not need to juggle with git to avoid accidentally committing a password. Search github for 'removed password' and you find over 300,000 commits with that commit message... Good: create a Config class, and DevelopmentConfig, and ProductionConfig and handle the differences there.
Use object relational mappers (ORM), instead of writing SQL, as you constantly worry about SQL injections.
How to standardise database migrations? A tool like alembic helps, to create a way to upgrade and downgrade. But please change the alembic file template to start with a date instead of a random hash.
Have a standard database seed, that users can use to get a database with standard content.
Use a requirements.txt file with all the requirements that you need. You can get packages from a git repository too. And you can install packages from the local file system, which is especially handy if you develop two packages at the same time, so you don't need to push changes all the time for package A when you yet know if they actually work with package B.
Is the setup code in your README working? If you can replace this by a single command, that would be better.
Create unit tests. You may hear:
- "But it wastes time." No, you still have to verify your code before you deploy it to production. And you do not just need to verify the new code, also the older code, as it may be affected by your change.
- "But writing it is boring." Manually clicking to verify that it works, is even more boring.
Database flush versus commit. A flush reserves a placeholder spot in the database. A commit saves records in the database. If you explicitly commit, and later something goes wrong, your commit is still in the database; usually you do not want this: either all changes should be written, or none at all. Also: use flush instead of commit when you need to get an auto increment id back: what is the id of the row that will be committed to the database.
Flexible
Create an app factory, so there is one way to instantiate your app. This may also help for avoiding circular imports, like when your API depends on your database code, and the other way around.
Create blueprints. For example create one decorator for registering an admin view, and another for anonymous views. That makes it very clear which view is for what kind of user.
Reliable
How to maximize uptime? Prevent downtime by not shipping bugs: have automated testing, have staging servers
Versioning: have a way to know what the used version is. Keep backwards compatibility. Communicate when you will drop a feature.
Get some analytics about API usage. If one of your front end developers is still calling a deprecated part of the API, detect this and send them an email.
First rule of endpoint design is: be consistent.
Document your API, or it does not exist.
A tool like Postman can be used to show the end points, and is helpful for debugging: paste the exact payload that gave an error on production.
Use Python profiling: what lines of your code take the most time?
Caching: store common responses in memory for quick retrieval.
Find me on Twitter: @devonwbernard.
Eric Steele: State of Plone
Keynote talk by Eric Steele at the Plone Conference 2017 in Barcelona.
Welcome to Barcelona! Great to be here. The Plone Barceloneta theme was made here, guillotina started here.
Plone 5.1 release candidate 1 is out.
Some of the features:
- collective.indexing integrated in core, to speed up indexing operations
- new portal actions control panel, so you can manage these in the Plone UI
- we support Retina (high definition) scales
- auto rotation for images
When: soon.
The stories that preserve our past. I want to talk about stories today. People leave projects, other people replace them, if you are doing it right. Contributors start of doing small things now and then, and then get hooked and start to do more and more, and then it usually goes down, and they may go to another job and another project than Plone. When all is well, others persons then stand up, and the project still continues.
Debian: developer half life of about 7.5 years: after that time, half of the developers are no longer involved. Others step up. Continuous renewal is needed, keeping the project fresh. You run into problems when a new generation does not step up quickly enough. If knowledge isn't shared, it is lost.
Alex Limi: we can have heated discussions in the community online, but usually you have seen the other person on a conference or sprint, and this helps in keeping the discussion healthy.
Antoine de Saint-Exupéry: if you want to build a ship, you don't tell people to get hammers and nails and wood and sail, but you teach people to yearn for the sea.
Blog post Martin Aspeli years ago: Pete and Andy try Plone 4, where he highlights some things that they like and that they miss. Several of these ideas are in current Plone.
Cal Doval was a designer who created a mockup of how the add-ons control panel could work. People saw this and knew that they wanted it. It is now in Plone.
It's time for new stories. For example: the Pastanaga UI, which Timo Stollenwerk will now talk about.
Timo
Pastanaga UI was created by Albert Casada. Pastanaga means carrot, and the logo is a stylised carrot. Albert worked on this in his free time, creating hundreds of icons, etc. I was impressed before I even saw it. Main ideas:
- Simplify. Do not show what the user does not need to see.
- Adaptive user interface. More people are viewing web sites on mobile devices than on desk tops now. You UI should work good on all kinds of devices.
- Focus. I was writing a blog post in my code editor. [Note from Maurits: me too.] We want users to do that in Plone. Most current CMSes use TinyMCE or a similar editor, like Plone does, and we no longer stand out. The most time of our users is probably spent in TinyMCE, so we should make this better again.
Pastanaga is a carrot. Albert said he wanted a carrot on a stick, to entice people into using Plone again.
Another story. Ramon did all the stuff on plone.rest, I saw he wast smart, so I just nodded and created a few tests. I did not have much time to work on it afterwards. Others came and made it work with Archetypes, which we did not initially want, but they improved our initial code to make it flexible, so why not. Then Eric Bréhault came along for plone.restapi. Then people worked on an angular client. So: we told a story, did some work, and others jumped in.
It is hard to maintain stuff on our own. We must become consumers of libraries that are already there, especially javascript libraries. Do not reinvent, but reuse.
For us developers, it is cool to create a version of Plone that works on Zope 4, or Python 3, but clients mostly do not care. They need something else, like the Pastanaga UI.
Make Plone outstanding again.
Eric
It's time for new stories. Start writing them. Thank you.
Genesis
Column Klokgelui voor Overschiese Krant.
U snapt wel waar deze column over gaat: de tentoonstelling Genesis, in het Fotomuseum. Ik had als titel eerst 'blote borsten' gekozen. Die zag ik namelijk op de tentoonstelling, en zo'n titel trekt meer de aandacht, als ik mijn eigen gedachtengang tenminste mag doortrekken naar de gedachten van de gemiddelde lezer van deze krant. Maar om borsten nu ook al als lekkermakertje bij een column namens de kerken te gebruiken, ging me te ver. Niet getreurd: in de inhoud komt u ze zeker tegen.
De fototentoonstelling bestaat uit prachtige zwart-witfoto's uit de natuur. Daarbij zitten ook foto's van mensen die in deze natuur leven. En dat ziet er anders uit dan de meeste familiekiekjes van een gezin uit Overschie. Je ziet blote borsten, beschilderingen, vaak met modder, weelderige pruiken, peniskokers, schotels door lippen of oorlelletjes, pinnen door een neus of zelfs staven door de kin heen. Ze zouden flink opvallen bij een wandeling over de Burgemeester Baumannlaan.
Maar ook zij zouden hun ogen uitkijken als ze ons zagen. Geen beschilderingen met modder, maar wel tattoos. Geen pruiken, hoewel, maar wel hanenkammen. Of gewoon lang, blond haar: ook ik vind dat exotisch. Geen lange peniskokers, wel grote auto's of dure mobieltjes, met wellicht hetzelfde doel. Geen schotels of pinnen door lippen en neuzen, maar wel overal piercings en oorbellen. En blote borsten zie je niet snel op de openbare weg, maar daar hebben we dan weer TV, internet en snapchat voor uitgevonden.
En zo zien wij dit soort volken misschien op het eerste gezicht als primitief, maar eigenlijk zijn ze heel hip en niet zo verschillend van ons.
Ik lees nu een boek, Het recht van de Radch, waarin iedereen wordt aangeduid als 'zij' of 'haar', ook als het over een man gaat. Er staat 'zij gaf haar een hand', en je hebt geen idee of het over twee vrouwen, twee mannen, of een man en een vrouw gaat. Aan kleding of haardracht kan je het ook niet zien, want alle combinaties daarvan vind je bij mannen en vrouwen. Dus waar in de tentoonstelling het verschil tussen man en vrouw duidelijk is, is dat in dit boek totaal anders.
In een ander boek, de bijbel, zegt God: 'Wat je van buiten ziet, is niet belangrijk. Daar kijken mensen altijd het eerst naar, maar ik let erop hoe een mens van binnen is.' Dat is een goede blik om mee door de fototentoonstelling te lopen, of door Overschie.