Plone
This is here to serve as contents for the atom/rss feed for Plone, also read by planet.plone.org.
Michele Finelli: Software engineering during the Italian Renaissance
Talk by Michele Finelli at the Plone Conference 2019 in Ferrara.
My first Plone conference was in Vienna. Glad to be here now.
Everything begins in ancient Babylonia. They had problems like measuring the size of fields after a flood. So they developed geometry (in Egypt too). They had to solve quadratic equations, like this:
a*x*x + b*x + c = 0
They used a different formula than what we are used to, because they had difficulty with the concept of negative numbers: you cannot have a field with a length of minus one meter.
This is basically software engineering, right?
Fast forward to the Renaissance. It is the year 1494, and Luca Pacioli publishes a book on mathematics. He said that the cubic equation (similar to the last, but raising to the third power) would probably never have a general solution. He was wrong. I show you a solution in Python, taken from Wikipedia.
Scipione dal Ferro found a solution a few years later. He seems to have come up with an algorithm to calculate it. Start with a formula, turn this into a series of instructions, and you have an algorithm.
Enter Gerolamo Cardano, Niccolo Tartaglia, and Lodovico Ferrari, all from the sixteenth century. Tartaglia also found a solution, shared it secretly with Cardano, who taught it to his student Ferrari. Cardano found the original solution from Dal Ferro, thought himself no longer sworn to secrecy, and eventually published the solution with Ferrari, together with a solution for the quadratic equation. Tartaglia (nickname for stutterer) became an enemy, and they fought mathematical battles, and we have some papers from this.
When you calculate with cubic roots, part of the solution uses the square root of minus one. At the time this was garbage. Currently we know this as an imaginary number. So it raised questions, which were finally answered two centuries later by Lagrange, Gauss, Ruffini, and Abel, between 1770 and 1826.
Why did this take so long? They lacked the abstractions needed to move forward. Babylonians did not know how to deal with negative numbers. Dal Ferro and company did not know how to deal with imaginary or complex numbers.
Moving forward. Is there a field where the lack of abstractions is preventing progress. I think yes, and it is called software engineering.
- How do we specify what a software should and should not do?
- How do we validate the behavior of code we write?
- How do we describe our architectures?
The answer is currently text: sentences, diagrams, graphs.
The history of the discovery of the mathematical solutions was a tale to tell that in science, progress usually stalls until we are able to go from one language to another. In software, of course there are new programming languages, but I don't mean that.
The Agile Manifesto tells fruitful and interesting things, and gives important results. But it does not improve how we do the software thing.
Software has built a different world, just for example the internet. It should also be a better world. Privacy is going down, are we building a police world, without intending to?
We need a quest to find a better theory of software.
Lightning talks Wednesday
The Wednesday Lightning talks at the Plone Conference 2019 in Ferrara.
Michele Finelli: Salama
This week I will do three easy pieces on Ferrara. And a rant.
You need to read this book: La Salama da Sugo. Salama = ninino (pig). This food is cooked, for a very long time, so the fat goes away. You do NOT eat it with bread. Eat it with purè, like mashed potatoes but better.
PyConWeb
A conference about web tools in Python. In Munich Germanu in May this year. We have many Plone talks and a training. See you next year?
Asko Soukka: plonetheme.webpacktemplate
One more way to theme your Plone. Webpack development server. Live demo. This is how we themed all our Plone 5 sites.
Code: https://github.com/collective/plonetheme.webpacktemplate
Sally and Kim: Plone Open Garden 2020
April 19-26 2020 in Sorrento, Italy. Registration is open soon.
Armin Stross-Radschinski: Search in Docs matter
(I lost a camera, if you find it, please give it to me.)
The default search in Sphinx really sucks. Local JavaScript, ugly, not translation proof.
Pretty live search with Algolia. Commercial SAAS engine. Free for open source docs.
It is already implemented for https://docs.plone.org. It uses Algolia, but not properly setup yet. Join my in a sprint on this!
Maik Derstappen: plonecli
We did a lot of work continuing on plonecli, the Plone command line client. Talk in detail tomorrow. If you are new to backend development in Plone, you should have a look.
Nejc Zupan: PyCon Balkan
We will do a Python conference in Slovenia. 18-20 April 2020 in Ljubljana, and three days of sprints afterwards. A track for web and data science. Also a workshop track. Come with the family. Everybody speaks English. We will do some excursions before.
Alexander Pilz and Johannes Raggam: py-spy
Debugging performance issue in production. We usually use Products.LongRequestLogger, it gives you the backtraces of long-taking requests on your server. But it is not compatible with WSGI and Python 3.
py-spy is non-intrusive. Just pip install py-spy. And then py-spy top and other commands. You can make flame graphs. You can attach it to a running process and get information from it.
Michael Töpf: React
Video with Manuel Bieh on the hype of React and why it is taking over the UI development. https://www.youtube.com/watch?v=ykaWoBDzuYA&feature=youtu.be
Panel: Future of Plone
Panel on the future of Plone at the Plone Conference 2019 in Ferrara.
Panel with Paul Roeland, Philip Bauer, Timo Stollenwerk, Asko Soukka, Kim Paulissen. Panel chaired by Tim Nguyen.
This panel is not about Plone 6, but about what happens after: what could be in Plone 7, without thinking about limits.
What would you like to see as the biggest changes in Plone 7?
- Some sort of machine learning, for example to provide tags for all images. Important for accessibility. The same for giving keywords.
- Scheduling that works. Press release to go out at 9.00 on Monday should not be visible even when you know the url, which is what happens now.
- Reliable intelligent pasting from Word and email.
- Get rid of lots of old cruft in Plone. Not have so many ways to do the same thing. No more browser views, because we have a REST API and front end.
- Get rid of the server rendered interface. Move all Plone UI to the ZMI, for admins who know what they are doing.
- Make it easy for users to customize things without knowing Python.
- Get rid of ZODB, ZMI, pickles, GenericSetup.
- More Python, not too many different technologies.
- Make Plone friendlier for smaller websites. Custom easy themes.
- A form editor that can easily be reused as schema editor. Export as json schemata, and the Python schemas.
- Customization is really hard. We had Products.Gloworm a long time ago. Click on a part of the site, and it shows you where it comes from, and allows you to customize it. This would make the first step of customizing easier.
What are the main pain points you have with Plone today?
- When you create a new content type, and you add it to your site, it does not show up in the navigation. We should fix this. It bothers me that no one created an issue for that. We have a smart community, but people think that things have been decided by a secret committee, and there must be a good reason for it, so they never question it. Usually decisions are made five minutes before a final release, so it may not be the best solution.
- The byline: I don't want the author to show, and you can switch this off, but it always shows somewhere.
- Language and date/time notation should not be linked.
- Accessibility should be more enforceable.
- The entire front end stack: that is why we wrote Volto.
- Behaviors: how we structure behaviors is done differently in Guillotina. We have lots of layers of flexibility, which is good, but also too much. At some point we have to take a step back.
- I hate our TinyMCE link dialog.
- Sane key bindings please, like CTRL-Enter to save the page.
What makes you envious when you look at other systems?
- I regularly use the web forms from Drupal. Immensely powerful and user friendly.
- Castle CMS has content quality checks.
- Doing migrations is hard, so I am envious about systems that don't care about their users. I am joking.
- Speed of GatsbyJS
- Stability of Guillotina
- Quick-start documentation. Short screen casts.
How can we remain approachable for enthusiasts?
- Django had a perfect introduction. We need people that grow up with Plone. Easy tools to get your content in and out: we have those ways, but can be improved and documented more clearly.
- For beginners it is hard to install add-ons. Buildout is new to them. Installing add-ons in a live site would be helpful.
- Our developer team got energy from using the latest front end technologies. Keep up with the pace.
- The feeling of the Plone community is very good. We try to do that for React as well. Building a community is incredibly important. Plone can give this to the JavaScript communities.
What will our theming story be? Will everything need to be done through Volto, and where will that leave Diazo.
- Technically you can still use Diazo. With or without Diazo, theming has basically taken the same amount of time. Diazo gets you in the right direction faster, but you still need to do more.
- Keep things simple: don't create and maintain our own difficult tooling. Volto builds on React and Semantic UI, who did the heavy lifting.
- Plone is a do-ocracy: do what you want, you can use and improve Diazo.
- For non-programmers, Diazo is still hard.
When will Plone drop server side rendering?
- We have server side rendering of JavaScript. But I guess the question means Page Templates.
- Plone 6 will keep the classic UI fully functional, is the plan.
- After that, we can decide what to do. We could reduce two thirds of our code if we drop it.
- It hurt when we moved to Python 3, because it was a lot of code. Currently it does not really hurt. But we can propose to remove it.
- For Plone 6 we can say that PLIPs do not need to have traditional Page Templates.
How much of the REST API would Django CMS need to implement before we can call it Plone?
- If the API contract is implemented 100 percent, then I don't care whether it is Django or Plone.
- 180 percent, because Plone is also add-ons. Or 240 percent, because Plone is also the community.
- Guillotina could replace Plone eventually. Django is just really different, making it hard to match the API.
How will Plone run 30 percent of sites on the Internet. In other words, how would Plone recapture mind share?
- I refuse to discuss marketing in a bigger group than two.
- Be better at introducing new types of developers to Plone. One for Python developers, one for React developers, for tinkerers, for users, etcetera. Make it easier to get into, and to grow.
- Have plone.io website where people can try out Plone, deploy small sites.
- Reaction to Volto was overwhelmingly positive.
- Creating custom content types is really important for a CMS now. If we can provide that in an easy way, that really helps. People create things of which I think: Why are you doing this, Plone does that out of the box.
If you have improvement ideas, please do not suffer in silence. Talk with people, create a PLIP issue.
Nathan, Ramon and Eric: Status of Guillotina
Talk by Nathan van Gheem, Ramon Navarro Bosch and Eric Brehault at the Plone Conference 2019 in Ferrara.
Guillotina is a full stack REST API data framework to build applications. It is built on AsyncIO. We think it is simple, but with batteries included. We are inspired by the best of Python: ideas from Pyramid, Plone. We want to keep to the spirit of the Plone community. We want to build tooling so that creating a front end on top of Guillotina is easy.
Everything is defined in a JSON Schema / Open API 3 / Swagger. We have hierarchical content with security, so very Plone-like. Transactionally safe with PostgreSQL. Pub/Sub, indexing, lots more.
We released Guillotina 5 this Summer, requiring Python 3.7 or higher. AsyncIO has matured there, with a great context vars api, letting us simpligy our code.
Use cases. What are we using it for in production?
- CMS.
- Data Framework mix. Secure personal data and application data, gathered automatically. Reports. Partially data that needs to be searched: pgcatalog and ElasticSearch. Partially data that needs to be secure and massive: cockroach.
- Machine learning processing. I often meet data scientists that always want a csv export. That takes too long for so much data. So have a front end that directly accesses the data in Guillotina, without needing an export. We use pickles to store data, and now this can contain numpy arrays.
- Web / Mobile applications.
- Knowledge Management. At Onna we work with some companies that have really big data, in emails, Jira, other tools. We can plugin this information from different sources and index it fast, because of AsyncIO.
How is Guillotina different from other frameworks?
- First: AsyncIO has won. Django is also moving towards this.
- Django: great OOTB feature set, lots of tooling, add-ons, support. Not a great REST API framework, and still comes from the days of server side templates.
- Pyramid: no batteries included.
- Plone: full feature CMS. Less good for large amounts of structured data. ZODB cannot really use AsyncIO.
Deployments:
- Guillotina is good for large amounts of data, but also for small amounts.
- The memory footprint is really low, again due to AsyncIO. If you use ElasticSearch for indexing you need much more memory.
- PostgreSQL has lots of tooling around it.
- In most cases we have three Guillotina processes, accessing 1 Postgres database.
Front end:
- Volto has 80 percent compliance with guillotina_cms.
- Grange (French for a barn, where you put all your farm tools).
Grange assembles your application fast. A lot of ready to use components.
Powerful principles: - traversing - state management - view overriding
Featureful:
- Pastanaga-based UI library
- Dynamic form generation, using the schemas that you get via json from Guillotina
- Standard views and components, like breadcrumbs and navigation
- i18n
- npm install @guillotinaweg/grange. This covers the standard CRUD scenario with any schema, and you only need to declare your custom views.
Roadmap:
- Guillotina 5 will be LTS.
- Guillotina 6: ASGI, dbusers in core, tooling, more behavior APIs. May have longer alpha phase.
- Some ideas for a Guillotina Object Server in Rust.
- Protobuffer instead of pickles, as pickles tie us to Python.
Fred van Dijk: Generating a Word document from collaboratively edited structured content in your Plone Website
Talk by Fred van Dijk at the Plone Conference 2019 in Ferrara.
At Zest in The Netherlands we have several clients in Flanders, a part of Belgium. One is a government organisation for the environment. One of their sites is about water management. Information about water levels in separate areas, floodings, pollution, etcetera. The country has several partially overlapping governments, and water is flowing through all of them. So it is a bit complex.
For a new part of the site, they said: wouldn't it be nice to let Plone help in getting the next big report with plans for the next six years for the government together? So the report should be viewable as web pages. But wouldn't it be nice to also let this generate the official Word document?
A business analysis followed, with requirements and scope:
- collectively edit it
- seperate between core and background information,
- export only the core going to a main PDF document
Ah, PDF export. Don't we love it? Always tricky, always corner cases, always too many pages, locking up your Zope server or crashing the export. For another site also dynamic graphics, table of contents. So: alarm bells went off. And do we really need PDF, can't it be something else?
Begin with the end in mind. Do I even want this project? Do we have the required skill sets in the team?
So what did we do? We made an MVP, a Minimum Viable Product. A prototype. Can we do all the parts of the process and have a basic document? Let the client test early to see we are on the same page.
Our internal customer was used to writing lots of plans, but a business plan was something else. Lots of things in there that would be nice. But which of those are really absolutely necessary?
Actually, the customer already made a mockup in a website. We had a meeting, things changed, new plans.
In the end, the Word document did not seem that much of a problem. But our initial main structure with two content types was too rigid. We simplified to one content type, and it was fine. Some different views for the content type to choose from.
We use:
- plone.api to find content.
- beautifulsoup4 to analyze the html.
- python-docx for generating a Word document.
Interesting extra problems were internal links, and later footnote support. Not really supported in python-docx yet, but with some lower level code it worked.
Risk: some html things may be too difficult to put in Word reliably. But they can edit the document later.
Something extra: we use pas.plugins.ldap and it is working smoothly now. We have some code to migrate from plone.app.ldap to this, and want to publish that.
Remark from Andreas: with a similar project we discovered that it was difficult for editors to make a section from say level 1.2 to 3.2 in the default Plone UI. So we made a special view for that. And watch out: tricky to lock everything down in TinyMCE.
Question: did you try collective.documentgenerator?
Answer: no. The things we found, were too big and generic.