Luistxo: The eibartarrak, gun toting ruffians

Eibartarrak are the people from Eibar. You probably expected to find some Spanish seaside resorts here, as in the propaganda on the conference site. Still, there is #PrettyEibar. Eibar in 1937: Francist soldiers destroyed the whole town, except for the church. So we had to build it all back. It is in progress. But it's the people that matter. Eibar in 1346 a Castilian king told us to build a town. What is the X that appears all over town? It is Saint Andrew's cross, which also comes back in the flag of Scotland. It is a poor geographical area, nothing grows on the slopes, we live in a hole. From the 15th century on, Eibar was making guns. For the government, not personal use. That was my dad's first job, making guns. If you can make guns, you can make bikes, they are tubes of metal after all. Also creating cork screws, staplers, other household materials. Helmholtz electronics. G93: realtime cycling tracking, comes from Eibar.

And CodeSyntax does Plone, fully i18n-ized thanks to Erral. It's the people that matter.

Reminder: at tonight's party, I am the DJ.

Julien Chandelle (jimbiscuit): shortcut pattern bookmark

Extension for Plone. Add shortcuts / bookmarks. Available on Chrome browser.

https://chrome.google.com/webstore/detail/shortcut-patern-bookmark/fneleejokehnmfkbnjfeahieblhaecho?hl=en-US

https://shortcut-pattern-bookmark.affinitic.be/

Philip Bauer: Plone Tagung 2024

March 4-6 2024, followed by community sprint. In Giessen. Most talks are in German, but happy to have English talks. See https://plonetagung.de/

Eric Steele: Unsung Hero

A person stepped up to fill a vital role for Plone. Largely doing so on their own. Who? There are several of you here. Single points of failure for Plone. We lean heavily on you. It is a Plone community problem. It happens everywhere. Cloning would be great, but two Freds would just debate each other to death. We had the 10% Plone Manifesto, where 10% of the time of Plone companies would be for Plone.

We again have Plone TuneUps. One day a month. You get an education and a warm glow. We need not just coders, also project managers, documentation writers, people who do not know Plone. Send me an email or register, link on plone.org.

David Glick: Ideas I don't have time to build

1. Easier input of smart quotes and EM dashes in Volto.
2. TUS uploads (progressive uploads)
3. Blocks import/export as markdown
4. Editor discussions: discuss revisions to documents.
5. ChatGPT: generative Plone text
6. Editor dashboard: show content that you edited recently, content that has not changed in a while.
7. Don't restrict ids. You can't name something 'image', or give it the name that is used as catalog index.
8. collective.exportimport: separate file for each item
9. docker image with unreleased changes from coredev (Erico: there is a nightly image)
10. consistent command line, whether you use docker or something else

Elisabeth and Martin: iMio and OSOR

iMio is finalist of the OSOR awards: Open Source Observatory of the European Commission. iMio serves 404 local initiatives. Please vote for us.

Erico and Kim: Board election

There is an election for the Plone Foundation board. You still have one hour to nominate yourself.

Foundation members will get an email on how they can vote.

See https://plone.org/foundation/meetings/membership/2023-membership-meeting/call-for-nominations-plone-foundation-board-of-directors-2023-2025

If you don't get voted in, please try again next year. It is not a popularity contest.

Kim: Plone Foundation sponsorship

We have a bunch of sponsors, listed on our sites. I did a poor job of updating the list this year. You can also sponsor individually via GitHub. We pay for all kinds of stuff with this money, like some servers.

We also have a Plone service provider list. We want to open up this list for everyone, without needing to pay anything. Contact us, email me.

Fred: Website Team

We have a Website Team in the Foundation? No, we don't. We have teams involved: AI team, Marketing team, PloneConf team, Release team, Volto team, Release managers, etc. But who actually maintains plone.org: Nobody does. Members from the other teams are standing in. plone.org is not a one-off project: worked on it, shipped it, done. No, it is ongoing.

We need a small pool of available developers to work on this, maybe 4-12 hours a month. I like to give training, but we need some experience on a Website Team first. We need a few hands to maintain this site.

Full CI/CD is already there, so that helps. Talk to me, Rikupekka, Erico.

I have created https://github.com/plone/plone.distribution which will end up in Plone 6.1.

With this you can create your own Plone CMS, or:how integrators use 20 years of collective work as the secret weapon to control a market / segment.

I talked about Plone distributions before. Maurits blogged about my talk. I talked about a distribution of Plone specifically meant for Brasil. And after a while I saw that some organisations were actually using it.

The idea is not new. You have Drupal distributions, Joomla templates, Typo3 distributions. For example CiviCRM is a distribution of Plone.

Plone was never very friendly for distributions. Newer Plone versions would come out that made it hard to upgrade the distribution.

The logic to create a new Plone site is currently distributed:

• browser view in CMFPlone
• content in plone.volto, plone.app.contenttypes
• translations in plone.app.locales

So there is no clear contract. How do you create a new site from Python code? No idea, I would copy some codes There is no RESTAPI service. Without such a service, how would you provide a SAAS for Plone?

Still, there are many examples of "distributions" in our family, for example: Quaive, SENAITE, Bika, Portal Modelo, Vindula, Volto, CastleCMS, io-Comune, Classic UI. Reality check: maybe we are not really selling Plone CMS. Plone CMS should not be targeted at the end user, but at integrators.

We need to define what a Plone Distribution is. It is a contract to create a new instance of a Plone Site. It follows conventions about how to create initial content. It provides a Python API and a RESTAPI. Use cases: starting point for a ne CMS project (website, intranet, headless backend, ...), basis for SAAS offering.

plone.distribution is the package that is the basis for this, the factory. Distributions should be first-class citizens in the Plone world. Convert the initial content from plone.volto and plone.app.contenttypes. Cleanup the codebase. It will sit above Products.CMFPlone, so we can use plone.api in the code to make things easier.

You can use it now as an add-on in Plone 6.0.  In 6.1 it will be a core package.

We are not yet another framework, Plone is a CMS. It is good to concentrate, be specific. But I can say: "Plone is a platform, not a CMS." We have solid foundations: permissions, workflow, basis for content types.

Marketing for Plone: focus on a market and segment, not on the common denominator. For some Plone companies Drupal is a competitor, for others Typo3. Plone marketing should target integrators.

By default you will see two distributions when you start the Plone instance: default (Volto) and ClassicUI. You can add your own, and optionally hide the default ones.

Example distributions:

• collective.ploneintranet
• We have portalbrasil.edu/leg/gov/jus as base for public websites in Brazil.
• kitconcept.voltolighttheme. Allows developers to showcase new blocks.
• A to-be-named distribution for personal site / blog. It is scratching my own itch, for ericof.com, and as additional target Maurits, who is blogging right now.

A distribution is not a "policy package". You do not need to have a GenericSetup profile. You can have complex forms, like with collective.plone.intranet.

Next steps.  How to create a front-page in 45 languages? We should think about that. Should we add features in a distribution package? We need to improve content export/import. It works, but I have questions.

We ship this with 6.1. With 6.2 I want to ship with 6 or 7 distributions. It is a seed we plant for integrators to think about.

With production I mean: serving public traffic (or an intranet), highly available, with maintenance and monitoring.

I will talk about dlr.de, about 15 million hits per month, in their own data center.

First availability. We have stacks in two zones, so if one zone goes down, the second zone should still run.

Internet traffic comes in at a load balancer / cache. This points to one of the apps. The apps talk with the database (primary or secondary) and to one of the two search machines (Solr).

We use Docker Swarm mode. Docker swarm used to be a separate project, but now it is part of Docker machine. It provides a good way to manage several apps on different machines. It is containerised: we build an image once, and deploy it on multiple machines. So you do not have a buildout that you need to run a multiple machines before you can run the result. It takes care of process management: restart processes when they fail or run out of memory. Or it can startup the service on a new machine.

It allows for rolling deployments: take down one of the apps, update it, start it, wait till it receives traffic, and then start updating the next app.

With Docker Swarm you can have a declarative infrastructure. We write it, it gets checked into git, others can review it.

How do you design for availability? Avoid single points of failure. What I dislike about this idea, is that it makes failure seem like something that rarely happens. Instead you should plan for downtime: for example a machine may need to restart after getting some package updates. Services should be stateless: the data should be somewhere else (except if the service is the actual database server).

Varnish as proxy cache is a bit trickier. Varnish stores responses, so it is not really stateless. We have two copies of Varnish. This means we need to send purge requests to different places. You could add multiple addresses in the caching control panel, but here it would be two times http://varnish. So we send the purge requests to a purger that sits in front of the two Varnishes.

For databases it also gets tricky: you need a primary database and a secondary that is kept up to date.

What if the two zones cannot talk to each other? On top, maybe the first zone, with the primary database, cannot be reached by the internet. Then you need cluster managers in three different zones, so there are always two left who can decide what to do.

Think about what resource limits you need to enforce.

Now maintenance. Tune backend environment variables, like the Relstorage blob cache size, and ZODB cache size. Tune postgres configuration, especially related to used memory. Configure cron jobs: we use the crazymax/swarm-cronjob image to run these jobs, like backup and packing. With Postgres you also need to do vacuuming. This is done automatically, but especially the blobs table may not get vacuumed often, because not much changes.

Lastly monitoring.The most basic type of information you need, is: is the server down for everyone or just me, use for example Uptime Robot or Uptime Kuma. You can use SigNoz for log aggregation. There are metrics that you can check over time, for example metrics on Traefik can tell you how long requests take, how many requests per minute you get, what percentage of requests are failing.

I like "distributed tracing" better than logs, where you can drill down into details of a request, like how many SQL queries are done.

Also use something like Sentry to get reports/alerts about errors.

With collective.opentelemetry, which I am working on, we get more OpenTelemetry data about Plone and Zope.

Panelists are: Sally Kleinfeldt, Tiberiu Ichim, Timo Stollenwerk, Eric Steele, Eric Bréhault, Rikupekka Oksanen and Érico Andrei.

Timo: we use Solr on a regular basis, for most clients. For collective.solr we had a buildout solution which was supposed to make it easier, but it was adding an extra layer of indirection: it is better to rely on the Solr documentation. There should be a good default, and we can have a search control panel, but will need to learn about Solr to really configure it.

Rikupekka: we run small and large sites at the university. For small sites the standard Plone search is fine. For larger sites we use Solr. One problem with 50,000 documents is when hundreds have a title "Research". Would be nice to have a warning message then: "We already have this many documents with the same title, please be more specific."

Eric Steele: Would be good if we market this correctly.

Tiberiu: At EEA we use Elasticsearch. Lately alternative vector based solutions start popping up. Currently we simply fetch the html of the the page, just like Google Search does.

Guido: At Quaive we use Solr, so much better than Catalog. Tuning it to give more weight to some fields should be an easy way to improve the results.

Erico: We could get rid of ZCatalog in all Plone instances. If navigation works in one way, and search in a different way, it is going to be a nightmare to debug. If we have money, we should hire Nuclia.

Eric Brehault: It needs to be opinionated, configured correctly.

Sally: Solr is open source, Elasticsearch is not.

Timo: I don't care about Solr versus Elasticsearch, we can make any decision there. Integration is important: might be that collective.elasticsearch is doing some things smarter than collective.search.

Guido: If you use an external service, you should remove the SearchableText index from the ZCatalog. And you need to make sure the indexers work: can you extract text from PDF, Word, etc.

Erico: We want real faceted navigation/search, like eea.facetednavigation did. Danger is that some add-ons do not work if the search is different. With a well defined search api, this should be no problem.

Erico: Sounds like there are benefits to each solution. People will want to choose. An abstraction layer will make it a lot safer.

Timo: Solr and Elasticsearch differ a lot, especially on how they handle facets. It is difficult to have an abstraction layer for this. And the responses will be different. If you try to transform the results so you get the same answer from Solr and Elasticsearch, then it kills performance.

Erico: It should be the same type of info as you get from the ZCatalog.

Guido: We need a solution that can handle and fix inconsistencies. The ZCatalog takes part in the transaction machinery in Plone, the external solutions typically do not.

 Philip: At one point in Sorrento (Plone Open Garden) we picked Solr and said it should be a first-class citizen in Plone. Just a single sentence in a Google Doc somewhere.

Currently we have quaive.app.onlyoffice and quaive.app.libreoffice for realtime collaboration. As html editor we use TipTap, using the pat-tiptap pattern, with Hocuspocus for synching changes. This can be used for example for collaboratively writing notes for a weekly meeting.

TipTap is a headless/designless editor, so you need to build your own. It fits nicely in our Patternslib ecosystem. It is based on ProseMirror. This means it has a strict content schema: you define exactly what html structures are allowed. The outcome is potentially nicer HTML than what you get from TinyMCE, the default editor in Plone. Our integration is opinionated, but well-reasoned.

We use the Yjs library, which is a bit like conflict resolution in the ZODB. Also offline support, with sync once you are back online.

Hocuspocus is Tiptap's oen collaboration library. Based on web sockets.

We have a first prototype. We don't have a collaboration history: the document is just saved. Currently only the initiating user can save. Permission check is only client-side for now.

Uses plone.app.textfield, with the Yjs document in the raw value.

Code:

• https://github.com/Patternslib/pat-tiptap/pull/6
• https://github.com/Patternslib/tiptap-collaboration-server

We have to see if Plone as storage backend is good for this. There are some BBQ (Boring Bundling Questions).

We want to make this available in standard Quaive. Maybe even for Plone and Volto.

We would like multi-channel editing, so at the same time you can edit text, title, description, data, list of participants, etc.

A collaborative whiteboard would be nice. Collaborative card notes. Collaborative anything!