Welcome from Fred van Dijk and Philip Bauer. This is episode 11 of the Plone Newsroom podcast.

Plone 6 beta 3 was released last week. Your password needs to be longer. Tables like nicer now. There were four Volto alpha releases meanwhile. We need more documentation and marketing.

Community member news: Ramon got married. Peter Holzer has left Plone, working for a new employer. We appreciate everything he has done, and what others have done who have left. People come and go, and some come back: David Glick.

What were some highlights of the past season? Having Victor on the show. The composite pages episode. The growing up of Plone Classic and Volto side by side.

How about the presentations this week so far? Fred's talk obviously, about how some managers and organisations handle Plone. Volto talks: we can see it is working. Plone beyond 2022, with technical planning on the frontend. Keynote about minerals. Diversity talk: we can fix stuff, improve stuff.

Trainings: Happy that we have the Effective Volto training now, to have the brains of Tiberiu and Victor not only in their heads, but also on a site for us to read. Classic UI Theming: good to see how fast it is to start a new theme with plonecli. Patternslib is great, and it was there before React.

Conference advice:

• The talks here are live recorded and almost live uploaded to YouTube, which is great. Skip a beer tonight and watch an extra video.
• When talking with people, be an open circle, so others can join.

For next year's conference organisers, here is some advice from Martijn: 

• Do it.
• Push for early bird tickets earlier. You want to know earlier how many people are coming.

The software is an artefact of the Plone community. Johannes, you organised a sprint? Yes, the Buschenschanksprint in Austria. Buschenschanks are in the Südsteiermark area of Austria close to Slovenia. Lots of vineyards there. Organising is not that hard. Have a place for people to work, maybe a place to stay at night as well, organise some food and drinks and internet. Then announce it. The Plone Foundation has funding for this. The sprints are important, are social, and so much work is done, it is awesome!

Rikupekka, please come on stage. He has been working on marketing and especially the new plone.org. We are done for eight months already, so it will be done soon. There are designs, there is code, there is migration, there is progress. This weekend you can sprint on the site. First we need a running site with Volto with the design that the Italians made. Write good content for plone.org, come to me with that.

When you support a website, be it Plone or Wordpress, and a feature request comes in, you can duck. Deny the request. Or accept them all and it will grow into an unmaintainable monster. With both these approaches, every other system starts looking appealing. I call this product fatigue. We see talks here with universities with thousands of Plone sites, and other universities step away from Plone, why?

Plugins:

• pas.plugins.oidc for authenticating with openid connect
• pas.plugins.authomatic got a Volto port

More about Volto. We both agree that Volto is by rights the default frontend of Plone 6. Will this be the only frontend in five to ten years? A few hands go up in the audience. The difference is: do you have an html canvas with some javascript, or a javascript canvas with some html? Requirements that I see coming in, are better met by Volto. Based on the reaction, you would say that no one here needs to fear the the Classic UI will die out in the next five to ten years. The path to the more dynamic frontend is there, but we are not pushing you.

See you next time!

pip is the tool almost every Pythonista learns early to use. Plone 6 installs just with "pip install -c ... Plone".

But it needs more: Zope configuration, including existing or development of own add-ons, using newer package-versions, configuring RelStorage, CI-CD integration, building Docker images, deployment on Swarm or K8s, an ingress, some load balancing, caching, ...

The talk is not a tutorial but gives a 3000 foot view and acts as a starter to dig deeper.

In the past we only had Plone as backend. Now also the frontend, running on a different port, different process. The backend talks to the database.

The Plone frontend is a Node project. It pre-renders pages on first request, and for this it talks to the backend.

Plone backend is a WSGI server based on Zope, with a complete CMS Rest API. The backend still has all the Classic UI features in it, even when this is not used with the new default front end.

You would use pip to install the backend: pip install Plone. But it is not perfect, so I created mxdev to override constraints. Previously Buildout was used to generate a directory structure and configuration files, but currently you use cookiecutter-zope-instance for this. And then there is a WSGI server to start Zope, by default this is waitress.

A web request comes in at the web server (nginx, traefik), and then most requests go to frontend, api requests to backend, and the frontend talks to the backend for the first request. You can scale the backend horizontally by adding more backends (zeoclients). The backends talk to the ZEO database, with usually a shared filesystem for the blobs (binary large files). You could scale the frontend as well, if that turns out to be the bottle neck.

If you use multiple backends or frontends, you will need to put a load balancer in between.

If your sysadmins don't like this ZEO setup, you can use a relational database. This uses the RelStorage package. Postgres would be the best choice, but MySQL should work as well. The blobs are stored as large objects in the database, which is more performant for a transaction.

At some point, you need some kind of caching. You can go to a cloud provider for a cache. But you can do it yourself: use a varnish cache. You put this between the web server and backend, and/or between web server and frontend. You need to configure it. Under very high load you could even say: cache this request for three seconds.

Now on to hosting. Let's focus on Docker.

Disclaimer: Docker swarm could die in the long term. But it is nice to start with, and it gives you knowledge that is also useful for Kubernetes.

You get images from docker hub, or other container registries. Usually you will build your own frontend, so you need to create a custom image for this. For the backend you can also do this. You could create an image on your laptop and upload it, but it is better to build and store them in CI/CD, to avoid big uploads on slow connections.

Now it is time to deploy it. Basics: There is docker compose, but you should use Docker Swarm with 1 to n nodes, or same with Kubernetes. You need storage for your database. This could be a managed database by your provider. And you need fixed IPs.

Simple deployment for one site would be a single node docker swarm, 1 docker stack with all included, all in one yaml file. 1 traffic, 2 frontends, 2 backends, 1 postgres. The 2 frontends and backends are so that you can upgrade 1, bring it up, the upgrade the second and bring it up, so no downtime.

You can add varnish in here. Traefik sends all requests there. Varnish returns a cached response, or it adds a header. With the header, Traefik sends the requests to frontend or backend.

Now some tools. With Traefik UI you can inspect what is configured or wrong. Traefik has Let's Encrypt built in. Portainer is a UI and management interface for Docker Swarm and Kubernetes, and you can add this with an image. You inspect the state of the cluster, stop and start services, view logs, open a web console.

You really need a CI/CD system, otherwise this get nasty with images. You need workflows and a container registry. GitLab and GitHub have both.

Want to play with this? See the deployment training.

"Untested code is broken code" according to Martin Aspen and Philipp von Weitershausen. But tested code is broken code too: even if all tests pass, it is not a guarantee that everything works, even with 100 percent test coverage.

So why do we write tests?

• Avoid regressions
• Improve your confidence when you change the system, for example from Python 2 to Python 3.
• Really: to minimise the chance of being ashamed.

History

When did the first tests get into Plone? I got to October 7 2002. A kind of continuous integration was done first in 2004. Golden age of Plone testing was between say 2009 and 2013, when PloneTestCase was there, plone.app.testing was created as new version, there was a Testing Team, Jenkins was added in 2010. Travis CI was used since 2012, a lot of add-ons started using that. We got robotframework around 2012, for end user testing in the browser, using Selenium. In 2013 mr.roboto was born so we could make sure that when one package changed, the full test suite of all Plone was run.

In 2018 GitHub Actions started. In 2019 Cypress could be used as alternative for robotframework, Volto is using this. Around now, some tools around code analysis have gotten updated.

Our current process

• You make a pull request.
• GitHub Actions checks are run, mostly code analysis, in some cases building docs.
• With a comment on GitHub you can start Jenkins jobs.
• When everything is green, the PR can be merged

Future

These are some ideas, I will be asking questions.

• Are we using the right tools? Most of the Python community uses pytest, we use zope.testrunner. You can actually use gocept.pytestlayer now so your Plone testing layers are treated as pytest layers. You need to move the Python code into src. And don't mix doctests and unit tests in the same module.
• Browser testing: Selenium? Cypress? Playwright?
• Move from Jenkins to GitHub Actions? Thank you Gil Forcada for keeping Jenkins up and running so far!
• Can the tests go faster? Any easy wins? With parallelisation on my M1 I ran the tests in four minutes.
• Go to a monorepo? For things that are used outside of Plone: no. But it may make a lot of things easier. Recently I changed the minimum password length and needed changes in eight packages or so; with a mono repo this would have been much easier.
• Do Classic UI and Volto test the same features? Could the plone.restapi tests run on Guillotina or Nick?
• Which version are we testing? We are not testing the backend of Plone against Volto until it is too late.
• Do we have the right tests? 350 acceptance tests (robot plus cypress), 800 integration tests (Python), 8000 unit tests (Python plus Jest)
• What else could we be testing? Accessibility testing. Visual regression testing. Performance tests. Type checking. In VS Code: go to Python, Analysis, Type Checking Mode, and set it to basic.
• Error tracking
• Distributed tracing for seeing how long things take to execute.

Audience:

• The Classic UI tests could be split from the REST api tests.
• For pytest, you could move the tests out of the package code to the root.
• Could we make pytest the default for add-ons?
• Mono repo does not solve everything. We need to think about architecture. But fewer repos is good.

Plone 6.0 will be matched with the Volto version 16, which is an LTS version. Volto 16 is in alpha stage, a few more features are pending, hopefully merged during the conference sprint.

Volto 17, 18, 19 will be worked on, first as "canary" (testing) and then stable, and released faster. Volto 16 will still be supported then. Bug fixes and features will be back ported to Volto 16, but no breaking changes.

At some point Plone 6.1 will be released. We will have to see which Volto version goes with that, for example 18. It will be clear at the time.

Core features in Volto 16:

• Configuration registry
• Shadowing and the component registry
• Add-on architecture
• Seamless mode and internal proxy, with variables that can be used when building or when running.
• Routing
• Data fetching
• Server side rendering
• Theming
• Pastanaga editor

We should continuously curate these features. They should remain high quality, also when the React ecosystem constantly evolves.

What's ahead?

• React 18 is out. New Suspense SSR Architecture.
• Routing: currently we are stuck with RR5, React Routing 5. We have AsyncConnect and connected-react-router. We cannot upgrade to RR6, so not to React 18. Routing libraries have improved a lot lately, including data loaders when the route changes. Some modern frameworks for this: Remix and NextJS.
• Server side rendering: we need to modernise and simplify this. AsyncConnect is obsolete. New Suspense SSR is amazing.
• Global state handling: paradigm of global state handling and Redux has changed.
• Data fetching: this is again influenced by the Suspense SSR in React 18. Need to adopt to @tanstack/react-query.
• Reusable components: we can extract basic Volto components to their own add-ons, unconnected. Make it easier to use other JS frameworks with Plone 6 as backend.
• Build modernisation: go to yarn 3, pnpm. Use next generation builders/transpilers, like Vite, ESBuild, SWC? Integrate it in Razzle or in another boiler plate.

What features are coming next?

• New theming architecture. Would be nice to support Bootstrap. We would need to separate the theme and CMSUI, and use a vanilla, agnostic CMSUI based on neutral frameworks.
• Quanta: the new design system by Albert Casado. We need to make that happen, probably in Plone 7, see when this is ready. Use the Quanta toolbar. Implement the design system and update the story book.
• Slots: a mix between viewlets and portlets.Inheritable, lightweight, containing blocks. Assigned statically and/or defined in the backend.

More features, working on this during the sprint would be good, especially the first three:

• Row / grid and columns / group block
• Nested drag & drop blocks support
• Teaser block
• Relations control panel
• Improve transform run detection based on agnostic markers, JSON-LD like
• Subscribe-able context for every block (widgets)
• JSON-schema form generatio, adopt a specialised library.
• Rework the object browser, improve its api and features

Work on alternative backends: Guillotina, Nick.js.

Mining/extraction of mineral resources (for ICT and other usages) logically involves geological, technological and engineering concerns. However, it also deals with numerous other essential aspects, although less discussed by medias/citizen/experts, such as economy (“circular economy”), sociology (“social acceptability” or “perception of mining by the citizen”), ethics (“artisanal mining”), geopolitics (“strategic/critical commodities”, “national strategies”), environment (“waste”, “post-mining”), teaching (including popularization), land management (zones dedicated to extraction), philosophy (“needs” of ICT for Humans), history (current impacts of former supplies), law (how to legislate/regulate?)…

With his experience in various countries/projects, Johan Yans will expose some of these aspects, with a particular focus on the supply of geological resources for ICT needs. Olivier Vergeynst will then introduce the basics of Sustainable IT, highlighting some best practices to start reducing your impact as ICT professional or as simple user of digital devices and services.

Copper and gold prices were already high before the war in Ukraine. True for lots of minerals. Not for gas, that was influenced a lot by the war.

Each year we extract 92 billion tons of minerals each year. That is one swimming pool every two seconds. 60 percent of cobalt in the world is extracted from the DRC, Congo. A lot is done artisan ally, by hand.

The list of critical raw materials is rising, so materials that are important, in our view from Europe, but that have supply risks. Europe needs to import a lot.

International Resource Panel says we need to extract less minerals if we want to keep below 2 degrees warmup in 2060. But the expectation is that we will extract two times as much.

We can extract minerals in Belgium, there is a lot of zinc and lead. Of course we say: not in my back yard! There are a lot of downsides.

The ICT industry accounts for more green house gases than civil aviation. And between 60 and 90 percent of our equipment ends up in land fills, instead of being recycled.

We have an Institute for Sustainable IT. Impact is mostly for user equipment, that takes the most energy and minerals. Manufacturing an iPhone 11 taks 72 kg of carbon emissions, which is about 80 percent of the emissions for its total life cycle.

For data usage, the main impact comes from videos. Online video is 60 percent, 20% other videos, like meetings, 20 % other content. If you can choose a lower transmission rate on Netflix, that helps.

Sustainable IT is good for business. Get ready ahead of regulations, and you will have a chance to catch up with your competitors and go beyond them. Risk management: unsustainability is a vulnerability.

Choose for eco-design of digital services. First question: what can I avoid developing? Write efficient code. Reduce bandwidth. Divide infra needs. Extend the lifetime of your product: if your product can still run on older phones, you avoid the need for your users to buy a new phone. More clients. Business growth.

Demonstrate your engagement. You can become a member. Or just sign.

A few tips:

• Buy less equipment
• Reduce video streaming
• Limit use of emails, avoid sending it, avoid sending large attachments.
• Use https://www.printwhatyoulike.com/