Weblog
Fabiano Weimar: Plone on Kubernetes
Talk by Fabiano Weimar (Xiru) at the Plone conference 2024 in Brasilia.
Link to talk information on Plone conference website.
Kubernetes (k8s) is an open source system for managing containerised applications It provides mechanisms for deployment and scaling. It is part of the Cloud Native Computing Foundation.
Kubernetes' benefits are scalability, fault tolerant, self-healing, efficient resource utilisation, declarative configuration, rich ecosystem.
Kubernetes basics:
- pods: the smallest deployable unit of computing, consisting of one or more containers. Containers in one pod can talk with each other like they are all on localhost.
- deployments: manage the replication and updates of pods
- RepicaSets: manage a fixed number of identical pods.
- StatefulSets: manage stateful apps, lik databases
- Jobs: one-off tasks
- CronJob: regular tasks
- ConfigMap: store configuration data
- Secrets: store sensitive information
- PersistentVolumeClaims: request storage
- PersistentVolumes: actual storage
- Pod Disruption Budgets (pdb): limit the number of pods that can be unavailable at any given time, for example during upgrades, self-healing during disaster recovery.
- Autoscaler: automatically scale the number of nodes in a cluster based on workload. Resource optimisation.
- Ingress: routing external traffic to services, SSL termination, load balancing, caching, rate limiting.
- Operator: software extensions to Kubernetes that make use of custom resources to manage applications and their components. Think of this as a human operator who would go in and do some stuff.
Plone components as Kubernetes resources:
- Plone backend: StatefulSet
- Plone frontend: Depoloyment
- PostgreSQL: StatefulSet
- Varnish: Deployment
- nginx
- cronjobs
Kubernetes for Devops. It has a steep learning curve. For your laptop you have microk8s and k3d so you have the Kubernetes api, although without the options that you would have with multiple machines. Basics: `apt install docker.io` and `snap install kubectl --classic` and `snap install heml --classic`. And read https://k3d.io/
At the Alpine City Sprint this year we worked on helm charts for Plone: https://github.com/plone/helm-charts/
Tips and tricks:
- You can use ArgoCD to automatically do a deploy when something changes in a git repo. You need to configure this correctly of course.
- Plone can do better on integration with proper observability tools like Grafana and Prometheus. For example, how can such tools get the logs when they are in multiple places.
- Use ZODB with PostgreSQL and relstorage. I think it is the most stable implementation today.
- Varnish Operator from IBM looks promising when you want to run multiple instances of varnish.
- If you run the `test.sh` from Plone help charts this may use a lot of bandwidth downloading docker images. k3d supports import of docker images so they are in a cache.
Rikupekka Oksanen: Building a secure video portal for researchers
Talk by Rikupekka Oksanen at the Plone conference 2024 in Brasilia.
Link to talk information on Plone conference website.
We needed a video platform for our university. Difficult, but we had a dev team of 8 with some architecture experience, so let's try. We did a successful POC in 2023.
We use: Plone 6, VideoJS player, Wowza streaming engine, Keycloak, WhisperX, Camunda 7, Nomad. Most important: open source and license free.
Security. We have it in a dedicated and separate server environment. All data is stored on own servers of the university. All data and files are encrypted. Plone is very secure. All transactions are logged. There is a process for data protection, and it has been reviewed and documented.
There was no definite deadline, which is both good and bad. We could focus on quality, but also needed to focus on other projects.
Now a nice working demo.
[Sorry, I was distracted by client work.]
Melissa Weber Mendonça: Writing and maintaining friendly (and interesting) docs
Talk by Melissa Weber Mendonça at the Plone conference 2024 in Brasilia.
Link to talk information on Plone conference website.
I do open source for a living now. No Plone, but scipy
and other scientific libraries. I was at the Python/Plone Brasil conference in 2013. At that percent 10 percent of women at the conference. First Python conference with a code of conduct. I am very happy with that. This year at Python Brasil we had 30 percent women, so very different.
I was using scipy, numpy, and I like teaching, so the first thing I had to do at my job was documentation.
Who likes to write docs? Who writes the docs in your company/project? How often do you read the docs? How valuable is documentation for you?
I want to talk about information, tools, open source, and empathy and anti patterns.
Part 1: information
Some people like to say we are not living in the information age anymore, but in the content age.
Documentation can be narrative: tutorials, how-to guides, books. Also doc strings, code comments, code review comments. Video, talks, slides. Communications, social media. Internap processes, handbooks.
How do we use documentation? We can have various goals. Document technical decisions, explain code logic, a concept, a workflow, a process, structure. Help onboard new people. "You can set the tone for the project. Reach other people, communicate news, events, announcements. What binds it together is that we try to reach other people. This is a very human interaction.
Can you find what you are looking for? Curating information is important, the writer has a responsibility to signpost what is important. Search can be very useful. Keep in mind that people like to skim documentation pages, so help them quickly jump somewhere.
There can be a sea of information. Help people navigate this. We have more information than we can process as humans. Danger is that people find old information on the internet and then file bugs because it does not work. Having official documentation helps.
For organising your content, have a look at https://diataxis.fr. Four types of docs: tutorials, how-to, explanation, reference. This is not definitive, explore and choose one taxonomy that works for you.
Reader profiles: organise your target audience. Are you writing for beginner, experienced, sighted, student, non-dominant culture/language, a child, artist, scientist? Think about who your audience will most likely be. Take care of accessibility. Identify who your reader is and organize around this. When your docs talk about a person let it not always be a white male, be inclusive.
Part 2: tools define the docs that we write.
If you use Restructured text, you probably try to avoid adding links, because it is a difficult syntax.
Often nowadays projects have "docs as code". So if you use GitHub for your code, you will use issues and PRs for your documentation as well. Often this comes with docs-driven development: if it is not documented, it is broken or does not exist. Technical writers (if they are relevant in your context) may not be used to GitHub and have their own tools. Be careful with credit, authorship, and the limitations of your tools: people who contribute to docs deserve just as much credit as someone who contributes to the code.
Media: where are your docs? On a blog, web, book, in docstrings? Be mindful of how you present your information: audience, goals, voice.
Write/build the docs. As markup language you should look at MyST Markdown (the Plone documentation uses this too). Documentation engines; Sphinx, MkDocs, AsciiDoc, Quarto. More traditional approaches Confluence, Notion, plain text, docusaurus, GitBook/MdBook.
Share it. Is it help for a command line tool? A PDF (probably less links needed, as people may print it)? Web, a book, video, etc. Does your audience know where to find you?
Update the documentation. Your docs may be perfect, but then the code changes and the docs don't match anymore. This is especially hard for images and videos. If you can generate those automatically, that would be a very good idea. You may just need to do frequent reviews. And you may need to version your docs, that old and new versions can be found.
Invite. Interactive documentation draws people in. JupyterLite, PyScript, MyST JS. Translations help for people who are not so good in English. Create docs on how to contribute to the docs.
Part 3: open source documentation
What makes this different from other kinds of docs? So great, there is no boss, so no one can tell us what to do. Oh, but there is no one to tell us what to do, so I don't do know what to do.
You want to lead by example, write docs when you are a project lead. Not RTFM, but WTFM: write the fine manual. Make sure contributors are recognised, valued. You don't need to be an expert to start writing docs, preferably not even!
Onboarding: welcome new contributors. Docs are usually the first point of contact between users and your work. Be clear about prerequisites, unwritten rules push people away. In your project it may be perfectly fine to tell potential new documentation contributors that they need a decent knowledge of C code.
The practice: When there are meetings, committees, other in-person decisions, record this somewhere, make it available for the people who were not there.
The elephant in the room: should we use AI for docs, just let ChatGPT write them?
- No: LLMs are frequently wrong, poisoned or non-sensical, telling you to call functions that have never existed.
- Yes: LLMs can be useful to summarise information for a quick starting point. They can help you index information and identify patterns in your corpus.
Dont't rely entirely on AI tools. If you use them, be transparent about it. Agree on a policy, for example: don't accept PRs that were generated by AI.
Avoid ephemeral media like Discord. Don't say "this is easy", that just creates unnecessary anxiety. Don't use foo, bar, a, b: you can use long variable names, it is allowed.
Celebrate human intelligence. Writing documentation is an exercise in empathy. You are not your audience. Docs are living things, they will never be finished, it is okay to deliver something now, and update later.
"You can't teach people everything they need to know. The best you can do is position them where they can find what they need to know when they need to know it." Seymour Papert
Lightning talks Thursday
Lightning talks on Thursday at the Plone conference Brasilia.
Martin Peeters: Plone Foundation membership
You can apply to become a member of the Plone Foundation membership. Use this form. Ask for help, especially if you are not so good in English. Don't be shy, we all suffer from imposter syndrome. If you are in this room, you probably should be a foundation member.
Paul: Sprints
In the weekend there are sprints. You don't need to be a programmer, there are lots of other things you can do. So join us this weekend.
Astrid: Democracy
There were lots of elections in democracies this year. Plone is also a democracy. Join a Plone team.
Subscribe to the Plone newsletter.
Alexander Loechel: Alpine City Sprint
The Alpine City sprint will happen in Innsbruck Austria, February 10-14.
See https://alpinecity.tirol/. Join us, and go skiing if you want.
Dante: Axolote sprint
Join us for this sprint in Mexico, June 23-27.
Erico: "Axolote was the best sprint this year by far."
Antoine: collective AI
This is a collection of packages that brings artificial intelligence in Plone. You can use it for generating summaries of articles. At IMIO plan a release in January. Contact us if you are interested.
Fred van Dijk: Site go live tips, tricks, checks, caveats
https://vmm.vlaanderen.be/ went live this morning. Went fine mostly, but of course there were issues.
The domain had to change from vmm.be to vmm.vlaanderen.be. There were absolute urls to this or a pre-production domain in some of the texts, so those need to be fixed. I did a search and replace to the blocks with a script.
Check settings in Analytics consoles, update the allowed domains.
We have iframe blocks. But where: we use collective.tileindex, meant for Classic UI tiles but reused for Volto blocks, so we know where they are.
Traefik: you can install a file provider.
Renan: electro magnetics and screws
I have a bachelor's degree in physics. I learned something about electro magnetism. An electric current going upwards, generates a magnetic field counter clockwise. Now you know which way to turn a screw driver.
Mohammed: GSoC student
This is my first Plone conference as a GSoC student. It is a global online program connecting students with open source programs. We get a mentor. It teaches us how to communicate and engage. You gave us a warm welcome at the conference, thank you. What I learned here, I did not learn in college. I'll be back. I will give a talk tomorrow about Hydra. I won't bore you, I promise.
Kim: Plone funding
You can sponsor Plone, donate to Plone.
I have been helping with sponsorships. It is 100 percent of our income. Here are our sponsors: https://plone.org/foundation/sponsorship/sponsors
You can also donate individually via GitHub: https://github.com/sponsors/plone
It is not only about us getting money. It is also about how it looks on GitHub: it helps if people there see that people are sponsoring us.
The money we have as foundation is stable. But what are we not doing? We could advertise more. Sponsor local sprints. Have some paid positions. What is the effect of not doing these things? We have a marketing team creating material, but it is not reaching a lot of people. Let's find creative way to raise funds. I will buy you a cerveza.
David Glick: Cookieplone
Talk by David Glick at the Plone conference 2024 in Brasilia.
Link to talk information on Plone conference website.
Cookieplone is a project generator for Plone. It has several templates to choose from: backend add-on, frontend add-on, project with both.
It is recommended for Volto 18 with Plone 6.0 or 6.1.
See the Plone documentation.
A brief history of Plone installers. What makes a good installer? Minimal: you don't want to need to install a bunch of things before you can use the installer. Reliable: works each time. Portable: works on your laptop and the server, Linux and Mac (and Windows hopefully). Easy to use, batteries included, flexible. Both easy to use and flexible can be hard to get right.
Why can't you just run `pip install Plone`? You can, then you have the Python packages. But then you need scripts, configuration, add-ons, databases, maintenance jobs, a front-end, etc.
Long time ago: download a tarball and unpack it. No dependency management.
Buildout came along, this handled dependency management, could create configuration and scripts using recipes. It has served us well. It has a bit of a learning curve. The reliability and portability could be a problem, especially when the server would be different from your laptop, or you did not pin all versions, or a package could no longer be downloaded.
Then we had the Unified Installer and Windows Installer. It was a wrapper around buildout, including all the needed packages and some scripts to make sure you had the right prerequisites.
plonecli / bobtemplates were more in the area of project scaffolds.
Somewhere in the past decade it become possible to install Plone with pip.
You can use Docker to install Plone. Less useful for development.
cookiecutter-plone-starter was the way to start with Volto and Plone 6. It would give you a Plone backend package and run a separate command to generate a Volto frontend.
And this year cookieplone. It was born in May this year during the Axolote sprint in Mexico City.
What is new compared to cookiecutter-plone-starter?
- It is a mono repo for frontend and backend.
- There are now minimal differences between an add-on and a project.
- We now use pnpm-workspaces for the frontend, which helps if you want to try an unreleased version of a package like Volto.
How does it work? It is a wrapper around Cookiecutter, running several templates, and then some actions at the end: modifying some files or deleting files that you don't want. It has custom filters.
What if I prefer Buildout, or don't want Volto, prefer unittest over pytest? Partial answer: cookieplone is opinionated, so no. But we can talk about it. You can contribute. You can also fork it if your company has different opinions.
It is expected that the templates will evolve over time. We will fix bugs or do things differently. How can I update an existing project? You could generate a new one and then copy in the custom parts that you already have.
What's in progress:
- GitLab pipeline
- Support for bob templates has recently landed, with some commands to extend an already generated project.
- Add a documentation section in the generated project.
- Option to create a Plone distribution.
- Use
hatch
for the backend. - Run via web-based UI or GitHub code spaces.