Weblog
Ziming Yuan: How a Beginner Built an RSS Feed Provider in Volto
Talk by Ziming Yuan at the Plone conference 2024 in Brasilia.
Link to talk information on Plone conference website.
I am studying computer science. This summer I have participated in the Google Summer of Code for Plone. Thank you to mentors Paul, Roeland and Mike!
I have built several full-stack web apps with React, but am new to open source.
What is RSS? It is Really Simple Syndication. It is used by websites to publish updated information, such as blog or news. It lets users decide what they read every day, instead of algorithms.
In Plone the RSS feed should use a catalog query to filter contents, for example a feed of science articles or of events.
For an RSS feed content type we could use a collection behavior or a blocks behavior with a listing block. The listing block is the way to go for Volto. So I did that. Plus extra fields like maximum title length and maximum description length.
I created a view component for the content type. Important is the Express middleware with which we got information from the backend.
We decided on `rss` as Node package for the feed. We looked at another newer one called `feed`, but it did not work for us or was missing features.
There were challenges for me as person new to Plone. Do I really understand the code? Often the answer is: no. For example ZCML? i18n? XML files in the profiles? There is a really nested structure. Create a new project, then you have backend / src / rss / src / rss, at least something very deeply nested. It was always confusing to me as I did not really know what should end up in git or in a release. The worst is that I don't know what are the most important files.
And what if my code fails, it was hard to understand errors on the front end. There are just too many things that may be the source of the error.
Another challenge: what if I want to create something new? The training lets you learn by example. But what does each line of the example code actually do? Are there other options?
Missing documentation: how do prevent the user from removing the title or rss block? We found out about initialBlocks per content type. And prevent adding new blocks? There is a feature disableNewBlocks, that could be set on an individual block, but what about doing this for all instances of this content type?
An overview of the code base would be good. How different components are connected, how different files work together. A guide about how to approach the source code, what files I should look at first.
For developers new to Plone: the source code is your friend, the Plone community is helpful, but be specific in the questions you ask, and tell what you have already tried.
Source code:
Érico Andrei: Plone + pytest: Tips and tricks
Talk by Érico Andrei at the Plone conference 2024 in Brasilia.
Link to talk information on Plone conference website.
Plone loves tests. The first was committed in October 2002 by Sidnei da Silva. We have used lots of different testing framework. I like using pytest
. The unittest
framework was inspired by Java testing and you see it. pytest is simple and pythonic. It does dependency injection, which can look magical, okay. There is a large ecosystem of plugins. It has support for unittest. There is a Brazilian maintainer.
I have created pytest-plone, built upon zope.pytest. It provides fixtures and helpers. it is the default in cookieplone templates. Take a look at plone.distribution and plone.exportimport for examples.
Golden rules
If you need a Plone site in your test, you need to wrap your test in a class. That helps for not tearing down everything after every test. Start with a method that gets portal and probably get_fti, and use @pytest.fixture(autouse=True)
.
Use @parametrize
to run the same test function with multiple inputs and output.
Use fixtures.
Trust and love pytest plugins. There are plugins for running the tests in random order, or in parallel. Code coverage is a plugin. You can start a docker container.
Use helpers in pytest-plone to check that your latest upgrade step matches the metadata profile version. Check that your add-on is installed.
plone-pytest 1.0 is coming. Small problem: the tests are breaking on Python 3.9.
Write more tests with less code. I will sprint on pytest-plone.
Fabiano Weimar: Plone on Kubernetes
Talk by Fabiano Weimar (Xiru) at the Plone conference 2024 in Brasilia.
Link to talk information on Plone conference website.
Kubernetes (k8s) is an open source system for managing containerised applications It provides mechanisms for deployment and scaling. It is part of the Cloud Native Computing Foundation.
Kubernetes' benefits are scalability, fault tolerant, self-healing, efficient resource utilisation, declarative configuration, rich ecosystem.
Kubernetes basics:
- pods: the smallest deployable unit of computing, consisting of one or more containers. Containers in one pod can talk with each other like they are all on localhost.
- deployments: manage the replication and updates of pods
- RepicaSets: manage a fixed number of identical pods.
- StatefulSets: manage stateful apps, lik databases
- Jobs: one-off tasks
- CronJob: regular tasks
- ConfigMap: store configuration data
- Secrets: store sensitive information
- PersistentVolumeClaims: request storage
- PersistentVolumes: actual storage
- Pod Disruption Budgets (pdb): limit the number of pods that can be unavailable at any given time, for example during upgrades, self-healing during disaster recovery.
- Autoscaler: automatically scale the number of nodes in a cluster based on workload. Resource optimisation.
- Ingress: routing external traffic to services, SSL termination, load balancing, caching, rate limiting.
- Operator: software extensions to Kubernetes that make use of custom resources to manage applications and their components. Think of this as a human operator who would go in and do some stuff.
Plone components as Kubernetes resources:
- Plone backend: StatefulSet
- Plone frontend: Depoloyment
- PostgreSQL: StatefulSet
- Varnish: Deployment
- nginx
- cronjobs
Kubernetes for Devops. It has a steep learning curve. For your laptop you have microk8s and k3d so you have the Kubernetes api, although without the options that you would have with multiple machines. Basics: `apt install docker.io` and `snap install kubectl --classic` and `snap install heml --classic`. And read https://k3d.io/
At the Alpine City Sprint this year we worked on helm charts for Plone: https://github.com/plone/helm-charts/
Tips and tricks:
- You can use ArgoCD to automatically do a deploy when something changes in a git repo. You need to configure this correctly of course.
- Plone can do better on integration with proper observability tools like Grafana and Prometheus. For example, how can such tools get the logs when they are in multiple places.
- Use ZODB with PostgreSQL and relstorage. I think it is the most stable implementation today.
- Varnish Operator from IBM looks promising when you want to run multiple instances of varnish.
- If you run the `test.sh` from Plone help charts this may use a lot of bandwidth downloading docker images. k3d supports import of docker images so they are in a cache.
Rikupekka Oksanen: Building a secure video portal for researchers
Talk by Rikupekka Oksanen at the Plone conference 2024 in Brasilia.
Link to talk information on Plone conference website.
We needed a video platform for our university. Difficult, but we had a dev team of 8 with some architecture experience, so let's try. We did a successful POC in 2023.
We use: Plone 6, VideoJS player, Wowza streaming engine, Keycloak, WhisperX, Camunda 7, Nomad. Most important: open source and license free.
Security. We have it in a dedicated and separate server environment. All data is stored on own servers of the university. All data and files are encrypted. Plone is very secure. All transactions are logged. There is a process for data protection, and it has been reviewed and documented.
There was no definite deadline, which is both good and bad. We could focus on quality, but also needed to focus on other projects.
Now a nice working demo.
[Sorry, I was distracted by client work.]
Melissa Weber Mendonça: Writing and maintaining friendly (and interesting) docs
Talk by Melissa Weber Mendonça at the Plone conference 2024 in Brasilia.
Link to talk information on Plone conference website.
I do open source for a living now. No Plone, but scipy
and other scientific libraries. I was at the Python/Plone Brasil conference in 2013. At that percent 10 percent of women at the conference. First Python conference with a code of conduct. I am very happy with that. This year at Python Brasil we had 30 percent women, so very different.
I was using scipy, numpy, and I like teaching, so the first thing I had to do at my job was documentation.
Who likes to write docs? Who writes the docs in your company/project? How often do you read the docs? How valuable is documentation for you?
I want to talk about information, tools, open source, and empathy and anti patterns.
Part 1: information
Some people like to say we are not living in the information age anymore, but in the content age.
Documentation can be narrative: tutorials, how-to guides, books. Also doc strings, code comments, code review comments. Video, talks, slides. Communications, social media. Internap processes, handbooks.
How do we use documentation? We can have various goals. Document technical decisions, explain code logic, a concept, a workflow, a process, structure. Help onboard new people. "You can set the tone for the project. Reach other people, communicate news, events, announcements. What binds it together is that we try to reach other people. This is a very human interaction.
Can you find what you are looking for? Curating information is important, the writer has a responsibility to signpost what is important. Search can be very useful. Keep in mind that people like to skim documentation pages, so help them quickly jump somewhere.
There can be a sea of information. Help people navigate this. We have more information than we can process as humans. Danger is that people find old information on the internet and then file bugs because it does not work. Having official documentation helps.
For organising your content, have a look at https://diataxis.fr. Four types of docs: tutorials, how-to, explanation, reference. This is not definitive, explore and choose one taxonomy that works for you.
Reader profiles: organise your target audience. Are you writing for beginner, experienced, sighted, student, non-dominant culture/language, a child, artist, scientist? Think about who your audience will most likely be. Take care of accessibility. Identify who your reader is and organize around this. When your docs talk about a person let it not always be a white male, be inclusive.
Part 2: tools define the docs that we write.
If you use Restructured text, you probably try to avoid adding links, because it is a difficult syntax.
Often nowadays projects have "docs as code". So if you use GitHub for your code, you will use issues and PRs for your documentation as well. Often this comes with docs-driven development: if it is not documented, it is broken or does not exist. Technical writers (if they are relevant in your context) may not be used to GitHub and have their own tools. Be careful with credit, authorship, and the limitations of your tools: people who contribute to docs deserve just as much credit as someone who contributes to the code.
Media: where are your docs? On a blog, web, book, in docstrings? Be mindful of how you present your information: audience, goals, voice.
Write/build the docs. As markup language you should look at MyST Markdown (the Plone documentation uses this too). Documentation engines; Sphinx, MkDocs, AsciiDoc, Quarto. More traditional approaches Confluence, Notion, plain text, docusaurus, GitBook/MdBook.
Share it. Is it help for a command line tool? A PDF (probably less links needed, as people may print it)? Web, a book, video, etc. Does your audience know where to find you?
Update the documentation. Your docs may be perfect, but then the code changes and the docs don't match anymore. This is especially hard for images and videos. If you can generate those automatically, that would be a very good idea. You may just need to do frequent reviews. And you may need to version your docs, that old and new versions can be found.
Invite. Interactive documentation draws people in. JupyterLite, PyScript, MyST JS. Translations help for people who are not so good in English. Create docs on how to contribute to the docs.
Part 3: open source documentation
What makes this different from other kinds of docs? So great, there is no boss, so no one can tell us what to do. Oh, but there is no one to tell us what to do, so I don't do know what to do.
You want to lead by example, write docs when you are a project lead. Not RTFM, but WTFM: write the fine manual. Make sure contributors are recognised, valued. You don't need to be an expert to start writing docs, preferably not even!
Onboarding: welcome new contributors. Docs are usually the first point of contact between users and your work. Be clear about prerequisites, unwritten rules push people away. In your project it may be perfectly fine to tell potential new documentation contributors that they need a decent knowledge of C code.
The practice: When there are meetings, committees, other in-person decisions, record this somewhere, make it available for the people who were not there.
The elephant in the room: should we use AI for docs, just let ChatGPT write them?
- No: LLMs are frequently wrong, poisoned or non-sensical, telling you to call functions that have never existed.
- Yes: LLMs can be useful to summarise information for a quick starting point. They can help you index information and identify patterns in your corpus.
Dont't rely entirely on AI tools. If you use them, be transparent about it. Agree on a policy, for example: don't accept PRs that were generated by AI.
Avoid ephemeral media like Discord. Don't say "this is easy", that just creates unnecessary anxiety. Don't use foo, bar, a, b: you can use long variable names, it is allowed.
Celebrate human intelligence. Writing documentation is an exercise in empathy. You are not your audience. Docs are living things, they will never be finished, it is okay to deliver something now, and update later.
"You can't teach people everything they need to know. The best you can do is position them where they can find what they need to know when they need to know it." Seymour Papert