Weblog
Thijs Feryn: Varnish caching in Plone
Talk by Thijs Feryn at the Plone conference 2024 in Brasilia.
Link to talk information on Plone conference website.
If your website is slow or is down, you lose money. How do you solve this. Some say: throw servers/money at the problem. Yes, this can work in the short term, but mo' money, mo' servers, mo' problems.
Optimize: make it faster: your Python, javascript, css, html. At some point you hit a limit.
Use caching. That is what we will talk about. Basic idea: why would you recompute if the data has not changed.
I am tech evangelist at Varnish software. I have a colleague here at the conference who speaks Portuguese, and a German colleague who speaks Spanish. So talk to them as well if you want.
New book by me: Varnish 6 by example.
The visitor of your site first gets sent to Varnish, and this either sends the request on to Plone, or serves a previous answer from its cache. That is basically how it works.
Everything that is static is easy to cache. We listen to any cache headers that are available in the request, or set by Plone in the response.
But by default, if the request has a cookie, we do not cache. But these days every website uses cookies! So we just have to deal with them properly.
You do this using the VCL language. This is a domain-specific language for Varnish only. It is not a top-down language, it builds on other settings, and you can write extra routines for this.
Let's talk about caching in problem. Disclaimer: I am not a Plone expert.
I have setup a Plone backend and Volto frontend locally with varnish, in Docker containers.
In the cache settings:
- I see good things in here, you can enable caching.
- Very good: you can set a url for purge requests.
- A bit strange: you need to select which content types should be purged, I selected them all.
- For various sorts of contents and files you can choose strong caching, moderate, weak or terse, which sets all kinds of cache headers, even some e-tags, which is cool.
E-tags are sent on again by the browser if it requests this again. Plone then checks this e-tag and if nothing has changed, it sends back a 304 Not Modified response.
What if we could store the e-tag somewhere, then you could return very fast maybe even without contacting the framework. Maybe this happens already.
Some of the rules use a Vary header, but I wonder if the Accept header is the best here. Also, watch out for filling the cache when an attacker makes request after request with a different Accept header. In Varnish we have an `accept` module that can help here.
Cache purging: on a PURGE request we clear the cache for a url. You allow this from a white list of IP addresses. Plone purges a lot of paths, but maybe not enough. I think the `/Plone` is missing from some urls. And some urls with query parameters are not purged.
I have written a VCL for this, but not sure if that is the way to go. What about tag-based cache invalidation instead? You can do that in the VCL with `add_key`. You can also strip off query parameters.
Also, you can mark things as required, instead of actively purging it. Note that you can also serve stale content while Varnish gets a fresh answer from Plone.
Can we set the stale-while-revalidate keyword in Plone, for the Cache-Control header? That could help.
Question: should you use the varnish load balancer or a specialised load balancer like haproxy? Answer: I am biased of course, but if you are already using Varnish, just use that. We have some advanced options.
Plone is already very advanced here in how it does caching. But look at the Vary Accept header, and at tag-based caching.
Guido Stevens: Public Sector & Open Source: Natural Allies
Talk by Guido Stevens at the Plone conference 2024 in Brasilia.
Link to talk information on Plone conference website.
The worldwide Crowdstrike outage on Windows computers this year: crazy how such a thing can happen. I have better quality control than that! How resilient is our tech infrastructure?
What is the point of writing code if the world is going to pieces?
A short history of the internet revolution. The Berkeley software was started in the sixties. It grew a lot, in the nineties Web 1.0 started. Metcalfe's law: the value of each new connection increases.
Mandelbrot wrote The (mis)behavior of markets. From bell curve to power law or the long tail: a few people get very rich and most people get very poor. That happens in networked connections. People are more geared towards linear connections.
Web 2.0, in the 2000s, went from a read-only web, to a read-write web.
Trump, Brexit, Covid, they all rocked the world.
"Post-truth is pre-fascism", writes Timothy Snyder in On tyranny. You should read this.
Web 3.0 is skipped and we just have AI. AI will swamp the web with nonsense, and no one can find your content anymore.
I had higher hopes, too. I did not sign up for this dystopia. I have an MBA in environmental economics, done OSS web development for decades, am on the Plone Foundation, and founder of Quaive.
I wrote a book Systems of intent: Open source (Plone) plus knowledge management = Quaive. Quaive changes the Plone UI. It shows an activity stream, documents, news, personally configurable dashboard. It enables a shift in the company from top-down to bottom-up. Technology can also cause problems. But is technology the problem? Or is capitalism the problem? Resistance is not futile.
Chaos theory, butterfly effect: small acts can have huge effects. You and I are all complex individuals, complex systems. It is a shift from a clockwork universe to an alive universe. Emergence: on a higher level something more happens than what you can imagine looking at the lower levels.
Adrienne Maree Brown: Emergent strategy. Adaptive: constant change. Move at the speed of trust.
Principles:
- Local autonomy. Have control over your technology.
- Strive for technological excellence. Our stuff needs to be good and secure.
- Networked agility: how we cooperate and coordinate effectively.
- Community sharing: celebrate that we are not alone.
The clients that use Quaive may not be interested in our GPK license, but they do care about those principles.
Asterix: the whole of France is occupied? No, one corner not. Same now: Half of Belgium (the Walloon region) uses Plone in basically all their towns, supported by IMIO. We got into an agreement when I visited personally.
Technical detail: using Docker, IMIO was able to host 8 sites on a server where we previously could only host one site.
Think global, act local. Open source is not only free as in beer, but free as in freedom. The community makes this possible: Free as in community. Public sector and open source are natural allies.
Fred van Dijk: Volto Information Architecture
Talk by Fred van Dijk at the Plone conference 2024 in Brasilia.
Link to talk information on Plone conference website.
I am using Plone since 2002, but am not a frontend expert. So currently I still have a bit of an outsider view on Volto, but I am working at Kitconcept now so will gain more experience.
Central thesis in this talk:
We need to rethink and upgrade or extend our content types and blocks system for Plone 7 and beyond.
For a big project at Zest we needed to migrate a website from Plone 4 to Plone 6 with Volto. I and colleague Maurits are mediocre frontenders at best, so we had help from two frontenders, though they had no experience with Plone or Volto.
We have almost all the building blocks already: distributions, the most secure CMS, the composite page editor, dexterity behaviours.
To do: generalised and connected frontend-backend configurations, devcontainers, version 3 of the block model, more documentation.
The big project: upgrade of the main website of the Flemish environmental agency, kind of the little sister of the European environmental agency. A communication agency came up with 20+ content types and a functionality geared towards Drupal. But the existing site was Plone 4. There were far too many inconsistencies in their design. Let's just start.
Old site is still online: https://www.vmm.be. The new site will go live tomorrow: https://vmm.vlaanderen.be. Not the greatest timing, but at least I have no talks tomorrow.
Versions used in this project: Plone 6.0 backend, Volto 18 alphas at first, now 18.1.1, Volto Light Theme. Still a Cookiecutter template as basis, if we would start now we would use Cookieplone.
Main assumptions in the Volto information architecture:
- Composite pages are now built-in, with blocks layout almost everywhere.
- Pages are folders.
Challenge: Where do you store images that belong to a page? If you are still adding the page, there is no place yet for storing the image. Store it in the block? No: use references. Put the images in a folder and reference it from a page.
Metadata images: we have lead images for listings and header images for the top of a page. The design should limit the aspect ratios and sizes, preferably just two are three sizes. That is more pleasing to the eyes than 10+ differently sized images on a page.
Media and data (for example graphs) belong in data content types, and their presentation should be in blocks.
One block to rule them all: we have the teaser block in core Volto now. Then we define adaptations/variations for presenting the different content types.
With Volto the editors can go wild on landing pages. Danger: they also do that in detail pages, which may not be what you want. Templates to the rescue: you can create an initialBlocks
configuration. We added metadata (subtypes) with which an editor could say what kind of page it is.
"Listy" blocks: you can either use a grid block with teasers, or a listing/search block. They can look the same. But the design had lots of combinations that should be supported, and then all kinds of exceptions came up. To the rescue: default variants. The default teaser block variant can adapt itself to the listing.
Responsiveness: we sometimes have 2 blocks in a listing or grid, sometimes 3 or 4, needed to deal with that. Needed to fix some restrictions with the search facets too, needing to be able to choose different vocabularies.
We have three layers: the visual side of Volto, then information architecture concepts, then the backend. Idea: connect frontend blocks with backend instance behaviours. Then you have no explosion on content types.
Lightning talks Friday
Lightning talks on Friday at Plone conference 2023, Eibar, Basque Country.
Jakob Kahl: Beethoven Sprint
We will again host the Beethoven Sprint in Bonn, 13-17 May 2024.
Lukas: PloneGov-BR
Plone portal from Brasil. https://github.com/plonegovbr
1.2 billion page view per year, 28 sites, 800k+ content items.
Plone conference Brasilia 18-24th November 2024.
Victor Fernandez de Alba: Volto PLIPs
PLone Improvement Proposals for Volto. See on GitHub.
Mikel Larreategi: Version inspection
What to do when Maurits tells you to update five packages with security fixes? We made a product for this. DigitalOcean API to get all the project lists, scp to get the instance files (bin.instance, yarn.lock), and search.
CodeSyntax: #PrettyEibar awards ceremony
The winner is Kim Nguyen with a picture of a blue house.
Michael McFadden: Have you heard about Tau?
The Tau manifesto. Tau is 2 times pie. Pie is only half the story. 1 tau is 1 turn. Much easier to teach to children. See https://tauday.com/tau-manifesto.
Kim Nguyen: Do you want customers?
Do you want glory? Do you want to help Plone? Help Plone help you. Get your Plone provider listing today!
Go to https://plone.org/providers and register.
Dylan Jay: Python meetups
In 2014 I moved from Australia to Bangkok. I started a meetup for Python. We are at meetup 94 now. There is a PyCon in Thailand December 13-15 this year.
Do you live in a town that does not have Python meetups? Create one! Build it and they will come. You need a venue. Anyone who is interested in developers, can help you. How do you get people to talk: twist their arm, get them drunk. Have two short talks rather than one big one.
The Python foundation now pays for meetups. You can use meetup.com, but you do not have to. You need a code of conduct. I needed a page, I used pyscript, Python in the browser.
Philip Bauer: Erico, I want a beer
Erico promised me a beer if I did something useful for him. Open an Plone site, do exportimport, you can now export it to one file per item.
Mikel Larreategi: Some random things
I had ideas for talks, but did not do it, so quickly they are here.
pas.plugins.oidc. Created by Mauro Amico. OpenID Connect is a layer of identification. You can install the PAS plugin in Plone and talk to such a server [works in a test setup for me as well, Maurits]. Works with various identity providers, like Google Workspace, Keycloak, EU Login. Click a link, redirect to the provider and identify there, callback, internal connection to provider, user is created in Plone and group management granted. We use it in production in two scenarios. See https://github.com/collective/pas.plugins.oidc.
Lichess: Open source chess server. Free, free, free, no ads, no tracking. 93M games played last month. I wanted to translate it into Basque. They use CrowdIn. Can we use something similar for Plone? Weblate maybe? We use this at CodeSyntax. Perhaps I will work on this the coming months for the core Plone translations.
This Plone conference started at Beethovensprint 2022. Last of of the sprint, after last dinner, with last beer in our hands, a few guys were there and approached us: "You are going to organise the conference, right?"
Thank you everyone!
Plone Foundation annual meeting
Open annual meeting of the Plone Foundation membership at Plone conference 2023, Eibar, Basque Country.
This year the Foundation membership voted in favour of a change to the bylaws, making it more inclusive, and moving to two cohorts for the Plone Foundation board, each cohort serving for two years.
New Foundation members this year: Mauro, Tanya, Jan, Karel, André, Brian, Joao, Martin. We have 100 active members from 21 countries. It you are an emeritus member and want to reactivate, contact the board.
There is a new contributor agreement process, more digital now, not fully automated yet, but easier to handle. There were 43 real new contributors, not including some hopeful GSoc students that never did anything. There were 5 real active GSoc students, thank you for your work, and thank you Google for sponsoring this. Others see that we are handling this well as organisation. And we try to bring the students here to the conference to present their work. Unfortunately there are strict rules in the EU making it hard to get visa for everyone.
Election for the next Plone Foundation Board of Directors. The result of the vote is in. For a one year term: Brian Davis, Kim Paulissen, Martin Peeters, Paul Roeland. For a two year term: Eric Brehault, Guido Stevens, Mikel Larreategi. Congratulations. Thank you outgoing directors: William, Jens, T. Kim and Erico.
Meeting adjourned.