Weblog

published Nov 03, 2021, last modified Nov 04, 2021

Eric Bréhault: AbFab! Come back to easy frontend development

published Nov 08, 2021

Talk by Eric Bréhault at the online Plone Conference 2021.

My talk from last year, presented on a T-shirt, was about second-guessing the SPA pattern, or Single Page App. After this year's introduction with T-shirts, I come again with a T-shirt thing, showing: AbFab: Absolutely Fabulous. It refers to the comic show from the nineties. British humor was always an inspiration for the Plone community. Or is it Spanish? Ab fab, away from the factory.

In the zeroes, the question was: why do I use 5 different jQuery versions on my page?
In the tens: why 5 different JS frameworks?
In the twenties it is just: WHYYYYY?
If you don't have the most recent problem, you are a big loser.

What if I want something simpler? Back in the nineties you had good British TV shows, and easy web development. HTML is still fabulous. HTML contains content.

Components are contents too, which is good. JS frameworks use this a lot. Idea of AbFab: push this to the backend.

Use a tree of components and subcomponents, like a filesystem with folders.

You request /polls/poll1.
You get HTML, json, a few components.
You then request /polls/poll2.
You have the HTML already, so you just needs the json, and we already have most components, so we only need for example one extra.
Updated component: serve a new version, and all pages will use it.

You have no bundles. You just serve separate components, which are dynamically loaded as dependencies.
Your browser can do that.

Low code.
I use Svelte for this. In the basis, you simply write HTML. <h1>Hello world!</h1> is a valid Svelte component. In React or Angular it is harder. In Svelte, if you need css, use a <style> tag. Need javascript? Use a <script> tag. To use another component:

<script>
import Hello from `./hello.svelte`;
</script>
<Hello></Hello>

So what is AbFab doing?
A component is stored in Guillotina, together with a compiled version.
Data in Guillotina is linked to a component.
Ask for the data, you get the needed components.
It will be tons of requests, but they will be small, and via HTTP/2 this goes quite fast.
We are using Guillotina at its finest, good for storing lots of small things in a hierarchy.

Low deployment.
My way to make it simple: Docker.

Low build.
I don't want npm installed, or webpack. At some point you do need it. I know this guy who edits CSS on GitHub, then has to wait 30 minutes for it to deploy, because it is too painful to build on his laptop. How to do this? Through The Web. Yes, it is naughtly, maybe forbidden, but I like it: the ability to change yourself.
I am 47 and have to accept who I am.
I am Eric.
I like Through-The-Web-shit.

The core of AbFab is about 200 lines. The editor is bigger.

Using an IDE is fine too. You can sync with the database. I make sure that the code I do is consistent with the documentation I am writing.

What is it good at? Small applications, like we would have done with Plomino previously. To be honest, for now I just made this for fun, so I don't care what it is good at.

Is it totally stupid? I made some disputable choices there. But it is about 200 lines frontend, 300 backend, so it can't be that bad.

Site updated from Plone 2.5 to Plone 6.0

published Nov 04, 2021

It only took ten years of making plans.

I am one of the Plone Release Managers, and have been working on Plone 6, which is now in alpha stage. But my personal website was still using the ancient Plone 2.5:

Screenshot of my website end of October 2021: still Plone 2.5

Often I have made plans to update my site to:

  • Plone 3
  • Grok
  • Plone 4
  • Plone 5
  • Plone 5.2
  • finally Plone 6

Long ago it was clear to me that an inline migration would not be practical. It would take too many steps: update the code to Plone 3.3, migrate the data, to Plone 4.3, migrate data, to Plone 5.2 Python 2.7, migrate data, Plone 5.2 Python 3, migrate data, Plone 6, migrate data.

Additionally, the question was how to handle the weblog, which is the main content. This was using Products.Quills, a Plone add-on for blogs. Updating to Plone 3 could have worked at the time, but this was made harder by some some custom code I added. This enabled me to use it as a podcast. I used this to enrich some of my summaries of sermons from my church with the actual audio of the sermon. I doubted whether to even include this content in Plone 6, as the last sermon was from 2008. I hate breaking links, so I kept it, although a bit hidden.

Another point that needed some extra attention, was that most, if not all, blog entries were not written in html, but in ReStructuredText. I make a lot of summaries of talks when there is a Plone Conference. The html editor on Plone 2.5 did not work anymore, or I disabled it to a simple textarea. I always open up the same text editor that I use for programming (previously Emacs, currently VSCode), and type the summary there. I much prefer writing ReStructuredText there, especially when I simply need text without any markup. I then paste it in Plone, without fear of losing all my work when my internet connection dies.

Lastly, I have an RSS/atom feed which is used by planet.plone.org and maybe others to stay updated when I add a new blog entry. I did not want this url to change.

Anyway, about six years ago I decided that I would use collective.jsonify to export my site, and then import it using transmogrifier. But time passed without any progress. This year, collective.exportimport was shaping up to be the preferred way to import the data. For export you can use it in Plone 4.3 as well, but definitely not in Plone 2.5.

At the beginning of this week I looked at jsonify. Didn't I have a local copy of my website on my laptop, with collective.jsonify installed? No! And it was not installed on the live site either. And installation would be hard, because the site uses Python 2.4, and currently you cannot even reach PyPI with older versions of Python 2.7.

Mildly panicked, I started on a custom script to export the content, still as json. Some details aside, this actually was not so hard. I looked at what collective.exportimport expected, and created the Python list of dictionaries accordingly. Then do a simple json.dumps() call and you are done. Except that this gave an ImportError: the json module is not available in Python 2.4. Okay, install simplejson package. But you need PyPI access for that, which does not work. Workaround:

  • Manually download an egg of Python 2.4-compatible simplejson 1.7 and save it in the buildout directory.
  • cp bin/instance bin/instance-json
  • Edit the new script and add the simplejson egg to the system path.
  • bin/instance-json run export_mvrsite25.py

After that, it was not too hard anymore. I used plonecli to create a new add-on with Plone 6.0.0a1. I actually do not yet use the add-on code, except for loading a minor patch that I added, but this gave a reasonable, modern Plone 6 buildout. Create a Classic Plone site, tweak a few settings (let Folder use folder_workflow, allow English and Dutch, navigation depth 1, enable syndication, configure caching and mail), import the content with collective.exportimport, edit a bit, and done.

The weblog now consists of standard Folders and Pages. To improve the view, I added a Collection, showing the latest pages first, and showing all content of the last seven blogs. I enabled syndication here, so it has an RSS/atom feed.

The weblog has always advertised two atom feeds:

  1. One for all weblog entries, at https://maurits.vanrees.org/weblog/atom.xml
  2. One for weblog entries with keyword 'plone' at https://maurits.vanrees.org/weblog/topics/plone/@@atom.xml

In the new site, the first one kind-of worked out of the box, but it only showed the items that were directly in the weblog folder, and this is not where my weblog entries are. I solved this with a patch to Products.CMFPlone.browser.syndication.views.FeedView: when atom.xml is viewed on a folder, check if it has a default page, and show the atom.xml of this default page instead. In this case, the default page is a Collection with the correct settings. So the general feed will keep working.

For the second one, my first idea was to create a folder 'topics' and within it a Collection with id 'plone'. Problem: 'plone' is a reserved word and cannot be used as id of a content item, so the id became 'plone-1'. Solution here: create the 'plone-1' collection directly in the weblog folder, and do a redirect in the frontend server (nginx):

 rewrite ^/weblog/topics/plone/@@atom.xml /weblog/plone-1/atom.xml last;

And that's it! My website is now on Plone 6.0.0a1:

Screenshot of Site overview in new site.

There are some more details that I could go into, like splitting up the buildout into multiple parts, with tox as main way to build everything, in preparation for moving more and more parts to pip-only. But that will have to be for another time.

Meanwhile: have a look around, and enjoy the fresh look!

Steering Circle

published Oct 29, 2021, last modified Oct 31, 2021

Meeting of Plone's Steering Circle at the online Plone Conference 2021.

Volto 14 alpha 23 is out. So still in alpha, but companies are using it in production. Should be final soon. Some plans for Volto 15. Created plone.volto integration package, where we try to give an easy transition from earlier company-specific versions. plone.restapi as always is pretty boring, stable. Erico worked on Docker integration.

Plone 6 alpha 1 is out. Eric sent an email for some coordination, like docs, training, accessibility, installers. If you want to be involved, let me know.

Franco has stepped out of the Framework Team, thank you for all your work. There is discussion about the role of the Framework Team. Plan is to keep it running, some more people have been asked.

Membership team: we have some people in sight as new members. Erico is stepping down as team lead, Victor is stepping up.

Security: Plone 6 will have 5 years security support. Synchronizing with Zope team. Some new members may be coming.

Marketing: busy with conference, also after the conference. Busy with plone.org renewal.

Installers: see the talk by Jens Klein earlier. Plone 6: no more installer, but we do have tooling. There are Docker images. We may want to reduce the role of buildout, and focus more on pip.

Plone Conference: you are looking at it. Some tasks to do afterwards. If anyone is interested in getting a certificate for following a training, ask us, and we can send it.

Internationalization: new branches for Plone 6, so Plone 5 uses different branch. New releases for 5 and 6. Updating po files, looking at i18n in Mockup.

Admin/Intrastructure: servers are still running. Cat herding sysadmins for doing stuff, keeping things up to date.

Trainings: relaunch is complete. We have three new trainings: Theming Plone Classic, Deploying Plone 6, Getting Started with Plone 6 (that one only in video). Various have seen major updates. Need to work on landing pages (we have two), remove the number 5 from the url, update some more trainings. Maybe Mastering Plone Classic training, but hard with navigation due to duplicate section targets when copying. Migration training would be good. We need to prune and tag some.

Plone Classic: We did polishing on Barcelonate, it is pretty ready. Focussing on bobtemplates before the trainings, making theming easier. JavaScript/ES6 is the remaining big issue. Plan is to finish it this year, we are quite far. We need other people helping us out.

Documentation: will be releasing a new Plone 5 branch today. For the new stuff, the tech stack is ready. New version of automated screen shot is about ready. We don't want a duplicate of the training, but we can automatically include code of it, so there is only one source of truth. The style guide is not always followed, seeing what we can do about that. Biggest point is missing documentation. There are now branches where the various teams can add and edit their content. We may change things, but we take this as input for the final structure.

Fred van Dijk: 7 things that can surprise you when you start customising or developing for Plone

published Oct 28, 2021, last modified Oct 29, 2021

Talk by Fred van Dijk at the online Plone Conference 2021.

This is a basic rundown and summary of our beloved subjects like ZODB persistence. Traversal. The view/viewlet/portlet trinity. How is a call handled in Plone. The differences between zcml and generic setup. Utilities and the ZCA. restrictedTraverse. The Plone catalog.

These are surprises that I have encountered myself, or that I have seen on faces of people I have trained or worked with during the years.

  1. Everything can or could be done through the web (TTW).

Zope vision from the nineties. Why don't we use this dynamic language called Python so we can change things TTW? It's so easy.

  1. The learning curve.

It starts easy, but then you hit what we call a Z-shaped learning curve. Dynamic Python makes things easy, and then it makes things hard, at least when you put Zope on it, then CMF, then Plone. Plone the product on top of a CMS on top of a framework on top of a language. We have a product, a framework, a stack, so it is hard.

  1. Five levels of conceptual complexity.

It helps to teach all the levels. Give new users a drawer for each level, so they have a box that they can put some info in.

You have:

  • browser: html, css, js
  • frontend: transforms, templates, zpt, metal, diazo
  • application logic: views, viewlets, portlets, adapters, Zope Component Architecture
  • dynamism: GenericSetup, zcml, xml, zope schema
  • programming language: Python, buildout, pip
  • package WorkManager (OS): apt, rpm

4 Same language/formats on different levels:

  • XML is used for the ZCA, GS, zope.schema, Diazo rules
  • package manager: buildout, pip, setuptools, GS

But: there is no magic. It is just advanced technology.

Startup:

  • Python uses sys.path modules to start bin/instance
  • ZCA loads site.zcml, package includes, other zcml to change configuration.
  • Then we have a runtime environment with objects and classes, the ZODB. GenericSetup is then some XML that you can use to change the ZODB.

So the ZCA overrides components in runtime. The alternative is to edit core files, maybe compile them, and restart. Much less nice and not sustainable.

So now we have a Plone process running.

  1. Zope is not that complicated.

Over HTTP Zope gets a request, does traversal, finds an object in the ZODB, loads it in memory. Then on top of this object we show a browser view / template. The template hooks into a main template, maybe does some sub requests, some Diazo, and we end up with some html and we are done.

This is all still 'lies to children'. It is simplified until we are able to understand more. With increment exposure to these concepts, it will stick more. It is complicated, but there is no magic.

  1. Acquisition.

It is traversal in the wrong direction.

  1. When you try to explain things, you improve your own understanding.

There is so much Plone content online: training, docs, youtube, github, discourse. We all learn in different ways, with own preferences, reading, viewing. There are so many repositories on github that you can explore for new ideas. Just yesterday Philip did a talk about exportimport and afterwards I did it, but from a different angle. It helps.

The community is our biggest asset.

Annette Lewis and Will Gwin: From Zope to Plone: Thinking User-First During Migration

published Oct 28, 2021

Talk by Annette Lewis and Will Gwin at the online Plone Conference 2021.

Migrating a site is always a challenging task, but when you have dozens of subsites with specific brand standards and custom user functionality, the challenge becomes mammoth. Six Feet Up worked hand-in-hand with Purdue's College of Engineering to migrate their existing Zope site and its subsites into a new Plone instance running on Plone 5.2 / Python 3. Throughout the migration process, we considered the project scale, timelines, and limiting the impact on end users, all while managing the balance between user needs and best practices. During this presentation, you will learn:

  • why it matters to think user-first during migration,
  • about creative solutions for translating content and functionality into Plone, and
  • how to successfully migrate subsites.

An overview of the project:

  • HigherEducation always seems to go a bit slower, certainly with migrations.
  • Our previous CMS was built in Zope and was getting extremely old.
  • We have been using Python since 2001.
  • There were security concerns and modernization issues.

The impact:

  • Only 15+ content editors.
  • 40+ public facing subsites
  • 30,000 total pages.
  • 20+ departments and units

Why did we select Plone as the next CMS?

  • It is a modern CMS solution
  • Python-based, so that fit what we currently have.
  • It is built on top of the Zope web framework that we were already using.
  • We looked at Drupal, Wordpress, and more, but that would have been a too big undertaking.

This was in 2018.

Laying out the solution. From requirements to action. Who are our users and what do they want? Not just our direct client (Will and his team) but their clients/users.

Challenges during development:

  • Purdue University changed its brand look. We had to seamlessly blend subsites into the existing parent site.
  • Convert all content types and templates from Zope to Plone.
  • Keep sight of the user experience in both environments. Could they use the new environment without too much training, or needing to have too much tech knowledge?

Determine the project essence. Distill the requirements down into broad categories: accessibility, usability, flexibility, security.

What is the path to successful collaboration?

  • The absolute best might not be the right answer.
  • It's okay to say no to an idea, but you dhouls have an alternative ready.
  • Aim for the best, avoid the dangerous, end up somewhere in the middle sometimes.

On to our development goals:

  • Do things in a Plone way. Plone uses Zope, but Zope may do some things in a different way than is the best way in Plone.
  • So observe best practices.
  • Make it intuitive and keep it familiar for the editors.

Solutions at a glance

Migrating site content:

  • We wanted to move subsites one by one, as needed.
  • Translate existin content to Plone content types
  • We could re-import content over existing content non-destructively.

Theming: retrofitting Plone into an existing theme. That is what Diazo was made for, bridging the gap between Plone and the theme, especially since the theme is 'living', with subtle changes coming in often.

Each subsite had a browser view named local.css to change some things. Not really what you want in Plone, but they really needed it, as a way to make subsites or sections look different. So we added an action to edit the local css, inheriting from parent folders.

We created a subsite settings control panel. We used lineage.registry for this. All kinds of customizations can be done based on that, for example add extra text and links in the navigation menu. They used to be able to do this in Zope as well, but that was with various properties, and much more code oriented.

We use Mosaic for flexibility of layout. In Zope we had blocks for layout. Mosaic took this a step further into a nice UI with drag and drop. It gives faster site prototyping and development.

The sandbox: we wanted to have safe spaces for content experimentation. Completely separate from production environment. It is used for new user training and testing. It is quick and easy to reset.

This migration project has been a constant collaboration between the Purdue communications office, Engineering Computer Network, and Plone company Six Feet Up. The content editors feel empowered to make complex changes, without constant oversight from my team.