Weblog

published Nov 03, 2021, last modified Nov 04, 2021

Steering Circle

published Oct 29, 2021, last modified Oct 31, 2021

Meeting of Plone's Steering Circle at the online Plone Conference 2021.

Volto 14 alpha 23 is out. So still in alpha, but companies are using it in production. Should be final soon. Some plans for Volto 15. Created plone.volto integration package, where we try to give an easy transition from earlier company-specific versions. plone.restapi as always is pretty boring, stable. Erico worked on Docker integration.

Plone 6 alpha 1 is out. Eric sent an email for some coordination, like docs, training, accessibility, installers. If you want to be involved, let me know.

Franco has stepped out of the Framework Team, thank you for all your work. There is discussion about the role of the Framework Team. Plan is to keep it running, some more people have been asked.

Membership team: we have some people in sight as new members. Erico is stepping down as team lead, Victor is stepping up.

Security: Plone 6 will have 5 years security support. Synchronizing with Zope team. Some new members may be coming.

Marketing: busy with conference, also after the conference. Busy with plone.org renewal.

Installers: see the talk by Jens Klein earlier. Plone 6: no more installer, but we do have tooling. There are Docker images. We may want to reduce the role of buildout, and focus more on pip.

Plone Conference: you are looking at it. Some tasks to do afterwards. If anyone is interested in getting a certificate for following a training, ask us, and we can send it.

Internationalization: new branches for Plone 6, so Plone 5 uses different branch. New releases for 5 and 6. Updating po files, looking at i18n in Mockup.

Admin/Intrastructure: servers are still running. Cat herding sysadmins for doing stuff, keeping things up to date.

Trainings: relaunch is complete. We have three new trainings: Theming Plone Classic, Deploying Plone 6, Getting Started with Plone 6 (that one only in video). Various have seen major updates. Need to work on landing pages (we have two), remove the number 5 from the url, update some more trainings. Maybe Mastering Plone Classic training, but hard with navigation due to duplicate section targets when copying. Migration training would be good. We need to prune and tag some.

Plone Classic: We did polishing on Barcelonate, it is pretty ready. Focussing on bobtemplates before the trainings, making theming easier. JavaScript/ES6 is the remaining big issue. Plan is to finish it this year, we are quite far. We need other people helping us out.

Documentation: will be releasing a new Plone 5 branch today. For the new stuff, the tech stack is ready. New version of automated screen shot is about ready. We don't want a duplicate of the training, but we can automatically include code of it, so there is only one source of truth. The style guide is not always followed, seeing what we can do about that. Biggest point is missing documentation. There are now branches where the various teams can add and edit their content. We may change things, but we take this as input for the final structure.

Fred van Dijk: 7 things that can surprise you when you start customising or developing for Plone

published Oct 28, 2021, last modified Oct 29, 2021

Talk by Fred van Dijk at the online Plone Conference 2021.

This is a basic rundown and summary of our beloved subjects like ZODB persistence. Traversal. The view/viewlet/portlet trinity. How is a call handled in Plone. The differences between zcml and generic setup. Utilities and the ZCA. restrictedTraverse. The Plone catalog.

These are surprises that I have encountered myself, or that I have seen on faces of people I have trained or worked with during the years.

  1. Everything can or could be done through the web (TTW).

Zope vision from the nineties. Why don't we use this dynamic language called Python so we can change things TTW? It's so easy.

  1. The learning curve.

It starts easy, but then you hit what we call a Z-shaped learning curve. Dynamic Python makes things easy, and then it makes things hard, at least when you put Zope on it, then CMF, then Plone. Plone the product on top of a CMS on top of a framework on top of a language. We have a product, a framework, a stack, so it is hard.

  1. Five levels of conceptual complexity.

It helps to teach all the levels. Give new users a drawer for each level, so they have a box that they can put some info in.

You have:

  • browser: html, css, js
  • frontend: transforms, templates, zpt, metal, diazo
  • application logic: views, viewlets, portlets, adapters, Zope Component Architecture
  • dynamism: GenericSetup, zcml, xml, zope schema
  • programming language: Python, buildout, pip
  • package WorkManager (OS): apt, rpm

4 Same language/formats on different levels:

  • XML is used for the ZCA, GS, zope.schema, Diazo rules
  • package manager: buildout, pip, setuptools, GS

But: there is no magic. It is just advanced technology.

Startup:

  • Python uses sys.path modules to start bin/instance
  • ZCA loads site.zcml, package includes, other zcml to change configuration.
  • Then we have a runtime environment with objects and classes, the ZODB. GenericSetup is then some XML that you can use to change the ZODB.

So the ZCA overrides components in runtime. The alternative is to edit core files, maybe compile them, and restart. Much less nice and not sustainable.

So now we have a Plone process running.

  1. Zope is not that complicated.

Over HTTP Zope gets a request, does traversal, finds an object in the ZODB, loads it in memory. Then on top of this object we show a browser view / template. The template hooks into a main template, maybe does some sub requests, some Diazo, and we end up with some html and we are done.

This is all still 'lies to children'. It is simplified until we are able to understand more. With increment exposure to these concepts, it will stick more. It is complicated, but there is no magic.

  1. Acquisition.

It is traversal in the wrong direction.

  1. When you try to explain things, you improve your own understanding.

There is so much Plone content online: training, docs, youtube, github, discourse. We all learn in different ways, with own preferences, reading, viewing. There are so many repositories on github that you can explore for new ideas. Just yesterday Philip did a talk about exportimport and afterwards I did it, but from a different angle. It helps.

The community is our biggest asset.

Annette Lewis and Will Gwin: From Zope to Plone: Thinking User-First During Migration

published Oct 28, 2021

Talk by Annette Lewis and Will Gwin at the online Plone Conference 2021.

Migrating a site is always a challenging task, but when you have dozens of subsites with specific brand standards and custom user functionality, the challenge becomes mammoth. Six Feet Up worked hand-in-hand with Purdue's College of Engineering to migrate their existing Zope site and its subsites into a new Plone instance running on Plone 5.2 / Python 3. Throughout the migration process, we considered the project scale, timelines, and limiting the impact on end users, all while managing the balance between user needs and best practices. During this presentation, you will learn:

  • why it matters to think user-first during migration,
  • about creative solutions for translating content and functionality into Plone, and
  • how to successfully migrate subsites.

An overview of the project:

  • HigherEducation always seems to go a bit slower, certainly with migrations.
  • Our previous CMS was built in Zope and was getting extremely old.
  • We have been using Python since 2001.
  • There were security concerns and modernization issues.

The impact:

  • Only 15+ content editors.
  • 40+ public facing subsites
  • 30,000 total pages.
  • 20+ departments and units

Why did we select Plone as the next CMS?

  • It is a modern CMS solution
  • Python-based, so that fit what we currently have.
  • It is built on top of the Zope web framework that we were already using.
  • We looked at Drupal, Wordpress, and more, but that would have been a too big undertaking.

This was in 2018.

Laying out the solution. From requirements to action. Who are our users and what do they want? Not just our direct client (Will and his team) but their clients/users.

Challenges during development:

  • Purdue University changed its brand look. We had to seamlessly blend subsites into the existing parent site.
  • Convert all content types and templates from Zope to Plone.
  • Keep sight of the user experience in both environments. Could they use the new environment without too much training, or needing to have too much tech knowledge?

Determine the project essence. Distill the requirements down into broad categories: accessibility, usability, flexibility, security.

What is the path to successful collaboration?

  • The absolute best might not be the right answer.
  • It's okay to say no to an idea, but you dhouls have an alternative ready.
  • Aim for the best, avoid the dangerous, end up somewhere in the middle sometimes.

On to our development goals:

  • Do things in a Plone way. Plone uses Zope, but Zope may do some things in a different way than is the best way in Plone.
  • So observe best practices.
  • Make it intuitive and keep it familiar for the editors.

Solutions at a glance

Migrating site content:

  • We wanted to move subsites one by one, as needed.
  • Translate existin content to Plone content types
  • We could re-import content over existing content non-destructively.

Theming: retrofitting Plone into an existing theme. That is what Diazo was made for, bridging the gap between Plone and the theme, especially since the theme is 'living', with subtle changes coming in often.

Each subsite had a browser view named local.css to change some things. Not really what you want in Plone, but they really needed it, as a way to make subsites or sections look different. So we added an action to edit the local css, inheriting from parent folders.

We created a subsite settings control panel. We used lineage.registry for this. All kinds of customizations can be done based on that, for example add extra text and links in the navigation menu. They used to be able to do this in Zope as well, but that was with various properties, and much more code oriented.

We use Mosaic for flexibility of layout. In Zope we had blocks for layout. Mosaic took this a step further into a nice UI with drag and drop. It gives faster site prototyping and development.

The sandbox: we wanted to have safe spaces for content experimentation. Completely separate from production environment. It is used for new user training and testing. It is quick and easy to reset.

This migration project has been a constant collaboration between the Purdue communications office, Engineering Computer Network, and Plone company Six Feet Up. The content editors feel empowered to make complex changes, without constant oversight from my team.

Alan Runyan: Building a Secure Cross Platform Mobile/Tablet Application (Flutter) using Plone as Backend Server

published Oct 28, 2021

Talk by Alan Runyan at the online Plone Conference 2021.

Enfold has been working on a secure cross platform mobile application the past eight months. Walk through of the Requirements, Security, Flutter framework, Backend configuration of Plone, Authentication and Lessons Learned. Our goal is to have a free public release of a limited version of the application Q4 2021.

It was an adventure for us. The core team never built a large mobile application. We did not know what we did not know.

The big picture:

  • A mix of devices (Android, iOS) needs to synchronize files to and from Plone.
  • All services are self-hosted in GovCloud. So we have no central database or server that we control.
  • If this becomes a success, then future phases may require a lot of certification of the codebase, code reviews.
  • Initially we worked with 15k devices, supporting 40k would be a success, the ceiling that we might support is 300k.
  • Users are completely offline for longer times.

We used Flutter to create a React native app, see https://flutter.dev It is a UI toolkit. Why did we use Flutter?

  • It is cross platform mobile.
  • It uses Dart, which is statically typed, making code analysus much easier.
  • Google seems to be prioritizing developer user experience, it really shows of quite a bit.

Dart has asynchronous code as a first class citizen. Quite different from Python. Runtime reflection (pdb) is unavailable. It has good ergonomics, with generics and closures. It is a driving force behind the Flutter toolkit.

Thoughts on mobile development:

  • It is a lot to take in.
  • You need to understand lots of languages, for us: kotlin/java, swift/obj-c, and Dart.
  • No idea how to test platform integration.
  • Native libraries are managed using Cocoapods/Gradle. Flutter drives those. Setting it up is yet another new thing to learn.
  • There are lots of inconveniences, like how do you read sqlite off a device, because that is how we store some of the info?
  • Also inconsistencies: if the app works on an emulator, that does not mean it works on a device.

On the server side:

  • Plone operating as a Webdav server
  • We need to support OIDC (mod_openids/oauth2)
  • Not many writes, maybe 100-1000 per day, but lots of reads.
  • 20k+ devices daily

Alternatives to the server/protocol could be nice:

  • Honestly, are there any standards other than WebDav?
  • An S3 api would be reasonable.
  • So ideas are welcome, let me know.
  • We have been working on prototypes with guillotina_webdav + openidc.

The good parts of Flutter:

  • UI/UX development is very fast, with lots of widgets. We had two developers who were used to Angular, and they took it up quite fast.
  • The bridge to native code (Pigeon) is straight forward.
  • Drift is an amazing sqlite library.
  • Riverpod for state management
  • Dependency management is good (flutter pub). You can tell that they learned a ton from others, like pip. Except that very occasionally the package cache is broken so you need to clean it.
  • They have a good community, with add-ons.

The not so good parts of Flutter:

  • Inconsistent platform features, like WorkManager (Android) versus NSUrlSession (iOS)
  • Dependency churn: often new versions come in, which you then need to check.

The mobile app:

  • We are still wrapping up the remote file operations.
  • After we deploy into production, we will improve the UI.

Yes, we hope to open source the synchronization framework, and maybe the foreground/background transferring subsystem. Yes, we have built a Flutter web-app of this, but it looks just like the mobile app currently. Needs a separate layout really. No, we have not done a Desktop app.

Lightning talks Wednesday

published Oct 27, 2021

Lightning talks on Wednesday at the online Plone Conference 2021.

Peter Holzer: My first Plone 6 site

This was in April 2020. I friend wanted a shop for his post cards. Lots of search filters.

We create a new Plone theme on Bootstrap 5. Updated our bda.shop stack.

See https://www.fotoeigenart.ch

Michael McFadden: Web developer confessions

I had a web form up where people could submit confessions, totally anonymous.

  • I used Perl
  • I sometimes test things in production.
  • I voted against removing the tag from HTML.
  • I store user passwords, as we often need to login as a user to fix their... stuff. We take precautions, but still.
  • I allowed editors to add css at a top level in the site.
  • I have a marquee tag on our site.
  • I am using iframes to include those who are unwilling to move into our big Plone Site. It is a way to do imports.
  • I use pdb on production all the time with a client instance.
  • I use
    for my page layout.
  • I don't setup the whole translation machinery on our Plone Site. Hello USA!
  • I use heavily duplicated code. What's wrong with coding with copy and paste.
  • I did not backup my data, and I knew the RAID aray was bad.
  • I think downtime is good for mental health.
  • I still hate javascript. And now we have node.
  • I don't always do web development. But when I do, I do it through Zope using plone_custom.css on a production site.
  • I fixed it with css. .class {display: none !important}
  • I suggested to a user to try it in another browser, like Edge. I am still ashamed.
  • I am not trained in UX or web design, but I do it anyway, because I just know what's right.

Fulvio Casali: Plone sponsorships

We are looking for financial sponsorship on a regular basis. Open source is a labor of love, but we also need money. The Plone Foundation supports the community financially for sprints, conferences, releases. We have administrative costs, like for trademarks, lawyers, hosting.

So you support independent media producers via Patreon, Substack, Medium, Youtube? How much? If not, maybe your boss does? Have you watched listened to the Plone podcasts?

Sponsors can be providers, customers, universities, companies that somehow use Plone. We couldn't do it without you.

Mail sponsor@plone.org.

Dylan Jay: Why Drupal won down under

Two stories of government websites, in this case UK and Australia, where other CMSes won the day. Maybe with Plone 6 we can gain some ground back.

PreviousNext is a Drupal shop in Australia. In 2012 aGov launched, a distribution of Drupal specifically for Australian Government websites. That was cleverly done. They organized a Drupal conf in Canberra, the capitol. Most of the stuff could not have been done without support by Acquia, the behemoth company behind Drupal. It helps to be able to say that they have got you backed.

Then the UK. In 2012 Gov.uk launched. 2013 Service manual published.

Ideas for Plone:

  • Open CMS for a specific government.
  • Acquia equivalent
  • How to sell o governments that have been burnt by Plone?
  • Target internal React developers.
  • Use a better term than 'enterprise'. Governance CMS for React Developers?
  • Concentrate on services, not websites?

Maik Derstappen: Current state of Mockup

We are moving Mockup and Patternslib to ES6. We replaced some Mockup patterns with patterns from Patternslib. Most mockup patterns are finished, just a few left that need a bit of work. Plone Classic UI frontend works basically, some issues left in control panels mostly.

Things will get easier. No more RequireJS yelling at you. Add-ons can provide, require and ship any javascript module. Plone will only load everything once, thanks to Webpack module federation.

Timeline: ES6 branch will be merged in the coming months.

Give it a try using the buildout.coredev repository and:

./bin/buildout -c plips/plip-3211-mockup-redone.cfg

Help us! Join us during the conference sprint or every Thursday on Discord, the classic-ui voice channel.

Johannes Raggam: Javascript integration

This is about Webpack module federation.

Goals:

  • add-on without recompiling
  • No or negligable code duplication, small bundle size

Hard to achieve with Plone 5 and RequireJS.

New concept: Webpack module federation, or actually just module federation, as it is not tied to webpack.

  • separate bundles
  • dependencies between each other
  • define shared bundles
  • define exported and remote modules
  • fallback to own dependency

David Glick: Snowfakery

Snowfakery is a tool for generating fake data that has relations between tables. Every row is faked data, but also unique and random, like a snowflake. To tell Snowfakery what data you want to generate, you need to write a Recipe file in YAML.

You may want realistic data in tests, but not production data.

I write a yaml file and tell it how many folders and documents to create. I let snowfakery export this to json. Actually, I have defined a slightly different format for Plone. This outputs a json file that can be used by collective.exportimport.

Snowfakery documentation: https://snowfakery.readthedocs.io/en/docs/

Philip Bauer: One small VS Code trick

For the training this year for the first time I used one editor for both Volto and Classic. This is because of one VS Code trick.

I use the Pylance language server for VS Code, which is fine, but it cannot initially find the Plone code. Simple fix for that: I have a script zopepy in my bin folder. Find it, copy the path, in VS code configuration go to 'Python: Select interpreter', and paste the path. After a few seconds, or maybe reloading an open file, it works, and VS Code finds all packages and modules.

Erico - New Docker images

Plone loves containers. Some have had a love/hate relationship with Docker, but we are over it. Maybe Docker is over it too.

We have a new generation of Plone Docker images:

- ``plone/plone-backend`` 5.2.6 and 6.0.0a1
- ``plone/plone-frontend`` Volto 14.0.0-alpha
- ``plone/plone-zeo`` is still available (5.2)
- ``plone/plone-haproxy``

Use your own Plone image, with an example extra add-on:

FROM plone/plone-backend:6.0.0a1
RUN ./bin/pip install "pas.plugins.authomatic --use-deprecated legacy-resolver"

The use-deprecated option should hopefully not be needed in the near future. Maurits has opened a [PR in pip](https://github.com/pypa/pip/pull/10574) for that.

All these images are based on pip. One point: do not use autoIncludeDependencies in your zcml, as this does not work with pip.

Examples for Docker compose: https://github.com/plone/plone-frontend#using-docker-compose

Thank you to the installers team: Jens, Alin, Silvio, Steve, Kim, Erico.