Plone

published Nov 03, 2021

This is here to serve as contents for the atom/rss feed for Plone, also read by planet.plone.org.

Maurits van Rees: 3 authentication add-ons

published Oct 25, 2019

Lightning talk by Maurits van Rees at the Plone Conference 2019 in Ferrara.

Three PAS plugins:

  • collective.denyroles
  • pas.plugins.headers
  • collective.contentgroups

collective.denyroles

Deny access to roles like Manager and Editor

  • Use case: Manager only logs in to edit-domain, not live site.
  • By default deny access to Manager, Editor, etc.
  • env DENY_ROLES=0 to disable
  • or Apache/nginx header X_DONT_CHECK_ROLES
  • Actually not a plugin, but a patch.

Code: https://github.com/collective/collective.denyroles/

pas.plugins.headers

PAS plugin for authentication based on request headers.

  • Use case: Apache/nginx adds SAML headers to requests.

  • Configuration in ZMI or profiles/default/pas.plugins.headers.json:

    {
    "userid_header": "uid",
    "required_headers": ["uid"],
    "roles_header": "roles",
    "allowed_roles": ["student", "teacher"],
    "deny_unauthorized": true,
    "redirect_url": "https://maurits.vanrees.org",
    "memberdata_to_header": [
        "fullname|HEADER_firstname HEADER_lastname"
    ]
}

Code: https://github.com/collective/pas.plugins.headers/

collective.contentgroups

Plone PAS plugin for content as groups.

  • Use case: create content item that works as a group.
  • dexterity behavior
  • No Products.membrane, no Products.remember, no dexterity.membrane.
  • No separate membrane_catalog.
  • Only groups, not users.
  • No multiple inheritance, just AccessControl.users.BasicUser.

Code: https://github.com/collective/collective.contentgroups

Rodrigo Ferreira de Souza: Data migration to Plone 5.2 and Volto

published Oct 25, 2019

Talk by Rodrigo Ferreira de Souza at the Plone Conference 2019 in Ferrara.

At KitConcept we have some websites that need to be deployed in Plone 5.2. We could have used collective.transmogrifier to migrate from 4.3 to 5+. And from 5.1 to 5.2 migrate the ZODB from Python 2.7 to Python 3. Transmogrifier is also an option for that part.

Why would we use transmogrifier? There are many generic pipelines available for common cases. Flexibility to deal with different use cases. And it is actually a brilliant wayto use the iterator design pattern. Every piece of the pipeline changes a small thing, and hands the information over to the next piece.

Clients:

  • Large university site
  • High-profile government client
  • Large research institution client website

Challenge: go to Python 3, Plone 5.2 and Volto. With Volto, they spare a migration from Plone 5 to 6, at least partially. Volto is a way to sell the client a Python 3 upgrade: it is the right time to use Volto.

We use Jenkins to test the migration.

General backend things:

  • Old ATTopics to core Collections. Use collective.jsonify to export it, in a way that already looks more like a Collection.
  • Rich text: migrate to Volto blocks.
  • Post migration: collective.cover needed special handling.

What we polish to enter in Volto land:

  • We use collective folderish types.
  • deal with default pages
  • Convert RichText to Volto DraftJS
  • easily point to old website when content is not imported
  • fix some urls (we want to use resolveuid)
  • simple folders we should turn info page with listing block
  • simple collections we should turn info page with collection block

Question: can you open source the module that handles the RichText to Volto blocks migration?

Answer: yes, it is actually just 38 lines with React in Node. I tried to do it in Python, but could not get it to work. And we can sprint on it.

Open Plone Board Meeting

published Oct 25, 2019

Board Meeting at the Plone Conference 2019 in Ferrara.

See the annual report.

Jen Myers has graciously offered to stay on next year as treasurer.

The Zope Foundation has been added to the Plone Foundation. Still ongoing, complex process.

With the Pylons community, those talks have stalled at the moment. Might happen in the future.

Financially we lost more money than we usually do. There was a trademark conflict that cost a lot more than we thought.

Some money should come from the Zope Foundation by the way.

Results of the vote for the board.

48 valid votes, one invalid. 2 went to the spam folder, but we found them. 30 percent of votes were in paper.

We had a vote by the foundation membership. Voted into the Plone board have been: Victor, Erico, Chrissy, Andy, Paul, Jens, Fulvio.

Meeting closed by Paul.

Thanks to Alexander, Kim and Carol who are leaving the board.

Riccardo Lemmi: Deployment Automation

published Oct 25, 2019

Talk by Riccardo Lemmi at the Plone Conference 2019 in Ferrara.

We wanted to find an easy way to reproduce the installation process. We use:

  • Vagrant
  • Fabric
  • AWS
  • Boto 3 / awscli

Vagrant manages virtual machines and containers, for example with VirtualBox. I use the init command to do some more configuration on a default box.

Then I use Fabric to make an ssh connection to the machine and do remote actions. It uses invoke and paramiko for this. You can let it run the same actions on different machines. You can also use it to transfer files to and from the server, or use sudo to restart Apache.

More libraries with Fabric:

  • fabtools to make sure that for example Python is installed, or a user is created.
  • Cuisine: update files. You can also use this tool to ensure packages and users, so parts are very similar to fabtools.

Next as AWS, Amazon Web Services. With this we deploy production and test machines in a simple and replicable way. You can choose to add more CPUs, bigger disks, more memory, etcetara. I use EC2 (elastic compute cloud), EBS (elastic block storage) and EIP (elastic IP) for the most. Snapshots as simple backup tool. Security group rules as a firewall.

I create a machine with Boto 3 or awscli, both available with pip install. Why would I script this? To have "infrastructure as code". When you manually try to replicate a server, you can easily forget things.

Philip Bauer: Migrations! Migrations! Migrations!

published Oct 25, 2019

Talk by Philip Bauer at the Plone Conference 2019 in Ferrara.

I have some upgrade code that you can use, or for some parts copy and adapt for your use case.

Code: https://github.com/collective/collective.migrationhelpers

Upgrade steps:

  • Do imports in the function.
  • Do them conditional, so it does not fail if some package does not exist.
  • Do not let a step fail when run a second time.

You may want to disable LDAP temporarily in an upgrade step.

Get the birds eye view, with some code that reports:

  • how many items are there
  • which portal types
  • how big
  • local roles
  • etcetera
  • How many items are there that need to be replaced or removed, like PloneFormGen or Collage.

Divide and conquer:

  • Deal with one problem at a time.
  • Ignore problems that don't block you. You may try to solve something in the old Plone 4.3 site which is already fixed just by completing the migration to Plone 5.

You can register upgrade steps conditionally in zcml if needed, for example with zcml:condition="installed plone-52".

Make big problems small:

  • Write something that removes 98 percent of your content, for testing. Keep the structure in place though: Folders may have portlets or local roles that give problems that you want to know of.
  • Do not migrate blobs. The PDF that lives on the filesystem will not change. Move the blobstorage out of the way, use experimental.gracefulblobmissing during migration, and move blobstorage in again.
  • Copy the Data.fs.index too, if it is a really big site.

Forget the past:

  • remove all revisions
  • maybe manually use collective.revisionmanager for this.
  • pack the database with zero days (bin/zeopack keeps the last day by default).
  • Remove no longer needed portlets.
  • You can use uninstall and upgrade profiles. I like having the upgrade code in Python, but sometimes a profile is much easier.
  • Remove utilities and adapters from add-ons that will be removed.
  • Sometimes you cannot easily remove a package. You can make an alias for it. plone.app.upgrade has code for this, for example to not crash on an old site that still expects the kupu editor package somewhere.

Migrating LinguaPlone:

  • Content editors may have done crazy things, combining folders and content from different languages.
  • We have code to migrate this to plone.app.multilingual in migration.plonehelpers.

Update to Plone 5.2:

  • Use Python 2.7 at first.
  • Include Archetypes if needed for migration.
  • Run the migration to migrate to dexterity.

Archetypes to dexterity:

  • Use the methods from pac_migration from plone.app.contenttypes.
  • Start with the containers/folders.
  • Do one type at a time.
  • Especially for blob-like items there are options to speed this up, like disabling updating SearcheableText.
  • You can write custom migrators, which might just need a few lines.

Alternative: inplace migrator from ftw.upgrade. Interesting for large folders. Might be a nice PR to get this part into core.

To Python 3:

  • We need to remove some archetypes tools.
  • Then remove the Archetypes eggs from buildout and use Python 3.
  • Run the zodbupdate script.
  • When using RelStorage, you may want to switch to filestorage temporarily.
  • Read the documentation please, especially: - upgrade to Python 3 - upgrade zodb to Python 3

Why do it this way? I don't want every Plone company to have their own migration code stashed away somewhere. Please contribute to the core.

Upgrade to Plone 6? Install plone.restapi, use Volto, done.