Weblog
No ICT without mineral extraction
Keynote on Thursday at the Plone Conference 2022 by Johan Yans and Olivier Vergeynst.
Mining/extraction of mineral resources (for ICT and other usages) logically involves geological, technological and engineering concerns. However, it also deals with numerous other essential aspects, although less discussed by medias/citizen/experts, such as economy (“circular economy”), sociology (“social acceptability” or “perception of mining by the citizen”), ethics (“artisanal mining”), geopolitics (“strategic/critical commodities”, “national strategies”), environment (“waste”, “post-mining”), teaching (including popularization), land management (zones dedicated to extraction), philosophy (“needs” of ICT for Humans), history (current impacts of former supplies), law (how to legislate/regulate?)…
With his experience in various countries/projects, Johan Yans will expose some of these aspects, with a particular focus on the supply of geological resources for ICT needs. Olivier Vergeynst will then introduce the basics of Sustainable IT, highlighting some best practices to start reducing your impact as ICT professional or as simple user of digital devices and services.
Copper and gold prices were already high before the war in Ukraine. True for lots of minerals. Not for gas, that was influenced a lot by the war.
Each year we extract 92 billion tons of minerals each year. That is one swimming pool every two seconds. 60 percent of cobalt in the world is extracted from the DRC, Congo. A lot is done artisan ally, by hand.
The list of critical raw materials is rising, so materials that are important, in our view from Europe, but that have supply risks. Europe needs to import a lot.
International Resource Panel says we need to extract less minerals if we want to keep below 2 degrees warmup in 2060. But the expectation is that we will extract two times as much.
We can extract minerals in Belgium, there is a lot of zinc and lead. Of course we say: not in my back yard! There are a lot of downsides.
The ICT industry accounts for more green house gases than civil aviation. And between 60 and 90 percent of our equipment ends up in land fills, instead of being recycled.
We have an Institute for Sustainable IT. Impact is mostly for user equipment, that takes the most energy and minerals. Manufacturing an iPhone 11 taks 72 kg of carbon emissions, which is about 80 percent of the emissions for its total life cycle.
For data usage, the main impact comes from videos. Online video is 60 percent, 20% other videos, like meetings, 20 % other content. If you can choose a lower transmission rate on Netflix, that helps.
Sustainable IT is good for business. Get ready ahead of regulations, and you will have a chance to catch up with your competitors and go beyond them. Risk management: unsustainability is a vulnerability.
Choose for eco-design of digital services. First question: what can I avoid developing? Write efficient code. Reduce bandwidth. Divide infra needs. Extend the lifetime of your product: if your product can still run on older phones, you avoid the need for your users to buy a new phone. More clients. Business growth.
Demonstrate your engagement. You can become a member. Or just sign.
A few tips:
- Buy less equipment
- Reduce video streaming
- Limit use of emails, avoid sending it, avoid sending large attachments.
- Use https://www.printwhatyoulike.com/
Wednesday Lightning Talks
Lightning talks at the Plone Conference 2022 in Namur, Belgium.
Vincent Paulis: From IT to brewery
I grew up in Spa, studied IT in Liege. I like comic strips. My company is called Bridge IT. A few years ago I adapted my company to officially sell beer. Brasserie du Point, le Gout du Plaisir. If you would stop working in IT, what would you do? I followed a micro brewery training. And I have a comic from Renaud, a well known comic artist, on my beer.
Andreas Jung: Typesense integration for Plone
A new full text solution for Plone 6.
A long time ago I created Products.TextIndexNG3, but after a long time it died. I was searching for a decent replacement. I came across Typesense, a few fault-tolerant, full text search solution. Faceted search.
The browser gets javascript from Plone and then talks directly to the Typesense server, so it is very fast.
Every widget is configured through javascript, very customisable.
You can deploy it on-premise or in a private cloud, either as single binary or as Dicker,.
Code: https://github.com/zopyx/zopyx.typesense
Mikel Larreategi: volto-react-table-widget
We had a use case for a kind of datagridfield. Standard stuff worked okay, but hat if the user needs more than 1000 rows? The browser dies.
How about react-table? Really fast. We integrated it into Volto.
For bonus points: CSV export import.
See https://github.com/eea/volto-react-table-widget
Erico Andrei: collective.elasticsearch
Developed originally by Nathan van Gheem. The Brazilian electoral justice site uses this. We have been updating it to Python 3. Also integrates with Volto.Simple to configure. Specify which indexes you want to use for Elasticsearch, press a button, and it will migrate. You can even delete the indexes from the portal catalog.
See https://github.com/collective/collective.elasticsearch
Fred van Dijk: Improv wisdom
Useful self help for your personal life. And how to participate in the Plone Community.
Improv wisdom is a book from 2005 by Patricia Plan Madson. See http://improvwisdom.com/
Improvisation theater: earlier in life I thought: how can this possibly be fun?
I realised today that this is also an interesting take on how the Plone community works.
Ideas from the book:
- Say yes. Be more adventurous. Don't say no by default.
- Don't prepare. Give up planning.
- Just show up. Just being somewhere enables things.
- Start anywhere.
- Be average. Dare to be dull. Be nothing special.
- Read the book.
Philip Bauer: What's awesome?
A community-curated list of awesome Plone add-ons: https://github.com/collective/awesome-plone
A curated list of things that make Volto awesome: https://github.com/collective/awesome-volto
We need you to help with this list: create an issue or pull request to add packages to these lists, or to remove no longer relevant ones.
It is fine to add your own, you are good.
Jon Pentland: Volto use case
NSW Government released a design system In 2018, now third version, we updated it. Result is 29 new Volto components. Hundreds of variations.
Lessons learned:
- Time consuming to convert extern html/css into Volto components
- Lots of shadows of core were needed, which makes it hard to upgrade.
- Other than that, upgrading Volto is pretty painless.
- We have same idea as https://github.com/collective/collective.patchwatcher
See https://github.com/pretagov/nsw-design-system-plone6
Victor Fernandez De Alba: Auto login
There was some problem that I created a fix for as Chrome extension. Click it and you are logged in. Also works in Firefox.
See https://github.com/collective/plone6-autologin-extension
Jens Klein: pipx
When you install an executable Python based tool, for example cookie cutter, this pollutes your virtual environment. pipx creates virtual envs on the fly, one for each tool. It updates, unless you tell it not to. Just use e.g. pipx run black
. Or pipx run --spec zest.releaser fullrelease
if the command differs from the package name
Flip McFadden: POSKeyError: how not to fix this
This is a serious talk.
You could use backups, but those are for the weak. So you have a POSKeyError. I like to debug things. A lot of people tell you how to do things, for example this smart guy. Don't belief him, that would be boring.
So try stuff. Use RelStorage. Reindex. Get advice from community forums. Step six or so: invent a new word: Kadosectomy. We will remove a bucket from the BTree. Seventh step: manually bisect a BTree and delete the bucket. Remove the BTree. Replace it with a fresh copy. Your problem is now fixed, and you can let someone else fix the AttributeError.
University of Oxford - Plone 4 to Plone 6 - Upgrading the beast
Talk by Philip Bauer, Lukas Zdych, Jay El-Kadhi and Tim Jones at the Plone Conference 2022 in Namur.
Haiku is a software as a service CMS utilising Plone as the engine. Haiku is aimed at research and higher education institutes. With over 140 websites at Oxford University using Haiku, we will talk about some of its key features and the long anticipated upgrade from Plone 4 to Plone 6, covering the good, the bad, and the epic. We will cover the small fish in the logistical pond, as well as the whales under the water.
The University of Oxford lets you do what you want to do. No one tells you. You can hire your nephew to build a website. Not always good for stability.
After a few pilot projects we are at 140 websites running Haiku. Non technical site owners can theme their site.
Now we want to move this beast to Plone 6. With a small team we had fear of the unknown: what dangers did we not know about? So we asked Philip Bauer and Lukas Zdych as external consultants and programmers.
Requirements for the migration: Keep all features and UX. Usually not a good idea, you would use a relaunch, do new things or theming. But here it looked good already, so it was okay to do it this way.
- Content: everything was already Dexterity, so that is modern.
- A ton on code, 2000 Python files, 500 page templates, good quality. The test coverage was not good though, so it was hard to say if a change was good.
- collective.cover was used a lot. I thought of going to Mosaic, but there was already work getting pretty close to Plone 5.2, so we went for it.
Since it was Dexterity already, we chose in-place migration. Now I am not so sure, exportimport would have been good too. We targeted Plone 6 alpha as soon as possible.
Initial task: update application code for Python 3. Lots is automated, but there is still manual work. After a while we got everything working on Python 3, and we could export and import the content, so we decided to change plans: use exportimport.
With exportimport you do not write upgrade steps when you need something special, but you write migration hooks. The project lead to some code and examples added in exportimport. For details on migration code, see the migration best practices training.
Lessons learned:
- collective.cover is actually great.
- collective.exportimport can solve even the toughest migrations, and is faster by magnitudes. You also get earlier results: you can export and import just one content type, or one part of the site. It makes migrations affordable. About 15 procent of the time Philip billed for this project was for the real migration of data, the rest for improvements in the code.
We have context behaviours to enable features on content items.
Comment from Kim: you can replace Oxford with Leuven and give basically the same presentation.
Module federation
Talk by Johannes Raggam at the Plone Conference 2022 in Namur.
Module Federation is a JavaScript technology based on webpack that handles dependency resolution and the loading/importing of modules, even if they come from external, remote bundles.
A bundle is a set of JavaScript modules which provides some functionality and can be run on its own. A typical web project often includes multiple bundles to provide all the necessary JavaScript functionality. In Plone, Module Federation helps us to prevent loading the same dependencies multiple times.
Think of a Plone add-on with a JavaScript bundle which uses Leaflet, Select2 and jQuery. Those dependencies are also used by the Plone core JavaScript bundle "Mockup" and other add-ons. These dependencies can - and should - be shared among the different bundles.
This talk explains the concepts behind Module Federation and how we use it in Plone to optimize loading speed.
Module Federation is a feature that landen in webpack 5. It helps for loading javascript only once, and in smaller chunks, as needed. Recently it was extracted as separate library.
Think of two bundles that both load a version of Patternslib. With Module Federation, we load the highest compatible version, which is specified in package.json.
Module Federation was invented by Zack Jackson in 2018. He wanted multiple small applications to work together without loading code multiple times.
For Plone 6, the whole Mockup stack was rewritten, this was ongoing. But how would we do add-ons? If they just needed jQuery, it should be alright, but how about add-ons that need more. I then found out about Module Federation. Manfred Steyer helped me, he lives close to me and he already spoke about Angular at the Plone conference in Barcelona. At the BuschenschankSprint we managed to merge this approach and release Mockup 8. Zack Jackson added an issue in Patternslib with some more possible improvements. It is nice that these people at the core of Module Federation are involved in Patternslib.
To use this in a new add-on, use bobtemplates.plone 6.0b14/15 or higher, or use a template.
Question: Could this help Volto too?
Answer: Yes, that should help. For Classic Plone, the main bundle went from 1.5 MB minified to 300 kB normal. Plus the chunks then, but those are only loaded when needed.
Plone 6 frontend and backend automated release and unified changelog
Talk by Valentina Balan at the Plone Conference 2022 in Namur.
This demo covers how the automated release works on Plone 6 projects ( frontend and backend ) for the github.com/eea Volto add-ons and Python eggs.
On each release on frontend and backend projects, a text is extracted that contains all the relevant information using the changelog of the updated Volto add-ons, Python eggs and the Plone/Volto releases. The unified changelog is then saved on the release on GitHub, and can later be viewed in the control panel from volto-eea-kitkat addon.
The GitHub repositories eea/eea-website-frontend and eea/eea-website-backend will be used in the demo. The automation jobs are running in Jenkins.
How is this done? On the backend you have a Plone site with eea.kitkat and on the front end volto-eea-kitkat. This saves old versions and update date in Plone registry. Plus Volto code.
We have the eea/plone-backend Docker image, based on plone/plone-backend. And then an image per project.
For Python egg releases, we use the GitFlow procedure, using eea.docker.gitflow. This means any commit to master is considered to be fine for a release. We extract the change log text from the commits, and is reviewed before merge.
On the front end we use auto-changelog.
When we release a new version of a Volto add-on, a script goes to all our other add-ons, and edits the package.json to update the version.