Fabio Kaiser Rauber: Interlegis Infrastructure

From Brazil. Interlegis is a program to merdernise and integrate the Brazilian Legislative Power. Hosted at a federal data center.

Small team requires more efficient processes. Automation is key to maintain quality. We started with shell scripts, then Puppet, then Docker in Rancher 1.0., then Ranger 2.1-2.3, now Rather 2.5. VMWare vSphere cloud native storage. So we use Kubernetes, more complex, but more powerful.

Helm is the packer manager for Kubernetes, allows templating of Kubernetes yaml files. You get Helm Charts.

See https://interlegis.leg.be.

Guido Stevens

In Ferrara we hatched a plan for world domination with Quaive. First step: The city of Namur will deploy a Quaive intranet next week.

Sorry I was not at the conference: I got married a few weeks ago.

Nicola Zambello: Introducing plone-remix

Working to minimalise the impact of IT on the environment. Remix is a full stack javascript web framework based on react-router. Everything is a form. #usetheplatform.

Now we have plone-remix. SSF-first with partial hydration. I need something light weight. plone-remix is a Plone front end with TypeScript, css framework agnostic. A catch-all route for contents, i18n, it is a Plone Rest API client

See https://github.com/raw-material/plone-remix and demo: https://github.com/nzambello/plone-remix-demo

Franco Pellegrini: Python blockchain conference

Mikko Ohtamaa asked me to show you this video. He organises the first Python blockchain conference. It is remote and free

See https://www.pychain.org/

collective.contentsections

Default page content type is replaced by new one where you can add text sections. Demo.

Steve Piercy: Documentation

Git clone https://github.com/plone/documentation/, do make install and make livehtml and you have a live preview on your computer while you are improving the documentation.

The search is much nicer, you can see in which context results are, and you can filter.

We hope to have a Dark Theme available soon.

Jens Klein: Plone mental model

See the Plone core development docs.

You have Products.CMFPlone at the top, then the core, then plone.base, and then other base packages. On top of CMFPlone you have for example restapi, working copy support. With this model in mind, you can start thinking about what really needs to be in core.

Jon Pentland: volto-plone-reloader

This makes plone.reload available via Volto.

See https://github.com/instification/volto-plone-reloader

Joël Lambilotte: François et Djoseph. And IMIO.

This is a statue near this building. There is a snail in a box, and they are afraid it gets out, because Namur people are called slow.

IMIO is a public company with 5 million euros revenue. 15% growth, 50 employees.

eDemocracy and FLOSS appeals to municipalities. Decisions of the public government are now automatically published. Unique project in Europe.

Neyts Zupan: Nix

How do I install Python? Elm? I need it to develop on https://paretosecurity.com/. cd into the directory, boom, I have the correct versions. Also for black, gecko driver, etc. The magic is Nix.

Maurits van Rees: Plone future

I work at Zest in the Netherlands, but live in Belgium now. I am starting my own company. No website yet, but you can mail me at maurits@py76.be. I will have no clients at the beginning, so I will have lots of time for core Plone at first. So Plone 6 will be finished within a week, and then I will get bored. So I have some ideas for the Plone future that I could work on. Maybe you could help during the sprints.

• Several password policies. See https://github.com/collective/Products.PasswordStrength/issues/1 and https://github.com/plone/Products.CMFPlone/issues/3370
• Policies for CMFEditions.
• revisionmanager: allow purging all revisions older than 1 year
• Image transform chain. See my focal points talk last year.
• Make the code more Pythonic, use the walrus operator, etcetera.
• More separation between backend and frontend, also in Classic.
• Can Classic UI call the Rest API more?
• Move all Classic UI code (p.a.layout/portlets/etc) into plone.classicui "mono repo".
• Only load GS in zcml when plone.app.upgrade is available. Or some flag.
• Toolbar by default at the top
• Store the catalog in Postgres. Add-on. One per Plone site. <plone-site-id>_catalog . Integrate with querystring operators? NickJS (from Rob) does this. Replace the @search endpoint?
• Get rid of relation catalog and intids, use a utility comparable to plone.app.redirection. Migration. Run side by side for comparison?
• Unify viewlets and portlets into slots / blocks.
• Auto save when editing a page. Or a block. Or only in TinyMCE or Slate.

Eric Brehault: Add GSOC ideas

We have lots of ideas for cool things in Plone. So add ideas for Google Summer of Code. Do not be lazy, file an idea, go for a mentorship.

Mikel Larreategi: plone i18n

Let's play a game with the language Kaixo. Kaixo means hello. Agur means Bye. Long string of word: "next year in our country". We come from the Basque country between Spain and France. Welcome in October/November next year.

Kim: Thank you

Thank you organisers from IMIO and Affinitic, and the city of Namur. This has been three years in the making. It was wonderful to meet each other after a long, long time.

mxdev is a utility that makes it easy to work with Python projects containing lots of packages, of which you only want to develop some. It works on top of stable constraints and uses the power of pip and git.

You learn

• why we need it/ what problems it solves,
• how it is used in your day to day work (based on Plone Backend),
• how it compares to zc.buildout
• how to extend it,
• the current state of development and what our future plans are.

The release manager provides us with a pip constraints file with over 300 versions. You pip install Plone with these versions. Works fine. Now try to install a different version of plone.restapi, which is one of the pinned packages. Create an own constraints, interit from the upstream constraints, add new plone.restapi version. Try it: pip does not allow it, because there now are two conflicting version pins.

Now add mxdev to the requirements with a git url, in editable mode. Change something in it, commit.  Run pip install for this again. Boom, my changes are gone, because the package is checked out again, or reset to the given upstream commit.

Manually, you could download the constraints, edit them, and then it would work. Clone checkouts manually, and use filesystem path in the requirements.txt. It works, but you do not want this manually.

So I created a bash file to do this, but then thought, isn't there a Python tool for this? Well, there is Buildout. We love this, but it is unknown in the Python community.

So, I rolled my own solution: mxdev. It is a preprocessor. It reads its own configuration, fetches requirements and constraints files, replace constraints with version overrides, directly clone or update from vcs (git, etc). We have mx.ini for configuration, with version overrides and sections for packages to clone.

Packages and documentation: https://pypi.org/project/mxdev/

You can try it for Plone with cookiecutter-plone-starter.

The average client has no idea what Plone is, and they don’t really care. They won’t trust something they’ve never heard of, and there are plenty of other solutions out there that claim to offer the same things.

Designers and developers, on the other hand, do care about Plone: an elegant, powerful, secure, scalable, flexible solution to clients’ content management dilemmas. However, clients generally don’t care about the technical stuff – they just want to be online. It’s our job to tell them why Plone is the way to go.

In this talk, we share what we’ve learned about selling Plone to people who don’t care over the last 15 years.

When planning the trip here from South Africa, I was looking for a backpack. Why did I buy this one?

1. It has some specific features.
2. It looks good.
3. It feels good.

I actually started at the bottom of the list. The brand is known to care about environment and their workers. Then number two, I like this black one. And only then I looked at the first item om the list.

We work at Juizi: websites worth having. We use Plone, and need to convince clients that Plone is a good choice. Mostly, they don't care.

Why don't people care about Plone? They don't know the name. Similar functionality is offered by other names that are more familiar.

What is Plone? You are here, so you know it. Plone is not so unique: most features are offered by most other CMSes as well. Some features we like: easy user management, content workflow and rules, many different content types. But the client is after a solution. What are benefits to the client? Explain the Plone features in terms of the customer. Will it get me online? Can my website grow? Is my website safe?

How do we sell Plone to people who don't care? Focus on the benefits, like: presence, freedom, safety.

Solve problems, rather than build new things. Clients think they are unique, but try to see the general feature that the really need. Think inside the box that Plone arrives in. Listen to wants and address needs. Never say never. Help the client funnel their budget to where it is needed.

Our team is our unique selling proposition. It sets us apart from anyone else. We don't just say we know how to use Plone. We highlight use cases. Clients stick around for years. A lot of business comes in through word of mouth.

In the backpack I bought, I could see the maker cared about it. His name was even in the label. We care about Plone.

Remember what clients are buying. They buy presence, freedom, and safety. Ensure they get this and they'll buy you.

Audience:

These are calls to your emotions, offering presence, freedom, safety. Same as years earlier we said: "Plone gives you peace of mind." Can we prove that we are better at this than others? You can put up testimonials that highlight these things. Marketing in general is an ongoing conversation. Sales is story telling. Get them to listen, and they fall deeper and deeper in love with Plone.

If you enable your client to nicely and easily tell their story to their clients, then you both win.

Send them to the demo sites to try it out.

[Maurits: idea for my own company that I am starting up: have a few pages or paragraphs with features, and link them to a demo page where they can try it out themselves.]

Find out how to level up recommendations to your website based on machine learning open source Python library scikit-learn. Now that we’ve tried its simple and efficient tools ourselves, we will show you hands-on how you can benefit from them. We developed a useful add-on for both Plone Classic and Plone Volto. Get smart content recommendations by using basic Natural Language Processing to integrate this content recommendation system which is accessible for everybody.

Vectorisation: create a vocabulary of all unique words, a bag of words. With countvectorizer you calculate the amount of appearances of each word. This leads to a vector, think of a line in a graph. Then for an unknown text you can compare its vector with the list of known vectors, and see what the nearest neighbour is, using sklearn.neighbors.

interaktiv.recommendations is a Plone package for Classic and Volto. It provides a behaviour which can be added to the content type of your choice, plus a control panel. Say you add this for the Page content type. Then on each page you can show a list of other pages that are similar, so are recommended for the user to read.

Future plans:

• auto tagging
• dimensionality reduction with part-of-speech-filtering and lemmatisation, which breaks the words down to their base form.

Slow tests suck. They are annoying and slow down you and your team. Hours and hours of engineer time is lost waiting for the CI to finish. I'll go through a number of approaches and principles to help you write fast tests.

I started plone.api way back, now in core. I have organised about twenty sprints. Next up is the Nix(OS) sprint 21 to 25 November in Lanzarote.

I developed an app for security on Mac, see https://paretosecurity.com/. We had tests for this, they were fast, but after a while I checked again and it took 3 minutes for only 300 unit tests. What? We need to bring this down. Long tests can cost money, and at least they cost developer time. If the CI pipeline takes twenty minutes, then you cannot wait for it, and you cannot start anything new in that time. There are a few things you can try.

Try one approach, measure it locally, measure it on CI, merge when it is an improvement, wait a few days before trying the next thing.

Use hyperfine to compare results of commands.

Running tests

Throw money at the problem: make sure your team has fast laptops and that you fast have CI runners. Buy a Mac mini for less than 1000 euros and use it as a fast CI runner in GitHub or GitLab.

Try pytest --collect-only. Ballpark: should take 1 second for 1000 tests. If it takes longer, there is something wrong. You are probably telling it to look in too many directories.

Try pytest --collect-only --noconftest. Does this differ much with the previous result? 

export PYTHONDONTWRITEBYTECODE=1

Usually we know a few tests that are slow. You can mark them as slow, and do not run them by default, only if you run them explicitly. Then do run them on CI.

Try pytest-incremental or pytest-testmon.

Writing tests

Make sure tests are not doing any network I/O, like to GitHub or gravatar. Use pytest-socket to get them to fail, and then fix them by mocking the network access.

Use pyfakefs to use a fake filesystem in memory.

Do all tests need a database? If tests only do simple things, like doing some calculations, do not setup a database. Or maybe not all tables are needed for this test.

Can I create the database only once? At the end of a test, you can truncate tables, so you don't need to recreate them.

Can I populate the database only once?

Parallellisation

The big wins are here, but it is more involved, so start with the others first.

pytest-xdist helps here. Make sure your parallel runs don't influence each other, like writing to the database when another process truncates it.

pytest-split is good for the CI, not for local runs.

Now that your tests are fast, how do you keep them fast? We made BlueRacer.io. Free for per personal use and open source software.

Extra tip: run pytest --lastfailed to only run the tests that failed in the last run.

See https://github.com/zupo/awesome-pytest-speedup for the full list with more information.

Audience: gocept.pytestlayer helps for running Plone tests in pytest.