I came to know Plone during my Google Summer of Code this year. I never used Plone before, but I started loving it.

My project was about transforming safe html, to do that with lxml now.

Plone is one of the most frequent participants in the Google Summer of Code, which I found a plus. The people on IRC were really helpful during the initial phase. You really helped me a lot! And there was awesome documentation. If you start from the first page, and on and on, then it will be a nightmare, but if you know what you are looking for, it is awesome.

I came across this ticket: https://dev.plone.org/ticket/14929. Proposed by Tom Gross, who had earlier tried it, but ran into many test failures. plone.transform was an earlier try as well, during GSOC'07. We had discussions, about moving it from how CMFDefault did it, to using lxml. Thought about the intelligenttext transform too.

I had difficulties working with Plone. It is not easy for anyone who is new. The code base is vast. The first two months it felt very alien. Difficult for me to figure out where to start. The safe_html of portal_transforms is quite old, and I was not used to those libraries, but I did need to know what this code was actually doing.

So lots of difficulties, but also lots of fun. I learnt how to work in a team, often asking 'can you help me?' I learnt new things about Plone, and how fun it is. Learnt how to write efficient code, and how to document how I did it. Pep8, indentation, etcetera, producing good code. Test driven development, I never tried that before. This is one of the best parts of Plone. I got to interact with cool people. They taught me a lot, including some new beers of Europe.

Main goal: get rid of the transform we were using from CMFDefault.

The new filter. Lots of things required to setup the add-on, generic setup, browser view, control panel, interface, an uninstall profile. We have automatic registration and unregistration, done when installing the add-on. Package is: experimental.safe_html_transform

Code: https://github.com/collective/experimental.safe_html_transform

New releases of Plone were done, and suddenly tests started failing. plone.app.widgets was not pinned and got pulled in, and the latest version was trying to pull in the latest CMFPlone version, which was too high at that point, giving an version conflict during buildout.

We created a new control panel, with separate permission.

The new transform uses lxml. It converts the document into the tree form and parses each node of the tree. It checks if the tags or attributes are safe, strips or removes them if not. lxml is faster than the previous solution from CMFDefault.

We used unit tests to test the transform, with sample html. Also automatic robot tests for the control panel.

Now TinyMCE should use our new transform. It uses getToolByName to get the portal_transforms to get the registered transform by its name: it wants safe_html. So we renamed the transform to safe_html.

One thing left: integration of control panel and script.

I learned test driven development, collaboration with people on a project, also a big project, how to understand errors from the error logs, how to keep logs for huge projects (blogging, task managers, etc), because you will forget what you have done five months ago.

Plans: I started loving Plone, so I look forward to learning more about it. First I will finish this project.

Most of the people in India do not use Plone. In my university no one used it. That is a barrier for picking up such a project from GSOC. More awareness is needed, so people are not stuck with Wordpress and need to learn the basics of Plone still.

I never took a training of Plone, that would have helped, like I did this week.

I will be here for one day of the sprints. I would like to work with you.

I have been involved with ICT accessibility since the mid nineties. I was a developer of the FreeBSD project. I am a Unix server admin, web admin, accessibility advocate, podcaster. And I have used Plone for about five years.

I am totally blind, almost since birth. But I can use a PC and such.

There is no universally accepted definition of accessibility, as far as I know. It depends on the context. For me: information or service can be accessed by 'everyone' with virtually equal costs.

Who is everyone?

• With or without disability.
• Age.
• Situational: if you are driving a car, you cannot pay much attention to visual clues; in a quiet library you should probably not use audio; maybe you temporarily cannot use your hands.
• With different devices, not just PCs.
• Language, not everyone speaks English or Japanese.

So how about the costs of access?

• Can be simply a financial cost.
• Time: without hands, you may need more time for some actions.

You first need to know content is there: I cannot see an image without an alternative text.

Key to accessibility is:

• Make everything machine-readable.
• Help assistive technologies.

What are assistive technologies?

• Software or hardware to assist users with specific needs. For example screen readers. Several operating systems have packages for this installed by default.
• Magnifying part of the screen.
• Alternative input devices. Keyboards with very large keys. Complicated switch interfaces.

Machine readable:

• If you show an image of a character, versus text data, most people will be able to see them both. Machines cannot, unless they have image recognition.
• You can make text visually larger and bolder, or you can explicitly enclose it in an h1 tag.

What needs to be machine readable?

• Content to be presented. Have alternatives for images.
• Roles and states of UI elements. Identify menu items as menu items.
• And everything else if you can.

Besides this:

• Is there enough color contrast?
• Are the characters large enough? I can still 'see' 1-point fonts, but most people cannot.
• Can text be resized by the visitor?
• Make it easier to operate. Timeouts, like you need to finish a form within ten minutes otherwise you are logged out, can be hard for people with a disability on their hands.
• Do not autoplay audio, because then I cannot hear my screen reader anymore, so I cannot even stop it and will just leave the page.

So what specifically can you do? See Rob Porter's presentation: http://maurits.vanrees.org/weblog/archive/2015/10/rob-porter-how-you-can-become-an-accessibility-superhero

From a technical aspect, you are opening up to a broader audience and an increasing variation of access devices.

There may be social or legal aspects. There is an aging population that needs reading glasses. Don't worry about me there. The US has legislation, other countries too.

Why Plone? Possible choices for me were Movable Type, Drupal, Plone. What did I want?

• I should be able to manage it, edit it, and use it.
• Multilingual support.
• I want an inexpensive license.

Movable Type was not a practical choice.

Drupal: I did not want to use PHP. Actually it is doing really well for accessibility.

From a blind programmer's perspective: Python sucks. You have to count the spaces. But still better than PHP's incompatible upgrades and stuff.

Accessibility in Plone:

• Really good up to around Plone 3 days.
• In Plone 4 it was outdated.
• Plone 5: not perfect, but a bright future.

The accessibility info page was well written at first, but was forgotten after a while.

Up to Plone 4.x: Kupu was inaccessible, at least for me. No critical issues with generated content. With third-party products it was a different story.

Plone 5:

• Many outdated techniques have been removed.
• Accessibility page rewritten.
• Recent TinyMCE is completely accessible. Really good. I don't have to disable it anymore.
• I see increasing interest and attention from developers.

Room for improvement:

• Javascript widgets need to integrate WAI-ARIA: use roles and states for assistive technology.
• Some issues with color contrast of default theme.
• Diazo editor is not accessible.
• Need to raise awareness in community.
• Well written documentation for theme editing is needed.

Then Plone 5 could be a good choice.

If you are using assistive technology, give more feedback to the developers please.

Look at the web content accessibility guidelines: http://www.w3.org/TR/WCAG20/

Slides: http://www.accessibility.org/~max/talks/201510-plone.html

Twitter: @mnakane (English), @ma10 (Japanese)

This is part two of a talk I did in the Brazil conference two years ago. Recap:

• Every non-trivial upgrade should be approached as a relaunch. Look at content, site structure, design, underlying packages, different languages to use in the site.
• The primary challenge is not development, but communication and project management. It is not just a new version number.
• Bring time. And space. You need to do all steps multiple times. Get lots of disk space. Keep several copies of the database so you do not have to start over at point 1 again.
• Expect everything to break. Templates, python files, add-ons, etc.
• Don't experiment, document. Write every problem and solution down. It is not done, you have just done it on a test migration on your laptop, you will have to do it later on the live site again.
• Write your code as if your own kids will inherit the code one day. You don't want them to hate you. You need to understand what it does one year from now.
• Write upgrade-steps in your add-ons, so you can just run it and not have to do all kinds of stuff manually.
• Divide and conquer. Once one step works, commit the changes and make a copy so you can worry about the next step without worrying about the old steps.
• Don't try more than two new technologies at once.
• Use the help. Use plone.api. Use ftw.upgrades, several nice helper steps.

Now upgrades from Plone 4 to 5.

Why would you want to migrate? Why would the client want it? Do you simply want to play with shiny new tools? If the site is running just fine, they can stay on Plone 4. Are there any new things in Plone 5 that they really need or want?

• Great responsive design out of the box. But they may already have it in Plone 4.
• Newer TinyMCE? The editor just always has some problems, no matter which Plone version.
• Security? Hard to sell.
• Prototyping can be easier, say for dexterity, but how often to you prototype on a live site?
• New folder contents, new UI: yes, many will like this.

So do not force your client.

Demo of upgrade of small site. Includes optional upgrade from Archetypes to Dexterity content types.

You lose one thing: the content history, so the CMFEditions revisions. That is really hard to do.

Demo worked fine. It can't be that easy, right? When the old site has lots of configuration and installed add-ons, it gets trickier. Many add-ons still need changes.

• LinguaPlone is hard to get rid of. You need to migrate this to plone.app.multilingual 1.x in Plone 4 already.
• Theming is tricky. I got an error when trying rendering anything. I saw I had to install plone.app.theming manually.

Use PDBDebugMode to debug tracebacks.

Please work on problems like this, document problems, fix problems, report them.

Read the upgrade guide and work on it during this sprint: http://docs.plone.org/manage/upgrading/version_specific_migration/p4x_to_p5x_upgrade.html We will upload this to a Google doc so more people can work on it this weekend and get it properly integrated later.

Disable your custom theme before migrating, otherwise it will break. Use the new Barceloneta diazo theme as basis and work from there.

Archetypes content types will still work just fine, maybe some less nice widgets in editing. You should migrate to dexterity.

Move everything to the portal registry. All settings are there. So no longer set something in for example the portal_properties.

Before you work on your custome code:

• Figure out if you still need it, maybe the problem it tried to solve is already gone.
• Use plone.api for anything you write new or that you change.
• Formlib is kind-of gone. Custom forms should be done with z3c.form.
• Portlets are also moved to z3c.form.
• Check any templates that you override, take over changes, or remove the override.
• For custom migrations from Archetypes to Dexterity, look at these views: pac_installer, atct_migrator, migrate_from_atct, custom_migration.

ZODB is pluggable, with lots of options for storage: FS (Data.fs accessed by single zope instance), ZEO, ZRS, demostorage, RelStorage.

RelStorage uses as backend MySQL (gave us problems to setup), Oracle (expensive), PostgreSQL (just right for us).

It can be up to twice as fast, mainly because transactions are faster, so you can have more concurrent writes. The limit is always the transaction conflicts.

PostgreSQL is an active open source project, mature, flexible, fast, standard. Sysadmins can handle it just fine.

You can use RelStorage in history-free storage. With history you can undo transactions in the ZMI. With large sites, history-free makes more sense, because usually so many write are done that mostly anything you want to undone has already been overwritten, so undo will not work.

You can use memcached. You need to poll the database for changes in intervals, with ZEO this happens automatically. You can replicate the database with standard tools. You can migrate from RelStorage to FileStorage and back.

Caching. Different threads and instances can use the same memcached client, saving memory. Use an own memcache per zope client machine: so on that same machine. And do not let this same memcache handle for example LDAP.

You should use blobstorage, so check your code, your content types: you do not want files in either FileStorage or RelStorage. [Note from Maurits: PostgreSQL has large object support, which works, but we switched to a shared filesystem blob storage for a client.]

You can split in edit and read-only clients and use a different poll interval. Use 0.5 to 10 seconds for read-only.

Do have a look at configuring and tuning PostgreSQL.

Now lots of time for questions.

Compared to normal setup, you need to replace the ZEO server by RelStorage, add a memcached server on each machine that has the Zope client. Configure the Zope clients to talk to RelStorage.

We could sprint on it to use more options of Postgres 9, making it even faster.

Original goal maybe ten years ago was that some customers required a specific relational database. Things have changed, NoSQL storage is more common now. But how is it possible that it may still be twice as fast as ZEO, as ZEO is quite simple actually? The portal_catalog is part of it probably. The code is hard, takes time to get into. In ZEO things could be tweaked. But for policy reasons it can still be handy to be able to say to an IT department: just give us a Postgres database.

Transactions are still serialized, not concurrent. Switching from pickled format to JSON may make it easier to use more Postgres features.

Can't we change ZEO so you can have a memcache between the zeo client and zeo server, like for RelStorage? So you need less disk cache and can use the memory more.

Avoid write on read.

Python 3? Does not currenly work as far as I know, but should not be that hard. It is just plain Python. ZODB itself works on Python 3.

We, Plone developers, are architects. We build a sand castle. Then users play with it, love it, and break it. Children play with it and it gets broken. (Watch short video Les Vacances.) Conflict: users love it and want new features, and we despair. So we build a nuclear power plant. Unbreakable, but the users cannot play with that any longer. Well, maybe terrorists can play with it, but they are not are target audience.

Pharmakon: any medicine is also a poison.

Build an inflatable castle instead: it is bouncy (more fun for the children), robust (more fun for us), and terrorists do not like it.

Hackability is a feature. It is not a flow. It must be provided as a tool, something on top. If we lose this, we will lose part of our audience. We should have a hacking tool, not a hackable system. A moderen and pleasant web interface.

And we have it: the ZMI. Well... Using framesets, really?

We have the Plone theming editor. It is already a hacking tool. Non-Plone experts can change the entire theme. The Diazo theory behind it, is: "We write XSLT so you don't have to." In reality I am writing tons of XSLT, because there is a really detailed theme layout and lots of pieces of content that need to be hand-picked into it.

We need on-the-fly content changes with before and after: on-the-fly content insertion, also from 'remote' to insert a small page. We have this working, but it is not merged yet.

We want more. Create our own chunks of content and scripts when you only know Python and javascript, and some simple diazo to include it in your rules. This is what Rapido offers. Rapido is not a new Plomino, it is something different.

Rapido lives in the theme. Editable in the Plone UI or in the source. And then we use Diazo to inject it into Plone.

It is very simple. No zca, no zpt (or any templating language), no jbot, just html and python.

Demo. [Looks quite nice, check the video once it is uploaded.]

We use souper for persistance. You can store on the context, or on the user, or what you want

Is it secure? Python scripts are executed by zope.security.untrustedpython. All the regular security stack is applied, like current user privileges, CSRF policy, PostOnly. Just as secure as Diazo.

Rapido comes with a complete REST api with GET, POST, PUT, DELETE, PATCH. It is a ready-to-use JSON backend.

Rapido is Mosaic compliant, but does not depend on it. We just make a tile available to insert a Rapido element.

Questions?

Yes, you can import and export it and include it in a package or have it as a zip file, so you have it on the file system.

When will it be included in core? Not now. We may talk with the security team.

Debugging? More difficult: you cannot add a pdb, same as in the Python skin scripts. We can make something possible, at least show on which line something goes wrong.

Philip: You still need to know Plone when you access for example the portal root in the code you create. I am not sold on it, just like I was never a fan of .cpy files. You lose debuggability. But yes, it lowers the barrier of entry. I am not the target audience.

There are of course permissions, so you can decide who can use this. And you can use access lists in a yaml file.