Why another migration tool for Plone? plone.app.contenttypes already has migration for Archetypes to Dexterity. This atct_migrator is in most cases the right tool. Watch and listen to Philip Bauer's talk (tomorrow) about this: How to upgrade sites to Plone 5.

We have Plone 3.3.5, more than 20,000 items. Custom content types, with archetypes.schemaextender. Merging of content from different sites, cluster of five sites sharing a mounted zeo. This live implementation was actually uninstallable anywhere else, so we could not get a good test migration using the default ways.

interaktiv.migrator gets source as json and pushes it to the target.

Pro:

• import content directly to a newer Plone version.
• migrate our own content types without mapping fields that were already matching
• archetypes.schemaextender data worked fine with this
• This could be done from a live system.
• You get a new and clean database.

Contra:

• Only content is migrated, so settings and users are lost. So you should do something else to get this part of your setup back.
• References have to be migrated at a second run.

I wrote a transmogrifier based workflow to export and import the data.

In the old site there is a configuration form for the migrator, with a target url and for security reasons an api key that must match between source and target. On the target site, you set mappings for types, fields, views.

Live demo, migrating some content, with portlets and local roles on the Sharing tab.

After migration some things still need to be done:

• Go to portal_workflow and update the security settings (button).
• Go to portal_catalog and clear and rebuild the catalog (button).

You can write code to export only some portal_types, or only published items, or only content from one folder.

• interaktiv.migrator is in beta.
• Target audience: developers.
• Does not work yet for collections.
• Only migrates from Archetypes to Dexterity.
• Content gets a different uid currently.
• Needs better documentation, testing, refactoring.

Code:

• https://github.com/jmevissen/interaktiv.migrator
• https://github.com/jmevissen/interaktiv.migrator_p3

Questions:

• Are dates migrated correctly, so creation and modification date? No, but for example we added this for the Creators field and this took only half an hour to implement.
• Which Plone versions? Tested on migrating from Plone 3 or 4 to 5.
• plone.app.contenttypes migrates schemaextender, dates, uids, etcetera. Did you report any issues there? No.

I am Tiberiu Ichim from Rumania. I have 12+ years experience with Plone and 2+ years with Kotti. I started small projects with Kotti. I am going to show some of my experience with it. How can you use it, benefit from it? When should you favor it over Plone or some other solution?

Kotti was started in 2011 at a Plone conference. The developers have Plone as common history, so Kotti will show this heritage inside of it. You will see familiar concepts, even though the platform is completely different. It is already stable for many years, you can just use it.

License: BSD. About 9,000 downloads per month, so still small, but an active, healthy and welcoming community.

Kotti: fast, light, flexible, well tested. That is its philosophy.

Create an add-on for Kotti:

$ bin/pip install Kotti
$ bin/pcreate -s kotti kotti_myaddon
...
$ cd kotti_myaddon
$ ../bin/pserve development.ini

This scaffold of a new add-on is tested in the core Kotti tests, so you know it will create a correct add-on.

The Node base class gives you traversal, as you are used to in Plone. It uses the adjacency list pattern, with parent and children. From a single root Node you can traverse a node tree. Access children with the dictionary protocol.

Class hierarchy is broken up among dependent tables. Each content class has its own SQL table. Transactions work, so changes are rolled back when a transaction is aborted on an error.

View definitions are done with a decorator.

You can use Zope page templates, specifically Chameleon. In a template you can use api.render_template('kotti:templates/views/tags.pt') to render a template.

deform and colander are used for schema definitions and form validation. Standard add and edit forms are just a few lines to define.

Configuration is done with .ini files. Kotti is a WSGI app.

Choices made by Kotti:

• Pyramid as framework.
• SQLAlchemy as most advanced object relational mapper for Python.
• Bootstrap css.
• Colander and deform (version 2) for forms.
• repoze.workflow, very similar to DCWorkflow
• filedepot for blob storage. You can store them on custom backends: file system, Amazon S3, GridFS, roll your own. It integrates nicely with SQLAlchemy and is transaction aware.
• Bleach, Babel, Alembic, py.test, jquery, etc.

95 percent test coverage of python code, using py.test.

You can use a lot of other pyramid extensions.

It is an alternative for Plone, Django, SubstanceD:

• Plone has a lot more features, many add-ons, TTW development. Heavy on resources, lot of features that you may not need for your site, hard to understand and get into development for newcomers. Many competing technologies: Dexterity / Archetypes, Zope2 / ZCA / Grok, Diazo / browser views / portal_view_customizations / portal_skins, z3c.form / CMFFormController / zope.formlib / plone.autoform / Archetypes. Plone is of course a source of inspiration for Kotti. Kotti is similar to core CMF in scope. With Kotti you have only one global site, instead of possibly multiple sites in one application, so Kotti has no local settings stored in the database, no need for GenericSetup, no need for most portal utilities. Also no need for the portal_catalog, as you can search with SQLAlchemy.
• Django has many add-ons, large community. Also opinionated. Not quite a CMS. Weird for a Zope or Plone developer, especially looking at templates and the ORM.
• SubstanceD: very good pedigree. Made by Pyramid developers. Similar features on the surfaces. Uses the ZODB instead of SQLAlchemy.

Kotti avoids burnout:

• smaller development stack
• starts faster
• faster test runs
• no lengthy builds

If you already know Kotti, it is easier to dive into Plone, compared to starting with Plone first.

If you are a Plone developer: yes, you can use the ZCA and write zcml. There is no undo or versioning yet. Smaller number of extensions, less builtin features. Not tweakable by editors, no TTW development, composable layouts, portlet management. But you could create it.

In the future, Kotti wants to say "lean and mean in all the right ways." Keep the base package light, suitable for everyone. Python 3 support is almost done. Probably undo and versioning using sqlalchemy-continuum. A REST api partially implementing JSONAPI.

See http://kotti.pylonsproject.org and https://github.com/Kotti/kotti and #kotti and the freenode irc channel.

Helpful hints:

• Use headings.
• Do not use "Read more" or "Click here". It will just get said over and over again by screen readers. Append the title to "read more" and throw it off the canvas with css so it is not in the way.
• Use big targets for touch devices.
• Use closed captioning on videos.
• Do not flash people, watch out for people with epilepsy.
• No tables, unless it is really tabular data.
• In the viewport tag, do not have a maximum-scale: let people zoom in so they can read it.

Forms:

• Every single field needs a label. I mean it! Just use a for and an id that match. If you click on the label and the field is selected, then you have got it right. You can move labels out of the way with CSS if you want, they will still be read by screen readers.
• Put special instructions in the label, like how you want the date formatted. Put helptext inside the label. Put errors in there too.
• What about placeholders? They do not work consistently in screen readers. You enter the input field and the placeholder is gone, so it never gets read.
• Do not say "required," but "name is required," much clearer.
• Use form validation alerts and go to the input that is invalid.
• Forms need a fieldset and label.
• Use type="tel" when you need to input a telephone number. Use those new types.

Modal windows:

• Check that Escape can close it.
• Have more than just X as label for the close button.
• Get back to where you were on the page after closing.
• Make sure users can tab around only on the modal.

Testing: browser tools.

• AXE: http://www.deque.com/products/axe/ is at least partically open sourced recently.
• Firefox has the Fireeyes plugin. Integrates with Firebug. Does a pretty good job of warning you about accessibility problems.
• Bookmarklet HTML_codesniffer.
• Colour Contrast Analyser for picking color and background-color.
• Color Blindness Simulator.

Testing: screen readers. Use the best combinations.

• JAWS, costs about a thousand dollars. Use with IE
• NVDA for Firefox
• Voice Over, Mac only, use for Safari.
• Talkback, on Android.

Checklist: http://pauljadam.com/wcag20checklist.html

WAI-ARIA: web accessibility initiative, accessible rich internet interfaces. HTML 5 makes a good bit of this redundant.

You really, really need to test. Make some friends that have these issues.

Plone 5: the editing interface needs some love. Everything else looks great.

Can you get automated reporting? At least the contrast checker can be run in Firefox in CI, in robotframework. We have permission from the author to do that.

Look at these Python packages: pep8, pyflakes, flake8, bobtemplates.plone, plone.recipe.codeanalysis. Sysadmins like to be lazy, or should be lazy: he does not want to get a call at midnight to get back to work putting out a fire, because everything is running smoothly, and anyone can do it because everything is setup correctly.

From one colleague I used to need to check all his work, even after explaining, explaining, explaining. There might be tests, maybe not, maybe failing, non standard code, etcetera.

Instead of spending time on real reviewing, it turned into a fight about how to do things, which style to use. So I stopped complaining. I did it differently: let the machines work for you. Let continuous integration tell you all the things.

Are tests passing? Pull requests on Plone are passed through our Jenkins test server.

Does code adhere to a style guide? Is it automatically checked, also by Jenkins or Travis?

QA is about asking why? Why did you make the changes this way? Why did you create this new buildout part? Step back from the line-to-line review and get the full overview. Maybe the change should be in a new package, or the change should not be done at all.

Side note: NASA's take on software development. See http://www.fastcompany.com/28121/they-write-right-stuff

End user or developer documentation. Maybe you create a new option for a buildout recipe, but nobody knows about it unless you document it. Maybe you write everything in English and not everyone can read it. Does it run in all browsers, did you test that? Maybe there is a broken layout, because you forgot to add an upgrade step. Is a dependency missing?

Side note: for checking layouts, see huxley, screenster, PhantomCSS.

So how do we fix it?

Have a style guide and conventions, some documentation on how your team wants the code to look like, what best practices you should adhere to. The best style guide is a coded guide, an automated check, especially as a commit hook. Use the plone.recipe.codeanalysis[recommended] recipe. Use flake8, not only in this recipe, but look at it yourself, integrate it in your editor, look at all the plugins that have been created for it. Maybe create one of your own, it is not hard.

Use a CI (continuous integration) system, it saves time. It is a newcomer's best friend, or even the seasoned veteran's best friend.

Fill your toolbox with zest.releaser, z3c.dependencychecker, lots more. Look for tools, share tools and experiences that help you adhere to the style guides and best practices. Improve and master the tools you use.

Side note: isort. Tool that automatically sorts your imports: https://pypi.python.org/pypi/isort

Some optional flake8 plugins: flake8-copyright, flake8-exact-pin, flake8-naming, flake8-docstrings, flake8-diff, flake8-tuples.

How about manual Quality Assurance? Checking a responsive design is probably something you need to do by hand, or spend time learning some new tools.

Single, one-time cleanups? Fine. But then also add tools to check that the package stays cleaned up, otherwise you keep spending your time on this issue.

Should a pep8 error end with a CI failure? If you have decided that you want to adhere to pep8: yes!

Before QA, you should do some pre-QA. Use bobtemplates.plone to create new Plone packages.

What kind of things can you check? Tests, coverage, plone.recipe.codeanalysis, state of translations, documentation syntax on PyPI, working documentation references and typos, a proper MANIFEST.in to avoid 'brown bag' releases (missing files), undeclared dependencies, accessibility, check javascript on major browsers and screen sizes, commit messages, no XXX or TODO markers in the code, have a change log entry for each fix or feature.

When do you run them? You can do it manually. Have a pre-commit hook. Before a release, but don't throw this job at the one doing the release, but do it yourself as initial committer. A per distribution CI job.

Some ideas. Keep the noise level low and targeted: if the whole team needs to look at a ten megabyte CI report, that is not nice. When you work on pull request branches, you can use git commit --amend and git rebase to restrict the change to a few or even just one single commit. Integrate checks in your editor. Maybe goal for Plone 5.1 to have zero code analysis failures, see http://jenkins.plone.org/job/code-analysis. Check the dependencies of packages. Zuul as project gating system: http://docs.openstack.org/infra/zuul/. Moving to Python3, will that ever be possible without best practices? Can a newcomer release Plone 6, or does it take one person who knows all the small details?

Final thoughts:

• No bike shedding, just use linters.
• No manual steps, let CI warn.
• Always room for improvement, fill your toolbox.
• Always have an eye for quality on every action.

Question: is setting up Jenkins difficult? You can look at https://github.com/plone/jenkins.plone.org which is a buildout, with nothing really specific for core Plone, you can use it for yourself. When I started using it at Der Freitag, it was basically half a day of work to setup.

How to enforce the style guide for every single package? Setup a Jenkins job or step to run this check. See the Github repo above.

Plone had its 14th birthday last week. Lovingly cared for, with the weight of experience behind it.

First of all, thank you. A lot of work and love has been put into Plone 5. We have built something truly special.

Plone 5 has an all new design. A toolbar on the left for editors, with basically everything you used to be able to do from the edit bar. It lives outside of the theme, so no more problems with a theme that would mess up your edit experience.

Folder contents, with for example bulk adding of tags or changing the state. Upload multiple files.

New TinyMCE, much better than what we had previously. Works responsively, so also on other devices.

We redesigned the control panel. Add-ons page has a complete revamp, much nicer and easier to understand. Social media settings.

Theming has been changed. A Diazo theme as base. You can copy the base theme in the UI and edit it right there. A really powerful tool, it makes the initial theming experience much nicer.

Our default content story is Dexterity now. You can still use Archetypes if you want, for now.

We have finally got recurring events rights, like adding an event that recurs each Sunday until the end of the year.

Plone 5 is accessible, meeting lots of accessibility requirements. You will hear more about that in the keynote tomorrow. New multilingual framework, in the core, working for both Dexterity and Archetypes. So: all the content, all the time, for all the people.

There is lots of room for extending Plone and integrating it into all kinds of other systems.

We have sturdy security, especially with automatic CSRF protection.

Evolution of hackability. In earlier Plones you could work through the web, but this was not considered best practice, because you could not really manage your code nicely. Eric Brehault said: we should build an unhackable core, but add something hackable on top of it. Empower people in all the right places. Customization is a feature. We can use through the web prototyping to rapidly come up with a solution that you can then export.

Annette Lewis is a designer of website on Penn State University. New Plone user. She runs about 40 sites. Base theme and policy products, and the rest is done entirely through the web, with dexterity, diazo, theme editor. "It feels like you can do a lot from the front without touching the web." She now has a two week release cycle for sites.

Mosaic is the successor to the oft-repeated Deco project. Not in core. It allows you to stick content on a page, move it around however you want. Look for plone.app.mosaic. Asko Soukka has been working on this two years straight. In beta. We are close to having it for real. Dexterity plus Diazo plus Mosaic is really powerful.