Nejc Zupan talks about plone.api during the Plone conference 2012.

This is a story about how Plone development went to a bit of a dark place and is now coming into the light again.

http://eestec.net is a website for more than 2500 European engineering students. Running in Plone 3. Two of those engineers stayed in Plone. The main reason was that we switched to Plone 3 during development. It got harder to train people. Plone 4 has improvements, but there are still problems.

You always have to remember from where you should import that function or class. Copying or moving objects is hard to remember how to do programmatically. Sample code is harder to read because of this.

So the idea of plone.api was born during the Plone Konferenz in Munich early this year. Work was done during the Belgium Beer Sprint. Eestec students new to Plone were working on rewriting the site in Plone 4. They gave feedback about what should become part of plone.api.

Inspiration came from:

• PEP20 (import this)
• PEP8
• Pareto principle: we document and support the 20 percent of tasks that you use 80 percent of the time.
• Documentation first.

Getting the site root: api.portal.get().

Moving an item: api.content.move(source=..., target=...).

It is documented. We document first. We write narrative documentation. The online documentation is pulled from the code and the comments in the code.

It is tested. We have about 95 percent test coverage. Narrative documentation is included in the tests. We have Continuous Integration set up with Travis CI.

Import and usage style:

from plone import api
portal = api.portal.get()
user = api.user.create(username='bob')

We have api.content, api.user, api.group.

It is used in the wild. There is tutorial.todoapp as a tutorial application you can try out. eestec.portal uses it. We have been using it in production for about a month on a portal with 100k objects and 300 users. No problems, although it is alpha code.

Where should you use it? In integration code? Yes. In add-on code? Likely. In core code? No.

In the future we should get api.role:

with api.role('Manager'):
    # Do something as manager

• api.env.debug_mode(), api.env.plone_version()
• api.system for running upgrades and things like that

We could create plone.jsonapi to package code around json.

We need lots of help.

• We want more translations of the documentation.
• Multilingual Sphinx on readthedocs.org?
• Show us your home-baked maintenance scripts.
• Sprint on this during the conference sprints.

Utopia would be that the Plone learning curve gets lower because of this and we can put up a small cheat sheet for new (or old) developers.

Fight for your rights as a developer!

Download plone.api and give it a spin on your development environment.

Code: https://github.com/plone/plone.api

Documentation: https://readthedocs.org/docs/ploneapi/en/latest/index.html

Steve and Andy: I think we should actually use this in the core code too. At least if a new package is using plone.api and this package gets moved into the core, then we should not require a rewrite without plone.api before it is accepted into the core.

Lightning talks on Wednesday during the Plone conference 2012.

This is an open space, actually two, during the Plone conference 2012.

Encolpe Degoute talks about ways to investigate for the future of Plone, during the Plone conference 2012. Let's look at Plone on various levels, from low to high, and see what can be changed.

Plone uses Python and C code.

What is the next Python to use? Options could be:

• CPython 2.7
• CPython 3.3
• Pypy
• Stackless Python

I think we should head for CPython 3.3 or Stackless Python.

Pypy does not (currently) use distutils, which is a big problem for Plone.

Does someone want to remove the GIL, Global Interpreter Lock? It would be good for performance, allowing you to use multiple CPUs, but it is hard to do.

For the network layer, Twisted is one of the worst frameworks to use. But it is written in Zope. So we should improve that, or switch. Synchronous versus Asynchronous. In Django you can switch the code that handles the network layer, for example Gunicorn. It is a big work, to be done in Zope 4 or Bluebream.

For storage we use the ZODB, storing objects as Python pickles. It uses a simple file (Data.fs) or a database (using RelStorage). A problem is that all transactions are stored in the ZODB, also when they are not useful anymore, which is why you need to pack the ZODB regularly.

Why Pickle? It is a Python specific format that can store binary objects. For more speed you need cPickle. If you want to replace it, you could use:

• JSON. It lacks schema validation. There is some variation in binary format.
• XML. You can use this to exchange messages, but it can be used as a storage format too. That should fit all the needs.

What can we improve in the ZODB?

• For B-Trees we can use stratified B-Trees, which is interesting for bigger sites.
• Better catalog indexes: use relational database power with RelStorage. The main developer of RelStorage does not want this in the core, it should be an extra package.
• Pack the ZODB: have a sub process that automatically cleans up old transactions.

Can we replace the ZODB? There are some production ready alternative databases, but they cannot completely replace the ZODB:

• Cassandra
• MongoDB

What about the basic framework, what can we improve there?

Have you looked with ZopeProfiler at a Plone site? Lots of things are called lots of times.

For a function of five lines, the initializing of the function normally takes more times than the execution. So we may want to use less functions. We need to use more Zope functions directly.

We need more asynchronouw services in Plone:

• Mail service
• transaction cleanup
• REST api

Can we switch to Bluebream and the ZTK instead of Zope 2? Are we ready for Zope 4?

For Plone, should we keep the CMF or push it in the ZTK? It was agreed that it was useful, but nobody wanted to do the work.

Which visual editor should we use? There are several options:

• TinyMCE
• FCKeditor
• Aloah
• VIE and Create.js

Main problem of all of them: integration.

In Plone we can use dexterity. We could switch everything to this instead of Archetypes. There is no toolkit currently to migrate content types and content objects from Archetypes to dexterity. That would be good to work on.

Have som file format integration. Plone should be able to create a documents from its internal format, like Open Document and PDF.

What about semantics? For example Schema.org. How can we include microformats in a Plone site? Did somebody work with Virtuoso?

So: lots of things that we could work on!

Asko Soukka and Jukka Ojaniemi talk about Plone, RabbitMQ and messaging that just works, during the Plone conference 2012.

Why message queues? We needed to run asynchronous tasks. Difficult to do well within the ZODB, once you are talking about a decent amount of requests. So we use something external.

AMQP is like a postbox: Advanced Message Queue Protocol.

• A message consists of a label and payload.
• Producer is a program which sends messages to exchange.
• Consumer is a program which waits to receive the messages.
• Exchange sits between producer and consumer
• Bindings are routes from exchange to a queue.
• Queues store messages.

You can build several messaging systems, like publication/subscription and RPC (client to server and then back to client).

RabbitMQ implements AMQP. Written in Erlang. Industry tested and reliable.

Sounds difficult? Well, messaging in Plone can be easy.

Example with collective.zamqp for publish and subscribe announcement. Get a utility, register a producer and publish a message in basically three lines. Handling incoming messages: register an event handler for IMessageArrivedEvent.

Beaker comes with an asynchronous connection utility. Zope uses a similar technique under the hood, which handles transactions for us. To use collective.zamqp you need to define some additional zope configuration in your buildout.

Code and demos: https://github.com/datakurre/collective.zamqp https://github.com/datakurre/collective.zamqpdemos https://github.com/datakurre/chatbehavior

Other relevant links: https://www.rabbitmq.com/getstarted.html https://www.amqp.org/about/examples

Slides of this talk: http://goo.gl/RDYZc