This talk is about the ENCODE portal, an encyclopedia of DNA elements.

I want to talk here about a pattern, not really about the specific technologies involved.

I work at the data coordination center, generating and organizing data. We get metadata submissions, store it in a metadata database. It is really a knowledge management system. It could have been built in Plone, but it would not be a great fit. I started from scratch based on Pyramid and ReactJS. Nowadays services more and more have a Javascript UI, where Javascript talks to the backend.

Embrace Javascript. There has been progressive enhancement. Single page web apps really need it. For building a portal, I have been looking at isomorphic web applications. Originally from the Render framework. It is important that pages are loading quickly. The exit rate for visitors goes just as quickly up with the loading time.

Json is the lowest common denominator for data. xml is more flexible, but more complex. In Python it is much easier to use json.

JSON-LD: json link data. Adopted recently by the w3c. It is partly about semantic data, but we are not using that yet.

At first we needed to duplicate the routing information on the server and the client side. JSON-LD allows us to express type information, which avoids the duplication.

You can have JSON-LD like this [in "pseudo json" here, just for the idea, Maurits]:

{
  @context: context/jsonld
  @id: ...
  @type: [biosample, item]
  @submitted_by: {
    @id: /users/lrowe
    @type: [user, item]
    name: Laurence Rowe
    }
}

Defined using JSON Schema, which is an IETF draft. Schema version, field definitions, types, validators. It is an extensible format.

All our normalized data is stored in Postgres as json. JSON-LD has the concept of framing, which objects should be embedded (mapping to a view in Plone terms), possibly doing some elasticsearch. Above it is server rendering (with NodeJs) that is creating html, and sends it to the browser. After the first page load, the browser uses Javascript to talk the the JSON-LD backend directly, instead of via the NodeJS server, letting the ReactJS Javascript do its own rendering.

The browser opens a page. Gets html back. Then it does a query for json data and gets that back and shows it on the page.

Indexing linked data. Updating one item may need a reindex of other items that link to it. We have written code for that. Using elasticsearch as a cache.

Come work with us, we are hiring.

See code at https://github.com/ENCODE-DCC

Watch the video and slides of this talk.

This is a big picture presentation. But don't worry, there will be a demo at the end.

United we stand, divided we fall. Plone is in a rough spot. Let's design and build some solutions. Yes, Wordpress is eating our lunch and our cake, but we should not sit back.

Code: https://github.com/ploneintranet/ploneintranet.suite

I have my own company, Cosent.

The Plone community is good. The backend code is good. The user interface was good, but has fallen back. Good enough is not good enough anymore. We need to do better. Only excellent user experience will win us customers.

When as a Plone company you want to sell Plone as an Intranet, you have a good starting point, but you are missing lots of pieces. You would have to build that yourself and let one customer pay for it. That is not going to happen, or at least it is a hard sale.

In the Plone community, the number of commits per month is rising, and the number of committers per month is also rising. Doing pretty well.

So what is wrong? We need to evolve or we will die, like dinosaurs. Plone Core currently is Web 1.0. We need a Plone Social, web 2.0: read/write, social networking, activity stream, time centric, personal perspectives, bottom-up sharing.

Roadmap: http://cosent.nl/roadmap

In 2010 I had the choice to ditch Plone or fix Plone. I chose to put all my eggs in one basket: Plone. Is Plone Social good enough? No. Look at Jive, that is some serious competition. We do not need to beat such a SAAS product, but we need to be close. Then customers will still prefer the local company that is closer to them and more attuned to their needs.

I think the "spare time" model of development is broken. It has brought us this far, but it is not enough anymore. Stuff is nearly finished on sprints, and then lags too long.

We need a new model. We have the talent, the technology. We can do it. We need to invest in a high quality, out-of-the-box product baseline. Low purchase cost, immediate sales appeal, fast delivery, shared maintenance across the community, new community ethos in collaborating together.

As Plone Intranet Consortium we band together and want to work on this. We had a meeting after the Plone conference last year. Every company there had their own workspace solution, everyone was maintaining their own stack. But they did not have enough resources to generalize it for the community.

Design first. Design is not about eye candy. You start with a decent vision of what your strategy is, what your project is trying to solve.

Roadmap-driven Scrum development. Normal working day, in company time. Legitimate leadership serves the community. The consortium board funds the roadmap. Investment per company: 1000 euro per month, plus one developer day per week. Cash is used to hire people to help with the design. Sprint every Wednesday.

It is 100 percent open source. It is not a product that we will make money on. We will make money on the service we deliver. We want to move the license to the Plone Foundation, we will talk about that.

What we are developing, are add-ons, not a core fork. Plone 5 compatible. Will port to mockup. You are welcome to join the consortium.

Cornelis has made a patternslib-based clickable prototype, that needs no backend to operate.

Demo by Alexander Pilz.

User experience sells. We showed this demo to a client last week and he thought it was an impressive preview of social functions in future Plone.

Roadmap. Phase one: activity streams, team spaces, dashboards, document structures/wiki. Phase two: calendaring, search, news hub.

We are pioneering a new business model for open source.

1. Dream a vision.
2. Combine investment.
3. Design first! Use dedicated designers.
4. Develop and release.
5. (or really 3.1 already) Win customers.

We can boldly go where no one has gone before. We are Plone, we can do anything.

We have an open space tomorrow. Welcome! Sprint on Saturday and Sunday.

Code: https://github.com/ploneintranet/ploneintranet.suite

Watch the video and slides of this talk.

I am owner of Klein & Partner, member of Blue Alliance, Austria. Doing Plone since version 1.0.

Default Plone is not so fast. Scales great horizontally (so adding machines), but there are still bottlenecks, primarily loading stuff from zodb.

First customer Noeku, over 30 Plone sites, hi availability, low to mid budget, self hosted on hardware, VMs. The pipeline is: nginx, varnish, pound, several Plone instances, databases (zodb, mysql, samba).

Second customer Zumtobel, brand specific international product portals, customer extranets, b2b e-shop, hosting on dedicated machines.

Third customer HTU Graz, one Plone site with several subsites (with lineage), lots of students looking at it, so we have a peak load.

Main Plone database is ZEO or PostgreSQL, plus blobstorage (NFS, NAS). Load balancer: haproxy or pound. Caching proxy: varnish (don't use squid please). Webserver: nginx (better not use apache).

The Plone instances and the client connection pool (between Plones and database) can use memcached (maybe multiple), LDAP, other third party services.

If you want to improve things, you must measure it: use munin, everywhere you can. fio is a simple but powerful tool to get measures on your io. Read up on how Linux manager disk/ram. Know your hardware and your VMs (if any).

If you still think documentation is about "how stuff works", you are doing it wrong. There is a lot of software around, people have lots of choices, people being future clients, developers, users, community members.

Also, your future self will thank you. It is good to know what that idiot was thinking half a year ago.

Sometimes you just want to use a four letter word: RTFM. Read the fine manual. But then there has to be a manual. If people don't know Plone, they won't use it.

A year ago, in Brasilia, a plan was born. We were really unhappy with the state of documentation last year. Spread around over the internet.

A robot master, documenter, marketeer and 9 others walk into a bar and... Well, it was an office, well there was a bar involved, but anyway. Sisi Nutt did a lot there. Without her the documentation would not be so far as it is now. It has been tested on live humans (testing on animals is not allowed).

We had a reality check on the Write The Docs conference in Berlin. Good to get input and feedback from other people. We are now testing broken links using continuous integration, which was new to them.

So now we want more, better, nicer, shinier, sweeter. What we are now telling about documenting Plone, you should apply that to your add-ons too. If it is not documented, it is not used.

• Define what the audience is of your documentation, or for a specific part of your documentation. Structure your docs accordingly.
• Soft landings are important. So add introductions. Let people see if this chapter is what they are looking for.
• Get the tone right. Friendly and professional go a long way. Don't scare off users, unless you really want to. Ask for help, especially if English is not your first language.
• Consistency. Don't leave your readers hanging, nor your contributors. Always use the same terminology. Write and follow style guides. Do not make it too complicated. Read the Zen of Python (import this) and apply it to your documentation as well.
• Versioning. Different software versions need different docs. Benefit: you can retire the old. This is what we will do soon. On http://docs.plone.org you will see the Plone 4 docs by default, and can choose a different version.
• Spellcheck is a wonderful invention, please use it.
• We love humor. Please do not do that in documentation. Highly culturally charged. It gets old quickly and docs are meant to last.
• Sarcasm is never a good idea in docs. Nor in irc, for that matter.
• Testing is essential. Test it on somebody who is not an expert in the field. Does your audience get it?

Now for some tools. Search docs right from Sublime. It has integrated search for Chrome, Firefox, DuckDuckGo. Or using Dash (OS X) and Zeal (Windows, Linux).

Quality control. We have robot screenshots. Automated spell check. We add some words to the known words list, which doubles as a Kudo list, for example we have added David Glick to that list, otherwise we get errors from the spell check.