Erico Andrei: Building a Collaborative News Platform with Plone

Here are the slides.

With some other people I have created pendect.com. The idea was to build a TL;DR (Too Long, Did not Read) platform. Create short cards, linking to long articles. Cards are submitted by our community (Pentributors). Also: for every card, we plant a tree.

Technical requirements:

• Collaborative workflow
• Permission control, who can do what, where and when
• Metadata, categorization
• We are a startup, so we need SEO.
• Open source

We decided for Plone as Content Management System. It has most features out of the box. I know Plone of course. But I considered building a simple API on top of Pyramid. But with Plone we could go to market immediately. It has a proven record with news portals, and also a friendly community with smart people.

We use DBpedia for Metadata and categorization. This works on a Wikipedia dataset. They have a tool Spotlight to detect entities in texts. We use a Sparql connection to DBpedia to query for more information.

We use a bit of everything else: Cloudflare, nginx, varnish, HAProxy, Ansible, Thumbor, Sentry, Mailhun, IFTTT, Zapier, Gravatar. We already survived twelve different surges in traffic from Reddit. We have tens of thousands of users. We use Thumbor for image scales. Actually easy to integrate in Plone.

Our toolset and add-ons:

• We have development speed thanks to some friends, like Python 3 with F-strings, Type hints (I put them in from the start, helping me to think about my code), data classes (better than returning dictionaries everywhere). Also: black, isort, flake8, PyCharm.
• We do not use many Plone add-ons: collective.z3cform.datagridfield for table-like content, collective.sentry to talk with Sentry, contentrules.slack to send messages to Slack, souper.plone to store many small data records/

Content types:

• Default folders, documents, images, collections
• Category: collection behavior with sub-objects
• Card: similar to news item, but more categories

Adapting Plone:

• Browser views: dashboard became "My Feed", Author page became the profile
• Some content rules

New features:

• Aggregator pages for tags, people, locations, organizations.
• Development with souper.plone: you can follow categories, tags, people, locations, organizations, and Pentributors

Sync to archive.org. Takes too long, so this needed to be asynchronous. Also other external services, like translating.

Lessons learned:

• Always create upgrade steps. Be aware of existing registry configurations: use purge=false.
• Plone training materials are the de factory documentation for Plone.
• I hate the current resource registries.
• Plone lacks a simple and working async/delayed solution.
• Webp images are not for all. Be careful with caching them.

Future:

• Translation and auto summary to all Pentributors.
• Card threads, similar to Twitter threads, using content relations.
• Use ElasticSearch.
• Move to RelStorage.
• Plone as headless CMS, with Volto and mobile applications
• Move away from the default user folder, it is getting slower already.