Plone
This is here to serve as contents for the atom/rss feed for Plone, also read by planet.plone.org.
Nathan van Gheem and Ramon Navarro Bosch: guillotina and asyncio
Talk by Nathan van Gheem and Ramon Navarro Bosch at the Plone Conference 2017 in Barcelona.
At first we had plone.server as name, but it was not really Plone anymore, so we were looking for a new name and come up with guillotine, and then Ramon said we should call it guillotina, the Catalan name for it.
Some background. There are web frameworks like Angular and React. Server rendering frameworks are dying out: the front end just wants to talk to an API. We love Plone, but wanted to be able to use it in high performant situations with modern front ends.
History:
- 18 years ago, Zope and the ZODB were created, an object oriented application and database.
- 16 years ago Plone got built on top of it.
- 7 years ago Pyramid created.
- 2 years ago Plone REST API created
- a bit more than one year ago plone.server/guillotina was born.
Guillotina is an evolution, done in the spirit of Plone. It is not necessarily a replacement for Plone. We are taking lessons learned. We are okay with forking some code to change it to our needs. We are not going to provide everything out of the box. Zope and Plone are inspirations, they are in our heads, like the hierarchical data model. It is not a re-implementation of Plone, it is not plone.restapi compatible. It is an asynchronous REST API service.
We have transactions, with conflict resolution policies that reduce conflict errors and gives better performance. We want to use the best database systems available, we support PostgreSQL and CockroachDB.
Information is organised in trees of objects. We love that in Plone and decided to keep it. Objects are resources, with schema attributes and annotations, OO inheritance and static or dynamic behaviors. The data is serialised as json.
Other features:
- Security: similar to how this is in Zope and Plone, but simplified.
- All code is based on asyncio for network integrations with external indexers, database, caching, services. It is based on aioHTTP.
- We want the installation to be simple. You can do pip install guillotina or docker run guillotina/guillotina.
- You can configure how CORS should be handled.
- It is easy to use web sockets.
- It is perfect for micro services, like handled in lots of containers with kubernetis.
- It is extensible, uses zope.interface and based on zope.component, with utilities and adapters.
- You can use Cookie Cutter to create a new project.
- We have a persistent configuration registry for each container. A container is comparable with a Plone Site inside a Zope database.
- You can point to a static directory for static files and javascript apps.
- You can mount multiple databases at the same time.
- We have open sourced our S3 and GCloud storage packages, and ElasticSearch indexing.
- Automatic API documentation generation using Swagger. With guillotina-swagger you get clickable documentation where you can try everything out.
- Give a container 200 MB of RAM and it will handle hundreds of requests per second.
- Urls are like: /db/container-name/obj1/obj2
- zope.interface is the only package that we kept from Zope/Plone.
- Only Python 3.6 or higher.
- Hopefully we get some reusable components that are also used for the Plone REST API.
Links:
- code: https://github.com/plone/guillotina
- docs: http://guillotina.readthedocs.io/en/latest/
- docker: https://hub.docker.com/r/guillotina/guillotina/
- modules: https://github.com/guillotinaweb
- our Onna company: https://github.com/onna
Devon Bernard: How to design a great API using Flask
Talk by Devon Bernard at the Plone Conference 2017 in Barcelona.
At Enlitic we study lots of medical data using computers. I hope to teach you how to create an api that developers and users love. If you are a backend developer, you can see the front end developers as your user. It should be intuitive, durable, flexible.
Intuitive
Your API should not surprise your users.
Hard coded variables are bad. Keep runtime variables in one central place, so users can find them. Make it so that you do not need to juggle with git to avoid accidentally committing a password. Search github for 'removed password' and you find over 300,000 commits with that commit message... Good: create a Config class, and DevelopmentConfig, and ProductionConfig and handle the differences there.
Use object relational mappers (ORM), instead of writing SQL, as you constantly worry about SQL injections.
How to standardise database migrations? A tool like alembic helps, to create a way to upgrade and downgrade. But please change the alembic file template to start with a date instead of a random hash.
Have a standard database seed, that users can use to get a database with standard content.
Use a requirements.txt file with all the requirements that you need. You can get packages from a git repository too. And you can install packages from the local file system, which is especially handy if you develop two packages at the same time, so you don't need to push changes all the time for package A when you yet know if they actually work with package B.
Is the setup code in your README working? If you can replace this by a single command, that would be better.
Create unit tests. You may hear:
- "But it wastes time." No, you still have to verify your code before you deploy it to production. And you do not just need to verify the new code, also the older code, as it may be affected by your change.
- "But writing it is boring." Manually clicking to verify that it works, is even more boring.
Database flush versus commit. A flush reserves a placeholder spot in the database. A commit saves records in the database. If you explicitly commit, and later something goes wrong, your commit is still in the database; usually you do not want this: either all changes should be written, or none at all. Also: use flush instead of commit when you need to get an auto increment id back: what is the id of the row that will be committed to the database.
Flexible
Create an app factory, so there is one way to instantiate your app. This may also help for avoiding circular imports, like when your API depends on your database code, and the other way around.
Create blueprints. For example create one decorator for registering an admin view, and another for anonymous views. That makes it very clear which view is for what kind of user.
Reliable
How to maximize uptime? Prevent downtime by not shipping bugs: have automated testing, have staging servers
Versioning: have a way to know what the used version is. Keep backwards compatibility. Communicate when you will drop a feature.
Get some analytics about API usage. If one of your front end developers is still calling a deprecated part of the API, detect this and send them an email.
First rule of endpoint design is: be consistent.
Document your API, or it does not exist.
A tool like Postman can be used to show the end points, and is helpful for debugging: paste the exact payload that gave an error on production.
Use Python profiling: what lines of your code take the most time?
Caching: store common responses in memory for quick retrieval.
Find me on Twitter: @devonwbernard.
Eric Steele: State of Plone
Keynote talk by Eric Steele at the Plone Conference 2017 in Barcelona.
Welcome to Barcelona! Great to be here. The Plone Barceloneta theme was made here, guillotina started here.
Plone 5.1 release candidate 1 is out.
Some of the features:
- collective.indexing integrated in core, to speed up indexing operations
- new portal actions control panel, so you can manage these in the Plone UI
- we support Retina (high definition) scales
- auto rotation for images
When: soon.
The stories that preserve our past. I want to talk about stories today. People leave projects, other people replace them, if you are doing it right. Contributors start of doing small things now and then, and then get hooked and start to do more and more, and then it usually goes down, and they may go to another job and another project than Plone. When all is well, others persons then stand up, and the project still continues.
Debian: developer half life of about 7.5 years: after that time, half of the developers are no longer involved. Others step up. Continuous renewal is needed, keeping the project fresh. You run into problems when a new generation does not step up quickly enough. If knowledge isn't shared, it is lost.
Alex Limi: we can have heated discussions in the community online, but usually you have seen the other person on a conference or sprint, and this helps in keeping the discussion healthy.
Antoine de Saint-Exupéry: if you want to build a ship, you don't tell people to get hammers and nails and wood and sail, but you teach people to yearn for the sea.
Blog post Martin Aspeli years ago: Pete and Andy try Plone 4, where he highlights some things that they like and that they miss. Several of these ideas are in current Plone.
Cal Doval was a designer who created a mockup of how the add-ons control panel could work. People saw this and knew that they wanted it. It is now in Plone.
It's time for new stories. For example: the Pastanaga UI, which Timo Stollenwerk will now talk about.
Timo
Pastanaga UI was created by Albert Casada. Pastanaga means carrot, and the logo is a stylised carrot. Albert worked on this in his free time, creating hundreds of icons, etc. I was impressed before I even saw it. Main ideas:
- Simplify. Do not show what the user does not need to see.
- Adaptive user interface. More people are viewing web sites on mobile devices than on desk tops now. You UI should work good on all kinds of devices.
- Focus. I was writing a blog post in my code editor. [Note from Maurits: me too.] We want users to do that in Plone. Most current CMSes use TinyMCE or a similar editor, like Plone does, and we no longer stand out. The most time of our users is probably spent in TinyMCE, so we should make this better again.
Pastanaga is a carrot. Albert said he wanted a carrot on a stick, to entice people into using Plone again.
Another story. Ramon did all the stuff on plone.rest, I saw he wast smart, so I just nodded and created a few tests. I did not have much time to work on it afterwards. Others came and made it work with Archetypes, which we did not initially want, but they improved our initial code to make it flexible, so why not. Then Eric Bréhault came along for plone.restapi. Then people worked on an angular client. So: we told a story, did some work, and others jumped in.
It is hard to maintain stuff on our own. We must become consumers of libraries that are already there, especially javascript libraries. Do not reinvent, but reuse.
For us developers, it is cool to create a version of Plone that works on Zope 4, or Python 3, but clients mostly do not care. They need something else, like the Pastanaga UI.
Make Plone outstanding again.
Eric
It's time for new stories. Start writing them. Thank you.
Keynote Google - Machine Learning APIs for Python Developers
Keynote talk from Google about Machine Learning APIs for Python Developers, at PyGrunn.
See the PyGrunn website for more info about this one-day Python conference in Groningen, The Netherlands.
Lee Boonstra and Dmitriy Novakovskiy give this talk, they work for Google Cloud, one of the gold sponsors of PyGrunn.
Python at Google
Google loves Python. :) It is widely used internally and externally. We are sponsoring conferences. We have open source libraries, like a Google Data Python Client Library, libraries for youtube, app engine, etcetera. We use it for build systems, report generation, log analysis, etc.
How can you use Google Cloud Platform for your app or website? You can deploy at scale. You can embed intelligence empowered by machine learning, we provide multiple pre trained models. You can use serverless data processing and analytics.
Machine learning
Let me explain it in a simple way. You want to teach something to a kid: what is a car, what is a bike? You point at a car or bike and explain what it is called. With machines we shoot in lots of data and they start to see patterns.
- Artificial intelligence: process of building smarter computers
- Machine learning: process of making a computer learn
Machine learning is much easier.
Our CEO: "We no longer build mobile-first applications, but AI-first."
We have a lot of data, better models, and more computing power. That is why machine learning is happening a lot now.
Google created the open source Tensorflow Python framework for machine learning. And we have hardware to match. We have ready to use models for vision, speech, jobs, translation, natural language, video intelligence.
- Vision API: object recognition (landmarks), sentiment on faces, extract text, detect inappropriate content. Disney game: search with your phone for a chair, and we show a dragon on the chair. Point your camera at a house, and you see a price tag.
- Speech API: speech recognition (write out the transcript, support for 80 languages).
- Natural language API: really understand the text that is written, recognise nouns and verbs and sentiment.
- Translation API: realtime sub titles, automatic language translation using more context than earlier versions.
- Beta video intelligence: label detection, enable video search (in which frame did the dog first appear).
Demo
Go to the Google Cloud console and create a free account to play with. You need to enable the APIs that you want to use. Install the command line tools if you want to run it on your local machine. And pip install google-cloud.
We use machine learning for example in GMail to show you a possible answer to send for an email you receive.
Walkthrough of machine learning and TensorFlow
Google Cloud Dataflow. Dataflow is a unified programming model for batch or stream data processing. MapReduce-like operations. Parallel workloads. It is open sourced as Apache Beam, and you can run it on Google Cloud Platform.
You put files in Cloud Storage. Process this in batches, with Python and Dataflow. This uses pre-trained machine learning models. Then store results in BigQuery, and visualize the insights in Data Studio.
Ploneconf Sprint Report Saturday
Sprints on Saturday at the Plone Conference 2016 in Boston.
- Updating add-ons for Plone 4.3 and 5. social.like, FacultyStaffDirectory, contentrules mailtogroup, collective.cover.
- Working on nicer listing of add-ons on Plone.
- Plone 5 toolbar UI improvements, default icons if they are missing, changed manage portlets sidebar with sensible texts, edit all portlets.
- plone.restapi. Long discussion about the framing, options listed as url, building basic Angular app, tutorial for search explaining how to setup al kinds of stuff.
- RestrictedPython to Python 3: lot of work
- Porting away several packages from ZopeTestCase. Looking for new server for Jenkins nodes. Removed lots of old upgrades from plone.appgrade.
- Release team: working on signing PyPI uploads.
- Review Plone documentation, looking how to use Sphinx in more sophisticated way, linking to the source if we mention classes or modules.
- Documentation on Plone support channels is done.
- Pyramid. Start using cookiecutter instead of pcreate to create a new project. Tutorials. Working on deform, colander, and demoes of that.
- Rapido in Mosaic tiles.
- Video and VR, Plone 5 support for c2.app.streamingaws, created template for 360 degrees viewer
- Update Zope dependencies, working at fixing some breakage when using newer versions.
- Resource handling, working on bug with legacy scripts, discussions.
- eea.facetednavigation, fix issues during upgrade from version 9 to 10, fixed batching issue, working on final blocker for release.
- Jasonic api for ZODB, working remotely with Jim.
- plone.app.multilingual, better visibility for selecting translations, inline svg.
- Make icons great again with inline svg instead of fonts.
- bobtemplates.plone theming improvements, separate the theming template from plone_addon, add a fat theme template for TTW, added fat theme buildout template, adding Rapido and other stuff, working on wrapper script.
- Plone marketing, prepared more news items for plone.org, plone.com content changes, marketing ideas for headless CMS, Carol will be interviewing some of you.
- Plone Cleanup, made it so that you can really get Products.CMFPlone without Archetypes, removed zope.formlib from GenericSetup which was the last one that was using it, zope.globalrequest, reviewing work.