Matthew Wilkes - Performance for product developers

Matthew Wilkes talks about performance for product developers during the Plone conference 2012.

I do performance and security work at the Code Distillery. Member of security team.

About performance. Plone is pretty fast. Low edit sites are really easy to make fly. It runs fine on a 256 MB VM. plone.app.caching takes about five minutes to configure. Add varnish and it goes really rather quick. Performance when logged in is harder. So site admins and integrators have it rather easy.

But what about product developers? No one of you deliberately writes bad, non-secure, non-performant code.

Do not add things to the catalog if you can help it. The catalog is already really big. An Archetypes object takes up less memory than a brain usually. If you have to, then use an indexer, not just a FieldIndex. This means you do not get a value for content types that do not have the field.

Build your types on dexterity, not Archetypes. Dexterity is the best thing Martin Aspeli has written. If you are doing some more low level things, use an OOBTree.

Keep Lengths of things that you need, instead of iterating over everything just to know how many things are in a list. Find a balance between huge objects and annotation soup. The more annotations you have, the less times you are going to write large objects to the database, but it takes more time to get the complete object with everything in it.

A for loop within a for loop is probably not a good way. Rethink this. Do not forget sets and itertools, they are good.

We have some tools available in Plone.

• Define cache:ruleset in zcml for custom browser views. You can go from, in an example, 79 to 0 milliseconds with a cache ruleset. You can directly assign rules to contexts by interface. You can map to an operation, like strongCaching, or to a ruleset, like plone.content.itemView.
• Hey, integrators: get write access to the Plone collective today and add rulesets to your favorite products. I will bring "I cleaned up your mess" stickers to Brasilia.

Edge Side Includes (ESI). Varnish is your friend. It lets integrators do page composition, but it is easy to leak data, so you need to be careful. With ESI you can arrange that the basic content of a page gets cached and the few spots that are not so static (say the navigation portlet), are included dynamically. In the cache rules you can set ETags to roles|lastModified, but ETags can be faked, so a malicious user can fake a Manager role. So integrators will need to segment their cache manually, which is hard work. But it can be done, or maybe it is not a problem on your site. It is a lot easier if add-ons are written with this in mind. So plan your templates for eventual ESIs. Using many small templates is appropriate, this lets them be cached seperately. If there is a block of mostly static html, make sure the user-specific bit is separated out. It should just take a few minutes.

You must test your caching, also automatically. Get a browser view in a test, then get it again and check that it is the same and Zope has only served it once. If you have plone.app.testing based tests, it is easyish. Create a new layer using a ZServer and start up Varnish. Make some testbrowser calls. Probably about half an hour or an hour to set up if you have the basis set up and are used to it. We need helper classes and methods for writing caching tests. We need actual tests in popular packages so we can point to them as examples. Tests in core Plone would be good too.

The current plone.app.caching rulesets are integrator focused. We need some standard ones for product developers to use. We should create some more standard rules for everyone, instead of everyone writing there own. So these are some sprint topics.

Read the documentation of plone.app.caching and certainly what is shown in the UI in the control panel. It can really help you.

Want help putting this into practice? Contact http://thedistillery.eu/