Weblog
Nejc Zupan: Turbo-boost your Python development with these cloud services
Talk by Nejc Zupan at the Plone Conference 2014 in Bristol.
Development time is a very limited resource. I think that time should be dedicated to coding, and not do things that can be outsourced.
Why should you listen to me? I like to be on the water, surfing, diving, and I do that a lot of days, much more than previously. So I have less time to do actual coding. I really need to get stuff done quickly, no distractions.
You don't want to tinker with your mail server if you can outsource that to someone else who is probably better at it than you.
Continuous delivery: deliver value for the customer all the time. When something is pushed to master, it is automatically rolled out on production. It makes the developers really attentive to details: it really has to work, not just roughly.
There is a problem: a bug, feature request, anything. You code it, push it to a branch, do a pull request. Another developer looks at it. Travis (continuous integration server) runs the tests. If all is fine, it is pushed to Heroku. This usually happens several times a day.
When you have code running in production, errors will happen. So we use Sentry for errors and Papertrail for logs.
Why is not everybody using Travis? Free for public code on github. Run a test script when changes get pushed. Private repos are supported too; not the cheapest service, but worth it.
Sign in to travis-ci.org with your github account, enable travis on your repo, add a .travis.yml file and you are good to go.
Heroku: cloud application platform. I am using it for Pyramid and Plone deployments. They take care of configuring Apache/nginx, firewall, etcetera. You just focus on your code. See http://heroku.com
In .travis.yml you can tell Travis to deploy to Heroku when all is fine. BTW, you can do the same with releasing to PyPI.
Last year I had a slide on using Fabric to release a Plone project. But now you can do that on Heroku with buildpacks. In a nutshell: bin/detect and bin/compile. It is out there and it is being used. See https://github.com/niteoweb/heroku-buildpack-plone It is using RelStorage. Our company website is hosted on Heroku since a few months.
They handle the database, which stays online. When you deploy, the buildout is run (the compile step) and once it is up and answering requests, the old Heroku instance is brought down. So no downtime.
Heroku is free for basic uses. But with Plone you hit the free database row limit after about 100 content items. After that, it costs about 9 dollars until the next limit that is a lot higher. On the free plan, when you have no traffic for an hour, your app is shut down, and then for Plone it will take about a minute for the next request.
We are missing a Plone 5 button on http://buttons.heroku.com. I am working on it.
Heroku forces you into good practices. If a requests takes longer than a minute, it is killed. So you don't keep that page that is taking longer.
Sentry: cloud error reporting. I used to use mailinglogger in Plone, but not anymore. Benefit of Sentry: it is smarter about sending only one email when there are lots of errors. And you see the stacktrace. Easy to set up in a buildout. See http://getsentry.com
Papertrail: cloud log aggregation. Everything in one interface. See https://papertrailapp.com
For most of these, you can choose to run it yourself, instead of depending on third parties.
Watch the video of this talk.
Timo Stollenwerk: The Beauty and the Beast. Modern Javascript Depelopment with AngularJS and Plone
Talk by Timo Stollenwerk at the Plone Conference 2014 in Bristol.
What is the current state of javascript in Plone, version 4.3?
Javascript was always a needed skill for Plone developers. Progressive enhancement: the site works okay without Javascript and works nicer with it. We introduced overlays and other sugar on top of Plone.
My experience the last years is that I have a small project where I use Plone and do a small bit on top. And larger projects, for which I do not use anything from the standard Plone front-end at all. If the project is large enough, this is sensible.
Lazy loading of parts of the page. You do not want to load a thousand comments at once. That can still be done with jQuery and a bit extra on top of Plone.
Problems:
- Page load is too slow.
- With jQuery you end up with ever more spaghetti code, adding bindings to buttons everywhere.
For a project we made a list of requirements:
- full feature solution
- no best of breed solution where you need to combine three or more frameworks and know them all
- good documentation and books
- What I like about Zope is: it teaches me stuff. PHP did not teach me anything. It just did not work. I wrote crappy applications or I was using a crappy framework. Or both. Then Zope and Plone came along. I would learn new stuff, knowing I was learning really useful stuff. So I wanted a good framework, well thought through.
- A framework that was here to stay for a while.
I won't do a comparison, but we chose Angular. Why?
- Full feature frontend framework.
- Developed in 2009 by Google.
- Google is using it for more than a thousand apps internally. Very popular online and in conferences.
Angular does not try to reinvent the wheel.
- Model View Controller. You have a model in Javascript and bind it to your angular template.
- Two way binding: changing the model changes the input, but also if you change an input in a browser, the model is updated.
- Keep the complex code in your Javascript, not in your template.
- Create directives that contain code for your use case, for example for easy bootstrap integration.
- Dependency injection. You write an Angular script, with a function and the function call has getUsersService in it. Then it gets this service (from a file that is already loaded) and you can use it within that function.
BTW: it is cool to edit a css or js file and have the browser reloaded automatically, with grunt or gulp. Really fast development.
Might be nice to keep the good parts of Plone, like permission management, and do the front end in Angular (or a similar framework), where Plone mostly serves up some json.
See https://github.com/tisto/plone.app.angularjs
My personal opinion is that we need a RESTful json api.
Reasons not to use Javascript:
- You might still have users without javascript.
- Search engine optimization (Google cannot crawl data that is loaded by Angular).
- May not be nice for mobile users or for old browsers.
- You need a lot of testing, because a single error may break your complete javascript.
Let's see what we can come up with as Plone community, with Angular or ReactJS or something else. I certainly recommend that you check out Angular.
Watch the video and slides of this talk.
Watch the screencasts shown in the talk: BrowserSync, Two Way Binding, and plone.app.angularjs
Matt Russell: Optimising local role security
Talk by Matt Russell at the Plone Conference 2014 in Bristol.
This talk is about experimental security indexing.
What is the problem we are trying to solve? Performing operations on the sharing tab can take very long, for large sites. There are complex hierarchies around the problem, the allowedRolesAndUsers index in the portal_catalog. Depending on the number of objects that are affected, a request can timeout or worse.
Local roles can be assigned to any user, stored in __acl_local_roles__ on the context, inherited by default.
For the Plone Intranet Project, we started working on this: Matt Hamilton, Ben Cole, and me.
Various approaches were attempted.
- Using PAS groups to represent groups of permissions or roles. You then assign users to these groups. Can become a lot of groups, that you preferably do not want to see.
- Implementing a show tree as a ZCatalog index: https://github.com/netsight/experimental.localrolesindex Abandoned, because it was too complex at a wrong level.
Current implementation: https://github.com/netsight/experimental.securityindexing
Patches the reindexObjectSecurity method for each content type. A utility stores a shadow tree, mirroring the live site. Stores a hash of the local roles. Make sure the security is only reindexed if the local roles change. Some subscribers to make sure the tree is kept up to date.
- Keep shadow tree up to date.
- Compute hash of local roles.
- Reindex the main object.
- Compute hash of local roles again.
- If the local roles are the same, do nothing more with the children.
- If applicable, do the same for the children, grandchildren, etc.
We did benchmarks, showing that this solution is a lot faster, both for Archetypes and Dexterity.
Next phase is to test it extensively on some real world scenarios and sites, and make sure it meets the workspace requirements for Plone Intranet.
Don't go into the ZMI and add local roles there, or do other stuff that Plone does not know about. Don't clear the allowedRolesAndUsers index.
The buildout extends the Plone 4.3 core development buildout and somehow the speed gain is much less; I don't know yet why that is the case.
If you have a project where you have this problem, try it out, for now on a development instance. You could also try to install the [benchmark] extra, generating benchmark-at and benchmark-dx scripts to create sample data and time it.
Watch the video of this talk.
Ramon Navarro Bosch and Rob Gietema: Resource Registries
Talk by Ramon Navarro Bosch and Rob Gietema at the Plone Conference 2014 in Bristol. Original title of talk: front-end development with Plone.
Current state
js registry, css registry, tools. Register your files in cssregistry.xml and jsregistry.xml. There are limitations. We are missing out:
- js and css have evolved. No one wanted to touch javascript a few years ago and now it is quite fun.
- dependency management was added in the javascript world
- now there is precompiling with less/sass
- versioning
- Mockup to do this in Plone.
Other tools
- RequireJS: you include require.js, add all your dependencies in javascript, asynchronous loading of files and modules.
- grunt: task runner, run tests, concat, minimize, jshint, etc. Kind of like a Makefile, but more suited for Javascript.
- npm: nodeJS server based package management tool, dependencies, versions.
- Bower: same thing on client side.
Less/sass: precompilers for css, variables, nesting, macros.
Design of the solution
- Diazo on top of static html
- bundle, resource, registry
Bundles are the main resources, loaded in dependency order. They can have js and less files that are compiled. You can enable/disable a bundle. There is a special legacy bundle with the old stuff as you are used to in current Plone 4.
Resource: defines the js, any needed export/init code, css, url. All defined in the Plone configuration registry.
Define Less variables, in the config registry record plone.lessvariables. For example define your own favorite plone-link-color.
Pattern options, for example for the pickadate widget.
Development versus production
- Compile on browser on each reload using RequireJS/Less.
- Develop static html with compiled bundles for testing
- Grunt using browser config.js/mixins.less.
Diazo
You can now say which bundles are enabled.
Views
- config.js has requireJS config.
- less-variables.js: has the standard less variables.
- plone.less: import files, patterns
- plone.js: requires the files.
This all is fully customizable through the web.
Legacy Plone: cssregistry.xml and jsregistry.xml still work. You probably want to convert it though. There are some incompatibilities: there is no ordering in the new stuff, only dependencies. So you may need to tweak a bit.
generate_gruntfile.py extracts all the information from the resource registries and creates a Gruntconfig.js file so you can test it on a static site.
A new Plone 5 site uses production settings for the theme. For development you obviously want to put it in development mode in the Site Setup. You can edit files through the web, have it compile and see the changes.
Migration
- Define resources as pattern and less file.
- Always generate production bundle.
- Move jsregistry.xml and cssregistry.xml to registry.xml.
- On/live jquery 1.11 modifications.
- Never loose hope.
See a plone.app.mosaic branch for the changes I did for migrating that package. It took me less than one hour to migrate.
More information: http://www.nathanvangheem.com/news/plone-5-resource-registries
We are going to sprint on this.
Watch the video of this talk.
Víctor Fernández de Alba and Albert Casado: Plone 5 theming unleashed
Talk by Víctor Fernández de Alba and Albert Casado at the Plone Conference 2014 in Bristol.
Plone 5 has a toolbar. On the left by default, but you can move it to the top if you want.
New markup, so less Diazo. Updated html of header, navigation, portlets, etcetera.
Remember that when styling you can use the test_renderer page to see all the standard elements of Plone, to check that all is fine.
We all like bootstrap, but use Plone classes instead, like context for buttons, which you can extend.
A designer should learn basic rules of Diazo, reuse the fine existing Plone markup, understand Plone structure, have in mind Plone default classnames, prototype a lot in in static html.
How to transfer a prototype into a Diazo theme for Plone
- Prototype based on the default Plone markup as far as possible.
- Know the limits of prototyping and know when to stop and use the real site.
- Use Diazo to move blocks around, do easy modifications to the default markup. Do not forget to make space for the toolbar just after the tag.
- Only if strictly needed, modify existing templates to match the prototype ones, using z3c.jbot.
- Use iterative improvement. Demo it to the customer frequently.
- Add custom styling for standard user generated markup in TinyMCE.
main_template is now a browser view, instead of a skins resource: Products.CMFPlone/browser/main_template.py.
Resource registries have been completely updated for modern front-end developing. Plone bundle, Barceloneta theme bundle, Legacy bundle. In your Diazo theme in manifest.cfg you can enable or disable bundles.
Development mode: compilation on browser of less/js resources. Compilation on console via grunt watch.
Barceloneta theme
- Pretty well ordered files: plonetheme > barceloneta > static > less.
- Deductible LESS structure and file naming.
- You can reuse the Barceloneta LESS files in your own theme by importing it with the sitePath variable.
- We did not use much Diazo: the rules file is very simple.
Toolbar is not completely isolated, we ran into problems using an iframe for this, it is still the plan for the future, it is now about halfway.
Maybe we can add a checkbox for developing in LESS and not Javascript and the other way around, so your browser does not need to compile everything all the time.
Twitter: @albertcasado and @sneridagh.