Weblog

published Nov 03, 2021, last modified Nov 04, 2021

PLOG Tuesday evening

published Apr 07, 2015

Laurence Rowe on cache invalidation, at Plone Open Garden Sorrento.

Laurence Rowe: Cache invalidation

Sharing experience from a non-Plone project. Cache invalidation is always hard. Pages may contain snippets from other pages, from other content items. It is hard to invalidate all pages that may contain such a snippet when updating this item.

ZODB caches the objects. Zope stores the transactions. A transaction has a list of objects that are changed. Other versions of the same object can then be removed from the cache.

I wondered if I could do this for caching pages in Varnish or in Elasticsearch. I tried this in the experimental.depends package. The first simple way kind-of worked. But new items that would need to end up on the page and were never there before, did not get inserted.

I looked at the Darcs version control system, which does similar things.

A folder contents page that shows the first five items, needs to take all items into account for possible inclusion.

I have an indexing process, asynchronous, that updates the info for anything that needs to be invalidated. It leads to a materialized view of the data, allowing faceted search across a deep join.

I think this principle could be extended to Plone. It would allow indexing to a page cache like varnish, or a search cache like Elasticsearch.

Varnish helps making often visited pages fast, it does not help with the long tail of pages that are hardly ever visited.

PLOG Tuesday afternoon

published Apr 07, 2015

Current state of various teams. Plone Open Garden Sorrento.

Kim Nguyen presents results from the Strategic Summit Survey

35 results back. Mostly: developer, solution provider, site-admin, deployer. Mostly North America and Europe, a couple from Asia, Africa, South America, Caribbean or world wide. Most have ten or more years of experience with Plone. Majority is not a Plone Foundation member. Some of you should be.

Now to some answers by those people.

What should be on the technical agenda? Nothing: this should be an open discussion of the future of the product. API. Plone 5 release. Moving away from Zope 2 and CMF. Making Plone more customizable through the web, easier to get started. Block editing, mosaic, tiles. Improving learning curve for developers, making it easier to come on board. Plone Intranet, Patternslib, Mockup, Python 3 (one), Continuous Integration. Refocus Plone as secure workflow tool, not necessarily a CMS. Replace catalog with SOLR. Better custom themes. Sidebar UI, some of the choices around this. Better mobile theming out of the box, use for mobile applications.

What should be on the non technical agenda? Marketing, visibility, promotions, formalise it, maybe professionalise? New plone.com and plone.org, get your success stories or your provider on it. Better communication from sprints, team leaders to the community and to marketing or external audiences. Get more money for the Plone Foundation. Who is the target user, client, customer? Concerns about upgrade from Plone 4. Be more welcoming to developers and other contributors. Monthly global sprints, similar to the earlier Plone TuneUps, re-energize the community. Releasing Plone 5. Better sustained documentation and consolidation, more help needed there. Consolidated bug tracking, merging into one place. Better Plone training, getting new people up to speed. Improve collaboration with Python, Pyramid, Django, Javascript communities. Decision process for PLIPs and their inclusion.

What should be the outcome of the summit? Roadmap, structured plan. Vision, better marketing. Plone 5 ship date (six months?). New community energy and synergy.

Paul Roeland and Eric Steele: summarizing complaints

  • TinyMCE showing its age.
  • People being spoiled by Google Docs.
  • Where to do content editing.
  • Scheduling content, how that works, expiring.
  • Old and new style collections do different things. Difficult programmatically.
  • Documentation: dexterity without Grok is not documented well, if at all.
  • Sorting portlets. There is an add-on though.
  • Deploying to different sites. Some add-on as well, so: notifying people what add-ons are available.
  • ZCatalog takes ages, indexing, reindexing, unindexing.
  • Adapters are hard to grasp as concept for new developers.
  • Changing a label in a dexterity behavior for just one content type is not possible.
  • Difficult to find Plone devs to hire, to train up new devs.
  • Nobody wants to touch Zope and CMF parts.
  • Collective add-ons, hard to get overview.
  • How to get extra content on the page. Several ways.
  • Plone complexity makes estimating a project complex.
  • Editing complex pages. Deco, cover, mosaic.
  • Concerns about migration process to Plone 5.
  • Challenging to keep up with what is happening in community.
  • Where are we at and where are we going.
  • Clients like concrete release dates.
  • What to do with the underlying stuff. More boring, but somebody needs to do that, clean it up.
  • Not good as community at making decisions, which means nothing gets thrown away.
  • Look at it from a business standpoint: how much would it cost to support Archetypes, how much to drop it, etcetera. Cost of current inefficiency is high, you can be bug hunting for hours. Cost of throwing lots of stuff away, is also high. Starting from scratch is very costly as well.
  • Feature changes ending up in bug fix releases.
  • LDAP integration is bad.
  • Not updating outdated documentation.

Eric Steele: Plone 5

There is a buildout for the first beta. Mockup has had issues with frequent rewrites. Control panels have now been modernized, using z3c.form and the configuration registry. plone.app.contenttypes is in there. Migrations for standard content types is working, for custom content types it is there but needs more love and/or money; you still have to write your own content types, views, templates of course. Dexterity is default content type story. New Barceloneta theme. Improved folder contents tab.

We have completely rewriting the way we use Javascript in Plone. Using more modern techniques, like RequireJS. A new Resource Registries panel in the Plone UI to edit this. You can edit less variables through the web, kind of like the old base_properties. We are now able to use more modern widgets, without needing to maintain outdated ones ourselves.

When will it be out? The more you test it and report bugs, the earlier we can ship it.

Eric and Paul: Community and communication

We have semi-regular team leader meetings. We are putting notes up on http://community.plone.org. There are lots of duties to take on, certainly also for non-core developers. You are very welcome.

Documentation is in reorganisation. Restructuring documentation that was already there. The documentation team is not the one to write a doc about say creating a dexterity content type without Grok. Someone else will need to write it, and we can then help getting it more understandable, in better English, etcetera. You do not need to be good at writing, you just need to write.

A lot of the teams are currently one person. We need to do a better job of getting more people in. Some people are in multiple teams, overcommitted, like Timo and Ramon.

Víctor Fernández de Alba is working on new plone.org, mostly on his own. Migrating some of the content using transmogrifier. Using Plone 4, with plone.app.contenttypes. Documents, events, news, images, files are migrated, PloneSoftwareCenter will not be used anymore, so that content will not be migrated. We wanted to have the PLIP process on plone.org, so we no longer need dev.plone.org. Maybe profile page for Plone contributors, listing which teams they belong to, other information. Maybe we want to rework the theme, change the landing page. Should be discussed, maybe during these days. Package is plone.org.core. You can see on github what is left and what is done.

Paul: dev.plone.org is now spam-free again, mainly thanks to Chrissy Wainwright. Down to about 600 tickets to export and import to github. There is a way of moving tickets from one github repo to another, and a way of creating a ticket in an initial spot when you do not know where it should be reported. Needs a champion, someone to do it and/or slave drive others into doing it. Are you good at scripting, with at least minimal people skills, please step forward. Even better: a group of people.

Eric: https://community.plone.org is being used. We need to make the decision to flip the switch to make this the main forum. Yes, we will switch it this Friday. Not the developer list though.

Paul: http://paragon.plone.org is a site for searching the very best Plone add-ons. There is a jury. Work still needs to be done. Also: get your setup.py classifiers correct so they mention Plone and the Plone version, and they will show up in a list here as well.

Kim Nguyen: http://plone.com is live, soft-released. We used an issue tracker and systematically used it, filed and assigned tickets. We had weekly meetings to keep on track and to keep us honest. It is so important, it re-energizes people. Please step up in a project, help a little bit. [Christian Theune: Too many people working on their own: create a bigger team again, so people do not get stuck in their own little corner. It helped us a lot.] We want more success stories. We want to hear from you if you think the message needs more refinement. To get your provider up on plone.org, contact us and become a sponsor. It is a small amount, we want it to be affordable also if you are in an area of the world where you do not make a lot of money. The goal is not to get a lot of money from this, just to get an up to date list. Translations: we will discuss it on Thursday. If you want to be a part of plone.com as your country, contact us. Also fine to have your own site, like the Dutch http://plone.nl

Paul: documentation is there, but it is mostly for Plone 4. Plone 5 is getting closer, so this needs updating. Looking into getting robot tests for screen shots. That is actually easy to add in your code when you have something that is worthy of a screen shot. We will be contacting you about that,

Christina McNeill: one-person communication team. Plone Newsletter has been there for a year. About 160 subscribers. 80 percent is actually reading it, which is really high according to mailchimp. Marketing is not really my area of expertise, so we can use someone there. [Philip: please add a nicely visible button on the plone.org homepage to subscribe.]

Philip Bauer: David Glick and Tres Seaver have recently been working on getting Zope parts to run on Python 3. Is there a master plan? Eric: no.

Asko Soukka: Mosaic. We are working on it. It may be doing too much. May be hard for people to decide whether to use something in Mosaic versus Diazo. Lots of things in tiles are broken for lots of reasons, needing work. Many concepts. Store site-layouts as content types. Needs specific versions of plone.app.widgets and plone.app.events. There is a PLIP configuration to run it on Plone 5 in the core development buildout. Ramon: it works fine, but I am not trying to finish it for production. Philip: Mosaic should be a priority for us, let's not wait until our javascript story is completely finished. Crowd: do not try to do this on Plone 4 anymore, only on Plone 5. Asko: hide all tiles that do not work. Eric: everyone help out on sprints, donate time, get a customer to build this for.

PLOG Tuesday morning

published Apr 07, 2015

Notes from Plone Open Garden Sorrento, april 2015.

Maurizio Delmonte welcomes us to Plone Open Garden. Organized by Abstract. About sixty people will be here. This is a Plone Strategic Summit. Ninth year of the Open Garden.

No laptops in the hall please, let's give the other clients of the hotel a sense of vacation still. Do not go everywhere, give others a bit of space. We have enough room for you.

Paul Roeland speaks. Last Strategic Summit was in 2008. Was anyone there who is now here? Only Matt Hamilton. Last Plone conference in Bristol there was talk about Plone 2020, more about the backend. There will be Hangouts tonight at 2200 local time, and Friday too, same time, connection with PyCon in the USA.

Ground rules:

  • one conversation at a time
  • not talking over someone else
  • letting two others speak before I speak again
  • be nice

Always appoint a person who reports, writes down, draws it, whatever. Maybe snippets of code will do. Sticky notes, flip charts. Tell the rest of the people here, tell the world, because not everyone is here.

If you want to complain about how Plone 4.2 works, this is not the time. We look from Plone 5 forwards.

Talks in the morning. Technical track in the afternoon.

We love it when you talk a lot to each other. Having a not so fantastical internet connection helps, so be happy about it. Do not be afraid of talking if this is your first time here. Step out of your comfort zone. Do not talk about just the backend, which some of you could talk for three weeks about. If you are a Plone provider, try to think like a Plone user.

Timo Stollenwerk speaks about the technical track. Goal of today: full report about where we are from a technical point of view.

Guido Stevens talks briefly about Plone Intranet Consortium. No sprinting planned, just talks about direction.

Complaining session

What are your major pain points in working with Plone at the moment? We split into three groups.

  • Almost impossible to hire new developers. Even just people who I can train up. Django or other Python web developers. There are no Plone developers walking around that you can snatch up. But you can train the others. Better have web developers than people who have only worked with Python in a scientific way. Still, there is lots of Plone code, divided over hundreds of packages, and that makes it hard for new people to get productive quickly.
  • Lots of old CMF or Zope code from the nineties, or early this century, that mostly works, but that nobody dares to touch and improve or cleanup.
  • Collective is nice, but there are thousands of packages, and no known good set of add-ons that really work and are still maintained. You do not want to reinvent the wheel, but do not want to use somebody else's abandoned wheel either.
  • You can easily click around in the settings, but then you want to export it, create an upgrade step, just too much trouble to get that done right, at least for new developers. Or how do you get extra content on the page: viewlet? portlet? diazo? browser view? Through The Web is still a good starting point for getting new people enthousiastic. Sean Kelly's video is still awesome, drawing people in, but it uses technology that is eight years old and that we no longer advise.
  • We miss completing things. Things start out nice, but developers get bored with it and it is much more fun to create new stuff without finishing the last five or ten percent. But to finish it, you need to put it in the wild and then people complain because it does not work completely. For example Dexterity, it had nice concepts, I tried using it a few years ago, in a way that should work (by looking at code, not at the then not existing documentation), and it did not work.
  • Hard to estimate how much work it is to fix things in a new client project, so you estimate higher, so you do not get the project. It is not a problem specific to Plone. Problem of classical fixed price projects. Difficult for me to have profitable projects in Plone. Every project has a new black box, a new part of Plone that I have not touched before, so where I run into problems that are hard to estimate and solve.
  • Plone has lots of features, it is hard to learn them all, keep them in your head. We might need to drop more stuff. It is easier and more fun to write something new.
  • Editing of complex pages is hard, content on one page being shown on other pages. Too hard to build UI in Plone. We add a prefix for each component so we can know css for this component only affects that component instead of needing to know which other pages might be adversely affected too. Mosaic should be finished, it had been a problem for years to solve lots of theming problems we have.
  • Some packages are old, for example last release of plone.app.tiles is from 2012, but there have been several helpful commits on it which have not been released.
  • Plone is ahead in some areas, but in the user experience of content editing, Plone is lagging behind in enterprise level.
  • I am concerned about migration from Plone 4 to 5. I generally customize login form, really specific theme now with Diazo. Now all the login form stuff was removed, done differently, so it is going to be hard to redo. I may not be able to sell this migration to the customer, and stay on Plone 4 for years. Currently we use bootstrap 3, and Plone 5 theme uses bootstrap 2, so that will give difficulties. Lots of add-ons cannot support both Plone 4 and 5. Upgrade guide still needs to be written. Also upgrade guide from Archetypes to Dexterity, and from LinguaPlone to plone.app.multilingual. It would already help to add a header in the documentation to remind others that something needs to be written.
  • Several packages have been released as a bugfix release, where it suddenly no longer works in Plone 4, without creating a branch.
  • If stuff is broken, open a ticket. But it should be clear where to file it, on the front page of docs.plone.org.
  • Lots of pull requests are open, please look at them. If you are not sure about merging, at least add a comment. We need a process. dev.plone.org is slowly being moved to github.
  • Current LDAP integration is old. Lots of layers. Very large or remote LDAP: very slow. Should be on the roadmap, in core Plone.
  • Releases are not happening often enough, should be every month. Now releases take up too many changes, which can give more unexpected problems. There is just so much development going on that it is hard to keep on top as release manager. Working on a release team.
  • Complaint about companies using Plone and not contributing back. But sometimes it is hard to contribute. Plone TuneUp may be reintroduced, with team leaders available. Get people into the community. Make donations easier.
  • It used to be easy to create complex workflow with ArchGenXML, but now you need to code it yourself. There is a plone.app.workflowmanager that does similar things, rewrite going on.
  • Setup a review service so you can pay someone to look at your project to check that it is state of the art.

Paul Harland Prijs: 34e

published Feb 10, 2015

Ik ben als 34e geëindigd in de editie 2014.

De Paul Harland Prijs is de grootste Nederlandstalige schrijfwedstrijd op fantastisch gebied: fantasy, science fiction en horror. In 2013 was ik 38e en nu 34e.

Je kan mijn verhaal, getiteld Een wereld vol gedachtenlezers, lezen als pdf of je bekijkt meer opties op Smashwords.

Summary of Plone conference 2014 Bristol

published Nov 03, 2014

What did I take home from Plone Conference 2014 in Bristol?

During the Plone Conference, as usual, I made summaries. You can see them all by checking the ploneconf2014 tag. I do this for you, with pleasure, but also for myself. Now I can read my own notes and gather my thoughts and remember what I actually heard.

Here are a few highlights.

  • Plone 5 is coming. I would be very surprised if a final release is done this year. I would be even more surprised if it did not happen in 2015. I may want to try to redo my own website with the next alpha of first beta version.
  • Transmogrifier can be a good way to migrate sites. I have some experience with it already. I may want to use collective.jsonify to migrate my current Plone 2.5 site...
  • The new Plone 5 theme is called Barceloneta. It uses Diazo. It uses the new resource registries that you can configure in the Plone UI, replacing the old portal_css and portal_javascript. Your custom Plone 5 themes should do that too.
  • A lot is happening in Javascript for Plone. The conference gave food for thought on AngularJS and json. A json api on top of Plone is being worked on, which can help in creating a completely different UI if your website needs it, or it can make showing Plone content in a completely different framework possible. Mockup should be the basis for your widgets or other Javascript work. Quote: We have to get the whole community into thinking 'Javascriptish', really push it into the head of more developers.
  • Documentation is (as always) being worked on, including versioning. Using five.grok in code is not officially recommended practice (it should work fine, but too many people in core development do not like its 'magic'), so the documentation that uses it is being copied to a grok-only part and edited in the original location to use best practices.
  • The Plone Intranet Consortium is working on improving Plone as an intranet. Several companies are banding together, with online sprints each Wednesday. The plan is to give the license of the code to the Plone Foundation. But for the moment development is tightly controlled so that effort is focused, with a design first approach. Contact us if you want to join.
  • The Plone Roadmap team talked about the roadmap and vision for Plone in 2020. Most important is what we want to keep, which is first and foremost: the community. Particularly, the Roadmap Team is the community, which means you. Favorites for changing: have one canonical way to do something (practical: improve plone.api); create a json api to help enable new stuff; improve end user experience.

I had a great time. Thank you, each and every one. Hopefully see you next year in Bucharest.