Embrace and Extend: The Zope 3 Way

published Jun 04, 2007, last modified Jun 05, 2007

Use case: extending Quills

This blog entry explains the use of the keywordannotator product from the Plone collective and its accompanying example product quadapter, which lives in the examples/ directory. Both are in use on this weblog. This tutorial has also been published in a report (400 kB pdf) I made for school. And the tutorial is now up at plone.org.

Introduction

I have a weblog. At the time of writing this is a hand-made Zope blog, consisting entirely of page templates and scripts, which sit in the Zope Database. See the status from early 2006 on the wayback machine. It works, it is fast, but it is kludgey. I have to do too many things before I can publish a new weblog entry. So there is room for improvement. Plus several other weblogs are available in Plone. Quills seems to be good, and I know a few persons who also use it, for instance my brother, so I choose to use that too.

Actually, I have two weblogs at my site. One is a general blog where I post personal things or entries relating to programming. The other is a completely Dutch blog for my church. Look at the old version. Actually it is more of a podcast. It contains small entries with a link to an external mp3 file containing a complete service of my church, and bigger entries with a link to a local mp3 file with just the sermon in case our pastor Erik was leading this service. So one service could actually have two entries. Again, it works, it is fast, but it is kludgey.

For this second blog, Quills seems like a reasonable choice as well. But to really make it useful, I would want two extra fields for the audio blog, that can contain links to those mp3 files. For the sermon it could be even nicer to be able to upload this file via Quills and have the link correctly made. But that may be added later. For now, I am happy with just two extra links in a Quills WeblogEntry.

Brave New Content Type

Alright, let's create a totally new content type, shall we? With ArchGenXML we can easily create a basic new content type, let's call it AudioEntry, that has the exact attributes that we need. Then again, other people have already thought a lot about what a weblog entry should contain and what it should do. We would introduce yet another weblog application, which Plone does not really need and that is currently only maintained by us. There are certainly occasions where this is a splendid option, but not here.

The olden days of Zope 2

Our second option is to extend Quills in the Zope 2 style: we create a class AudioEntry that inherits from the Quills WeblogEntry. If our class really offers something new, this is a fine choice. A good example is RichDocument. This extends the Schema of ATDocument to add a few attributes. The class inherits from ATDocument and so it uses the functions already defined for that basic ATDocument and adds a few functions that make sense for RichDocument. This is very well documented in Extending ATContentTypes by Martin Aspeli.

What would that mean for our use case of Extending Quills? AudioEntry would inherit from WeblogEntry and extend it with two attributes for the links. But it would not stop at that. For starters, a Quills Weblog can currently only contain Folders and WeblogEntries. Adding our AudioEntry there is not possible out of the box. Of course this is changeable, but then we might also have to override the Weblog class with our own AudioWeblog class. This can easily get out of hand and result in a cascade of overrides. Also, when someone else has a similar idea and introduces a VideoWeblogEntry, these two extension products would be incompatible.

There are certainly use cases like RichDocument where this Zope 2 way makes sense. But for our use case we simply want to add two links. Is there no easier and less intrusive way to do this?

Zope 3: A New Hope

When we extend a product in Zope 3 style, we adapt an existing content type without making a new content type. In our case this means that we only need to adapt WeblogEntry, without the need for changing the Weblog class. Also, we can arrange that our adaptations work for the VideoWeblogEntry class as well, as long as that class plays nice and implements the IWeblogEntry interface that Quills defines.

So already we can see that the Zope 3 way indeed brings new hope, in that our changes can be much less intrusive and much more compatible with other changes.

We will be using and introducing the following Zope 3 technologies:

  • events
  • utilities
  • adapters
  • annotations

General Strategy

We are going to make two products: one general product for extending an object when it fulfils some condition, and one specific product for extending a Quills WeblogEntry when it has audio content. The strategy for the general product is this:

  • An event is handled by an event handler, which uses a...
  • utility to decide if an...
  • adapter is necessary, in which case...
  • annotations are added to the original object.

So: when an event takes place, we ask a utility if this event warrants an adaptation of an object. The adaptation is implemented by adding annotations to that object.

This is implemented by the keywordannotator product. And this strategy probably has most people blinking their eyes and rereading that list. :) Don't panic. You probably do want to reread it, but if you do not fully understand it, you can just go to the next section, which will translate this general idea into a specific strategy for our use case. It should be much clearer then.

Quills Strategy

Do you remember the four Zope 3 technologies that we planned on using? Events, utilities, adapters and annotations. This is how our specific strategy uses those:

event
Someone adds a keyword audio to a WeblogEntry. When this happens, Zope fires a so-called IObjectModified event. We will register an event handler that springs into action when a WeblogEntry (or actually any object that claims to implement the IWeblogEntry interface) is modified or added.
utility

The code that handles this event, calls a utility. This utility looks at the WeblogEntry and decides that the addition of this keyword means that this entry now also implements the marker interface IMaudio that we will define. If an object implements or provides a 'normal' interface, then this object has all the functions and attributes that are defined by that interface. A marker interface has no functions or attributes. So it is basically just a label: the object is marked as being IMaudio.

We also make sure that we can later add annotations to this entry by letting it provide the standard IAttributeAnnotatable interface defined by Zope 3.

adapter
We can now actually adapt (extend) this object that implements IMaudio and add annotations to it. Adapting basically does the same that inheritance (the Zope 2 way) does, but with a different technique. You temporarily put a wrapper around an object, do something to it (in our case add annotations) and then remove the wrapper or just forget about it. Any other code (for example the code in Quills itself) does not know or care that the WeblogEntry object has been adapted or wrapped and now has something extra. To that code, the WeblogEntry is still a normal WeblogEntry, even though it now has annotations.
annotations
Annotations can be added in several ways, but most used is the method associated with the IAttributeAnnotatable interface that I mentioned above. In our case, two links are added in a new hidden attribute of that WeblogEntry.

This is implemented by the quadapter (Quills Adapter) product, which uses the keywordannotator product. In fact, this quadapter product is added in the examples directory of keywordannotator. By the way, both products have tests, so you can just run them and see for yourself that all this actually works.

This section is rather important. If you understand what our goal and our strategy is, you stand a good chance of understanding the next sections which present the actual code. On the other hand, if you prefer a bottom up approach, then reading the next sections may help you in understanding this strategy.

Each of the next few sections first present the code from the general keywordannotator product and then the code from the specific quadapter product. Sometimes I find it hard myself to follow what keywordannotator is actually doing, but reading the corresponding quadapter code usually helps. :)

Event handler

zcml registration

Okay, for the general keywordannotator product we want an event handler that gets activated whenever an object implementing the IAttributeAnnotatable interface gets modified or added. We register that in the configure.zcml file like this:


That takes care of modified objects. For added objects we replace 'IObjectModifiedEvent' with 'IObjectCreatedEvent'.

In the case of quadapter we want an event handler that gets activated only for object implementing the IWeblogEntry interface:


Event handler code

In keywordannotator the code is this:

def annotationEventHandler(ob, event):
    from zope.component import getUtility
    decider = getUtility(IAnnotationDecider, context=ob)
    if decider.matchesKeywords(ob):
        decider.provideInterfaces(ob)

So this event handler looks for a utility. That utility tells us whether the object the event was fired for has a keyword that has been marked as special in our code or not. If this condition has been met, we instruct the utility to make sure that the object now provides a few more interfaces.

Note: quadapter simply uses this event handler from keywordannotator without overwriting it.

Now let's look at what that utility looks like.

Utility

zcml registration

In keywordannotator it looks like this:


This means that when some code wants to have a utility that provides the IAnnotationDecider interface (the event handler code wants this, see above), such a utility can be created by calling the DefaultAnnotationDecider class in the events.py file in the keywordannotator product.

The quadapter overrides this utility in overrides.zcml:


Since this is an override, this means that the only known way to create a utility that provides the IAnnotationDecider interface, is now calling the AudioDecider class in the events.py file in the quadapter product.

Utility code

So what does that code look like? In keywordannotator it is this:

class DefaultAnnotationDecider(object):
    implements(IAnnotationDecider)
    keywords = KEYWORDS
    ifaces = (IKeywordMatch,)
    def matchesKeywords(self, object):
        ...
    def provideInterfaces(self, object):
        for iface in self.ifaces:
            if not iface.providedBy(object):
                alsoProvides(object, iface)

The implementation of the function 'matchesKeywords' is not interesting here. It simply checks whether the keywords of the object match one of the special keywords. By default, only the literal word 'special' is considered special.

The function provideInterfaces is more interesting. It makes sure that a certain object provides all wanted interfaces. By default this is the IKeywordMatch marker interface. In the next section we will register an adapter for objects implementing that interface.

In quadapter the utility code is just four lines:

class AudioDecider(DefaultAnnotationDecider):
    implements(IAnnotationDecider)
    keywords = KEYWORDS
    ifaces = PROVIDE_INTERFACES

It uses a different list of keywords that are special, and different interfaces that need to be provided to objects that match one of the keywords. These values are specified in the config.py file:

KEYWORDS = ['audio', 'preken']
PROVIDE_INTERFACES = (IMaudio, IAttributeAnnotatable,)

Note: 'preken' is the Dutch word for 'sermons'.

So, with just a few lines, the quadapter product changes the default adapter so it reacts to different keywords and provides different interfaces.

Note that a Quills WeblogEntry does not by default provide the IAttributeAnnotatable interface, so we must instruct the utility to make sure that WeblogEntries with one of the special keywords now implement that interface as well. If wanted, we could add code to quadapter that makes sure that all WeblogEntries also provide the IAttributeAnnotatable interface. If you want that, look at the utils.py file of keywordannotator.

At this point you may want to look back at Event handler code to see how this utility is used by the event handler.

Current Status

We have covered a lot of ground already. So before we continue with adapters and annotations, let's take a break and see what we have accomplished so far.

If you are only using the keywordannotator product, the situation is this:

  • An object implementing the IAttributeAnnotatable interface,
  • given the special keyword ('special'),
  • now also implements the IKeywordMatch interface.

If next to keywordannotator you are also using Quills and the quadapter product, then your situation is this:

  • An object implementing the IWeblogEntry interface,
  • given one of the special keywords ('audio' or 'preken'),
  • now also implements the IMaudio interface
  • and the IAttributeAnnotatable interface.

If this is not clear yet, then it is better to reread some of the previous sections. If you do understand the current status, then it is time for adapters and annotations.

Adapter

zcml registration

Remember what we want to do: we want to adapt objects implementing a certain interface by giving them annotations. In keywordannotator we register an adapter for objects implementing the IKeywordMatch interface. This adapter should provide such an object the new IKeywordBasedAnnotations interface, which is just a marker interface (or label). This adapter can be created by calling the KeywordBasedAnnotations class in events.py. We register that adapter in the configure.zcml file:


In quadapter we do something similar:


This means that the adapter can be created by calling the AudioAnnotations class in the events.py file of quadapter. This adapter provides the IAudioAnnotations interface for objects that already implement the IMaudio interface.

IAudioAnnotations is not a marker interface but a 'normal' interface, if you want to make that distinction. In the interfaces.py file of quadapter we state that any object that claims to implement the IAudioAnnotations interface should have the attributes completeURL and partURL:

class IAudioAnnotations(Interface):
    """Provide access to the audio annotations of an IMaudio object.
    """

    completeURL = schema.URI(title=u'URL to complete audio content')
    partURL = schema.URI(title=u'URL to a part of the audio content')

At this point it may be good to say that Zope 3 tends to use a lot of interfaces. But you have probably already noticed that. :)

Adapter Code

We have registered two adapters. Now how does the code look? In keywordannotator:

class KeywordBasedAnnotations(object):
    ...
    def __init__(self, context):
        self.context = context
        annotations = IAnnotations(self.context)
        self._metadata = annotations.get(self._anno_key, None)
        if self._metadata is None:
            self._metadata = PersistentDict()
            annotations[self._anno_key] = self._metadata

The __init__ function is the factory; in other words, it is the function that creates a KeywordBasedAnnotations object based on another object that is passed in via the 'context' parameter. This is the spot where we first really see annotations. It is this line:

annotations = IAnnotations(self.context)

Here the context object is adapted to the IAnnotations interface and the resulting wrapped or adapted object is stored in the annotations variable. That variable is now basically a python dictionary which at this point probably does not contain any values. The other lines make sure that a basic structure for storing annotations is now available in that object.

Now on to the adapter code in quadapter:

class AudioAnnotations(KeywordBasedAnnotations):
    ...
    def __get_completeURL(self):
        return self._metadata.get(COMP_ANNO)
    def __set_completeURL(self, url):
        self._metadata[COMP_ANNO] = url
    completeURL = property(__get_completeURL, __set_completeURL)

This class subclasses the KeywordBasedAnnotations class from keywordannotator, so it inherits the init function we saw above. But the lines in this code do the stuff that we actually started this product for: they add the completeURL property, where we can store the link to the complete service of my church. This link is stored in the self._metadata property, which is an annotation to this object.

The same is done (in some code that is not shown here as it is basically the same) for the partURL property, where we can now store a link to the partial service or more sane terms: the sermon.

Annotations

In the previous section we have actually already seen how annotations are added to an object and that they are basically just a python dictionary. But before this is possible, we need to add one line to the configure.zcml of keywordannotator:


This directive loads another zcml directive which is in the zope software directory, in the file lib/python/zope/app/annotation/configure.zcml:


This actually concludes our strategy. All the code and configuration is now in place to add those annotations to a WeblogEntry by adapting it if it implements or provides the IMaudio interface, which is added to it by a utility if if decides that the condition is met after an event takes places that is handled by an event handler. If you understood what I just wrote, then you are very smart. :)

Adding Annotations

At this point, if you look at the state of things on the Plone level, we can add a normal WeblogEntry and give it a keyword 'audio'. Then our code makes sure that it provides the IMaudio interface. And then it stops! There is not yet a way to actually put something in the annotations. But we can arrange that. First we install a new product: CMFonFive.

That product may get assimilated into the core of CMF itself some day (Plone is based on CMF, the Content Management Framework). But for now this extra product is needed for the following lines that we add in the configure.zcml of quadapter:




This adds a menu item in the Plone site to the object tabs of any object that implements the IMaudio interface. When you add this code, restart your Zope instance, and look at a WeblogEntry that has one of the special words and thus provides the IMaudio interface, you will see a tab that links to the action maudio_edit.

At that point we are almost done. We now need to make an edit form for that tab, that calls the code that sets the Annotations for this object. If you know how to make an edit form in html, then you should be able to make this yourself, possibly with the use of the Archetypes product.

Viewing Annotations

When you have added those links in the annotation of a WeblogEntry, you also want to be able to see them when you actually look at a WeblogEntry in your browser. We can use one more Zope 3 technique for this: the BrowserView. Essentially this also is an adapter. We need to define a BrowserView in quadapter:


This basically means that for an object providing the IMaudio interface we have a python class that gives us some functions to call in an html page template. In fact, we can now copy the file entry_macros.pt from Quills and add these lines at the right spot:


  

This uses a maudio_macros.pt file that gets the links from the object. Now when you view a normal WeblogEntry you just see the normal page that you would otherwise see. But when you view an entry with the IMaudio interface, for which you have added links with the edit form in the previous section, you will now see some extra text containing those links. The specifics are left as an exercise to the reader as they are just standard page template techniques, which should be familiar and which are not too interesting for this tutorial. If you do not get it working, contact me. Well, okay, I have just added them in the skins folder of quadapter.

Conclusions

Using Zope 3 like this is:

  • clean:
    • Only one page template from Quills is changed and no code is overwritten.
    • If we added a metal:macro with a fill-slot to Quills, even that single overwrite would not be necessary.
  • compatible:
    • We only add features, we do not change existing behaviour of WeblogEntries.
    • Our audio enhanced WeblogEntries are still first class WeblogEntries. Existing code that expects a normal WeblogEntry can handle our audio entries just fine.
    • Our changes work on any content type that implements the IWeblogEntry interface. So if someone really does create a VideoWeblogEntry implementing IWeblogEntry, our code would work for such an object as well.

In other words: Zope 3 is the place to be!