Plone

published Nov 03, 2021

This is here to serve as contents for the atom/rss feed for Plone, also read by planet.plone.org.

Search forms with zope formlib and batching

published Nov 23, 2007 , last modified Jan 30, 2008

Combine a search form and a results page by using zope.formlib and plone.app.content.batching.

In this tutorial we want to have a search form on our web site. After a visitor fills in the form he should see a page with results. But on that same page he should see the search form again, with his previous values filled in. On top of that, the results should be batched, displaying only the first ten results and having links to the next pages. On Plone 3.0 you can do this by combining zope.formlib and plone.app.content.batching.

Some details are left out, like explaining how files and classes are called and where they are located (most are in the browser/ directory), or not showing some methods whose goal and code are hopefully clear enough from their name. The idea is not to make this too long and I am failing at that already. :-) If I omitted too much for your taste and you have a specific question, just ask.

Building search forms

We start with a browser view. On the top we need some imports:

from zope import schema
from zope.interface import Interface
from zope.interface import implements
from zope.formlib import form
from Products.Five.formlib import formbase

With our search form we want to search for some searchable text and for Products of a specific Brand. Brand is a content type we (Zest Software) made for a customer. A Product is another content type, which is added to a Brand. We made a vocabulary for this which maps the title of a Brand (shown in the form) to the path where this Brand is found.

We define an Interface for our search form. Very important: let the parameters match the names of the catalog indices that you want to search on:

class ISearchForm(Interface):
    """These are the fields of our search form.
    """
    path = schema.Choice(title = _(u'Brand'),
        vocabulary=u"customer.brand",
        description = _(u'Brand, mark, formula.'),
        required = False,
    )
    SearchableText = schema.TextLine(title = _(u'Search word'),
        description = _(u'Search for specific words.'),
        required = False,
    )

For schema.Choice the default widget is a SelectWidget or RadioWidget. As empty value it displays "(no value)" in the drop down box. At least for a Dutch web site this is not very nice, as there is no translation. You can provide an own version of that message by doing something like this:

from customer.product import customerMessageFactory as _
from zope.app.form.browser.itemswidgets import SelectWidget
SelectWidget._messageNoValue = _("vocabulary-missing-single-value-for-edit",
                      "Selecteer indien gewenst een waarde.")

Or do this in English and use i18ndude of course.

Now write a class that inherits from:

  • Products.Five.formlib.formbase.PageForm to have a complete page.
  • Products.Five.formlib.formbase.SubPageForm to get a partial page.

In our case the search form will be only part of a page, so we choose the second option:

class Search(formbase.SubPageForm):
    implements(ISearchForm)
    form_fields = form.FormFields(ISearchForm)
    label = _(u"Working independently")
    description = _(u"You are looking for a product for:")

Add an action button to that class:

@form.action(_(u"Search"))
def action_search(self, action, data):
    """Perform search.
    """

On a SubFormPage this can actually be empty (pass): the surrounding form tag that you have to supply yourself takes care of redirecting. For a complete FormPage you would at least need something like this:

self.request.response.redirect(target)

where target should be a page with search results.

Search form and results on one page

Hook up your browser view in configure.zcml. We only register this view for the Site Root in this case:


Same for a results page:


And the main page showing search plus results:


Use something like CMFPlone/skins/plone_content/document_view.pt as base and add the structure of the search and results subpages:





...

        
...

Overwriting the search form template

If the default template from zope.formlib does not give you what you want, you overwrite it. Configure some named adapters in zcml:



The factory for the first adapter is in zope.formlib. Our own alternative factory is this:

from zope.formlib import interfaces, namedtemplate
from zope.app.pagetemplate import ViewPageTemplateFile
subpage_template = namedtemplate.NamedTemplateImplementation(
    ViewPageTemplateFile('searchform.pt'), interfaces.ISubPageForm)

Let your class know it should use the new template:

class Search(formbase.SubPageForm):
    ...
    original_template = namedtemplate.NamedTemplate('default.subform')
    template = namedtemplate.NamedTemplate('customer.search')

Now write the template, using the original template via the original_template attribute from the view. In this case we add the form description in a slot:


  

A nice description

Remembering form values

The form now submits to itself. Currently its values are forgotten. So we fix that:

from zope.formlib.form import setUpWidgets

class Search(formbase.SubPageForm):
    ...
    # We set an empty prefix, otherwise we would end up with
    # 'form.SearchableText', etc, in the request while the code here and in
    # results.py expects 'SearchableText'.
    prefix = ''

  def setUpWidgets(self, ignore_request=False):
      """From zope.formlib.form.Formbase.
      
      Difference: pass extra data=self.request.form.
      """
      self.adapters = {}
      self.widgets = setUpWidgets(
          self.form_fields, self.prefix, self.context, self.request,
          form=self, adapters=self.adapters, ignore_request=ignore_request,
          data=data)

While we are here, let's fix a possible unicode error. There is a difference between getting a value from the request or from request.form when unicode is involved:

>>> request.form.get('SearchableText')
u'\xff'
>>> request.get('SearchableText')
'\xc3\xbf'

In the method zope.app.form.browser.textwidgets.TextWidget._getFormInput() the value is fetched from the request and not request.form. So you get a UnicodeDecodeError. We need to guard against that:

def setUpWidgets(self, ignore_request=False):
    """From zope.formlib.form.Formbase.
    """
    self.adapters = {}
    fieldnames = [x.__name__ for x in self.form_fields]
    data = {}
    for key in fieldnames:
        value = self.request.form.get(key)
        if value is not None and not value == u'':
            data[key] = value
            self.request[key] = value
    
    self.widgets = setUpWidgets(
        self.form_fields, self.prefix, self.context, self.request,
        form=self, adapters=self.adapters, ignore_request=ignore_request,
        data=data)

Batching on the results page

The search form is working fine now. So we move on the the search results. The results template can be quite simple: a ul with an li for every search result if that is enough for you. The only interesting part is batching:

...

So we need a view with attributes has_results and batching. This is the base of the browser view:

from plone.app.content.batching import Batch
class IResults(Interface):
    """List of Products.
    """

    has_results = Attribute("The search has matching results.")
    items = Attribute("List of Products")
    batch = Attribute("Batch of brains")
    url = Attribute("URL for this context")


class Results(BrowserView):
    implements(IResults)

    # We want to use the batching.pt file from plone.app.content, but
    # that has a few drawbacks so we copied it ourselves.
    batching = ViewPageTemplateFile('batching.pt')

We overwrite it because it has one flaw. It has a question mark where we want an ampersand, so we can add more options in the url. It was:

tal:attributes="href string:${view/url}?pagenumber=${batch/previouspage}&sort_on=${request/sort_on|string:getObjPositionInParent}">

and we change that to:

tal:attributes="href string:${view/url}&pagenumber=${batch/previouspage}&sort_on=${request/sort_on|string:getObjPositionInParent}">

This change means we can add our filled in search terms (path, SearchableText) to the url method of the view:

@property
def url(self):
    """Base url, needed by the batching template."""
    url = self.context.absolute_url()
    terms = ["%s=%s" % (key, value) for key, value in self.search_filter.items()]
    query = '&'.join(terms)
    return url + '?' + query

We add a search_filter to our view that gets the filled in values from our search form:

@property
@memoize
def search_filter(self):
    """Construct search filter.

    Only add valid search terms from the request.
    """
    context = aq_inner(self.context)
    form = self.request.form
    search_filter = {}
    for key in ['path', 'SearchableText']:
        value = form.get(key)
        if value is not None and not value == u'':
            search_filter[key] = value
    # When viewing a Brand, add its path to the filter.
    if search_filter.get('path') is None:
        if IBrand.providedBy(context):
            search_filter['path'] = '/'.join(context.getPhysicalPath())
    return search_filter

Incidentally, our search form and results will also be used on the view of a Brand, which is why the last few lines were added.

Our view needs a batch property:

@property
def batch(self):
    """Batch of Products (brains).
    """
    context = aq_inner(self.context)
    # We only search for Products.  We need to copy the
    # search_filter as we need it original contents somewhere
    # else, without the portal_type.
    search_filter = self.search_filter.copy()
    search_filter['portal_type'] = 'Product'
    catalog = getToolByName(context, 'portal_catalog')
    brains = catalog.searchResults(search_filter)
    batch = Batch(items=brains, pagesize=10,
                  pagenumber=self.pagenumber, navlistsize=5)
    return batch

We use that batch in the shown batching.pt template, but also in our own view, as we do not want to pass brains to our template, but a nice list of dictionaries:

def items(self):
    """List of Products.

    Here we get some info for all the Products that are in the
    current page of the batch.
    """

    batch = self.batch
    items = []
    for brain in batch:
        product = brain.getObject()
        info = dict(
            url=product.absolute_url(),
            title=product.Title(),
            isbn=product.getIsbn(),
            )
        items.append(info)        
    return items

There you have it: a search form with batched results on the same page. And all thanks to the authors of zope.formlib and plone.app.content.batching!

Plone 3.0 and instancemanager

published Oct 16, 2007 , last modified Jan 30, 2008

Two quick instancemanager configs for using Plone 3.0

Buildout is all the rage now in Zope and Plone land. I am using it more and more myself. But instancemanager still works fine for me too. I heard someone having problems getting instancemanager working with 3.0, so here are two config files for that.

First with the official Plone 3.0 tar file:

zope_version = '2.10.4'
archivebundle_sources = [
    dict(url='http://plone.googlecode.com/files/Plone-3.0.tar.gz',
         internalBundles=['Products', 'lib']),
    ]

internalBundles is the key here. It tells instancemanager that the tar ball has two directories that have products in them. Not those directories but only their contents should be put in the correct directories in the instance. Since one of those directories is called lib instancemanager knows that it has to put its contents in the lib directory of the instance instead of in Products.

And here is a version using svn checkouts, also showing an alternative way to specifying the dictionaries:

zope_version = '2.10.4'
symlinkbundle_sources = [
    {'url': 'https://svn.plone.org/svn/plone/bundles/3.0'},
    {'url': 'https://svn.plone.org/svn/plone/bundles/3.0-lib',
     'pylib': True},
    ]

Here pylib tells instancemanager to link the contents into the lib/python directory.

GetPaid - Making money with Plone

published Oct 12, 2007 , last modified Jan 30, 2008

Plone conference 2007, Naples. Speaker: John Lenton Company: Except

Plone conference 2007, Naples. Speaker: John Lenton Company: Except

(with many thanks to Kapil Thangavelu (who helped prepare but could not give this presentation) and Chris Johnson)

Show of hands: about half of the people attending are integrators and the other half developers.

E-commerce is hard. There is astounding complexity: payment processing, shipping, discounts, taxes, etcetera. Every one of those items is essential to somebody. So if you want to please everyone you need to have all of that in your system.

Getpaid does not do that. It tries to be a minimal framework. It uses zope 3 style python packages on one side and plone content integration and UI on the other. It has no content types of its own, so you are not forced to use say GetPaidBook or GetPaidDVD because they do not exist. You can use the zope 3 adapters of getpaid on your own content types or the core Plone content types. This means it is pretty small and clean.

Getpaid is currently in production on a few sites. There is:

  • Content integration (like mentioned above), you can make content items payable.
  • a shopping cart
  • pluggable order workflows. It uses the hurry.workflow package. It adds an adapter and parallel workflow.
  • order history
  • order types: shipable, donation orders. Classify and introspect orders.
  • order management: you can tracks workflow changes, searching, reports, import and export.
  • payment processors:
    • authorize.net: synchronous payment, for US-only merchant accounts
    • ogone.com: asynchronous, for EU-only merchant accounts probably.
    • more to come
  • order payments: workflow driven. Synchronous, asynchronous, and someone wants to do offline payment to (that would be me :), ed.)
  • admin interface: very simple, needs more UI love
  • internationalisation and localisation. getpaid.core and PloneGetPaid are mostly i18n, for about five languages now.

5 examples of best technologies that we use:

  • multistep wizard views
  • using formlib and viewlets all round
  • custom indexing and querying
  • zope3 workflow engine (hurry.workflow) which is unbelievably simple
  • doctests, event subscibers and more

Work in progress:

  • We currently support any currency, but only one at the same time, so we want to do something there.
  • product catalog
  • UPS shipping
  • Plone 3
  • etc

Wishlist: pure zope 3 UI, as that is the only part that is Plone dependent now.

We have a sprint here this weekend, so join us to finish version 1.0.

Customizing

  • Use a policy; see Martin Aspeli's talk.
  • Come to the sprint and learn by writing an example. :)
  • Wait until after the sprint and learn from reading the example.

Everything is pluggable, as it is pure zope 3, which helps a lot in customizing.

Questions?

Is there SQL database integration? No, but you could use sqlalchemy.

What do you mean with a product catalog? I mean that you can query for all products in a category. Could be done by querying the portal_catalog, but that would tie it to Plone, which we want to avoid.

Is there a demosite? Yes: http://dev.plonegetpaid.com

Untested code is broken code

published Oct 12, 2007 , last modified Jan 30, 2008

Plone conference 2007, Naples. Speakers: Philipp von Weitershausen and Martin Aspeli

Plone conference 2007, Naples. Speakers: Philipp von Weitershausen and Martin Aspeli

"I know I should write tests, but I will just let my customer or the community do that for me. They should just report bugs." Not a good idea, especially if your app only fails in some code that only gets executed by your boss. He will not be happy if it fails.

So first write a test, then start coding. That is Test Driven Development (TDD). Core developers should know that and should not need to follow this talk. :)

With TDD you catch design mistakes early on. It can keep your code simple: if the tests all pass, why would you add more code because you think you might need it in the future, complicating the code?

Tests should exercise API. Demonstrate how to use them. Other developers are happy with this kind of tested documentation. You do this with doctests. They look like a python interpreter session with some paragraphs in between. We use restructured text, so it can be turned into html and pdf and is still readable on its own.

We also have Documentation Driven Development (DDD). You write doctests first. You write a science fiction story. You tell it to an imaginary user. Just use the words "we" and "you". You can put that story on the product homepage (plone.org, PyPI). It helps you keep in mind who you are actually talking to, instead of making documentation that only you or some core developers can understand.

Now Martin Aspeli gave a demo. He just wrote to the attendees list giving some links. I am sure he will not mind if I quote him here so others can read it too:

I've checked in the examples to my two talks in the collective:

- http://svn.plone.org/svn/collective/examples/policy-buildout/ contains the sample code for the "Extending and customising Plone 3" talk. The slides are there too, in PDF format.

- http://svn.plone.org/svn/collective/examples/example.tests/ contains boilerplate examples for the different types of tests covered in the "Untested code is broken code" talk, which I gave with Philipp von Weitershausen this morning. This may be useful as it has well-commented test setup examples. I always copy and paste this stuff between my products, so now hopefully we can keep this in one place.


Open Source and Open Society: using Plone to build community online

published Oct 12, 2007 , last modified Jan 30, 2008

Plone conference 2007, Naples. Speaker: Tom Moroz From: Open Society Institute

Plone conference 2007, Naples. Speaker: Tom Moroz From: Open Society Institute

I hope at the end of this talk you will see that you are all part of the open society.

I am from the Open Society Institute (OSI; do not confuse this with the Open Source Initiative). We aim to shape public policy to promote democratic governance, human rights, and economic, legal, and social reform. On a local level we have several initiatives to support education and other things. We build alliances across borders.

We have national foundations is 32 countries. Founder is billionaire George Soros from Hungary. He was effected by the philosophy of Karl Popper. We have programs and give grants in over 60 countries.

Key initiatives

Some of our key initiatives are:

  • advocacy
  • education, scholarships, school building
  • media, information
  • local government, make them more effective
  • human rights, we look at that in all the other programs
  • justice
  • public health, HIV/AIDS mostly in Africa, tuberculosis in Russia

Open Source

A broad definition of open source: a set of principles that promote open access to the design and production of goods and knowledge.

This applies to software development, but also government, media (blogs are part of this), education, and business.

Key elements of both open source and open society:

  • Knowledge is provisional and fallible. You have to check it. You have to be able to check it. An open source release is not going to be perfect in the first release. That release brings more people to the software and they check how good the software/knowledge is.
  • Responsiveness: fix reported bugs quickly. Get a deep understanding of the problem and then fix it, which is the same in open society.
  • Transparency: the code is open or the information is open. We publish numbers of how much is being spent where to rebuild Iraq.
  • Pluralistic and multi-cultural. We respect minorities. (Plone cares about internationalisation, ed.)
  • High degree of responsibility. The individual should interpret her value. No-one is forcing anyone to do anything. You feel responsible anyway. You are involved and want to be a good member of the community.
  • Freedom and human rights are at the foundation.
  • Social mobility and a matter of openness. It does not matter where you come from.

The path to Karl

In 2001 we did an intranet study but not much was done. It was not felt needed: surely the OSI would not be needed in 2010 anymore so why build a knowledge base? We were wrong. In August 2005 we had a board meeting. Several programs were working on HIV, but not much knowledge was shared. So we needed something to help us here.

We looked at closed source options. In July 2006 we decided to go with an Open Source solution as this was in line with our own goals and vision. Plone was chosen. May 2007: launch of our website Karl, version 1.0. Next Monday we will have a new launch.

The name is the first name from Karl Popper and it is also an acronym for Knowledge And Resource Locator. About 160 communities are already using Karl. We are amazed at that. People were not at all resisting the change. They wanted to use it soon.

The path forward

OSI shows the future global potential of collaboration. We are like a caterpillar that needs to change into a butterfly. It takes people collaborating in order effect that change.