Weblog

published Nov 03, 2021 , last modified Nov 04, 2021

Report on eXtremeManagement

published Jun 21, 2007

I finished a report for school about work I did on eXtremeManagement at Zest Software the past months. Biggest parts: getting xm to run on Plone 3.0 and use Zope 3 technologies. Download the report or read the preface, abstract and conclusion.

I made a report (link to pdf) for school about the eXtremeManagement project management tool for Plone. Now I just need to finish two other classes and then I am done studying. Read on for the preface, conclusion and abstract.

Preface

This report describes my final assignment for my study Informatics at the Rotterdam Institute for Informatics Studies (RIVIO) of the Hogeschool Rotterdam. The assignment lasted from February till June 2007. It was carried out for web development company Zest Software in Hoogvliet, The Netherlands. I have been working there since November 2005. Previous reports for school of my work for Zest Software (including this report) can be found on my website.

The main subject of this report is eXtremeManagement. This is a project management tool for the open source content management system Plone. At Zest Software we use this tool on a daily basis to keep track of what our customers want and how much time we have worked for them.

This eXtremeManagement tool can use some updates, which is the goal of this assignment. The focus is on the underlying technology: using more and more features made available by newer versions of Plone and the Zope web development framework that Plone is built on. Also some user interface improvements will be made.

I thank Hans Manni from RIVIO for keeping me on track for finishing my study. I thank Aad van Raamt for being the second teacher next to Hans on the committee. I thank Jean-Paul and Esther Ladage for giving me the opportunity to work on eXtremeManagement for five months. I thank Reinout for the photo on the front page. I thank my colleagues from all over the world for their feedback, their own additions to eXtremeManagement and for making Zest Software a very nice team to work and have fun with.

Conclusion

eXtremeManagement has been given a thorough cleanup. It runs on the current Plone 3.0 beta. Where useful, Zope 3 technologies have been put to good use. Personally, I have learned a lot about these new software versions and I am sharing that knowledge on my weblog and in mailing lists. At Zest Software we are happily using the latest version of eXtremeManagement and are full of ideas for further improvement.

Some recommendations and further actions to take:

  • Release a new version of eXtremeManagement soon: it is ready.
  • Building on the foundation laid during this assignment, do more work on the user interface. There are lots of ideas floating around. See Future plans (Appendix A).
  • Once Plone 3.0 is officially out, copy the current subversion trunk to a maintenance branch and continue development for Plone 3.0 only on trunk.

In closing, I will say I had a great time with eXtremeManagement and its users and co-developers on the mailing list. eXtremeManagement is a rocking product, ready to handle the future.

Abstract

This report describes my final assignment for school, which is: improve the eXtremeManagement tool: a project management tool based on Extreme Programming principles and running on content management system Plone.

The Introduction (chapter 1) paints the landscape of this assignment. What is Zope? What is Plone? What is eXtremeManagement? How do they fit together?

I made some improvements to the User Interface (chapter 2). Most of the original ideas there were not implemented however, for various reasons, ranging from simple lack of time to fresh insights that invalidated the original plan. The focus of the assignment was shifted to improving the core, instead of the front door.

Plone 3.0 (chapter 3) tells the tale of getting eXtremeManagement ready to run on the new (still in beta status) version 3.0 of Plone. I did some standard fixes applicable to all third party Plone products. I also did some other changes that were found to be needed. Finally I added an improvement to core Plone to make this upgrade easier for other products.

With Zope 3 (chapter 4) we come to the heart of the matter. More than originally envisioned the focus needed to be put here. I added marker and functional interfaces. I created browser views to make a clearer Model-View-Controller distinction. I introduced annotations for keeping track of estimates and hours worked. All three work together to make a far cleaner version of eXtremeManagement than was there at the beginning of this assignment.

I draw the Conclusion (chapter 5) that eXtremeManagement is clean and future-proof and that I have learned a lot in this assignment.

Rapport over eXtremeManagement

published Jun 21, 2007

Ik heb een rapport ingeleverd voor school over wat ik de laatste tijd bij Zest Software heb gedaan aan eXtremeManagement: een projectmanagement tool voor het CMS Plone. Download het rapport (Engels) of lees de Nederlandse samenvatting.

Download het rapport (pdf, Engels) dat ik heb gemaakt over eXtremeManagement voor Plone. Of lees hieronder de Nederlandse samenvatting.

Samenvatting

In dit rapport doe ik verslag van mijn afstudeeropdracht voor de studie Informatica aan de Hogeschool Rotterdam. Ik verbeter de eXtremeManagement tool: een project management tool gebaseerd op Extreme Programming principes en draaiend op het content management systeem Plone.

De Introductie (hoofdstuk 1) schetst de achtergrond van deze opdracht. Wat is Zope? Wat is Plone? Wat is eXtremeManagement? Hoe passen ze in elkaar?

Ik heb wat verbeteringen aangebracht in de gebruikersinterface (hoofdstuk 2). De meeste oorspronkelijke plannen werden niet uitgevoerd, om redenen uiteenlopend van simpel tijdgebrek tot nieuwe inzichten die het originele idee onderuit haalden. De focus van de opdracht verschoof daarmee naar het verbeteren van de kern, in plaats van het mooi maken van de buitenkant.

Plone 3.0 (hoofdstuk 3) vertelt het verhaal van het geschikt maken van eXtremeManagement voor de nieuwe versie 3.0 van Plone, die overigens nog de beta status heeft. Ik heb wat standaardaanpassingen doorgevoerd die alle Plone producten van derden moeten doen. Ik heb ook andere veranderingen gedaan die nodig bleken te zijn. Tenslotte heb ik een verbetering aan de kern van Plone bijgedragen om deze aanpassingen voor 3.0 simpeler te maken voor andere producten.

Met Zope 3 (hoofdstuk 4) komen we bij het hart van de zaak. Meer dan oorspronkelijk gedacht, moest de focus op dit deel gericht worden. Ik heb markeer- en functionele interfaces toegevoegd. Ik heb browser views gemaakt om een duidelijkere scheiding te brengen tussen data, presentatie en logica (Model-View-Controller). Ik heb annotaties ingebracht voor het bijhouden van inschattingen en gewerkte uren. Deze wijzigingen leveren samen een veel schonere versie van eXtremeManagement op dan er was bij de start van de afstudeeropdracht.

Ik trek de conclusie (hoofdstuk 5) dat eXtremeManagement schoon en toekomstklaar is en dat ik veel heb geleerd bij deze opdracht.

eXtremeManagement on Plone 3.0

published Jun 18, 2007 , last modified Jun 19, 2007

Part of my big final assignment of my study Informatics is getting eXtremeManagement to run on Plone 3.0 (beta). That is functioning nicely now. Some remarks about what I needed to do for that.

eXtremeManagement used to be fit for Plone versions 2.1 and 2.5. We want it working with Plone 3.0 too (well, trunk should work on 2.5 and 3.0), as that is expected to be released in spring or summer 2007. At the time of writing (halfway through June) Plone 3.0 is in beta status. So some things may still change that eXtremeManagement needs to react to. But currently all automatic tests of the eXtremeManagement pass for Plone 2.5 and 3.0. Plus: a test of migrating the current database of the Zest projects site to Plone 3.0 worked fine. So what was done to reach that point?

General actions

The general steps that every product needs while upgrading from Plone 2.5 to Plone 3.0 are described in the Upgrade guide on plone.org.

Some steps were already taken for Plone 2.5, like setting up the workflows with GenericSetup. Some steps are very simple. An example is that this:

get_transaction().commit(1)

needs to be replaced with:

transaction.commit(1)

The other ones that needed checking in eXtremeManagement were:

  • Searching users/groups via the Membership tool is deprecated; it turns out eXtremeManagement does not use any of the deprecated methods, so fifteen minutes of checking was all that was needed here.

  • Portlets have a new infrastructure. Needed action: check that our classic portlets still work in Plone 3.0. They turn out to work fine. In the future we may want to investigate what the benefits of switching to the new portlet system are. A downside is that this would be incompatible with Plone 2.5, so it would require a separate branch.

  • main_template.pt now uses Zope 3 viewlets. eXtremeManagement is not using an own version of that template, so no action was needed. In fact, we were not even using that template, which does not sound like the best idea. So I changed our templates to use the page layout as given by the main_template.

  • The "Sharing" tab is now a global action. In previous Plone versions you had to explicitly add a Sharing action to your content type if you wanted to see the sharing tab when viewing that content in the Plone user interface. On the sharing tab you can share this content with others. Specifically in the case of eXtremeManagement: we have this tab on Projects, so we can give developers the Employee role and assign Tasks to them.

    On Plone 3.0 this action is now always available. This means that currently some of our content types actually list this tab twice, as they explicitly add it to themselves already. The problem now is that you cannot actually do this correctly on Plone 2.5 and 3.0 at the same time: either you have no sharing tab in 2.5 or you have two sharing tabs in 3.0. Since 3.0 is beta still, I opted to have it working correctly in Plone 2.5 only. Once Plone 3.0 is officially released, we can easily fix this on a branch. Or probably: at that point we can move 2.5 compatibility to its own branch and have trunk specifically for 3.0 support.

More fixes

There were some other steps that needed to be taken before eXtremeManagement would work on Plone 3.0. Some might need to be added to the general list and some may be specific for eXtremeManagement.

Viewing content

When you have a Content Management System like Plone it is nice if you can actually see some content. At first try after migrating the current Zest database to Plone 3.0 on a local test instance, none of our content types were actually visible. Change set 43353 lists the changes that were done to make ProjectFolders viewable again. (Change set 43354 has the changes to the other content types.) Some lines could be removed from content/ProjectFolder.py as they were already in the GenericSetup profile. That profile needed some changes too, in this case the file profiles/default/types/ProjectFolder.xml. The most important are the additions of the aliases. Those lines should now look like this:

It could be that this is only needed because our types inherit from OrderedBaseFolder and not from e.g. ATFolder. If your own content type is also not visible on Plone 3.0, I would advise you to add these aliases as a first try.

Actually, the properties alias is superfluous on Plone 3.0 (but not on 2.5, so we will keep it for now). On 3.0 the properties of content can be reached via the normal edit form.

Portlet classes

Some html classes have been deprecated, so you should not use them in your templates anymore. Specifically, get rid of portletItemLast and lastItem in portlets. See change sets 43533 and 43987. And get rid of portletDetails in favour of portletItemDetails. See change set 43555. Adding a header and footer like in 43535 is also nice:


  
    Portlet title
  


...


  
  








Workflow actions


The general list of things to do when preparing your product for Plone
3.0 already mentions workflows: they need to be done in GenericSetup.
That was already done for eXtremeManagement.  But this turned out not
to be enough.  Our content types had a drop down list with workflow
actions, but they were not clickable.  So you could not do any
workflow actions, at least not from the usual drop down workflow menu.
This has been put right in change set 43600 by adding a url to the
workflow action, instead of having it empty (which worked fine in
Plone 2.5).  This is the change for
profiles/default/workflows/eXtreme_Iteration_Workflow/definition.xml:



-  Finish
+  Finish



But this means that most add-on Products for Plone have no functioning
workflow in Plone 3.0, assuming that most Products have an empty url
for that action.  This also means that when you change the id of a
transition id, you also need to remember to change the url, else the
old transition is still done or at least tried.  In fact, did you
notice that exact mistake in the code above?  Probably not.  I know I
did not. :)  The transition id is complete, so the action url
should have ended with that as well and not with finish.


Can we make this simpler?  Yes.  I provided a fix for core Plone so it
would accept the empty urls that most workflows have.  I only just got
commit privileges to core Plone, partly based on this proposed fix
probably, so change set 15335 is my first actual commit there.  This
meant that I could revert my earlier change to eXtremeManagement.







        


      

        


        




      


      

      

    

  


              
                  
                
                  
                    
                
  

    


      


      

        Minimally overwriting a content type
      


      

  

  
    
      published
      Jun 08, 2007
      ,
    

    
      last modified
      Jun 09, 2007
    
  

  





      
If you want a new content type slightly different from an existing one, you might be able to use only GenericSetup to accomplish this. Update: 'diff -u' added; alternative way using the ZMI added; point to TemplateContentTypes by Rocky.


      


        


        


          

        

          
Sometimes you want to use an existing content type, but with slightly different settings.  You might have a client who is technically happy to add a Document to a Folder, but who would rather add Eggs to Baskets.  In other words: he wants to use the Document and Folder content types, but with a different name.  Also, he might want to put only Eggs in his basket, and not Images and Files. You might be able to do this with zero lines of python code! The trick is that you can use GenericSetup to do your tweaking.
You make a new product: basket. You put that in your Products directory. You add an empty __init__.py. You add a profiles/default directory and register it in configure.zcml:


    xmlns="http://namespaces.zope.org/genericsetup"
    i18n_domain="genericsetup">
        name="default"
      title="Basket profile"
      description="Basket"
      provides="Products.GenericSetup.interfaces.EXTENSION"
      for="Products.CMFPlone.interfaces.IPloneSiteRoot"
      />


You add types.xml in profile/default:





After this, you copy CMFPlone/profiles/default/types/Folder.xml to profiles/default/types/Basket.xml. Then you change a few lines there . The diff  (update: the 'diff -u' as rightfully requested by Reinout) is here:


 
-+    meta_type="Factory-based Type Information with dynamic views"
    xmlns:i18n="http://xml.zope.org/namespaces/i18n">
- Folder
+ Basket
  -    name="description">A folder which can contain other items.
+    name="description">A Basket for Eggs
  folder_icon.gif
- ATFolder
+ Basket
  ATContentTypes
  addATFolder
  folder_listing
  True
- False
- 
+ True
+ 
+  
+ 
  False
  folder_listing
  
   
-  
-  
   
  
  


Then you restart Zope, go to portal_Setup, in the Properties tab select your profile, in the Import tab select 
    "Types Tool", press "Import selected steps" and you are done. Well, we have not looked at how to do this for normal Documents too, adding a content type Eggs based on them, but that should be the same technique, really.


Or as Wouter Vanden Hove says in a reaction: you can also go to portal_types, copy an existing content type (Document) to a new one (Egg), change Egg to your liking, go to portal_setup, export the Types Tool, unzip the resulting file in profiles/default and then remove all that is not an Egg or Basket from types.xml and remove all files other than Egg.xml and Basket.xml from the types directory. Those are a lot of steps, but it is probably takes longer to write this down than to actually do this. ;-)


If you really use this and submit this to a code repository like subversion, I recommend to first do a commit with the original Document.xml and Folder.xml (though renamed to Egg.xml and Document.xml) so you can more easily see what was actually changed.


By the way, I was trying this out and then I was triggered to write this blog entry by some commits I saw from Rocky in the collective: a new product TemplateContentTypes. That is a new product that he created on behalf of ServerZen Software and
Zest Software. It does something similar but also some more. I have not tried it yet. It probably contains some things that would be useful to add to the files above too, like profiles/default/factorytool.xml. Here is part of the README.txt of TemplateContentTypes:




A product designed to be used as the template or skeleton layout of a new set of types to completely replace ATContentTypes.  Each custom type subclasses it's ATCT's equivalent.
Once the product has been installed inside a site, all standard actions that use/create/modify ATContentTypes based content will instead use the new template types.




Now create a Basket, add an Egg and enjoy!



        


      

        


        




      


      

      

    

  


              
                  
                
                  
                    
                
  

    


      


      

        Poi, intelligenttext and Plone 3
      


      

  

  
    
      published
      Jun 06, 2007
      ,
    

    
      last modified
      Jan 30, 2008
    
  

  





      
Poi has seen some interesting changes recently.  Which branch or bundle can you use in combination with which Plone version?  An investigation.


      


        


        


          

        

          


Introduction


Poi is an issue tracker for Plone.  The code is in the collective.


The eXtremeManagement product from Zest Software and others depends on
Poi.  The biggest reason: Daniel Nouri has added a content type
PoiTask to eXtremeManagement.  You can add such a PoiTask to a Story
and there easily link to one or more Poi issues that should be fixed.
So I am interested in what happens with Poi and I occasionally add
some code or run some tests on various Poi versions.


Poi has several branches.  The ones that interest me here are 1.0
and plone3-support.


Poi has several dependencies: AddRemoveWidget, contentmigration,
DataGridField and intelligenttext.


It can be annoying to have to hunt down where all dependencies can be
downloaded and which version is needed.  To make it easier on users,
Poi has subversion bundles for this.  The ones that interest me here
are 10 and plone3.  The bundles have in common that all
dependencies are using trunk.  So what are the differences?  And with
which Plone version can you use them?






Testing Poi


First of all: I improved the tests of Poi slightly, on all three
branches that I investigate here.  When you have an smtp server on
your localhost, running the Poi tests would actually cause a few emails
to be sent.  They go to non-existing email addresses, so you (or some
administrator) will get some mail delivery failures in your inbox, which
is not nice.  So in tests/ptc.py I replaced the default mail host
with a simple mock mail host, which avoids this.  That code was taken (and
drastically simplified) from PasswordResetTool: thanks!


There is always one test that fails.  This is a known issue: a deleted
response causes stale SearchableText for an issue.  If someone has an
idea on how to fix this, that would be nice.


When below I say a branch or bundle is compatible with some Plone
version then I mean that all tests except that one pass for that Plone
version.  The test combinations of Zope and Plone that I tried were:


    

  • Plone 2.1.4 (tar ball) with Zope 2.8.9; for the tests I added
PloneTestCase 0.8.2.
    • 

  • Plone 2.5.3 (tar ball) with Zope 2.9.7
    • 

  • Plone 3.0 (subversion trunk) with Zope 2.10.3
    • 



All were using python 2.4.


Note: I did not run any tests in combination with PloneSoftwareCenter.






instancemanager


I ran the tests with help from instancemanager.  I use that tool a
lot, so I am used to it, which helps.  But I really like it for
quickly running tests for (various versions of) a product on various
versions of Plone.  Here is my .instancemanager/poi.py file, ready
for running the tests for the combination of Poi 1.0, Zope 2.9.7 and
Plone 2.5.3.  Commented out are the lines that are used for the other
combinations:


python = 'python2.4'
#zope_version = '2.8.9'
zope_version = '2.9.7'
#zope_version = '2.10.3'

archive_sources = [
    #'PloneTestCase-0.8.2.tar.gz',
    ]
symlinkbundle_sources = [
    {'source': 'Poi10bundle',
     'url': 'http://svn.plone.org/svn/collective/Poi/bundles/10'},
    #{'source': 'Poitrunkbundle',
    # 'url': 'http://svn.plone.org/svn/collective/Poi/bundles/trunk'},
    #{'source': 'Poi30bundle',
    # 'url': 'http://svn.plone.org/svn/collective/Poi/bundles/plone3'},
    #{'url': 'http://svn.plone.org/svn/plone/bundles/3.0'},
    #{'url':'https://svn.plone.org/svn/plone/bundles/3.0-lib',
    # 'pylib': True},
    ]
archivebundle_sources = [
    #{'url': 'http://heanet.dl.sourceforge.net/sourceforge/plone/Plone-2.1.4.tar.gz'},
    {'url': 'http://plone.googlecode.com/files/Plone-2.5.3-final.tar.gz'},
    ]
main_products = ['Poi']



Then I just create an instance, fill the products directory and run
the Poi tests with this single command:


$ instancemanager poi --create --products --test=MAIN



When the tests have run, I add and remove some comment signs to get
the right versions and run that command again for the next
combination.






Poi 10 bundle


    

  • This has Poi branch 1.0.  That branch was created by Daniel Nouri on
2007-05-17, before he added new functionality on trunk.
    • 

  • Compatible with Plone 2.1 and 2.5
    • 







Poi trunk bundle


    

  • This has Poi trunk, which has version number 1.1.  Since putting 1.0
on a maintenance branch, trunk has seen the addition of link
detection by Daniel (links are automatically created from #123
to issue 123 in this tracker; and r1234 points to
revision/changelog 1234 if you specified a base url for that in your
tracker).  Plus some small changes by others that I did not
investigate.
    • 

  • Compatible with Plone 2.1 and 2.5.
    • 







Poi plone3 bundle


    

  • This has Poi branch plone3-support.  That branch was created by
Alexander Limi on 2007-04-25.  Goal is to add Plone 3 support.
    

    • 

  • This bundle does not have intelligenttext.  The reason is that
intelligenttext is already available as a python module in Plone 3.
It should be in your instance, in
lib/python/plone/intelligenttext.  The other
versions of Poi throw this error, when they try to install
intelligenttext:
    

    Products.CMFQuickInstallerTool.QuickInstallerTool.AlreadyInstalled: intelligenttext

    

    This python module is not exactly the same as the Product from the
collective though.  It misses some very recent additions there, like
adding rel="nofollow" to links; this also means that the Poi
tests had to be changed a bit.  Or rather: I had to change the Poi
tests on the other two branches to fit the intelligenttext changes.
    

    Update: Thomas Mueller (deepdiver) has updated plone.intelligenttext so it is in line with the collective product.  He fixed the Poi tests accordingly.  Thanks!
    

    • 



    

  • Compatible with Plone 3.0 only.
    • 







Conclusions


    

  • On Plone 2.1 or 2.5:
      

    • For a Poi version without sudden changes that would require a
reinstall or a schema update: use branches/1.0 from
bundles/10.
      • 

    • For recent improvements: use trunk from bundles/trunk.
      • 

    

    • 

  • On Plone 3.0: use branches/plone3-support from
bundles/plone3.
    •