Jordan Baker: Plone Testing Tools and Techniques

Geared towards developers this presentation walks you through developing a simple Plone 3 product using a test-driven or test-first approach. Techniques covered include unit testing, PloneTestCase, test layers, mocking and more. This is an updated and expanded version of the talk "Hone Your TestFu" given at the Plone Symposium East 2009.

I am using Plone since version 1.0. Senior consultant and founder of Scryent, in Toronto, Canada.

Slides are at http://tinyurl.com/pttt-slides

You don't always have to test. Sometimes you are just doing a prototype. But in my opinion: if you write production code, you should write tests. It improves code quality; reduces defects; makes your code more maintainable; it nudges you into more modular, loosely coupled code as that is easier to test; it keeps you focused: if the tests pass then you are done; when you have good test coverage, you can refactor with confidence.

Test first: write the test first, then start coding. It forces you to think about requirements up front. And manual testing is slow and error-prone.

Test levels: functional/system (black box), integration (your code in combination with Plone), unit (just your code).

More tests is not always better. Those tests need to be maintained too. With too many tests, it may start taking too long to run all the tests. Focus on lots of lower level tests as they are fast to run.

Unit tests: written from the developer's perspective. Test your components in isolation; use mocking techniques when you need to test integration with other components.

When working on a particular problem, try running only the immediately relevant tests. Try bin/instance test --help for some options you can pass. When those tests pass, run all tests again to check for unexpected failures.

Integration tests: test the integration of your code with Plone. Can you create the content types you made in the actual Plone Site? Do the correct values end up in the portal_catalog? Are the extra roles you defined actually added when you install your product?

Recently I started using collective.testcaselayer to simplify test setup. It simplifies repetitive Plone testing boiler plate.

Add it in your setup.py in extras_require and tests_require. Then in the eggs of your buildout add your.package[tests].

Use roadrunner to speed up testing: it sets up the layers (the Plone Site) and keeps that available so the setup is much much faster the next time you run those tests. There are some problems with for example z3c.autoinclude. I want to sprint on roadrunner this weekend.

Doctests look like interactive python sessions with documentation/explanation added.

Functional and system tests: end-to-end tests, black box tests, often zope.testbrowser tests, or with Twill or Selenium. Perhaps smoke tests: add specific tests for the most important paths that are used in your Plone Site. Twill is a domain specific language for performing web tests. It also provides a python API.

Use mocking to e.g. patch the time.time method, or urllib, or the MailHost. Martin Aspeli created plone.mocktestcase.

Coverage: answers the question: are all code paths executed in the tests. Does not guarantee that there are no bugs in the code. Use z3c.coverage to get html reports of the coverage.

Write tests at the lowest lelel possible. Take your time, write out what you are trying to do in English first. Is it too hard to test, then simplify your code, split it out.

I want to have an open space on testing.