Quality assurance for third party products
Open space
When a customer has installed 25 extra products and wants me to migrate his site, I want to be able to give him a good idea of how hard that is going to be, how much trouble, without needing to be convincing. Do the products have tests? Is there good test coverage?
On pypi you can see how many times a package has been downloaded. Gives an indication.
I want at least some automatic information, like number of tests, coverage.
It takes about one or two hours per package to configure a buildbot. You can grab statistics from that. Maybe automatically run pyflakes and pep8 tests.
Each package has an owner.
We can calculate a value for a package, like lines of code times the coverage. Can be touchy.
Perhaps create a directory buildbot in the collective, where people can put buildout config files for their packages.
It is better to have a src directory in a package instead of having the code next to the setup.py file.
Add automatically collected information to PloneSoftwareCenter.
Could be tricky to prevent someone uploading code to the collective that is then executed by the buildbot and for instance starts sending spam.
Show translation coverage.
We should not do rating (yet).
Look at the pylint package.
You can make pyflakes less complaining.
pep8 and pyflakes tests could be optional.
Action points, we can sprint on this:
- get buildbot infrastructure
- add fields to PSC for code coverage, and number of tests; could be annotations
- POST request for writing the info there from the buildbot
- scripts to parse the buildbot output and post the info