Weblog

published Nov 03, 2021, last modified Nov 04, 2021

Templating Systems Compared and Contrasted

published Jul 05, 2006, last modified May 18, 2007

By Chris Withers from Simplistix Ltd

Independent Zope and Python consultant. Using Zope and Python since 1999.

Python 2.4 gives string templating, which is better than e.g. print statements.

Quixote - PTL:

import quixote quixote.enable_ptl()

Positives:

  • allows template re-use

Negatives:

  • broken html possible
  • what would an html-only person think?

Zope - DTML:

  • It is a generic scripting language
  • Highly tied to acquisition
  • Tell the html writers to leave the dtml variable alone and they can do the rest.

Positives:

  • simple templates
  • not just xml/html

Negatives:

  • acquisition
  • one big namespace
  • not good with dreamweaver and friends

Zope - ZPT

Well known in the Zope community.

Positives: - Clean namespaces

  • limits you from introducing business logic in templates
  • source is valid xml/html

Negatives:

  • 2 or 3 new languages
  • macros are confusing, especially nesting them
  • very tied to Zope
  • tied to generating xml/html

Cheetah

  • not limited to xml
  • not tied to one framework

Positives:

  • familiar, python-like
  • compact
  • caching

Negatives:

  • not enough python-like, so yet another language
  • we're not helping the web monkeys

Nevow: really aimed at twisted.nevow

Positives:

  • Clean separation of data, logic and presentation
  • does help the web monkeys

Negatives:

  • very verbose
  • meant for Twisted and yes it is twisted!

Others: - Myghty

  • Preppy
  • XSLT
  • PyMeld, meld2, meld3
  • Kid attribute language like ZPT

Types: preprocessor, class based, attribute languages, DOM based

We want to:

  • Replace attributes, values and tags.
  • Repeat tags
  • Remove attributes, values, tags and node
  • html quoting

What if we did that?

  • simple
  • no new languages
  • work with raw html

Twiddler

  • n = t.getById(Something)
  • n.replace(value, tag, **attributes)
  • n.repeat(value, tag, **attributes)
  • n.remove()
  • Put code in html comments, mostly at the top.
  • t.clone() for reusing common material
  • Use name attributes:

Positives:

  • works with real html
  • no new language
  • simple as possible

Negatives:

  • verbose
  • not battle-proven
  • maybe slow

I hadn't seen meld3 when i started developing Twiddler, but they are scarily similar. It uses meld:id attribute.

But it's not just about html. Think about sending an email. Or a css file. Or things that are just not xml.

What about i18n, html quoting, stx, rst? We can do that.

See the presentation.

The Future of Python

published Jul 04, 2006, last modified May 18, 2007

Europython 2006 Keynote Talk by Guido van Rossum

Lambda is not going away, so don't worry about that. (Applause)

Python 3000 philosophy

  • Don't design a new language. No pet peaves like adding a switch statement, redoing the if statement or whatever.
  • Fix early design bugs.
  • Allow incompatible changes (within reason). We're not maximizing breakage of course.
  • Get rid of (morally) deprecated features. Old style classes are no longer the default.

Python 3000 process:

  • We have to make selections. Too many proposals compete for time.
  • Maintaining python 2 is a big issue. Back porting may be partly possible.

Release Schedule

  • First alpha: not before next year.
  • Final release: probably a year after that
  • 3.1 and 3.2 may follow soon after for fairly small but logical changes.
  • 2.6 will come, 2.7 likely, 2.9 is as far as we'll go.

How incompatible can it be?

  • new keywords allowed
  • dict.keys(), range(), zip() won't return lists. This kills dict.iterkeys(), xrange(), itertools.izip()
  • All strings will be unicode. Mutable bytes datatype, represented as a character array underneath. For strings always use encoding.
  • Binary file I/O redesign. Use bytes objects when reading from or writing to files.
  • Drop '
  • as alias for !='.
  • But not:
    • dict.keys instead of dict.keys()
    • change meaning of else-clause of for/while
    • change operator priorities

How to migrate code?

  • Perfect mechanical translation will not be possible. Many changes are semantic rather than syntactic. So a live programmer will need to look at it. Tools like pychecker can help though.
  • Most likely approach:
    • Use pychecker-like tools to do an 80 percent job.
    • Create python 2 version that warns about dead ends.

Python 3000 will not:

  • have macros, programmable syntax, etc
  • add syntax for parallelization (use zip())

Python 3000 features: read PEP 3100 for a large list. See the peps

Basic cleanup:

  • kill classic classes
  • all exceptions must derive from BaseException
  • int/int will return a float
  • remove last differences between int and long
  • kill:
    • sys.exc_type()
    • dict.has_key() (use in)
    • file.xreadlines() and actually file and readlines()
    • apply(), input(), buffer()

Minor syntactics:

  • exec becomes a function again
  • kill `x` in favor of repr(x)
  • range() will behave like xrange()
  • xrange() dies
  • zip() becomes izip(0
  • lambda lives! The lambda lovers tried for a year to come up with a better version, but failed. But it has its uses, so it can stay.

String types reform:

  • bytes and str instead of str and unicode
  • maybe bitwise operations on bytes, but that might add too much to to API.
  • All data is either binary or text.

New standard I/O stack

  • C stdio has too many problems
  • bytes/str gives an opportunity to fix all this.

Print becomes a function!

  • print x, y, z becomes print(x, y, z)
  • print >> f, x, y, z becomes print(x, y, z, file=f)
  • This makes it much easier to switch use logging.
  • Skip the space or newline? Use printf() or printraw() or whatever we come up with.

Drop default inequalities: <= and friends are not useful between unrelated objects of different types. '== and !=' should always be possible.

Thanks to the Google team for T-shirts and sponsoring!

If it is possible to make a 2.x version that is somehow compatible with 3.0 then that would be nice.

Panel: Getting Started with the Zope Foundation

published Jul 04, 2006, last modified May 18, 2007

by Jim Fulton, Martijn Faassen and Aroldo Souza-Leite

The Zope Foundation was legally formed in March 2006. Initial board since June: Aroldo Souza-Leite, Martijn Faassen, Rob Page and Jim Fulton.

We have strategic developers, consumers, solution providers, associate members and board members. It is a non-profit organisation.

Purpose: advance the community.

Practical: hold the copyright, trademark, manage zope.org.

The foundation is you! The community is doing the work and should keep the responsibility. We'll try to help and not get in your way. The foundation helps make things easier for others.

Initial committees: membership, finance, Zope management organisation and the www.zope.org web site. We need more volunteers on all committees.

Q: What services should the Zope Foundation offer to developers of the Zope platform?

Martijn: We have subversion, mailing lists, wikis, issue tracker. Not all our services should necessarily be based on Zope, e.g. an issue tracker need not have been made in Zope.

Aroldo: it is a frame. We seek commitment between parties. More meta technical and legal. It is a movement.

Jim: I don't want to be a Zope Pope anymore and I am not. I tried to add a page for a talk i held earlier and I had to manually add a Folder as Manager on the zope site, which feels a bit dirty. So we need something better there.

Christian Theune: +1 on better bug trackers.

Martijn: Help people to not build Zope specific software if it can instead be used outside of Zope. Right now many people don't even try it.

Q: What should the Zope Foundation do for people with an interest in Zope?

Martijn: Give people clearly the impression that Zope and its projects are there to stay, that they are stable. Give them peace of mind when using it.

Aroldo: The Zope Foundation could be a frame for evangelising. A good place to collaborate with other institutions, for example the python.org organisation. We should take care of documentation.

Jim: +1 on all that.

Chris: How are new members invited?

Jim: Dieter has been invited.

Martijn: We had a board decision for that.

Jim: We'll add some info on the site.

Christian: There should be info on where and when conferences are, which companies in the neighbourhood can help customers.

Mark Hamilton: Make the role of Zope clear. People may now step in at either the python or the plone level without really realizing what the Zope level is about.

Several in the room: A sort of Zope newsletter would be good, maybe some development news? There is Planet Zope with an accompanying blog and a planetzope del.icio.us tag. Done by Philipp von Weitershausen?

Jim + Martijn: There are several Zope related projects: zope2, zope3, Five (well, part of zope2), zodb. Maybe we need to split those.

Aroldo: the Zope Foundation can help you decide if you want to join the main Zope effort or start a separate project.

Room: more centralised support for small projects. More info about those projects on www.zope.org, instead of just a subversion repository.

Jim: I agree.

Room: I like the idea of Popes for several projects actually.

Kit Blake: When I talk to a customer, it helps to have a Zope Foundation, because I can point them to a web site. The web site may be ugly, as long as it gives good info, a good image. A Zope Foundation with a good image is good and then I don't mind a messy www.zope.org.

Geir: A neat www.zope.org is good, but not necessarily a task for the Zope Foundation.

Martijn: Whose job would it then be?

Geir: It could be done without the Zope Foundation. The community can do that.

Jim: The Zope foundation owns the domain name, but we don't have the man power to handle the web site.

Tarek Ziade: How about more marketing people?

Martijn: We have to use the right marketing to the right market. So more marketing people are helpful.

Jim: When more than one person steps forward, we'll worry about how to coordinate marketing efforts.

Jim: Do we want to split Zope into more projects? So give some people far more authority and responsibility for e.g. zodb or zope3? Andreas doesn't have the authority to decide where Zope2 is going to be heading.

Russ: You are looking to spread out the load. There is no money available. Only recognition now, so you need to do that part well.

Martijn: Giving credit to people is important.

Christian: Right now, if somebody wants to modify something, maybe large changes that I am unhappy about, then I have no authority to do anything about it. So splitting could help there.

Steve: There are a few roles for a project: technical officer, security officer, some more. These may be the same person. Knowing who has these roles up front would help. Make it clear. Also: define compatibility. What versions is a software program compatible with.

Martijn: Maybe you can help?

Steve: I'm busy applying to become a member. :)

Jim: We are in the bootstrapping phase. I'm not sure of the requirements for new members, but we don't want to turn anyone away.

Q: What shall we do with www.zope.org?

Chris Withers: rm -rf /

Martijn: We have enough ideas, we just need some time. Identity pages for projects are the most important. We should look at TurboGears and Django, because they are doing this well. We should go low tech. Just simple pages. Steve brought up LaunchPad, so we may want to use that. Maybe just use Apache. Anything that works!

Room: Applause!

Martijn: It is a very important part of our effort.

Chris: How do I get in?

Jim: Ask Martijn.

Chris: Okay, I want in.

Christian: Okay, you are in. :)

Martijn: There are ideas, but there are some practical problems, like access to the machines that host zope.org.

Jim: Zope Corp is still hosting that. Okay, we have a few volunteers. Maybe everyone who is still left in the room here? :) We have them on film. :) We have the authority, we just need people to get involved.

Introduction to pywinauto

published Jul 04, 2006, last modified May 18, 2007

by Mark McMahon

He's a localization software engineer, uses python for interfacing with legacy applications, for tools, and automating stuff. Python is unfortunately not a standard in his company.

What is pywinauto?

It automates clicking in Windows. E.g. open up Microsoft Paint, create a new file, paint something. Hard to write tests for.

If you set the delays between commands too short, the right window may not have come up yet so you get an error. Be aware of that.

It was first written in C to analyze localized dialog boxes for GUI bugs (1998). Then moved to C++ (2000), then python (2005). Conversion to C++ took two years, to python two or three months. :)

Example:

from pywinauto.application import Application

app = Application.Start("Notepad")

app.Notepad.Edit

app.Notepad.Edit.TypeKeys("Hello World")

WinSpy or Winspector help you to figure out which classes to use, e.g.

canvas = app.untitledPaint.Afx1000000008

Pywinauto is all dynamic:

  • no dialog declaration needed
  • fuzzy matching of titles (resistance to software changes)

Easy to understand.

It could use a script recorder, integration with test tools and a merciless refactoring.

Microsoft Office is done very differently, uses other controls. You can't use pywinauto for that.

See Mark's blog

Subversioned system configuration

published Jul 04, 2006, last modified May 18, 2007

by Holger Krekel from merlinux

Linux distros need configuration and customisation. People want to modify text config files. Some changes need reviews to prevent bad things from happening. Without tool support this is fragile. Problems are then hard to track.

You can version the /etc directory with subversion or the like. But permissions and ownerships is hard to do right there. They can be different from system to system. This is intrusive. You don't want that many versioned directories.

vadm gives you indirect versioning. vadm add /etc/passwd This maps to-be-versioned files into user-specific working copies. It delegates to an underlying versioning system. It gives bidirectional transform of ownership and permission info. History and diffs are available.

vadm: using it

  • prerequisites: svn installation, sudo rights for executing user, and a repository
  • vadm init file:///sysrepo/mysystem
  • vadm add/remove/commit/diff/log path/to/configfile
  • Start using it just like subversion

Setup notifications:

  • Install a post-commit hook into subversion repo to signal admins of changes.
  • daily cronjob may commit any pending changes

We use it to control some 20 systems now. Daily auto-committing has proven to be valuable. It signals changes to others who can then review your changes. You can find out who edited particular lines.

Versioning clusters

  • this is future work
  • versioning multiple similar machines:
    • distinguish between local and global (cluster-wide) changes
    • allow groups of systems (postfix systems, subversion systems)
    • per system would take preference
  • a cluster vadm should only require ssh + local svn config
  • vadm is slow now
  • don't require a server-side daemon other than sshd?

Suggested semantics

  • a URL defines a group of versioned files.
  • each system has a stack of multiple such urls
  • url marked manual-update or auto-update

Other considerations:

  • versioning package installation information. An update of the apache package probably has a new config file. Do this in the same commit.
  • speed up versioning of hundreds of system files
  • plug in other versioning systems. Subversion is best supported now.

vadm future:

  • release planned in 2006, under GPL
  • development happens on a demand basis
  • training/support possible

Use the source