Martin: Testing is about Specification, not Verification

• Attendance was good -- about 80 to 100 folks. There were five notebooks present. The presenter's was some Wintel (Toshiba I think), while there were three TiBooks and one iBook in the audience. Interesting.

• Robert believes nonagile software projects are black boxes to managers. "When you can't see inside a project...you manage to the date". Everything becomes about the date. Managers attempt to manage the date by dictating ("Bad Things™ will happen if you slip") and/or motivating ("I believe in you, you can do it!").

• Agile methods provides two valuable feedback streams to managers: velocity and story points remaining. These, plus deliverables every two weeks, provide needed project control to managers. Happy managers are happy clients. We like happy clients.

• Acceptance tests are key to an agile process. Programmers can say they're done, when they really aren't. The coder isn't lying, it's just that while his module is code complete, the three it depends on aren't. Or perhaps its functionality hasn't been exposed yet. A "slice" of work (two week iteration) is only completed when the acceptance tests pass 100%.

• I was unclear on the differences between unit tests (which I adore) and acceptance tests. Robert's differences:

• Unit tests are written by the programmers.
• Acceptance tests are written by the stakeholders.
• Unit tests are small, written by the hundreds or thousands.
• Acceptance tests are larger, written by the tens or hundreds.
• Unit tests use white boxes (internal knowledge known)
• Acceptance tests use black boxes (no internal knowledge known)

• Unit tests and acceptance tests exist in one of three states: Pending (specified, but not coded yet), Failing and Passing. Both need to be automatically run regularly, otherwise they won't be run in the crush of time.

• Robert's main point is that tests can replace "requirement" documents. They are unambiguous, cannot grow out of sync with the project and are active (executable).

• The topic of GUI testing came up. Robert's response was great: you can do it, but it leads to madness. He gave an example of a project stuck in the 1980s with a DOS-style pseudo-GUI that hasn't been modernized largely because the tests work by stupidly poking at the GUI. Select item 3 of menu 2. Ensure text field 9 of dialog 43 is "Customer Name". And so on. Their GUI was effectively frozen in time, held captive by passing tests!

• Finally, Robert demonstrated FitNesse, ObjectMentor's open source Wiki-driven acceptance test generation project. Interesting factoid: when queried about if they know what a wiki is, only ~5% of the audience raised their hand. Hmm, I guess wikis aren't as widespread as I previously thought.

Tuesday, March 11, 2003
12:00 AM