On November 17th, I had the pleasure and privilege to participate in the Developer Testing Forum help at PARC, in Palo Alto, CA.
There we 3 parts to the program: a talk by Sriram Sankar from Google, a panel of 4 developer testing experts, and a talk by Kent Beck. Below are some of the highlights that appealed to me.
The event was reasonably well attended for a workday, with over 100 participants from some of the biggest names in The Valley. Google’s presentation was very interesting in two ways: first Sriram described the slow, but deliberate adoption of XP methods and especially developer testing into Google’s egalitarian engineering culture. One of the concrete proofs that progress was being made was the increased stability of the build.
Sriram then went on to discuss how Google fights bugs that cannot possibly be reproduced by unit tests, either due to their irregularity as in race conditions or simply due to the fact that they obviously cannot have a copy of their production environment dedicated to testing. Among the techniques they employ are custom log, thread, and heap dump analysis tools. Another tool analyzed exception stack traces, then located the responsible developers via the SCM, and emailed them the problem report.
The panel largely focused on the issue of how to get developers “test-infected” and keep them that way. Rob Mee quoted my current boss, Christian Sepulveda, who is fond of saying that “developers who do not write unit tests are committing a form of professional negligence”. I subscribe to this opinion as well.
The most interesting part of the program in my opinion was Kent’s presentation on the future of developer testing. He based it around two assertions:
1. We must strive to develop healthy software, not just quality software. According to Kent, Quality is a measure of how well the software performs at the time it is deployed, but software Health forecasts how well it will respond to significant stress which may be extra load or requirements changes. The Health of the software was directly related to both the physical and mental health of its developers and to how well it is designed. Since Agile development achieves clean design incrementally through the application of refactoring, unit tests play an important role here. Kent stressed that when unit tests are difficult to write – it is almost always a design problem – making software testable is the hallmark of good design.
2. His second, and in my opinion most significant insight, was to present developer testing as a form of Accountability, specifically that tests clearly document what scenarios the developer considered when designing the software. According to Kent, the off-shoring phenomenon represents business’ search for accountability, not necessarily cheaper developers. He pointed out that many of the offshore development organizations carry high CMM certifications, which mostly make them accountable for each step in the development process.
This event demonstrated that we have made significant progress in getting developer testing recognized as an important practice in the software development process, but much remains to be done before we no longer have to make a fuss about it at all. For the rumor mill, there was talk of another developer testing event early next year with Microsoft’s participation.
Lastly, I plan to spend the next 3 months working at Agitar with their Agitator tool. I am thrilled to have an opportunity to work with their ground-breaking technology and hope to learn much about software testing in the process.
Regards,
David Vydra
. In fact this library has already been in existence for two years, and offers several well-known approaches from the Java world (as far as I understood them).