testdriven.com Wrangling quality out of chaos

Archive for November, 2004

Report from the Developer Testing Forum at PARC

11.21.2004 · Posted in News

On November 17th, I had the pleasure and privilege to participate in the Developer Testing Forum help at PARC, in Palo Alto, CA.

There we 3 parts to the program: a talk by Sriram Sankar from Google, a panel of 4 developer testing experts, and a talk by Kent Beck. Below are some of the highlights that appealed to me.

The event was reasonably well attended for a workday, with over 100 participants from some of the biggest names in The Valley. Google’s presentation was very interesting in two ways: first Sriram described the slow, but deliberate adoption of XP methods and especially developer testing into Google’s egalitarian engineering culture. One of the concrete proofs that progress was being made was the increased stability of the build.

Sriram then went on to discuss how Google fights bugs that cannot possibly be reproduced by unit tests, either due to their irregularity as in race conditions or simply due to the fact that they obviously cannot have a copy of their production environment dedicated to testing. Among the techniques they employ are custom log, thread, and heap dump analysis tools. Another tool analyzed exception stack traces, then located the responsible developers via the SCM, and emailed them the problem report.

The panel largely focused on the issue of how to get developers “test-infected” and keep them that way. Rob Mee quoted my current boss, Christian Sepulveda, who is fond of saying that “developers who do not write unit tests are committing a form of professional negligence”. I subscribe to this opinion as well.

The most interesting part of the program in my opinion was Kent’s presentation on the future of developer testing. He based it around two assertions:

1. We must strive to develop healthy software, not just quality software. According to Kent, Quality is a measure of how well the software performs at the time it is deployed, but software Health forecasts how well it will respond to significant stress which may be extra load or requirements changes. The Health of the software was directly related to both the physical and mental health of its developers and to how well it is designed. Since Agile development achieves clean design incrementally through the application of refactoring, unit tests play an important role here. Kent stressed that when unit tests are difficult to write – it is almost always a design problem – making software testable is the hallmark of good design.

2. His second, and in my opinion most significant insight, was to present developer testing as a form of Accountability, specifically that tests clearly document what scenarios the developer considered when designing the software. According to Kent, the off-shoring phenomenon represents business’ search for accountability, not necessarily cheaper developers. He pointed out that many of the offshore development organizations carry high CMM certifications, which mostly make them accountable for each step in the development process.

This event demonstrated that we have made significant progress in getting developer testing recognized as an important practice in the software development process, but much remains to be done before we no longer have to make a fuss about it at all. For the rumor mill, there was talk of another developer testing event early next year with Microsoft’s participation.

Lastly, I plan to spend the next 3 months working at Agitar with their Agitator tool. I am thrilled to have an opportunity to work with their ground-breaking technology and hope to learn much about software testing in the process.

Regards,

David Vydra

Choosing an Automated Testing Tool

11.19.2004 · Posted in Links

For many IT and quality assurance managers, the decision of which testing tools to use can cause confusion. The first decision is which category of tool to use: one that tests specific units of code before the application is fully combined, one that tests how well the code is working as envisioned, or one that tests how well the application performs under stress. And once that decision is made, the team must wade through a variety of choices in each category to determine which tool best meets its needs.

This guide will help you sort through some of the many options available with Developer-Oriented Tools, Functional-Testing Tools, Load-Testing Tools, and Performance Monitoring and Management Tools.

Author: Karen D. Schwartz
Published: September 29, 2003
link

Tags:

OOP Case Study: The Bank Account Class (Part 3)

11.19.2004 · Posted in Links

I have been looking mock objects and test driven design lately. The main issue, I see brought up over and over again is that people want to modify existing code by inserting mock objects, but that it is hard to do because of code dependencies. I argue that proper use of abstraction at the implementation level would make it much easier.

Author: Christopher Diggins
Published: Artima, November 18, 2004
link

Tags:

Agile User Interface Development

11.19.2004 · Posted in Links

What is preventing people from building GUIs in an Agile way? Whether their application is web-based or a desktop application, most developers don’t do test-driven development (TDD) of the user interface. This is for a simple reason: unit testing GUI software is hard. Tests that exercise the GUI can be tedious and error-prone, involving complex code to simulate user events, wait while events propagate and controls redraw, and then attempt to check the state as it would appear to the user. Agility depends on doing TDD, but effective tests of specific behaviors are difficult to write for the GUI. The quality and design benefits of Agile have yet to be fully realized on the GUI side of the cube farm.

Author: Paul Hamill (author of Unit Test Frameworks)
Published: OnJava.com, November 17, 2004
link

Ward Cunningham’s Fit Acceptance Testing Framework

11.19.2004 · Posted in Links

Testing framework by Ward Cunningham and others and specifically applicable to acceptance tests like those prescribed by Extreme Programming.

Our goal is to facilitate cooperation between customers, testers and developers who share the values of communication, simplicity, feedback and courage.

The framework includes Java and .Net flavours.
link

Agitator & Agitar Management Dashboard

11.19.2004 · Posted in Links

Agitator helps software development teams implement Developer Testing (a.k.a. unit testing). The automation provided by Agitator makes developer testing practical, and is the perfect complement to deliver improved software quality and reduced overall software lifecycle costs.

The Agitar Management Dashboard is created specifically to monitor and manage Developer Testing efforts, to evaluate that resources are allocated in the best possible way.

Commercial tool.
link

Using Cargo for functional testing

11.19.2004 · Posted in Links

Automated tests are good. Automated Functional tests are even better as they are the proof that your application is working. In addition, with automated functional tests you can also automate your delivery process. However, writing automated functional tests is hard. The main reason it is hard is because you need to control your execution environment (database, application server, etc).

Cargo is a framework that you can use to automatically install, configure and execute J2EE containers. Thus it allows you to control your execution environment (for the J2EE container at least) and permits completely automated functional tests for J2EE applications.

Author: Vincent Massol
Published: November 17, 2004
link

WmUnit – webMethods Unit Testing

11.18.2004 · Posted in Links

WmUnit is a unique package designed to help webMethods 4.6 & 6 customers achieve a higher standard of quality and reliability with their webMethods services by allowing them to use automated tools for developing and testing their services.

WmUnit is based upon popular industry unit testing techniques and does NOT require developers to write complex Java code.

WmUnit is a commercial tool.

Documentation is found at http://www.customware.net/repository/display/WMUNIT/Home
link

TDD Tools Advisory, November 1-15, 2004

11.16.2004 · Posted in Advisories

The following tools were recently updated:

Abbot 0.13.1
Check 0.9.2
Clover 1.3.2 & JBuilder plugin 1.0RC1
Clover.NET 1.1 Mono
CruiseControl.NET 0.7
EasyMock.NET 1.1RC1
Fit-GUI 1.0b1
FitNesse 20041105
FxCop 1.312
JTestCase 2.1.2
Marathon 0.90
STAF 3.0.0 b5

Also: Antifact, KDevCppUnitAbbot 0.13.1
Abbot is a Java GUI testing framework. The framework may be invoked directly in hand-coded Java tests, or more simply using XML-based scripts. Both methods are designed to be used with the JUnit testing framework. You can launch a GUI, invoke arbitrary user actions on it, and examine its state. The scripts may be invoked from JUnit or with minor modification any other testing environment.

Check 0.9.2
Check is a unit test framework for C. It features a simple interface for defining unit tests, putting little in the way of the developer. Tests are run in a separate address space, so Check can catch both assertion failures and code errors that cause segmentation faults or other signals. The output from unit tests can be used within source code editors and IDEs.

Clover 1.3.2 & JBuilder plugin 1.0RC1
Clover is a Java code coverage analysis tool. It discovers sections of code that are not being adequately exercised by your unit tests. Clover reports its findings in multiple formats for easy use by the whole team at the project level down to each line of source code.

Clover.NET 1.1 Mono
Developers and Team leads use Clover.NET to quickly find untested code and to measure testing completeness. This feeds back into the testing process to improve tests. Clover.NET does not just produce a set of figures which are hard to relate to your code. Clover.NET reports, published in HTML format, provide an accessible, easily navigable view of your source. These reports are easily understood, yet comprehansive. They can be shared with the development team or project management.

CruiseControl.NET 0.7
CruiseControl.NET is an automated Continuous Integration server for the Microsoft .NET platform. It is a functional port of the Java-based CruiseControl to the .NET platform.

EasyMock.NET 1.1RC1
EasyMock.NET is a class library that provides an easy way to use mock objects for given interfaces or remote objects. EasyMock.NET is a port of the EasyMock framework, which can be found for the Java(TM) platform.

Fit-GUI 1.0b1
Fit-GUI is a .NET FIT test runner. It allows you to execute test or group of tests and see input test specification and results right away.

FitNesse 20041105
FitNesse is a collaborative testing and documentation tool, providing a very simple way for teams to collaboratively create documents, specify tests, and run those tests.

FxCop 1.312
FxCop is a code analysis tool that checks .NET managed code assemblies for conformance to the Microsoft .NET Framework Design Guidelines. It uses reflection, MSIL parsing, and callgraph analysis to inspect assemblies for more than 200 defects in the following areas: Library design, Localization, Naming conventions, Performance & Security. FxCop includes both GUI and command line versions, as well as an SDK to create custom rules.

JTestCase 2.1.2
JTestCase uses XML file to organize multi-testcase data and asserting conditions (action and expected result), and provides a set of easy-to-use methods to retrieve test cases out from XML, and asserts results against those expected ones in data file.

MarathonMan 0.90
Marathon runs GUI based acceptance tests against Swing applications. It is composed of a runner, a recorder, and an editor. Tests scripts are expressed as Python code.

STAF 3.0.0 b5
The Software Testing Automation Framework (STAF) is an open source, multi-platform, multi-language framework designed around the idea of reusable components, called services (such as process invocation, resource management, logging, and monitoring).

Newly registered SourceForge project:

Antifact is meant to provide project dependency management and continuous integration with Ant.

KDevCppUnit is both a standalone application and a plugin for KDevelop (http://www.kdevelop.org) to support unit testing based on CppUnit.

Fake It; Evolutionary DB; Team System Test

11.15.2004 · Posted in Blogosphere

James Newkirk shows an example of the Fake It (‘Til You Make It) technique.

Peter Hancock finds out that Database design and deployment can be evolutionary.

Udi Dahan mocks a unit.

Dave Bost reports on a full-day track on Microsoft Team System and highlights features of its built-in testing framework.

Brian Button posts Part Deux of his test-driven developed Payroll system.

Mike Spille tackles IoC.

FlexUnit is coming soon…

Jim Little finds the tradeoff between Mock Object complexity and Playground manageability, while Santosh Benjamin is asking for pointers.

Oliver Kiessler shows how to set up a project and unit tests with the karma J2EE framework.

Erik Bianchi points to a quirk in Unit-testing Flash.

Dave Thomas explains Dependency Injection with Ruby on Rails.

Automate GUI tests for Swing applications

11.15.2004 · Posted in Links

Transition from unit tests to acceptance tests

Automation is necessary for frequent and consistent testing, which is the foundation of agile development. However, acceptance tests of GUI applications are not always easy to automate. This article explains a simple way of automating Java Swing application acceptance tests, starting from Swing components’ unit tests and extending them to acceptance tests without human intervention.

Author: Ichiro Suzuki
Published: JavaWorld, November 15, 2004
link

JUnit testing Velocity

11.15.2004 · Posted in Links

Clifton Craig writes: "I’ve recently been having some trouble getting velocity to do what I want it to do. I dreamt up an ad-hoc GUI widget that would evaluate arbitrary VTL strings and display the result for testing. The nifty little widget was never completed. After reading this tip I may scrap my GUI code all together."

(via Matt Raible)
link

Tags:

karma: lightweight J2EE application framework

11.15.2004 · Posted in Links

karma is a lightweight j2ee open source application framework based on the well known model view controller (mvc) pattern. Its IoC design makes servlets, EJBs or WebService controllers (or even POJOs) containers for the karma components which are themselves exchangeable through your own implementations.

karma provides Mock Objects for ease of testing.
link

A lightweight nonintrusive EJB testing framework

11.15.2004 · Posted in Links

Bring your EJB 2.0 applications in line with testing best practices

This article presents a simple, easy-to-deploy framework that enables fine-grained tests to be run on the container, making it possible to develop and maintain Enterprise JavaBeans components using a test-driven development (TDD) process.

Author: Phil Zoio
Published: JavaWorld, November 15, 2004

link

FxCop: .NET code analysis tool

11.15.2004 · Posted in Links

FxCop is a code analysis tool that checks .NET managed code assemblies for conformance to the Microsoft .NET Framework Design Guidelines. It uses reflection, MSIL parsing, and callgraph analysis to inspect assemblies for more than 200 defects in the following areas: Library design, Localization, Naming conventions, Performance & Security.

FxCop includes both GUI and command line versions, as well as an SDK to create custom rules.
link

Cruise Control Installed

11.15.2004 · Posted in Links

This document is meant to help you get running with CruiseControl 2.0 (CC). Starting from the download, it will take you through installing, configuring, and starting CC for the first time.

Author: Jonathan Julian
Published: Wiki, ongoing
link

More NUnit and Nant Tricks, Tips and Examples

11.15.2004 · Posted in Links

Nant (the .NET version of the Ant build tool) handles a wide range of build, deployment, and test tasks. Stewart Baird digs a little deeper into the .NET development tools Nant and NUnit, with tips and examples to help you with your next project.

Author: Stewart Baird
Published: InformIT, december 27, 2002
link

Mouse Gestures Library for Java

11.11.2004 · Posted in Links

Mouse Gestures is an open source pure Java library for recognition and processing mouse gestures. With Mouse Gestures you have a fast way to execute commands without using the keyboard, menus or toolbars. The user just holds down a mouse button and moves the mouse in a certain predefined way. It can be used in any Swing application.
link

Practically Groovy: Unit test your Java code faster with Groovy

11.11.2004 · Posted in Links

Not long ago, developerWorks contributor Andrew Glover penned an article introducing Groovy, a new proposed standard language for the Java platform, as part of our alt.lang.jre series. Reader response was fantastic, so we’ve decided to launch this column to offer a practical guide to using this hot new technology. This first installment introduces a simple strategy for unit testing Java code with Groovy and JUnit.

Author: Andrew Glover, CTO, Vanward Technologies
Published: IBM DeveloperWorks, November 9, 2004
link