testdriven.com Wrangling quality out of chaos

Archive for March, 2005

Tool-Assisted Unit Test Selection Based on Operational Violations [PDF]

03.30.2005 · Posted in Links

Unit testing, a common step in software development, presents a challenge. When produced manually, unit test suites are often insufficient to identify defects. The main alternative is to use one of a variety of automatic unit test generation tools: these are able to produce and execute a large number of test inputs that extensively exercise the unit under test. However, without a priori specifications, developers need to manually verify the outputs of these test executions, which is generally impractical. To reduce this cost, unit test selection techniques may be used to help select a subset of automatically generated test inputs.
Then developers can verify their outputs, equip them with test oracles, and put them into the existing test suite. In this paper, we present the operational violation approach for unit test selection, a black-box approach without requiring a priori specifications.

Authors: Tao Xie, David Notkin (Department of Computer Science & Engineering, University of Washington)
Published: October, 2003
See also:
Jov (Java Unit Test Selection Tool Based on Operational Violations)
Jusc (Java Unit Test Selection Tool Based on Residual Structural Coverage)


JUnitX 5.2: test private methods without polluting production code

03.30.2005 · Posted in Java

[First, this news is not about the alphaworks JUnitX project (assertions extensions) but about the original extreme-java.de one (private/protected methods testing).]

I just wrote a little tutorial explaining how to use JUnitX to test private parts of your code. Its purpose is not to choose a camp (testing private parts or not), but to explain how to use JUnitX in case you need it, not the "why" of its use.

For our project needs, I’ve recently played with the JUnitX source code and patched it to make it easier to use: combined with javassist, this version not longer needs you to create TestProxy classes in your production code. (see section " JUnitX 5.2 -> No need for TestProxy !", to download this version)

Original version was created by Andreas Heilwagen but this project doesn’t seem to be supported anymore :(

Changelog for this version:
.avoid TestProxy classes creation
.static private methods testing.

Comments and remarks welcome.

JUnitX How to

JUnitX how-to: testing private classes

03.30.2005 · Posted in Links

JUnitX is an extension to JUnit allowing to go around the access restrictions and thus to be able to test private or protected methods, as well as methods inside inner private or protected classes. All this without polluting classes to test. In general it is not advised to abuse too much of this kind of tests, but it may become handy to be able to test a crucial, private method, only to reassure oneself of its efficiency. (even if we unit test the public method using it).

Author: Jérôme Layat
Published: March 24, 2005

Announcing TestNG for Eclipse

03.30.2005 · Posted in Java

The TestNG team is happy to announce the first release of the TestNG plug-in for Eclipse.

This first release covers the basic functionalities of TestNG, among which:

* Multiple ways to launch tests (from a method, from a class, groups or an entire suite).
* Convenient report view that lets you directly jump to failed tests.

A more detailed overview can be found here, and the documentation and snapshots here.


The TDD train, Agile contrasts, Erratic behavior…

03.29.2005 · Posted in Blogosphere

Brian Button shares the train of thoughts that come to an agile coder in TDD defeats Programmer’s Block — Film at 11: "Here is the scene. I’m working with Peter Provost and Jim Newkirk, building a Continuous Integration system to be used internally in patterns & practices. Peter and I have been the devs on it, and Jim provided the customer’s point of view. It has taken longer than we thought, mostly because things were harder than they seemed at first [..] But we’re winding up the initial implementation of this system, with one final piece left before we can put this thing into use." Brian also gives a few pointers on how to Explain TDD to a non-techie, and is also starting a series on legacy testing: his group is planning on implementing tricks from Michael Feathers’ new book, Working Effectively with Legacy Code.

Glen Stampoultzis shares his experience with an Agile shop: "I’m a contractor and for the past six months I’ve been working in smallish agile software development company. I’ve just recently left this position to join a ‘Big Company’. There really is a big contrast between the two organisations."

Martin Fowler ponders Erratic Test Failure.Jim Alateras takes a second look at Grinder and shows how to conduct Load Testing Web Services: "First impressions have been very positive indeed. I was able to install and run a simple Jython-scripted example in less than 1 hour. After one day with Jython and Grinder I was able to leverage some existing JUnit test cases to do some ad hoc load testing on my web service."

Azad Bolour shows how to conduct Web Application Testing with Embedded Browsers.

Brian Marick keeps delving into DDTDD.

Jason Bell is Refactoring by pressing delete and starting again.

Cyrille Martraire advocates ONE loop per method: "I usually follow this rule, with very few exceptions. The point is that I think a loop is already complicated enough to be the hot spot in a method."

Discussion at SitePoint on SimpleTest and other PHP unit-testing frameworks: [url=http://www.sitepoint.com/forums/showthread.php?t=245100&page=1&pp=25&highlight=test driven]How many actually use unit testing?[/url]

This one is from the Agile Software Testing list: Where do you find testers?

Laurent Ploix blogs on using Jython and Python to write Junit tests: "Using Jython/Python to make unit tests enables you to write your tests before to write the code. Then you can encapsulate the python/jython tests in JUnit tests so that they can be easily displayed in JUnit plugins in your IDE."

For those looking for a Java IDE comparison chart, IT Manager’s Journal compares [url=http://productguide.itmanagersjournal.com/pg/05/03/26/148205.shtml?tid=14487&tid=11105&tid=106]Four Java IDEs[/url]. You may also be interested to know that NetBeans can import Eclipse projects, and that Eclipse 3.0.2 has been released.

Agitar Software Wins Software Development Magazine Jolt Award.

[url=http://www.computerweekly.com/articles/article.asp?liArticleID=137532&liArticleTypeID=20&liCategoryID=2&liChannelID=105&liFlavourID=1&sSearch=&nPage=1]Dixons uses agile method for Epos project[/url].

Web Application Testing with Embedded Browsers

03.29.2005 · Posted in Links

"Recently I started experimenting with an embedded browser to develop tests for web applications. By using an embedded browser with an in-process web server, I can now write web application tests that run in their entirety in a single process. Such end-to-end in-process tests provide a seamless client-server execution context for a web application under test, which simplifies the debugging of failing tests. As a bonus, end-to-end in-process tests with embedded browsers also exercise any Javascript associated with loading pages and with clicking their buttons and links. In this note, I’ll discuss the rationale for end-to-end in-process tests, and demonstrate simple examples of such tests."

Author: Azad Bolour
Published: Bolour Computing, March 2005

CruiseControl With CMSynergy

03.29.2005 · Posted in Links

"As with most things in life, there are many ways to accomplish the same task. This page will attempt to document my own experiences using Cruise Control with Telelogic’s CM Synergy, and what has worked well for me."

Author: Robert J. Smith
Published: March 22, 2005

JTiger: unit-testing framework for Java

03.29.2005 · Posted in Links

JTiger Unit Testing Framework for Java 2 Standard Edition 1.5 provides a robust and feature-rich abstraction to develop and execute unit test cases. Test case metadata and unit test documentation is expressed through annotations. Assertions, from the basic to the complex, can be made using the JTiger assertions package rather than having to write the logic for those assertions yourself.

Visual Build Professional

03.22.2005 · Posted in Links

Visual Build Pro is a tool that enables developers, software process engineers, and build specialists to create an automated, repeatable process for building their software. Visual Build provides built-in support for Microsoft Visual Studio .NET, Visual Basic, Visual C++, Visual J++, SourceSafe, Installer, eMbedded Tools, Borland Delphi, C++Builder, JBuilder, and more.

(Commercial tool)

Grand Unifitation, Design-Driven Test-Driven Design…

03.21.2005 · Posted in Blogosphere

The Grand Unifitation is underway! Robert C. Martin calls for a summit: "I think it might be a good idea to get all the major Fit/FitNesse contributors (and perhaps a few major users) together for a one or two day summit. The purpose of this summit is to decide how to unify FIT and, more importantly, how to keep it unified."

Brian Marick explores Design-Driven Test-Driven Design: "At the Canadian Agile Network workshop earlier this week, Jeff Patton had what we think could be a great idea: 1. The conceptual connection between Interaction Design and programming can be the Presenter class (from Model View Presenter); 2. The performative connection can be Fit DoFixture scripts."Mike Clark explains how he learned Ruby and quickly found the Test::Unit module.

Dave Thomas is writing his books incrementally with a Wiki.

Scott Bellware picks up TDD practice, by and by: "The one resounding problem with picking up TDD and agile from the primary agile community message boards was the language barrier. The agile community spoke code, and I didn’t — not to the extent I needed anyway. I was a visual developer, and they were hard core OO practitioners. My background in Microsoft approaches to application development left me quite lost when it came to understanding the significance of a lot of the dialog."

Sebastian Bergmann translates the JUnit Cookbook for PHP developers, resulting in the PHPUnit Cookbook.

Chris Shiflett and Marcus Baker join their efforts to make SimpleTest output a TAP-compliant protocol to integrate with Apache-Test.

Perryn Fowler thinks some Mocks are ill-behaved: "I see things being called ‘mocks’ that IMNSHO aren’t mocks at all. My basic rule of thumb is If it has behaviour, it isn’t a mock."

Phillip J. Eby is Mocking wxPython.

Gianugo Rabellino is doing some [url=http://www.rabellino.it/blog/index.php?title=hardcore_unit_testing_in_swing&more=1&c=1&tb=1&pb=1]unit testing in Swing[/url] to "test a JTextField on which [he’s] attaching a custom key listener".

Mike Kelly reports on the March session of the Indianapolis Workshops on Software Testing, which addressed unit testing.

Not specifically test-driven, but interesting none the less: Steve Hayes is providing extensive notes taken during the Software Education Conference (Day one, two, three). Steve has also had some disagreements of late on the meaning of "done".

JUnit Cookbook

03.21.2005 · Posted in Links

A short cookbook showing you the steps you can follow in writing and organizing your own tests using JUnit.

Authors: Kent Beck, Erich Gamma

Hopeless dysfunctionality, JB brings TDD to J2EE, It’s a revolution…

03.17.2005 · Posted in Blogosphere

On InfoWorld, Paul Krill relates a topic raised on a Microsoft/Google panel on agile development, personnel issues cloud software development: "Following an audience question about personnel issues in software development, panelist Granville ‘Randy’ Miller, an architect and developer for Visual Studio at Microsoft, noted the problems that can arise with dysfunctional teams. He cited an incident where developers used interfaces against each other and developers were settling vendettas or satisfying egos rather than delivering business value."

J.B. Rainsberger made people happy with his tutorial on Test-Driven J2EE at SD West.

Read Cem Kaner’s excellent article The Ongoing Revolution in Software Testing.Michael Hunter has been stuffing his brain at SD West, and blogs about the experience. In particular, he reports on Ken Pugh’s talk on prefactoring (yes, PREfactoring), Elisabeth Hendrickson’s tutorial on Exploratory testing and her talk on how in the world testing can survive in an Agile world.

Ed Burns avoids injecting a bug thanks to TDD and falls to his knees: All Hail TDD!

Ori Peleg posts a Threaded test suite runner for Python. (via [url=http://www.artima.com/forums/flat.jsp?forum=256&thread=98567]Artima[/url])

Trey Hutcheson asks for advice on Unit testing sockets with dotnet.

Lasse Koskela posts on Pair Programming and War Zones, or solutions to difficulties getting in the zone.

In MSDN Magazine, Chris Menegay presents some of the new features in Visual Studio 2005 Team System: "Visual Studio 2005 Team System integrates within the IDE several tools that will help you write better code. These include a profiler to analyze potential performance problems in your application, code analysis tools for both managed and unmanaged code, and unit testing tools with code coverage analysis."

Mike Attili posts his slides from a recent session on Continuous Integration for .NET.

Jeff Atwood redefines the Ambient Orb build with Automated Continuous Integration and the BetaBrite LED Sign.

Ryan Nelsestuen asks what ratio of server-side Java is Object-oriented vs. procedural, and finds TDD compelling: "In the little bit of coding I have done in that fashion, I find that I wind up writing things in a more ‘OO correct’ fashion, without really having to think about it… writing a client for your object at the same time you write the object seems to naturally drive lower coupling, making code more maintainable."

DrKW Banking platform uses JUnit: [url=http://www.eweek.com/article2/0,1759,1773936,00.asp]DrKW Open-Sources Better Communication[/url].

Agile Scotland: Invite: April Agile Training Session: "Agile-Coach Rachel Davies will be presenting two agile training sessions in Edinburgh on Tuesday, 26th of April, 2005."

Last minute: There is a presentation on Continuous Integration using CruiseControl and ClearCase at the Arizona Rational Users Group in Scottsdale, AZ, this evening (March 17, 2005).

Using Agile Software Development (3/3)

03.17.2005 · Posted in Links

In part one of this three-part series on Agile Software Development, we looked at developer practices and how technical excellence can lead to marked improvements in quality. Part two looked at development team practices and how to create the most effective development team possible, with a focus on coding standards, continuous integration and a shared language for describing the system. Now we’ll take a look at the outer-most circle — the "one team practices" that include developers, testers, customers — and help better align business and IT.

Author: Brian Swan
Published: uk.builder.com, March 16, 2005

Pair Testing: How I Brought Developers into the Test Lab [PDF]

03.17.2005 · Posted in Links

"As a black box tester working on a variety of development projects, I was often approached by developers who were trying out test-driven development and wanted to learn more about testing. To show them how I worked, I involved them in pair testing, a technique in which two people test an application at the same computer. Interestingly enough, the developers taught me as much as I taught them."

Author: Jonathan Kohl
Published: Better Software Magazine, January 2004


The Ongoing Revolution in Software Testing [PDF]

03.17.2005 · Posted in Links

"Twenty-one years ago, I started writing Testing Computer Software (Kaner, 1988), a book whose second edition (Kaner, Falk, & Nguyen, 1993, 1999) outsold every other book in the field. It didn’t seem destined for the mainstream back then. [..]

This paper for STPCon is my next crack at a paper that reappraises the assumptions about testing and development that I accepted or left alone in 1983, and perhaps the start of a new map for TCS 3.0. Some of the assumptions were probably always wrong. Others work under some circumstances, but not others—and in 21 years evolution of software development, circumstances have changed a lot. Others, I think, retain lasting value."

A thorough review of many assumptions, on:
– The role of testers
– Test planning and documentation
– The practice of testing

Author: Cem Kaner, PhD
Published: Software Test&Performance Conference, December 8, 2004

TDD Tools Advisory, March 1-15, 2005

03.16.2005 · Posted in Advisories

Here’s a short list of recently-released tools related to test-driven development practice:

Convergence 2.0
FinalBuilder 3.0
JxRef 0.6.0
MockPP 1.6.0
PHPUnit2 2.2.0
SQLUnit 4.5
ZXF Snapshots 03/2005 Convergence 2.0
Convergence is for software development managers, architects, and developers who are dissatisfied with the traditionally limited view of quality as software is being constructed. Our product is a quality management dashboard for the Java platform that enables tracking of application quality early in the software development life-cycle.

FinalBuilder 3.0
FinalBuilder is an automated build and release management solution for Windows software developers and SCM professionals. Commercial tool.

JxRef 0.6.0
JxRef is a Java code analysis toolset. It is a tool that will help you identify refactoring opportunities in a Java code base. It can also provide information that will help you identify potential risks of making a code change.

MockPP 1.6.0
mockpp’s goal is to facilitate developing unit tests in the spirit of Mock Objects for Java, EasyMock and jMock. [url=http://www.testdriven.com/search.php?query=mock&action=results]Mock objects[/url] allow you to set up predictable behaviour to help you test your production code by emulating some functionality your code depends on. This might for example be a huge database which is too difficult and time-consuming to maintain just for testing purposes.

Pear PHPUnit2 2.2.0
PHPUnit is a family of PEAR packages (PHPUnit, PHPUnit2, PHPUnit2_MockObjects, PHPUnit2_GtkUI, and PHPUnit2_WebUI) that supports the development of object-oriented PHP applications using the concepts and methods of Agile Programming, Extreme Programming, Test-Driven Development and Design-by-Contract Development. It provides an elegant and robust framework for the creation, execution and analysis of Unit Tests.

SQLUnit 4.5
SQLUnit is a regression and unit testing harness for testing database stored procedures. A SQLUnit test suite would be written as an XML file. The SQLUnit harness, which is written in Java, uses the JUnit unit testing framework to convert the XML test specifications to JDBC calls and compare the results generated from the calls with the specified results.

ZX Framework Snapshots 03/2005
A java framework for extreme development of database systems. The framework is based on the theory of self-describing objects that allow separation of ‘what’ from the ‘what with’ (e.g. ‘search form’ for ‘client’).

Advanced Deployment with Ant (and DbUnit): Put yourself in control of your development process

03.16.2005 · Posted in Links

This article discusses some of Ant’s more advanced features that are available in Ant’s two task packages: the core task package and the optional task package. The core package contains tasks for building, packaging, and deploying projects, as well as tasks to execute SQL statements and tasks for integrating with Concurrent Versions System (CVS). The optional package extends Ant’s functionality in a variety of ways, such as integrating with other source control systems and running unit tests using JUnit.

The author also provides an example of how to use the DbUnit Ant task. DbUnit is a JUnit extension targeted for database-driven projects that, among other things, puts your database into a known state between test runs.

Author: Harry Klein
Published: ColdFusion Developer’s Journal, March 15, 2005

Agitar Management Dashboard: Monitor and Manage Developer Testing efforts

03.16.2005 · Posted in Links

For successful application of Developer Testing, a total of 20-40% of time dedicated to the project will be spent on the creation and execution of unit tests. It is therefore critical that teams can evaluate that they are allocating resources in the best possible way. The Agitar Management Dashboard is the first of its kind, created specifically to monitor and manage Developer Testing efforts.

Convergence: Java Quality Management Dashboard

03.16.2005 · Posted in Links

Convergence is for software development managers, architects, and developers who are dissatisfied with the traditionally limited view of quality as software is being constructed. Our product is a quality management dashboard for the Java platform that enables tracking of application quality early in the software development life-cycle.

Joshua: Java test distribution mechanism

03.16.2005 · Posted in Links

Joshua was created to investigate the conceptual foundation, design, and implementation of an approach that distributes the execution of regression test suites. Our technique can complement existing regression test selection and prioritization approaches or be used in an independent fashion when other approaches are not likely to increase regression testing efficiency.

Our current implementation of Joshua uses Jini and JavaSpaces to build upon the JUnit Test Automation Framework and create a simple and efficient test distribution mechanism. We are currently developing new implementations that rely upon the Frugal Metacomputing Framework, mobile agents, and the Condor High-throughput Computing Framework.