testdriven.com Wrangling quality out of chaos

Archive for October, 2004

Source code analysis breaks new ground

10.30.2004 · Posted in Links

Large-scale software systems are staggeringly complex works of engineering. Bugs inevitably come with the territory and for decades, the software profession has looked for ways to fight them. We may not see perfect source code in our lifetime, but we are seeing much better analysis tools and promising new approaches to remedy the problem.

Author: Jon Udell
Published: InfoWorld, October 29, 2004


Static Analysis for Security [PDF]

10.30.2004 · Posted in Links

All software projects are guaranteed to have one artifact in common: source code. Together with architectural risk analysis, code review for security ranks very high on the list of software security best practices. Here, we’ll look at how to automate source-code security analysis with static analysis tools.

Authors: Brian Chess, Gary McGraw
Published: IEEE Computer Society, 2004


Sessions on TDD and related practices at XP Day Benelux 2004

10.30.2004 · Posted in Events

The XP Day Benelux is a one day conference about all aspects of agile software development methods like Extreme Programming, DSDM, Scrum, Feature-driven development, and Crystal.

The conference will be held in Mechelen, Belgium, November 19, 2004, and will include several sessions dealing with TDD and related practices:The extreme power of short releases with Koen De Wel & Vera Peeters
Refactoring, keeping software malleable with Duncan Pierce & Ivan Moore
Refactoring thumbnails with Sven Gorts (more)
Getting your Story Straight with Rachel Davies & Andy Pols
Xtreme Databases with Pramod Sadalage & Nick Ashley

Visit the home page.
View the full program.
Register for the event (room limited to 100 participants).

TDD-related sessions at XP Day 2004

10.30.2004 · Posted in Events

Several sessions related to Test Driven Development will be held at the upcoming XP Day 2004, London, November 25-26.Refactoring: Keeping Software Malleable with Duncan Pierce
Agile Deployment with John Birtley & Hubert Matthews
Practices Make Perfect Pair Programming with Andy Marks & Julian Boot
The Agile Customer with Lisa Hunter & Jackie Mitchell
Automation: Begrudge Every Keystroke with Ray Gallagher & Sean O’Donnell
jBehave: Evolving Test Driven Development with Damien Guy, Manish Shah & Dan North

Visit XP Day‘s web site.
See all session summaries.
Register for the event.

Selenium: Functional and Browser Compatibility Testing Framework

10.28.2004 · Posted in Links

From the home page: "Selenium is a testing tool for browser-based testing of web applications. It can be used both for functional and compatability testing. Selenium tests run directly in supported browsers, just as real users do."

Driven by test-definition tables, similarly to FIT/FitNesse, it provides simple keyword driven interactions with the web-application front-end.

Selenium can also be integrated into CruiseControl for continuous integration.

Chromatic’s Katas; Decouple!; Help for customers…

10.27.2004 · Posted in Blogosphere

Chromatic, Technical Editor of the O’Reilly Network, starts a series of Perl Katas with focus on testing:

"How do you find new ideas? One way is through code katas, short pieces of code that start your learning. This article is the first in a series of code kata for Perl programmers. All of these exercises take place in the context of writing tests for Perl programs.

"Why give examples in the context of testing? First, to promote the idea of writing tests. One of the best techniques of writing good, simple, and effective software is to practice test-driven development. Second, because writing tests well is challenging. It often pushes programmers to find creative solutions to difficult problems."

The first Kata challenge is Testing Taint.

Antony Marcano explains why it’s a good idea to decouple test data from test code.

Brian Marick gives help for customers, highlighting common problems met by customers and solutions found.

Dan Crevier is reading Test-Driven Development in Microsoft.NET, and cringes at refactorings affecting performance.

Jonathan Cogley presents the ultimate pair-programming setup.

Daniel Bradby reports on his first session with Dave Thomas and does a quick summary of the ways to inject mocks into code.

Mike Gunderloy wonders whether Edit and Continue is a Useful tool or disaster in waiting.

Static Substitution: using Service Stubs for testing purposes

10.27.2004 · Posted in Links

As I listen to our development teams talk about their work, one common theme is their dislike of things held in statics. Typically we see common services or components held in static variables with static initializers. One of the big problems with statics (in most languages) is you can’t use polymorphism to substitute one implementation with another. This bits us a lot because we are great fans of testing – and to test well it’s important to be able to replace services with a Service Stub.

Author: Martin Fowler
Published: October 20, 2004

Dializer: Erlang software discrepancies analyzer

10.27.2004 · Posted in Links

The Dialyzer is a static analysis tool that identifies software discrepancies such as obvious type errors, unreachable code, redundant tests, virtual machine bytecode which is unsafe, etc. in single Erlang modules or entire applications. Currently, it starts its analysis from BEAM bytecode and reports to its user the functions where the discrepancies occur and an indication of what the discrepancy is about.

Deep Dive into Test Driven Development

10.27.2004 · Posted in Links

In a class I recently taught, the students created a simple payroll system using Test Driven Development. Different students made different amounts of progress, some almost finishing the example I’m about to post, and some not quite so far. But they all asked for my solution. I have already posted the code to my solution, but now I want to talk about how I went about creating it. Hopefully this will share my Test Driven thought processes with my students and anyone else who might be reading.

Author: Brian Button
Published: October 25, 2004

Austria C++ library

10.27.2004 · Posted in Links

The Austria C++ library provides a host of essential C++ tools including but generic factories, smart pointers, logging, unit testing framework and parameters/configuration system. Works with both Linux/GCC and win32/MVC++.

Automatic mock object creation for test factoring [PDF]

10.26.2004 · Posted in Links

Test factoring creates fast, focused unit tests from slow system-wide tests; each new unit test exercises only a subset of the functionality exercised by the system tests. Augmenting a test suite with factored unit tests, and prioritizing the tests, should catch errors earlier in a test run.

One way to factor a test is to introduce mock objects. If a test exercises a component A, which is designed to issue queries against or mutate another component B, the implementation of B can be replaced by a mock. The mock has two purposes: it checks that A’s calls to B are as expected, and it simulates B’s behavior in response. Given a system test for A and B, and a record of A’s and B’s behavior when the system test is run, we would like to automatically generate unit tests for A in which B is mocked. The factored tests can isolate bugs in A from bugs in B and, if B is slow or expensive, improve test performance or cost.

This paper motivates test factoring with an illustrative example, proposes a simple procedure for automatically generating mock objects for factored tests, and gives examples of how the procedure can be extended to larger change languages.

Authors: David Saff, Michael D. Ernst
Published: ACM SIGPLAN/SIGSOFT Workshop on Program Analysis for Software Tools and Engineering (PASTE’04), (Washington, DC, USA), June 7-8, 2004, pp. 49-51.


Managing your Way through the Integration and Test Black Hole

10.24.2004 · Posted in Links

"How do you know how much effort to schedule for integration and test? Once in test, how do you know how much longer it will take? How do you know when you have found all the defects that you can find? When is integration and test really done? Often, integration and test is seen as a black hole. The product enters the phase, but no one seems to know when it will come out."

This article presents approaches to estimate and manage more accurately the integration and test efforts.

Author: Ellen George
Published: Methods & Tools, Fall 2004


Dear Manager; TDD is great, except when it isn’t; Cross-functional TDD…

10.23.2004 · Posted in Blogosphere

Dear Manager, Mike Clark has written an open letter to tell you why the developers in your team need a build machine.

Craig Andera makes the point that TDD is great, except when it isn’t: "Sometimes what you’re testing is a little more complicated, like a form. In cases like that, testing gets a bit more complicated too, because now you have to simulate a user clicking buttons and entering text."

Brian Marick returns from a SCRUM Master session addressing cross-functional teams issues with an abstract diagram, and finds test-first practice to act as a means of integration between teams.

Brendan Tompkins describes ReSharper as a TDD tool.

Trey Hutcheson finds out How to test remote code that targets the PocketPC.

Peter Provost finds out why XSLTUnit doesn’t work under Internet Explorer, and provides the missing registry entry that’ll make it work.

Clarke Ching relates a dinner discussion on Waterfall and chopsticks and gets inspired to ruthless comparisons of XP and Waterfall following a go at the XP Game.

iCompile: automated build system for C++ projects on Linux and OS X

10.23.2004 · Posted in Links

iCompile makes the reasonable assumption that you want to compile all C++ files in the current directory into a single executable whose name matches the directory name. It analyzes your code and then, like an experienced Unix programmer, executes the compiler and linker with just the right options.

For Linux and MacOS X.

VSNUnit: Execute Unit tests from within Visual Studio

10.23.2004 · Posted in Links

VSNUnit is an integration tool that allows you to execute your NUnit tests from within the IDE. Instead of dumping the results as a text stream to the output window, VSNUnit provides the graphical tree view that NUnit and JUnit users have come to love. The tree view is a dockable toolwindow inside the IDE, allowing you to integrate it with your standard development environment layout.

See NUnitAddin and TestRunner for alternatives.

Driving on CruiseControl, part 2

10.23.2004 · Posted in Links

As we discussed in Part 1, there are some very good reasons for wanting to put your Continuous Integration builds’ results online for everyone to see. Many don’t like filling their inbox with full-blown reports, many projects don’t have easy access to a mail server, and many projects would like to integrate the build results to other dashboard-like applications which don’t quite fit in with the HTML email publisher solution.

In the case of CruiseControl, the web application used for presenting build results needs to work with the XML log files produced by CruiseControl during a build cycle. Not a surprise, the most used application used for exposing CruiseControl’s build results over the Web is the reporting application that comes as part of the CruiseControl distribution.

Author: Lasse Koksela
Published: JavaRanch, October 2004

TDD not a mere safety net…

10.17.2004 · Posted in Blogosphere

Andreas Schaefer disagrees with the view that Unit Testing is only a safety net: "The only person responsible of the quality of the software is the author himself and he/she should be held accountable for that. Therefore, in my opinion, writing Unit tests is not just a good habit but an essential part of the deliverables."

Trond Andersen reviews an article on How to do unit testing in BEA Workshop.

DotNetJoe is looking for a TDD pet project.

TDD Tools Advisory, October 1-15, 2004

10.16.2004 · Posted in Advisories

Antlion 0.2.0
buildtemplate r0-7-18
Cargo 0.2
DotNetMock 0.7.1
GhostDoc 1.2.0
HTTPtUnit 1.6
JavaTT 0.4
JML 5.1RC1
JSystem 1.3.2
SAMIE 1.2a
SimpleTest 1.0RC1
TestDriven.NET RC2
TestGen 0.02
WebTester b10
XRadar-base 0.92

Also: InRunner, JCollector, jEquals, JSystem, PublicTest, The Refactoring Planner, SuiteShop, TEST, TestCenter, VBMock, WabZilla, WebInjectorAntlion 0.2.0
Antlion helps Apache Ant build files manage external libraries, and the paths and filesets that are associated with those libraries. This also includes inter-project relationships for multi-build file projects.

buildtemplate r0-7-18
This is an installable third party library that comes with ant, junit, emma, build files, ant-contrib, and other utils all built into one jar. It lets you create projects fast, and upgrade everything much easier.

Cargo 0.2
Cargo provides a Java API to start/stop and configure Java containers. Possible use cases for Cargo include starting containers for integration and functional tests, or starting containers for applications that require a container to be started (Plugins for IDEs, etc).

.NET MockObjects 0.7.1
The .NET Mock Objects project is a .NET framework whose goal is to facilitate developing unit tests in the mock object style.

GhostDoc 1.2.0
GhostDoc is an add-in for Visual Studio .Net 2003 for automatically generating those parts of a C# documentation comment that can be deduced from name and type of e.g. methods, properties or parameters.

HTTPUnit 1.6
Combined with JavaUnit, HTTPUnit allows automated testing of web applications, with the tests written in Java. Obviously, it doesn’t matter what language the web application is in.

JavaTT 0.4
JavaTT separates the test data from the actual test cases; test cases may be implemented once and reused at will with different test data values. Test data values are stored in XML files, and grouped together in a scenario whose test results are also written to an XML file.

JML 5.1RC1
The Java Modeling Language (JML) is a behavioral interface specification language that can be used to specify the behavior of Java modules (as in design by contract — DBC). It has many tools to do assertion checking, unit testing, etc. JMLUnit is an integral part of the JML project.

JSystem 1.3.2
JSystem is a framework for writing and running automated tests, based on JUnit. Its main goal is to support automation of functional/system testing.

SAMIE 1.2a
S.A.M. for I.E. is a Perl module (SAM.pm) that allows a user to automate Internet Explorer. This free tool is designed for quality assurance engineers that need to run automated tests for their browser applications (Windows only).

SimpleTest 1.0RC1
SimpleTest aims to be a complete PHP developer test solution. It includes all of the typical functions you would expect from JUnit and the PHPUnit ports, but also adds mock objects and JWebUnit-style testing including cookies, authentication and form handling.

TestDriven.NET RC2
Point and Test NUnit integration for Visual Studio.NET. The aim of this project is to simplify testing and debugging using the NUnit framework inside of Visual Studio.NET.

TestGen 0.02
Python based framework for generating and running unit and integration tests. The framework supports unit testing of C, C++, and Python and integration testing of Python. See forums for detailed intial project focus.

WebTester b10
Webtester is a collection of java applications that can be used to record and playback test cases for either static HTML or dynamic web applications. It is intended to provide a way of performing end-to-end regression testing for existing websites.

XRadar-base 0.92
The XRadar is an open extensible code analysis framework and report tool currently supporting all Java based systems. The XML/XSL batch-processing framework produces HTML/SVG reports of the systems current state and the development over time.

Newly registered SourceForge Projects

InRunner aims to create a simple functional testing framework that uses user input recording and screen-capture comparison for unit and regression tests. Written in a combination of Java and C++, it can be used for testing any GUI application.

JCollector will dynamically collect JUnit test cases from a classpath into a test suite.

jEquals is to provide a Java framework for defining how objects should be tested for equality, eg when using JUnit for testing. Detailed trace is also provided when objects aren’t equal. A range of utility implementations are provided, for Collections, arrays, Iterators.

JSystem is a framework for writing and running automated tests, based on JUnit. Its main goal is to support automation of functional/system testing.

PublicTest is an automated unit testing tool. In essence, it changes all private/protected accessibility members to public, allowing direct testing of members that would normally be inaccessible to a testing class.

The Refactoring Planner aims to support users of the Eclipse platform to plan refactorings. The user will be able to generate and work with refactoring plans, which especially includes connecting steps of a refactoring to other resources, i.e. source code.

SuiteShop aims to provide a Web-based Testcase Management environment. SuiteShop allows, potentially, a completely different set of test suites to be run against each software build in a version, but still features version-based reporting.

TEST (TEsting Session Tool) is a set of solutions for tracking context-driven and Exploratory testing efforts by the use of the Session Based Testing Management (SBTM) model. It also enables the automated generation of reports and metrics.

TestCenter will be a tool for automated batch application testing to support all major flavors of UNIX/Linux, with a simple GUI, a flexible test suite enviroment and automated result checks.

VBMock aims to serve as a mock object builder library similar to NMock and EasyMock, but for VisualBasic 6. Starting with a template MockObject, you implement the interface of the target object you wish to mock using Implements.

WabZilla plans to provide a Perl-based Software Distribution System For Windows.

WebInjector plans to offer a lightweight, zero-footprint, web application testing framework, requiring only minimal changes to JSP pages. It simulates user interaction with web pages by injecting sequences of JavaScript, sent from the application server.

Testing Multithreaded Code with Mock Objects

10.15.2004 · Posted in Links

A question that is frequently asked on the jMock users’ mailing list is "how do I use mock objects to test a class that spawns new threads"? The problem is that jMock appears to ignore unexpected calls if they are made on a different thread than that which runs the test itself. The mock object actually does throw an AssertionFailedError when it receives an unexpected call, but the error terminates the concurrent thread instead of the test runner’s thread.

Author: Nat Pryce
Published: October 14, 2004