testdriven.com Wrangling quality out of chaos

Hamburg/Newton. Mar. 07, 2007 – Telerik, Inc. and froglogic GmbH announced a technology cooperation providing a specialized automated GUI testing tool for Telerik’s award-winning radControls UI suite and AJAX framework.

The focal point of this joint cooperation is to combine the technical expertise and technology of froglogic and Telerik to provide developers with a robust testing framework for their web applications. Squish, froglogic’s web testing tool, will help radControls users to reduce the cost and time needed for their QA cycles by automating the GUI testing efforts."The technology relationship with froglogic will further strengthen our position in the web development market. With this cooperation we look forward to provide our customers with best in class solutions.", says telerik’s CEO Svetozar Georgiev.

"We feel that this technology synergy between our companies would be of significant benefit of both telerik and froglogic" says Reginald Stadlbauer, forglogic’s CEO. <em>"Both telerik and froglogic share a common focus in developing great products and in treating our customer as valued partners in the process."

In the scope of this technology cooperation, froglogic will add dedicated support for Telerik’s radContols suite to its automated Web testing tool Squish for Web. radContols users will be able to use Squish for Web to create advanced, robust and maintainable automated tests for their classic or Ajax-enabled user interfaces.

Squish for Web and tests created with it are completely cross-platform and support all popular web browsers such as Internet Explorer, Firefox, Mozilla, Safari and Konqueror.

For more information about Squish, please contact squish@froglogic.com.

About Telerik

Telerik is a leading vendor of User Interface (UI) components for ASP.NET and Windows Forms. Building on its expertise in interface development and Microsoft technologies, Telerik helps customers build applications with unparalleled richness, responsiveness and interactivity. Created with passion, Telerik products help thousands of developers every day to be more productive and deliver reliable applications under budget and on time. http://www.telerik.com

About froglogic

froglogic GmbH is a software company based in Hamburg, Germany. Their flagship product is Squish, the market-leading automated testing tool for GUI applications based on Qt, Java (Swing, SWT, etc.), Tk or XView.

If you are trying to find a way to do Test-Driven Development in JavaScript, you can learn some ways to do it here.

Server-side JScript .NET for ASP .NET is an important topic at this site as well as client-side JavaScript in the browser. The documentation for JSNUnit is found here as well. Links for dynamic language benefits, agile practice, ajax, and JavaScript in one place.

A new version of Getting Fit With .Net, an introductory guide to testing .Net applications with Fit/FitNesse, has been made available.

Version 0.2 is updated to cover Fit.Net 1.1, and has a new chapter on DoFixture, working with arrays and business objects and writing better test scripts.

Gojko Adzic

"IEEE Software seeks articles for a special issue devoted to both the state of the practice and the state of the art of TDD. We’d like articles to address TDD use in a wide range of project types and styles, and we invite articles pointing to ways to succeed as well as ones describing problems encountered—whether or not they were ultimately resolved favorably. We’re especially interested in experience reports of test-driving large software projects, dealing with legacy code, and test-driving the maintenance of a legacy system."

Publication: May/June 2007
Submission deadline: 1 December 2006
[url=http://www.computer.org/portal/site/software/menuitem.538c87f5131e26244955a4108bcd45f3/index.jsp?&pName=software_level1&path=software/content&file=cfp-May07_TDD.xml&xsl=article.xsl]Call for articles[/url]

VersionOne recently conducted a global survey to highlight the value teams deliver from Agile development. Survey respondents work in companies of all sizes from small and mid-sized organizations to the largest global corporations. This survey summarizes the state of Agile Development in organizations covering every industry vertical from financial services, health care, and education to video games, government, and defense.

Read the results of the survey here.

More and more developers are using development tools produced by the open source community like JUnit, MySQL, Eclipse, PHP or JBoss. A recent poll asked to compare the quality of open source and commercial software development tools. And the winner is… not obvious ;o)

Open source versus commercial tools 2006 (2004 answers)
Same quality: 38% (32%)
There is no easy answer to this question: 22% (24%)
Superior in quality: 20% (26%)
Inferior in quality: 12% 13%
I do not use open source tools: 6% (4%)
I do not use commercial tools: 2% (1%)
Participants: 524 (312)

Source: http://www.methodsandtools.comFor many participants, there is no difference in perceived quality between open source and commercial tools for software development. For 22% of the participants, it was difficult to give a precise answer. Diversity exists in both worlds and it is not easy to give a clear indication when you have experiences giving opposite indications. Things have not changed a lot since our 2004 poll, even if the usage of open source tools has surely increased in the mainstream development shop. This may be a reason of the 6% decline in percentage in the "OS software is better than commercial" category, as open source has been more used and could revealed some limits.

The claim that open source software is as good as commercial one seems easy to understand. Besides their open source label, there is little difference in the available support infrastructure between products like JBoss, PHP or MySQL and their commercial competitor. Backed by large companies like IBM, products like Apache or Eclipse will surely receive more testing than a small project in SourceForge. For open source software development tools, a large user base also increases the probability that associated professional services are created to provide commercial support and that the quality of the software is "commercially" managed.

For 20% of the participants, open source development tools are superior in quality to commercial ones. Besides the results of our informal pool, there have been some studies to compare the quality of open source and commercial products. Part of these studies have investigated a claim by many open source software advocates that their code quality was higher. Peer review and the amount of feedback from users are quoted as allowing open source software to achieve high quality results. When it decided to release some software in the open source world the NASA gives "to increase NASA software quality via community peer review " as its first motivation (see references). But if the size of the development team and a smaller user base could be a problem for small vendors, larger commercial organisations could also have implemented internal peer review and they have also a user community with adequate feedback channels. So why could the feedback loops and quality perception be better in the open source community?

Several factors could influence this perception:

- Developers and users (not customers!) have a higher sense of product’s ownership. They feel that they both contribute to something special and it is not "just a job" or "just a product"

- The relationship between users and developers are less confrontational because
a) money is not the matter
b) expectations are often different: the product is "younger" and… there is not a marketing organisation sometimes over-selling the benefits ;o)
c) open source organisations seems to have a better responsiveness to customers request/bugs as the process is more collaborative than confrontational

Some references on the quality of open source software:

http://opensource.arc.nasa.gov/
http://scan.coverity.com/
http://www.artima.com/weblogs/viewpost.jsp?thread=21730
http://www.cyrius.com/publications/michlmayr_hill-reliance.pdf
http://opensource.mit.edu/papers/michlmayr_hunt_probert-quality_practices_problems.pdf

http://www.theglobeandmail.com/servlet/story/RTGAM.20060420.gtflkhaledapr20/BNStory/Technology/einsider

Zutubi is pleased to announce the immediate availability of free Small Team liceses for the Pulse continuous integration server. Small Team licenses are fully-featured licenses for up to two users and two projects on a single server. Zutubi has decided to make these licenses available to support small development teams and home users who would like to take advantage of continuous integration.

Small Team licenses are available from the Zutubi website. The licenses are free: there is no charge or obligation on your part. Enjoy using Pulse!

When we write unit tests for our functions, we usually initialize some input data, call our function and then "assert" on the output data. But some functions take files as input or return their output in files. Where do you place these files so that tests will run on the integration server as well as all developer machines?

This was a problem we faced on our current project, and we solved it by putting the files inside the unit tests.

See how we did it on BestBrains.

The second, in a monthly podcast series on early software quality techniques and solutions – personalized with a disco twist – introduces test categorization as a technique for reducing software build durations.

Have a listen!! (MP3 file)

Steve Matyas of 5AM Solutions and Paul Duvall of Stelligent developed an Ambient Orb Ant task that changes the color of an Ambient Orb based on a build event.

The Ambient Orb Ant task project is hosted at Quality Labs, a development and distribution environment for the open-source community.

Read the announcement.

Methods & Tools is a free e-newsletter for software developers, testers and project managers.

Summer 2006 issue’s content:

* Preventing Project Failure
* Agile Adoption at British Telecom
* Managing an Open Source Project

30 pages of software development knowledge.

Click here to download or read this issue.

To the tune of the disco era, Stelligent announces their new series of short podcasts related to early quality techniques and solutions. This month’s podcast is on Continuous Integration.

If you’re curious about the quality of your software and like disco music, check out this 5 minute audio podcast, available for a free download, with Stelligent CTO Paul Duvall and President Andy Glover.

Have a listen!

Haven version 1.2 released!

This new version of Haven introduces browser compatibility testing as well as continuous integration for Selenium Remote Control users.

More about Haven here.

Allan Wick, the founder and lead developer on the CruiseControl config project, which provides a configuration and monitoring mechanism for CruiseControl was interviewed by Stelligent Incorporated recently. The interview was broken into two parts:

What is CruiseControl config?
Where is the project going?

Ever since Martin Fowler and Matthew Foemmel’s influential article on Continuous Integration was originally published, the CI ecosystems has spawned a host of tools from entire CI platforms like LuntBuild and DamageControl and value adding tools such as CruiseControl config.

Read the full interview: http://www.stelligent.com/content/articles/article.php?topicId=111

In a recent interview, Ward Cunningham talks about social software, organizing for collaborative development and the future of software.

SQLSummit.com Podcast
(MP3 audio, playing time 20:01)

He discusses an evolution from using compilers for custom languages, to OOP and Smalltalk, extreme programming (XP) and agile development. He also discusses global collaboration, scripting, static vs. dynamic typing and the future of software.

Ward is the father of Wiki and Director of Community Development at the Eclipse Foundation.

At the beginning of this year, Methods & Tools run a poll on its web site to examine the way unit testing is performed in software development organizations. Here are the results:

Unit testing is not performed 13%
Unit testing is informal 46%
Unit tests cases are documented 11%
Unit tests cases and their executions are documented 16%
We use a Test Driven Development approach 14%

Participants: 460

These results do not claim any scientific value, but they give some information on the adoption of TDD in organizations. The rate of adoption of TDD could be considered as encouraging for a relatively young approach. This rate could also be put in perspective with a poll on agile approaches targeting a similar audience. In this poll run at the beginning of 2005, 40% of the participants have adopted and deployed, partially or completely, agile practices. This should mean that around 1/3 of agile practitioners are using TDD.Unit testing is still performed informally by a majority of participants. This is symptomatic of the small consideration that is given to the testing activities in most software development projects. When the pressure to deliver is big, testing informally makes it easier to execute poorly without being noticed as you don’t have to provide evidence of your activity. It is however recognized that unit testing is an important building block of system quality and that it costs more to correct errors discovered in later project phases. Good documentation of unit tests allows also to improve maintenance when the original developer has left the project or the company, because it can limit the occurrence that the correction introduced a negative side-effect.

From this point of view, it is already encouraging to see that 41% of the participants are documenting their unit testing efforts. These percentages are already important, as we know that documentation is not the preferred activity of software developers. They could be explained by the emergence of a wide range of open source unit testing frameworks in the xUnit family. They are the tools that should lead to more and more repeatable unit tests.

Source: Methods & Tools (www.methodsandtools.com)

Issue 9 of ObjectiveView magazine is now out – with articles on Ruby, Rails, AspectJ and Ajax. Of particular interest to test-driven development fans is the Rails article, which desribes how to do TDD using Rails.

Issue 8 also contains two articles of interest to TDDers: Unit Testing by Elfreide Dustin, and Combining TDD with more traditional UML approaches by Matt Stephens.

See Objective View’s home page.

Installer v. 1.3.1 build 1.0.2239 allows to use any edition of AnyUnit free of charge for non-profit and academic development (Community Edition) at www.anyunit.com.

Installer v. 1.3.1 build 1.0.2213 is moustly a bugfix release with one new important feature: ability to persist unit test XML output for better integration with automatic build tools such as CruiseControl.

AnyUnit is an add-in for MSVS 2003 that supports many unit-testing frameworks for C/C++/Managed C++/C#/VB.NET. Visit anyunit.com for details.

Methods & Tools is a free e-newsletter for software developers, testers and project managers.
Winter 2005 issue’s content:

* Choosing and Managing the Ideal Test Team by Lloyd Roden
This article presents how to build and manage efficient test team. It also introduces the "tester’s style analysis
questionnaire" to discover the 4 types of tester that exist within organisations.

* Risk Based Testing, Strategies for Prioritizing Tests against Deadlines by Hans Schaefer
This article explains how to improve testing efficiency with risk evaluation.

37 pages of software development knowledge.

To download or read this issue go to the PDF area of http://www.methodsandtools.com

Scott Bellware offers eloquent criticism of Microsoft’s article about TDD on Scott’s Blog.

There is an article on Test Driven Development in the September issue of the Computer magazine of the IEEE with some data on TDD research in the industry.

Research 1: controlled experiment, 3 companies, 24 programmers
Research 2: case study, 1 company, 9 programmers
Research 3: case study, 1 company, 9 programmers

Results – Quality – Productivity
Research 1 – 18% more test passed – TDD took 16% longer
Research 2 – 50% reduction in defect density – minimal change
Research 3 – 40% reduction in defect density – no changeIn two studies, programmers were new to TDD. As we see, TDD seems to improve quality without damage for productivity. The productivity gap for the first experiment is explained by the fact that the control group wrote far fewer tests than the TDD group.

References:
Research 1: B. Georges and L. Williams, "A Structured Experiment of Test-Driven Development", Information and Software Technology, vol. 46, no 5, pp. 337-342.
Research 2: E. Maximilien and L. Williams, "Assessing Test-Driven Development at IBM", Proceedings of the 25th International Conference on Software Engineering (ICSE 03), IEEE CS Press, 2003, pp. 564-569
Research 3: L. Williams, E. Maximilien and M. Vouk, "Test-Driven Development as a Defect-Reduction Practice", Proceedings of the 14th International Symposium on Software Reliability Engineering (ISSRE 03), IEEE CS Press, 2003, pp. 34-45

Source: David Janzen and Hossein Saiedian, "Test-Driven Development: Concepts, Taxonomy, and Future Direction", IEEE Computer, September 2005

In a recent poll, Methods & Tools was trying to determine if the software developers have automated tools to validate the behaviour of their software. The question was: do you use tools to automate execution of functional software tests?

Here are the answers:

38% – My organisation has no tool for functional software tests
26% – My organisation has tools, but my project or I do not use them
36% – I use tools for functional tests

Participants: 147

In these results, you can see either a half-full or a half-empty glass. A majority of organisations have functional test tools, but a majority of the respondents do not use them. There are many good reasons for this, like the fact that these tools could be restricted to a specific environment within the organisation and that the respondents are developing software in a different context. In my experience, the problem is also often that managers had the budget to buy the tools, but did not take into consideration the budget and time needed to train people to the new tools and use them in real projects.

Source: Methods & Tools (www.methodsandtools.com)

Summer 2005 issue’s content:

Agile Development with ICONIX Process by Doug Rosenberg, Matt Stephens and Mark Collins-Cope
This book’s excerpt provides an introduction to a process mixing the Use Case approach and Test-Driven Development.

Using Customer Tests to Drive Development by Lisa Crispin
This article reports a concrete experience of incorporating customer input in a Test-Driven Development approach

The Enterprise Implementation Framework (EIF): Beyond the IDEAL Model by Sinan Si Alhir
This article introduces the CMM’s IDEAL model and EIF, a broader framework for software process improvement wherein IDEAL model activities are balanced with the human perspective.

More than 30 pages of software development knowledge.

Methods & Tools is a free e-newsletter for software developers.
To download or read this issue go to the PDF area of methodsandtools.com.

In a recent poll, the Methods & Tools newsletter asked the following question: At what stage is the agile approach (XP, Scrum, FDD, …) adoption at your location?

Not aware – 26%
Not using – 16%
Investigating – 14%
Analysed and rejected – 3%
Pilot projects – 4%
Partial implementation (adoption of some agile practices) – 17%
Partial deployment (some projects are using this approach) – 12%
Deployed (all new projects are using this approach) – 8%

Participants: 232 There is a good balance between the three main situations of agile approaches’ adoption (no, maybe, yes):
- around 40% of the participants’ organisations are not using an agile approach or practice
- around 20% are investigating it or running pilot projects
- around 40% have adopted, fully or partially, an agile approach to software development.

The numbers on agile approach usage could be slightly biased. As Methods & Tools has published some articles on this topic in its recent issues, the proportion of agile practitioners in its readership and web sites visitors could be higher than in the "real world".

If you look at each question, you could be shocked that 26% of organisations are even not aware that something like an Agile movement have been active in the software development world these recent years. It is however a sad reality that some organisations are functioning in a "not invented here" mode. They think that their situation is "the best they can get" and they are not interested in what is happening "outside". It is also true that agility is people oriented and that most organisations have a tendency to be focused on process oriented activities. There is also a high number of respondents with partial implementation of agile practices. Even if some purists could argue that you cannot be 50% agile, this is another point showing that agility is reality oriented. You can rarely change completely and immediately a software development process.

Source: Methods & Tools