testdriven.com Wrangling quality out of chaos

Archive for March, 2009


03.31.2009 · Posted in Links

jsUnity is a lightweight JavaScript testing framework that is context-agnostic. It doesn’t rely on any browser capabilities and therefore can be run inside HTML, ASP, WSH or any other context that uses JavaScript/JScript/ECMAScript.

There are already several unit testing frameworks for JavaScript that are browser-centric. jsUnity aims to just focus on JavaScript as a language that’s used beyond the browser context. jsUnity can be used in server-side JavaScript implementations as well as all popular browsers.

Automated GUI Testing: Q1 Labs – Squish for Web users since 2007

03.15.2009 · Posted in Advisories

Q1 Labs is a global provider of high-value, cost-effective, security information and event management (SIEM) products. They have been using Squish since late 2007 to test the Web UI of their entire QRadar-branded family of products.

Q1 Labs products are web applications that use JSP to render HTML pages with JavaScript used for dynamic elements. Their test automation takes place on several Windows virtual machine images, and using the browsers they support, such as Firefox.

We spoke to one Q1 Labs’ QA Engineers, Chris Fredericks, and asked him about their experience of using Squish.

Why Q1 Labs is using Squish

Q1 Labs first learned about Squish through web research. Chris told us that:

We were already using a GUI test automation tool but we were not happy with it, nor with the direction its development was taking. This made it essential for us to find a new tool that would suit our needs.

Several factors were influential in Q1 Labs’ decision to choose Squish over competing products. Chris summarized some of them for us:

"We wanted scripting support, and Squish provides this with its support for four well-known and highly supported scripting languages. Some of the other tools we considered either provided their own proprietary scripting language, or no scripting support at all."
"We also liked Squish’s approach to interacting with the AUT via scripting—it seemed the most logical way. And in our case we chose to use JavaScript, since we already had a lot of experience with that language."
"We needed the flexibility to automate our testing, and Squish made this easy with its command line tools. We currently use our own in-house batch running and reporting tool, and it was easy to plug Squish script execution into our framework."
"We like to get value for our money—and we felt that Squish’s pricing met this requirement."

We asked Chris more generally about Q1 Labs’ experience with using Squish, starting with the feature they found most useful:

"Given that our product contains quite a lot of data tables we would have to say that the inclusion of XPath has been a huge advantage for our automation effort."

Squish’s support for XPath allows Squish/Web testers to perform XPath queries on arbitrary elements in HTML documents. (XPath is a standardized XML querying language that many consider to be more powerful and flexible than multi-property DOM queries as a means of navigating and accessing attributes in XML documents.)

We also asked what problems and challenges Q1 Labs had encountered in their use of Squish:

"We really have encountered no big challenges; Squish does the job we want it to do. The only issue we ran into was in the handling of Alert/Confirm windows—an area we know that froglogic is working to improve.

Squish’s technical support is something that many customers depend on. We asked Chris what his experience had been, and he told us:

"From the demo phase up until now the team at froglogic have worked hard to answer questions and work on issues. So yes, we are very satisfied with the technical support we have received."

Scope and Benefits

Q1 Labs already have more than 800 test cases, with more in production. We asked Chris what benefits their automated testing had delivered:

One of the main benefits has been the decrease in the time it takes for our regression testing cycles. What used to take a few QA members weeks can now be done in a few days through automation. This also means that we can run full regression suites more often during our development cycle, instead of waiting until the end.

Chris also mentioned that recently Q1 Labs has extended its testing effort:

"Through the past couple of development cycles we have been automating some functional tests during the initial development phases. This has allowed us to thoroughly test new features as soon as they are available, and to raise bugs much earlier in the cycle."

Chris also told us that although at present Q1 Labs use their own in-house batch test running and reporting tool, with Squish as the only third party product in their test tool chain, they are currently considering using some complementary tools.

"It’s good to know that from Squish’s end we’ll be able to integrate our current automation suites with, for example, Testlink, if we decide to go that route."


Q1 Labs now has considerable experience with testing using Squish and has built up an impressive set of test cases which they are actively extending. And their experience of integrating Squish with their own in-house tools has given them the knowledge and confidence to integrate Squish with other tools if they decide to extend their tool chain.

Chris told us how test automation with Squish had enabled Q1 Labs to reduce their regression testing cycles from weeks to just a few days, with all the consequent benefits that such time saving and fast turnaround can deliver.

froglogic’s team would like to thank Chris for taking the time to share Q1 Labs’ experience with Squish, and we looking forward to a continued successful relationship.

About froglogic

froglogic GmbH is a software company based in Hamburg, Germany. Their flagship product is Squish, the market-leading automated testing tool for GUI applications based on Qt, Java AWT/Swing and SWT/RCP, Mac OS X Carbon/Cocoa and for HTML/Ajax-based web applications running in a variety of web browsers. froglogic also offers services in the areas QA/automated testing and C++/Qt programming and consulting. For more about froglogic visit http://www.froglogic.com.


03.15.2009 · Posted in Links

The Software Quality Planet is a aggregator for RSS feeds concerning software testing (unit testing, functional testing, regression testing, load testing), code review and inspection, bug and defect tracking, continuous integration.


JUnitMax beta is worth the price

03.07.2009 · Posted in Java

Kent Beck has released JUnitMax for Eclipse as paid beta – currently USD $2/month. JUnitMax is a selective testing tool – it chooses a subset of tests to run so you get feedback very fast. I especially like the fact that the results are written into the same window as compiler failures. Think of JUnitMax as the twin sister of the incremental compiler.

I’ve tested it with the latest version of Apache Commons Collections and it works quite well. Of course its usefulness depends on whether you have fast and reliable tests in the first place, but if your project can benefit from fast tests – you have one less excuse not to write them. :)


New Agile Browser Automation Testing framework released – LiquidTest

03.05.2009 · Posted in News

Why is it so hard to test dynamic and Ajax based web pages? They work in your browser, so why do so many test tools struggle with this? This was a core focus for JadeLiquid as we developed LiquidTest, our brand new automated Web Application testing framework.

Built upon Browser Automation, LiquidTest consists of an Eclipse rich client application (and Plug-in) for recording Browser actions in Firefox™ and Internet Explorer™ and outputting succinct and robust
test cases in Java™, JUnit and TestNG, C# NUnit, Ruby and Groovy™.

Core LiquidTest functionality:
– Browser Automation – Firefox and Internet Explorer
– Succinct cross-platform script generation and execution
– Easy recording and replay of test cases
– Integrations with JUnit, Continuous Integration, Eclipse, .NET and other frameworks
– Language support for Java, C# (.NET), Groovy and Ruby
– IDE plugins (Eclipse) and standalone, and headless server component
– Open SDK for plugins and custom integrations
– Headless test case execution and scalable across a Grid
– Data-driven support: Powerful GUI, and direct support for Excel spreadsheets
– Advanced testing features: Expectation model – no sleeps! Drag and drop, Ajax support etc.

The LiquidTest framework consists of three separate editions; Developer Edition, Tester Edition
and Server Edition.

The LiquidTest Developer Edition has been created for Software Developers aiming to
complement their Agile software development practices by integrating Functional Tests (as Unit
Tests) into their Software Development Process. LiquidTest supports headless test case
execution which allows for regression tests at every step of the build process. Support for
Continuous Integration means that the LiquidTest Server can be integrated on your Continuous
Integration machine and utilized in an automated test execution manner to provide complete
Functional and Acceptance Test coverage.

The LiquidTest Server Edition is a Server component specifically designed to add support
for headless LiquidTest test case execution on Continuous Integration servers. See our
Continuous Integration and LiquidTest documentation for more information on
LiquidTest/Ant/Continuous Integration setups.

The LiquidTest Tester Edition is designed for Test and Quality Assurance teams that have less
technical knowledge than the Development team. LiquidTest Tester Edition outputs concise
scripts in LiquidTest Script (Groovy derivative) which in turn is powerful but not syntactically
complicated. LiquidTests recorded with the Tester Edition can be replayed with the Developer
Edition and vice-versa. This provides close integration between Development and Test/QA

Download a Free 30 day trial to see for yourself why LiquidTest is a unique offering in the
Automated Software Quality market: www.jadeliquid.com

Application architecture data published from 2,500 open source projects

03.05.2009 · Posted in Tools

Coverity Inc., the software integrity company, today announced the publication of application architecture data from over 2,500 popular open source software projects at www.scan.coverity.com . As an outgrowth of its contract with the U.S. Department of Homeland Security (DHS), Coverity™ collected this data via the Scan site using the Coverity Architecture Analyzer product. This new, publicly accessible resource includes application architecture files and generated diagrams that will benefit developers planning to incorporate open source packages in their applications, as well as developers that want to learn the architectures of successful projects to improve the structure of their own applications.

"I’ve tried the Coverity Architecture Analyzer application, and it showed me some unexpected references between parts of the code that I want to investigate,” said Volker Lendecke of the Samba Team. “I’m glad to hear the architecture diagrams can be included on our project wiki as documentation to help new developers get up to speed more quickly."

The Scan architecture library is a database of application architecture diagrams from over 2,500 open source projects such as Amanda, NTP, OpenPAM, OpenVPN, Overdose, Perl, PHP, Postfix, Python, Samba, and TCL. Each diagram displays dozens to hundreds of components that comprise a given software project.

According to Forrester Research, “A high-performance scalable architecture depends on continually optimizing your application architecture so that different components built in myriad languages interoperate according to performance best practices. Application architecture is not about "set it and forget it." It requires continuous care and feeding to identify and solve problems, take advantage of new platforms, and handle new loads and new features.” (Best Practices: Attaining and Maintaining Blazing Fast Web Site Performance”, Forrester Research, Inc., February 2009.)

The ability to study a visual presentation of an application’s architecture and related data offers a number of benefits to developers. For example, developers planning to use or build on top of a project in the Scan library can optimize their use of it by comparing architecture plans to other codebases that use the same project. This allows developers to more fully understand the structure and capabilities of the open source software they intend to consume.

For developers writing a plug-in or module for an existing open source package, understanding its architecture helps ensure their design is aligned with the expectations and intent of the target open source application. Finally, developers creating a particular type of application, such as a web server, can study the architecture of similar, successful open source projects to improve and accelerate the development of their own application architecture.

“This new resource is another part of Coverity’s ongoing efforts to provide valuable resources to the software development community, while advancing the state of the art in developing high integrity software,” said David Maxwell, open source strategist for Coverity and NetBSD developer. “Sharing the structural data of a wide variety of application types will be a resource for both open source and commercial developers who are planning to use a specific open source project, as well as developers that are simply curious to learn more about application architectures in general.”

Since 2006, the Scan site has leveraged Coverity’s static analysis technology to analyze 50 million lines of code on a daily basis from more than 250 projects. Since its launch, open source developers have used the Scan site to identify and eliminate over 8,500 security vulnerabilities and quality defects. The new Scan architecture library is publicly available at Coverity’s Scan site http://scan.coverity.com/arch/

The Scan architecture library was created using Coverity Architecture Analyzer. The product automatically maps the relationships between code elements at the function and file levels, identifying the underlying structure of software to help developers identify violations of architectural standards. Coverity Architecture Analyzer requires no change to the source code or build environment.

Open source developers may request inclusion in the Coverity Scan, or complimentary licenses of Coverity Architecture Analyzer for use with the data collected in the Coverity Architecture Database by emailing scan-admin@coverity.com. Trials of Coverity Architecture Analyzer can be downloaded immediately, for use with the Coverity Architecture Library, by registering at http://scan.coverity.com/arch/trial

Developers who participate in the Scan project also receive access to an interactive Coverity Architecture Analyzer application for browsing the library in greater detail.

About Coverity Scan
The Coverity Scan site was developed by Coverity with support from the U.S. Department of Homeland Security as part of the federal government’s Open Source Code Hardening Project. The site divides open source projects into rungs based on the progress each project makes in resolving defects. Projects at higher rungs receive access to additional analysis capabilities and configuration options. Projects are promoted as they resolve the majority of defects identified at their current rung. The Coverity Scan site is freely available to qualified open source projects at: http://scan.coverity.com

About Coverity
Coverity (www.coverity.com ), the software integrity company, is the trusted standard for companies that have a zero tolerance policy for software failures, problems, and security breaches. Coverity’s award winning portfolio of software integrity products helps customers prevent software problems throughout the application lifecycle. Over 100,000 developers and 500 companies including ARM, Phillips, RIM, Rockwell Collins, Samsung and UBS rely on Coverity to help them ensure the delivery of superior software. Coverity is a privately held company headquartered in San Francisco with offices in 6 countries and more than 150 employees.

Ivonna — a Typemock based Asp.Net testing tool.

03.01.2009 · Posted in Links

Ivonna is an Asp.Net testing tool that allows writing unit tests for your Asp.Net applications. What makes it different is that Ivonna runs in the same process as the test runner.

Unlike other testing tools, Ivonna is not a "client-side" tool. It doesn’t just test the HTML output (although such a possibility exists), it creates and executes an Asp.Net request and allows you to examine the intrinsic objects, such as the Page object. This opens many new possibilities, such as

* Examining properties of page controls and other objects, such as the HttpContext object.
* Mocking external dependencies in order to isolate the page behavior.
* Injecting setup code and assertions into the page’s lifecycle event handlers.
* Injecting mock HttpModules, Filters, and test-specific configuration values.

Ivonna is being developed in partnership with TypeMock and runs on top of the TypeMock Isolator framework.

Ivonna — a Typemock based Asp.Net testing tool.

03.01.2009 · Posted in Tools

Ivonna is an Asp.Net testing tool that allows writing unit tests for your Asp.Net applications. What makes it different is that Ivonna runs in the same process as the test runner.

Ivonna is being developed in partnership with TypeMock and runs on top of the TypeMock Isolator framework.

Site: http://sm-art.biz/Ivonna.aspx