testdriven.com Wrangling quality out of chaos

Archive for September, 2009

Automated GUI Testing: Squish Success at Perforce Systems

09.30.2009 · Posted in Articles

Introduction

Perforce Software, founded in 1995, markets and supports Perforce, the Fast Software Configuration Management (SCM) system. Perforce is an award winning SCM system used to version and manage source code and all digital assets. Perforce SCM seamlessly handles distributed development and multi-platform environments and is used by more than 330,000 developers worldwide.

Recently, Perforce adopted Squish for GUI performance testing of their SCM system’s flagship GUI, the Perforce Visual Client (P4V). We discussed Perforce’s use of Squish with Tim Brazil, a Perforce Performance Lab engineer.

Why Squish?

In mid-2009 Tim Brazil chose Squish to round out his testing arsenal and to help ensure that performance remains high as new features and bugfixes are applied to P4V. P4V, written in C++ and using the Qt GUI library, provides a graphical interface on Windows, Mac OS X, Linux, Solaris, and FreeBSD.

One feature in particular Tim liked was Squish’s support for multiple standard scripting languages for writing tests—these currently include JavaScript, Perl, Python, and Tcl. Using a standard scripting language avoids the need for testers to learn an obscure proprietary testing language, and means they can benefit from the large number of third party modules available for standard scripting languages.

Squish provides a complete GUI-based testing IDE capable of recording and playing back GUI tests, but for Tim, it was also Squish’s support for command line use that was particularly appealing.

"I prefer to work in a command-line environment that facilitates the use of scripting languages," Tim explained. "It is evident that froglogic’s approach took engineers like myself into account when they designed the product. For example, features like the envars file, suite.conf file, squishserver, and squishrunner, allow me to design a fairly complex test environment with relatively minimal work."

Tim has many years of experience in the testing field and is a strong proponent of using both automated and manual testing.

"I believe the best software testing solution is to use a good mix of automation and manual tools," said Tim. "By utilizing the strengths of both, producing superior products can be achieved."

"The benefits of automated testing include reliability, repeatability, comprehensiveness, and speed," Tim continues. "Furthermore, automated tests can quickly and dependably navigate through test scenarios and are, therefore, more likely to uncover subtle timing problems." He pointed out that manual testing alone is often both slow and fallible, and that even with a comprehensive test plan at hand it can be difficult to reliably recreate the exact test actions and environment.

Squish at Perforce

Tim uses Squish to test the performance of P4V, specifically on Windows Professional, Mac OS X, and SLED (SUSE Linux Enterprise Desktop). Squish is used to test the nightly build as well as the previous three P4V releases. A daily performance report is generated, identifying performance trends and pinpointing any possible areas of concern as P4V continually evolves.

Each test Tim uses is designed to work independently—if one test fails, it has no effect on subsequent tests. No third party tools are used; instead, a custom Perl script (runner.pl) runs the Squish tests. This script runs each set of Squish P4V tests for each version of P4V on each client machine, accumulating results into an XML file (see the screenshot below). Once the tests are complete, the XML results file is automatically parsed and converted into an HTML report for the test team’s review each morning.

Squish in Practice

We asked Tim what features of Squish he liked most, apart from the multiple scripting language and command-line tools support he’d already mentioned.

Squish’s Object Map came to mind: "At previous companies I have used graphical test tools that were heavily invested in using coordinates to identify objects. This was a maintenance nightmare. Squish’s objects.map is unique and greatly facilitates test readability, robustness, and maintenance."

Squish uniquely identifies application objects such as widgets, using the values of their properties. For every identified object, Squish also creates a corresponding symbolic name—the symbolic name is the one that is normally used inside test scripts. This means that if a developer changes one of an object’s properties, the test engineer only has to update the corresponding object property once in the Object Map to reflect the change. The symbolic name used for the object will continue to work in all the tests where it is used.

In some cases, an object property’s value may vary depending on the platform the application is being run on. Squish’s Object Map can accommodate such challenges since the properties used to identify an object can not only be matched for equality, but also using wildcard or regular expression matches—a feature that Tim has found to be particularly useful.

"One of the biggest challenges I’ve faced with automated GUI testing is the ability to maintain a robust test as the product changes and evolves over time," Tim continues. "This is a problem because GUI application performance can vary on different operating systems. One particular issue we’ve had is being able to identify when a web page or GUI application object is ready to be tested, since load times vary. The wait*() functions Squish provides have been invaluable in helping me tune a test so that it provides consistent behavior when run on various platforms."

In addition to Squish’s documented features, Tim has found Squish’s technical support team very helpful: "I was impressed with froglogic’s support from the moment I started to evaluate Squish. Just because something isn’t stated in the documentation does not mean it cannot be done. Squish’s technical support team has quickly and efficiently provided solutions that work."

Conclusion

Perforce’s Performance Lab depends on the reliability and repeatability of Squish tests to check application performance as well as behavior across multiple platforms. Squish’s usefulness and flexibility has allowed Perforce to rapidly adopt Squish as an integral part of their performance quality monitoring process. This has lead to time and cost savings compared with the previous manual testing, and at the same time ensured that tests are automatically and reliably repeated to ensure product quality.

Making a third party tool an integral part of P4V’s testing process is a significant commitment, but one that Perforce has chosen to make with Squish.

froglogic’s team would like to thank Tim for taking the time to share Perforce’s experience with Squish, and we are looking forward to a continued successful relationship.

More about Squish and froglogic at http://www.froglogic.com

Simplify creating mock objects with Mockito and the Builder Pattern

09.07.2009 · Posted in Blogosphere

You know that if you do unit testing at some point you will run into mocking. There are many libraries out there like JMock, EasyMock, but my favorite is Mockito. In this post I explore the option of using a Builder Pattern to simplify the creation of a hierarchical mock object that contains nested mock objects itself.

Take a look at the posting:
http://www.dzone.com/links/simplify_creating_mock_objects_with_mockito_and_t.html

GUIdancer 3.1 supports GEF components

09.07.2009 · Posted in News

Bredex GmbH announced today that version 3.1 of GUIdancer, the automated functional testing tool, is now available for download. As well adding support for the testing of GEF (Graphical Editing Framework) components, the new version is available for MAC OSX as a beta, and is compatible with Eclipse 3.5.

Increased platform and application support are not the only areas where the Eclipse-based test tool has been expanded. GUIdancer’s efficient keyword-driven approach has also been extended to improve the testing of dynamic tables.

“Version 3.1 allows us to reach testers working on GEF applications and MAC OSX who want to attain a long-lasting and maintainable test-coverage, yet see the first successes from day one" said Hans-Joachim Brede, a Managing Director at the company. “We all know that a structured, modular approach is the key to successful test automation. GUIdancer’s simple yet powerful concept supports reusability at every stage of the test process. Test coverage grows quickly and maintenance is drastically reduced compared to other approaches where programming effort is necessary to automate tests.”

GUIdancer allows testers to automate tests from the user perspective without writing or editing any program code. Tests are created either by drag-and-drop from a library of actions or by recording actions from a running application. Both options lead to read-able tests which are easy to expand, change and maintain against changes in an application.

Since releasing version 3.0 in April, the GUIdancer team has also added new actions to the GUIdancer library, including a simpler way of testing native dialogs. The software is available for download on the GUIdancer website, and a fully functional version can be tested with a demo-license. Live-demos of the software are available on request.

BREDEX GmbH

Since 1987, BREDEX GmbH has been providing high quality information technology services in the areas of consultancy, analysis and design, software development, quality assurance and training. Based in Braunschweig, the company has a history of long term customer satisfaction and a focus on forward thinking, developing individual solutions that bring future standards to current project needs. BREDEX sells GUIdancer, which is a powerful and cost effective automatic test tool for Java and HTML applications.

For more information: www.bredexsw.com.

This press release can be downloaded from: http://www.bredex.de/en/news/first.html

TestRail Test Management

09.07.2009 · Posted in Links

TestRail is a comprehensive web-based test case management software to efficiently manage, track and organize your software testing efforts.
link

Tags:

Coding and Testing: Testers and Programmers Working Together

09.07.2009 · Posted in Links

The idea of driving development with tests has been popularized by the agile development movement. The fact is that testing and coding are inseparable components of software development. We get the best results with testers and programmers work closely together. How can we deliver real value to the business frequently? How can we know how much testing is enough? Let’s look at how testers and programmers collaborate to produce high-quality software.
link

Tags:

Responsibility Driven Design with Mock Objects

09.07.2009 · Posted in Links

Responsibility driven design is an approach that helps you shift focus from object state to interactions and responsibilities. This article shows how test driven development with mock objects facilitates responsibility driven design and drives you towards a more cohesive, loosely coupled design. Examples in Java.
link

Klaros-Testmanagement 2.0 released

09.07.2009 · Posted in News

Klaros-Testmanagement, the free Ajax-based web application for management of the quality assurance process in software development projects is now available in Version 2.0.

The multitude of enhancements include as the most prominent changes:

– A completly redesigned user interface
– A separate client application for a journaled execution of manual tests
– An interface to the Atlassian JIRA issue management system

All further details can be found in the release notes at http://www.klaros-testmanagement.com/forum/.

For more product information, the download of the free of charge Community Edition, the user forum and much more please visit http://www.klaros-testmanagement.com/.

For more information about verit Informationssysteme GmbH please visit http://www.verit.de/.

New Release: web2test web testing software version 1.3

09.07.2009 · Posted in News

New release offers improved handling of AJAX and already supports the new FireFox version 3.5

The new release of itCampus’ innovative test software for automated GUI tests of web applications – web2test – supports besides Internet Explorer 6, 7 and 8, Mozilla 1.8, Seamonkey 1 and 2 Beta and FireFox 2 and 3 already the newest FireFox version 3.5.
To further improve the handling of AJAX the mutation listeners have been revised.
In addition version 1.3 offers the possibility to record and replay get/post requests. Hence web2test is even better suited for GUI driven performance tests.The general handling of web2test has been improved as well thanks to several new features:
The Checker interface of the new version makes it even easier to extend the already integrated standard checks.
Test suites, run logs or individual test suite nodes can now be bookmarked to make them directly accessible via the new sub-menu File -> Bookmarks.
Furthermore several new variants for converting and (un)packing nodes have been added.

Note: The English version of the web2test website is now available at www.web2test.com.

Contact: Benjamin Franke, Marketing web2test, Tel: +49. 341. 4 92 87 29, Fax: +49. 341. 4 92 87 01, E-Mail: b.franke@itcampus.de

About web2test
web2test has been developed as a co-operation project between itCampus and Quality First Software (QFS). It is a tool for functional system tests of web based applications, portals and complex websites. Further USPs: Tests are platform and browser independent and tolerant towards layout changes. In addition web2test offers a real user simulation for a 100 % AJAX support. The test tool can be used under Windows as well as under Linux. www.web2test.com

About itCampus
itCampus is an innovative software development company closely cooperating with international scientists and researchers. Founded in 1999 in the cities of Halle and Leipzig in Germany, the company today has offices in Germany, the UK, Switzerland, Italy, Austria and Slovakia. itCampus provides communication solutions, IT consultancy services and custom solutions in the fields of software development and software ergonomics. Since April 1 2009 the Software AG is itCampus’ majority shareholder. www.itcampus.eu

TestRail Test Case Management Software introduced

09.07.2009 · Posted in Advisories

Gurock Software is introducing their upcoming test management software TestRail. TestRail is intended to help software development teams and quality assurance departments manage test cases, track test results and organize software testing efforts. Gurock Software is still looking for additional beta testers who are interested in taking a look at the new tool, which is scheduled for release in Q3 2009. The first details and screenshots of TestRail are available on Gurock’s website.