Improving software quality and QA productivity through automated testing
By David Barron, Senior Systems Software Engineer
Automating tests helps improve the quality of software and the productivity of software engineers by replacing manual tests, which can be difficult to run, very time consuming, and prone to errors. At Synergex, we evaluate every bug that is fixed and every feature that is added to see if a test for it can be automated, and we automate whenever possible. We run a specific set of automated tests whenever we make changes to our products to ensure we don’t reintroduce one bug while fixing another. As you can imagine, our test suite gets larger with each release!
Most of the automated tests we have at Synergex are written in Perl, which is cross-platform, installed on all the machines on which we need to run scripts, and free. With about 250 Perl scripts written, it has proven to be a versatile and powerful tool. Most of our Perl scripts run a command (or a set of commands) and verify the output. Perl scripts are used to test a broad range of products, from the compiler and runtime to the XML API. Some of our Perl scripts are quite complex, running thousands of commands.
Perl scripts run quickly. Our compiler test suite runs over 2500 compile commands and compares the output to saved output in less than 8 minutes! That time includes the building of the various repository files used to test the Repository .INCLUDE syntax and the building of prototype files used in testing the IMPORT statement. You can find more information about Perl at www.perl.org.
For user interface testing, we use TestComplete from SmartBear. TestComplete enables us to test user interaction with applications, such as clicking the mouse at a specific place and seeing what happens. We originally chose TestComplete for its support of Windows, its cost, and the ability to view the things we’re testing in both Windows and Visual Studio.
We use TestComplete to test UI Toolkit, Repository, Workbench, Composer, and our Synergy DBL Integration for Visual Studio. TestComplete enables us to write scripts in a wide variety of programming languages, but we chose to implement our tests in JScript because we felt it was more versatile than the other languages TestComplete offers. It allows for a number of different checkpoint types; for example, you can compare screen regions or files or strings on the screen.
We also use TestExecute to execute the automated tests. This allows us to run TestComplete scripts in TestExecute while developing and testing other tests in TestComplete. TestExecute is a scaled-down version of TestComplete that enables us to run tests and view the results on systems that don’t have access to TestComplete. It supports all of the scripts we create with TestComplete and allows us to run large groups of tests in any test environment required. The TestExecute license must be purchased separately, but it costs considerably less than TestComplete. You can find more information about TestComplete and TestExecute at www.smartbear.com/TestComplete.
