The reigning management maxim is, “If you don't measure it, you can't manage it.” For IT organizations, the operative maxim is, “If you don't test it, you can't trust it.” That trust carries a hefty price tag: In 2002, AMR Research reports, software defects cost business $60 billion to fix, and 50 percent of software costs were related to finding and fixing those defects.
In an age of accelerating product lifecycles and restrictive cost pressures, the impact of the traditional approach to testing has had serious consequences that have put IT organizations between a rock and a hard place. In today's business climate we were asked to do more with less and they're expected to deliver higher quality systems in less time with fewer resources.
The solution to this is framework-driven test automation. Framework driven test automation can greatly reduce testing cycle times, especially regressions. It also reduces costs and effort, and supports iterative development and more frequent releases.
Framework-driven test automation, just like Rome, wasn't built in a day. It takes a great deal of work and time to build and maintain the test automation tools that drive the process and contribute to the results. And framework driven test automation doesn't produce instant gratification.
Framework-driven automation is an innovative approach to software testing that leverages proprietary automation frameworks and tool sets from leading vendors to maximize the benefits of test automation.
Test automation framework
Here i will provide the brief intorduction about various types of the framework, and the analysis of the framework for the application under test (AUT). This also detailed description of the format of the input (test data table) that is given to our test automation framework.
Different frameworks available are
- Test Modularity or Record and Playback
- Test Library Architecture or Procedural apporach
- Data - Driven
- Keyword Driven or Table Drive0n Testing
- Hybrid Testing
Source: http://www.wilsonmar.com/
Test Modularity
Most basic of the frameworks also known as record and playback method. It's a well-known programming strategy to build an abstraction layer in front of a component to hide the component from the rest of the application. This insulates the application from modifications in the component and provides modularity in the application design. When working with test automation scripts this is achieved by creating small, independent scripts that represent functionality, modules, and sections. Then these small scripts are taken and combined them mostly in a hierarchical fashion to construct larger tests. The use of this framework will yield a higher degree of modularization and adds to the overall maintainability of the test scripts.
Merits of Test Modularity Approcah
- Scripts may be developed are independent
- There is a higher degree of Modularization
- Easy to build
- Mainly used for automation for small independent projects (3-4 months)
Test Library Architecture
The test library architecture framework is very similar to the test script modularity framework and offers the same advantages, but it divides the application-under-test into procedures and functions (or objects and methods depending on the implementation language) instead of scripts. This framework requires the creation of library files (SQABasic libraries, APIs, DLLs, and such) that represent modules, sections, and functions of the application-under-test. These library files are then called directly from the test case script. Much like script modularization this framework also yields a high degree of modularization and adds to the overall maintainability of the tests.
Data-Driven Testing
A data-driven framework is where test input and output values are read from data files (ODBC sources, CVS files, Excel files, DAO objects, ADO objects, and such) and are loaded into variables in captured or manually coded scripts. In this framework, variables are used for both input values and output verification values. Navigation through the program, reading of the data files, and logging of test status and information are all coded in the test script. This is similar to table-driven testing (which is discussed shortly) in that the test case is contained in the data file and not in the script; the script is just a "driver," or delivery mechanism, for the data. In data-driven testing, only test data is contained in the data files.
Merits of data-driven testing
- Scripts may be developed while application development is still in progress
- Utilizing a modular design, and using files or records to both input and verify data, reduces redundancy and duplication of effort in creating automated test scripts
- If functionality changes, only the specific "Business Function" script needs to be updated
Keyword-Driven Testing
This requires the development of data tables and keywords, independent of the test automation tool used to execute them and the test script code that "drives" the application-under-test and the data. Keyword-driven tests look very similar to manual test cases. In a keyword-driven test, the functionality of the application-under-test is documented in a table as well as in step-by-step instructions for each test. In this method, the entire process is data-driven, including functionality.
Merits of keyword driven testing
- The Detail Test Plan can be written in Spreadsheet format containing all input and verification data.
- If "utility" scripts can be created by someone proficient in the automated tool’s Scripting language prior to the Detail Test Plan being written, then the tester can use the Automated Test Tool immediately via the "spreadsheet-input" method, without needing to learn the Scripting language.
- The tester need only learn the "Key Words" required, and the specific format to use within the Test Plan. This allows the tester to be productive with the test tool very quickly, and allows more extensive training in the test tool to be scheduled at a more convenient time.
Hybrid Test Automation Framework
The most commonly implemented framework is a combination of all of the above techniques, pulling from their strengths and trying to mitigate their weaknesses. This hybrid test automation framework is what most frameworks evolve into over time and multiple projects. The most successful automation frameworks generally accommodate both Keyword-Driven testing as well as Data-Driven scripts.
This allows data driven scripts to take advantage of the powerful libraries and utilities that usually accompany a keyword driven architecture. The framework utilities can make the data driven scripts more compact and less prone to failure than they otherwise would have been. '
The utilities can also facilitate the gradual and manageable conversion of existing scripts to keyword driven equivalents when and where that appears desirable. On the other hand, the framework can use scripts to perform some tasks that might be too difficult to re-implement in a pure keyword driven approach, or where the keyword driven capabilities are not yet in place. The following sections describe its architecture, merits and demerits.
