Automation Testing

Test Automation using Selenium & JMeter

A Logsheets management system designed by Amicus keeping in minds the lack of integration between the production floor and the enterprise floor, in most manufacturing plants that utilize ERP solutions. The application is a Data Collection system that digitize the paper trail data, enabling manufacturing plants to save time in capturing inputs, ensuring accuracy, maintaining compliance and most importantly, enabling the company to receive real-time updates of plant floor level activity and reporting (downtime report, production report, customized reports etc.), irrespective of locations.

Business Situation

Application was used across multiple locations by different users, there were different browsers used for accessing the application like IE, Chrome and Mozilla etc. Due to different browsers used, some functionalities were not supported as expected in all the browsers. The business also wanted to facilitate change management and control,to speed up the creation, updating of test scenarios/scripts and bringing scalability to the existing test cases. Furthermore it reduces costs of maintenance for test operation activities. User experienced slowness in the application while executing some particular business scenarios, and less response time when number of concurrent users increases.

Frequent product releases were required to accommodate bug fixes & enhancements for upgrading the product. Beta releases and production releases were vital part before marketing the product.

---

Technical and Business Challenges

The client was looking to automate the regression test suite for our web-based application to minimize the effort required to execute them any time a change is made to the application. They wanted a robust framework which would allow to add and remove tests from the test scenarios with minimal changes in automation test scripts. They were also looking to provide redundant test data in an external file which can be changed any time without having to break existing scripts and require less maintainability. The key technical challenge was establishing a Selenium data driven framework using configuration property file along with JMeter. The automation testing framework was also required to take understanding and convenience of end users into account.

Frequent product release cycles for bug fixes & enhancements became challenging to adhere systematic releases i.e. Controlled Availability (CA) and General Availability (GA) due to manual testing efforts involved.

---

Our Solution

Team proposed a hybrid framework using Selenium which would take advantage of both data-driven and keyword-driven testing features of Selenium. The whole application was divided into different modules by the functionality and consulted with the customer to prioritize the modules and delivered the most important ones first. We developed the test automation framework from scratch and written scripts for test automation, which provided significant ROI by minimizing maintenance and facilitating ease-of-use. While executing the test cases, integration complexities were taken into account and defects were well documented. Every time any change was incorporated, the application underwent the Selenium automation testing process again to establish integration credentials. Test results were reported in HTML files using Test NG which could in turn be emailed to the stakeholders automatically after test execution. Different types of testing scenarios were created for validation of performance checks at various levels, for generating JMeter scripts. JMeter scripts were developed and configured in a distributed test lab to simulate different usage scenarios for their application. Concurrent user load was created on the system by simulating normal functions performed by end users with different roles such as, supervisor, administrator, foreman and operator. These scripts were executed after every major deployment and results were captured in excel spreadsheets.

Ease of Automation Testing helped product team to plan and execute Controlled Availability validations for timely General Availability releases.

---

Solution Benefits

Lower cost of automation as the open source tool Selenium and JMeter were used to perform the testing. Automation helped to reduce the regression testing from 80 hours to 10 minutes tremendously minimising the execution time & increasing the test cases reusability and ultimately the productivity. The Unit Testing & Component Testing, was performed which resulted in test coverage of critical functionality by 200%, it increased with each subsequent automation build-out. The results output and various reporting graphs were used to analyse the behaviour of the system with various combinations. Results helped the customer’s management team to prioritize further enhancements and benchmark the applications performance. Controlled Availability releases provided ample opportunity to validate the product by internal users for full proof system, before opening it to external users for General Availability releases.