Reengineering Application Infrastructure

Reengineering Application Infrastructure Highlights

The Chief Information Officer (CIO) of a large east-coast company was faced with the task of upgrading the company's IT infrastructure without disrupting everyday business. The company started reviewing some 25 servers that were hosting 300+ custom-built applications to manage their data relating to quality, customer facing portals, research and development. The applications on these servers were failing constantly, causing several business disruptions, data losses and adding great amount of IT overhead. To make matters difficult, the IT staff that had originally designed the network had left the company and there was no documentation in-place to figure out which applications relied on which servers. Amicus was called in to assist in the up-gradation of the servers. We delivered that, and much more, well within budget and time.

Business Situation

The company's IT division had been receiving complaints from many business units that critical business applications were performing sluggishly and affecting their workflow. Further, the frequent failures of various applications were frustrating their customers and, therefore, the business leaders wanted a permanent solution for this long-standing problem as soon as possible. The company had decided that the best plan would be to upgrade in stages so as not to cause a complete disruption in IT services. They wanted Amicus to inventory the servers, document how the servers were interconnected and then reproduce the same network design as-is onto the new servers. In addition, they wanted Amicus to eliminate applications, which were no longer in use.

---

Technical and Business Challenges

There were a total of 300 custom-built applications running across 25 servers and most of the applications were interconnected. The servers were about a decade old and were still running on older technology platform like Windows 2000 and SQL Server 2000. Five of these servers were in particularly bad condition and had to be replaced. The IT division was looking to upgrade to Windows 2008 and SQL Server 2005. There wasn't a robust application-testing environment within the company, and the CIO wanted Amicus to design a 3-tier DEV-QA-PRD environment for each application, without spending excessively on new servers.

---

Our Solution

When Amicus began to inventory the servers, we discovered many applications and databases hosted on the servers which were not considered initially. Also, we identified that there was a lot of repetition of data and processes. We interviewed around 100 personnel across the company to find out about each application and their usage. We looked into each physical machine to classify the databases and jobs according to each application. We identified that the servers were often crashing because there were multiple jobs scheduled at the same time that were competing with each other for resources. Furthermore, the lack of an IT-production environment (i.e. separate servers for Development, Quality Assurance and Production) meant that updates to applications were made directly on the production servers, further affecting server performance and possibility of unintentional downtime. We quickly realized that documenting and porting the same IT design to upgraded servers would eventually lead to the same situation within a few years. We decided instead to focus on consolidation wherever possible and developed a detailed plan to upgrade all the applications and databases to newer server environment and thoroughly test each one of them. Additionally, due to the interconnected nature of their existing IT network; we understood that there were no advantages of migrating in phases and suggested to upgrade the entire infrastructure at once.

---

Solution Benefits

Besides executing the project flawlessly, within time and budget, several improvements were made to the overall application infrastructure. We reduced the size of the database to 40% of its original size through data archiving and purging. The number of database jobs reduced from 322 to 104 through consolidation and we mitigated the risk of overlaps and improved the application performances by up to 40%. Some of the servers, which were running at 100% CPU during peak hours, were now consistently running well under limits. This was one of prime causes for frequent server failures and resolving it, removed any immediate need for more expensive hardware. There were 70-80% improvements in average run durations on almost all data transfer jobs. One of the major accomplishment was that we were able to reduce the application portfolio by archiving/cleanup of 29 applications and 30 databases along with retiring one application server. There was a significant reduction in the number of job failures, which meant less IT maintenance work and business disruptions. Existing servers that were running on a mix of Windows 2000/2003 and SQL2000/2005 were migrated to Windows2008/SQL2005. The technology stack was modernized and standardized. Source code for the programs running on the servers were documented along with the usernames and passwords, complete architecture diagrams of all servers and data flows and procedures to easily rebuild the complete application environment from scratch in the event of any server failure or disaster.