Technical Debt - Part 2 - Making the monthly payments

Back in March 2012 I wrote an article about technical debt that made the case for incorporating the accumulation of technologies as one way we create new technical debt in Operations. I'm happy to report that the phrase "technical debt" is now firmly embedded in our lexicon and I often hear people using this as part of a justification to change portions of our ecosystem. The acceptance of this concept further strengthens the importance of having a way to identify and address technical debt. We're only talking about it because we want to improve the accumulation of this debt by paying it down.

Throughout the course of 2012 we have reduced or virtualized our MySQL footprint by over 25%. You might not expect MySQL to be something I lump into technical debt given that this is still a core competency and a technology we continue to recommend and support. We started the year with nearly 7,100 instances, many of them supporting over built environments and running on dedicated servers. If left unchecked, there is no doubt this becomes a debt problem of tomorrow.

We won Uptime Institute's inaugural Server Roundup contest for removing about 9,500 servers from our data centers. This year we've kept a similar pace and we could remove north of  8,500 additional servers. I still count over 8,000 servers that are over 4 years old, so this will continue in 2013 as we look to refresh technologies and lower our power consumption.

In this last quarter of the year we've made two huge decisions related to legacy technologies and technical debt. We've negotiated agreements for maintenance, software licenses, and new hardware that provide us a time horizon for replacing technology that powered critical components of AOL's ISP business for the past 2 decades. This will be a tremendous amount of work extended over many months. I'm excited about the willingness to tackle this aging technology and allow us to improve our services offerings in ways that the current technology stack limits.

Looking ahead to 2013, we've already kicked off efforts to revamp our approach to networking and revise our long term data center strategy. We are looking at combining our out-of-band and in-band services to reduce our networking equipment footprint. We are keeping pace with advances in Software Designed Networking (SDN) to evaluate where we can take advantage of commodity hardware capable of making line speed, rules-based, networking decisions. We continue to enhance our private cloud offerings with the expectations of improving collapse ratios from bare metal servers. Each month we are paying both the interest and some principal on our technical debt. A great beginning...