Infrastructure Drift Detection & Remediation
Restoring infrastructure consistency for an energy sector client
Overview
An energy sector client had accumulated manual changes made outside Terraform across its application portfolio, leaving infrastructure state out of sync with what was defined in code on more than 30 applications. With no process in place to catch it, this drift went unnoticed until it surfaced as an incident. The engagement recovered every affected application, corrected the drift, and put alerting in place so new drift is caught immediately instead of accumulating silently again.
Problems
Manual changes made outside Terraform over time caused infrastructure state to fall out of sync with code across the application portfolio
More than 30 applications had drifted, several with multiple discrepancies, making the Terraform code an unreliable record of what was actually deployed
Undetected drift meant unauthorized or unreviewed changes could introduce security gaps without anyone knowing, since nothing was comparing deployed state against approved configuration
No process or tooling existed to detect drift, so it accumulated silently until something broke
Without ongoing detection, any one-time fix would only be temporary
Goals
Restore an accurate, trustworthy picture of what infrastructure actually exists across the portfolio
Bring every drifted application back into alignment without introducing new outages during remediation
Turn a one-time cleanup into a maintained standard, so the same drift doesn't quietly reaccumulate
Give the team visibility into drift the moment it happens, instead of finding out when something breaks
What Was Done
Recovered more than 30 applications containing multiple instances of drift
Documented a maintenance process now used as the team's standard approach to drift remediation
Built automated alerting so new drift is caught immediately instead of resurfacing as an incident
Results and Impact
More than 30 applications brought back into alignment with their Terraform code
Closed the security exposure created by unreviewed manual changes, since every application's actual state is now verified against its approved Terraform configuration
Infrastructure across the portfolio is now centralized and consistent with a single source of truth
New drift is now caught automatically through alerting, instead of accumulating unnoticed
The team has a documented, repeatable process for handling drift going forward, not a one-time fix
Stack