← Back to Engineering Blog
πŸ—“οΈ Jan 15, 2025⏱️ 5 min read

Uncovering Hidden Tech Debt: Cloud Landing Zone Security Audits

How we uncovered 140+ orphaned Public IPs, over-privileged Service Principals, and legacy TLS endpoints during an enterprise Azure Landing Zone security audit.

πŸŽ™οΈ Listen to ArticleREADY
AI Audio Synthesis Narrator
Share Post:

β€œTechnical debt in the cloud doesn’t look like dusty server racks in a basementβ€”it looks like 140 orphaned Public IPs, legacy TLS 1.0 storage endpoints, and Service Principals with Contributor rights that nobody has owned since 2021.”

In early 2025, in my current role as Associate Director, Cloud Architecture & AI, I led a cloud governance audit for a financial services client across 45 Azure subscriptions.

The client had grown rapidly through acquisitions over five years. Each acquired team brought its own Azure sub-tenants, custom deployment scripts, and architectural habits.

The internal security team believed their landing zones were compliant because quarterly manual portal checks showed clean dashboards.

The reality was very different.


The Discovery Phase

When you audit cloud infrastructure manually, you only see what you’re looking for.

To get an accurate picture of the environment, we built automated discovery scripts using Azure Resource Graph and Azure CLI to audit identity boundaries, network exposure, and storage security configurations across all 45 subscriptions simultaneously.

Within two hours of running the scan, the findings exposed millions of rupees in wasted spend and critical security vulnerabilities:

  • 140+ Orphaned Public IP Addresses: Unattached static public IPs costing thousands of dollars annually, several of which still had permissive Network Security Group (NSG) rules allowing RDP (port 3389) from any source IP.
  • Over-Privileged Service Principals: 32 automation Service Principals created by long-departed contractors holding full Owner or Contributor permissions at the root subscription level.
  • Legacy Storage Endpoints: 60+ storage accounts operating with public blob access enabled and accepting legacy TLS 1.0/1.1 connections.
# Azure CLI query to identify unattached public IPs across all subscriptions
az graph query -q "Resources | where type =~ 'microsoft.network/publicipaddresses' and isNull(properties.ipConfiguration) | project subscriptionId, resourceGroup, name, properties.ipAddress" --output table

The Mess: The Automated Cleanup Script That Deleted a Cold Backup

Finding the tech debt was easy. Cleaning it up safely was the hard part.

A junior engineer on the team wrote an automated cleanup script to delete any public IP or managed disk whose attachment state was null. He ran it against a staging subscription without a dry-run flag.

The script worked as written. It deleted 18 β€œunattached” managed disks.

Ten minutes later, an application team lead came running into the channel: an unattached secondary disk containing a monthly cold database backup had just vanished. The disk wasn’t attached to an active VM because it was only mounted during monthly reporting runs.

# The dangerous deletion script that bypassed quarantine tagging
az disk list --query "[?managedBy==null].id" -o tsv | xargs -n 1 az disk delete --yes
# Result: Immediate deletion of unattached database backup disks

The recovery took four hours from snapshot storage.

That mistake taught us a critical lesson: in enterprise environments, unattached does not always mean unused. You cannot automate deletion without a quarantine buffer.


The Solution: The 14-Day Quarantine Tag Pipeline

We completely redesigned the remediation workflow by introducing a mandatory Quarantine & Tagging Pipeline.

Instead of deleting orphaned resources immediately, our automated remediation job applies a state=quarantined tag and attaches a restrictive NSG rule blocking all inbound traffic.

# Azure Policy definition enforcing TLS 1.2 minimum version on all Storage Accounts
resource "azurerm_policy_definition" "enforce_tls12" {
  name         = "enforce-storage-tls12"
  policy_type  = "Custom"
  mode         = "Indexed"
  display_name = "Enforce Minimum TLS 1.2 on Storage Accounts"

  policy_rule = <<POLICY
{
  "if": {
    "field": "type",
    "equals": "Microsoft.Storage/storageAccounts"
  },
  "then": {
    "effect": "audit"
  }
}
POLICY
}

The remediation pipeline operates in three distinct stages:

  1. Tagging Phase: Resources identified as orphaned are tagged with pending-deletion and an expiration timestamp set to 14 days in the future.
  2. Notification Phase: An automated summary is sent to the respective subscription owners via Slack/Teams with a direct link to claim the resource.
  3. Purge Phase: If no team claims the resource after 14 days, the pipeline executes a safe deletion pass.
# Safe quarantine tag assignment script
az resource tag --ids $RESOURCE_ID --tags state=quarantined deletion_date=2025-02-01 owner=unclaimed

We also introduced Checkov static analysis gates into the Bitbucket CI/CD pipelines to block developers from deploying storage accounts without TLS 1.2 or unencrypted OS disks at pull-request time.


The Impact

  • Financial Savings: Eliminated β‚Ή12 Lakhs ($15,000 USD) in annual wasted spend from orphaned public IPs, unattached disks, and oversized idle gateways.
  • Privilege Reduction: Stripped root Owner rights from 32 legacy Service Principals, replacing them with scoped custom RBAC roles.
  • Security Baseline: Enforced TLS 1.2 and disabled public blob access across all 45 Azure subscriptions using Azure Policy guardrails.

Key Takeaway

Never Delete Unattached Cloud Resources Without a Quarantine Buffer.

Cloud technical debt accumulates quietly through automated pipelines and human turnover. Regular automated audits are essential, but remediation must be staged. Use quarantine tagging and 14-day hold windows before purging orphaned assets to prevent accidental production outages.


Architecture and decisions: mine. Debugging sessions at odd hours: mine. AI assistance: structure, syntax, first draft. β€” Sachin

SKS

Sachin Kumar Sharma

Associate Director (Infrastructure & Cloud Architecture Strategy) | 20+ Yrs Exp

Architecting resilient multi-cloud enterprise landing zones, SDN overlay fabrics, DevSecFinOps automation pipelines, and autonomous Agentic AI platforms.

πŸ“¬

πŸ“¬ Stay Updated on Tech Releases

Sign up to get notified when I publish new production war stories, agentic AI architecture blueprints, or open-source infrastructure tools.

⚑ Theme Adaptive Shift
Switching layouts matching domain reading affinity...