← Back to Engineering Blog
πŸ—“οΈ Nov 20, 2018⏱️ 4 min read

The Magic of HCX: Bulk Live vMotion Across Hybrid Clouds

How we evacuated 200 production VMs to IBM Cloud in 72 hours using VMware HCX Replication-Assisted vMotion and WAN deduplication.

πŸŽ™οΈ Listen to ArticleREADY
AI Audio Synthesis Narrator
Share Post:

β€œWatching 50 production virtual machines vMotion live across 1,000 miles of WAN without losing a single ping packet looks like magic. In reality, it’s deduplicated replication, L2VPN encapsulation, and precise memory state synchronization.”

In late 2018, during my tenure as a VMware NSX Consultant at IBM Cloud, we received a high-stress escalation.

An enterprise client had received a hard eviction notice from their colocation datacenter provider. They had exactly 30 days to evacuate 200 production virtual machines hosting their core ERP and database workloads into IBM Cloud SDDC.

The client’s IT director was on the verge of panic.


The Math of Cold Migration

Under traditional cloud migration playbooks, workloads are moved via β€œCold Migration”:

Shut down VM -> Export OVF/OVA appliance -> Upload over WAN -> Import into Cloud vCenter -> Re-IP Subnets -> Test Connections.

We did the math for 200 VMs averaging 500GB each across their existing 1Gbps WAN connection:

  • Total Data Volume: 100 Terabytes.
  • Upload Time per VM: ~3.5 hours under optimal WAN throughput.
  • Total Migration Window: 700 hours (nearly 30 continuous days of downtime).

The business refused to grant 30 days of application maintenance windows. Worse, re-IPing the servers would require rewriting hundreds of legacy database connection strings.

Cold migration was mathematically impossible within the deadline.


The Mess: The 14-Hour Corrupted OVA Transfer

Before our team arrived on site, an internal engineer tried to execute a manual cold migration on the client’s primary 2TB SQL Server database VM.

He powered down the database at 8:00 PM on Friday, initiated an OVF export, and started an HTTP upload over the 1Gbps WAN link.

At 10:00 AM on Saturdayβ€”14 hours into the upload passβ€”a transient 2-second packet drop occurred on the WAN circuit.

Because raw HTTP file uploads lack resilient block-level resume capabilities, the file transfer aborted instantly:

# OVF Upload Failure Log
HTTP 504 Gateway Timeout: Connection reset by peer during chunked upload at byte 1842901248000
# Result: 14 hours of upload time wasted; 2TB state file corrupted.

The team had spent their entire weekend maintenance window on a failed transfer. The database had to be powered back on without moving an inch.

Fourteen hours of downtime yielded zero progress.


The Solution: VMware HCX Replication-Assisted vMotion (RAV)

We threw out the manual OVF exports and deployed VMware HCX (Hybrid Cloud Extension) across the on-premise and IBM Cloud SDDC environments.

HCX replaces manual file transfers with Replication-Assisted vMotion (RAV), combining background block replication with live memory state synchronization over an encrypted Layer-2 VPN tunnel.

# VMware HCX Service Mesh Status Check
hcx-admin > show service-mesh status
  Inter-Site Transport (IX): UP
  Network Extension (NE): UP [VLAN 102 -> Stretched-Segment-102]
  WAN Optimization (WO): ACTIVE [Deduplication: 3.2:1 | Compression: LZ4]

The 3-Step RAV Bulk Migration Mechanics

  1. Background Seed Sync (Zero Downtime): HCX begins replicating 100% of the VM’s disk blocks to IBM Cloud storage while the VM is live and servicing end-users on-premise.
  2. Delta Sync & WAN Optimization: As users modify data, HCX replicates only changed blocks (deltas). Built-in WAN Optimization (WO) applies line-rate deduplication and LZ4 compression, achieving a 3.2:1 compression ratio across the WAN.
  3. Sub-Second Live Cutover: When the delta replication reaches near-zero, HCX executes a live vMotion flip. It transfers the VM’s active RAM memory state across the stretched Layer-2 network (NE) and registers the VM on IBM Cloud vCenter in under 2 seconds.
# Triggering an automated RAV Bulk Migration wave via HCX API
POST https://hcx-mgr.corp.local/api/migrations/bulk-rav
{
  "vm_ids": ["vm-201", "vm-202", "vm-203", "vm-204"],
  "destination_datastore": "vsanDatastore",
  "retain_ip_mac": true,
  "schedule_cutover": "2018-11-25T02:00:00Z"
}

Because HCX Network Extension (NE) stretched the on-premise VLANs directly into IBM Cloud, the VMs kept their original IP addresses, default gateways, and MAC bindings.


The Impact

  • Evacuation Completed: Successfully migrated all 200 production VMs into IBM Cloud in 72 hours over a single weekend.
  • Zero Downtime Re-IP: Achieved 100% zero-IP-change cutover; application teams executed zero connection string updates.
  • WAN Optimization: Reduced total data transfer volume across the WAN from 100TB to 31TB using real-time deduplication.

Key Takeaway

Combine Background Replication with Live Memory Cutover.

Never perform cold OVF/OVA file uploads over WAN links for large-scale enterprise migrations. Use advanced hybrid engines like VMware HCX Replication-Assisted vMotion (RAV) to seed data in the background while applications remain live, reducing final cutover windows to seconds.


Architecture and decisions: mine. Debugging sessions at odd hours: mine. AI assistance: structure, syntax, first draft. β€” Sachin

SKS

Sachin Kumar Sharma

Associate Director (Infrastructure & Cloud Architecture Strategy) | 20+ Yrs Exp

Architecting resilient multi-cloud enterprise landing zones, SDN overlay fabrics, DevSecFinOps automation pipelines, and autonomous Agentic AI platforms.

πŸ“¬

πŸ“¬ Stay Updated on Tech Releases

Sign up to get notified when I publish new production war stories, agentic AI architecture blueprints, or open-source infrastructure tools.

⚑ Theme Adaptive Shift
Switching layouts matching domain reading affinity...