The finance team flags that the cloud bill has ballooned. When you look, you find dozens of virtual machines that were sized "to be safe" — 32 GB, 8-vCPU boxes that are sitting at 8% CPU and 20% memory usage. How do you cut the spend without risking performance?

What is actually happening (the mental model).

This is the Day-3 lesson turned upside down. The original problem was under-using giant physical servers; the cloud makes it just as easy to over-provision virtual machines, and you pay for the capacity you allocated whether or not you actually use it. "Sized to be safe" really means sized to a guess, and guesses tend to run high. The fix is to right-size the machines to their measured usage.

How to work through it.

1. Measure first. Pull the CPU, memory, and IO usage over a representative window of time, and make sure that window includes the peaks, such as month-end or a marketing campaign. Never resize based on a single afternoon's numbers.
2. Right-size to the peak plus some headroom, not to the average. Drop the 32 GB, 8-vCPU box down to the smallest instance type that covers the real peaks with a safety margin.
3. Automate the ongoing fit: use autoscaling for variable load, and schedule development and test machines to scale down out of hours.
4. Prove it: track the cost per workload before and after, so that the saving is visible and you can confirm that performance was not harmed.

Root causes, and how to tell them apart.

It is usually one of two things. It might be fear-driven sizing, where someone gives a machine plenty of resources so that it never breaks. Or it might be copy-paste sizing, where every machine gets the team's default large type. If the usage is flat and low everywhere, it is a default-sizing problem. If a few boxes spike occasionally, those genuinely need the headroom, so do not shrink them blindly.

How to fix it, verify it, and prevent it.

Resize the machines in a change window, watch the performance for a cycle, and then lock it in. To prevent the problem coming back, use a tagging and ownership policy, review right-sizing recommendations every month, and use autoscaling so that capacity tracks actual demand instead of a static guess.

The trap that less-experienced engineers fall into.

Sizing to the average utilisation and then getting paged when the month-end peak hits, or shrinking everything by the same amount and clipping the one workload that actually needed the room.

🎯 Interviewer follow-up questions you should expect.

• "Average or peak?" You size to the peak plus headroom over a representative window, never to the average.
• "How do you make it stick?" Autoscaling, a scheduled scale-down out of hours, and a monthly right-sizing review are what make it stick.
• "Reserved or spot instances?" Once the machines are sized correctly, you commit the steady workloads to reserved or savings plans and put the interruptible ones on spot instances.

A cloud cost audit turns up 40 running virtual machines with no clear owner, no tags, and no one on the team who remembers what they're for or whether they're safe to turn off.

What is actually happening (the mental model).

A VM someone launched for a one-off task — a proof of concept, a migration, a "just for today" test — doesn't announce when it's no longer needed; it simply keeps running and billing until someone notices, and without an owner recorded anywhere, "someone notices" can take years. Every VM launched without a tagging policy is a zombie in waiting.

How to work through it.

1. Don't delete blind. A VM with no recorded owner might still be load-bearing — check recent network traffic, CPU activity, and any DNS/load-balancer references pointing at it before assuming it's safe to remove.
2. Stop first, delete later. For anything genuinely ambiguous, shut the VM down (don't terminate it) and wait a defined period — a week or two — watching for anyone to complain, before permanently deleting it.
3. Trace ownership through indirect evidence when tags are missing — billing account, the IAM user or role that launched it, deployment timestamps correlated with known projects.
4. Fix the source, not just the backlog: enforce mandatory tagging (owner, purpose, expiry date) at creation time going forward, ideally blocking VM creation entirely without it.

Root causes, and how to tell them apart.

This usually comes down to one main cause with two things that make it worse. The main cause is that there was no enforced tagging policy at the moment the VM was created, so nothing ever recorded who owned it or why. It gets worse when people leave the company or the team without handing over a list of what they had launched, and worse again when a resource was only ever meant to be temporary but nobody circled back to clean it up once its one-off purpose had ended.

How to fix it, verify it, and prevent it.

Stop any suspected zombie rather than deleting it immediately. Verify that nothing breaks over a defined waiting period before you actually delete it. Prevent the problem from recurring by enforcing mandatory tagging at creation time, running an automated report that flags untagged or long-idle resources, and tying a regular ownership audit to whenever someone leaves the team.

The trap that less-experienced engineers fall into.

Either leaving zombie VMs running indefinitely out of fear of breaking something unknown, or deleting them immediately without a safety window — both extremes avoid the actual fix, which is a brief, deliberate verification step before permanent removal.

🎯 Interviewer follow-up questions you should expect.

• "You find an untagged, unowned VM — do you delete it?" Not immediately — you stop it first, watch for a defined period, then delete it once nothing breaks.
• "How do you stop this recurring?" You enforce mandatory tagging for owner, purpose, and expiry at creation time, ideally blocking creation entirely without it.

Storage costs have crept up steadily for a year. The VMs themselves haven't grown much — but nobody ever looked at how many disk snapshots have accumulated behind them.

What is actually happening (the mental model).

Taking a snapshot before a risky change is good practice, but a snapshot is a real, billed storage object that outlives the moment it was useful for — if nothing ever deletes old snapshots, they accumulate indefinitely, and because each one is individually cheap, the growing pile is easy to miss until the aggregate cost is substantial.

How to work through it.

1. Audit snapshot age and count per volume — most cloud providers can list this directly; look specifically for volumes with dozens of snapshots going back years.
2. Apply a retention policy — keep, say, the last N daily snapshots plus a handful of weekly/monthly ones for longer-term recovery points, and automatically expire anything older.
3. Confirm nothing depends on an old snapshot before bulk-deleting — check whether any are referenced by a documented recovery runbook or compliance requirement before assuming they're all safe to remove.
4. Automate the retention policy going forward (most cloud providers support lifecycle rules for exactly this) so cleanup isn't a manual, easily-forgotten chore.

Root causes, and how to tell them apart.

Most setups have no automated retention or expiry policy on snapshots unless someone deliberately configures one, so that is the default state to expect. On top of that, many teams build the habit of taking a snapshot before any risky change but never build the matching habit of going back to delete the old ones — they only ever practise the create half of the routine, never the delete half.

How to fix it, verify it, and prevent it.

Audit snapshots older than an agreed retention window, confirm that none of them are referenced by a runbook or a compliance requirement, and then bulk-clean the rest. Verify the saving by checking the storage cost afterward. Prevent the pile from building up again by setting an automated lifecycle policy that expires snapshots on a schedule, so nobody has to remember to do it by hand.

The trap that less-experienced engineers fall into.

Treating snapshots as free because each individual one is cheap, and never noticing the aggregate cost of hundreds accumulated over years — until a cost audit finally surfaces it as a single large number.

🎯 Interviewer follow-up questions you should expect.

• "Snapshot storage costs crept up over a year — what's the fix?" You audit snapshot age and count per volume, apply a retention policy, and confirm nothing depends on the old ones before bulk deletion.
• "How do you prevent this recurring?" An automated lifecycle policy that expires snapshots on a schedule prevents it, rather than relying on someone remembering to clean up manually.

VMs launched from the same "golden image" template a year ago now behave inconsistently — some have manual patches applied, some don't, and nobody's sure which machines are actually still faithful to the template.

What is actually happening (the mental model).

A golden image is only a snapshot of "correct" at the moment it was built — every VM launched from it starts identical, but any manual change made directly to a running instance afterward (a hotfix, a manual patch, an ad hoc config tweak) makes that specific instance diverge from both the template and from its siblings, and those divergences accumulate silently unless something actively prevents or tracks them.

How to work through it.

1. Treat running instances as disposable, not editable. The actual fix for drift is discipline: changes get made to the golden image definition and redeployed, never patched directly onto a live instance.
2. Rebuild the golden image regularly (a scheduled pipeline, not an ad hoc one-off) so it keeps including the latest patches and configuration, rather than aging further behind reality with every month that passes.
3. Detect existing drift with configuration-management or compliance-scanning tools that compare a running instance's actual state against the intended template definition.
4. Replace drifted instances rather than reconciling them by hand — relaunch from the current golden image instead of trying to manually patch a diverged instance back into line, which is slower and less certain to actually succeed.

Root causes, and how to tell them apart.

The core discipline failure is making manual, direct changes to running instances instead of updating the source image itself. That gets compounded when the golden-image build pipeline isn't run on any regular schedule, so the template quietly ages behind reality, and again when there's no drift-detection tooling in place at all, so nobody even notices that divergence has happened until it causes a real problem.

How to fix it, verify it, and prevent it.

Detect the drifted instances with a compliance or configuration scan. Replace them by relaunching from a freshly rebuilt golden image rather than patching them in place. Prevent the problem recurring by treating running instances as immutable — you rebuild them, you never edit them — and by scheduling regular golden-image rebuilds so the template never drifts far behind current patches and configuration.

The trap that less-experienced engineers fall into.

Patching individual running instances directly to fix an urgent issue, which solves the immediate problem but adds one more silent divergence from the golden image — the fleet becomes less uniform with every "quick fix," not more.

🎯 Interviewer follow-up questions you should expect.

• "VMs from the same template have drifted apart — how do you fix it?" You detect the drift with a configuration or compliance scan, then replace the drifted instances from a freshly rebuilt golden image rather than patching them in place.
• "How do you prevent drift going forward?" You treat running instances as immutable, so changes go into the golden image and get redeployed rather than applied directly to a live instance, and you rebuild the image on a regular schedule.

A VM's performance degrades unpredictably at seemingly random times, even though its own CPU and memory metrics look completely normal.

What is actually happening (the mental model).

Multiple VMs share the same underlying physical hardware, and while a hypervisor isolates memory and CPU allocation reasonably well, some shared resources — disk IO bandwidth, network bandwidth, the CPU cache — can still be contended for by other VMs on the same host, a phenomenon called the "noisy neighbour" problem. Your VM's own metrics can look fine while it's still being starved of a shared resource by another tenant's spike.

How to work through it.

1. Look beyond your own VM's metrics — check disk IO latency and network throughput specifically, not just CPU and memory, since contention often shows up there first.
2. Correlate the timing of the degradation against anything else known to be happening on the same host or hardware pool, if that visibility is available from the cloud provider or virtualization platform.
3. Move to a resource tier with stronger isolation guarantees if the pattern persists — dedicated instances/hosts (no shared tenancy) or an instance type with guaranteed (not best-effort) IOPS and bandwidth.
4. Use provisioned/guaranteed IOPS storage instead of best-effort shared storage for genuinely latency-sensitive workloads, which sidesteps the contention risk entirely for that specific resource.

Root causes, and how to tell them apart.

The classic cause is shared, best-effort disk or network bandwidth being contended for by another tenant on the same physical host, which is the noisy-neighbour problem itself. You can rule out a genuine problem in your own application once you've confirmed your own CPU, memory, and IO metrics are normal during the degradation, since that points the blame away from your code and toward the shared infrastructure underneath it.

How to fix it, verify it, and prevent it.

Move latency-sensitive workloads to dedicated instances or guaranteed-IOPS storage. Verify improvement by confirming the degradation pattern stops recurring after the move. Prevent recurrence by defaulting genuinely sensitive workloads to isolated or guaranteed-resource tiers from the start, rather than discovering the need for it after an incident.

The trap that less-experienced engineers fall into.

Assuming unpredictable performance must be a bug in the application itself, and spending significant time debugging application code, when the actual cause is infrastructure-level contention entirely outside the application's control.

🎯 Interviewer follow-up questions you should expect.

• "A VM's performance degrades randomly but its own metrics look fine — what do you suspect?" Noisy-neighbour contention on shared disk IO or network bandwidth from another tenant on the same host is the likely cause.
• "How do you fix it long-term?" You move the workload to dedicated instances or hosts, or to guaranteed-IOPS storage, for anything genuinely latency-sensitive.

Traffic spikes and autoscaling correctly triggers new VMs to launch — but each one takes over three minutes to become ready, so users hit degraded performance for several minutes during every spike.

What is actually happening (the mental model).

A VM's boot time is the sum of booting a full operating system plus whatever application setup happens after boot (installing dependencies, pulling configuration, warming caches) — and because a VM always carries a complete OS (T1/T2's core trade-off), that startup cost is structurally higher than a lighter-weight compute model, which matters enormously the moment "how fast can we add capacity" becomes the thing users actually feel.

How to work through it.

1. Break down where the three minutes actually goes — OS boot itself, versus application install/config, versus application warm-up — since the fix differs for each.
2. Bake more into the golden image (from the drift scenario) — pre-install dependencies and pre-warm what can safely be baked in ahead of time, so less work happens at boot rather than every single time a new instance launches.
3. Over-provision slightly ahead of predictable spikes (a scheduled scale-up before a known traffic pattern, like a marketing campaign) rather than relying purely on reactive autoscaling for entirely predictable load.
4. Reconsider the compute model for this specific workload if boot time remains a hard constraint even after baking in as much as possible — containers or serverless functions start in seconds, not minutes, precisely because they skip the full-OS-boot cost entirely.

Root causes, and how to tell them apart.

One cause is heavy post-boot application setup happening on every single launch instead of being baked into the image ahead of time, and that one is fixable without leaving the VM model at all. The other is more fundamental: the full operating-system boot cost itself may simply be the wrong fit for how fast this particular workload needs to scale, and if that's the case it's a signal to reconsider the compute model entirely rather than just trying to optimise within it.

How to fix it, verify it, and prevent it.

Bake dependencies into the golden image and add predictive scheduled scale-up for known spikes. Verify by measuring the new time-to-ready against the old three minutes. Prevent regression by keeping the golden image current (tying back to the drift scenario) so the baked-in savings don't erode over time.

The trap that less-experienced engineers fall into.

Treating "add more autoscaling" as the fix for slow scale-out, without addressing that the actual bottleneck is what happens during each individual instance's three-minute boot — more instances launching slowly is still slow.

🎯 Interviewer follow-up questions you should expect.

• "New VMs take three minutes to become ready during a spike — how do you speed that up?" You break down where the time actually goes, bake more setup into the golden image ahead of time, and add predictive scaling for known spikes.
• "What if VM boot time is still too slow after optimising?" Then it's worth reconsidering the compute model entirely — containers or serverless start in seconds, because they skip the full-OS-boot cost altogether.

A critical OS security patch needs to go out to 300 VMs across the fleet this week, and none of them can go down all at once without taking the service with them.

What is actually happening (the mental model).

Patching at this scale is fundamentally a rollout problem, structurally identical to a software deployment — the risk isn't the patch itself, it's applying it to everything simultaneously with no way to catch a problem before it's everywhere, so the same discipline that governs safe application deployments (small batches, canaries, monitoring between waves) applies directly to patching a VM fleet.

How to work through it.

1. Patch in small batches, never everything at once — take a portion of the fleet out of rotation (behind a load balancer, with capacity to spare), patch it, verify it's healthy, then move to the next batch.
2. Patch a canary batch first and watch it under real traffic for a defined period before proceeding to the rest of the fleet, exactly as a canary release works for application code.
3. Automate the batching and health-checking with configuration-management or fleet-management tooling, rather than manually SSHing into machines one at a time, which doesn't scale to hundreds of hosts and invites human error.
4. Have a rollback plan specifically for the patch — for VMs, this is often "revert to the previous golden image" rather than trying to manually un-patch a live system, since forward-fixing a botched patch is usually harder than replacing the instance.

Root causes, and how to tell them apart.

The main cause is having no batching or canary discipline for patch rollouts at all, which usually comes from treating patching as fundamentally different from a software deployment when it's actually structurally the same risk. That gets worse when the patching process itself is manual and unautomated, since that kind of process simply doesn't scale safely past a handful of machines.

How to fix it, verify it, and prevent it.

Roll the patch out in small batches with a canary first, automated via fleet-management tooling. Verify each batch's health before proceeding to the next. Prevent future patch incidents by keeping this batched rollout process as the standing, default way any fleet-wide change is applied, not a one-off invented under pressure.

The trap that less-experienced engineers fall into.

Treating a security patch as urgent enough to justify skipping the batching/canary discipline and just pushing it everywhere at once — urgency is exactly the condition under which skipping safety discipline causes the biggest, most visible outages.

🎯 Interviewer follow-up questions you should expect.

• "How do you patch 300 VMs without an outage?" You batch it like a deployment — a canary group first, verified health, then the rest in waves, all automated through fleet-management tooling.
• "What's the rollback plan if a patch breaks something?" You revert to the previous golden image rather than trying to manually un-patch a live system.

A host fails and the team goes to restore its VM from backup — only to discover the backup is corrupted, or restores to a state that doesn't actually boot.

What is actually happening (the mental model).

A backup that has never been restored is, for all practical purposes, an unverified claim, not a guarantee — the only thing that actually proves a backup works is successfully restoring from it, and if that's never been tried until a real disaster, you're discovering whether your disaster-recovery plan works at the worst possible moment to find out it doesn't.

How to work through it.

1. In the moment: have a fallback beyond the failed backup — a more recent snapshot, a golden-image rebuild plus reapplying known configuration, or failing over to a replica if one exists — since a single backup with no verified alternative is a single point of failure in the recovery plan itself.
2. Afterward, make restore testing a recurring, scheduled exercise — actually restoring a sample of backups periodically and confirming the result boots and functions correctly, not just confirming the backup job itself reported success.
3. Treat "the backup job succeeded" and "the backup is restorable" as two different claims — a backup job can report success while still producing an unusable or corrupted artifact if nobody's ever tested the restore path.
4. Document and rehearse the actual restore procedure, not just the backup schedule — under real incident pressure is the worst time to be improvising unfamiliar restore steps for the first time.

Root causes, and how to tell them apart.

The usual cause is that backups are only ever verified by checking that the backup job reported success, never by actually testing a real restore. That's compounded by having no scheduled, recurring restore-testing practice, and often by having a documented backup policy with no matching documented and rehearsed restore procedure to go with it.

How to fix it, verify it, and prevent it.

Recover using whatever verified fallback exists, then add or fix the missing restore procedure once the immediate incident is resolved. Prevent recurrence with scheduled, recurring restore drills that test the full path end to end, not just the backup job's own success status.

The trap that less-experienced engineers fall into.

Treating "we have backups" and "we have a working disaster-recovery plan" as the same claim — a backup that's never been restored is a hope, not a tested capability, and the difference only becomes visible during an actual disaster.

🎯 Interviewer follow-up questions you should expect.

• "A DR restore fails during an actual incident — what does this reveal?" It reveals that the backup was never actually tested by restoring it — "the job succeeded" and "the backup is restorable" are different claims, and only the second one actually matters.
• "How do you prevent this?" Scheduled, recurring restore drills that test the full path end to end prevent it, rather than just checking the backup job's own reported status.

The fleet has grown 20% this year, but the Windows Server and database licensing bill has grown 60% — nobody can immediately explain the gap.

What is actually happening (the mental model).

Many commercial OS and database licenses are priced per-core or per-instance, not per-workload — so decisions that look purely like infrastructure choices (bigger instance types, more VMs, a licensed database engine chosen without comparing alternatives) directly multiply a cost that's easy to overlook because it shows up on a separate line item from the compute bill, evaluated by a different team entirely.

How to work through it.

1. Break down the licensing bill by resource — which specific VMs or databases are licensed, at what core count, and whether that count actually reflects real utilisation, exactly like the right-sizing exercise in scenario 1 but for a cost that scales differently.
2. Check for oversized instances specifically driving licensing cost, not just compute cost — a per-core license means an oversized VM is doubly wasteful, paying extra for both idle compute and idle licensed cores.
3. Consider licensing-included alternatives where they exist — a cloud-native managed database service, an open-source equivalent, or a different instance family with more favourable licensing terms — evaluated specifically against the licensing cost, not just the sticker price of compute.
4. Bring licensing into the same right-sizing and tagging discipline as compute, since it's currently being decided ad hoc by whoever provisions a VM, disconnected from whoever actually reviews the licensing bill.

Root causes, and how to tell them apart.

This usually traces back to per-core licensing on oversized instances, which is really the same over-provisioning problem from earlier, just showing up on a different budget line. It's made worse when licensed software gets chosen without ever comparing it to a viable open-source or managed alternative, and worse again when the person provisioning the infrastructure is a completely different person from whoever actually reviews and owns the licensing budget, so nobody connects the two decisions.

How to fix it, verify it, and prevent it.

Right-size the specific instances driving licensed-core costs, and evaluate lower-licensing-cost alternatives for new provisioning. Verify with a licensing-cost-per-workload metric, the same discipline as compute cost. Prevent recurrence by bringing licensing cost into the same provisioning review and tagging policy as compute, rather than treating it as a separate, disconnected budget line.

The trap that less-experienced engineers fall into.

Treating licensing cost as a fixed, unavoidable overhead disconnected from infrastructure decisions, rather than recognising that the same over-provisioning and instance-choice decisions driving compute waste are often directly multiplying licensing waste too.

🎯 Interviewer follow-up questions you should expect.

• "Licensing costs grew much faster than the fleet — why?" Many licenses are priced per-core or per-instance, so oversized VMs and licensed-software choices directly multiply that cost, often unnoticed because it sits on a separate budget line from compute.
• "How do you bring licensing cost under control?" You right-size the instances actually driving the licensed-core costs, evaluate lower-licensing alternatives, and bring licensing into the same review discipline as compute provisioning.

A new workload needs to be deployed, and the team is debating whether it should run on a VM, in a container, or as a serverless function.

What is actually happening (the mental model).

All three are compute options on the same underlying spectrum, trading isolation and control against startup speed and operational overhead — a VM gives full OS-level isolation and control at the cost of heavy boot time and full OS patching; a container gives lightweight, fast isolation by sharing a host kernel, at the cost of slightly less isolation than a full VM; serverless gives the least operational overhead (no server to manage at all) at the cost of the least control over the runtime environment. The right choice depends on what the workload actually needs, not a default habit.

How to work through it.

1. Choose a VM when you need full OS-level control, a legacy application that assumes a complete traditional OS, or strict isolation requirements (regulatory or security) that specifically call for hypervisor-level separation rather than a shared kernel.
2. Choose a container when the workload is a modern, stateless-ish application that can be packaged once and run consistently anywhere, and you want fast startup and efficient density — most new application workloads default here today.
3. Choose serverless when the workload is event-driven, bursty, or genuinely intermittent (a webhook handler, a scheduled batch job) and you want to pay only for actual execution time with zero idle-capacity cost, accepting less control over the runtime and typically a maximum execution duration.
4. Don't default to whatever the team already knows without checking the workload's actual shape — a bursty, occasional job forced onto an always-on VM is paying for idle capacity exactly like the over-provisioning scenario, just architecturally rather than through a sizing mistake.

Root causes, and how to tell them apart.

Often a team defaults to VMs simply out of familiarity, even for a workload that would genuinely fit a lighter model better, and that's usually a cost and agility problem rather than something actually broken. The other case is genuinely needing VM-level isolation or a legacy operating-system dependency that containers or serverless can't provide, which is a real and valid reason to choose a VM rather than a habit to correct.

How to fix it, verify it, and prevent it.

Match the compute model to the workload's actual isolation, startup, and cost-shape needs rather than habit. Verify the choice by checking utilisation and cost patterns after deployment match what was expected for that model. Prevent future mismatches by making this evaluation an explicit step in new-workload design, not an assumed default.

The trap that less-experienced engineers fall into.

Treating this as a purely technical or trendy choice ("containers are modern, use containers") instead of matching the actual isolation and cost-shape needs of the specific workload — the right answer is genuinely different depending on what's being built.

🎯 Interviewer follow-up questions you should expect.

• "When would you still choose a VM over a container in 2026?" You'd still choose a VM for full OS-level control, a legacy app that assumes a complete traditional OS, or a strict isolation requirement that specifically needs hypervisor-level separation.
• "When does serverless make more sense than either?" Serverless makes more sense for genuinely event-driven or bursty workloads, where paying only for actual execution time beats paying for any idle capacity at all.