A developer accidentally committed a long-lived IAM access key and secret into a public GitHub repository. It was live for six hours before anyone noticed.

What is actually happening (the mental model).

A leaked long-lived key is a live credential in an attacker's hands the moment it's public — bots scan GitHub for exactly this pattern continuously, so "six hours" isn't a small window, it's ample time for automated abuse (crypto-mining instances, data exfiltration) to already be underway. The response has to start with containment, not investigation.

How to reason about it.

1. Deactivate or delete the key immediately in IAM — this is the single highest-priority action, before anything else.
2. Check CloudTrail for activity from that key during the exposure window, to understand what, if anything, was actually done with it.
3. Rewrite the Git history to remove it (the same discipline as the Git chapter's leaked-secret scenario) and force-push, coordinating with the team.
4. Prevent recurrence structurally: move to short-lived credentials via IAM roles (for EC2/Lambda) instead of long-lived access keys wherever possible, and add automated secret-scanning on commits/pushes.

The root causes and trade-offs.

The root cause is simply having long-lived IAM access keys at all in a place where a role would do the job instead, combined with no automated secret-scanning in place to catch the commit before it was pushed. The deeper fix, eliminating long-lived keys in favour of roles, is more work upfront, but it removes this entire category of incident going forward.

The trap that less-experienced engineers fall into.

Spending the first hour investigating what happened before deactivating the key — every minute the key stays live is additional exposure, and containment always comes before root-cause analysis.

🎯 Interviewer follow-up questions you should expect.

• "A key was leaked — what's your very first action?" You deactivate or delete the key in IAM immediately, before investigating anything.
• "How do you prevent this structurally?" You use IAM roles with short-lived credentials instead of long-lived access keys wherever possible, plus automated secret-scanning on commits.

A security review finds a role attached to dozens of EC2 instances with an IAM policy granting "Action": "*", "Resource": "*" — full admin access — because "it was easier to get things working."

What is actually happening (the mental model).

A wildcard policy is the IAM equivalent of leaving every door in the building unlocked because it was faster than cutting individual keys — it works, until any one of the many things that role touches is compromised, at which point the blast radius is the entire account instead of one narrow, intended capability.

How to reason about it.

1. Use IAM Access Analyzer or CloudTrail's actual usage history to see what the role has genuinely called in practice, rather than guessing at what it might need.
2. Write a scoped policy matching only that observed, real usage — specific actions on specific resources — rather than a broad category "to be safe."
3. Roll it out gradually, monitoring for access-denied errors that reveal a genuinely missed permission, and widen only that specific gap rather than reverting to the wildcard.
4. Make least privilege the default going forward — start every new role narrow and add permissions as concretely justified, rather than starting broad and meaning to narrow later (which rarely actually happens).

The root causes and trade-offs.

The root cause is usually time pressure during initial setup, which leads someone to grant everything as a shortcut, combined with no process ever circling back afterward to actually tighten it. Tightening the policy from real, observed usage, using a tool like Access Analyzer or CloudTrail, is far safer than tightening it by guessing at what "should" be needed.

The trap that less-experienced engineers fall into.

Treating "it was easier to get working" as a permanent justification rather than a temporary shortcut that was always meant to be revisited — broad IAM policies granted under time pressure very rarely get tightened unless something forces the issue.

🎯 Interviewer follow-up questions you should expect.

• "How do you tighten an overly broad IAM policy safely?" You use Access Analyzer or CloudTrail to see actual usage, write a scoped policy matching it, and roll out gradually while watching for access-denied errors revealing a real gap.
• "Why is a wildcard policy dangerous even if nothing's gone wrong yet?" The blast radius of any single compromise becomes the entire account instead of one narrow capability.

An IAM user's policy clearly grants s3:GetObject on a bucket, and yet every request comes back "Access Denied" with no further detail.

What is actually happening (the mental model).

AWS authorization is an intersection of every applicable policy layer, not just the one you happened to check — the identity-based policy (attached to the user/role), the resource-based policy (attached to the bucket itself), any Service Control Policy at the Organization level, and any permissions boundary all have to agree; and critically, an explicit deny in any one layer always wins, regardless of what any other layer allows.

How to reason about it.

1. Check the resource-based policy (the S3 bucket policy) — a common cause is the identity policy allowing it while the bucket policy has no matching allow, or an explicit deny for a different condition (like requiring a specific VPC endpoint).
2. Check for an SCP at the Organization/OU level that might be denying the action account-wide, invisible from looking at the IAM user's own policy alone.
3. Check for a permissions boundary on the user or role, which caps the maximum permissions it can ever have regardless of what its attached policies say.
4. Remember explicit-deny-always-wins — if any layer has an explicit deny matching this action, no other layer's allow can override it, so hunt specifically for a deny, not just for a missing allow.

The root causes and trade-offs.

The root cause is checking only the identity-based policy and assuming that's the whole picture, when AWS authorization is genuinely the intersection of several independent layers. The actual deny is often an explicit one sitting somewhere in an SCP or a resource policy, and it simply isn't visible unless you specifically go looking for it there.

The trap that less-experienced engineers fall into.

Staring at the IAM identity policy that clearly says "allow" and concluding AWS must be malfunctioning, instead of checking the other authorization layers — the resource policy, SCPs, permissions boundaries — where the actual deny is very often sitting.

🎯 Interviewer follow-up questions you should expect.

• "The IAM policy clearly allows it but access is still denied — why?" AWS authorization is an intersection of multiple layers — identity policy, resource policy, SCPs, permissions boundaries — and an explicit deny in any one of them wins regardless of what the others allow.
• "Where do you look beyond the IAM policy itself?" You look at the resource-based policy on the object or bucket, any Organization-level SCP, and any permissions boundary on the role.

A routine audit finds a database's security group allows inbound traffic on port 5432 from 0.0.0.0/0 — the entire internet — and it's been that way for months.

What is actually happening (the mental model).

A security group open to the world on a database port means anyone on the internet can attempt to connect — the only thing standing between that and a breach is the strength of the database credentials themselves, which is a single point of failure nobody should be relying on when a network-level restriction is nearly free to add.

How to reason about it.

1. Check for evidence of actual exploitation first — review database logs and CloudTrail for unfamiliar connection attempts before assuming it's merely a latent risk versus an active breach.
2. Narrow the security group immediately to only the specific sources that genuinely need access — the application servers' security group as the source, not an IP range, wherever possible, so the rule stays correct even as IPs change.
3. Rotate database credentials regardless of whether exploitation is confirmed, given the exposure window is unknown and credentials can't be un-exposed after the fact.
4. Add guardrails to prevent recurrence — AWS Config rules or Security Hub checks that flag any security group with a sensitive port open to 0.0.0.0/0 automatically, rather than relying on periodic manual audits to catch it.

The root causes and trade-offs.

The root cause is usually a security group rule that was created broadly during initial setup just to make things work, and then never narrowed afterward, combined with no automated detection in place for this specific, well-known misconfiguration pattern. Setting up an automated Config or Security Hub rule costs a small amount of effort, in exchange for continuous protection against this exact class of mistake happening again.

The trap that less-experienced engineers fall into.

Treating a security-group-only fix as sufficient without also rotating credentials, on the assumption that if nothing's been proven exploited yet, the existing credentials are still safe — but an unknown exposure window means you can't actually make that assumption with confidence.

🎯 Interviewer follow-up questions you should expect.

• "A database port is open to the whole internet — what do you do?" You check logs for evidence of exploitation, narrow the security group immediately to specific sources, rotate credentials regardless, and add automated detection to prevent recurrence.
• "Why rotate credentials if nothing's confirmed exploited?" The exposure window and its actual usage can't be fully verified, so treating the credentials as potentially compromised is the safer default.

A security scanner flags an S3 bucket containing customer data as publicly readable — anyone with the URL can download the objects inside.

What is actually happening (the mental model).

S3 buckets are private by default, so public exposure is always the result of an explicit choice somewhere — a bucket policy, an object-level ACL, or (historically) a "Block Public Access" setting that was deliberately disabled — which means the fix always starts with finding exactly which of those three actually granted the access, not assuming which one it was.

How to reason about it.

1. Confirm the actual exposure first — check whether it's the bucket policy, an object ACL, or the account/bucket-level "Block Public Access" setting, since the fix differs depending on which one it is.
2. Re-enable Block Public Access at the bucket (and ideally account) level as the fastest, broadest containment — this overrides bucket policies and ACLs that would otherwise allow public access.
3. Check access logs for evidence of actual downloads during the exposure window, to scope the incident's real impact rather than assuming worst-case with no evidence either way.
4. Prevent recurrence with automated detection (AWS Config, Security Hub, or Macie for sensitive-data-specific scanning) that continuously checks for public buckets, rather than relying on the next scheduled scan to catch the next one.

The root causes and trade-offs.

The root cause is usually a bucket policy or object ACL that was set too broadly, often just for convenience during initial setup. It's also common for Block Public Access to have been deliberately disabled for a specific static-website use case, and then never re-scoped afterward once that same bucket started holding sensitive data too.

The trap that less-experienced engineers fall into.

Fixing the specific policy or ACL that was found without also checking and re-enabling Block Public Access as a broader safety net — leaving the bucket one future misconfiguration away from being exposed again the same way.

🎯 Interviewer follow-up questions you should expect.

• "An S3 bucket with customer data is publicly readable — what's your first move?" You re-enable Block Public Access immediately as the fastest broad containment, then determine exactly which mechanism — the policy, the ACL, or the setting itself — caused the exposure.
• "How do you prevent this recurring across the whole account?" Automated detection through Config, Security Hub, or Macie continuously flags public buckets, rather than relying on periodic manual scans.

"Design a VPC for a typical three-tier web application" — testing whether you understand why not everything belongs in a public subnet.

What is actually happening (the mental model).

The core VPC design principle is that only what genuinely needs to be reachable from the internet should be able to be reached from the internet — a public subnet (with a route to an Internet Gateway) holds load balancers and bastion hosts; a private subnet (no direct route in) holds application servers and databases, which never need a direct inbound connection from the public internet at all.

How to reason about it.

1. Public subnet: load balancers (ALB/NLB), and a bastion host if you need direct SSH access for debugging — these are the only things that genuinely need to be internet-facing.
2. Private subnet: application servers and databases — they receive traffic routed through the load balancer, never directly from the internet.
3. A NAT Gateway in the public subnet lets resources in the private subnet reach out to the internet (for package updates, external API calls) without allowing anything to initiate a connection in — outbound-only by design.
4. Spread subnets across multiple Availability Zones for the high-availability design covered in scenario 8, rather than putting everything in a single AZ's subnets.

The root causes and trade-offs.

Putting application servers or databases directly in a public subnet "because it's simpler" trades away a whole layer of network-level protection for no real benefit, since a load balancer in front already handles the actual internet-facing traffic. NAT Gateways cost money per hour plus data processed — a real, deliberate trade-off against the security benefit of private-subnet isolation.

The trap that less-experienced engineers fall into.

Putting everything in public subnets to avoid the extra NAT Gateway complexity and cost, which removes a meaningful layer of defence for no real operational upside, since the load balancer was always going to be the actual entry point regardless.

🎯 Interviewer follow-up questions you should expect.

• "What goes in a public subnet versus a private one?" Load balancers and bastion hosts go in the public subnet; application servers and databases go in the private one, since they never need a direct inbound connection from the internet.
• "What does a NAT Gateway actually do?" It lets private-subnet resources reach out to the internet, for updates or external API calls, without allowing anything to connect in — it's outbound-only.

This month's AWS bill is 40% higher than last month's, with no obvious new feature or traffic spike to explain it.

What is actually happening (the mental model).

AWS cost surprises hide in two places people don't naturally look first: data transfer (moving data between AZs, regions, or out to the internet is billed and easy to overlook, since it's not an obvious "resource" like an instance) and orphaned resources — an EBS volume left behind after its instance was terminated, an unused Elastic IP, an old load balancer nobody deleted — each individually small but adding up unnoticed over time.

How to reason about it.

1. Use Cost Explorer to break down the increase by service first, rather than guessing — this immediately narrows whether it's compute, storage, data transfer, or something else entirely.
2. Check specifically for cross-AZ or cross-region data transfer if the breakdown points that way — a recent architecture change (like spreading instances across AZs for HA) can introduce data-transfer costs that weren't there before, as a direct trade-off for the reliability gained.
3. Audit for orphaned resources — unattached EBS volumes, unused Elastic IPs, idle load balancers — using AWS's own cost-optimization recommendations or Trusted Advisor as a starting point rather than manually hunting.
4. Set up billing alerts and regular cost reviews going forward, so a 40% jump is caught within days, not discovered a month later when the bill arrives.

The root causes and trade-offs.

One common cause is an architecture change — like spreading instances across AZs for high availability — that added cross-AZ or cross-region data transfer as a side effect of a different, deliberate goal. The other is resources simply left behind after a migration, or a decommissioned service that nobody explicitly cleaned up. Both are common, and the fix — billing alerts and regular cost reviews — is cheap relative to catching this kind of drift only after it's already large.

The trap that less-experienced engineers fall into.

Assuming a cost jump must mean a traffic spike or a new feature, and looking only at compute costs, when the actual driver is very often data transfer or an accumulation of small orphaned resources that don't show up as an obvious single culprit.

🎯 Interviewer follow-up questions you should expect.

• "The bill jumped 40% with no obvious cause — where do you look?" You check Cost Explorer's service-level breakdown first, then specifically data transfer and orphaned resources, which are the two most commonly overlooked cost drivers.
• "How do you prevent being surprised by this again?" You set up billing alerts and a regular, monthly or more frequent, cost review, so a jump is caught within days rather than discovered when the bill arrives.

"Design this application so a single failure — an instance, or an entire Availability Zone — doesn't take it down." Testing whether you actually understand redundancy, not just naming services.

What is actually happening (the mental model).

High availability means eliminating single points of failure at every layer: at the compute layer, an Auto Scaling Group spread across multiple AZs replaces a failed instance automatically; at the traffic layer, an Application Load Balancer spread across those same AZs stops routing to any instance or AZ that's unhealthy; at the database layer, RDS Multi-AZ gives an automatic failover to a synchronous standby. No single layer's redundancy compensates for a gap in another — all three have to be addressed together.

How to reason about it.

1. Spread the Auto Scaling Group across at least two (ideally three) AZs, so losing one AZ still leaves healthy instances running elsewhere, and the ASG launches replacements automatically.
2. Put an ALB in front, also spanning multiple AZs, with health checks that stop routing traffic to any instance (or entire AZ) that's failing.
3. Use RDS Multi-AZ for automatic database failover to a synchronous standby, typically within a minute or two — and make sure the application handles the writer-endpoint change on failover (from scenario 11) rather than assuming failover alone is sufficient.
4. Size for N+1 at minimum — enough capacity that losing one AZ's worth of instances still leaves enough to handle the load, not just enough for the happy-path average.

The root causes and trade-offs.

Designing redundancy at one layer, say the Auto Scaling Group, while leaving another layer as a single point of failure, like a single-AZ RDS instance, gives a false sense of high availability, because the whole design is only ever as resilient as its weakest layer. Making everything Multi-AZ does cost more than staying single-AZ, and that's a deliberate trade of cost for availability that should be sized to the actual availability requirement, not applied blindly everywhere.

The trap that less-experienced engineers fall into.

Making the compute layer highly available with an ASG across AZs while leaving a single-AZ RDS instance underneath it — the database becomes the single point of failure the rest of the design was supposed to eliminate.

🎯 Interviewer follow-up questions you should expect.

• "How do you design for HA at every layer, not just one?" You use an ASG spanning multiple AZs, an ALB spanning the same AZs with health checks, and RDS Multi-AZ for the database, all three together, since the design is only as strong as its weakest layer.
• "What's the trade-off of Multi-AZ everywhere?" It's cost — Multi-AZ should be sized to the actual availability requirement, not applied uniformly regardless of need.

An Auto Scaling Group keeps rapidly adding and then immediately removing instances during a sustained traffic period, instead of settling at a stable size.

What is actually happening (the mental model).

Flapping almost always means the scaling policy's thresholds and the actual metric behaviour are mismatched — a scale-out threshold and a scale-in threshold set too close together (or a cooldown period too short) means the ASG reacts to normal, brief fluctuations in the metric as if they were a sustained trend in either direction, repeatedly overcorrecting.

How to reason about it.

1. Check the gap between scale-out and scale-in thresholds — if they're too close (say, scale out above 70% CPU, scale in below 65%), ordinary metric noise crosses both repeatedly; widen the gap so there's real separation between the two triggers.
2. Check the cooldown period — too short, and the ASG evaluates and acts again before the previous scaling action has had time to actually affect the metric; lengthen it so each action gets a chance to take effect before the next evaluation.
3. Consider a step-scaling or target-tracking policy instead of simple threshold-based scaling, since these are specifically designed to respond proportionally rather than as a binary in-or-out trigger.
4. Verify new instances are actually ready before being counted as healthy — if health checks mark an instance healthy before its application has actually finished starting up (tying back to the VM boot-time scenario), the ASG can misjudge capacity and scale again prematurely.

The root causes and trade-offs.

The root cause is usually scale-out and scale-in thresholds that are set too close together, often combined with a cooldown period too short for the metric to actually reflect a completed scaling action, and sometimes with health checks marking an instance ready before it's genuinely ready. Widening the thresholds and the cooldown trades a little responsiveness for real stability, and that's almost always the right trade for anything that's actually flapping.

The trap that less-experienced engineers fall into.

Reacting to flapping by disabling autoscaling or setting a large fixed instance count instead — which solves the symptom by giving up the actual benefit of autoscaling, rather than fixing the threshold/cooldown mismatch causing it.

🎯 Interviewer follow-up questions you should expect.

• "An ASG keeps flapping — what do you check first?" You check the gap between the scale-out and scale-in thresholds, and the cooldown period — flapping almost always means these are too tight relative to the metric's normal noise.
• "Why not just disable autoscaling if it's causing problems?" That gives up the actual benefit entirely — the real fix is widening the thresholds and cooldowns or switching to target-tracking, not abandoning the mechanism.

"Would you run this on EC2, ECS/Fargate, or Lambda?" — testing whether you match the compute model to the workload instead of defaulting to what you know best.

What is actually happening (the mental model).

This mirrors the Virtual Machines chapter's VM-vs-container-vs-serverless decision, one layer up inside AWS specifically: EC2 for full OS control or workloads that don't fit a container model cleanly; ECS/Fargate (containers, with Fargate removing the underlying server management) for modern, containerised applications wanting fast, consistent deployment without managing EC2 instances directly; Lambda for event-driven, bursty, or intermittent work where paying only for actual execution time beats any idle capacity cost at all.

How to reason about it.

1. EC2: full control needed, a legacy app, or specific licensing/OS requirements Fargate/Lambda can't accommodate.
2. ECS on EC2: containerised, but you want to manage the underlying instances yourself (for cost control at scale, or specific instance types).
3. Fargate: containerised, and you'd rather not manage servers at all — AWS handles the underlying compute.
4. Lambda: genuinely event-driven or intermittent (a webhook handler, a scheduled job, a lightweight API) where idle cost matters and execution time is well within Lambda's limits (from the serverless-gotchas scenario).

The root causes and trade-offs.

One common cause is defaulting to EC2 purely out of familiarity, even for a workload that would genuinely fit Fargate or Lambda better, which is usually a cost and agility issue rather than a correctness one. The opposite mistake is pushing everything into Lambda regardless of fit, which is what runs straight into the cold-start, concurrency, and downstream-overwhelm problems covered in the serverless-gotchas scenario.

The trap that less-experienced engineers fall into.

Treating this as a trendiness contest ("serverless is the future, use Lambda for everything") rather than genuinely matching the workload's actual shape — control needs, execution duration, and traffic pattern — to the right compute model.

🎯 Interviewer follow-up questions you should expect.

• "When would you still choose EC2 over Fargate or Lambda?" You'd still choose EC2 for full OS control, a legacy application, or specific licensing or OS requirements the other two can't accommodate.
• "Fargate vs ECS on EC2 — what's the actual difference?" Both run containers through ECS; Fargate removes the underlying server management entirely, at a cost premium over managing your own EC2 instances for the containers.

Either the production database has been running noticeably slower under normal load for the past week, or a Multi-AZ failover just happened and the application still went fully down even though RDS itself recovered within about ninety seconds.

What is actually happening (the mental model).

These are two faces of the same underlying skill — knowing where to look on a managed database, and knowing that Multi-AZ solving the database's availability doesn't automatically solve the application's availability, since the application still has to notice and adapt to the writer endpoint changing.

How to work through it.

1. Performance Insights — top SQL and wait events (missing index? lock waits?). CloudWatch for CPU/connections/IOPS/replica lag.
2. Connection storms — each app instance's pool multiplies connections; put RDS Proxy in front to pool/multiplex.
3. Multi-AZ vs read replicas — Multi-AZ = synchronous HA standby, auto-failover; read replicas = asynchronous read scaling. Don't confuse them.
4. App-side failover handling — the writer endpoint changes on failover; connections need retry-with-backoff or the failover is an outage.
5. Right-size instance/IOPS; add read replicas for read scaling.

Root causes, and how to tell them apart.

Missing index / bad query (Performance Insights); connection exhaustion (RDS Proxy fixes); undersized instance/IOPS; a failover the app didn't handle. Performance Insights + CloudWatch distinguish them.

How to fix it, verify it, and prevent it.

Resize the machines in a change window, watch the performance for a cycle, and then lock it in. To prevent the problem coming back, use a tagging and ownership policy, review right-sizing recommendations every month, and use autoscaling so that capacity tracks actual demand instead of a static guess.

The trap juniors fall into.

Confusing Multi-AZ (HA) with read replicas (read scaling), not using RDS Proxy for connection storms, and not handling failover in the app (so a recovered database still means an outage).

🎯 Interviewer follow-ups you should expect.

• "RDS is slow — how do you diagnose?" You use Performance Insights for the top SQL and waits, and CloudWatch for connections and IOPS — it's likely a missing index or a connection storm.
• "Multi-AZ vs read replica?" Multi-AZ is a synchronous HA standby with automatic failover; a read replica is asynchronous and built for read scaling.
• "A failover happened and it was an outage even though RDS recovered — why?" The application didn't handle the writer-endpoint change, so it needs connection retry logic.

A Lambda-based system hits limits — cold-start latency, throttling, or it flattened the database it calls.

What is actually happening (the mental model).

Serverless is great until you hit its edges, and a senior engineer can name them. There are cold starts, for which you use provisioned concurrency on latency-sensitive paths. There are account-level concurrency limits, where one runaway function throttles everything else — a nasty coupling. And then there is the big one: Lambda scales faster than your database can. Lambda can go to thousands of concurrent executions almost instantly, which flattens a relational database that cannot take that many connections; you put SQS or RDS Proxy between them to smooth the load. On top of that, there are the 15-minute and payload limits, and a cost crossover at high steady volume.

How to reason about it.

1. Cold starts: use provisioned concurrency for latency-sensitive paths, which adds cost; and note that a VPC-attached Lambda has extra cold-start and ENI considerations.
2. Concurrency limits: these are account-wide, so a runaway function can throttle everything else, which is a coupling you have to manage.
3. Downstream overwhelm: Lambda scales to thousands of concurrent executions instantly and flattens a relational database, so putting SQS or RDS Proxy between them smooths the load.
4. Limits: a 15-minute maximum runtime and payload size limits.
5. Cost crossover: at high steady volume, Lambda can cost more than containers, so you need to know when serverless stops fitting.

The root causes and trade-offs.

Serverless removes operational work at the cost of these edges. The senior skill is knowing where the model stops fitting.

The trap that less-experienced engineers fall into.

Pointing thousands of concurrent Lambdas straight at a relational database, which causes connection exhaustion and an outage; and not knowing that the account-wide concurrency limit couples all the functions together.

🎯 Interviewer follow-up questions you should expect.

• "Lambda flattened your database — why and fix?" Lambda scales faster than the database can accept connections, so you put SQS or RDS Proxy between them.
• "What's the concurrency-limit gotcha?" It's account-wide, so one runaway function throttles everything else.
• "When does serverless stop being right?" It stops being right at high steady volume, where the cost crossover kicks in, for long-running work that hits the 15-minute limit, or for latency-critical paths affected by cold starts.

"SQL vs NoSQL — which AWS database?" — testing whether you choose from access patterns or from hype.

What is actually happening (the mental model).

Choose from the access pattern and the consistency needs, not from fashion. Use RDS or Aurora for relational, transactional work with flexible queries. Use DynamoDB for massive scale with known, fixed access patterns, where you model the table around your queries in what is called single-table design. Use ElastiCache for caching, OpenSearch for search, and Redshift or Athena for analytics. The senior mistake to avoid is picking DynamoDB "for scale" and then fighting it because the access patterns were not nailed down first, since it is painful for ad-hoc or flexible querying.

How to reason about it.

• RDS or Aurora is for relational, transactional work with flexible or ad-hoc queries.
• DynamoDB is for massive scale with known access patterns, where you model the table around the queries (single-table design); it is the wrong choice for flexible or ad-hoc querying.
• ElastiCache is for caching, OpenSearch is for search, Redshift and Athena are for analytics, and services like DocumentDB cover more niche needs.
• Decide based on the access patterns, consistency, scale, cost, and operational model.

The root causes and trade-offs.

DynamoDB is cheap and scales almost infinitely, but only if the access patterns are modelled up front; it is painful if you later need flexible queries. Aurora gives you flexible queries, but at more operational cost at extreme scale.

The trap that less-experienced engineers fall into.

Choosing DynamoDB "for scale" without modelling the access patterns first, and then fighting it when they need the flexible queries it was never designed for.

🎯 Interviewer follow-up questions you should expect.

• "How do you pick a database?" You pick it from the access pattern and the consistency needs, not from hype.
• "When is DynamoDB the wrong choice?" It's the wrong choice when you need flexible or ad-hoc queries, because it needs the access patterns modelled up front.
• "What's single-table design?" It's modelling one DynamoDB table around your specific query patterns.

"How do you structure accounts for a growing org?" — testing whether you use accounts as isolation boundaries and enforce guardrails top-down.

What is actually happening (the mental model).

The account boundary is the strongest isolation that AWS offers — for limiting the security blast radius, for attributing cost, and for keeping a mistake contained to one environment. So you use separate accounts for each environment or team under Organizations, group them into organizational units (OUs), and enforce guardrails with SCPs from the top down (for example, "no one can disable CloudTrail" or "only approved regions"). You centralise the things that should be central, such as logging and security tooling in dedicated accounts and networking through a Transit Gateway, and you bootstrap new accounts consistently with a landing zone or Control Tower.

How to reason about it.

1. Use separate accounts for each environment, team, or blast-radius boundary under Organizations, since this is the strongest isolation.
2. Use OUs together with SCPs for top-down guardrails, such as denying anyone the ability to disable CloudTrail, or restricting which regions can be used.
3. Centralise logging and security tooling in dedicated accounts, connect networking through a Transit Gateway, and use consolidated billing.
4. Use a landing zone or Control Tower to bootstrap new accounts consistently.

The root causes and trade-offs.

One giant account gives no isolation, so everything is within the blast radius; a multi-account structure is isolated but takes more setup. SCPs enforce the guardrails without your having to police each account individually.

The trap that less-experienced engineers fall into.

Running everything in one account, so that a single mistake or breach affects everything and there is no cost attribution; and having no SCP guardrails at all.

🎯 Interviewer follow-up questions you should expect.

• "How do you structure accounts?" You use separate accounts for each environment and team under Organizations, with OUs and SCP guardrails, plus centralised logging and security accounts.
• "What's an SCP?" It's a top-down guardrail that limits what any account can do, for example preventing anyone from disabling CloudTrail or restricting it to approved regions.
• "Why multi-account over one big account?" Because the account is the strongest isolation boundary, for security, cost, and blast radius.

A security/compliance question, or a review: "how do you handle encryption?" — and an "Access Denied" on an encrypted resource that turns out to be a KMS key-policy issue.

What is actually happening (the mental model).

You encrypt at rest (backed by KMS on S3, EBS, RDS, and so on) and in transit (with TLS), and the piece that people underuse is the KMS key policy. A KMS key policy is a separate authorisation layer, which means that accessing an encrypted resource requires access to both the resource and its KMS key. KMS uses envelope encryption, where a data key encrypts the data and the KMS key in turn encrypts the data key. And you choose between AWS-managed and customer-managed keys (CMKs) based on your control and audit needs, because CMKs give you control over key rotation, key policies, and CloudTrail auditing of how the key is used.

How to reason about it.

1. At rest: enable default encryption everywhere (S3, EBS, RDS, and so on), backed by KMS.
2. In transit: use TLS, and enforce it (for example, with an S3 bucket policy that denies non-TLS requests).
3. KMS key policies: these are a separate authorisation layer, so access to an encrypted resource needs key access as well (this is the hidden "Access Denied" from scenario 3).
4. CMK versus AWS-managed: use CMKs when you need control over rotation, the key policy, and CloudTrail auditing of key usage, or cross-account sharing through the key policy.
5. Envelope encryption: KMS encrypts the data keys rather than the data directly, which scales well and keeps the master key inside KMS.

The root causes and trade-offs.

The classic issue is missing key access on an encrypted resource, which is the hidden denial. AWS-managed keys are simple but give less control, whereas CMKs give control and audit at the cost of a little management overhead.

The trap that less-experienced engineers fall into.

Forgetting that the KMS key policy is a separate authorisation layer, and so granting access to the bucket or resource but not to the key, which causes a confusing "Access Denied."

🎯 Interviewer follow-up questions you should expect.

• "Access to an encrypted resource is denied but the resource policy allows it — why?" The KMS key policy doesn't grant access — it's a separate layer.
• "AWS-managed vs customer-managed keys?" CMKs give you control over rotation, the key policy, and CloudTrail auditing of key usage, plus cross-account sharing.
• "What's envelope encryption?" It's KMS encrypting a data key that in turn encrypts the data, so the master key stays inside KMS.

You inherit a messy AWS environment and must assess and improve it — without boiling the ocean.

What is actually happening (the mental model).

Run a structured Well-Architected review across all the pillars (operational excellence, security, reliability, performance, cost, and sustainability), rather than reacting to whatever is loudest, so that you have a complete picture. Tooling does much of the discovery for you, including Security Hub, Config, Trusted Advisor, and Cost Explorer. Then prioritise by risk multiplied by effort: security holes and reliability single points of failure come first, cost comes next (since it is usually high-ROI), and performance comes after that. Present the findings in business terms — meaning risk and dollars — and sequence the remediation as a realistic roadmap.

How to reason about it.

1. Do a structured review across all the pillars, to get a complete picture rather than a reactive one.
2. Use tooling for discovery: Security Hub, Config, Trusted Advisor, and Cost Explorer surface the issues for you.
3. Prioritise by risk multiplied by effort: security exposures and single points of failure first, cost next (which is high-ROI), and performance after.
4. Communicate in business terms — risk and dollars — rather than presenting a wall of technical findings.
5. Sequence the remediation: fix the scariest things first and show progress, rather than trying to boil the ocean.

The root causes and trade-offs.

The choice is between reactive firefighting and a structured review, and between prioritising by risk times effort and trying to fix everything at once, which fails.

The trap that less-experienced engineers fall into.

Reacting to whatever is loudest, so that you never get a complete picture; presenting a wall of technical findings that leadership cannot act on; and trying to fix everything at the same time.

🎯 Interviewer follow-up questions you should expect.

• "You inherit a messy AWS account — where do you start?" You start with a Well-Architected review across the pillars, using tooling for discovery, and then prioritise by risk times effort.
• "How do you prioritise?" Security holes and single points of failure come first, cost comes next since it's high-ROI, and performance comes after.
• "How do you get leadership buy-in?" You present it in business terms — risk and dollars — as a sequenced roadmap.