☁️Cloud Engineer

IaaS provides raw infrastructure like VMs, networks and storage (e.g. EC2) with you managing the OS and apps. PaaS provides a managed platform to deploy code without managing servers (e.g. Elastic Beanstalk, App Service). SaaS delivers finished software over the web (e.g. Gmail, Salesforce). Control decreases from IaaS to SaaS while operational burden shifts to the provider.

Virtualization: runs full guest operating systems on a hypervisor with strong isolation at the cost of higher overhead and slower startup. Containerization: shares the host kernel with isolated user spaces for fast startup and density, trading some isolation strength for efficiency, which suits microservices and CI jobs.

Elasticity: automatically acquiring and releasing resources to match demand in near real time, like scaling out at peak and to zero at idle. Scalability: the ability to handle larger loads by design, whether manually or automatically, without re-architecting, measured by linear performance as capacity grows.

Providers secure the physical cloud itself including hardware, hypervisors and managed service patching, while customers secure what they put in the cloud like data, identities, configs and app code. Responsibility shifts with service type: more customer duty on IaaS, less on SaaS. Misconfigurations like open buckets remain customer faults.

Regions are isolated geographic areas, availability zones are independent data centers within a region with separate power and networking, and edge locations cache content near users via CDNs. Deploy across zones for high availability, across regions for disaster recovery, and at the edge for latency. Choose regions for latency, compliance and service availability.

A service mesh adds sidecar proxies with a control plane to handle mTLS, retries, traffic shifting, telemetry and policy without app code changes. It suits large microservice fleets needing zero-trust and progressive delivery. Weigh latency, memory and operational overhead, and start with ingress plus observability before full mesh adoption.

VPC peering: direct one-to-one private links between two VPCs, simple and low cost but mesh complexity grows quickly. Transit Gateway: a central hub routing many VPCs and on-prem links with shared route tables and inspection, which scales cleanly at higher hourly cost and design effort.

During partitions, cloud stores choose consistency like strongly consistent databases or availability like highly available object and DNS-backed stores. Design for partition tolerance always, then match money paths to CP and content or carts to AP with reconciliation. Validate with fault-injection and failover drills.

Infrastructure as code defines networks, servers and policies in versioned text applied through automation for repeatable environments. It enables reviews, testing, audit trails and fast disaster recovery while reducing click-ops drift. Treat it like app code with modules, pipelines and ownership.

GitOps keeps git as the desired-state source with agents reconciling clusters and infrastructure automatically, giving auditable rollbacks via revert. Protect main with reviews and policy checks, promote by merge, and alert on out-of-band drift. Separate app and infra repos with clear ownership.

Edge runs logic at CDN PoPs or devices close to users for sub-round-trip latency, offline tolerance and bandwidth savings. Use Lambda at Edge or CloudFront Functions for auth, personalization and image shaping, keeping state centralized. Debug with staged traffic and centralized logs because distribution complicates tracing.

The AWS Well-Architected Framework reviews workloads across six pillars: operational excellence, security, reliability, performance efficiency, cost optimization and sustainability. For each pillar apply design principles like least privilege, multi-AZ deployment, right-sizing and automation, and remediate high-risk findings first. Use the review as a recurring checklist before launches and migrations.

Serverless runs code on provider-managed compute that scales to zero and bills per invocation, such as Lambda, Cloud Functions or Azure Functions. It removes server management and suits event-driven, spiky or glue workloads. Trade-offs include cold starts, execution time and payload limits, vendor coupling and harder local debugging, so keep functions small, stateless and observable.

Map workload needs to families: general purpose for balanced apps, compute optimized for CPUs, memory optimized for caches and databases, storage optimized for high IOPS, and accelerated for GPUs. Start from metrics for CPU, memory and network, test with representative load, and prefer newer generations for price performance. Revisit sizing after each release.

Match access patterns to classes: Standard for hot data, Infrequent Access for monthly reads, One Zone for replaceable data, Intelligent Tiering for unknown patterns, and Glacier tiers for archives. Automate transitions plus expirations with lifecycle rules and abort incomplete multipart uploads. Monitor retrieval fees before archiving.

Create public subnets with internet gateways for load balancers and NAT, private subnets for apps with NAT egress, and isolated subnets for databases without internet routes. Spread across three zones, restrict route tables and security groups, and log flows. Reserve IP space for growth and peer cleanly to shared services.

Target groups register healthy instances behind ALB or NLB with path and port health checks, while autoscaling policies add capacity on CPU, latency or request-count signals with cooldowns. Warm pools and lifecycle hooks reduce cold starts, and connection draining protects in-flight requests. Test scaling with realistic load ramps.

Accounts trust specific roles from other accounts or federated identity providers, issuing short-lived credentials after authentication with MFA and conditions. Use them instead of shared keys for third parties, CI and partner access, with external ids to prevent confused-deputy issues. Log every assumption centrally and expire aggressively.

Collect metrics, logs and traces in CloudWatch with SLO dashboards plus composite alarms, and record every API call immutably in CloudTrail with log-file validation. Alert on burn rates and anomaly detectors, forward to SIEM for security, and retain per compliance tiers. Link alarms to runbooks for fast response.

CloudFront caches content at edge PoPs by cache keys for paths, headers and cookies with TTLs and invalidations, shielding origins and cutting latency. Tune origin shielding, compression, signed URLs and WAF rules, and separate static from dynamic behaviors. Measure hit ratios per path to refine keys.

Land raw data in Standard, curate Parquet partitions for analytics, then lifecycle age cold prefixes to Infrequent Access and Glacier Deep Archive with retrieval-time expectations. Catalog with Glue, enforce encryption and access logging, and validate restores for compliance. Model access frequency before choosing tiers.

SQS: queue-based point-to-point messaging where one consumer processes each message with visibility timeouts and dead-letter queues for reliability. SNS: pub-sub fan-out where one publish reaches many subscribers like queues, functions and emails at once, favoring broadcast over competing consumption.

Use private subnets with VPC CNI plus security groups for pods, EBS and EFS CSI drivers for state, ALB ingress with WAF, and Cluster Autoscaler plus Karpenter with HPA on custom metrics. Bind IAM roles to service accounts, encrypt secrets, and upgrade in stages with PodDisruptionBudgets. Observe with Prometheus plus CloudWatch Container Insights.

Terraform is a declarative infrastructure-as-code tool: you describe desired state in HCL, and Terraform plans then applies changes via providers. It tracks reality in a state file with remote backends like S3 plus locking, and supports reusable modules, workspaces and drift detection. Emphasize code review for plans, remote state locking, and separating environments by workspace or directory.

Model infrastructure in version-controlled code with declarative tools like Terraform, CloudFormation or Pulumi, review plans like app code, store state remotely with locking, and apply through CI and CD with staged environments. Structure with reusable modules, lint and policy checks such as tflint and Checkov, drift detection, and secrets from a manager rather than plaintext.

Store state remotely in S3 with versioning plus DynamoDB-style locking or Terraform Cloud, encrypt at rest, and never commit state files with secrets. Split state by environment and service to limit blast radius, and apply only through CI with plan review. Recover with versioned history and regular state backups.

Build small versioned modules with clear inputs, outputs and examples, pin provider and module versions, and compose environments from shared modules without forking. Publish to a private registry with changelogs, lint and policy checks, and test with plan fixtures. Own modules like products with support and deprecation notes.

Reference secrets from a manager at apply time with data sources, never hardcode values or commit state containing them. Use short-lived dynamic credentials, separate sensitive variables with masked CI logs, encrypt remote state, and rotate regularly. Scan plans and repos for leaked strings before merge.

Write declarative HCL, run format and validate, then plan to preview a diff against state and providers, get peer review, and apply through CI with locked state. Store plans as artifacts for audited applies, and verify with drift checks plus smoke tests after apply. Roll forward with fixes rather than manual console edits.

Schedule drift detection with plan-only runs, config rules and state comparisons, alert on manual console changes, and reconcile via GitOps applies rather than clicks. Lock production to pipeline-only changes, require reasons for exceptions, and backfill code for any hotfix immediately. Track drift rate as a reliability metric.

Providers are plugins that translate HCL into vendor APIs for clouds and SaaS, pinned by version and configured with credentials. Resources declare managed objects like instances and buckets with arguments, attributes and lifecycles. Compose them into modules, import existing objects carefully, and upgrade providers in test environments first.

Run format, validate and plan reviews, lint with tflint, check costs and security with Checkov or Sentinel, and test modules with Terratest or kitchen fixtures on ephemeral stacks. Require peer approval and show human-readable plan summaries. Promote only passing versioned modules to prod registries.

Prefer separate directories or stacks per environment with shared modules for strong isolation, using workspaces only for lightweight copies. Isolate state, variables and credentials per environment, promote via versioned modules, and gate prod applies with approvals. Document which pattern each repo uses to avoid confusion.

Fix forward when possible by correcting code and re-applying reviewed plans, restoring from versioned state backups only if state corrupted. Use prior module versions, targeted plans cautiously, and verify with drift plus smoke tests. Never hand-edit cloud resources without backfilling code.

Design for failure: spread across availability zones and regions, use load balancers with health checks, build stateless services with shared state in managed stores, automate failover, keep backups with tested restores, and run chaos and game-day exercises. Define RTO and RPO targets first, then add redundancy, graceful degradation, retries with backoff, and runbooks for on-call recovery.

Containers package apps with dependencies for portable, long-running services you control, including scaling, patching and orchestration via ECS or Kubernetes. Serverless abstracts servers away with per-request billing and scale-to-zero, ideal for event-driven or bursty work. Choose containers for steady-state services needing control and serverless for variable or glue workloads; many architectures mix both.

Serve writes or reads from multiple regions with global routing, replicated datastores with conflict handling, and idempotent APIs that tolerate replication lag. Use health-checked DNS or anycast failover, test partition behavior, and define RTO and RPO per tier. Prefer active-passive for strong-consistency workloads that cannot merge conflicts.

Use managed control planes like EKS, AKS or GKE with private endpoints, managed node groups or autopilot, cluster autoscaler plus HPA, and cloud load balancers via ingress controllers. Integrate IAM roles for service accounts, encrypted etcd and secrets, persistent volumes with CSI drivers, and GitOps deploys with staged rollouts. Harden with Pod Security and network policies.

Build immutable artifacts once, promote the same image through dev, staging and prod with automated tests plus policy and security scans. Deploy with blue-green or canary strategies, automated rollback on SLO burn, and GitOps reconciliation for drift. Require approvals for prod, sign artifacts and keep full audit trails.

Define RTO as max tolerable downtime and RPO as max tolerable data loss per tier, then pick backup-restore, pilot-light, warm-standby or active-active patterns to match. Automate cross-region backups with tested restores, rehearse failover game days, and document decision rights plus comms. Measure actual recovery times, not hoped ones.

Route domain events through buses like EventBridge to small single-purpose functions with schema registries, idempotent handlers and dead-letter queues. Control fan-out, concurrency and payload limits, trace end to end with correlation ids, and prefer step functions for orchestration. Load-test cold starts and quota ceilings early.

Blue-green runs two identical environments and flips traffic after health checks for instant rollback, at double capacity cost. Canary shifts a small traffic slice to the new version, watches SLOs, then ramps gradually for early detection. Automate promotion and rollback on error budgets and keep database changes backward compatible.

Hypothesize steady state with SLOs, inject bounded faults like zone loss or latency in staging then prod with automatic halts, and verify detection plus recovery runbooks. Start with dependency kills and progress to region evacuations, always with blast-radius limits and game-day owners. Turn findings into backlog fixes with retests.

Use queues like SQS for one-to-one task distribution with competing consumers and dead-letter handling, and pub-sub like SNS for one-to-many fan-out to queues, functions and emails. Filter at the topic to cut noise, keep messages small with payload offloading, and trace with message ids. Match retention and ordering needs to the choice.

Centralize JWT validation, API keys and usage plans with per-method throttles plus quotas, validate schemas, cache safe responses, and emit per-client metrics with 429 plus Retry-After. Stage APIs separately, require WAF association, and document limits with self-serve quota requests.

Managed SQL such as RDS or Cloud SQL gives schemas, JOINs and ACID transactions for relational workloads needing consistency. Managed NoSQL such as DynamoDB, Firestore or Cosmos DB gives flexible schemas, massive horizontal scale and single-digit millisecond access for key-based or high-write workloads. Choose by access patterns: complex relational queries favor SQL, extreme scale or variable shapes favor NoSQL.

Automated backups enable point-in-time recovery within retention windows while snapshots support cloning and cross-region copies. Multi-AZ keeps a synchronous standby in another zone with automatic failover for high availability, not for read scaling. Test restores, monitor replica lag and failover events, and document runbooks.

Choose DynamoDB for massive key-value scale with single-digit millisecond latency, serverless billing and global tables, when access patterns are known upfront. Choose Aurora for relational JOINs, transactions and ad hoc SQL with familiar Postgres or MySQL compatibility. Decide from query shapes and consistency needs, not hype.

DynamoDB hashes partition keys to spread data and throughput: even high-cardinality keys scale linearly while a hot key throttles one partition. Choose keys with uniform access, add sort keys for range queries, and use GSIs or write sharding for skewed patterns. Monitor consumed versus throttled requests and split hot items.

Global tables replicate writes across regions with conflict resolution by last-writer or app logic for low-latency local reads and writes. Monitor replication lag, design idempotent updates, and test partition behavior plus failover routing. Use them for shopping carts and profiles, not for ledger workloads needing strong consistency.

Replicate with DMS or native logical replication, validate with checksums and shadow traffic, cut over writes during low traffic with brief freezes, and keep rollback snapshots plus reverse replication. Tune parameter groups and instance classes beforehand and rehearse on clones. Freeze schema changes during the migration window.

Managed backups automate full plus incremental copies with retention, cross-region replication and encryption, while point-in-time recovery replays logs to any second within the window. Test restores for RTO and RPO compliance, lock vaults against deletion, and alert on failed jobs. Document table-level versus full restore paths.

Front databases with CDN plus ElastiCache or MemoryDB, add read replicas across zones, and shape hot keys with request coalescing and short TTLs. Separate OLTP from analytics traffic, prewarm after deploys, and monitor hit rates plus replica lag. Fall back gracefully to stale reads with clear headers.

Monitor slow query logs, Performance Insights and replica lag, right-size instance classes, and tune parameter groups for connections, buffers and timeouts with staged rollouts. Add reader endpoints for scale, optimize indexes and queries first, and test failover times. Snapshot before parameter changes.

ElastiCache for Redis or Memcached caches hot reads, sessions and leaderboards in memory with microsecond latency, absorbing spikes before they hit the database. Use cluster mode, right TTLs, lazy plus write-through fills, and failover replicas across zones. Monitor hit rates and evictions to prove value.

Gain visibility with cost allocation tags and budgets with alarms, then right-size instances, use Savings Plans or Reserved Instances for steady load, Spot or preemptible capacity for fault-tolerant batch work, lifecycle policies for storage tiers, and scheduled shutdowns for idle dev resources. Review top-cost services monthly and treat cost as an architecture metric.

Baseline CPU, memory, disk and network utilization over business cycles, then downsize idle heads, match families to profiles, and set autoscaling bounds with load tests. Enforce tagging and ownership so low-use resources have accountable owners. Re-check monthly because usage drifts after every release.

Tag every resource by team, service and environment, feed cost and usage into dashboards with budgets and forecasts, and charge or show costs back monthly with variance reviews. Set guardrails like required tags and policy denials, celebrate savings, and tie forecasts to roadmaps. Treat unit cost per request as the key efficiency metric.

Run fault-tolerant batch, CI and stateless web tiers on Spot for deep discounts with diversified pools across zones and types plus capacity-optimized allocation. Handle two-minute interruptions with checkpointing, draining and fallback to on-demand. Never place quorum or single-leader databases on Spot alone.

Savings Plans: commit to hourly spend for one or three years across flexible instance usage, ideal when families or regions shift. Reserved Instances: commit to specific instance types in a scope for steady-state loads with higher discounts but less flexibility, ideal for stable databases and baseload fleets.

Right-size volumes with gp3 tuning, delete unattached volumes and old snapshots, tier S3 with lifecycle rules to Infrequent Access and Glacier, compress and partition analytics data in columnar formats, and expire logs aggressively. Show per-team storage dashboards so owners clean their own waste.

Inventory untagged or low-use instances, volumes, IPs, load balancers and snapshots with cost explorer plus rightsizing reports, notify owners, stop or snapshot-then-delete after grace periods. Automate parking schedules for dev and require TTL tags on experiments. Report reclaimed spend to sustain the habit.

Require tags for owner, service, environment and cost center via policy denials and default-tag providers, backfill with automation, and report untagged spend weekly. Feed tags into budgets, dashboards and chargeback, and block deploys missing required tags. Review taxonomy quarterly to prevent sprawl.

Create budgets per account and service with actual plus forecasted thresholds, route alerts to owners in Slack and email, and enable anomaly monitors for spikes with root-cause links. Review weekly, attach runbooks for triage, and freeze experiments that breach guardrails. Forecast seasonal peaks ahead of launches.

Auto-stop nights and weekends with Instance Scheduler or Lambda, tag ephemeral stacks with TTLs for reaping, and use smaller Spot-backed sizes for non-prod. Notify owners before deletion, keep one-click recreate scripts, and report savings per team. Exempt only load-test windows by request.

Security groups are stateful virtual firewalls attached to instances or ENIs: return traffic is automatically allowed and rules are allow-only. NACLs are stateless subnet-level firewalls with numbered allow and deny rules evaluated in order, requiring explicit inbound and outbound entries. Use security groups for workload rules and NACLs as a coarse subnet backstop.

IAM authenticates identities like users and roles, then authorizes actions through JSON policies with explicit allow or deny on resources and conditions. Roles with temporary credentials suit workloads and cross-account access while users suit humans with MFA. Apply least privilege, use managed plus scoped inline policies, and review with access analyzers.

Segment into public, private and data tiers across zones with route tables, security groups as instance firewalls and NACLs as subnet backstops. Keep databases private without internet routes, use NAT or egress-only gateways for outbound, plus flow logs and IDS inspection. Peer or transit-connect VPCs with least-privilege routes and DNS controls.

Start from denied-by-default, grant only required actions on specific resources with conditions like MFA, IP and time, and test with policy simulators plus access advisors. Prefer roles over long-lived users, scope down wildcards iteratively, and review unused grants quarterly. Automate checks in CI to block broad permissions.

Enable KMS-managed encryption with customer keys and rotation for disks, buckets and databases, enforce TLS everywhere with modern versions, and separate duties for key admins versus data users. Audit with config rules, require encrypted volumes by policy, and log key use to detect anomalies. Rotate and scope keys per service tier.

Verify every connection with workload identity, mTLS and per-request policy instead of trusting VPC boundaries. Segment with security groups, private endpoints and service meshes, require MFA and device posture for admins, and log all flows centrally. Automate policy as code and review effective access often.

Store credentials in Secrets Manager with KMS encryption, automatic rotation and fine-grained IAM access, injecting at runtime rather than baking into images. Audit access with CloudTrail, separate rotation duties, and reference by ARN with version stages. Alert on failed rotations and stale unrotated secrets.

Scan images in registries, use minimal distroless bases with pinned digests, run as non-root with read-only filesystems, and enforce admission policies plus runtime threat detection. Sign images, rotate base layers promptly, and isolate namespaces with quotas and network policies. Keep secrets out of layers.

Detect with GuardDuty plus audit trails, isolate affected keys, instances and roles by revoking sessions and quarantining resources, preserve snapshots and logs for forensics, then eradicate and rotate broadly. Document timelines, notify stakeholders per policy, and convert lessons into detective controls and drills.

Centralize identities in IAM Identity Center with SAML or OIDC to your provider, require hardware or authenticator MFA, and use short-lived permission sets instead of static keys. Audit sign-ins, alert on MFA gaps, and automate joiner-mover-leaver provisioning. Test break-glass access regularly.