Auto Scaling
Auto scaling adjusts the number of compute instances (or container replicas) automatically in response to load signals such as CPU utilisation, request rate, or a custom metric. It improves availability under spikes and reduces cost during troughs by removing idle capacity. Effective auto scaling needs sensible minimum and maximum bounds, a warm-up period, and cooldowns to avoid thrashing.
Availability Zone
An availability zone (AZ) is one or more discrete data centres within a cloud region, with independent power, cooling, and networking, connected to sibling zones over low-latency links. Spreading replicas across AZs protects a workload from a single data-centre failure while keeping intra-region latency low. A region typically exposes two to six zones.
Blue-Green Deployment
Blue-green deployment runs two identical production environments - "blue" (current) and "green" (new) - and switches traffic from one to the other once the new version is verified. It makes releases near-instant and rollback trivial, because the previous environment stays intact. The cost is running double capacity during the cutover.
Canary Deployment
A canary deployment routes a small percentage of traffic to a new version, observes error rate and latency, and progressively increases the share only if the metrics stay healthy. It limits the blast radius of a bad release by exposing it to few users first. Automated canary analysis can roll back when service-level objectives degrade.
CDN (Content Delivery Network)
A CDN is a geographically distributed network of edge servers that cache content close to users, reducing latency and offloading the origin. It serves static assets and, increasingly, cacheable dynamic responses and edge-computed logic. Cache behaviour is controlled with TTLs, cache keys, and explicit invalidation.
Container
A container packages an application together with its dependencies into an isolated, portable unit that shares the host operating-system kernel rather than booting a full guest OS. Containers start in milliseconds and are far lighter than virtual machines, which makes them ideal for horizontal scaling and immutable deployment. The OCI image and runtime specifications standardise their format.
Container Orchestration
Container orchestration automates the deployment, scaling, networking, and lifecycle management of containers across a cluster of machines. It schedules workloads onto nodes, restarts failed containers, performs rolling updates, and balances load. Kubernetes is the de facto standard, with managed offerings on every major cloud.
Control Plane
The control plane is the set of components that manage the desired state of a system and make global decisions - scheduling, API serving, and reconciliation - as opposed to the data plane that carries actual user traffic. In Kubernetes it comprises the API server, scheduler, controller manager, and etcd. Keeping the control plane highly available is essential because it governs the whole cluster.
Cloud Region
A cloud region is a separate geographic area containing multiple availability zones, used to place resources close to users and to satisfy data-residency requirements. Most managed services and pricing are region-scoped, and cross-region traffic incurs latency and egress charges. Choosing a region trades off latency, cost, compliance, and service availability.
Disaster Recovery
Disaster recovery (DR) is the strategy and tooling that restores service after a major outage, defined by a recovery time objective (how fast) and a recovery point objective (how much data loss is tolerable). Patterns range from backup-and-restore through pilot-light and warm-standby to active-active multi-region. The right pattern is chosen by balancing cost against the business cost of downtime.
Edge Computing
Edge computing runs computation at points of presence close to users rather than in a centralised region, cutting round-trip latency and reducing origin load. It suits request transformation, authentication, personalisation, and IoT aggregation. Edge runtimes are constrained (limited CPU time, memory, and cold-start budgets) compared with regional compute.
Egress
Egress is data leaving a cloud provider's network toward the internet or another network, and it is typically billed per gigabyte while ingress is usually free. Egress costs are a common source of surprise bills, especially for cross-region replication, CDN origin pulls, and data-heavy APIs. Architecting to keep traffic within a region or behind a CDN reduces egress spend.
Elasticity
Elasticity is a system's ability to acquire and release resources automatically to match demand in near real time, distinct from raw scalability (the ability to grow). An elastic system scales out under load and scales back in when idle, so you pay roughly for what you use. It depends on stateless workloads, fast provisioning, and accurate load signals.
Failover
Failover is the automatic switch of traffic or workload from a failed component to a healthy standby, such as promoting a replica database to primary. It is measured by detection time plus switch time, which together determine the recovery time objective. Well-designed failover is tested regularly, because untested failover often fails when it is finally needed.
High Availability
High availability (HA) is a design goal of minimising downtime by eliminating single points of failure through redundancy, health checking, and automatic failover. It is quantified as an availability percentage (for example 99.9% allows about 8.8 hours of downtime per year). HA addresses component failure, distinct from disaster recovery which addresses large-scale loss.
Horizontal Scaling
Horizontal scaling (scaling out) adds more instances of a component to handle load, distributing work across them behind a load balancer. It is the cloud-native default because it has no hard ceiling and tolerates individual instance failure, but it requires stateless or externally-stored state. It contrasts with vertical scaling, which makes a single instance bigger.
IaaS (Infrastructure as a Service)
IaaS provides virtualised compute, storage, and networking as on-demand building blocks, leaving the operating system, runtime, and application to the customer. It offers the most control and portability among cloud service models, at the cost of more operational responsibility. Virtual machines, block storage, and virtual networks are typical IaaS resources.
Idempotency
In infrastructure and APIs, an operation is idempotent if applying it multiple times produces the same end state as applying it once. Infrastructure-as-Code tools rely on idempotency to converge resources to a desired state safely on repeated runs, and idempotent API operations are safe to retry after a timeout. Non-idempotent create operations are made retry-safe with an idempotency key.
Immutable Infrastructure
Immutable infrastructure treats servers and images as never-modified-after-deployment: to change a running system you build a new image and replace instances, rather than patching them in place. This eliminates configuration drift, makes rollbacks a redeploy of the previous image, and yields reproducible environments. It pairs naturally with containers and golden machine images.
Infrastructure as Code
Infrastructure as Code (IaC) defines and provisions infrastructure through machine-readable definition files kept in version control, rather than manual console actions. It makes environments reproducible, reviewable, and auditable, and enables the same change pipeline used for application code. Declarative tools converge to a desired state; imperative ones execute ordered steps.
Ingress
In Kubernetes, an Ingress is an API object that manages external HTTP and HTTPS access to services in a cluster, providing host- and path-based routing, TLS termination, and load balancing through an ingress controller. More generally, ingress is inbound traffic entering a network, usually not metered for cost. It centralises edge routing rules for many services behind one entry point.
Kubernetes
Kubernetes is an open-source container-orchestration platform that automates deploying, scaling, and operating containerised applications across a cluster, reconciling the actual state toward a declared desired state. It introduces abstractions such as Pods, Services, Deployments, and namespaces, controlled through a declarative API. It is governed by the Cloud Native Computing Foundation.
Load Balancer
A load balancer distributes incoming requests across multiple backend instances to improve throughput, availability, and resilience, removing unhealthy targets via health checks. Layer 4 balancers route on TCP/UDP, while Layer 7 balancers route on HTTP attributes such as path and host. They are a foundational building block for horizontal scaling.
Managed Service
A managed service is a cloud offering where the provider operates the underlying infrastructure, patching, scaling, and high availability, exposing only an API or endpoint to the customer. Managed databases, queues, and Kubernetes control planes trade some control and portability for reduced operational burden. They shift undifferentiated heavy lifting to the provider.
Multi-Cloud
Multi-cloud is the deliberate use of services from more than one cloud provider, for resilience, negotiating leverage, regulatory reasons, or best-of-breed services. It increases architectural and operational complexity and usually forfeits some provider-specific optimisations. It differs from hybrid cloud, which combines public cloud with on-premises infrastructure.
Network Address Translation
Network Address Translation (NAT) remaps IP address spaces by modifying address information in packet headers, letting many private instances share one public address for outbound traffic. A NAT gateway allows resources in a private subnet to reach the internet without being directly reachable from it. It is central to keeping backend resources unexposed while still able to fetch updates.
Observability
Observability is the degree to which a system's internal state can be inferred from its external outputs - its logs, metrics, and traces (the three pillars). Unlike basic monitoring of known dashboards, observability lets engineers ask new questions about novel failures without shipping new code. It is essential for operating distributed systems where failures are emergent.
PaaS (Platform as a Service)
PaaS provides a managed application platform - runtime, build pipeline, scaling, and routing - so developers deploy code without managing servers or operating systems. It accelerates delivery and reduces operational work, at the cost of less control and potential lock-in to the platform's conventions. It sits between IaaS and fully serverless models.
Pod
A Pod is the smallest deployable unit in Kubernetes: one or more containers that share a network namespace, IP address, and storage volumes, and are scheduled together onto a node. Containers in a Pod are tightly coupled and co-located, often a main container plus a sidecar. Pods are ephemeral; controllers like Deployments create and replace them.
Private Subnet
A private subnet is a network segment within a virtual private cloud whose instances have no direct route to the internet and no public IP addresses. Backend services and databases live here so they are unreachable from the public internet, reaching out only through a NAT gateway. Pairing public and private subnets is a core network-segmentation pattern.
Provisioning
Provisioning is the act of allocating and configuring infrastructure resources - instances, storage, networks, and identities - so they are ready to serve a workload. Modern provisioning is automated and declarative through Infrastructure as Code, making it repeatable and auditable. It encompasses both the initial creation and ongoing configuration of resources.
Region Failover
Region failover is the disaster-recovery action of shifting traffic and workload from a primary cloud region to a secondary region after a region-wide impairment. It depends on cross-region data replication, health-based DNS or global load balancing, and regularly tested runbooks. Because it is rare and high-stakes, it is the hardest failover to get right.
Reserved Instance
A reserved instance is a billing commitment to a specific amount of compute capacity for a one- or three-year term in exchange for a substantial discount over on-demand pricing. It suits steady, predictable baseline workloads, whereas spiky or short-lived workloads are cheaper on on-demand or spot pricing. Savings plans are a more flexible variant of the same commitment model.
Serverless
Serverless is a cloud model in which the provider runs and scales code in response to events, billing per invocation and execution time, with no servers for the developer to manage. Functions scale to zero when idle and out automatically under load, but face cold starts and execution-time limits. It shifts capacity planning and patching entirely to the provider.
Service Mesh
A service mesh is a dedicated infrastructure layer that handles service-to-service communication - load balancing, retries, mutual TLS, and telemetry - usually via sidecar proxies, transparently to application code. It centralises cross-cutting networking concerns and gives uniform observability and security across microservices. It adds latency and operational complexity, justified at scale.
SLA (Service Level Agreement)
A service level agreement is a formal contract between a provider and a customer specifying the guaranteed level of service, typically an availability percentage, with financial penalties (service credits) if breached. It is the externally promised target, distinct from the internal service-level objectives an operator sets. Cloud providers publish per-service SLAs.
Spot Instance
A spot instance is spare cloud capacity offered at a steep discount that the provider can reclaim with little notice when it is needed elsewhere. It suits fault-tolerant, interruptible workloads such as batch processing, CI, and stateless web tiers behind a queue, but is unsuitable for stateful, latency-critical services that cannot tolerate eviction. Diversifying instance types reduces interruption risk.
State (Terraform State)
Terraform state is the file that maps the resources declared in configuration to the real-world objects a provider created, so the tool can plan changes by diffing desired against recorded state. It is sensitive (it can contain secrets) and must be stored remotely with locking when teams share it, to prevent concurrent corruption. Losing or hand-editing state is a frequent cause of drift and outages.
Throttling
Throttling deliberately limits the rate of requests or operations to protect a service from overload and to enforce fair usage, returning a retryable error (such as HTTP 429) when limits are exceeded. Cloud APIs and managed services throttle to maintain stability under bursty load. Clients handle throttling with exponential backoff and jitter on retry.
Vertical Scaling
Vertical scaling (scaling up) increases the capacity of a single instance by adding CPU, memory, or faster storage. It is simple and needs no application changes, but it has a hardware ceiling, usually requires a restart, and leaves a single point of failure. It is often the first step before re-architecting for horizontal scaling.
Virtual Machine
A virtual machine (VM) is a software emulation of a physical computer, with its own guest operating system, running on a hypervisor that shares physical hardware among many isolated VMs. VMs provide strong isolation and run unmodified OS images, at the cost of more overhead and slower start than containers. They are the unit of compute in the IaaS model.
Virtual Private Cloud
A virtual private cloud (VPC) is a logically isolated, software-defined network within a public cloud, where you control IP ranges, subnets, routing, and gateways. It lets you segment workloads (public versus private subnets), peer with other networks, and enforce network-level security. It is the foundational networking construct for almost every cloud deployment.
Zero Trust
Zero trust is a security model that assumes no implicit trust based on network location and instead verifies every request's identity, device, and context before granting least-privilege access. It replaces the perimeter "trusted internal network" assumption with continuous authentication and authorisation. In cloud it manifests as strong identity, mutual TLS, and per-request policy enforcement.