Question 1

What is the difference between a Kubernetes Pod and a Deployment?

Nikola Pashov · Accepted Answer

A Pod and a Deployment solve different problems in Kubernetes. Understanding the distinction is fundamental to using Kubernetes correctly.

**Pod** is the smallest deployable unit in Kubernetes. It encapsulates one or more containers that share a network namespace (same IP address and localhost), storage volumes, and lifecycle. Containers in a pod are always co-scheduled on the same node. Pods are ephemeral by design — they are not self-healing. If a Pod crashes, is evicted, or the node it runs on fails, the Pod is gone. Creating a Pod directly with `kubectl apply` gives you exactly one instance that will never be replaced.

**Deployment** is a controller that manages a **ReplicaSet**, which in turn manages a set of identical Pods. You declare desired state — "I want 3 replicas of this container image" — and the Deployment controller continuously reconciles actual state to match. Key capabilities:

- **Self-healing**: if a Pod crashes or a node goes down, the Deployment's ReplicaSet replaces it automatically by scheduling a new Pod on a healthy node.
- **Scaling**: `kubectl scale deployment my-app --replicas=10` adds Pods; the controller handles scheduling.
- **Rolling updates**: update the container image in the Deployment spec, and Kubernetes progressively replaces old Pods with new ones (configurable with `maxUnavailable` and `maxSurge`). Zero-downtime deployments by default.
- **Rollback**: `kubectl rollout undo deployment/my-app` reverts to the previous ReplicaSet instantly — the old ReplicaSet is retained for this purpose.
- **History and audit**: Deployment revision history records what changed and when.

**When to use a Pod directly**: rarely, and only for debugging, one-off inspection tasks, or very simple local development. A standalone Pod provides no resiliency.

**Other controllers**: Deployments are for stateless workloads. For stateful workloads (databases, message queues with persistent storage), use a **StatefulSet** which assigns stable network identities and ordered rolling updates. For per-node daemons (log collectors, monitoring agents), use a **DaemonSet**.

The mental model: a Deployment is a specification of desired state for Pods over time; a Pod is a single instance of that desired state at a point in time.

Question 2

What is the difference between liveness and readiness probes in K8s?

Nikola Pashov · Accepted Answer

Kubernetes uses probes to determine the health of a container, but liveness and readiness probes serve fundamentally different purposes and trigger different control-plane actions.

A **liveness probe** answers the question: "Is this container still alive?" If the probe fails, kubelet kills the container and restarts it according to the pod's `restartPolicy`. Use liveness probes to recover from deadlocks or infinite loops — states where the process is running but making no progress. A classic example is a JVM that has deadlocked: the process is up, but no requests complete. Without a liveness probe the pod would stay in a broken state forever.

A **readiness probe** answers the question: "Is this container ready to serve traffic?" If the probe fails, the pod's IP is removed from the Endpoints object of any matching Service, so the load balancer stops routing new connections to it. The container is *not* restarted. This is the right tool for a pod that is temporarily overwhelmed, waiting for a dependency (database, cache), or performing a warm-up operation like loading a large ML model into memory.

Both probes support three mechanisms: `httpGet` (checks for a 2xx/3xx HTTP response), `tcpSocket` (checks that a port accepts connections), and `exec` (runs a command inside the container and checks the exit code). Key configuration fields include `initialDelaySeconds` (grace period before the first check), `periodSeconds` (how often to check), `failureThreshold` (consecutive failures before action), and `successThreshold` (consecutive successes to recover readiness).

A common production pattern is to run both together: the readiness probe uses a lightweight `/healthz` endpoint that also validates upstream dependencies, while the liveness probe hits a simpler `/livez` endpoint that only confirms the process is responsive. This prevents a cascade where a failing dependency causes a restart loop (liveness) when the correct behavior is to shed traffic (readiness).

Missing a readiness probe is one of the most frequent causes of failed rolling deployments — new pods receive traffic before they are ready, spiking error rates.

Question 3

What is the difference between a ConfigMap and a Secret in K8s?

Nikola Pashov · Accepted Answer

ConfigMaps and Secrets both decouple configuration from container images, but they differ in their intended sensitivity, storage handling, and access controls.

A **ConfigMap** stores arbitrary non-sensitive key-value pairs or entire config files — things like feature flags, log levels, or application properties. The data is stored as plain UTF-8 text in etcd. There is no additional encoding; what you put in is what you read out. ConfigMaps can be consumed as environment variables, command-line arguments, or files mounted into a pod via a projected volume.

A **Secret** is designed for sensitive data: passwords, TLS certificates, API tokens, and SSH keys. By default, Secret values are base64-encoded in the Kubernetes API — this is *encoding*, not *encryption*. The values are trivially reversible with `base64 -d`. The real security comes from layering additional controls: enabling **encryption at rest** in the API server (`EncryptionConfiguration` with an AES-GCM or KMS provider), tightening RBAC so only the pods and service accounts that need a Secret can `get` it, and using external secret managers (Vault, AWS Secrets Manager) via operators like External Secrets Operator.

Kubernetes applies some extra protections to Secrets by default: they are stored in a `tmpfs` (memory-backed) volume on the node rather than written to disk, and they are only sent to nodes that have a pod requesting them. However, anyone with `kubectl get secret` permission on the namespace can read all Secrets there, so RBAC hygiene is critical.

In practice, the injection patterns are identical: `envFrom`, `env.valueFrom.secretKeyRef`, or a `volume` mount. For Secrets in volumes, Kubernetes can also update the mounted file automatically when the Secret changes without restarting the pod, but environment variable injection requires a pod restart to pick up changes — an important operational difference for both resource types.

The key interview point is: base64 is not security. Treat Secrets as a namespace-level access-control primitive, not an encryption primitive, and back them with real encryption and least-privilege RBAC.

Question 4

What is a Helm chart and why would you use one?

Nikola Pashov · Accepted Answer

Helm is the package manager for Kubernetes. A **Helm chart** is a directory of files that together define a deployable unit: Go-templated Kubernetes manifests, a `values.yaml` file of defaults, a `Chart.yaml` with metadata, and optionally subcharts and lifecycle hooks.

The core value proposition is **parameterization with versioning**. Without Helm, teams maintain separate sets of manifests per environment (dev, staging, prod), leading to drift. With Helm, you write a chart once, define variable points with `{{ .Values.image.tag }}` or `{{ .Values.replicas }}`, and install the same chart with different `values.yaml` overrides per environment. The chart is versioned and can be published to a chart repository (Artifact Hub, OCI registries, Harbor), enabling reproducible installs and rollbacks with `helm rollback`.

A **release** is a specific installation of a chart into a namespace. Helm stores release state as Secrets in the target namespace (Helm 3), so the release history is durable and `helm history` shows every revision with its status.

Helm also manages the deployment lifecycle through **hooks** — Jobs or other resources that run at defined points: `pre-install`, `post-install`, `pre-upgrade`, `pre-rollback`. These are useful for running database migrations before an upgrade or sending a Slack notification on rollback.

For complex applications, charts can declare **dependencies** (subcharts) in `Chart.yaml`. For example, your app chart might depend on a `postgresql` subchart from Bitnami, pinned to a specific version, so a `helm dependency update` pulls it in before packaging.

The alternative to Helm is managing raw manifests with `kubectl apply -k` (Kustomize), which uses overlay patching rather than templating. Kustomize avoids template logic but requires more boilerplate for multi-environment variance. Many teams use both: Helm for third-party charts (databases, ingress controllers) and Kustomize for their own application manifests.

Helm 3 removed the server-side Tiller component from Helm 2, addressing a major security concern — all operations now happen client-side using the user's kubeconfig credentials.

Question 5

What is the difference between a Docker container and a VM?

Nikola Pashov · Accepted Answer

Containers and VMs both provide isolated execution environments, but they achieve isolation at different layers of the stack.

A **Virtual Machine** runs a complete guest operating system on top of a hypervisor (Type 1 like VMware ESXi or KVM, or Type 2 like VirtualBox). Each VM includes its own kernel, drivers, and OS userspace. The hypervisor virtualizes hardware — presenting virtual CPUs, memory, and NICs to each VM. This provides strong isolation: a kernel panic in one VM does not affect others. The cost is overhead: each VM consumes gigabytes of disk for the OS image, hundreds of megabytes of RAM for the guest OS, and takes minutes to boot.

A **Docker container** shares the host machine's kernel. Isolation is achieved through Linux kernel primitives: **namespaces** (pid, net, mnt, uts, ipc, user) give each container its own process tree, network stack, and filesystem view; **cgroups** (control groups) limit and account for CPU, memory, and I/O usage. The container image contains only the application and its userspace dependencies — not a kernel. A container starts in milliseconds, images are tens to hundreds of megabytes, and you can run dozens of containers on a machine where you might run only 3-5 VMs.

The trade-off is isolation depth. Because containers share the host kernel, a kernel exploit inside a container can potentially escape to the host. VMs have a harder boundary. For multi-tenant environments with untrusted workloads, VMs (or VM-based container runtimes like Firecracker / gVisor) are preferred. For microservices in a controlled environment, containers provide sufficient isolation with far better density and speed.

In practice, containers and VMs are complementary. Most cloud deployments run containers *inside* VMs: the VM provides the strong isolation boundary at the infrastructure level, and containers provide the packaging and density benefits at the application level.

Docker specifically adds a content-addressable layer filesystem (UnionFS / OverlayFS), a registry protocol for distributing images, and a daemon for managing container lifecycle — these are Docker-specific conveniences on top of the underlying Linux primitives.

Question 6

What are the stages of a CI/CD pipeline?

Nikola Pashov · Accepted Answer

A CI/CD pipeline automates the path from a code commit to a running production deployment, enforcing quality gates at each stage to catch issues early when they are cheapest to fix.

**Source trigger**: The pipeline starts on a push or pull request event from the version control system (GitHub, GitLab). Branch protection rules require the pipeline to pass before merge.

**Build**: Source code is compiled, transpiled, or containerised. The build artifact should be immutable and environment-agnostic—a Docker image tagged with the commit SHA, or a compiled binary. Build caching (layer caching for Docker, dependency caching for npm/pip) keeps this stage fast.

**Unit and integration tests**: Fast, isolated unit tests run first; slower integration tests (requiring databases, message brokers) run after or in parallel with test containers (Docker Compose, Testcontainers). Code coverage thresholds act as gates.

**Static analysis and linting**: Code style (ESLint, Black), type checking (mypy, TypeScript), and static security analysis (Semgrep, Bandit) catch issues without executing code.

**Security scanning**: Dependency vulnerability scanning (Snyk, Dependabot, Trivy for containers) and SAST (Static Application Security Testing) run here. Findings above a severity threshold block the pipeline.

**Artifact publishing**: The verified image or binary is pushed to a registry (ECR, Artifact Registry, PyPI) tagged with the commit SHA and semantic version. The artifact is immutable from this point—the same artifact is promoted through environments.

**Deploy to staging**: The artifact is deployed to a staging environment that mirrors production topology. Smoke tests and end-to-end tests run against staging. This is the last gate before production.

**Deploy to production**: In Continuous Deployment, this is automatic after staging passes. In Continuous Delivery, a human approval gate precedes it. Production deployments use progressive rollout strategies (blue-green, canary, rolling) with automated rollback on error rate or latency SLO breach.

**Observability gate**: After production deploy, the pipeline monitors error rates and latency for a bake window (5-15 minutes). Automated rollback triggers if thresholds are breached.

The key principle is **fail fast, fail early**—put the fastest and highest-signal checks first. A pipeline that takes 45 minutes to run is one that developers work around rather than wait for.

Question 7

What are best practices for logging and monitoring in production?

Nikola Pashov · Accepted Answer

Production observability rests on three pillars—logs, metrics, and traces—and effective logging and monitoring practices make the difference between a 5-minute and 5-hour incident resolution.

**Structured logging**: Emit logs as JSON objects, not free-text strings. Include consistent fields: `timestamp` (ISO 8601 UTC), `level`, `service`, `trace_id`, `request_id`, `user_id` (where applicable), `message`, and context-specific fields. Structured logs are machine-parseable, enabling fast filtering and aggregation in log platforms (Datadog, Splunk, CloudWatch Insights). Free-text logs require regex parsing, which breaks on message changes.

**Log levels with discipline**: DEBUG for development noise (never in production by default), INFO for significant state transitions (request received, job completed), WARN for recoverable unexpected conditions, ERROR for failures requiring attention, CRITICAL/FATAL for system-level failures. Tune log level per environment—DEBUG in staging, INFO in production. Never log sensitive data (passwords, PII, tokens) at any level.

**Correlation IDs**: Propagate a `trace_id` or `request_id` through every service call for a given user action. Log it at every layer. This enables reconstructing the full call path across microservices during an incident. Standards like W3C `traceparent` and OpenTelemetry make this interoperable.

**Metrics**: Logs answer "what happened"; metrics answer "how often / how fast". Instrument with the RED method (Rate, Errors, Duration) per service and the USE method (Utilization, Saturation, Errors) per resource. Export via Prometheus, StatsD, or OTLP. Dashboards in Grafana or Datadog surface trends; alerts fire on thresholds.

**Distributed tracing**: Tools like Jaeger, Zipkin, or Datadog APM stitch together spans across service boundaries. Essential for identifying latency contributions in microservice architectures.

**Log retention and cost**: Production logs have cost at scale. Apply sampling for high-volume debug logs (1% of INFO, 100% of ERROR). Set retention policies (30 days hot, archive to S3 for compliance).

**Alerting on symptoms, not causes**: Alert on user-visible impact (error rate > 1%, p99 latency > 500ms) rather than internal metrics (CPU > 80%). Symptom-based alerts reduce noise and map directly to SLOs.

Question 8

How do you design an effective alerting system?

Nikola Pashov · Accepted Answer

An effective alerting system maximises signal and minimises noise—every alert that fires should require a human response, and every condition requiring a human response should fire an alert. Achieving this balance is harder than it appears.

**Alert on symptoms, not causes**: Alert on user-visible impact (error rate, latency, availability) rather than internal resource metrics (CPU, disk usage). A CPU alert at 80% may never affect users; a 2% error rate always does. This maps alerts directly to SLOs and reduces false positives.

**SLO-based burn rate alerting**: Google's SRE model defines alerts in terms of error budget consumption. A fast burn rate alert fires when the error budget is being consumed at a rate that will exhaust it within 1 hour (high severity, page on-call). A slow burn rate alert fires when the budget will exhaust in 3 days (lower severity, ticket). This approach captures both sudden outages and slow degradations that symptom-threshold alerts miss.

**Severity tiers**: Define clear severity levels with expected response times. P1: system down, data loss risk—wake someone up. P2: degraded performance affecting users—respond within 15 minutes. P3: anomaly worth investigating—handle during business hours. Consistency in severity definitions prevents alert fatigue and sets clear on-call expectations.

**Alert routing and escalation**: Route alerts to the right team based on service ownership (PagerDuty, OpsGenie). Define escalation paths: if the primary on-call doesn't acknowledge within N minutes, escalate to the secondary. For P1s, auto-escalate to the incident commander.

**Reducing alert fatigue**: Deduplicate related alerts—a single infrastructure failure should not generate 50 application alerts. Use alert grouping and inhibition rules (Alertmanager). Set alert resolution auto-close to avoid stale firing alerts cluttering dashboards.

**Runbooks**: Every alert should link to a runbook with: what this alert means, probable causes, diagnostic steps, and remediation actions. Alerts without runbooks transfer cognitive load to the on-call engineer at 3am.

**Alert hygiene reviews**: Regularly audit firing alerts. If an alert fires frequently but requires no action, either fix the underlying issue or raise the threshold. Alerts that don't drive action are noise that erodes trust in the system.

Infrastructure Interview Questions — What Senior Engineers Need to Know

All 8 Questions

How to Prepare

Related Topics

Test Your Knowledge