Chapter 7 • Premium Interview Kit Kubernetes 1.30+ / eBPF Standard ⏱️ 40 Min Read

Chapter 7: Docker & Kubernetes Infrastructure Internals

Deep-dive architectural breakdown of Linux Namespaces, cgroups v2, Overlay2 filesystem, containerd/runc runtime, kube-apiserver pipeline, PLEG, eBPF Cilium networking, and oom_score_adj QoS allocation.

👨‍💻
Nagendra Rana
SDE-2 @ Digilytics AI • Lead Architect WhatInfoTech

1. Concept Introduction

From Heavy Hypervisors to OS-Level Virtualization

Before containerization gained widespread adoption, infrastructure isolation relied primarily on Hardware Virtualization (Type-1 and Type-2 Hypervisors). Containers replace hardware emulation with OS-Level Virtualization, using Linux Kernel primitives to constrain processes running directly on the host CPU.

virtualization-vs-containers.txt • Architecture Shift
+-------------------------------------------------------------------------+
|                 OS-LEVEL VIRTUALIZATION (CONTAINERS)                    |
|  +--------------------+  +--------------------+  +-------------------+  |
|  | Container A (App)  |  | Container B (App)  |  | Container C (App) |  |
|  | - Binaries/Libs    |  | - Binaries/Libs    |  | - Binaries/Libs   |  |
|  +--------------------+  +--------------------+  +-------------------+  |
|  | Container Runtime (containerd / runc)                                |  |
|  +-------------------------------------------------------------------+  |
|  | Host OS Kernel (Namespaces + cgroups v2 + OverlayFS)                |  |
|  +-------------------------------------------------------------------+  |
+-------------------------------------------------------------------------+

2. Theory & Low-Level Internals

A. Linux Namespaces (7 Isolation Primitives)

Namespace Flag (`clone(2)`) Isolated System Resource
PID`CLONE_NEWPID`Process IDs (Container process becomes `PID 1`).
NET`CLONE_NEWNET`Network devices, IP routing tables, firewall rules.
MNT`CLONE_NEWNS`File system mount points (Isolated view of `/`).
IPC`CLONE_NEWIPC`System V IPC objects, POSIX message queues.
UTS`CLONE_NEWUTS`Hostname and NIS domain name.
USER`CLONE_NEWUSER`User and group ID mappings (Non-root is `root` inside).
CGROUP`CLONE_NEWCGROUP`Virtualized cgroup root directory view.

B. Overlay2 Union File System

overlay2-filesystem.txt • CoW Layer Topology
+-------------------------------------------------------------------------+
| MERGED DIRECTORY (/var/lib/docker/overlay2//merged)               |
| Unified view exposed to the running container process                   |
+-------------------------------------------------------------------------+
                                     │
            ┌────────────────────────┴────────────────────────┐
            ▼                                                 ▼
+------------------------------------+   +------------------------------------+
| UPPERDIR (Read-Write Container)    |   | LOWERDIR (Read-Only Image Layers)  |
| Holds modified files, deletions    |   | Base OS, runtime, application code |
+------------------------------------+   +------------------------------------+

C. Kubernetes Control Plane Architecture

k8s-control-plane.txt • Master & Worker Subsystems
+-----------------------------------------------------------------------------------+
|                            KUBERNETES CONTROL PLANE                               |
|  kube-apiserver ──> etcd (Raft MVCC) ──> kube-scheduler ──> kube-controller-mgr   |
+-----------------------------------------------------------------------------------+
                                          │
                                          ▼ gRPC Watch Stream
+-----------------------------------------------------------------------------------+
| WORKER NODE: kubelet (SyncLoop / PLEG) ──> containerd ──> runc ──> CNI (Cilium)   |
+-----------------------------------------------------------------------------------+

3. Real Production Architecture Example

Enterprise Multi-Tenant High-Availability Cluster with Zero-Trust Cilium eBPF network policies, Karpenter dynamic compute node provisioner, and KEDA event-driven autoscaler.

4. Code Examples

A. Production Pattern: Multi-Stage Distroless Dockerfile

# Stage 1: Build Stage
FROM node:20-alpine AS builder
WORKDIR /build
COPY package*.json ./
RUN npm ci --only=production
COPY . .
RUN npm run build

# Stage 2: Production Distroless Stage (No Shell, No Package Manager)
FROM gcr.io/distroless/nodejs20-debian12:nonroot
WORKDIR /app
COPY --from=builder /build/node_modules ./node_modules
COPY --from=builder /build/dist ./dist

USER 65532:65532
ENTRYPOINT ["/nodejs/bin/node", "dist/server.js"]

B. Low-Level C Code Container Isolation Engine (`clone(2)`)

#define _GNU_SOURCE
#include <sched.h>
#include <stdio.h>
#include <sys/mount.h>

static int child_exec(void *arg) {
    mount(NULL, "/", NULL, MS_REC | MS_PRIVATE, NULL);
    mount("proc", "/proc", "proc", 0, NULL);
    sethostname("isolated-container", 18);
    char *cmd[] = { "/bin/sh", NULL };
    execv(cmd[0], cmd);
    return 0;
}

5. Tiered Interview Questions

Senior Level • Tech Lead

Q1: Walk through `kubectl run` end-to-end execution sequence

Ideal Answer: 1. APIServer RBAC & Admission Webhooks, 2. Persists spec to `etcd`, 3. `kube-scheduler` filters & scores nodes, setting `spec.nodeName`, 4. `kubelet` SyncLoop detects Pod via gRPC Watch stream, 5. Invokes CRI (containerd), CNI (Cilium IP allocation), and `runc` execution via `clone(2)`.

Architect Level • Principal Engineer

Q2: How does Kubernetes assign QoS classes and control Linux oom_score_adj?

Ideal Answer: Guaranteed (`requests == limits`) sets `oom_score_adj = -997` (immune). Burstable sets score dynamically ($2–999$). BestEffort sets `oom_score_adj = 1000` (first candidates killed during memory starvation).

6. Production Debugging Scenario

Symptom: 5-Second DNS Resolution Timeout & CrashLoopBackOff

Root Cause: Simultaneous parallel UDP A and AAAA DNS queries from glibc triggered a `conntrack` SNAT lock race condition in the kernel, dropping packets.

Resolution: Deployed **NodeLocal DNSCache** DaemonSet to bypass UDP network traversal and conntrack races.

Official Partner Verified Opportunity

📜 Official Developer Certification & Cloud Hosting

Validate your software engineering skills with official developer certifications and high-performance cloud hosting.

Support My Work