cgroups v2 Resource Quotas & OOM Watchdogs

1. Concept Overview & Systemic Problem

Autonomous AI agents are inherently unpredictable:

• An agent writes a recursive function to traverse a file graph, forgets the exit condition, and triggers infinite memory allocation.
• In 30 seconds, RAM usage spikes from 200 MB to 32 GB.
• The Linux Out-Of-Memory (OOM Killer) emergency mechanism activates, killing not only the stuck agent but also the primary PostgreSQL database and web server based on unpredictable heuristics.

Control Groups v2 (cgroups v2) is the engineering cage of the Linux kernel. It allows strict boundaries to be assigned to an agent process: even if the agent attempts to allocate a terabyte of memory, the kernel will forcibly limit it to the allocated cap and gracefully terminate only that specific isolated process.

2. Architectural Taxonomy & Mental Model

┌─────────────────────────────────────────────────────────────┐
│                 CGROUPS V2 UNIFIED HIERARCHY                │
├─────────────────────────────────────────────────────────────┤
│ ROOT CONTROL GROUP (/sys/fs/cgroup)                         │
│ ├── system.slice (Critical host services: SSH, Systemd, UFW) │
│ │   ➔ Guaranteed Memory & High CPU Priority                 │
│ └── agent-sandbox.slice (Isolated agent contour)            │
│     ├── memory.max = 1536M (Hard memory limit)             │
│     ├── memory.high = 1200M (Soft limit: throttling)       │
│     ├── cpu.max = 100000 100000 (Maximum 1 full core)      │
│     ├── pids.max = 150 (Protection against Fork Bomb)      │
│     └── io.weight = 100 (Low write priority on SSD)        │
└─────────────────────────────────────────────────────────────┘

3. Technical Pipeline & Internal Mechanics

01. Protecting the Host Server via systemd-run

Running a risky agent script directly in a dedicated cgroup without a container:

systemd-run --scope -p MemoryMax=1G -p CPUQuota=50% -p TasksMax=50 python agent_worker.py

If the script exceeds 1 GB of RAM, the kernel will immediately restart only the agent_worker.py service, leaving all other processes on the server unaffected.

02. Using PSI for Adaptive Load Management

A monitoring daemon reads /proc/pressure/memory. If the pressure metric exceeds 20%, the system automatically suspends the acceptance of new tasks for agents until current computations are completed.

4. Production Engineering Scenarios

• OOM Loops (Infinite Restart Loops): If the memory limit is set too low (e.g., 256 MB for a Node.js application), the agent will crash and restart every 10 seconds, clogging the queues.
• Disabled Swap: In the complete absence of a swap file on the server, a sudden memory spike triggers an immediate SIGKILL. It is recommended to have a small zram or swapfile on fast NVMe to mitigate peak spikes.

5. Pitfalls, Common Mistakes & Security

cgroups v2 is the foundation of stability for any infrastructure interacting with autonomous code. Setting strict hardware quotas ensures that no error or AI hallucination can disrupt the availability of the host server.