AWS and Intel Face Severe CPU Shortages From Agentic AI

Amazon Web Services recently ordered its engineers to minimize CPU usage due to exploding wait times for CPU server capacity. While the initial artificial intelligence boom focused almost entirely on GPUs for model training and inference, the rise of agentic AI has shifted the bottleneck. These autonomous systems rely heavily on CPUs to handle tool use, such as executing code, making API calls, and managing sub-agents. These agents can generate work at machine speed; for example, an OpenAI model once fired off 300 actions in an hour during an inadvertent hack of Hugging Face. According to AMD, seven out of eight stages in realistic agentic AI pipelines run solely on the CPU rather than the GPU.

This architectural shift has introduced severe latency challenges. Researchers from the Georgia Institute of Technology and Intel found that scheduling optimizations can reduce end-to-end latency by up to 1.8 times under sustained workloads. Another study co-authored by Georgia Tech's Euijun Chung revealed that insufficient CPU cores cause GPUs to stall while waiting for instructions. Furthermore, agentic tool calls require frequent tokenization of massive datasets. Adding a 1,000-token tool result to an ongoing 100,000-token sequence forces the CPU to re-tokenize the entire sequence. Chung's testing on Alibaba's Qwen 3-30B and Meta's Llama 3.1-70B showed that increasing CPU core counts can reduce time-to-first-token latency by 1.5 to 7 times.

The hardware market is already reacting. Intel has sold out of server CPUs through the end of the year, prompting it to cut client CPU production to prioritize enterprise chips. AMD has doubled its server CPU forecast, while Qualcomm and Arm have announced new processors tailored for agentic workloads. Even Nvidia is prioritizing Vera, its Arm-based CPU designed for agentic AI within its Vera Rubin platform. For AI practitioners, this shift means that optimizing CPU usage is now just as critical as managing GPU memory. As agentic systems scale to handle sequence lengths of up to one million tokens on models like Anthropic's Claude, developers must prepare for higher cloud computing costs and potential hardware shortages.