CoreWeave treats AI infrastructure as one giant computer

CoreWeave has announced a new multi-rack NVIDIA Vera Rubin NVL72 deployment, connecting hundreds of next-generation GPUs into a single scale-out cluster. According to Chen Goldberg, the company's Executive Vice President of Product and Engineering, this shift represents a fundamental change in how AI hardware is managed. Rather than treating data centers as collections of individual GPUs, modern AI demands that compute, networking, storage, cooling, and software act as one cohesive machine.

This unified approach is critical because today's massive AI workloads are tightly coupled. Goldberg noted that over 90 percent of CoreWeave's AI workloads still run on Kubernetes, but the underlying resource model must evolve to handle these highly interdependent jobs. In this environment, a single slow GPU, a marginal network link, or a minor cooling issue can bottleneck the entire cluster, dragging down performance for the whole workload.

For practitioners, this infrastructure evolution is driven by the rise of autonomous AI agents and complex coding systems. Unlike traditional chatbot requests that finish in seconds, an AI agent might run for hours and make hundreds of calls. This requires infrastructure that can actively observe, heal, and optimize itself during a run. Additionally, as AI-generated code reduces development times—with Goldberg citing an instance where a task was cut from a year to three weeks—the focus of software engineering is shifting from writing code to managing these complex, system-level operations. Ultimately, CoreWeave aims to make this advanced, highly integrated infrastructure accessible to a broader range of professionals rather than keeping it concentrated within a few elite labs.