AI Race Shifts Focus to Power Efficiency and Infrastructure Beyond Chips

The escalating artificial intelligence race is increasingly driven by the need for efficient power and infrastructure, not just faster chips. As AI models grow larger and user bases expand, companies face immense demand for processors, servers, and, critically, electricity. However, building new power plants is a lengthy process, and grid connections have become a bottleneck for data center expansion. To address this, companies are exploring ways to maximize computing power per megawatt, reduce cooling energy consumption, and accelerate data center deployment. This shift presents opportunities beyond the chip market, prompting questions for investors about who will benefit from efficiency gains and who might see reduced demand for equipment.

Software solutions are emerging to extract more computing power from existing hardware. A significant source of waste in data centers is not just chip power consumption, but inefficient utilization. Startups like Atom are developing systems to share GPUs in real-time, claiming up to a 2-3 times increase in workload capacity on the same infrastructure compared to dedicated, underutilized allocations. Expanse focuses on optimizing resource allocation for computing tasks to minimize over-provisioning and failed runs. These software approaches can potentially reduce cloud expenses and electricity usage without requiring immediate hardware upgrades.

NVIDIA is also innovating in data center efficiency with its DSX platform, which integrates power management, cooling, and software to achieve up to 30% less electricity consumption for the same AI output. Features like DSX MaxLPS dynamically allocate power budgets, allowing unused capacity to power more processors. Proof-of-concept tests with Lambda and Nscale showed significant increases in token throughput and processor capacity under fixed power budgets. NVIDIA's DSX Flex aims to allow data centers to adjust power consumption based on grid conditions, potentially reducing energy use by up to 40% in simulated tests without impacting critical workloads.

Beyond software and power management, advanced cooling and optical communication are becoming crucial. Traditional air cooling struggles with the heat generated by high-density computing, leading to a shift towards more efficient liquid cooling systems. NVIDIA is developing closed-loop cooling systems that can save substantial amounts of water compared to traditional cooling towers. In optical communications, companies are working on technologies like Co-Packaged Optics (CPO) to reduce energy consumption for data transfer. New solid-state transformers are also being developed to more efficiently convert electricity for data center needs, supporting higher voltage architectures.

In Israel, the demand for data center power significantly outstrips current generation capacity, with requests totaling approximately 27,000 megawatts against an installed capacity of about 18,800 megawatts. The Israel Electric Corporation has introduced new guidelines to manage this, limiting new facilities to 200 megawatts and allocating additional capacity. Several Israeli companies are advancing data center projects, but their realization depends on securing permits, grid connections, financing, and customer contracts. Investors are advised to look beyond project announcements and examine approved capacity, timelines, funding, and projected returns, as the AI race's success hinges on efficient infrastructure as much as on powerful chips.