How AI is pushing a data center network rethink

Twenty years ago, Jawed Karim uploaded a 19-second video to a nascent platform called YouTube. That “Me at the zoo” video triggered a massive change in the way people consume content and turned network design on its head. 

Fast forward, and AI is having a similar impact today, except the effect on the cloud and network is far greater, and new app adoption is happening much faster. YouTube took three years to reach 100 million users. ChatGPT took two months.  

Data volumes are pushing the outer limits of data center processing and storage capacity. The expectations around performance, reliability and security are magnitudes higher. And quality is AI’s lifeblood. While a glitch in a video can be frustrating, a glitch in an AI model can harm the validity of the result – and possibly even a client’s reputation.  

The network behind the cloud isn’t top of mind these days due to other priorities such as securing the required compute capacity, ensuring a reliable and sufficient energy supply, implementing effective cooling technologies, and locking down locations. These demands are so extraordinary that industry leaders are musing about harvesting solar energy directly from space and deploying data centers on the moon. 

Yet, the network will play a gating role in how far AI and the cloud can evolve. Ignoring the network risks bottlenecks by leaving expensive compute resources sitting idle while they wait for data to be transmitted between them. 

The cloud exists because of the network. The evolution of the cloud, whether it was to support video in the past or AI in the future, is inextricably tied to the evolution of the network. 

So, what’s needed for the network to help data centers succeed in the age of AI? Let’s look first at the AI use cases that will ratchet up the pressure on a data center network. 

The network-cloud continuum

Consider first centralized cloud-based gaming. When fast reaction times decide the winner, users will still choose a console over the cloud. That’s because latency degrades performance enough that the game winner is often the one with the best connectivity. 

Similarly, in the future, many AI use cases are expected to require fast reaction times. 

AI applications today are fairly limited and mostly text-based. Most of the AI traffic is generated to train large language models (LLMs) in big, centralized AI factories owned by hyperscalers, a few governments, research institutes, and very large enterprises. 

But by 2030, 60-70% of all AI workloads will be used for real-time AI inferencing, according to McKinsey & Company.

In other words, as AI application adoption grows, the focus will shift from training to predicting and answering requests from humans, machines and agents. For mission-critical applications, this will require rapid, real-time processing and data analysis where network speed and low latency are essential. 

AI workload distribution

Inferencing will also drive the distribution of AI workloads closer to the consumers of the applications. Reducing the round-trip delay will improve the overall response time of the AI models, improving the user experience and reducing bandwidth consumption. 

Limiting data transmission over networks can also be critical when there are privacy and security concerns. Applications that require rapid response times or that operate in environments with limited or high-cost connectivity will benefit from distributed AI workloads. 

Another perk of distributing AI workloads is the opportunity to locate data centers closer to power and cooling sources. This can make a noticeable difference in how data centers operate, helping to control costs and boost efficiencies. 

Economics will also drive data center network decisions. As data centers expand and grow, cloud arbitrage will come into play. This involves dynamically running workloads on whichever cloud offers the best price-to-performance, allowing for a workload to move anywhere and at any point in the processing. Request-response fanouts determine where to find the lowest cost compute while still meeting the quality of experience required by the end user. 

New needs for data center networking

While some of these use cases may seem fantastic today, the speed at which AI is forcing change makes them closer than one might think. That’s because it’s not just humans, or eyeballs, driving traffic – it’s machines. Bot traffic accounted for 30% of all HTTP requests in early 2025, according to Cloudflare Radar data.

Sensors strapped to our bodies, probes in deep space…humans are being connected to anything and everything that can provide data. As AI agents are deployed to process this contextual information, reason, exchange data with other systems, make decisions, and act upon them in a fully autonomous way, they will generate a massive influx of data that will drive machine-to-machine communications. 

With the massive challenges AI is creating within data centers around the world, the network can be a cloud’s best-kept secret.  

That’s because connectivity is vital to how a data center functions – both inside the facility and in the wide area. 

Given that the network will evolve in tandem with the cloud, the transition of data centers from a centralized model out to the far edge has significant implications. Beyond the additional network capacity required to enable new use cases and AI-based applications, the network architecture needs a rethink.

Extreme speed, reliability and security will be crucial to this highly distributed, massively interconnected infrastructure to support the business and mission-critical applications that run on top of it. 

Architecting for increased scale will matter, as AI applications trigger a cascade of data requests and responses, leading to rapid traffic bursts that can overwhelm existing networks. 

Network responsiveness will also be key. Robust and reliable automation will allow for dynamic adaptation as demands evolve. This is a contrast to traditional networks, which often rely on manual configurations and struggle to prioritize and allocate resources. 

Network the cloud

Much like how video changed network architecture twenty years ago, AI is driving an evolution in how we think about the cloud and the network. Even with a rigorous focus on compute, power and energy, the network can emerge as a vital component of a data center’s success. Evolving the network to meet the fresh demands of AI requires a visionary approach to networking within the data center itself and between distributed data centers.

By evolving the network in tandem with the cloud, data center operators will have the foundation for a seamless and efficient continuum – one that can respond to whatever happens next in the age of AI.