Evolving data centres: the transition to 40GBASE-T

For many years, Ethernet over twisted pair, in conjunction with RJ-45 connectivity, has been a widely accepted and cost-effective networking option in both data centre and commercial building environments. However, with bandwidth requirements escalating so rapidly, some commentators in the IT industry are asking how long this technology can continue to adapt to new demands.

The pressure on data centre capacity has never been greater. It is now commonplace for 10 or more virtual machines to be running on the same physical server, making a high-speed uplink essential to ensure all these applications can run without excess latency or degradation of service. As more and more servers migrate to 10GbE links to support this data deluge, the IEEE forecast 40G uplink speeds to be a requirement for server interfaces by 2018. While 40G transmission speeds can certainly be achieved with current technologies, such as fibre or twinaxial (Twinax) cabling, these solutions are in many cases not ideal.

Fibre transceivers are expensive, and Twinax solutions are limited to short distances of seven metres, making them a poor choice for interconnecting devices located more than one or two racks apart. Twinax cabling is also often a proprietary technology designed to fulfil a specific function in a data centre, making it inflexible in meeting new demands.

Historically, the most cost-effective solution for data centre connectivity has been Ethernet over twisted pair cabling. This is the preferred way of connecting servers to switches, where ubiquity and ease of use is crucial. Thus research into 40GBASE-T was started to create a more cost-effective alternative to existing fibre or Twinax solutions. The adoption of 40GBASE-T in data centres will bring huge advantages to enterprises, opening up fresh possibilities for connectivity, supporting a structured cable design and a host of new data-intensive applications. Lower latency connections also provide additional benefits, such as facilitating the movement of virtual machines around the data centre. 40GBASE-T is likely to primarily be used in data centres for server access.

Twisted pair cabling with the RJ-45 connector has always been the technology of choice for IT professionals, based on its low cost and ease of use. IT professionals are also hugely experienced with this technology, since it has been an industry standard since the 1980s. This means that specialised data centre personnel do not need to be retrained to install or maintain 40GBASE-T infrastructure, significantly reducing the time and costs associated with deploying the technology.

Twisted pair is also unique in being able to seamlessly support a range of different Ethernet speeds through auto-negotiation. Unlike fibre or Twinax solutions, twisted pair cabling can automatically switch to different data rates, such as from 100MbE to 10GbE. This ability makes twisted pair extremely versatile and allows data centre managers to gradually and cost-effectively evolve their networks.

For example, an IT manager could install a 40GBASE-T switch at the end of a data centre row to provide connectivity to a handful of servers that are highly virtualised and require high-speed connections. Meanwhile the remaining ports on the switch can be connected to existing 1000BASE-T or 10GBASE-T servers, which could be upgraded as needed over time. This means that the transition to 40GBASE-T does not require a forklift upgrade of equipment. Instead it can take place in an evolutionary manner in response to tangible business needs, since servers and switches are not always refreshed at the same time.

Finally, deploying 40GBASE-T connections will also reduce the overall operational expenditure (OpEx) of a data centre by alleviating the need to aggregate multiple 10GBASE-T links to achieve 40GbE data rates. By reducing the amount of links in a data centre, IT managers can also improve energy efficiency and thus lower the total cost of ownership (TCO).

It has been proposed in some corners that Category 7A cabling offers future-proofed support for 40GbE speeds. However, Category 7A is only specified to a frequency of 1GHz and PHY industry experts have shown that higher bandwidth, in the range of 1.5-2GHz, will be required to support 40GBASE-T transmission. To date, the work of standards bodies indicates that a higher bandwidth solution than Category 7A cabling is needed in support of 40GBASE-T.

Additionally, Category 7A supports three different types of connectors that are not interoperable or backwards compatible with the RJ-45 connector. Their use in DCs will require the use of hybrid patch cords and equipment cords making operations more difficult, error prone and slowing down MACs (moves, adds, changes) in data centres.

The IEEE has made it clear that it requires a low-bulk, high-density cabling solution for 40GBASE-T, given the limited real estate in data centres. Due to these factors, the wider cabling industry has agreed that Category 8 is the path forward for supporting 40GBASE-T. This is the future of cabling as we know it.