Crack open a firewall, load-balancer or storage array and on the motherboard you might find a chip named “Octeon” by Marvell.
Octeon’s job is giving appliance-builders a chip that handles all the messy bits of moving data securely across networks so they can focus on building brilliant firewalls or storage array. The chips can scale to 16 more cores and are programmable: firewall vendors can make them dance to their tune and array-builders can tweak them to handle the way disks spew out data.
The Register offers that short history lesson because it’s about to be re-told quite a lot thanks to another piece of hardware in which Octeon is at home: a network interface controller (NIC).
A few years back NICs got smart: vendors started adding some decently-specced compute cores and a little storage so they could do the same things Octeons do, namely offer a platform to handle work a device’s main engine is better off not doing. These beefed-up NICs came to be known as SmartNICs.
Hyperscale clouds liked the look of SmartNICs because they make money renting CPU cores, but some of those cores were busy running networking and security. Many servers in clouds host multiple VMs and customers, making cloud networking and security rather complex. Hyperscalers therefore liked to the idea of offloading that work into SmartNICs, both to free up CPU cores and to further isolate customer workloads from other tenants and the perils of the wider internet.
Oracle reckons it was first to use SmartNICs in its second-generation cloud, which may be true although Amazon Web Services’ “Nitro” was announced a few months before Big Red’s efforts became public.
Cloud-measuring contests aside, SmartNICs have now become standard for hyperscalers. Alibaba, and Baidu are known users, while Google is under suspicion of having them under the hood and Microsoft does similar things but with FPGAs.
Because they work to move data, SmartNICs are now being described as data-processing units – “DPUs” – and being advanced as essential for demanding workloads like AI.
“The DPU is really good at looking inside data and running storage and compression and security,” NVIDIA’s veep of marketing Kevin Deierling told The Register. Being good at that matters because users whose data-crunching requirements are so urgent that latency caused by clouds cruels their efforts make a big investment in RAM-rich servers and GPUs. Anything that leaves that hardware free to do AI, instead of running I/O, is therefore welcome.
5G is another likely use-case, as the new protocol assumes that network functions will be pushed into software, and running such code on the hardware that handles networking is useful. By adding co-processors to servers SmartNICs also improve compute density to devices that run in the space-constrained places like base stations and rooftops where 5G network hardware operates.
Deierling also points out that SmartNICs are handy because firewalls on servers are becoming important because applications like AI mean more data is now travelling East/West (between racks) than North/South (up and down a rack.”
“Traditional firewalls on the perimeter no longer are adequate,” he argues. “You need security to match what the accelerated distributed aps look like.”
Note his use of the term “accelerated apps” because Deierling assumes the presence of a GPU, and suggests its cores and memory also deserve the protection of DPU offload.
NVIDIA even thinks GPUs can benefit from being melded with a DPU. The company’s forthcoming EGX platform does just that.
Running a combined GPU/DPU sounds like fun in many situations.
Putting one to work is not. NVIDIA has an SDK and provides its own software to drive DPUs. Marvell makes sure its kit is ready to work with the Data Plane Development Kit, an effort founded by Intel, overseen by the Linux Foundation and operating with an aim “to accelerate packet processing workloads running on a wide variety of CPU architectures.”
John Sakamoto, veep of Marvell’s VP infrastructure business unit, told The Register he sees those who need to create custom code writing to the DPDK spec when they need the functionality of a DPU.
But while hyperscalers, appliance-builders and serious AI adopters are happy cutting networking code, most users are quite rightly far happier with off-the-shelf product.
If you fancy running a firewall in a SmartNIC/DPU, instead of using appliances or running software firewalls, your preferred vendor almost certainly has nothing that will run on the accelerators.
Analyst firm Gartner’s 2020 Hype Cycle for Enterprise Networking therefore rates SmartNICs – which it calls “function accelerator cards” – as currently applicable to “Less than 1% of target audience”.
But The Register has written this story because change is in the wind. VMware has demonstrated its ESXi hypervisor on SmartNICs. As mentioned above, NVIDIA, is bolting one to a forthcoming GPU. Marvell has news up its sleeve. And Arm just created a DPU design just to accelerate storage.
And in not too many years some of that news will offer new ways to consider running almost any data centre. ®