https://www.techradar.com/news/nvidia-builds-fastest-single-computer-humanity-has-ever-created

Nvidia has just announced its HGX-2 cloud server platform, which it claims powers "the fastest single computer humanity has ever created".

It combines 16 Tesla V100 graphics cards, which work together to create a giant virtual GPU with half a terabyte of GPU memory and two petaflops of compute power. This is achieved using Nvidia NVSwitch interconnect fabric technology, which links the GPUs together to work as a single GPU.