The race for faster AI inference is on. Cerebras’ purpose-built chips earned a warm market reception with the company’s IPO debut in May, but French startup Kog is betting that conventional GPUs still hold far more performance in reserve.
Kog’s technical preview reached the front page of Hacker News in May, with the goal of demonstrating that “extremely fast single-request decoding is possible on the standard datacenter GPUs enterprises already own.” The demo ran on AMD MI300X and Nvidia H200 models, two of the data center GPUs most commonly found in enterprise environments.
The company is taking a different approach from Cerebras, which relies on custom silicon designed specifically for AI workloads. Instead, Kog believes there is significant untapped potential in the hardware that companies have already deployed, and that smarter optimization can unlock faster inference without requiring new or specialized infrastructure.
Kog’s bet is that the next leap in inference performance will come not from a new generation of chips, but from going deeper into the ones already in data centers.