Europe's Fastest LLM Inference

Infercom serves open-weight models such as gpt-oss-120b on SambaNova dataflow hardware in Munich. Our highest measured output is 713 tokens per second, with peaks up to 772 tok/s on shorter prompts - up to 10x faster than GPU-based alternatives. Every figure is a server-side p50 from our production API, not a vendor estimate, and you can reproduce it with our open-source benchmark tool.

gpt-oss-120b leads output throughput at 713 tok/s and also has our lowest time to first token at 388 ms, which makes it the default for chat, voice and high-volume agent loops. MiniMax M2.7 Ultraspeed is our 229B frontier reasoning model at 428 tok/s with a 192K context window, for long agentic runs and hard coding tasks. Gemma 4 31B measures 199 tok/s and adds native image and text input. Chat and voice depend on time to first token; agents and coding depend on throughput.

Yes. Every number is a server-side p50 at 10K input and 1K output tokens, single request, measured against our production API with our open-source benchmark tool. Run it online against our API with your own key, or clone the repository and run it locally against your own prompts. Your client-side results will differ depending on network location and conditions.

Dataflow hardware keeps its compute pipeline busy at low batch sizes, so per-request speed does not depend on stacking many requests together the way GPU serving does. Batching still raises total system throughput, but it is not what buys the per-request numbers above. The figures published here are single-request p50s; if concurrency behaviour matters for your workload, the benchmark tool has a real-workload mode that replays variable request rates against our API.

The dataflow architecture Infercom runs on is continuously benchmarked by Artificial Analysis and has been covered by VentureBeat and TechRadar, with the underlying energy-efficiency methodology validated by Stanford Hazy Research. Those sources cover the technology; the per-model numbers on this page are our own measurements on our own Munich infrastructure, which is why we publish the benchmark tool so you can check them.

A GPU repeatedly moves model weights and activations between compute and memory, and during token-by-token decoding it spends much of its time waiting on that memory traffic. A dataflow architecture maps the model onto the chip and streams data through it, so far more of the silicon does useful work per token. That is where the up to 10x speed advantage over GPU-based inference comes from, alongside up to 5x better energy efficiency.