Nvidia's grip on AI compute just loosened. Not because a rival chip dethroned the H100. Because a 20-person team in Paris wrote software that makes the hardware irrelevant.

ZML dropped LLMD this week — a free inference server that runs Llama, Mistral, and other open-weight models across Nvidia GPUs, AMD Instinct, Google TPUs, Apple Metal, and Intel Arc. No vendor lock-in. No CUDA tax. Peak performance on each architecture, sometimes faster than the vendor's own stack.

That is the headline. The subtext is sharper.

The inference pivot is real

Training gets the press. Inference pays the bills. Every prompt processed, every token generated, every chatbot response — that is inference. And it is eating the world. Morin told me the industry has already crossed the threshold: inference optimization now outranks model training in strategic importance. Yet the software layer remains a patchwork of proprietary kernels, half-baked abstractions, and CUDA-or-bust assumptions.

LLMD attacks that mess directly. One server. One API. Heterogeneous silicon as a feature, not a bug.

Europe's chipmakers just got a lifeline

Morin rattled off names: Axelera, Fractile, Kalray, OLIX, Q.ANT, SiPearl, SpiNNcloud, VSORA. European silicon startups, nearly all of them. They share a problem — brilliant architectures, zero software ecosystem. Nvidia's moat was never just the chip. It was cuDNN, TensorRT, the entire stack that made the chip usable out of the box.

ZML changes that calculus. If LLMD supports your accelerator at launch, you have a path to customers. Morin calls it co-designing silicon. That is not marketing fluff. His team works with chipmakers pre-tapeout, mapping kernels to instruction sets before the first wafer ships. That is how you compress a two-year bring-up into six months.

Not anti-Nvidia. Post-Nvidia.

Morin refuses the bearish label. He speaks of Nvidia with respect — supply scale, roadmap execution, a genuine partnership. But he also describes a world where enterprises blend H100s for training, AMD MI300X for batch inference, Apple Metal for edge deployment, and a European ASIC for specialized workloads. All managed by one control plane.

That world terrifies the lock-in business model. It also terrifies the venture-backed inference startups racing to build the next abstraction layer.

The competition is flush and fierce

Baseten just landed a $13 billion valuation. Inferact commercializes vLLM, the open-source engine that powers half the internet's LLM serving. RadixArk backs SGLang, the other half. Both projects are excellent. Both are narrower — GPU-first, Nvidia-optimized, community-driven.

ZML's bet: breadth wins. When a bank wants to deploy on-prem with a mix of Intel Gaudi and AMD cards. When a telco needs edge inference on Qualcomm Snapdragon. When a sovereign cloud mandates European silicon. LLMD answers yes. The others answer "file an issue."

Speed as a byproduct of leanness

Twenty engineers. $20 million raised. That is a ratio most VCs would call dangerous. Morin calls it velocity. His track record — VP Engineering at Zenly, sold to Snap for nine figures — bought him trust. Investors include 20VC, Xavier Niel's Kima Ventures, Kindred, LocalGlobe. No board seats. No growth-at-all-costs mandates.

The team ships. They shipped a compiler. They shipped a runtime. They shipped LLMD. Next quarter: a kernel generator that writes optimized CUDA, HIP, Metal, and SPIR-V from a single IR. The scope expands while headcount stays flat.

The free tier is the wedge

LLMD is free. Apache 2.0. No telemetry, no phone home, no enterprise upsell locked behind a license key. Morin believes the standard wins by adoption, not extraction. The business model comes later — managed service, support contracts, custom silicon engagements.

That patience is rare. Most inference startups sprint toward revenue, locking features behind paid tiers, fragmenting the community. ZML's approach feels like Redis in 2010: give away the engine, sell the expertise.

Skepticism warranted

Hardware diversity is a maintenance nightmare. Every new chip vendor brings a new compiler backend, a new memory model, new bugs. Supporting six targets is hard. Supporting twenty — the implied roadmap — is a combinatorial explosion. Morin acknowledges this. His answer: automated kernel generation, not hand-tuning. The compiler writes the backend. Humans verify.

If that works, ZML becomes the Linux of inference. If it fails, they become another ambitious compiler project that bit off more than it could chew.

The real disruption is economic

AI costs are spiraling. Not training — inference. A single H100 hour costs $2–4 on major clouds. Multiply by millions of daily active users. The CFO notices. The ability to shift workloads to cheaper silicon — AMD at 60% the price, Intel Gaudi at 40%, custom ASICs at 20% — changes unit economics fundamentally.

Morin puts it plainly: give people the power to build their own system. Real efficiency gains. Dissemination at scale.

That is not a software feature. That is a market restructuring.

Watch the next release

ZML moves quarterly. The kernel generator arrives soon. Then a distributed runtime for multi-node, multi-vendor clusters. Then the silicon tape-outs they have been co-designing since day one.

Nvidia will not collapse. But its software moat just sprang a leak. The water is rising. Every chipmaker with a datasheet and a dream now has a path to production. The inference gold rush just got a new map — and it does not lead exclusively to Santa Clara.