France's ZML Launches LLMD, a Free Server That Runs LLMs on Any Chip

The Paris startup, backed by Yann LeCun, wants to break Nvidia's software lock-in with an inference server that runs open-weight models across five chip families from one stack — free, though its alpha is narrower than the pitch.

Jul 9, 2026Source: TechCrunch

France's ZML Launches LLMD, a Free Server That Runs LLMs on Any Chip

A French startup betting against Nvidia's software moat has shipped a tool meant to run AI models almost anywhere. ZML, a Paris company, released LLMD on July 8 — a free LLM inference server designed to run open-weight models across sharply different chips from a single stack. The goal, founder Steeve Morin says, is to "break existing silos" and end the vendor lock-in that ties AI workloads to one hardware maker.

Any model, any hardware

LLMD ships supporting five architectures out of the box: Nvidia CUDA, AMD ROCm, Google TPU, Intel and Apple Metal. "The idea is to give people back the power to create their own system and achieve real efficiency gains," Morin told TechCrunch, adding that ZML has "reached the point where we are co-designing silicon." The company's tagline is blunt: "Any model. Any hardware. Zero compromise."

How it's built

LLMD is built in the Zig programming language on top of MLIR and OpenXLA, deliberately sidestepping the Python and PyTorch dependency chain. It leans on explicit, ahead-of-time compilation so latencies stay "flat and predictable," with no hidden runtime compilation. The server sits atop ZML's open-source inference framework, first released in September 2024 and updated to v2 in March 2026 under an Apache-2.0 license.

Free, but not open

Here's the twist: LLMD is free but not open source — a departure from the framework beneath it. Morin's reasoning is commercial patience: "I'd rather measure and [generate revenue] where it is most effective without hindering my growth stupidly." Learn how people use it first, monetize later.

A reality check

The launch pitch runs ahead of the shipped code. ZML advertises tensor-parallel sharding, prefix caching and broad model support, but an independent hands-on test published on Hugging Face found the current alpha far narrower: single-GPU only, a max batch size of 16, no prefix caching, and support limited to Llama and Qwen3 model types. No hard throughput or latency numbers have been published; the "sometimes faster than native" speed claim remains qualitative. The advertised features read, for now, as roadmap.

Who's behind it

Founded in 2023 with a team of about 20, ZML is led by Morin, a former VP of engineering at Zenly, the location-sharing app Snap acquired in 2017. The company has raised $20 million, led by Harry Stebbings' 20VC with Xavier Niel's Kima Ventures, LocalGlobe and Kindred Capital, and a marquee angel list: Turing laureate Yann LeCun, Docker founder Solomon Hykes and Hugging Face's co-founders. "I couldn't do ZML anywhere but in Paris," Morin said. It competes with a crowded inference field that includes the $13 billion Baseten and the teams behind vLLM and SGLang.

Why It Matters

Nvidia's deepest moat isn't its chips but CUDA, the software layer rivals can't easily match — and portable inference stacks like ZML, Modular and Apple's MLX are aimed squarely at eroding it. If buyers can run any open-weight model on any silicon at full speed, it loosens Nvidia's grip and reshapes inference economics, the fastest-growing line in AI budgets. The gap between LLMD's pitch and its alpha is the catch: cross-hardware parity is exactly the hard part everyone keeps underestimating.

Filed underAI Chips AI Tools #AI inference #Nvidia #ZML #open source