Google Open-Sources HEIR for Encrypted AI Inference

Google Open-Sourced a Compiler That Runs AI Models on Data It Cannot Read

HEIR converts pre-trained models to run inference directly on homomorphically encrypted inputs. Four reference applications ship with it, and none of them are a hosted product.

Aug 16, 2026Source: Google

Google published HEIR — Homomorphic Encryption Intermediate Representation — on 14 August: an open-source compiler toolchain and development platform that takes a pre-trained model and emits a version that runs inference on homomorphically encrypted inputs.

What that means in practice

Fully homomorphic encryption allows computation on ciphertext. Applied to inference, the server executes the model without ever decrypting the input and returns an encrypted result only the client can open. The operator learns nothing about the query or the answer — which removes the single largest objection to sending medical, financial or biometric data to someone else's model.

What actually shipped

Four compiled reference applications, each with single-threaded CPU latency measured and published: a deep-learning recommendation model built with Belfort Labs, LG and NYU; credit-card fraud detection with Niobium and hardshell.ai; network intrusion detection on the Kitsune system with Niobium; and hotword detection for audio privacy with Belfort Labs. Source for all four is in Google's fully-homomorphic-encryption GitHub repository.

What it is not

This is a developer toolchain, not a managed Google Cloud service. There is no SLA, no hosted endpoint and nothing to buy. "Google launches encrypted AI service" would be wrong. The release is aimed at people who will compile and run this themselves.

The number nobody published

Google's post does not quantify the speedup against prior FHE work — it says only that the cost of homomorphic encryption is falling rapidly. That omission is the story's honest limit. FHE inference remains orders of magnitude slower than plaintext inference, and the fact that the reference latencies are single-threaded CPU measurements tells you which end of the performance range this technology still occupies.

Why publish it now

Compilers are how a cryptographic technique stops being a research artefact. Hand-writing an FHE circuit for a neural network is specialist work measured in months; a compiler turns it into a build step. Google pairing the release with named academic and startup collaborators, rather than a product page, signals it expects the ecosystem to close the performance gap rather than Google alone.

Why It Matters

Private inference is the unlock for regulated data that currently cannot leave a hospital or a bank at all, and shipping a compiler rather than a service is what moves homomorphic encryption from cryptography papers into ordinary build pipelines.