Glonce

Google releases HEIR, an open source compiler for private AI

securityThe Glonce editorsHacker News🔥81Glonce rating81Impact70Novelty70Relevance95Surprise60Players75Credibility85
Google releases HEIR, an open source compiler for private AI

Google has released HEIR, an open source compiler toolchain that is now part of its Private Computing Toolkit. HEIR (Homomorphic Encryption Intermediate Representation) takes a pretrained AI model that normally runs on unencrypted data and converts it to run inference on homomorphically encrypted inputs instead, so a server can process ciphertext and return an encrypted result without ever seeing the underlying data. Google frames this as a way around a long standing trade off: standard end to end encryption keeps user data safe from breaches, but it also stops a service provider from running features that need to read that data, such as spam or virus detection, and sectors like healthcare and finance face strict rules that limit sharing data across institutions in the first place. Homomorphic encryption removes that trade off at the level of capability, Google says, and turns it into a question of cost, a cost it says is falling rapidly. Google adds that homomorphic encryption's security guarantees are purely cryptographic, unlike hardware based approaches such as secure enclaves, but that manually converting an existing program to use it efficiently normally requires a team of cryptographers, which is the gap HEIR is meant to close. Google says it announced its intentions for the HEIR project in 2023, and that the homomorphic encryption community has since built on it: four peer reviewed publications have used HEIR so far, with more in preparation, and the project has drawn collaborations with Georgia Tech, Carnegie Mellon, UC Santa Barbara, the Illinois Institute of Technology, Purdue, the University of Edinburgh, Tsinghua University and others. Google has also partnered with companies building hardware accelerators for homomorphic encryption, including Belfort, Niobium, Cornami and Optalysys, and says it plans to demonstrate the latency benefits of those accelerators in the near future. To show what the compiler can do today, Google is sharing four private inference applications, each compiled with HEIR and each with source code published on GitHub, with latency numbers presented for a single threaded CPU. The first is a deep learning recommendation model built with Belfort Labs, LG and New York University that can serve content recommendations without the provider seeing the user's underlying features. The second, built with Niobium and hardshell.ai, is a credit card fraud detector. The third, built with Niobium, compiles the Kitsune anomaly detection system so a service provider can flag suspicious network traffic without reading the contents of the packets. The fourth, built with Belfort Labs, is a hotword detector that Google says could let an audio triggered AI agent recognize wake words while keeping the rest of the recording private. Google's stated long term goal is to turn HEIR into a one click solution that lets non experts add encrypted inference to production applications without needing cryptography expertise themselves.

Key facts

  • HEIR is Google's open source compiler that converts pretrained AI models to run inference on homomorphically encrypted data; Google says it announced its intentions for the project in 2023.
  • Four peer reviewed publications have been built on HEIR to date, with more in preparation, and it has drawn academic collaborators including Georgia Tech, Carnegie Mellon, UC Santa Barbara, Illinois Institute of Technology, Purdue, the University of Edinburgh and Tsinghua University.
  • Google is sharing four HEIR compiled demo applications with source code on GitHub: a recommendation model with Belfort Labs, LG and New York University; a credit card fraud detector with Niobium and hardshell.ai; a Kitsune based network threat detector with Niobium; and a hotword detector with Belfort Labs.
  • Google partners with hardware accelerator makers Belfort, Niobium, Cornami and Optalysys, and plans to demonstrate the accelerators' latency benefits in the near future.
  • Google's stated goal is for HEIR to become a one click solution that lets non experts add encrypted inference to production applications without a dedicated team of cryptographers.

Why it matters

Homomorphic encryption lets a server compute directly on encrypted data and hand back an encrypted result without ever seeing the plaintext. Google frames this as a way past a real trade off: standard end to end encryption protects user data but also blocks features that need to read it, like spam or virus detection, and regulated sectors such as healthcare and finance already restrict how data can be shared between systems. Homomorphic encryption keeps the protection while still allowing computation, turning what used to be a hard capability limit into a question of processing cost, which Google says is falling quickly.

Who it affects

The people most affected are companies that want to offer AI features on sensitive data without holding that data in the clear, in fields like healthcare, finance and content recommendation, plus the hardware accelerator makers Google names as partners, Belfort, Niobium, Cornami and Optalysys. It also affects the academic side of the field: Google says HEIR has become a shared research platform, crediting collaborations with Georgia Tech, Carnegie Mellon, UC Santa Barbara, Illinois Institute of Technology, Purdue, the University of Edinburgh and Tsinghua University, on top of four peer reviewed publications built on it so far.

How to use it

HEIR is open source, and the source code for all four demo applications described in the post, a recommendation model, a fraud detector, a network threat detector and a hotword detector, is published on Google's GitHub repository for developers to inspect or build on. Google states its longer term goal is to make HEIR a one click solution so non experts can add encrypted inference to production applications, but the post gives no release date or roadmap for reaching that point, and mentions no pricing since HEIR is offered as an open source project rather than a paid product.

How solid is it

The claims are backed by concrete, named work rather than a bare announcement: four peer reviewed publications, a named list of university collaborators, and four working demo applications built jointly with named industry partners, Belfort Labs, LG, New York University, Niobium and hardshell.ai. Google also says it measured latency for each demo on a single threaded CPU, and the project traces back to intentions Google says it announced in 2023, indicating multi year development rather than a one off release.

Risks and caveats

The post says latency numbers are presented for a single threaded CPU but gives no throughput or cost figures for any of the four demos, so the real world performance overhead of running these models under encryption is not fully disclosed. It also does not say whether the recommendation model, fraud detector, threat detector or hotword detector are running in production anywhere or remain research demonstrations. There is no timeline for turning HEIR into the stated one click, non expert tool, and Google's own framing acknowledges that homomorphic encryption still carries a nontrivial cost overhead even as that cost falls.