Engineering @ Scale

Signal of the Day

Google’s open-sourcing of HEIR demonstrates that complex homomorphic encryption compile pipelines can be simplified into a “one-click” toolchain, enabling pre-trained AI models to perform inference on encrypted inputs. This compilation model shifts the burden of cryptographic implementation away from software developers, making secure computation a scalable development capability.

Deep Dives

DRAM Controller Register Manipulation Breaks CPU Memory Isolation · Modern CPU Architectures · InfoQ In modern cloud and confidential computing environments, hardware-level memory isolation is critical to keep unprivileged tenant code from accessing protected host memory regions. Security researcher Christopher Domas developed “skitter-creek-bath-salts,” an open-source hardware security tool that disrupts CPU privilege boundaries by manipulating memory controller translation registers. By bypassing logical OS and hypervisor protections, this direct register manipulation allows unprivileged software to access protected memory regions. This exploit serves as a stark warning to infrastructure teams that logical virtualization and sandboxing are ultimately bound to physical processor architecture security.

OVHcloud Raises Prices as AI Memory Demand Reprices Non-AI Infrastructure · OVHcloud · InfoQ Infrastructure and platform engineering teams face severe capacity planning constraints as the global explosion in AI workloads spikes costs for standard, non-AI hardware. Facing DRAM costs that were six times higher in June than the previous year due to RAM suppliers shifting capacity to High-Bandwidth Memory (HBM) for AI, OVHcloud decided to raise server prices starting in September. These hikes are dramatic, with 2026-edition gaming servers increasing by 87% and other recent server models rising by 40% to 59%. While larger hyperscalers like AWS hedge this supply-chain volatility by purchasing inventory years in advance—repricing only a single reserved GPU product so far—smaller infrastructure providers are forced to pass these market fluctuations directly to customers.

Google’s HEIR Aims to Make Homomorphic-Encrypted Inference a One-Click Capability · Google · InfoQ Deploying machine learning models that can process encrypted user data has traditionally required manual, highly complex mathematical implementations of homomorphic encryption algorithms. To lower this entry barrier at scale, Google introduced HEIR (Homomorphic Encryption Intermediate Representation), an open-source compiler and development toolchain designed to make encrypted computation easier to deploy. HEIR’s core mechanism lies in its ability to compile pre-trained AI models built for conventional, unencrypted inputs so that they can operate directly on encrypted data. For engineering teams handling highly regulated data, HEIR demonstrates that complex security and privacy protocols can be integrated directly into developer workflows through smart compiler-level abstractions.

Cloudflare OS: Cloudflare’s Open-Source Corporate AI Platform Built on a Capability-Based Model · Cloudflare · InfoQ Deploying AI agents across an enterprise requires balancing deep data utilization with strict sandboxing and budget-friendly token management. To address this security-cost dilemma, Cloudflare open-sourced Cloudflare OS, a secure platform built on a capability-based, sandboxed model. The platform allows enterprise teams to generate work artifacts grounded in internal company knowledge and provisioned connectors, while building customizable work software for specific, complex use cases. Crucially, the architecture optimizes operational expenses by automating repetitive workflows with optimized token costs, introducing AI assistance only when strictly necessary.

Patterns Across Companies

This period highlights a converging focus on managing the dual pressures of security sandboxing and physical infrastructure constraints under the weight of AI growth. While Google and Cloudflare are building open-source, high-level software abstractions to secure AI inferences and workflows, hardware-level realities—from physical memory isolation exploits to soaring DRAM market costs—underscore that software scaling is ultimately bound by physical infrastructure security and supply chain economics.

🔍 I can research on the web how other major cloud providers are responding to the High-Bandwidth Memory (HBM) supply crunch and whether similar infrastructure price hikes are expected across the industry, and you can choose which findings to import.