Engineering @ Scale

Signal of the Day

Airbnb’s server-driven “Flexible Authentication” architecture, which cut client-side authentication code by 60% and slashed web bundle sizes by 100 KB, demonstrates how centralizing complex orchestration flows back to the server solves client-side drift and optimizes performance across multi-platform client systems.

Deep Dives

Twenty Years of jQuery: How a Little Library Rewired Web Development · jQuery · InfoQ jQuery originally solved the fragmentation of early web development by abstracting inconsistent browser APIs and manual DOM manipulation into a single, unified client-side library. Its architecture was built around chains of fluent interfaces and an extensible selector engine that made cross-browser JavaScript highly accessible. While modern reactive frameworks have reduced its active adoption for new applications, jQuery’s lightweight footprint and backward compatibility keep it running on a massive portion of the live web today. The key lesson is that developer ergonomics and stable, predictable APIs can secure a technology’s production relevance for decades, outlasting major shifts in architectural patterns.

Tether: Apple Continuity Like Experience Between iOS and Linux Desktop Machines · Tether · InfoQ Integrating services like shared clipboards, notifications, and instant messaging across Apple iOS and Linux platforms requires overcoming deep operating system silos and proprietary hardware integrations. Zack Bartel’s Tether achieves this cross-platform compatibility by employing secure local network communications coupled with a custom Bluetooth stack. By bypassing cloud brokers in favor of localized peer-to-peer syncing, the tool minimizes latency and ensures robust end-to-end data security. The architectural tradeoff is that implementing a custom Bluetooth stack introduces complex hardware-compatibility edge cases that must be managed on the client side. This project demonstrates that local-first synchronization patterns can deliver seamless, high-performance integration without the cost and security risks of cloud-based intermediaries.

Podcast: Personality Over Skillset: How Adam Wachtel Builds Engineering Teams · Click Boarding · InfoQ When software platforms hit architectural crises, engineering organizations often fail by hiring exclusively for static, highly specialized technical skill sets. Click Boarding CTO Adam Wachtel argues that successful platform transformations require restructuring teams around adaptive cognitive profiles, problem-solving habits, and soft skills. While vetting for personality takes longer in the hiring pipeline, it creates cohesive teams capable of shifting across tech stacks as architectures evolve. This approach also directly prepares teams for the AI-driven shift in software delivery, where individual engineers manage broader system behaviors rather than writing boilerplate. For tech leaders, the core lesson is that organizational resilience during a system rewrite depends more on cultural adaptability than on specific language mastery.

Copilot Code Review Reaches Azure Repos, Billed Per Review with Reporting Two Days Behind · Microsoft · InfoQ Teams with legacy constraints often cannot easily migrate from Azure DevOps to GitHub to access modern AI developer tooling. Microsoft addressed this block by bringing GitHub Copilot code review directly to Azure Repos, integrating model-driven PR analysis natively into legacy environments. The system uses a per-review billing model routed through Azure subscriptions, introducing a concurrency cap of five reviews per organization. However, because cost reporting runs 48 hours behind, and budget rules only trigger notifications rather than halting operations, teams face a risk of silent billing overruns. This decision highlights a major tradeoff in cloud system design: decoupling usage meters from real-time cost-control gates can lead to unpredictable operational spend.

Presentation: From S3 to GPU in One Copy: Rethinking Data Loading for ML Training · Linux Foundation · InfoQ ML training at scale is frequently bottlenecked by the CPU and storage I/O, where traditional data pipelines struggle to stream remote datasets fast enough to saturate high-end GPUs. The open-source Vortex columnar file format tackles this by bypassing conventional deep serialization stacks to stream data directly from S3 to GPUs at up to 60 Gbps. It leverages cascading lightweight encodings, layout-based segment pruning, and zero-copy memory pipelines to eliminate CPU translation overhead entirely. While this architecture demands a specialized file layout and upfront schema planning, it completely removes the need for expensive, time-consuming offline reprocessing. This shows that eliminating intermediate serialization hops is the most effective way to solve I/O bottlenecks in accelerator-heavy data paths.

Mini book: Next-Gen Architecture Playbook: Insights and Patterns for the AI Era · InfoQ · InfoQ The rise of stochastic models requires technical leaders to transition from designing highly deterministic systems to orchestrating how software behaves, learns, and fails. This playbook distills industry-tested practices for architects building applications around large language models and multi-agent systems. A recurring pattern is the expansion of the software architect’s responsibilities to encompass guardrail definition, prompt routing, and model observability. This shift introduces a tradeoff, requiring engineers to trade absolute execution predictability for cognitive flexibility and adaptive workflows. The takeaway is that future system architectures must be designed to guide and govern autonomous behaviors rather than strictly hardcoding logic paths.

Kubernetes Promotes KYAML as a Safer, More Consistent Way to Work with Manifests · Kubernetes · InfoQ Configuring large-scale distributed infrastructure using standard YAML is highly error-prone, often leading to silent typos, type-coercion issues, and production misconfigurations. To solve this, Kubernetes is promoting KYAML, a stricter dialect of YAML designed to make manifest definitions explicit and highly predictable. Enforcing strict typing and programmatic schemas prevents common parser ambiguities before manifests ever reach the cluster control plane. While adopting KYAML introduces syntax verbosity and increases configuration-time friction, it greatly reduces the blast radius of manual on-call errors. This emphasizes a core infrastructure principle: safety and predictability should be enforced at the configuration parsing layer, even at the cost of developer shorthand.

Airbnb Cuts Authentication Code by 60% with Server Driven Architecture · Airbnb · InfoQ Managing dynamic authentication flows across divergent web, iOS, and Android clients leads to major code duplication, high maintenance overhead, and security drift. Airbnb resolved this by building “Flexible Authentication,” a server-driven architecture that moves flow orchestration and challenge selection entirely to the server. This redesign eliminated 60% of authentication-related client code, reduced client web bundles by 100 KB, improved success rates by 2.6%, and lowered OTP costs by 11%. The architectural tradeoff is that pushing UI routing to the server introduces additional server-side state complexity and requires robust contract design between server and clients. For software architects, this proves that centralizing complex transactional flows back to the server is highly effective for simplifying multi-client codebases and optimizing performance.

How Intuit built an agentic disaster recovery assistant with Amazon Bedrock · Intuit · AWS Blog Coordinating regional failover across thousands of microservices at Intuit is technically solved by their internal EWOK orchestrator, but the actual decision-making under stress still relied heavily on human tribal knowledge. To bridge this gap, they built EWOK Agent, which wraps Bedrock foundation models with a thin, bounded reasoning loop over deterministic executors. Operational procedures are codified as “skills”—comprising YAML frontmatter for typed I/O schemas and markdown prompt bodies for strict decision logic. This architecture enforces a clean separation of concerns: the model decides what to do, while the executor deterministically carries out the API calls under strict IAM and guardrail policies. This pattern demonstrates that agentic infrastructure must treat the LLM as a stateless router and use conventional, audited code for all state-changing mutations.

Customizing your knowledge base on Amazon Bedrock for large and complex documents using Amazon Textract · AWS Professional Services · AWS Blog Implementing standard Retrieval-Augmented Generation (RAG) on highly structured documents like multi-page utility bills often fails due to dense tables, inconsistent layouts, and vector chunking limitations. AWS solved this by building an automated preprocessing pipeline that integrates Amazon Textract’s layout-aware text extraction with Bedrock Knowledge Bases. Uploaded documents trigger a series of Lambda functions that clean, parse, and format raw files into structured text before vector indexing. The tradeoff is that this multi-hop extraction pipeline increases initial document ingestion latency and compute costs compared to simple direct embedding. However, it completely eliminates hallucinations and incomplete extractions, showing that layout-aware parsing is a prerequisite for RAG quality on complex documents.

Run agent-driven Amazon SageMaker HyperPod operations with InstantStart · AWS · AWS Blog Deploying and operating large ML clusters on Amazon EKS requires coordinating multiple asynchronous AWS resources and Kubernetes operators, which can lead to high operational overhead and misconfigurations. The open-source HyperPod InstantStart control plane tackles this by providing a unified REST API fronted by both a web UI and an AI agent via Model Context Protocol (MCP). The agent leverages highly structured “skills” (playbooks in markdown) to safely orchestrate long-running, multi-stage workflows like cluster creation and storage mounts. A critical design choice is that the MCP tools wrap the backend REST APIs directly, ensuring that all security, validation, and idempotency guarantees protect both interfaces identically. This proves that control planes must be designed with clean API contracts first, allowing agents to act as a secure, predictable interface without fabricating low-level CLI commands.

Build a Physical AI model factory with NVIDIA Cosmos 3 on SageMaker HyperPod · AWS / NVIDIA · AWS Blog Physical AI pipelines (like robotics and autonomous vehicles) usually deploy separate, underutilized GPU clusters for synthetic data generation, policy training, and simulation evaluation. Using NVIDIA’s open omnimodal Cosmos 3 world model, this architecture schedules all three workloads onto a single persistent, time-shared GPU pool managed by SageMaker HyperPod with EKS. Training high-parameter Mixture-of-Transformers (MoT) designs with long sequence packing demands context parallelism on top of FSDP2 and low-latency EFA networking. The tradeoff is that co-locating these distinct I/O regimes on a single FSx for Lustre storage plane requires careful client-side tuning to balance bandwidth and metadata performance. This architecture illustrates that treating the physical AI loop as a unified, resilient substrate dramatically improves “goodput” (pipeline progress per reserved GPU-hour) while slashing data migration overhead.

Designing lifecycle policies for AgentCore memory · AWS Professional Services · AWS Blog Long-running conversational agents on Amazon Bedrock AgentCore naturally accumulate memory, but unbounded growth degrades retrieval performance, increases prompt token costs, and introduces compliance risks. This solution implements a nightly lifecycle pipeline using AWS Step Functions to manage memory through TTL-based expiration, relevance decay scoring, and LLM-based consolidation. The relevance scorer leverages CloudTrail logs to track memory access frequency and recency, applying an exponential decay formula to flag low-value memories. Battered memories are then consolidated into concise semantic facts using Bedrock models before the high-volume episodic records are safely pruned. This design shows that developer teams must treat LLM memory as a managed cache, enforcing active eviction policies to preserve context accuracy and comply with GDPR.

Deploy a multimodal WhatsApp ordering assistant with Amazon Bedrock AgentCore · AWS Enterprise Support · AWS Blog Quick-service restaurants often suffer from fragmented customer profiles because their ordering channels—web, app, voice notes, and calls—operate on completely isolated backends. This CDK-deployed solution unifies text, voice notes, and real-time WebRTC voice calls under a single WhatsApp number, backed by Bedrock AgentCore. An inbound API Gateway and SQS queue ingest messages instantly to process them asynchronously, and customer state is stored in a shared, cross-channel memory. While implementing WebRTC voice calls requires a complex NAT gateway and KVS TURN relay, it delivers true voice-in, voice-out processing with zero local transcription lag. This architecture demonstrates that decoupling conversational runtimes from backend business services via Model Context Protocol (MCP) enables high-performance omnimodal channels.

Project HydraFusion: Frontier quality via multi-model orchestration · GitHub · GitHub Blog Single-model code generation hits quality caps on complex repository-level tasks, while manual model switching is tedious and computationally expensive for developers. GitHub Copilot introduced Project HydraFusion, which orchestrates runtimes by dynamically choosing between Single, Cascade, and Critique execution patterns depending on task complexity. It leverages beam search over historical trajectories to build an optimal decision policy that routes drafting, review, and escalation tasks. This orchestration layer improved verified task quality on TerminalBench by 4.9 percentage points while slashing estimated costs by 67%. This proves that the next frontier of LLM application design is not relying on a single large model, but using runtime routing to coordinate specialized, cheaper models.

Web Excursions for September 4th, 2026 · Brett Terpstra · BrettTerpstra.com Personal and specialized workflows, such as local read-later archives, media storyboarding, or local software updates, are often over-engineered with heavy cloud dependencies. This curation highlights local-first, lightweight desktop and mobile applications designed to perform targeted tasks efficiently. For instance, Laytr offers iOS users a secure, private archive that stores reads locally without needing external servers or account databases. Similarly, Floodtide updates Mac applications locally through Sparkle, Homebrew, and GitHub without requiring recurring cloud subscriptions. This collection serves as a reminder that local-first application architectures remain highly effective for preserving privacy, eliminating server latency, and avoiding cloud lock-in.

GPT 6 Astra now available on Vercel AI Gateway · Vercel / OpenAI · Vercel Changelog Standard chat-based LLMs often fail in long-running agentic workflows because they struggle with complex state mutations and lose track of historical constraints. OpenAI designed GPT 6 Astra to address this by optimizing instruction following across evolving tasks, software navigation, and multi-step data simulation. When integrated through Vercel AI Gateway, developers can seamlessly connect the model to coding tools like Cursor while utilizing gateway features like zero data retention and unified usage tracking. The key architectural advantage is Astra’s ability to proceed with independent work and make reasonable assumptions while waiting for user clarification. This marks a major shift toward agent-native models designed explicitly to survive long-term planning loops without derailment.

Ling 3.0 Flash Sante is now available on AI Gateway for free · Vercel / inclusionAI · Vercel Changelog Executing highly specialized medical reasoning and clinical workflows on dense, general-purpose LLMs is computationally inefficient and expensive. inclusionAI designed Ling 3.0 Flash Sante as a medical-focused Mixture-of-Experts (MoE) model, boasting 124B total parameters with only 5.1B active per token to minimize inference costs. Available on Vercel AI Gateway, it features a 256K token context window and native function calling for evidence-based clinical retrieval. This architectural approach allows teams to deploy highly specialized clinical reasoning pipelines without paying the performance or cost penalties of fully dense models. It illustrates the power of sparse MoE routing to deliver domain-specific accuracy efficiently at scale.

Inside a Software Factory · Decoding AI · O’Reilly Radar Many teams building coding agents rely on “vibe coding” or overbuild complex “autonomous graphs” that quickly become un-debuggable monoliths when executions fail. To enforce software engineering rigor, Paul Iusztin built Squid, formalizing a “software factory” model structured across eight clear stages of the SDLC. The design splits the process cleanly: humans lead brainstorming and planning, while agents execute implementation, testing, and pull request reviews. A crucial lesson from his experiment is that a highly detailed human-engineered planning spec makes downstream execution by cheaper models significantly more robust and cost-effective. This demonstrates that AI-driven software development must be built on top of composable, interruptible local loops rather than fully autonomous, unalterable pipelines.

This Week in AI: The Frontier Is Getting Bigger · O’Reilly Radar · O’Reilly Radar The competitive landscape for AI is expanding beyond raw benchmark performance to encompass capital, infrastructure, custom silicon, and physics-based modalities. OpenAI’s custom Jalapeño inference chip delivered up to 1.9x more work per watt and 3.6x lower latency than comparable systems in testing, showing that custom silicon is a key differentiator. Concurrently, researchers from MIT and Tsinghua are pretraining models on physical particle interactions, establishing physics as a critical “third modality” alongside text and pixels. These movements signal that as transformer-based text scaling hits economic and data bottlenecks, the frontier is shifting toward hardware-software co-design and physics-grounded simulation. For system designers, the takeaway is that scalability will increasingly rely on domain-specific neural operators and co-designed accelerators.

Patterns Across Companies

Across these technical developments, we observe a strong convergence toward building agent-driven software factories and disaster recovery control planes that enforce a strict separation of concerns—confining LLMs to high-level reasoning and routing while relying on conventional, deterministic APIs for state execution. There is also a corresponding architectural divergence as teams either seek to bypass deep serialization stacks entirely for direct, high-throughput hardware access or build complex runtime orchestration systems that dynamically trade off cost and quality across multiple model providers.

💡 Which of these architectural patterns—like server-driven dynamic UI or the bounded agentic execution loop—would you like to drill down into for your team’s next system review?