Engineering @ Scale — 2026-07-13#

Signal of the Day#

Meta bypassed generalized Linux kernel schedulers to eliminate severe latency regressions by using sched_ext, an extensible BPF-based framework that allows user-space, workload-specific CPU partitioning. This architectural shift achieved a 28% latency reduction in their Ads service by keeping critical threads localized in L3 cache, proving that custom user-space scheduling yields massive scale returns without the overhead of maintaining kernel forks.

Deep Dives#

How to Build More Resilient Local-First Applications With AT Protocol Infrastructure · InfoQ Jake Lazaroff highlighted the AT Protocol’s potential as a backbone for distributed applications well beyond traditional social networking. The engineering focus is shifting toward a local-first architecture where users retain their data in Personal Data Servers (PDSs). This approach leverages shared infrastructure for state synchronization, drastically reducing the operational burden of maintaining app-specific backend services. For engineers, the tradeoff shifts complexity from traditional backend database management to building resilient client-side synchronization layers.

Podcast: Governance in the Age of AI: A Conversation with Sarah Wells · InfoQ Software architecture governance is often viewed as a bottleneck, but it is actually critical for minimizing system complexity at scale. Standardizing procedures actively improves security postures and eliminates repetitive operational tasks for engineering teams. By implementing targeted checklists, organizations can reduce the cognitive load and stress engineers face during complex deployments. Effective governance ultimately enables teams to work faster by providing safe, predefined paved roads for standard procedures.

Article: Removing a Hidden Round Trip from a Multi-Region AWS API · InfoQ When a team evaluated their multi-region AWS API for global failover capabilities, they uncovered a massive architectural blind spot. A pre-flight discovery call, which had been baked into every client session years prior, was creating a hidden network round trip. This legacy design choice acted as a hard blocker to seamless regional failover during AWS outages. Resolving it required systematically removing the dependency across all client sessions, demonstrating how legacy synchronous network calls can silently compromise modern multi-region resilience.

Presentation: Road to Compliance: Will Your Internal Users Hate Your Platform Team? · InfoQ Rolling out cloud infrastructure compliance often fractures relationships between platform teams and developers. At Sevdesk, the platform team pivoted to a “minimum viable governance” model on AWS to rebuild internal trust. They replaced rigid deployment enforcement with event-driven Slack alerting, which automated policy feedback directly to developers in real-time. This approach proves that shifting from hard gates to high-empathy, data-driven collaboration actually increases compliance adherence without blocking deployment velocity.

Java News Roundup: TornadoVM 5, JHipster, Google ADK, OmniFish Build of Payara, Introducing Vidocq · InfoQ The Java ecosystem saw several crucial updates, notably the general availability release of TornadoVM 5.0 for heterogeneous hardware acceleration. Point releases were also shipped for JHipster, Keycloak, and Google ADK to improve system stability. GraalVM Native Build Tools and Micronaut received maintenance releases, ensuring better native image compilation pathways. Additionally, Vidocq was introduced as a new implementation of the Jakarta EE 11 Core Profile and MicroProfile 7.1, expanding modern enterprise Java options.

How DoorDash Built an AI Shopping Assistant That Doesn’t Rely on the LLM Alone · InfoQ DoorDash engineered its “Ask DoorDash” conversational assistant to bypass the hallucination limitations of standalone LLMs. The architecture relies on an intelligence layer that orchestrates specialized AI agents using Model Context Protocol (MCP) tooling. It bridges the LLM with persistent consumer memory and live backend inventory data to ground responses in actual store reality. This memory-backed orchestration drove up to 24% higher checkout conversions, proving that RAG architectures must evolve into multi-agent, tool-connected pipelines for consumer applications.

The Path to Sovereign Data: Challenges and Priorities in Local-First Computing · InfoQ A recent engineering panel heavily critiqued current definitions of data ownership, noting that simple account control is insufficient for true sovereignty. True data ownership requires structural independence, broad system interoperability, and community-driven governance models. Panelists advocated for unbundled platforms and the strict adoption of shared data standards to prevent vendor lock-in. The overarching architectural lesson is that systems must provide users with exportable, standard-compliant data formats from day one to support genuine local-first computing.

Launching UI for generative AI inference recommendations in Amazon SageMaker AI · AWS Deploying generative AI to production traditionally requires a grueling cycle of manual benchmarking across instance types, containers, and optimizations. AWS launched an inference recommendations UI in SageMaker AI Studio to compress this iteration cycle from days to minutes. By selecting workload profiles like “Interact” or “Summarize”, the system automatically benchmarks options against goals like minimizing latency or maximizing throughput. This abstracts away complex hyperparameter tuning, allowing ML engineers to quickly find cost-performance optimized deployment configurations using actual GPU benchmarking under the hood.

Implement on-behalf-of token exchange for multi-tenant agents with Amazon Bedrock AgentCore Gateway · AWS Multi-tenant AI agents face the “confused deputy” problem: if an agent forwards a user’s token directly or impersonates the user, downstream systems must trust the agent unconditionally. To fix this, Amazon Bedrock AgentCore implements OAuth 2.0 Token Exchange (RFC 8693) to dynamically mint audience-bound tokens per tenant. The system preserves the user’s identity in the sub claim while rewriting the aud claim specifically for the downstream API. This enforces cryptographic least-privilege without requiring the agent to manage complex token-exchange logic itself.

Building an agentic AI solution at Bluesight with Amazon Bedrock · AWS Bluesight faced an immense data correlation problem for healthcare compliance, requiring over 4,000 manual audit hours to prove 340B Drug Pricing exceptions across 620 hospitals. They built “Prism”, a multi-agent solution on Bedrock AgentCore that orchestrates specialized data-worker agents to query live databases securely via Model Context Protocol (MCP). Instead of exposing raw data to the LLM, they wrapped APIs in Lambda functions to return agent-optimized structured data, dropping query latency from 5 minutes to 10 seconds. Crucially, the compliance determination remains a deterministic, auditable scoring pipeline rather than an LLM opinion, satisfying strict healthcare regulatory requirements.

When your brain works differently, AI isn’t a luxury—it’s accessibility · AWS For neurodivergent professionals, traditional productivity tools often fail due to competing cognitive needs for routine and novelty. One AWS solutions architect built an automated, local AI system using “Amazon Quick on your desktop” to fully offload executive function tasks like email triage and task state management. The architecture uses a custom MCP server connecting to Outlook and Asana, driven by configurable markdown rules rather than manual user interaction. The engineering lesson is that minimizing initiation cost to near zero allows AI to serve as a durable, invisible cognitive scaffold rather than just another abandoned workflow tool.

OpenAI GPT-5.6 Sol, Terra, and Luna are now generally available on Amazon Bedrock · AWS OpenAI’s latest GPT-5.6 model family has launched on Amazon Bedrock, introducing three capability tiers: Sol (flagship reasoning), Terra (balanced production), and Luna (fast/affordable). A massive optimization for agentic workloads is the introduction of explicit prompt caching breakpoints. This allows Bedrock to reuse processed context for shared system instructions or files, offering a 90% discount on cached inputs stored for up to 30 minutes. Security is enforced at the chip level via a zero-operator access (ZOA) model, ensuring enterprise data residency and preventing AWS operator access to prompts.

Building Service Topology at Scale: Architecture, Challenges, and Lessons Learned · Netflix Netflix needed a real-time service dependency map and opted for a streaming-first architecture over batch processing to ensure freshness during live incidents. They encountered catastrophic “hot node” problems when flow logs for highly popular services overwhelmed specific aggregation instances. To fix this, they shifted from a two-stage to a three-stage distributed pipeline, introducing graduated redistribution via dynamic consistent hashing to spread load evenly. They also migrated from gRPC to Server-Sent Events (SSE) to eliminate severe serialization overhead and garbage collection pressure.

How MAPFRE USA modernized fraud claims with Amazon EMR Serverless · AWS MAPFRE USA moved beyond traditional rules-based fraud detection by integrating ML models with 54 graph-based features using Neo4j and AWS. The data platform operates on an Iceberg lakehouse architecture, utilizing EMR Serverless and Apache Airflow to orchestrate ingestion, batch scoring, and graph enrichment. Model predictions are fed into Guidewire Claims via a resilient, asynchronous event-driven Lambda pattern equipped with automated retries and dead-letter queues. This direct workflow integration generated over $5 million in net present value by embedding transparent ML explanations straight into adjuster dashboards.

Unlocking the future of video data: March Networks cloud storage on AWS · AWS Enterprise video surveillance from thousands of distributed locations creates an unsustainable on-premise hardware storage bottleneck. March Networks engineered a hybrid cloud architecture moving petabyte-scale video archives to Amazon S3 and S3 Glacier. This centralized approach uses automated lifecycle policies to tier footage based on access frequency, significantly dropping storage infrastructure costs. By centralizing the data, teams can now apply AI-driven analytics, such as Amazon Bedrock-powered semantic search, across an entire organization’s video fleet.

Modernizing the Meta Ads Service With an Open-Source Kernel Scheduler · Meta When Meta upgraded their Linux fleet to kernel 6.9, the new EEVDF scheduler caused a severe latency regression in their Ads serving infrastructure. They mitigated this by deploying sched_ext, an extensible BPF-based scheduling framework that allowed them to soft-partition CPUs specifically for the ads workload. This custom, user-space BPF policy dynamically adjusts CPU pools to keep latency-critical threads localized in the L3 cache, avoiding costly DRAM access overhead. The resulting 28% reduction in p99 latency proves that overriding generalized kernel schedulers with domain-specific BPF policies delivers massive scale returns.

Verifying Rust cryptography in SymCrypt, from standards to code · Microsoft Microsoft is ensuring extreme security for post-quantum cryptographic code by formally verifying their Rust-based SymCrypt library. Because production cryptography requires highly optimized, architecture-specific code (like SIMD intrinsics), pure rewriting is insufficient. They utilize Aeneas to translate Rust’s mid-level representation into pure Lean models, allowing formal verification against mathematical standards without sacrificing hardware performance optimizations. AI agents are then deployed to automate the routine generation of proof scripts, making formal verification scalable for continuous production pipelines.

The context layer for AI: Bringing trusted Dropbox content into OpenAI workflows · Dropbox Dropbox has integrated its storage fabric deeply into OpenAI workflows, enabling tools like ChatGPT to securely access trusted enterprise content. This approach aims to provide LLMs with a reliable “context layer” that strictly respects existing organizational permissions and IT governance. By creating official Dropbox skills for ChatGPT, users can retrieve documents, generate summaries, and seamlessly save outputs back to shared folders. The engineering philosophy dictates that AI is only as useful as its context, and that context must remain bound by established enterprise access controls.

Preparing for the post-quantum era: Discover and prioritize now · HashiCorp The threat of “harvest now, decrypt later” (HNDL) attacks means organizations must prepare for cryptographically relevant quantum computers today. Migrating to post-quantum cryptography (PQC) begins with total visibility into existing cryptographic assets, dependencies, and vulnerable algorithms like RSA and ECC. HashiCorp advises evaluating systems based on data sensitivity, confidentiality lifespans, and quantum risk exposure to prioritize PQC upgrades. Tools like Vault enable crypto-agility by supporting new NIST-approved post-quantum algorithms (like ML-DSA) without requiring complete redesigns of hybrid infrastructure workflows.

How Microsoft Ships AI Agents at Enterprise Scale · Microsoft Shipping prototype AI agents is easy, but production agents break due to unhandled edge cases, context staleness, and poor telemetry. Microsoft’s AI Foundry combats this by treating retrieval as a subagent, replacing one-shot RAG lookups with iterative planning and retry loops to handle complex queries safely. Crucially, agents are treated as primary identities inside Microsoft Entra to enforce robust access control and maintain compliant audit trails. They also replaced generic LLM metrics with continuous, rubric-based evaluation on live traffic to enable automated self-improvement and strict guardrail enforcement.

Empowering India’s next generation of innovators with ATL Saathi · Google Google, in collaboration with AIM, has launched ATL Saathi, an educational tool powered by the Gemini AI ecosystem. The platform is designed specifically to empower educators managing hardware and robotics labs across India. By leveraging AI to assist in complex technical instruction, it reduces the friction teachers face when introducing advanced robotics concepts. This initiative demonstrates how targeted AI integrations can scale technical literacy and hardware education for the next generation of innovators.

Open-weight models surge to 29% of volume, price per token flattens · Vercel Vercel’s AI Gateway data from June 2026 reveals that open-weight models now account for 29% of total production token volume, while only consuming 4% of total spend. DeepSeek experienced a massive surge, securing 22.6% of all gateway tokens to directly rival Google’s footprint. Interestingly, while cheap open-weight volume exploded, frontier closed-weight models saw their average token cost rise by 12% as they handled more complex, high-risk requests. This indicates a maturing engineering strategy where enterprises route high-volume tasks to low-cost open models while reserving frontier models for critical reasoning.

Web Analytics and Speed Insights are now more cost-efficient · Vercel Vercel deployed infrastructure and event-processing improvements that made their Web Analytics and Speed Insights roughly 10% more cost-efficient. The underlying optimizations enable page views, referrers, and Core Web Vitals to be processed with significantly less compute overhead. This cost reduction scales automatically with event volume for all Pro and Enterprise teams. The update was applied transparently on the backend without requiring users to alter any tracking code.

AI Gateway leaderboards, now with open data and shareable charts · Vercel Vercel enhanced its AI Gateway leaderboards by making the aggregated production traffic data completely open and exportable. The leaderboards track models, labs, apps, and providers by metrics like token volume, request counts, and generated media. Developers can now query the data programmatically via an export endpoint, with results cached every 24 hours to ensure platform performance. Releasing this anonymized usage data under a Creative Commons license provides the engineering community with transparent insights into real-world AI adoption trends.

Configure which sources can create deployments with Deployment Policies · Vercel Vercel introduced Deployment Policies to give engineering teams granular control over how deployments are triggered. Administrators can now strictly limit deployment creation based on specific mechanisms, organizations, or authorized repositories. These security policies can be configured per environment at both the project and team levels. This architectural enhancement minimizes the risk of unauthorized or accidental code deployments across diverse frontend CI/CD workflows.

Manage Vercel Flags targeting rules from the CLI · Vercel Feature flag management in Vercel can now be handled entirely through the Vercel CLI via the vercel flags rules command. Engineers and automated agents can insert new targeting rules, adjust evaluation ordering, and query configurations without ever needing the web dashboard. It fully supports complex targeting structures, including progressive rollouts and environment inheritance logic. This CLI exposure is critical for teams wanting to treat feature flags as code and fully automate rollout strategies within CI/CD pipelines.

Chat SDK adds X adapter support · Vercel Vercel’s Chat SDK has expanded its write-once-deploy-anywhere bot framework by adding an official adapter for X (formerly Twitter). The adapter utilizes the X API v2 and Activity API to manage public mentions and DMs seamlessly. It automatically abstracts away the complexity of CRC verification, webhook signature validation, and continuous OAuth token refreshes. This allows developers to port existing chat agents to X without rewriting tedious platform-specific authentication and delivery logic.

Agent Runs now show subagent activity on eve projects · Vercel Observability in multi-agent systems is notoriously difficult, prompting Vercel to upgrade their Agent Runs view for “eve” projects. A new dedicated Subagents tab visualizes all spawned subagent activity on a shared timeline. It details prompt inputs, execution duration, token usage, and failures explicitly linked to the parent turn that triggered them. This nested telemetry is essential for debugging runaway tool calls or logic loops in hierarchical agent architectures.

The Frontend Verification Gap in AI-Assisted Development · O’Reilly AI code generators excel at producing visually complete frontend components, but they consistently fail at accessibility, keyboard navigation, and edge-case state management. This creates a “frontend verification gap,” where AI outputs look polished but fail under realistic interaction conditions. To combat this, engineers must enforce rigid boundaries by requiring AI to compose interfaces strictly from pre-verified design-system components rather than generating ad-hoc UI. Furthermore, review pipelines must shift from purely structural code checks to behavior-focused user-flow testing to validate actual usability.

Introducing Precursor: detecting agentic behavior with continuous client-side signals · Cloudflare Traditional CAPTCHAs and point-in-time bot checks are failing because modern bots can execute JS and simulate single interactions perfectly. Cloudflare launched Precursor to combat this by injecting dynamic scripts that monitor behavioral signals—like wrist-pivot arcs, tremors, and cognitive delays—continuously across an entire user session. The edge-side evaluation layer cross-references this buffered stream of pointer and keyboard activity against known human biomechanical constraints. By shifting detection from single-request checkpoints to persistent, session-scoped behavioral analysis, Precursor drastically increases the cost and difficulty for attackers attempting to scale automated agents.

Patterns Across Companies#

A massive shift is occurring around Agentic Infrastructure & Observability. Microsoft, AWS, DoorDash, and Vercel are all evolving beyond simple RAG architectures, instead building multi-agent “retrieval-as-a-subagent” patterns that require strict identity management (like OBO tokens and Entra integrations) and robust subagent telemetry. Concurrently, there is a distinct push toward User-Space & Edge Execution to solve scale bottlenecks; Meta is using BPF for custom kernel scheduling, while Cloudflare is leveraging edge computing for continuous behavioral analysis, bypassing generalized approaches to drastically reduce latency and improve precision.