Engineering @ Scale — 2026-07-28#

Signal of the Day#

The most instructive insight this period comes from how DoorDash, Instacart, and Uber Eats all solved the exact same semantic search problem using entirely different LLM architectures—proving that the “best” AI integration pattern is entirely dictated by a team’s pre-existing infrastructure constraints. Instead of forcing a one-size-fits-all AI solution, top teams either enrich existing knowledge graphs offline, cache head queries while fine-tuning models for the tail, or embed LLMs directly into legacy two-tower retrieval systems.

Deep Dives#

Eval-driven development: Lessons from evaluating GenAI at scale · Airbnb When building generative AI products, non-deterministic outputs render traditional software testing paradigms insufficient, often masking critical failure modes. To solve this, Airbnb developed “Eval-Driven Development” (EDD), treating evaluation as core infrastructure that continuously tests for failures via programmatic constraints and highly calibrated “LLM-as-judge” models. For agentic systems, they leverage DFS tree traversal on execution traces to reconstruct and evaluate multi-step reasoning and sub-agent invocations, rather than just checking the final answer. This highlights a generalizable lesson for AI engineering: evaluating the entire pipeline—including retrieval and tool use—is critical to preventing final-output correctness from masking a broken reasoning path.

Uber’s Zero Growth Stack: Scaling Services, While Optimising Infrastructure and AI Cost · Uber Uber is addressing the challenge of scaling services efficiently without a linearly proportional increase in infrastructure. They implemented the “Zero Growth Stack,” an architectural approach that explicitly decouples capacity growth from business demand to rein in runaway hardware requirements. A central pillar of this strategy is aggressive garbage collection optimization to reduce baseline server load. This approach demonstrates how hyper-growth companies must eventually pivot to deep resource optimization and strict cost-management measures to maintain viable unit economics at scale.

AWS Launches Amazon GuardDuty Investigation Agent to Automate Threat Triage · AWS Security teams face overwhelming alert volumes that delay critical threat triage across complex cloud architectures. AWS introduced a public preview of the GuardDuty Investigation Agent to automate this process by actively correlating findings with 90-day activity logs and topological resource graphs. The agent generates structured reports containing risk ratings, confidence scores, and MITRE ATT&CK classifications for rapid assessment. Notably, AWS exposes this agent through an MCP (Model Context Protocol) Server, enabling direct orchestration from external agentic tooling and standardizing autonomous security investigations.

Article: The Hard-Stop Rule: From 3 HCM Monoliths to 120 Domain Microservices · InfoQ Migrating legacy monoliths to microservices without dedicated budget is a notorious engineering trap. A payroll and HR team successfully decoupled three monoliths into over 120 domain services over five years using a strict “hard-stop rule”. Instead of retrofitting old systems, they forced every new feature to be built as an independent service. By utilizing a pull-based migration strategy and carefully selected tooling, they managed to keep infrastructure costs down while gradually starving the monoliths of new development.

Remix 3 Beta Preview Ditches React for a Web-Standards Full-Stack Framework · Remix Web frameworks are increasingly grappling with the overhead and complexity of React’s render lifecycle for high-performance applications. Remix 3 addresses this by pivoting away from React entirely, instead emphasizing web platform primitives and utilizing a forked version of Preact for the frontend. The architectural shift unifies routes, request handlers, and UI components into a single, cohesive structure that strictly prioritizes server ownership of the request lifecycle. This signals a growing trend of full-stack frameworks shedding heavy frontend libraries to optimize server-to-client data flow, even at the cost of backward compatibility.

Presentation: The Future of Engineering: Mindsets That Matter When Code Isn’t Enough · InfoQ As AI automation accelerates, the intrinsic value of raw code generation is commoditizing rapidly. Engineer Ben Greene argues that senior engineers must shift their focus from syntax generation to systems-level problem solving and direct customer impact. Key architectural mindsets for this transition include starting with simple designs, prioritizing system comprehension over output volume, and tackling the hardest structural problems first. Because AI handles rote implementation, human agency and pragmatic architectural tradeoffs become the irreplaceable core of the engineering profession.

Grafana Assistant Expands to More Than 30 Data Sources · Grafana Labs Correlating telemetry across disparate observability tools is traditionally a manual, query-heavy process during critical incidents. Grafana Labs tackled this by expanding its AI-powered observability assistant to support querying across over 30 distinct data sources. Engineers can now use natural language to filter and correlate incidents across these previously siloed backends. This architectural integration positions the AI as a unified translation layer, mapping natural language intent into complex, source-specific query syntaxes for faster incident resolution.

GitHub Introduces Default “Cooldown” Policy for Dependabot Version Updates · GitHub Supply chain attackers frequently rely on speed, hoping malicious package updates are merged before security systems can detect the threat. GitHub altered its Dependabot architecture to introduce a default three-day “cooldown” period for version updates. Instead of immediately opening pull requests upon a package release, the system intentionally delays the PR to allow community detection signals to surface. This tradeoff deliberately sacrifices absolute version recency to establish a significantly hardened defense against zero-day supply chain compromises.

Market surveillance agent with LangGraph and Strands on AgentCore · AWS Financial market surveillance requires orchestrating multi-agent systems that handle complex reasoning, strict compliance, and robust error recovery. AWS engineered a solution pairing LangGraph for deterministic macro-level workflow orchestration with Strands for intelligent, node-level agent reasoning. By isolating Strands agents inside individual LangGraph nodes, the system compartmentalizes context memory, preventing the monolithic LLM hallucination trap where agents lose track of complex instructions. The architecture runs on Amazon Bedrock AgentCore, utilizing DynamoDB for persistent checkpointing to enable safe, stateful human-in-the-loop recovery.

How AgentCore Gateway supports the MCP 2026-07-28 spec · AWS Scaling the Model Context Protocol (MCP) in enterprise deployments previously required sticky sessions due to its stateful initialization handshake. The new 2026-07-28 MCP specification overhauls this by transitioning to a stateless HTTP protocol, embedding protocol versions and capabilities directly into per-request metadata (_meta). AWS integrated this into AgentCore Gateway, enabling seamless horizontal scaling, HTTP-layer routing via standard headers (Mcp-Method), and native W3C Trace Context propagation without the need to parse the JSON-RPC body. This redesign proves that scaling agentic tools requires strict adherence to standard, stateless web infrastructure principles.

Disrupting supply chain attacks on npm and GitHub Actions · GitHub Software supply chain attacks ruthlessly exploit CI/CD vulnerabilities to execute untrusted code and exfiltrate credentials for broad malware distribution. GitHub and npm re-engineered their default security postures by blocking untrusted code execution from fork PRs (“pwn requests”) and introducing a read-only Actions cache for untrusted triggers. To safeguard publishing workflows, they forced npm high-impact accounts into a 72-hour read-only mode upon 2FA changes, and staged npm v12 to disable install-time scripts by default. These systemic architectural changes highlight that securing infrastructure at scale requires breaking backward compatibility to eliminate inherently risky default behaviors.

Terraform AzureRM provider 5.0 now generally available · HashiCorp Managing Azure infrastructure via Terraform previously suffered from slow initializations and permission errors due to the automatic registration of over 60 legacy Resource Providers. The release of AzureRM 5.0 abandons this behavior, shifting control to platform teams who must now explicitly define required providers. The update also introduces an opt-in Azure Preflight Validation API, allowing the provider to check configurations against Azure quotas and policies during the terraform plan phase before state mutation begins. This illustrates a broader infrastructure-as-code pattern: moving structural validation earlier into the pipeline (shift-left) while aggressively reducing implicit provider magic.

Looking back on Microsoft’s FY26: From AI experimentation to Frontier Transformation · Microsoft Enterprises are aggressively transitioning from AI experimentation to building interconnected “agentic systems” that operate continuously across core business functions. Microsoft details how organizations deploy ecosystems of thousands of governed AI agents to scale operations—for example, Banco Popular Dominicano expanded real-time risk monitoring coverage from 40% to 100%, processing 80,000 documents weekly. A recurring architectural theme is grounding these agents in proprietary data layers (like Microsoft Fabric) to create a compounding “intelligence platform” rather than relying solely on raw foundational models. This indicates that enterprise AI success hinges on secure governance layers and deep data integration, not just baseline model capability.

Why DoorDash, Instacart, and Uber Eats Integrated LLMs Into Search Three Different Ways · ByteByteGo DoorDash, Instacart, and Uber Eats faced the exact same challenge of handling complex semantic search queries but shipped drastically different LLM architectures dictated by their existing systems. DoorDash utilized an offline LLM to extract attributes and enrich their knowledge graph, using runtime RAG strictly as a constrained guardrail to map queries to a known taxonomy. Instacart deployed an offline RAG cache for head queries and a real-time, fine-tuned Llama-3-8B model for long-tail queries, keeping downstream retrieval classical. Uber Eats, leveraging massive multi-vertical requirements, embedded a fine-tuned Qwen model directly into a two-tower retrieval system, handling matching in vector space using Matryoshka Representation Learning and int7 quantization for ultra-low latency.

Scientific computing in the age of agentic AI · OpenAI Scientific computing often struggles with legacy codebases and highly bespoke software environments that throttle empirical discovery. OpenAI highlights how scientific researchers are incorporating AI coding agents to systematically modernize these specialized software stacks. By leveraging autonomous agents, scientists can accelerate the translation of complex mathematical and genomic models into scalable, performant software. This approach removes the bottleneck of manual software engineering in domains where pure scientific expertise traditionally outpaces software architecture skills.

Vercel Sandbox supports forking · Vercel Creating isolated, reproducible runtime environments rapidly is critical for testing multi-tenant AI agents safely. Vercel enhanced its Sandbox offering with a Sandbox.fork() capability, allowing developers to branch a new sandbox directly from a source’s latest snapshot. The fork inherits all configuration and environment variables, completely bypassing the need to cold-start a fresh runtime from scratch. This represents a powerful infrastructure optimization that brings Git-like branching semantics to ephemeral runtime environments for massive parallel testing.

WebSocket support is now available for Python Functions · Vercel Serverless environments have historically struggled with the persistent, bidirectional connections required for real-time applications. Vercel addressed this by introducing native WebSocket support for its Python runtime environments, accommodating popular ASGI and WSGI frameworks like FastAPI, Django, and Flask. This architectural upgrade removes the need for external long-polling or dedicated WebSocket microservices, allowing serverless functions to natively handle interactive AI streaming. This signals a maturation of serverless compute, successfully bridging the gap between stateless execution and stateful client-server communication.

How Sandstone grew 40x in 147 days on Vercel · Vercel Sandstone, a legal tech startup managing complex data ingestion across disparate systems, needed to iterate rapidly while strictly meeting enterprise security demands. They implemented a seven-app turborepo architecture on Vercel, heavily leveraging Vercel’s AI SDK to compose multi-step agentic workflows that parse sensitive procurement agreements. Crucially, they bypassed traditional compliance bottlenecks by utilizing Vercel Secure Compute, which establishes private, non-public network connections directly to their data layer. By combining componentized AI toolkits with zero-trust infrastructure primitives, they shipped enterprise-grade compliance at consumer-startup velocity.

Vercel Connect now supports Custom Environments · Vercel Managing third-party provider tokens and webhooks across diverse deployment stages is a persistent configuration and security nightmare. Vercel Connect updated its architecture to support Custom Environments, enabling developers to link specific connectors to non-standard environments beyond standard production or preview. Deployments in these isolated environments can securely request provider tokens and receive forwarded webhooks specifically routed to their verified domains. This allows infrastructure teams to enforce strict provider-level isolation, ensuring that sandbox credentials never leak into or conflict with production workflows.

5 ways to host the ultimate dinner party with Google Search · Google Consumer applications are increasingly masking complex AI orchestration behind simple search interfaces. Google highlighted how Search’s generative AI tools can autonomously craft menus, design tablescapes, and organize logistical party-planning tasks. From an engineering perspective, this requires dynamic prompt chaining and task-routing, where the system retrieves disparate knowledge graph nodes and synthesizes them into actionable outputs. It demonstrates the ongoing shift from traditional information retrieval toward intent-driven task synthesis in mass-market consumer software.

5 ways AI Mode in Search helps you enjoy the real world · Google Bridging the digital-to-physical divide requires AI to navigate highly dynamic, real-world data APIs like ticket availability and local business hours. Google’s AI Mode in Search parses complex user intents to execute multi-step queries—such as booking concert tickets—directly from the search interface. This implies a backend architecture capable of real-time API integrations and strict entity resolution, ensuring that AI-generated suggestions match actual, bookable inventory. This illustrates how search engines are evolving into action-oriented agents rather than static link aggregators.

Gemini API Managed Agents: 3.6 Flash, hooks, and more · Google Building reliable agents requires deep control over the execution lifecycle, not just raw model intelligence. Google released updates to Managed Agents in the Gemini API, introducing the 3.6 Flash model alongside powerful new execution hooks. These hooks allow developers to intercept and modify the agent’s state or actions mid-execution, a critical requirement for production-grade telemetry, safety validations, and dynamic routing. By providing managed infrastructure with deterministic interception points, LLM development is moving away from black-box prompting and toward standard software engineering lifecycles.

Powerful Compute So Compact, It’s Clutch — Build AI in Your Hand With NVIDIA Jetson · NVIDIA Deploying agentic AI in physical robotics requires massive compute constrained by severe size, weight, and power (SWaP) limitations. NVIDIA’s Jetson Orin Nano Super condenses 67 TOPS of generative AI processing into a module small enough to physically fit in a handbag. The architecture allows robots to run complex vision-language models (VLMs) and AI assistants entirely on-device, achieving low latency without relying on cloud APIs or persistent internet connectivity. By moving frontier models completely to the edge, this platform drastically reduces network latency overhead and enhances operational security for autonomous systems.

Teaching Coding When AI Can Write the Code · O’Reilly With generative AI commoditizing code production, evaluating software engineering competency based solely on a finished artifact is fundamentally broken. UT Austin’s creative coding department transitioned from artifact-based grading to a “studio model” focused entirely on process, public debugging, and live coding performance. To scale assessment, they engineered an inverted AI workflow: a custom Vera Molnár chatbot interrogates students on their architectural choices, forcing them to defend their logic while generating a rubric-aligned transcript of their understanding. This shift—treating AI as an assessor and demanding performative fluency—offers a blueprint for how technical interviews and code reviews must evolve when “the artifact is no longer the proof”.

Natural disasters and government interference: examining Q2 2026’s major Internet disruption event · Cloudflare The internet’s physical and cryptographic infrastructure remains highly fragile to regional disruptions. Cloudflare’s Q2 2026 disruption report analyzed how diverse root causes—including Typhoon Sinlaku in Guam, physical drone strikes on UAE AWS data centers, and a subsea cable cut in Saint Lucia—all manifest as near-identical telemetry drops. Cryptographically, an invalid DNSSEC key rollover at Germany’s .de registry triggered a massive query spike as validating resolvers continuously retried uncacheable SERVFAIL responses, effectively blackholing national web traffic. These incidents underscore that global service reliability requires immense redundancy in network routing, physical paths, and cryptographic key management.

Patterns Across Companies#

A dominant architectural theme this period is the transition of LLMs from monolithic “magic” processors into modular, deterministic pipelines. Companies like Airbnb, AWS, and Instacart are wrapping models in strict governance, utilizing RAG as a hard constraint or isolating agent context within deterministic state machines (like LangGraph) to ensure reliability. Simultaneously, the industry is standardizing around statelessness and lifecycle hooks—seen in MCP’s new 2026-07-28 spec and Gemini’s API updates—proving that scaling AI requires leveraging standard, observable software engineering primitives.