A
Microsoft is making its Cobalt 200 virtual machines generally available, with CPU improvements aimed at the sandboxes, tools, and code execution that keep AI agents running. At its Ignite conference in San Francisco on Tuesday, the company announced that the VMs, powered by its second-generation Arm-based CPUs, are available in 14 Azure regions, expanding to 23 by the end of the year. Microsoft says they offer up to 50% better performance compared with Cobalt 100. It’s been almost exactly a…
A
Production traffic shows failures that benchmarks or pre-deployment evaluations can’t fully reveal. It tells you which edge cases matter and what users need. Turning that signal into the next model version requires an upfront decision: does the model need to change, or is the problem in retrieval, context, prompts, tools, or serving? Changing weights won’t fix a failure elsewhere in the system. When the model is the right place to intervene, production evidence can improve both quality and…
A
AI agents have had a URL problem for a while. When an agent needs current information but has no search tool, it often makes up the web address it thinks the answer lives at. It might guess something like docs.example.com/v4.2/auth, try to load it, and get a “page not found” error because the page doesn’t exist. Then it either keeps guessing or falls back on outdated information from its training. “Live web search gives AI agents fresh pages. Deciding which ones to trust over your own data is…
A
Anthropic has unveiled a self-hosted version of its platform for building and running AI agents. Self-hosted Claude Managed Agents, as it’s aptly called, is available now in private beta. Customers host both the server that coordinates the agents and the sandboxes where the agents run their tools, either in their own cloud account or in an on-premises data center. Credentials and session histories stay on their infrastructure, while requests to Claude go to the inference provider they…
A
Accelerator bills have become one of the largest lines in the infrastructure budget. GPUs have become the most expensive resource in modern computing, and yet most organizations operate them with little visibility. A single training cluster can run for days and cost millions of dollars. Most organizations pool these accelerators across teams on shared clusters to keep them busy. But in these shared environments, platform teams cannot answer basic questions: which team is using which…
A
Moving an AI workload to another cloud means bringing more than a model and its data. Your serving layer, observability dashboards, security controls, and storage systems have to work together on the new infrastructure. For platform teams, proving those connections under real traffic can become the slowest and riskiest part of the move. As a stack grows across vendors and clouds, each project adds more integration questions. A compatible specification is a starting point. It doesn’t guarantee a…
A
Google’s eighth-generation TPU announcement on Monday included more than new chips, introducing TPU Sync, an open-source system designed to speed up AI inference by tackling a problem that worsens as context windows grow: moving data between different types of memory. The software is made for developers running large models, particularly ones handling million-token contexts and reinforcement-learning workloads. The company claims it can cut KV-cache transfer latency and time to first token by…
A
Perplexity’s new Teammates can coordinate multiple agents, remember where previous sessions left off, and keep working without another prompt. The challenge is keeping everything in sync, especially when agents are working at the same time, sharing project state, and operating with different permissions. Announced Wednesday, Teammates builds on infrastructure Perplexity has spent much of the year developing. The launch puts that earlier work to use in workflows where agents can delegate tasks…
A
Together AI is putting AI coding agents to work on the GPU code that runs its inference infrastructure, automating performance tuning that used to take engineers days or weeks of manual work. The company unveiled Together Compiler on Wednesday, a system that generates, tests, and improves GPU kernels without an engineer directing each experiment, and Together AI says it can deliver gains overnight. The process runs without a person steering each step, but it doesn’t ship anything on its own.…
A
When organizations across the industry first adopted generative AI, most consumed it through a hosted API, but open-weight models are changing that. Models like the Qwen family and GLM-5 are published on model registries like Hugging Face, many under Apache 2.0 licensing, and load directly into open-source serving engines like vLLM and SGLang. “For a growing set of workloads, that gap is small enough that the model is no longer the main constraint.” Closed models pulled further ahead in 2025,…
A
Dropbox is entering the AI infrastructure space in a big way, announcing plans on Thursday to acquire Tigris, a startup that provides object storage for AI applications. The cloud-storage giant tells The New Stack that it’s paying $350 million for Tigris, a move designed to accelerate Dropbox’s own ambitions in selling storage infrastructure to developers and businesses. Indeed, alongside the acquisition news, Dropbox also announced DBX Agent Cloud, an upcoming offering that will initially let…
A
Render is taking on Vercel with AI sandboxes that can run indefinitely, along with new tools for managing long-running agent workflows and shared storage. Announced Wednesday, the launch removes Render’s previous 30-day continuous runtime limit on paid workspace plans. Vercel caps individual sandbox sessions at 24 hours, while Render says its environments can suspend when idle and resume with their files, memory and running processes intact. Render founder and CEO Anurag Goel, a former Stripe…
A
OpenSearch has come a long way since Amazon Web Services first announced the project back in January 2021, in response to Elastic’s decision to move Elasticsearch and its accompanying visualization tool, Kibana, away from Apache 2.0 and onto a combination of proprietary and source-available licenses. Elastic said at the time that the reason was to prevent cloud behemoths such as AWS from offering Elasticsearch “as a service” without giving anything back, while also accusing AWS of violating the…
A
Welcome to the final edition of Road to KubeCon, where we track the Kubernetes and cloud-native ecosystem each week on The New Stack as we head into KubeCon + CloudNativeCon NA, November 9-12 in Salt Lake City, Utah. This week we explore how Kubernetes is getting new guidance for running AI workloads, including a new book from TNS, a dedicated Kubernetes working group, and advice on running LLMs across multiple GPUs. Also in this day-before-Halloween-timed column, we’ve got OTel operators…
A
Microsoft is opening up the GitHub Copilot agent harness to outside developers through Microsoft Foundry, bringing the same orchestration infrastructure used across its Copilot products into a managed service for building and deploying autonomous agents. Announced Wednesday, the Microsoft Copilot harness is now available in public preview through Foundry Agent Service. It comes with built-in sandboxing, reusable skills, MCP integrations, and support for subagents. Microsoft takes care of…
A
At its annual Universe ’26 conference in San Francisco this week, GitHub took to the question of the platform’s stability head-on. The code-hosting company spent a significant part of the event’s keynote to talk about the investments it is making. That includes adding 4 million cores and 160 petabytes of storage, along with a full rearchitecting of its backend infrastructure. But at a press conference and in interviews with The New Stack after the keynote, the leadership team of Jay Parikh, the…
A
When OpenAI announced at DevDay that ChatGPT had crossed 1.2 billion weekly active users (WAUs), roughly 15% of the human population, most engineering discussions naturally centered on model context windows, compute clusters, and distributed GPU inference. However, behind every chat session, API call, and model invocation sits an equally monumental infrastructure challenge: Identity and Access Management (IAM) at extreme web scale. “Behind every chat session, API call, and model invocation sits…
A
In AI agent training, you can pay for powerful GPUs and still spend much of your time waiting. Agents need to attempt tasks and produce results before the model can learn from them. A few slow attempts can hold up the next training update, making the systems that run those tasks a major factor in training speed and cost. Di Jin, VP of AI Training at Amsterdam-headquartered Nebius, says teams coming from pretraining or supervised fine-tuning can underestimate the work these systems do. “If you…
A
Entire, the startup founded by former GitHub CEO Thomas Dohmke, on Monday launched its GitHub alternative that replaces the traditional pull request model with a new agentic code review system called Trails. The company has been launching parts of its platform since it announced a $60 million seed round in February. Early on, Dohmke occasionally noted he wasn’t planning to compete with GitHub. But with Monday’s launch, Entire is launching a full GitHub alternative, with its own twist on what a…
A
Platform teams that mandated their internal platform were more than twice as likely to say it delivered most or all of its goals: 62.2%, compared with 27.5% for teams that made adoption optional. Over the same year, the share of mandated platforms in our research fell from 63.0% to 45.9%, and optional platforms became the majority. That puts the data at odds with the most repeated advice in platform engineering. Team Topologies, the CNCF’s Platform Engineering maturity model, and a 2026 review…
A
Kubernetes was built around a fairly simple assumption. It’s that software does what its developers told it to do. AI Agents complicate that assumption. An agent can reason about a problem, choose a course of action, call tools, write code, interact with other systems, and keep working toward a goal. That is precisely what makes agents useful. But it is also what makes running them in production different from running a conventional application. And increasingly, enterprises want to run them on…
A
For teams running AI workloads on Kubernetes, a fully allocated GPU pool and a growing batch-job queue might make expanding the pool seem like the obvious next step. But a growing queue could point to timing, serving capacity, or the choice of model and hardware. Before expanding a GPU pool, teams should define what success looks like, when it must finish, and what it costs to deliver. “An allocated GPU, a busy GPU, and a workload producing valuable results are not the same thing.” That starts…
A
Welcome to another edition of Road to KubeCon! We’re tracking movements in the Kubernetes and cloud-native ecosystem as we count down to KubeCon + CloudNativeCon North America 2026 November 9–12 in Salt Lake City. Half of our column this week focuses on untangling Kubernetes as we add the demands of agentic AI: automating pod-restart troubleshooting, syncing communication between the scheduler and autoscaler, and maturing how we distribute inference across a cloud-native stack. The other half…
A
AI agents are getting better at remembering past conversations and earlier tasks but remembering something doesn’t guarantee an agent can use it to do better work next time. Letta’s new FrontierMemory benchmark, released Wednesday, tests that gap by measuring whether agents can learn from experience rather than by just retrieving stored information. Early results show a widespread in performance across agent harnesses, the the infrastructure that manages how an agent operates. Letta Dream came…
A
Ramp spent months testing managed agent APIs, open-source frameworks, and third-party platforms before concluding it would be better off building its own. The result is Laminar, an internal agent harness that took six weeks to reach a first version and now powers every agentic product at the fintech company. “We studied the other offerings on the market and decided that we could do better, move faster, and have stronger alignment to where we want to go by building in house.” “We studied the…
A
At PyTorch Conference North America 2026 this week, Google open-sourced TorchTPU, a native PyTorch stack for Google TPUs, now available on GitHub. AWS also launched TorchNeuron in public beta, its open-source native PyTorch backend for AWS Trainium, also available on GitHub. Both releases seem aimed at making it easier for developers to move beyond Nvidia GPUs, whose CUDA software ecosystem has long dominated AI development. But while Google and AWS’s new offerings do ease some of the…
A
Enterprise security has traditionally operated on a relatively straightforward premise: establish identity, authenticate the requester, determine what that identity can access, and enforce those permissions. AI agents introduce a fundamental complication. They don’t simply access systems. They interpret objectives, select tools, create sub-agents, make decisions, and execute actions, often using credentials and permissions inherited from humans. The problem isn’t necessarily that an agent lacks…
A
Google will soon make the infrastructure that powers its Gemini agents available to Google Cloud users in preview. Announced on Thursday at the Linux Foundation’s AGNTCon+MCPCon in San Jose, Agent Compute Engine will become available in preview to Google Cloud users later this year. It will provide every AI agent with a hardware-isolated microVM that combines the speed of containers with the security of a sandbox. Who else offers agent sandboxes? The idea of agent sandboxes is anything but new,…
A
Every enterprise company is currently trying to figure out how to have its AI tools work with its data. Meanwhile, every major enterprise vendor that works with those enterprises (think SAP, Microsoft, Salesforce, Workday, etc.) is making a claim on becoming the central hub for managing and securing that data. But what about the companies that want to do it themselves? Nvidia, on Wednesday, announced how it wants to provide enterprise AI agents with the data they need to make sense of company…
A
Nvidia wants new AI capacity to show up at the data center fully built. At its GTC Berlin conference on Wednesday, the company introduced DSX POD, a reference design for shipping containers that partners fill with Vera Rubin racks, networking, and liquid cooling before they leave the factory. Installing and wiring up racks in a conventional data center can take months, Nvidia says. But a DSX POD can be ready to deliver tokens in as few as 14 days, as long as the site already has the necessary…