Several of the most-used hosted models retire between November 30, 2026 and January 14, 2027, including GPT-5, Gemini 2.5, and Sonnet 4.5. If you use any of these models, you’ll need to transition. But upgrading models usually requires more than just swapping out the model parameter in your API calls. On the most obvious level, […] The post GPT-5, Gemini 2.5, and Claude Sonnet 4.5 Retirement and Deprecation: What Should You Test Before Switching Models? appeared first on Comet.
Opik is Comet’s open-source platform for tracing, evaluating, testing, and monitoring LLM and agent applications. It tracks every call an application makes, including prompts, tool calls, retrievals, and LLM outputs. This gives teams tools to understand the application’s behavior, find where it goes wrong, and identify what to improve, from the developer workstation to production. […] The post How we made Opik dataset and experiment ingestion 30× faster appeared first on Comet.
Incident remediation agents are quickly emerging as one of the most compelling AI use cases for the enterprise. This is particularly true for large financial institutions, where complex infrastructure and the sheer volume of production signals make it difficult to distinguish genuine incidents from background noise. That’s where remediation agents step in — they can […] The post Optimizing Agentic Remediation for Financial Services with Opik appeared first on Comet.
Trace a multi-agent Google ADK app with Opik, fix handoffs that split into two traces, and evaluate it on 23 hard support cases to compare two Gemini models on cost, speed and refund safety. The post How to Trace and Evaluate a Google ADK Agent with Opik appeared first on Comet.
No customer data, accounts, or API keys were accessed. On October 26, 2026, Comet contained a security incident affecting Opik on comet.com. This post explains what happened, what was affected, and what we have done in response. What happened Between October 24 and 26, 2026, an unauthorized party used a comet.com user API key from […] The post Opik Security Incident: October 2026 appeared first on Comet.
At Comet, we’ve seen firsthand how teams in finance, tech, e-commerce, and beyond use the Opik platform to track critical metrics for monitoring and improving their agents. Those metrics vary widely — for instance, a financial services team checking that its internal support agent follows company policy will track very different metrics than an automotive […] The post Introducing Opik’s Generative Dashboard Builder appeared first on Comet.
Decision models drew wide attention in September, when TypeSafe AI launched Jev, a model that returns probabilities over a fixed set of answers instead of generating text. Open-source alternatives followed almost immediately: Latent.Space counted six in two days. Most writing about decision models since then defines them by their interface, which uses some sort of […] The post Train your own decision model appeared first on Comet.
A model that answers yes/no questions instead of writing text came out 3.5× cheaper and 3.8× faster than our gpt-4o-mini judge on 1,000 real production turns, and agreed with it on 897 of them. Here’s the test that showed it, what the 103 disagreements have in common, and how to run it yourself the next […] The post Jev vs. LLM-as-a-Judge for AI Evals appeared first on Comet.
Comet’s Opik platform is now listed in Nebius AI Cloud Applications, bringing agent observability, evaluation, and the full Comet AI development platform to one of the most powerful AI cloud platforms available today. It is the first step in a broader collaboration between Opik and Nebius to bring observability closer to the infrastructure where AI […] The post Comet Extends Agent Observability and Evaluation to Nebius AI Cloud appeared first on Comet.
We ran a small model over thousands of Claude Code session and discovered how people really use coding agents. Comet Cost Intelligence is a tool for optimizing engineering token spend on Claude Code and Codex. While working on it we started with common issues: default models set too high, skills and MCP servers loaded into […] The post Capturing a 400-Turn Claude Code Session in a Single Image appeared first on Comet.
Despite how often prompts change once an agent is live, most teams don’t have a real prompt management process. Somewhere on a shared drive is prompt_final_v2.txt, and somewhere else is a version someone pasted into a Slack DM that is two edits newer. Neither one matches what runs in production, and nobody can say for […] The post Prompt Management for AI Agents: The Best Tools Compared (2026) appeared first on Comet.
When an LLM call is slow, expensive, or wrong, you want to see the whole request: the prompt, the model, the tokens, the tool calls, the latency. If your traces live in one tool and your evaluation lives in another, that picture is split in two. This guide shows how to send TrueFoundry AI Gateway […] The post Export TrueFoundry AI Gateway traces to Opik with OpenTelemetry appeared first on Comet.
The way developers interact with products is changing, and opening your app’s dashboard is no longer the obvious entry point. They want to use their apps inside the terminal. Why? Because they’re already there. We’re seeing more users instrumenting MCP and it makes sense. As engineers work primarily in their terminals, they want to treat […] The post UX Without a UI: Notes from Researching Terminal UX for Opik MCP appeared first on Comet.
“Language model” used to be a very general term for statistical models trained on language tasks. It included everything from n-gram models to LSTMs and GRUs. But following the release of GPT-3.5 and ChatGPT, the term “language model” narrowed, at least in the popular sense, to mean one thing: autoregressive Transformers. In 2026, that definition […] The post Diffusion Language Models, From Scratch to Production appeared first on Comet.
Your new AI product is live. Infra dashboards are all green. Latency is low, error rates are flat, and CPU/GPU utilization looks healthy. Yet your Slack channels are full of screenshots from users asking, “Why did it do this?” The same user goal and high-level prompt can still produce wildly different behavior. Sometimes the agent […] The post What is AI Observability? A Complete Guide to Debugging and Monitoring Modern AI Systems at Scale appeared first on Comet.
Someone on your team defaulted to the latest and greatest model available, which is also the most expensive model. Maybe it was an agent calling the flagship model for every tool call or a coding agent set to its most capable model out of the box. It worked, so nobody revisited it. Then the bill […] The post LLM Model Selection: How to Pick the Right Model for Every Agentic Task appeared first on Comet.
LLM applications are everywhere now, and they’re fundamentally different from traditional software. They’re non-deterministic. They hallucinate. They can fail in ways that are hard to predict or reproduce (and sometimes hilarious). If you’re building LLM-powered products, you need visibility into what’s actually happening when your application runs. That’s what LLM observability tools are for. These […] The post Best LLM Observability Tools of 2026: Top Platforms & Features appeared first on…