The vector database category is undergoing a shift in response to the needs of agentic AI. 

The retrieval-augmented generation (RAG)-to-vector database pipeline doesn’t cut it anymore; agentic AI requires a different approach that incorporates context. VentureBeat’s Q1 2026 Pulse survey underscores this trend: Every standalone vector database is losing adoption share, while hybrid retrieval intent has tripled to 33.3%, the fastest-growing strategic position in the dataset.

Vector database pioneer Pinecone recognizes this and is pivoting to meet the specific needs of agentic AI.

The company today announced Nexus, which it positions as a knowledge engine rather than an improvement on retrieval. Nexus introduces a context compiler that converts raw enterprise data into persistent, task-specific knowledge artifacts before agents query them, and a composable retriever that serves those artifacts with field-level citations and deterministic conflict resolution.

Alongside Nexus, Pinecone is releasing KnowQL, a declarative query language that gives agents a vocabulary to specify output shape, confidence requirements, and latency budgets. In Pinecone’s own internal benchmark, one financial analysis task that previously consumed 2.8 million tokens was completed by Nexus with just 4,000. This represents a 98% reduction, although the company has not yet validated it in customer production deployments. Nexus is in early access starting today.

“RAG was built for human users,” Pinecone CEO Ash Ashutosh told VentureBeat. “Nexus was built for agentic users, because their language is very different. The responses they expect are very different. The task that an agent is assigned to do is very different from what a chatbot is supposed to do.”

Why RAG was never built for what agents actually do

RAG encompasses one query, one response, and a person in the loop to interpret the result. But agents work differently. They are assigned tasks, not questions — and completing these requires assembling context from multiple sources, resolving conflicts, tracking what has already been retrieved, and deciding what to query next.

The distinction matters. A RAG pipeline retrieves documents and hands them to a model at inference time. Each agent session starts cold, with no compiled understanding of the enterprise data estate — which tables relate to which, which sources are authoritative for which questions, and which formats an agent downstream will actually be able to consume. Every session re-discovers that from scratch.

“At the heart of all this stuff was a very simple problem,” Ashutosh said. “You’re asking agents — machines — to work on systems and data that was designed for humans.”

Pinecone estimates that 85% of agent compute effort goes to the re-discovery cycle rather than task completion. The downstream effects compound: unpredictable latency, runaway token costs, and non-deterministic results. Run the same task twice against the same data, and an agent may return different answers with no record of which sources drove either result. For enterprises where auditability is a compliance requirement, that is a structural disqualifier, not a tuning problem.

What Nexus is and how it works

Nexus moves reasoning work from inference time to compilation time. In a conventional RAG pipeline, the reasoning required to interpret, contextualize, and structure knowledge happens at the moment an agent queries — every session, every time, burning tokens on work that could have been done in advance. But Nexus reasons just once during a compilation stage that runs before any agent query, then stores the result as a reusable knowledge artifact. The agent receives structured, task-ready context rather than raw documents to interpret on the fly.

The architecture Pinecone is shipping has three distinct components, each addressing a different layer of the agent retrieval problem.

1. Context compiler. Nexus takes raw source data and a task specification and builds specialized knowledge artifacts — structured, task-optimized representations that agents consume directly without interpretation overhead. The same underlying data estate produces different artifacts for different agents: a sales agent gets deal context synthesized from CRM and call records, a finance agent gets revenue context linking contracts to billing schedules. Artifacts are persistent and reused across agent sessions, not regenerated at inference time.

2. Composable retriever. Compiled artifacts are served at query time with typed fields, per-field citations with confidence levels, and deterministic conflict resolution. Output is shaped to match the agent’s specified format rather than returned as raw text for the agent to re-parse.

3. KnowQL. Pinecone describes this as the first declarative query language designed for agents rather than humans. Six primitives — intent, filter, provenance, output shape, confidence, and budget — allow agents to specify structured responses and source grounding and latency envelopes in a single interface. Ashutosh compared the structural gap that KnowQL fills to what SQL did for relational databases: Before a standard interface existed, every application built its own data access layer from scratch.

The relationship between Nexus and Pinecone’s underlying vector database is additive. The context compiler produces knowledge artifacts that are indexed and stored in the vector database; the compilation layer shapes and serves knowledge; the vector layer handles storage, retrieval speed, and scale.

 “The vectors are still stored and managed by the Pinecone vector database,” Ashutosh said.

What analysts make of the architectural claim

Moving reasoning upstream from inference to a compilation stage is not a novel concept — ontologies, data catalogs, and semantic layers have pursued versions of it for years. What has changed is the ability to do this at scale without dedicated engineering teams for every domain. That is the specific argument Nexus is making, and it is where analysts see the genuine advance.

Stephanie Walter, practice leader for AI stack at HyperFRAME Research, told VentureBeat that Nexus is directionally important because it shifts knowledge work from runtime chaos to pre-compiled structure. She stressed, however, that it is an evolution of RAG architecture, not a complete reinvention. 

“The real innovation isn’t the idea itself, but the productization of knowledge compilation as a first-class infrastructure layer,” Walter said. “If Pinecone can operationalize that reliably, it becomes meaningful infrastructure, not just another RAG tuning trick.”

The technical mechanism behind that claim is what Gartner distinguished VP analyst Arun Chandrasekaran called the meaningful architectural distinction.

“Unlike traditional RAG, which relies on pure semantic search at runtime, architectural compilation embeds structural logic into the metadata layer, which can boost time to response and provide better reasoning,” Chandrasekaran told VentureBeat. “This is an important leap from simple retrieval to enhanced reasoning, allowing agents to navigate enterprise schemas and acquire better memory for contextualization.”

The competitive landscape

Multiple vendors acknowledge that a vector database and traditional RAG are not enough for agentic AI.

Microsoft has extended its FabricIQ technology to provide semantic context for agentic AI. Google recently announced its Agentic Data Cloud as an approach to help solve the same issues. There are also standalone contextual memory technologies, like hindsight, that provide yet another option for users.

But analysts are less focused on the feature comparison than on what buyers should actually be evaluating.

“The agentic AI stack is fragmenting into dozens of features, but enterprise buyers shouldn’t chase features,” Walter said. “They should chase control: cost control, governance control, and security control.”

Most enterprise failures in agentic AI, she argued, will not be technical. They will be operational — tied to cost overruns, governance gaps, and security discipline.

The capability bar goes beyond retrieval speed.

“The true differentiator is deterministic grounding,” Chandrasekaran said, pointing to techniques like knowledge graphs that ensure agents understand structural relationships within enterprise data rather than returning surface-level matches. Interoperability is a related consideration: Standards like model context protocol (MCP) matter for connecting agents to legacy data sources without creating new dependencies.

What this means for enterprises

RAG and standalone vector databases were built for a different era. Agentic workloads are exposing the limits of both.

The retrieval cost problem is architectural

Teams running complex agentic workloads on conventional RAG pipelines are burning tokens at inference time on work that could be done in advance — interpreting, contextualizing, and structuring knowledge, every session, from scratch. That is a design problem. Tuning the retrieval layer will not fix it. The question for data engineering teams is whether their current stack is structurally capable of pre-compiling knowledge for specific agent tasks, or whether it was built for a human user who never needed that capability.

Governance is what separates a pilot from a production deployment

The capabilities that determine whether agentic AI gets approved for enterprise use are not performance metrics.

“The real enterprise value proposition isn’t just faster retrieval, but governed knowledge pipelines,” Walter said. “Those are the capabilities that turn agentic AI from an experiment into something finance and risk teams will actually approve.” 

The budget has shifted

VentureBeat’s Q1 Pulse data shows that retrieval optimization investment rose to 28.9% in March, overtaking evaluation spending for the first time in the quarter. Enterprises have finished measuring their retrieval problems. They are now spending to fix them. 

“The future of agentic AI won’t be decided by who has the longest context window,” Walter said. “It will be decided by who can operationalize trusted knowledge at scale without blowing up cost or governance.”

Something shifted in enterprise RAG in Q1 2026. VB Pulse data spanning January through March tells a consistent story: the market stopped adding retrieval layers and started fixing the ones it already has. Call it the retrieval rebuild.

The survey covered three consecutive monthly waves from organizations with 100 or more employees, with between 45 and 58 qualified respondents per month across platform adoption, buyer intent, architecture outlook and evaluation criteria. The data should be treated as directional.

Enterprise intent to adopt hybrid retrieval tripled from 10.3% to 33.3% in a single quarter — even as 22% of qualified enterprise respondents reported having no production RAG systems at all. For data engineers and enterprise architects building agentic AI infrastructure, the data reveals a market in active transition: the RAG architecture most enterprises built to scale is not the one they expect to run by year-end. 

Hybrid retrieval has become the consensus enterprise strategy. Unlike single-method RAG pipelines that rely on vector similarity alone, hybrid retrieval combines dense embeddings with sparse keyword search and reranking layers, trading simplicity for the retrieval accuracy and access control that production agentic workloads require.

The standalone vector database category is under pressure. Weaviate, Milvus, Pinecone and Qdrant each lost adoption share across the quarter in the VB Pulse data. Custom stacks and provider-native retrieval are absorbing their displaced share.

A growing minority of enterprises are stepping back from RAG altogether — a signal that the market’s maturity narrative has meaningful exceptions.

Organizations that went wide on RAG in 2025 are hitting the same failure point: the architecture built for document retrieval does not hold at agentic scale.

Enterprises that scaled RAG fast are now paying to rebuild it

The two largest intent movements in Q1 are directly connected — enterprises confronting retrieval quality problems at scale, and hybrid retrieval emerging as the consensus answer.

Investment priorities shifted in parallel. Evaluation and relevance testing led budget intent in January at 32.8% and fell to 15.6% by March. Retrieval optimization moved in the opposite direction, from 19.0% to 28.9% — overtaking evaluation as the top growth investment area for the first time. 

Steven Dickens, vice president and practice lead at HyperFRAME Research, described the operational burden enterprise data teams are facing in a VentureBeat interview in March on Oracle’s agentic AI data stack. “Data teams are exhausted by fragmentation fatigue,” Dickens said. “Managing a separate vector store, graph database and relational system just to power one agent is a DevOps nightmare.”

That fatigue shows directly in the platform data. The custom stack rise to 35.6% is not a rejection of managed retrieval — many organizations run both. It is a consolidation response from engineering teams that have hit the limits of assembling too many components.

Not every enterprise has made it that far. The VB Pulse data includes a signal that complicates the market’s overall growth narrative: 22.2% of qualified respondents reported no production RAG by March, up from 8.6% in January.  The report attributes this cohort to organizations that have “not yet committed to any retrieval infrastructure, or have paused programs” — concentrated in Healthcare, Education and Government, the same sectors showing the highest rates of flat budgets.

Standalone vector databases are losing the adoption argument but winning the reliability one

Recent reporting by VentureBeat illustrates why the dedicated retrieval layer still matters in production. 

Two enterprises building on Qdrant show why purpose-built vector infrastructure still wins in production.

 &AI builds patent litigation infrastructure and runs semantic search across hundreds of millions of documents. Grounding every result in a real source document is not optional — patent attorneys will not act on AI-generated text. That requirement makes the architectural choice clear.

“The agent is the interface,” Herbie Turner, &AI’s founder and CTO, told VentureBeat in March. “The vector database is the ground truth.”

GlassDollar, a startup that helps Siemens and Mahle evaluate startups, runs an agentic retrieval pattern across a corpus approaching 10 million indexed documents. A single user prompt fans out into multiple parallel queries, each retrieving candidates from a different angle before results are combined and re-ranked. That query volume and precision requirement is what drove the choice of purpose-built vector infrastructure.

“We measure success by recall,” Kamen Kanev, GlassDollar’s head of product, told VentureBeat in March. “If the best companies aren’t in the results, nothing else matters. The user loses trust.”

The VB Pulse data shows that framing — retrieval as ground truth rather than feature — is gaining traction across the broader enterprise market, even as standalone vector database adoption declines. 

Why enterprises say they need a dedicated vector layer shifted significantly across Q1. In January the top reasons were access control complexity (20.7%) and retrieval precision (19.0%). By March, operational reliability at scale had surged to 31.1% — more than doubling and overtaking everything else. Enterprises are no longer keeping vector infrastructure primarily for precision. They are keeping it because it is the part of the stack they can rely on when query volumes scale.

How enterprises are redefining what good retrieval means

How enterprises judge their retrieval systems shifted notably across Q1 — and the direction of that shift points to a market getting more sophisticated about what good retrieval actually means.

In January, response correctness dominated evaluation criteria at 67.2% — far above anything else. By March, response correctness (53.3%), retrieval accuracy (53.3%) and answer relevance (53.3%) had converged exactly. Getting the right answer is no longer enough if it came from the wrong document or missed the context of the question.

Answer relevance was the only criterion that rose across the quarter, gaining five percentage points. It is also the hardest to measure — whether the retrieved context is actually the right context for that specific question requires purpose-built evaluation infrastructure, not just pass-or-fail correctness checks. Its rise signals that a meaningful share of enterprise buyers have moved past basic RAG testing entirely. 

The market’s verdict: RAG isn’t dead. The original architecture is

The “RAG is dead” narrative had real momentum heading into 2026. It rested on two claims. The first: that long-context windows — models capable of processing hundreds of thousands of tokens in a single prompt — would make dedicated retrieval unnecessary. The second: that agentic memory systems, which store what an agent learns across sessions rather than retrieving it fresh each time, would absorb the knowledge access problem entirely.

The VB Pulse data is the enterprise market’s answer to the first claim. The long-context-as-dominant-architecture position collapsed from 15.5% in January to 3.5% in February before partially recovering to 6.7% in March. January’s sample was heavily weighted toward Technology and Software respondents — the segment most exposed to long-context model announcements in late 2025. As the sample diversified, the position evaporated.

On the memory question, Jonathan Frankle, chief AI scientist at Databricks, framed the architecture clearly in a March interview with VentureBeat: a vector database with millions of entries sits at the base of the agentic memory stack, too large to fit in context. The LLM context window sits at the top. Between them, new caching and compression layers are emerging — but none of them replace the retrieval layer at the base. New agentic memory systems like Hindsight, developed by Vectorize, and observational memory approaches like those in the Mastra framework address session continuity and agent context over time — a different problem than high-recall search across millions of changing enterprise documents.

The most consequential signal: the share of respondents not expecting large-scale RAG deployments by year-end grew from 3.4% to 15.6% — nearly 5x. That is not a verdict against retrieval. It is a verdict against the retrieval architecture most enterprises built first.

The retrieval rebuild is not optional

The retrieval rebuild is the cost of scaling RAG without first deciding what architecture could actually support it.

If your organization is among the 43.1% that entered Q1 planning to expand RAG into more workflows, the VB Pulse data suggests that plan has already changed for many of your peers — and may need to change for you. Hybrid retrieval is the consensus destination. Custom stack growth to 35.6% reflects teams building retrieval infrastructure around requirements that off-the-shelf products do not fully address.

RAG is not dead. The architecture most enterprises used to implement it is. The data suggests the rebuild is not a future decision. For 33% of enterprises, the rebuild is already the stated priority.

For most data engineering teams, managing pipeline reliability often means waiting for an alert, manually tracing failures across distributed jobs and clusters, and fixing problems after they’ve already hit the business. Agentic AI needs the data to be there, clean and on time. A pipeline that fails silently or delivers stale data doesn’t just break a dashboard — it breaks the AI system depending on it.

That gap is what Definity, a Chicago-based data pipeline operations startup, is building into: embedding agents directly inside the Spark or DBT driver to act during a pipeline run, not after it. One enterprise customer identified 33% of its optimization opportunities in the first week of deployment and cut troubleshooting and optimization effort by 70%, according to Definity. The company also claims customers are resolving complex Spark issues up to 10x faster.

“You need three big things for agentic data operations: full stack context that is real time and production aware. Control of the pipeline. And the ability to validate in a feedback loop. Without that, you can be outside looking in and read only,” Roy Daniel, CEO and co-founder of Definity told VentureBeat in an exclusive interview.

The company on Wednesday announced that it has raised $12 million in Series A financing led by GreatPoint Ventures, with participation from Dynatrace and existing investors StageOne Ventures and Hyde Park Venture Partners.

Why existing pipeline monitoring breaks down at scale

Existing tools approach the problem from outside the execution layer — Datadog, which acquired data quality monitor Metaplane last year, Databricks system tables, and platforms like Unravel Data and Acceldata all read metrics after a job completes. Dynatrace has monitoring capabilities; it also participated in Definity’s Series A.

The Definity approach is differentiated from other options in the way the solution is architected. According to Daniel, that means by the time a platform monitoring tool surfaces a problem, the pipeline has already run — and the failure, the wasted compute or the bad data is already downstream.

“It’s always after the fact,” Daniel said. “By the time you know something happened, it already happened.”

How Definity’s in-execution agents work

The core architectural difference is where the agent sits — inside the pipeline rather than watching from outside it.

Inline instrumentation. The Definity system installs a JVM agent directly inside the pipeline execution layer via a single line of code, running below the platform layer and pulling execution data directly from Spark.

Execution context during the run. The agent captures query execution behavior, memory pressure, data skew, shuffle patterns and infrastructure utilization as the pipeline runs. It also infers lineage between pipelines and tables dynamically — no predefined data catalog is required.

Intervention, not just observation. The agent can modify resource allocation mid-run, stop a job before bad data propagates or preempt a pipeline based on upstream data conditions. Daniel described one production deployment where the agent detected that an upstream job had been preempted and the input table it was supposed to write was stale — and stopped the downstream pipeline before it started, before bad data reached any dependent system.

What is and isn’t real time. Detection and prevention are real time. Root cause analysis and optimization recommendations run on demand when an engineer queries the assistant, with full execution context already assembled.

Overhead and data residency. The agent adds approximately one second of compute on an hour-long run. Only metadata transmits externally; full on-premises deployment is available for environments where no metadata can leave the perimeter.

What in-execution intelligence looks like in a production environment

One early user of the Definity platform is Nexxen, an ad tech platform running large-scale Spark pipelines  for mission-critical advertising workloads, running on-premises.

Dennis Meyer, Director of Data Engineering at Nexxen, told VentureBeat that the core problem he was facing was not pipeline failures but the accumulating cost of inefficiency in an environment with no elastic cloud capacity to absorb waste.

“The main challenge wasn’t about pipelines breaking, but about managing an increasingly complex and large-scale environment,” Meyer said. “Because we operate on-prem, we don’t have the flexibility of instant elasticity, so inefficiencies have a direct cost impact.”

Existing monitoring tools gave Nexxen partial visibility but not enough to act on systematically. “We had existing monitoring tools in place, but needed full-stack visibility to understand workload behavior holistically and to systematically prioritize optimizations,” Meyer said.

Nexxen deployed Definity with no pipeline code changes. According to Meyer, the team identified 33% of its optimization opportunities within the first week, and engineering effort on troubleshooting and optimization dropped by 70%. The platform freed infrastructure capacity, allowing the team to support workload growth without additional hardware investment.

“The key shift was moving from reactive troubleshooting to proactive, continuous optimization,” Meyer said. “At scale, the biggest gap often isn’t tooling — it’s actionable visibility.”

What this means for enterprise data teams

For data engineering teams running production Spark environments, the shift from reactive monitoring to in-execution intelligence has architectural and organizational implications worth thinking through.

Pipeline ops is becoming an AI infrastructure problem. Data pipelines that previously supported analytics now carry AI workloads with direct business dependencies. Failures that were once an inconvenience are now blocking production AI delivery.

Troubleshooting time is a recoverable cost. According to Meyer, Nexxen cut engineering effort on troubleshooting and optimization by 70% after deploying Definity. For teams running lean, that time going back to the roadmap is the most direct near-term case for evaluating this category.

Enterprise teams that fine-tune their RAG embedding models for better precision may be unintentionally degrading the retrieval quality those pipelines depend on, according to new research from Redis.

The paper, “Training for Compositional Sensitivity Reduces Dense Retrieval Generalization,” tested what happens when teams train embedding models for compositional sensitivity. That is the ability to catch sentences that look nearly identical but mean something different — “the dog bit the man” versus “the man bit the dog,” or a negation flip that reverses a statement’s meaning entirely. That training consistently broke dense retrieval generalization, how well a model retrieves correctly across broad topics and domains it wasn’t specifically trained on. Performance dropped by 8 to 9 percent on smaller models and by 40 percent on a current mid-size embedding model teams are actively using in production.

The findings have direct implications for enterprise teams building agentic AI pipelines, where retrieval quality determines what context flows into an agent’s reasoning chain. A retrieval error in a single-stage pipeline returns a wrong answer. The same error in an agentic pipeline can trigger a cascade of wrong actions downstream.

Srijith Rajamohan, AI Research Leader at Redis and one of the paper’s authors, said the finding challenges a widespread assumption about how embedding-based retrieval actually works. 

“There’s this general notion that when you use semantic search or similar semantic similarity, we get correct intent. That’s not necessarily true,” Rajamohan told VentureBeat. “A close or high semantic similarity does not actually mean an exact intent.”

The geometry behind the retrieval tradeoff

Embedding models work by compressing an entire sentence into a single point in a high-dimensional space, then finding the closest points to a query at retrieval time. That works well for broad topical matching — documents about similar subjects end up near each other. The problem is that two sentences with nearly identical words but opposite meanings also end up near each other, because the model is working from word content rather than structure.

That is what the research quantified. When teams fine-tune an embedding model to push structurally different sentences apart — teaching it that a negation flip which reverses a statement’s meaning is not the same as the original — the model uses representational space it was previously using for broad topical recall. The two objectives compete for the same vector.

The research also found the regression is not uniform across failure types. Negation and spatial flip errors improved measurably with structured training. Binding errors — where a model confuses which modifier applies to which word, such as which party a contract obligation falls on — barely moved. For enterprise teams, that means the precision problem is harder to fix in exactly the cases where getting it wrong has the most consequences.

The reason most teams don’t catch it is that fine-tuning metrics measure the task being trained for, not what happens to general retrieval across unrelated topics. A model can show strong improvement on near-miss rejection during training while quietly regressing on the broader retrieval job it was hired to do. The regression only surfaces in production.

Rajamohan said the instinct most teams reach for — moving to a larger embedding model — does not address the underlying architecture.

“You can’t scale your way out of this,” he said. “It’s not a problem you can solve with more dimensions and more parameters.”

Why the standard alternatives all fall short

The natural instinct when retrieval precision fails is to layer on additional approaches. The research tested several of them and found each fails in a different way.

Hybrid search. Combining embedding-based retrieval with keyword search is already standard practice for closing precision gaps. But Rajamohan said keyword search cannot catch the failure mode this research identifies, because the problem is not missing words — it is misread structure.

“If you have a sentence like ‘Rome is closer than Paris’ and another that says ‘Paris is closer than Rome,’ and you do an embedding retrieval followed by a text search, you’re not going to be able to tell the difference,” he said. “The same words exist in both sentences.”

MaxSim reranking. Some teams add a second scoring layer that compares individual query words against individual document words rather than relying on the single compressed vector. This approach, known as MaxSim or late interaction and used in systems like ColBERT, did improve relevance benchmark scores in the research. But it completely failed to reject structural near-misses, assigning them near-identity similarity scores. 

The problem is that relevance and identity are different objectives. MaxSim is optimized for the former and blind to the latter. A team that adds MaxSim and sees benchmark improvement may be solving a different problem than the one they have.

Cross-encoders. These work by feeding the query and candidate document into the model simultaneously, letting it compare every word against every word before making a decision. That full comparison is what makes them accurate — and what makes them too expensive to run at production scale. Rajamohan said his team investigated them. They work in the lab and break under real query volumes.

Contextual memory. Also sometimes referred to as agentic memory, these systems are increasingly cited as the path beyond RAG, but Rajamohan said moving to that type of  architecture does not eliminate the structural retrieval problem. Those systems still depend on retrieval at query time, which means the same failure modes apply. The main difference is looser latency requirements, not a precision fix.

The two-stage fix the research validated

The common thread across every failed approach is the same: a single scoring mechanism trying to handle both recall and precision at once. The research validated a different architecture: stop trying to do both jobs with one vector, and assign each job to a dedicated stage.

Stage one: recall. The first stage works exactly as standard dense retrieval does today — the embedding model compresses documents into vectors and retrieves the closest matches to a query. Nothing changes here. The goal is to cast a wide net and bring back a set of strong candidates quickly. Speed and breadth are what matter at this stage, not perfect precision.

Stage two: precision. The second stage is where the fix lives. Rather than scoring candidates with a single similarity number, a small learned Transformer model examines the query and each candidate at the token level — comparing individual words against individual words to detect structural mismatches like negation flips or role reversals. This is the verification step the single-vector approach cannot perform.

The results. Under end-to-end training, the Transformer verifier outperformed every other approach the research tested on structural near-miss rejection. It was the only approach that reliably caught the failure modes the single-vector system missed.

The tradeoff. Adding a verification stage costs latency. The latency cost depends on how much verification a team runs. For precision-sensitive workloads like legal or accounting applications, full verification at every query is warranted. For general-purpose search, lighter verification may be sufficient. 

The research grew out of a real production problem. Enterprise customers running semantic caching systems were getting fast but semantically incorrect responses back — the retrieval system was treating similar-sounding queries as identical even when their meaning differed. The two-stage architecture is Redis’s proposed fix, with incorporation into its LangCache product on the roadmap but not yet available to customers.

What this means for enterprise teams

The research does not require enterprise teams to rebuild their retrieval pipelines from scratch. But it does ask them to pressure-test assumptions most teams have never examined — about what their embedding models are actually doing, which metrics are worth trusting and where the real precision gaps live in production.

Recognize the tradeoff before tuning around it. Rajamohan said the first practical step is understanding the regression exists. He evaluates any LLM-based retrieval system on three criteria: correctness, completeness and usefulness. Correctness failures cascade directly into the other two, which means a retrieval system that scores well on relevance benchmarks but fails on structural near-misses is producing a false sense of production readiness.

RAG is not obsolete — but know what it can’t do. Rajamohan pushed back firmly on claims that RAG has been superseded. “That’s a massive oversimplification,” he said. “RAG is a very simple pipeline that can be productionized by almost anyone with very little lift.” The research does not argue against RAG as an architecture. It argues against assuming a single-stage RAG pipeline with a fine-tuned embedding model is production-ready for precision-sensitive workloads.

The fix is real but not free. For teams that do need higher precision, Rajamohan said the two-stage architecture is not a prohibitive implementation lift, but adding a verification stage costs latency. “It’s a mitigation problem,” he said. “Not something we can actually solve.”

In a significant shift toward local-first privacy infrastructure, OpenAI has released Privacy Filter, a specialized open-source model designed to detect and redact personally identifiable information (PII) before it ever reaches a cloud-based server.

Launched today on AI code sharing community Hugging Face under a permissive Apache 2.0 license, the tool addresses a growing industry bottleneck: the risk of sensitive data “leaking” into training sets or being exposed during high-throughput inference.

By providing a 1.5-billion-parameter model that can run on a standard laptop or directly in a web browser, the company is effectively handing developers a “privacy-by-design” toolkit that functions as a sophisticated, context-aware digital shredder.

Though OpenAI was founded with a focus on open source models such as this, the company shifted during the ChatGPT era to providing more proprietary (“closed source”) models available only through its website, apps, and API — only to return to open source in a big way last year with the launch of the gpt-oss family of language models.

In that light, and combined with OpenAI’s recent open sourcing of agentic orchestration tools and frameworks, it’s safe to say that the generative AI giant is clearly still heavily invested in fostering this less immediately lucrative part of the AI ecosystem.

Technology: a gpt-oss variant with bidirectional token classifier that reads from both directions

Architecturally, Privacy Filter is a derivative of OpenAI’s gpt-oss family, a series of open-weight reasoning models released earlier this year.

However, while standard large language models (LLMs) are typically autoregressive—predicting the next token in a sequence—Privacy Filter is a bidirectional token classifier.

This distinction is critical for accuracy. By looking at a sentence from both directions simultaneously, the model gains a deeper understanding of context that a forward-only model might miss.

For instance, it can better distinguish whether “Alice” refers to a private individual or a public literary character based on the words that follow the name, not just those that precede it.

The model utilizes a Sparse Mixture-of-Experts (MoE) framework. Although it contains 1.5 billion total parameters, only 50 million parameters are active during any single forward pass.

This sparse activation allows for high throughput without the massive computational overhead typically associated with LLMs. Furthermore, it features a massive 128,000-token context window, enabling it to process entire legal documents or long email threads in a single pass without the need for fragmenting text—a process that often causes traditional PII filters to lose track of entities across page breaks.

To ensure the redacted output remains coherent, OpenAI implemented a constrained Viterbi decoder. Rather than making an independent decision for every single word, the decoder evaluates the entire sequence to enforce logical transitions.

It uses a “BIOES” (Begin, Inside, Outside, End, Single) labeling scheme, which ensures that if the model identifies “John” as the start of a name, it is statistically inclined to label “Smith” as the continuation or end of that same name, rather than a separate entity.

On-device data sanitization

Privacy Filter is designed for high-throughput workflows where data residency is a non-negotiable requirement. It currently supports the detection of eight primary PII categories:

• Private Names: Individual persons.

• Contact Info: Physical addresses, email addresses, and phone numbers.

• Digital Identifiers: URLs, account numbers, and dates.

• Secrets: A specialized category for credentials, API keys, and passwords.

In practice, this allows enterprises to deploy the model on-premises or within their own private clouds. By masking data locally before sending it to a more powerful reasoning model (like GPT-5 or gpt-oss-120b), companies can maintain compliance with strict GDPR or HIPAA standards while still leveraging the latest AI capabilities.

Initial benchmarks are promising: the model reportedly hits a 96% F1 score on the PII-Masking-300k benchmark out of the box.

For developers, the model is available via Hugging Face, with native support for transformers.js, allowing it to run entirely within a user’s browser using WebGPU.

Fully open source, commercially viable Apache 2.0 license

Perhaps the most significant aspect of the announcement for the developer community is the Apache 2.0 license. Unlike “available-weight” licenses that often restrict commercial use or require “copyleft” sharing of derivative works, Apache 2.0 is one of the most permissive licenses in the software world.For startups and dev-tool makers, this means:

1. Commercial Freedom: Companies can integrate Privacy Filter into their proprietary products and sell them without paying royalties to OpenAI.

2. Customization: Teams can fine-tune the model on their specific datasets (such as medical jargon or proprietary log formats) to improve accuracy for niche industries.

3. No Viral Obligations: Unlike the GPL license, builders do not have to open-source their entire codebase if they use Privacy Filter as a component.

By choosing this licensing path, OpenAI is positioning Privacy Filter as a standard utility for the AI era—essentially the “SSL for text”.

Community reactions

The tech community reacted quickly to the release, with many noting the impressive technical constraints OpenAI managed to hit.

Elie Bakouch (@eliebakouch), a research engineer at agentic model training platform startup Prime Intellect, praised the efficiency of Privacy Filter’s architecture on X:

“Very nice release by @OpenAI! A 50M active, 1.5B total gpt-oss arch MoE, to filter private information from trillion scale data cheaply. keeping 128k context with such a small model is quite impressive too”.

The sentiment reflects a broader industry trend toward “small but mighty” models. While the world has focused on massive, 100-trillion parameter giants, the practical reality of enterprise AI often requires small, fast models that can perform one task—like privacy filtering—exceptionally well and at a low cost.

However, OpenAI included a “High-Risk Deployment Caution” in its documentation. The company warned that the tool should be viewed as a “redaction aid” rather than a “safety guarantee,” noting that over-reliance on a single model could lead to “missed spans” in highly sensitive medical or legal workflows.

OpenAI’s Privacy Filter is clearly an effort by the company to make the AI pipeline fundamentally safer.

By combining the efficiency of a Mixture-of-Experts architecture with the openness of an Apache 2.0 license, OpenAI is providing a way for many enterprises to more easily, cheaply and safely redact PII data.

Enterprise data stacks were built for humans running scheduled queries. As AI agents increasingly act autonomously on behalf of businesses around the clock, that architecture is breaking down — and vendors are racing to rebuild it. Google’s answer, announced at Cloud Next on Wednesday, is the Agentic Data Cloud.

The architecture has three pillars:

• Knowledge Catalog. Automates semantic metadata curation, inferring business logic from query logs without manual data steward intervention

• Cross-cloud lakehouse. Lets BigQuery query Iceberg tables on AWS S3 via private network with no egress fees

• Data Agent Kit. Drops MCP tools into VS Code, Claude Code and Gemini CLI so data engineers describe outcomes rather than write pipelines

“The data architecture has to change now,” Andi Gutmans, VP and GM of Data Cloud at Google Cloud, told VentureBeat. “We’re moving from human scale to agent scale.”

From system of intelligence to system of action

The core premise behind Agentic Data Cloud is that enterprises are moving from human‑scale to agent‑scale operations.

Historically, data platforms have been optimized for reporting, dashboarding, and some forecasting — what Google characterizes as “reactive intelligence.” In that model, humans interpret data and decide what to do.

Now, with AI agents increasingly expected to take actions directly on behalf of the business, Gutmans argued that data platforms must evolve into systems of action.

“We need to make sure that all of enterprise data can be activated with AI, that includes both structured and unstructured data,” Gutmans said. “We need to make sure that there’s the right level of trust, which also means it’s not just about getting access to the data, but really understanding the data.”

The Knowledge Catalog is Google’s answer to that problem. It is an evolution of Dataplex, Google’s existing data governance product, with a materially different architecture underneath. Where traditional data catalogs required data stewards to manually label tables, define business terms and build glossaries, the Knowledge Catalog automates that process using agents.

The practical implication for data engineering teams is that the Knowledge Catalog scales to the full data estate, not just the curated subset that a small team of data stewards can maintain by hand. The catalog covers BigQuery, Spanner, AlloyDB and Cloud SQL natively, and federates with third-party catalogs including Collibra, Atlan and Datahub. Zero-copy federation extends semantic context from SaaS applications including SAP, Salesforce Data360, ServiceNow and Workday without requiring data movement.

Google’s lakehouse goes cross cloud

Google has had a data lakehouse called BigLake since 2022. Initially it was limited to just Google data, but in recent years has had some limited federation capabilities enabling enterprises to query data found in other locations.

Gutmans explained that the previous federation worked through query APIs, which limited the features and optimizations BigQuery could bring to bear on external data. The new approach is storage-based sharing via the open Apache Iceberg format. That means whether the data is in Amazon S3 or in Google Cloud , he argued it doesn’t make a difference.

“This truly means we can bring all the goodness and all the AI capabilities to those third-party data sets,” he said.

The practical result is that BigQuery can query Iceberg tables sitting on Amazon S3 via Google’s Cross-Cloud Interconnect, a dedicated private networking layer, with no egress fees and price-performance Google says is comparable to native AWS warehouses. All BigQuery AI functions run against that cross-cloud data without modification. Bidirectional federation in preview extends to Databricks Unity Catalog on S3, Snowflake Polaris and the AWS Glue Data Catalog using the open Iceberg REST Catalog standard.

From writing pipelines to describing outcomes

The Knowledge Catalog and cross-cloud lakehouse solve the data access and context problems. The third pillar addresses what happens when a data engineer actually sits down to build something with all of it.

The Data Agent Kit ships as a portable set of skills, MCP tools and IDE extensions that drop into VS Code, Claude Code, Gemini CLI and Codex. It does not introduce a new interface.

The architectural shift it enables is a move from what Gutmans called a “prescriptive copilot experience” to intent-driven engineering. Rather than writing a Spark pipeline to move data from source A to destination B, a data engineer describes the outcome — a cleaned dataset ready for model training, a transformation that enforces a governance rule — and the agent selects whether to use BigQuery, the Lightning Engine for Apache Spark or Spanner to execute it, then generates production-ready code.

“Customers are kind of sick of building their own pipelines,” Gutmans said. “They’re truly more in the review kind of mode, than they are in the writing the code mode.”

Where Google and its rivals diverge

The premise that agents require semantic context, not just data access, is shared across the market. 

Databricks has Unity Catalog, which provides governance and a semantic layer across its lakehouse. Snowflake has Cortex, its AI and semantic layer offering. Microsoft Fabric includes a semantic model layer built for business intelligence and, increasingly, agent grounding.

The dispute is not over whether semantics matter — everyone agrees they do. The dispute is over who builds and maintains them.

“Our goal is just to get all the semantics you can get,” he explained, noting that Google will federate with third-party semantic models rather than require customers to start over.

Google is also positioning openness as a differentiator, with bidirectional federation into Databricks Unity Catalog and Snowflake Polaris via the open Iceberg REST Catalog standard.

What this means for enterprises

Google’s argument — and one echoed across the data infrastructure market — is that enterprises are behind on three fronts:

Semantic context is becoming infrastructure. If your data catalog is still manually curated, it will not scale to agent workloads — and Gutmans argues that gap will only widen as agent query volumes increase.

Cross-cloud egress costs are a hidden tax on agentic AI. Storage-based federation via open Iceberg standards is emerging as the architectural answer across Google, Databricks and Snowflake. Enterprises locked into proprietary federation approaches should be stress-testing those costs at agent-scale query volumes.

Gutmans argues the pipeline-writing era is ending. Data engineers who move toward outcome-based orchestration now will have a significant head start.

Data teams building AI agents keep running into the same failure mode. Questions that require joining structured data with unstructured content, sales figures alongside customer reviews or citation counts alongside academic papers, break single-turn RAG systems. 

New research from Databricks puts a number on that failure gap. The company’s AI research team tested a multi-step agentic approach against state-of-the-art single-turn RAG baselines across nine enterprise knowledge tasks, reporting gains of 20% or more on Stanford’s STaRK benchmark suite and consistent improvement across Databricks’ own KARLBench evaluation framework. The results make the case that the performance gap between single-turn RAG and multi-step agents on hybrid data tasks is an architectural problem, not a model quality problem.

The work builds on Databricks’ earlier instructed retriever research, which showed retrieval improvements on unstructured data using metadata-aware queries. This latest research adds structured data sources, relational tables and SQL warehouses, into the same reasoning loop, addressing the class of questions enterprises most commonly fail to answer with current agent architectures.

“RAG works, but it doesn’t scale,” Michael Bendersky, research director at Databricks, told VentureBeat. “If you want to make your agent even better, and you want to understand why you have declining sales, now you have to help the agent see the tables and look at the sales data. Your RAG pipeline will become incompetent at that task.”

Single-turn retrieval cannot encode structural constraints

The core finding is that standard RAG systems fail when a query mixes a precise structured filter with an open-ended semantic search. 

Consider a question like “Which of our products have had declining sales over the past three months, and what potentially related issues are brought up in customer reviews on various seller sites?” The sales data lives in a warehouse. The review sentiment lives in unstructured documents across seller sites. A single-turn RAG system cannot split that query, route each half to the right data source and combine the results.

To confirm this is an architecture problem rather than a model quality problem, Databricks reran published STaRK baselines using a current state-of-the-art foundation model. The stronger model still lost to the multi-step agent by 21% on the academic domain and 38% on the biomedical domain. 

STaRK is a benchmark published by Stanford researchers covering three semi-structured retrieval domains: Amazon product data, the Microsoft Academic Graph and a biomedical knowledge base. 

How the Supervisor Agent handles what RAG cannot

Databricks built the Supervisor Agent as the production implementation of this research approach, and its architecture illustrates why the gains are consistent across task types. The approach includes three core steps:

Parallel tool decomposition. Rather than issuing one broad query and hoping the results cover both structured and unstructured needs, the agent fires SQL and vector search calls simultaneously, then analyzes the combined results before deciding what to do next. That parallel step is what allows it to handle queries that cross data type boundaries without requiring the data to be normalized first.

Self-correction. When an initial retrieval attempt hits a dead end, the agent detects the failure, reformulates the query and tries a different path. On a STaRK benchmark task that requires finding a paper by an author with exactly 115 prior publications on a specific topic, the agent first queries both SQL and vector search in parallel. When the two result sets show no overlap, it adapts and issues a SQL JOIN across both constraints, then calls the vector search system to verify the result before returning the answer.

Declarative configuration.  The agent is not tuned to any specific dataset or task. Connecting it to a new data source means writing a plain-language description of what that source contains and what kinds of questions it should answer. No custom code is required.

“The agent can do things like decomposing the question into a SQL query and a search query out of the box,” Bendersky said. “It can combine the results of SQL and RAG, reason about those results, make follow-up queries and then reason about whether the final answer was actually found.”

It’s not just about hybrid retrieval

Being able to source information from both structured and unstructured data isn’t an entirely new concept.

LlamaIndex, LangChain and Microsoft Fabric agents all offer some form of hybrid retrieval. Bendersky draws a distinction in how the Databricks approach frames the problem architecturally.

“We almost don’t see it as a hybrid retrieval where you combine embeddings and search results, or embeddings and tables,” he said. “We see this more as an agent that has access to multiple tools.”

The practical consequence of that framing is that adding a new data source means connecting it to the agent and writing a description of what it contains. The agent handles routing and orchestration without additional code. 

Custom RAG pipelines require data to be converted into a format the retrieval system can read, typically text chunks with embeddings. SQL tables have to be flattened, JSON has to be normalized. Every new data source added to the pipeline means more conversion work. Databricks’ research argues that as enterprise data grows to include more source types, that burden makes custom pipelines increasingly impractical compared to an agent that queries each source in its native format.

“Just bring the agent to the data,” Bendersky said. “You basically give the agent more sources, and it will learn to use them pretty well.”

What this means for enterprises

For data engineers evaluating whether to build custom RAG pipelines or adopt a declarative agent framework, the research offers a clear direction: if the task involves questions that span structured and unstructured data, building custom retrieval is the harder path. The research found that across all tested tasks, the only things that differed between deployments were instructions and tool descriptions. The agent handled the rest.

The practical limits are real but manageable. The approach works well with five to ten data sources. Adding too many at once, without curating which sources are complementary rather than contradictory, makes the agent slower and less reliable. Bendersky recommends scaling incrementally and verifying results at each step rather than connecting all available data upfront.

Data accuracy is a prerequisite. The agent can query across mismatched formats, JSON review feeds alongside SQL sales tables, without requiring normalization. It cannot fix source data that is factually wrong. Adding a plain-language description of each data source at ingestion time helps the agent route queries correctly from the start.

The research positions this as an early step in a longer trajectory. As enterprise AI workloads mature, agents will be expected to reason across dozens of source types, including dashboards, code repositories and external data feeds. The research argues the declarative approach is what makes that scaling tractable, because adding a new source stays a configuration problem rather than an engineering one.

“This is kind of like a ladder,” Bendersky said. “The agent will slowly get more and more information and then slowly improve overall.”