The enterprise AI industry has a math problem. Cisco data shows 85% of enterprises are piloting AI agents, but only 5% have shipped them to production. At VB Transform 2026 on Tuesday, Bryan Silverthorn, Director of AGI Autonomy at Amazon, explained why that gap persists — and why the answer isn’t better benchmarks.
Silverthorn, who joined Amazon through its acquisition of Adept AI and now leads multimodal agent training inside the company’s AGI lab, argued that reliability must be broken into four distinct dimensions: consistency, robustness, predictability, and safety — a framework he credits to research from Princeton.
“It unpacks different factors that I see tangled together in almost every eval I’ve ever seen,” he said.
The framework matters because agents routinely ace internal evaluations and then collapse in the wild. Silverthorn described a customer that deployed an agent for software QA involving serial number extraction from screens. It worked flawlessly for two months — then began intermittently reading wrong numbers. The culprit: the underlying vision encoder behaved differently depending on where the serial number appeared on screen, and a software change imperceptible to humans triggered the failure.
The lesson, Silverthorn said, is about measurement, not just models. “The models have to be better. Obviously, we’re working hard on making the models better,” he said. But the deeper takeaway, he added, is that teams need to identify their dimensions of variability and match measurement rigor to the stakes of the application. VentureBeat’s own proprietary research, presented before the session, reinforces the point: half of surveyed companies shipped agents that passed internal evals but failed real customers, and enterprises overwhelmingly track uptime while ignoring accuracy — checking the pulse without checking the diagnosis. A related finding underscored how few guardrails exist: most enterprises default to the model makers’ own evaluations and little else, leaving their testing strategy, as I described it on stage, a coin flip between trusting the vendor and trusting nothing.
Silverthorn’s most memorable prescription was cultural, not technical. Inside Amazon’s AGI lab, researchers literally call their agents “interns” — as in, “I’ll have my intern talk to your intern.” The joke carries a serious operational philosophy. Agents, like interns, are powerful but occasionally clueless, capable of amazing work and spectacular derailment.
Managing them, he argued, requires management skills rather than software skills: asking what could go wrong, adding backups and undo capabilities, and consciously deciding what risk you can accept. “You can ask the intern, ‘Hey, what might you do wrong here? How might you mitigate your negative outcomes?'” he said. Amazon’s lab has embraced that trade-off, accepting agents occasionally running the wrong experiment in exchange for research velocity — including one agent running experiments around the clock on its own high-level research plan.
Silverthorn was candid about the limits of today’s technology. Self-improving AI remains “a loaded term,” he said — Amazon uses AI to improve its models constantly, but fully autonomous self-improvement is distant. Computer use remains a core focus of his lab, with a commercial trucking customer already using browser automation to stitch together warranty claims across fragmented systems**, though he stressed that no future agent will rely on computer use alone — it will work alongside MCP, APIs, and other tools to complete end-to-end workflows**. And LLM-as-judge techniques, while promising, are just one of several strategies for aligning agent capability with acceptable risk.
For enterprises stuck in pilot purgatory, the path forward starts with a mindset shift: stop asking whether your agent can do something impressive once, and start asking whether it can do it correctly a thousand times in a row.
In other words, the enterprises that escape the 85% ceiling won’t be the ones with the smartest agents. They’ll be the ones with the best managers.
Hundreds of enterprise leaders and technical experts packed the main ballroom of the luxurious Hotel Nia in Menlo Park this week for VB Transform 2026, the year’s preeminent conference on using generative AI agents to drive business outcomes.
Rachad Alao, vice president of product engineering at the rising Canadian enterprise AI startup Cohere, joined VentureBeat CEO and editor-in-chief Matt Marshall for a fireside chat about building agentic systems without surrendering sensitive data, infrastructure control, or the ability to change vendors.
Alao, who previously led responsible AI and trust and safety engineering teams at Google and Meta, argued that AI sovereignty means more than downloading an open model or running an application behind a corporate firewall.
Asked how Cohere defines sovereignty, Alao pointed to organizations operating mission-critical systems, including banks, hospitals and governments.
“It is important to have very tight control on where the data resides, have tight control on the AI,” he said, adding that AI operations should take place in jurisdictions an organization understands or directly controls.
That extends from GPUs and private-cloud infrastructure through governance systems that route requests among models, as well as the connectors, search tools and agent frameworks acting on enterprise data.
“You want to have control on the entire stack,” Alao said.
Marshall challenged one of the central economic arguments for smaller, locally deployed models: Inference prices continue to fall rapidly, potentially weakening the case for optimizing every token.
Alao countered that total consumption is climbing even faster as enterprises move from relatively simple chatbots to agents that reason through problems, call tools, search internal systems and take multiple steps before returning an answer.
“Your token utilization is going exponentially up, because you’re dealing with more and more complex agentic use cases,” he said. Those workflows require “a lot of processing, thinking, tools interaction” to complete their objectives, he added.
Alao also drew a contrast between providers that bill customers according to token consumption and Cohere’s approach.
“If your whole way of charging customers is for token utilization, you want to maximize token utilization,” he said. “We do not sell our models and our platform that way.”
Instead, Alao said Cohere tries to help enterprises solve their hardest problems privately and securely while reducing unnecessary model usage. His prescription was straightforward: “Use the right model for the task at hand.”
Rather than sending every request to the largest available frontier model, enterprises should route work according to the intelligence required and the sensitivity or regulatory burden attached to the task.
Alao cited an unnamed Canadian bank that uses Cohere’s on-premises models for highly regulated workloads, while sending less sensitive tasks requiring greater intelligence through Cohere’s North platform to larger frontier models.
“So model routing can become super useful,” he said.
Asked by an audience member how Cohere’s open-source North Mini Code, released last month, could compete against proprietary coding models, Alao acknowledged that larger frontier models may perform somewhat better on the hardest tasks.
But that advantage may not justify using them indiscriminately.
“For 80% of the use cases that they needed, this was a lot more effective, a lot cheaper,” Alao said of developers adopting the model.
Cohere’s North Mini Code runs on a single Nvidia H100 GPU and targets agentic software engineering, including terminal work, code review and tool use.
The company has also released Command A+, a 218-billion-parameter mixture-of-experts model with only 25 billion parameters active during each generation step.
Its compressed four-bit version reduces the hardware required for private deployment, while its Apache 2.0 license gives enterprises broad freedom to operate and modify it.
Asked about Cohere’s longstanding work on embeddings and enterprise search, Alao said the field is moving beyond retrieving text and inserting it into a model’s context window.
“Today, the state of the art is around multimodal search,” he said. “It’s beyond just the text modality.”
Search across documents, images and other forms of information is becoming “an integral component of your agentic workflow,” Alao added, with the model deciding when and how to use retrieval like any other tool.
Asked what would persuade enterprises to move beyond bundled AI services from existing cloud providers, Alao returned to data control and portability.
“If you’re interested in sovereignty, you want to have more control on your data,” he said. Cohere’s governance layer, he added, lets customers route traffic to appropriate models, “breaking that vendor lock-in concern that a lot of our customers have.”
Organizations need to transform to meet the needs of agentic AI.
Meta VP of Engineering Barak Yagour opened his talk at VB Transform 2026 wearing a pair of Ray-Ban Meta AI glasses, a small sign of how far AI has already worked its way into physical life. His argument went further: enterprise infrastructure was built for humans, not for agents, and it’s starting to show.
Yagour, who leads its data infrastructure organization, told the audience that agentic queries hitting Meta’s data systems grew 30x in a single half, an inversion that he said is breaking assumptions the company spent two decades building around.
The shift is not confined to Meta. Automated traffic overtook human traffic on the internet last year, reaching 51% of the total, according to Imperva’s 2025 Bad Bot Report. That traffic is also growing roughly eight times faster than human traffic, according to HUMAN Security’s 2026 State of AI Traffic report. Yagour cited both figures to describe what he called an inflection point already underway inside his own organization.
Yagour framed the shift as an open question for infrastructure teams everywhere. “What happens to the infrastructure we’ve spent years building when agents and not humans become the main consumers of that,” Yagour said. “That’s the world we’re stepping into.”
Yagour said three assumptions are breaking simultaneously inside Meta’s infrastructure: capacity, identity and velocity.
On capacity, the math no longer works the way engineering teams are used to. “One engineer used to mean one unit of load,” he said. “Now one engineer spawns 10 agents, each spawning subagents. Your 1,000-person org can generate the load of 100,000 users practically overnight.”
His answer is not to block agent traffic but to make infrastructure agent-aware, with dynamic controls that understand agent hierarchies, cost attribution that traces consumption back to the use case that spawned it, and throttling that adapts based on priority.
Identity is breaking, too. Yagour said an agent does not fit the categories infrastructure teams built access controls around. It is not a human user, it does not carry a badge and it is not a deployed service, yet it makes decisions on its own.
Velocity is the third assumption under strain. Yagour cited a company-reported figure that GitHub Copilot writes 46% of the average user’s code, then noted that faster code generation does not make the rest of the pipeline faster.
“That code still needs to be built, tested, deployed, monitored,” he said. “The agent writes the code in seconds, but your CI/CD pipeline doesn’t get faster just because the machine is the author.”
Data is where Yagour said the pressure from agents is most direct.
“Data sits at the center of everything,” he said, pointing to the decisions, products, recommender systems and next generation models it drives.
Meta is also rethinking how much autonomy to grant agents inside its own data systems. In February, the company shipped what Yagour called agentic data apps. Within three months, 63% of dashboards published across Meta were built using the new tooling, part of the same 30x rise in agentic queries Yagour cited earlier.
That growth raises a governance question. Human analysts have traditionally sat between raw data and business decisions, curating it and serving as an informal check on quality. Yagour said Meta wants to grant agents more independence on harder problems, but was direct about the risk.
“Autonomy without governance is nothing but chaos,” he said. That’s why the company built what it calls trusted data environments, to preserve the human check as agents take on more of that work.
“Inside, the agent can explore data freely, but every output is traced back to its source and scrutinized. So you always know that the data shared back is trusted and governed,” Yagour said.
Sensitive fields are masked before an agent can reach them, and every access request is evaluated in real time against what the agent is trying to reach, why and whether it is allowed. Yagour summarized the approach as exploring broadly while releasing narrowly.
Meta’s models are also demanding more from data as they shift from correlation to reasoning.
“Reasoning is data hungry,” Yagour said.
Pattern matching works on sparse, summarized signals. Reasoning demands the full behavioral history, every interaction across every surface over time. Yagour pointed to two shifts already underway inside Meta’s infrastructure to keep up.
Real-time streaming is replacing batch ETL for ranking pipelines. A pipeline that takes 24 hours to run is not viable when a model is reasoning about a user’s current intent. Yagour said real-time streaming, not batch extract-transform-load processing, is becoming the backbone of Meta’s ranking and recommendation systems.
Storage is becoming schema-aware to stop GPU starvation. Meta previously stored user data as opaque blobs with no awareness of what the data contained, which Yagour said led to heavy overfetching and idle GPU capacity. The company is now building storage that understands what it holds, pulling only the columns and time ranges a given query needs. Yagour said Meta is building toward 500 million queries per second and a petabyte per second of throughput for training data reads.
That data feeds directly into how Meta’s recommendation systems behave. Yagour said 42% of Instagram users have told the company they want to fundamentally change the algorithm, not adjust a single session or setting. Meta’s response is what Yagour called fully conversational recommendations, where a user tells the system what they want more of and it reasons about intent rather than matching on keywords. Yagour said the same search term, soccer, would return different results for a casual fan looking for highlights than for a club athlete seeking training drills, because the system would reason about which one is asking.
Yagour described the three threads of his talk, agents, data and recommendations, as reinforcing each other rather than moving independently.
“Agents make data more accessible. Better data makes reasoning. Reasoning creates new demands that push agents and infrastructure forward,” he said. “This isn’t linear; it’s a flywheel.”
During the Q&A, an audience member asked whether Meta’s push toward more intelligent infrastructure signals the end of traditional file systems in favor of newer neural storage approaches, and whether agents will keep using SQL as their interface to data the way humans do. Yagour said Meta is experimenting at every level, including questioning whether SQL is the right interface for agents at all, and that storage at Meta’s scale already operates in the multi-digit exabyte range and needs to keep expanding.
Yagour closed his talk with the timeline he believes the industry is working against. “We spent 20 years building infrastructure for humans. We have maybe 20 months to rebuild the whole thing for a world where humans and agents co-create at scale,” Yagour said. “The window is open, but it won’t stay open for long.”
Drug discovery is notoriously inefficient. Pharmaceutical projects span years, moving from one specialized human team to the next through disconnected workflows that result in knowledge loss during each handoff.
A shocking 90% to 95% of drug discovery projects reportedly fail — one of the highest failure rates of any industry. A single successful drug can take over a dozen years and up to $1 billion from initial discovery to patient distribution, according to published reports.
Generative AI is being used to solve some of the challenges, but Stanford researchers have moved the ball forward with agentic AI.
A team led by James Zou, associate professor of Biomedical Data Science at Stanford University, has deployed thousands autonomous AI “scientist” agents in a virtual biotech that simulates the full lifecycle of drug development. The agents handle everything from initial discovery through safety testing and clinical trial design, while maintaining the continuity that’s lacking in today’s drug discovery processes, according to Zou.
The project uses a hierarchical orchestration framework. At the top sits a chief scientist officer agent that acts as a planner, delegating tasks to teams of specialized agents, Zou told VentureBeat during a call ahead of his upcoming session at VB Transform 2026.
While one team of agents focuses on discovery, another manages safety, and others handle specialized analytical tasks. Because these agents operate within a unified, hierarchical ecosystem, they retain the full context of a project, maintaining continuity from the first molecule identified to the final clinical outcome.
The “brain” of the system relies on a vast amount of primary data. The agents are granted access to data sources ranging from genomics and FDA chemistry data to clinical trial databases using a model context protocol.
The team has invested heavily in agent-native and agent-friendly data, allowing the AI to synthesize complex information more effectively. The system relies on a combination of models, with Zou noting that while Claude often serves as the backbone for coding and data analysis, the architecture employs a mixture of models, including those fine-tuned specialized use cases.
Zou is raising money at a roughly $1 billion valuation for his startup, Human Intelligence, based on the research.
During Zou’s session at VB Transform on July 15, titled How 10,000 agentic scientists in Stanford’s lab are set to revolutionize medical research and discovery, he will share valuable insights including strategies for managing context and long-running, multi-step workflows in a multi-agent system, the process of transforming and indexing raw enterprise data to make it agent native, and how to use human auditing and experimental reward signals to verify agent actions.
Another session at VB Transform focused on the value of agentic context includes Building a trustworthy agentic AI foundation: How Zillow accelerated engineering by 40%, with Zillow’s SVP of engineering and technology, Toby Roberts and Glean’s CEO Arvind Jain.
Interested in attending VB Transform 2026? Register here. A select number of complimentary passes are also available to senior technology leaders. Contact us to get yours.
Customer expectations have shifted from simple, fast conversational interactions to complex agentic AI-powered tasks that legacy IT architectures simply can’t handle.
To address this, Intuit made the bold decision to overhaul its technical infrastructure for its business platform. The company moved away from its multi-agent setup, which prioritized broad capabilities, to a granular, skill-and-tool-based architecture while embedding human experts directly into the workflow alongside AI. This shift involved decomposing its massive agents into specialized components, separating the brain from the hands, essentially.
“We went from a multi-agent system where we had large agents that did a lot to fully incorporating workflows, skills and tools down to the base level,” said Nhung Ho, VP of AI at Intuit. “We changed the orchestrator, we changed the planner, we changed the brain, and we also changed what everybody had to build across the whole company.”
At VB Transform 2026 on July 14 and 15, Ho will share details about the technology decisions behind building an abstraction layer behind Intuit’s system of intelligence. She’ll also share how the new architecture has allowed the company to decouple its orchestration from specific model providers, allowing Intuit to remain agile and use the best tools for the job, whether from large model providers or their own home-grown tools.
Other VB Transform sessions focused on agentic orchestration include:
From signals to shelves: How Target is engineering Agentic AI for the right product, right place, right time with speaker Siobhan McFeeney, SVP Technology, Target;
The engineer’s multiplier: How Instacart uses agentic AI to eliminate toil, elevate teams and slash costs with speaker Anirban Kundu, CTO, Instacart;
MCP connection isn’t orchestration: Building the agent execution layer with Arnab Bose, chief product officer, Asana;
Building the agentic workforce: A blueprint for scaling AI operations without the sprawl with Romit Jadhwani, Sr. Director, Enterprise AI, Data & Productivity, Rivian and Craig Wiley, VP of AI, Databricks; and
Inside Atlassian’s Living Lab: Deploying context-aware agents at scale with Dr. Molly Sands, head of the Teamwork Lab at Atlassian
Interested in attending VB Transform 2026? Register here. A select number of complimentary passes are also available to senior technology leaders. Contact us to get yours.
The security implications of advanced AI models were immediately clear to Visa’s technology team when they began testing Anthropic’s Mythos model.
Just weeks into Project Glasswing, the team observed how quickly attackers can identify and weaponize vulnerabilities in critical code bases, creating security risks, explained Rajat Taneja, Visa’s president of technology, during a call to prepare for his session at VB Transform 2026, VentureBeat’s upcoming agentic AI event.
Visa is among the companies selected to test Anthropic’s upcoming model — a version of which was released June 9 but abruptly disabled days later to comply with U.S. government directives.
The findings of Project Glasswing put a spotlight on widening enterprise security gaps and the vulnerabilities malicious actors can take advantage of.
“Security has always been important, but currently, in the age of AI, is going to be even more important because the attacks become autonomous,” Taneja told VentureBeat. “The defenses have to become autonomous. And we are not there. And there’s an asymmetry there, which is a very big risk for the world.”
Threat actors now have access to powerful AI agents that can work 24/7, “operating at a scale and speed that human teams cannot match, automating the tedious reconnaissance and exploitation phases of a cyberattack,” according to Cisco’s State of AI Security 2026 report. Amy Chang, Cisco’s head of AI threat intelligence and security research, will also be a speaker at VB Transform.
To mitigate these risks, Visa is building its own abstraction layers, observability, and data guardrails to secure its autonomous commerce frameworks. The payment services giant also rolled out an open‑source, AI-driven security framework that turns vulnerability discovery and remediation into a structured, repeatable pipeline.
Their work represents a shift enterprise IT teams must make to protect enterprise systems against threats posed by bad actors wielding autonomous agents. Taneja will share these insights and valuable technical details during his session at VB Transform, titled Inside Project Glasswing and Mythos: Securing the agentic future today, on July 15.
Other agentic AI security-focused sessions at VB Transform include:
CrabTrap: How Brex built an open source proxy to secure OpenClaw’s critical flaws for everyone with Brex co-founder and CEO Pedro Franceschi;
When AI Agents have wallets: Building the trust layer for autonomous B2B commerce with Mastercard’s Chief AI and Data Officer, Greg Ulrich;
Expedia’s blueprint for building autonomous agents for high-stakes transactional systems with Chief AI and Data Officer Xavier Amatrain; and
Securing agentic AI: A playbook for permissioning, sandboxing, and human-in-the-loop controls, a panel discussion with AI security leaders from Intuit, Box and Cisco.
Interested in attending VB Transform 2026? Register here. A select number of complimentary passes are also available to senior technology leaders. Contact us to get yours.