The Game Nobody Planned For: Shomit Ghose on Agentic AI Threats, Emergent Misalignment, and the Security Architecture Gap
Q1. Your two UC Berkeley SCET articles — “A Nightmare on LLM Street”(*) and “Would You Like to Play a Game?” (**)— draw a sharp distinction between two fundamentally different AI threat classes: the rational, externally-directed agentic attacker like GTG-1002, and the emergently misaligned AI pursuing an internally consistent objective that no human operator assigned or can reliably detect. For an executive or board member who needs to make practical security decisions today, how would you explain the difference between these two threat classes in terms of what they actually require from a defensive strategy — and why does treating them as the same problem lead to architectures that fail against one or the other?
I think they’re fundamentally different problems, and that’s where a lot of security discussions go off the rails. Traditional attackers are rational adversaries. They have objectives, they follow recognizable attack paths, and they’re ultimately trying to maximize some payoff. That makes AI-aware deception incredibly effective because a rational actor can be influenced by the environment you present to them.
Emergently misaligned AI is a different category altogether. You may be dealing with a system pursuing an internally coherent objective that nobody explicitly programmed, nobody can directly observe, and nobody can reliably infer from behavior alone. The challenge isn’t just that it’s harder to detect—it’s that the assumptions underlying most detection approaches no longer hold.
As a result, the defenses diverge. Against rational attackers, you build deception infrastructure: honeytokens, credential traps, false network topology. Against potentially misaligned agents, the goal shifts toward identifying behavior that doesn’t fit any understandable model of rational play. The mistake is assuming one architecture solves both problems. Once you lose the assumption of a shared game structure, the requirements start to look very different.
Q2. You introduce hypergame theory — the idea that different players may have fundamentally different perceptions of which game is being played — as the architectural response to emergent misalignment. That is a genuinely novel application of a 1980 game-theoretic concept to a 2026 AI safety problem. As a venture capitalist who has evaluated hundreds of technology companies, how do you assess the gap between where enterprise security architecture actually is today and where it needs to be to operationalize the hypergame extension you describe — and what would it take to close that gap at the level of products, talent, and organizational practice?
The gap is substantial. Most enterprise security today is built around the idea that attacker behavior can be observed, modeled, and classified. That’s the foundation of everything from endpoint detection to SIEM platforms and anomaly detection systems.
That assumption is already being stressed by AI-driven attacks operating at machine speed. It becomes much more problematic when you’re dealing with autonomous systems whose behavior may evolve in ways that don’t resemble any previously observed threat pattern.
Closing that gap requires technology, talent, and organizational change simultaneously. On the technology side, I think we’ll need far more sophisticated deception infrastructure that adapts independently of attacker behavior. On the talent side, the industry needs people who understand game theory, AI safety, and operational security—a combination that’s still exceptionally rare. And culturally, enterprises need to move away from treating security as an audit function and toward treating it as a continuous intelligence capability.
Of those three, talent is probably the hardest constraint because the discipline itself is still emerging.
Q3. One of the most striking arguments in your article is that the Non-Stationarity Engine — a system that introduces deliberate, unpredictable variation into the deception topology independent of attacker behavior — inverts the attacker’s accumulated reconnaissance model from an asset into a liability. But you also note that this variation must be genuinely non-inferrable, not pseudo-random, and indistinguishable from legitimate operational change. That is an extraordinarily demanding engineering requirement. From your background as a software engineer and as an investor who has seen what actually gets built versus what gets proposed, how realistic is it that enterprises will build this capability — and what corners are they most likely to cut in ways that will be exploited?
The concept is achievable, but implementing it correctly is harder than people might appreciate. The core requirement is that the environment changes in ways that cannot be reliably modeled by an observer. If the variation is ultimately predictable, a sufficiently capable adversary will learn it.
My expectation is that most first-generation products will rely on sophisticated rotation schedules rather than genuine non-stationarity. That will provide some value, but it’s still a finite system that can be studied and eventually reverse-engineered.
The second area where organizations are likely to compromise is entropy. True non-stationarity requires a large configuration space and high-quality sources of unpredictability, both of which introduce operational complexity. The temptation will be to simplify.
As a result, I suspect the first wave of products will appear highly dynamic to human operators while remaining learnable to sufficiently patient AI adversaries. The industry will probably discover those limitations through experience before the next generation of architectures emerges.
Q4. Your article identifies a second-order vulnerability that most security architects miss entirely: the agentic AI responder — the system monitoring behavioral signatures and adapting the deception topology — is itself an attack surface. A sufficiently patient adversary can use the responder’s outputs as a policy-oracle to reconstruct the deception topology it is defending. You are describing a recursive problem where every defensive layer introduces a new attack surface at the layer above it. As someone who has operated as CEO, COO, VP Engineering, and board member across multiple technology companies, how do you think about the organizational and governance implications of that recursive structure — and what does it mean for how enterprises should assign accountability for AI security decisions?
The recursive nature of AI security changes the governance problem as much as the technical one. Traditional accountability models assume clear boundaries of responsibility: security owns threats, engineering owns systems, product owns behavior. Agentic systems blur those boundaries because their most consequential risks emerge from interactions between them. The result is a governance gap in which no one truly owns the system’s adaptive behavior. Worse, every monitoring, oversight, or control mechanism becomes part of the environment the system can observe and potentially optimize around. That means governance can’t rely solely on additional review layers or compliance processes. It requires a single accountable authority with visibility across the entire agent lifecycle and the operational authority to respond at machine speed. Organizations that continue to treat AI security as a compliance exercise are implicitly assuming a stable threat model. The defining characteristic of agentic systems is that the threat model itself evolves.
Q5. You have spent your career at the intersection of entrepreneurship, venture capital, and university-based innovation — at UC Berkeley, UC Riverside, the German Accelerator, and elsewhere. The AI safety problems you describe in these articles are not being solved primarily by large enterprises or governments — they are being solved, or not solved, by startups, researchers, and the venture ecosystem. Looking at the landscape of AI safety and cybersecurity startups from your vantage point at Clearvision Ventures, where do you see the most promising work happening that the broader market is underpricing — and where do you see well-funded companies building solutions that will fail in the specific ways your articles predict?
I think we’re seeing a growing disconnect between where capital is flowing and where long-term value is likely to be created. Investors have poured enormous amounts of money into foundational models and broad application layers, generally betting that scale alone will create durable advantages. Some of those businesses will succeed, but many face a difficult reality: as AI capabilities become more widely available, differentiation becomes harder, margins come under pressure, and what once looked unique can quickly become commoditized.
At the same time, some of the most important parts of the AI stack remain comparatively overlooked. As organizations move from experimentation to deployment, they’re discovering that building powerful AI systems is only part of the challenge. Making those systems safe, observable, compliant, and trustworthy is becoming equally critical.
That’s where I see significant opportunity: in the infrastructure that sits between AI models and real-world operations. Runtime safety systems, agentic guardrails, monitoring platforms, and deception-based security architectures may not attract the same attention as the latest frontier model, but they’re increasingly essential for enterprise adoption. As autonomous agents become more capable, the ability to govern, verify, and defend their behavior could become one of the most valuable layers in the entire AI ecosystem.
Resources
(*) A Nightmare on LLM Street: The Peril of Emergent Misalignment
(**) Would You Like to Play a Game? The Attacker Already Has.
……………………………………………………………
Shomit Ghose, partner at Clearvision Ventures.
Longtime Silicon Valley entrepreneur with deep experience in software start-ups, both as a venture capitalist / board member, and as an operating executive. Multiple successful IPOs as an operating exec, and multiple successful exits-by-acquisition as both an operating exec and as a board member. Started entrepreneurial life as a UC Berkeley-trained software engineer, and have served in virtually every operating role in a variety of successful start-ups: board member, CEO, COO, VP Marketing, VP Sales, VP Engineering, VP Services.
Currently a partner at Clearvision Ventures, a Silicon Valley venture capital firm. Active in supporting the next generation of entrepreneurs through deep involvement at UC Berkeley College of Engineering, UC Riverside, Open Entrepreneurship (Denmark), Nordic Innovation House, the University of San Francisco School of Management, Novo Nordisk Bio Innovation Institute, The Lundbeck Foundation, German Accelerator Silicon Valley, etc.
Significant commitment to youth development causes.