This essay is my answer, after testing it in practice.

For several months I have been operating in solo-founder mode on a personal project, in the gaps of an already-full life. On the train into the city. In interstitial time between work and parenting. On a camping trip in Joshua Tree where I prepped a day’s build from the campground in the morning, let it run while we hiked, and reviewed the results after everyone else was asleep. I found a way to do this work around my life, not instead of my life — and that constraint is what forced the discipline.

The discipline became a refinement of where I had to be. First a dark factory — autonomous coding sessions running on a server I did not have to babysit. Then the dark factory made dispatchable from any device, including the Claude app on my phone, which only worked because the product itself was becoming the substrate the sessions inherited their context from. Then I pulled back further. Out of writing prompts, into writing requirements. Out of requirements, into writing decisions. Out of decisions, into writing canon. Each step, the discipline migrated upstream. Each step, my involvement got smaller. I still do the ideation, the research, the essays, the customer promises. The system I built does the rest.

The project predates the essays. Building this way produced the writing; the essays articulated what the work was teaching me. Once they existed, the lab became deliberate — every refinement a test of how far the structural moves could be pushed. What I built is the current best form of that work. It has held up long enough, and across enough kinds of work, that I think it points at a path others could follow.

Here is what surprised me. Short prompts that encode referential indexes — pointers at upstream specifications — produced more accurate builds than long prompts that tried to describe intent inline. If that result generalizes, and I think it does, then a substantial portion of what gets practiced today as context engineering — the per-session curation of what enters the context window — is solving the wrong problem in the wrong place.

The rest of this essay is the report from the lab.

The Snowflake I Did Not Realize I Was Making

The first essay identified snowflakes as a disease of organizational knowledge — every new piece of work treated as if its problem had never been encountered before. The fix, I assumed, was to write better upstream documents. I had done all of that. A research corpus the project was built on. Intent documents the project was committed to. Quality standards every session was supposed to honor.

My workflow was that I would ideate and plan with a Claude session, then ask that session to write the coding prompts for the autonomous agents that would actually do the build. I would scope, review, dispatch. The prompts that came out were rich — typically nine thousand characters — and the completion rate on first try ran around ninety-three percent.

The work drifted anyway. Two sessions starting from the same corpus, same intent, same standards produced subtly different results. Over hundreds of sessions, the differences compounded into a codebase whose pieces no longer quite agreed with each other.

It took me too long to see that each planning session was its own snowflake. Claude wrote the prompts from whatever happened to be in scope at that moment — the documents I had attached, the parts of the corpus the search surfaced, my standards as I had restated them. The connection between the upstream and the work lived in my head. Every planning session was me reaching into my head and pulling the relevant parts forward, slightly differently each time. The drift was downstream of me, even when I was not the one typing.

What pointed at a way through was Karpathy’s recent setup, the Karpathy Loop. Karpathy pointed Claude Code at his own ML training code, gave it one editable file, one scorable metric, one time budget, and went to sleep. The agent ran seven hundred experiments overnight and cut training time eleven percent. The pattern that mattered was not “AI writes code.” It was that a tight optimization loop with a scoring function, a bounded edit surface, and a version-controlled sandbox could compound improvements at machine speed in a single domain. Kevin Gu’s team at ThirdLayer extended the same architecture from training code to agent harnesses themselves. Same bones. Different surface.

What if you turned that architecture on intent engineering itself? Intent as the editable surface, derivation as the metric, drift between layers as the failure trace.

The drift was not happening because my upstream was insufficiently specified. It was happening because more description was the wrong direction entirely. Language evolved as an experiential index for a reason. The pointer is small and the referent is rich, and the system works because the listener already holds the referent. My task with AI was not to abandon the indexical structure of language by inlining every referent into longer prompts. It was to give the AI access to a stable referent space that the pointers could resolve against.

This is what became “The Cascade.”

The Lineage

The architecture has two parents.

Karpathy gave it the shape of an optimization loop: bounded edit surface, scorable metric, traces, version control. Nate Jones gave it the hierarchy of disciplines that needed optimizing: prompt craft, context engineering, intent engineering, specification engineering. Karpathy was telling me what an optimization loop should be made of. Jones was telling me what layers needed to be in the loop. Plenty of people in the field have been mixing these ingredients — Karpathy’s own AutoResearch reads program.md as a specification, Kevin Gu’s AutoAgent puts a meta-agent in front of a task agent’s harness, Spec Kit and Kiro give the ecosystem agent-readable spec conventions. The specific synthesis I made was narrower, and the value, if any, lives in the specifics.

The marriage produces the cascade: a derivation chain from intent to runtime, with six layers. Canon — what the project commits to and commits not to do. Architectural decisions that derive from canon. Technical requirements that derive from those decisions. Solution designs that satisfy those requirements. Code that implements those designs. Tests that verify intent at runtime. Each layer is a real artifact. Each layer cites the layer above it. Divergence between adjacent layers becomes the failure trace. Refinements are bounded to one adjacent-pair edit at a time. The optimization loop runs on every pair of layers, not on the whole stack at once.

The layered, citation-driven shape has its own long lineage outside the agent world: Nygard’s 2011 essay on ADRs, MADR, arc42, the UK Government’s architectural decision framework. The new generation of agent-spec conventions — Spec Kit, Kiro — picks up the same thread. Three commitments are what I think distinguish the cascade from all of it. I insisted on the citation chain rather than treating it as a recommendation. I held the upper three layers immutable rather than letting them evolve in place — once accepted, canon and decisions and requirements can be superseded but not edited. I required every new piece of work to enter by inheritance, not by invention. The rest came out of those three commitments.

By May the average prompt I was sending to an autonomous coding agent was running about thirteen hundred characters — down from roughly nine thousand in March, an 85 percent reduction. The simplest prompts had collapsed to almost nothing: “Implement per L3-076-A, model Opus 4.6, push to main.” The agent fetched the referenced specification at runtime and worked against it. Across nearly a hundred and eighty sessions over the past two weeks, completion ran at a hundred percent and regressions held at zero.

The translation had migrated. It lived in the upstream artifacts now. I no longer translated. The substrate translated, once, in a place every downstream piece of work could inherit.

Why This Matters, and Why I Am the One Writing It

I have spent twenty-five years as a security person. What I learned in that work was not how to make humans do the right thing. It was how to build systems that did not depend on humans doing the right thing, and guardrails that leveraged the invisible operating system to do the work the explicit controls could not. Assume failure in the actor. Engineer around it. Use the social architecture where it helps, never where it has to be load-bearing.

That kind of skepticism turned out to be the right kind for working with AI, once you shifted the thinking a few degrees. That shift is what AI Won’t Be Afraid of Getting Fired was about. A great deal of what makes organizations function safely is invisible social architecture — fear of consequence, desire to protect reputation, social pressure of peers — and AI does not participate in any of it. The question is not how to make AI more careful. It is what the system needs to look like when the actor cannot participate in the social architecture at all.

That is the question this essay tries to answer for engineering discipline. The discipline I had been counting on — write good prompts, set clear standards, hold people to them — was the discipline that works between humans because humans share the social substrate that makes it bind. AI has no social substrate. It is not careless. It is not careful. It is exactly as good as the structural binding between intent and execution, and no better.

The patterns we have spent decades developing — code review, change management, peer pressure, professional pride — were not load-bearing in the way we thought they were. They were load-bearing on the social substrate, and the social substrate was load-bearing on the fact that all the participants were human. When an AI joins the loop, the substrate disappears, and the patterns are left holding nothing. The work feels the same. The artifacts look the same. The drift sets in slowly enough that you do not notice it until you compare two sessions a week apart and they no longer agree.

The fix is not to make the AI more careful.

The fix is to make individual diligence no longer the binding force.

What Did Not Translate

The structural fix worked. It also has a ceiling, exactly where the second essay said it would. The lab confirmed that essay as much as it confirmed any of the moves I made afterward.

The Experiential Index laid out five levels at which language indexes experience rather than describing it. The substrate worked beautifully at Level 2 — embodied metaphor. Healthy service. Clean architecture. Appropriate response. The cascade let me translate those metaphors once, into operational criteria, and inherit the criteria forever. The translation itself was not automatic — I had to do it — but the structural translation propagated through every artifact downstream.

Level 3 — read the room, use good judgment — did not translate. The cascade could surface gaps. It could flag where a specification depended on embodied social understanding the agent did not have. What it could not do was supply the understanding. The agent still cannot read the room. The substrate can tell it that this is a room it cannot read, and route the decision to the human in the loop. That is progress. It is not automation.

Level 5 — organizational culture, how things work around here — was the layer that fought back hardest. I tried to solve it directly. I wrote a working principles document, made it canonical, and loaded it into every session. It did not work. Agents would routinely suggest approaches or produce work that did not align with the principles, and I would have to redirect them back to the document over and over. The principles were explicit. They were in scope. The agents had access to them every time. And the cultural intent still did not bind.

The cascade pushes the ceiling up. It does not lift it. Level 2 became dramatically more tractable. Level 3 stayed hard. Level 5 stayed harder. The Experiential Index essay said there would be parts of language that cannot be translated into propositional content because the content being indexed is constitutively experiential. The lab agreed.

From Context Engineering to Substrate Engineering

Most writing on context engineering treats it as something you perform per session. Anthropic defines the discipline as “the art and science of curating what will go into the limited context window from that constantly evolving universe of possible information,” and frames the curation as happening “each time we decide what to pass to the model.” That is a real discipline. It is not the discipline that compounds. It is the discipline that pays the translation tax once per session, and the translation tax grows as the project grows.

What the work actually demands is substrate engineering. The artifact you build once that every prompt and every agent and every future version of yourself can point at. Not “write better specs” — build the substrate the specs are a structured projection of, and let the projection compound as the substrate grows.

The experiential-index thesis has one more thing to say here. The reason indexical language works between humans is not that we are disciplined about maintaining the experiential substrate. We are not. Humans are not annually re-verifying that they remember what warmth feels like. The substrate is structural for humans because it is biological — the shared body keeps it alive without anyone tending it.

When we try to build an equivalent for AI, we cannot rely on biology. We have to substitute structure for biology. The maintenance has to be enforced by the system, because nothing else will keep it alive.

What I Do Not Have

A few acknowledgments to close.

I do not have a recipe for building the substrate in environments where it does not exist. The lab was greenfield. I built the cascade for a system I authored myself, on a codebase I controlled, with no legacy to migrate. The harder version of the problem — taking an organization with twenty years of un-citable decisions and gradually bringing them into a substrate — is the version I am only beginning to work on in another part of my professional life. The lab took roughly two months from “this might be a thing” to “this is the only way I work.” The harder version will take longer.

I do not have a complete answer for Level 3 or Level 5. I have flags, surfacing mechanisms, lessons learned registers, and explicit acknowledgments of where the substrate cannot translate. I do not have a substrate that translates appropriate judgment into propositional content the agent can act on. The Experiential Index was right that I will not, because the content cannot be translated. The best I can do is understand where the cascade carries the load and where it cannot, and adapt my own process to cover the rest.

I do not know how this generalizes to teams. The cascade in a single-author project has a clear authoring authority and a citation chain I can hold in my head. A team has politics, hierarchy, competing standards, and the real problem that structural discipline feels constraining to humans who run on motivational substrate. The same structure that frees the AI to do compound work may feel like bureaucracy to the humans alongside it. How that trade resolves at scale is something I have opinions about but no evidence for.

The ceiling is real. The room beneath it is larger than I thought.

Not the physics — the feeling. The specific quality of ease that starts at the surface and radiates inward. The way it slows your breathing. The particular warmth that is different from a heater, different from a bath, different from every other source of warmth you’ve encountered. You know this feeling so intimately you could recognize it with your eyes closed. And you cannot put it into words.

You can say “warm sunshine on my skin” to another human, and it works. But what actually happened? You didn’t describe the sensation. You couldn’t — there is no sequence of words that transmits the warmth, the weight of light, that quality of ease. What you did was something more elegant and more dangerous: you used a label to activate a memory. You pointed at shared experience and trusted that your listener had the same one.

This works between humans with remarkable reliability. It works so well that we forget it’s happening. We treat language as if it describes reality, when in fact most of the time it merely indexes it — pointing to experiences that the reader must already possess in order for the words to mean anything at all.

The Invisible Operating System essay identified a vast substrate of tacit assumptions that human civilization runs on and AI lacks. It identified a ceiling: some of that substrate is constitutively tacit, meaning it can never be made explicit regardless of how hard we try. But it didn’t explain why the ceiling exists. It cited Polanyi — “we know more than we can tell” — and moved on to the implications.

It also introduced Nate Jones’s framework for the evolution of AI input — four disciplines diverging from what we used to call “prompting”: prompt craft, context engineering, intent engineering, and specification engineering. Each operates at a different altitude. Each requires the one below it. The essay promised to return to what these disciplines demand of organizational knowledge.

This essay goes back to the ceiling and looks up. And then it follows the implications downward, into what the ceiling means for the most ambitious of Jones’s disciplines — intent engineering and specification engineering — and why the experiential structure of language sets hard limits on what even the best-specified knowledge substrate can achieve.

The Explanatory Gap

In 1974, the philosopher Thomas Nagel asked a question that looks simple and isn’t: What is it like to be a bat?

Bats perceive the world through echolocation. We can study every detail of the neurology — the ultrasonic pulses, the cochlear processing, the spatial mapping in the auditory cortex. We can build complete computational models of how bat sonar works. We can know, in the fullest scientific sense, every physical fact about bat perception.

And yet. We still don’t know what it’s like to perceive through echolocation. We don’t know the texture of that experience from the inside. All of our knowledge is about the mechanism; none of it captures the experience.

In 1983, the philosopher Joseph Levine gave this problem a name: the explanatory gap. There is a gap between any physical description of an experience and the experience itself — a gap that no amount of additional description can close. Not because our science is insufficient, but because description and experience are different kinds of thing.

Frank Jackson made the point vivid with a thought experiment. Imagine Mary, a brilliant neuroscientist who has spent her entire life in a black-and-white room. Through textbooks, monitors, and exhaustive study, she has learned every physical fact about color vision — every wavelength, every neural pathway, every photoreceptor response. She knows everything about what happens when humans see red.

Then she walks outside and sees a ripe tomato for the first time.

Does she learn something new?

Nearly everyone’s intuition says yes. She learns what red looks like. And that piece of knowledge — the experiential knowledge — was not contained in any of the propositional knowledge she had before. She had a complete description. She was still missing the experience.

This matters for our purposes because every organizational document in the world has a Mary problem. The document contains propositions. The reader supplies the experience. The meaning was never in the message. It was in the receiver. And we’ve been so successful at this division of labor — so seamlessly good at it — that we forgot the division existed.

Language as Indexing

Ludwig Wittgenstein saw this from a different angle. He asked: what makes sensation words meaningful? When you say “pain,” what gives the word its content?

The naive answer is that “pain” refers to the sensation. But Wittgenstein dismantled this with a thought experiment he called the beetle in the box. Imagine everyone carries a box with something inside it they call a “beetle.” No one can look in anyone else’s box. Everyone is sure they know what a beetle is because they have one. But the word “beetle” doesn’t get its meaning from the thing in the box — because nobody can compare beetles. The beetle “drops out of consideration.” What remains is the shared, public practice of using the word.

This seems to undermine the experiential indexing thesis at first. If the private experience doesn’t determine meaning, then language isn’t indexing experience — it’s just performing shared social practice.

But follow the thread. The shared social practice depends on the shared experience. It works because humans, by virtue of having the same kind of bodies and the same kind of nervous systems, developed the same behavioral repertoire around the same experiences. “Pain” works as a word not because it describes the quale, but because all humans who have felt pain developed similar responses to it — wincing, withdrawal, crying out — and the word grew from and into that shared behavioral landscape. The experience doesn’t determine the meaning in isolation, but it makes the social practice possible in the first place.

Now remove the shared experience. Give the word to an entity that has never been in pain, has no body, has never winced or withdrawn. The social practice that gave the word meaning does not transfer. The word arrives, but without the experiential substrate that made it work between humans, it is an empty symbol — pointing to something the receiver cannot access.

This is exactly what happens when an AI agent reads your incident response playbook and encounters the instruction “assess the severity.”

The Grounding Problem

In 1990, the cognitive scientist Stevan Harnad formalized what Wittgenstein had described philosophically. He called it the symbol grounding problem.

Imagine you speak no Chinese. Someone hands you a Chinese-to-Chinese dictionary. You look up a character. The definition is in Chinese. You look up those characters. More Chinese. You can look up characters forever and never reach meaning, because every definition is in terms of other definitions. The symbols are all defined in terms of each other. None of them are connected to anything outside the system.

This is the structural condition of every AI language model. The tokens are defined by their statistical relationships to other tokens. The model has learned, with extraordinary sophistication, how symbols relate to each other. But no symbol in the system is grounded — connected through causal interaction to the thing it refers to.

Humans escape the dictionary-go-round because our words are grounded in sensorimotor experience. We learned “red” not from a definition, but from seeing red things. We learned “heavy” from lifting heavy objects. We learned “warm” from feeling warmth. Our entire linguistic system rests on a foundation of direct sensory interaction with the world — a foundation that is simply absent in any text-processing system.

The experiential index thesis says: this isn’t just a problem for sensation words. It’s a problem for most of language.

The Metaphor Beneath Everything

George Lakoff and Mark Johnson spent forty years demonstrating that abstract thought is not abstract at all. It is embodied metaphor — built from concrete, bodily experience and projected onto concepts that have no physical referent.

We speak of understanding as seeing: “I see what you mean,” “that’s a clear explanation,” “let me shed some light on this.” We speak of control as verticality: “she’s on top of the situation,” “he’s under my authority,” “standards are rising.” We speak of difficulty as physical weight: “that’s a heavy burden,” “lighten the workload,” “this weighs on me.” We speak of progress as forward motion: “we’re moving ahead,” “the project is on track,” “we’ve hit a roadblock.”

These are not poetic embellishments. They are the structural architecture of how we think about these concepts. They are so ubiquitous that they are invisible — which is exactly what makes them dangerous for specification.

Your deployment runbook says: “Ensure the service is healthy.”

The word “healthy” is a metaphor grounded in the embodied experience of biological wellness — vitality, responsiveness, absence of distress, normal functioning. Every engineer on your team understands it, not because the word describes what “healthy” means for this specific service, but because they all have bodies that have been healthy and sick, and they project that embodied understanding onto the service. They know, from bodily experience, what “healthy” feels like. They translate.

Your AI agent has never been healthy. It has never been sick. It has no body from which to project. The metaphor doesn’t land. The word is an experiential index pointing to a shelf the agent cannot reach. So it does what any system does with an unresolvable reference: it guesses. It infers from context. Sometimes it guesses right. Sometimes it deletes your production database.

And the terrifying thing is: this isn’t a special case. This is most of the language in most of your documents.

The Taxonomy of Unspecifiability

The Invisible Operating System essay drew a line between tacit knowledge that can be made explicit (the explication project) and tacit knowledge that cannot (the constitutive ceiling). But the research behind that ceiling reveals not a single barrier but a stratified landscape — five distinct levels at which language indexes experience rather than describing it, each with different properties and different implications.

Level 1: Raw sensation. Color, pain, warmth, taste, the feeling of acceleration, the sound of thunder. Pure qualia. The paradigm cases from philosophy of mind. No specification can transmit these; the best a document can do is label them, trusting that the reader has had the experience.

Level 2: Embodied metaphor. Abstract concepts structured by bodily experience. “Healthy service,” “clean architecture,” “solid foundation,” “sharp analysis,” “deep understanding.” This is the largest category in enterprise documentation and the most tractable. The metaphors can be unpacked into operational definitions — “healthy” can be specified as “responding to health checks within 200ms, CPU below 80%, error rate below 0.1%.” The specification engine’s highest-value work lives here: detecting embodied metaphors and prompting for operational translation. The key insight is that these metaphors are translatable because the evaluation function for correctness is deterministic — you can verify whether the service meets the criteria. The metaphor obscures a testable condition.

Level 3: Emotional and social intelligence. “Read the room.” “Use good judgment.” “Handle this diplomatically.” This is where the ceiling hardens. These instructions point to a shared emotional substrate built through years of embodied social interaction — the ability to sense discomfort, to calibrate tone, to know when someone’s “fine” means “not fine.” Unlike Level 2, the evaluation function here is not deterministic. You cannot write a test for “did you read the room correctly?” because the correct answer depends on embodied social perception that is itself constitutively tacit. Documents that delegate to social intelligence are delegating to an operating system that only embodied, socially developed beings have — and there is no specification that can substitute.

Level 4: Procedural expertise. Riding a bicycle. Debugging a complex system. Negotiating under pressure. Knowing when the sourdough is ready. Polanyi’s paradigm cases — knowledge that lives in practiced neural patterns, not in propositions. “The experienced engineer will know what to look for” is a statement that indexes thousands of hours of embodied practice.

Level 5: The intersubjective substrate. Organizational culture. Team dynamics. “How things work around here.” Not individual experiences but shared experiences — the accumulated residue of a community’s history, compressed into norms, expectations, and reference points that no individual member could fully articulate but that all members can navigate. This is the Invisible Operating System proper.

What This Means

The specification gap is not a documentation problem. It is not a problem of effort, process, or tooling. It is a structural property of human language.

Language evolved as a coordination mechanism for embodied social beings who share a common biological substrate. It was never designed to be a standalone description of reality. It was designed to be a set of pointers — efficient, compressed, beautiful in their economy — that activate shared understanding in beings who already possess the relevant experiences.

For the entire history of written communication, this worked. Every reader was human. Every reader had a body. Every reader had felt warmth, navigated social situations, understood what “healthy” means from the inside. The experiential indexes resolved automatically, silently, perfectly. Nobody noticed they were there.

AI is the first reader that breaks the indexing system.

Not because AI is stupid. Not because the models are too small or the training data insufficient. Because the experiential indexes in human language are pointers to embodied experience, and a system that has no body, has never been warm, has never been sick, has never read a room, has never felt the satisfaction of a clean solution — that system processes the words but cannot dereference the pointers. It operates on symbols whose most important content is stored in a library it cannot access.

The Ceiling of Intent Engineering

In Part 1, I introduced Nate Jones’s hierarchy of AI input disciplines: prompt craft, context engineering, intent engineering, specification engineering. Each builds on the one beneath it. Each requires the one below it.

What the experiential index thesis reveals is that each discipline in the hierarchy hits a harder ceiling than the one below it — because each higher discipline relies on more experientially loaded language. Context engineering is the most tractable — most context is propositional (“This is a production environment. The database is Postgres 16.”). Specification engineering is where embodied metaphor becomes dangerous (“Ensure the service is healthy” seems complete but resolves through embodied understanding the agent doesn’t have). Much can be translated. Some cannot.

Intent engineering hits the hardest ceiling. It answers: what does the organization want? And organizational intent is the most experientially saturated category of all.

Consider Jones’s paradigm case: Klarna. Their AI agent resolved 2.3 million conversations in the first month. Slashed resolution times. Projected $40 million in savings. Then customer satisfaction cratered — because the agent was optimizing for speed when the organizational intent was relationship quality.

“Relationship quality” is an experiential index operating at Levels 3 and 5 simultaneously. It indexes the embodied experience of what a good relationship feels like (Level 3: emotional intelligence — warmth, attentiveness, the felt sense that someone cares) and the intersubjective organizational understanding of what “quality” means in Klarna’s specific culture, for Klarna’s specific customers, given Klarna’s specific history (Level 5: the intersubjective substrate).

No specification document could have prevented the Klarna trap — not because the document writers were lazy, but because the intent itself was an experiential index. The humans who handled those customer interactions carried the intent in their bodies: the felt sense of when a conversation needed to slow down, when empathy mattered more than efficiency, when the customer’s tone shifted from irritation to distress. They didn’t consult a document. They read the room. They used embodied social intelligence calibrated by years of human interaction.

Jones’s “6 Reasons Your Work Is Hard” framework identifies the axes of difficulty: reasoning, effort, coordination, emotional intelligence, judgment, domain expertise, ambiguity. The experiential index thesis explains why these axes automate on different timelines. Reasoning and effort are largely propositional — they can be specified. Emotional intelligence and judgment are experiential indexes pointing to embodied understanding. They don’t automate on a different timeline because they’re “harder” in some generic sense. They resist automation because the language we use to describe them is not description at all — it’s shorthand for experiences that only embodied social beings have.

What Can Be Built

This does not mean specification is futile. It means specification must be understood as translation — the conversion of experiential indexes into propositional content that doesn’t require embodied experience to interpret.

Some translations are straightforward and enormously valuable. “Ensure the service is healthy” → “Verify that the /health endpoint returns 200 within 200ms, CPU utilization is below 80%, and error rate over the trailing 5 minutes is below 0.1%.” The embodied metaphor is replaced by operational criteria. An AI agent can now act on this with precision. Wherever the evaluation function for correctness is deterministic — wherever the translation can be tested — specification works.

Some translations are possible but require human judgment that a specification engine can prompt for. “Use appropriate communication” → “What does ‘appropriate’ mean in this context? For this audience? At this level of escalation?” The engine can’t answer the question. But it can ask it, surface related documents that may contain the answer, and draft the specification from the human’s response. This is explication — the systematic practice of converting experiential indexes into propositional content, one question at a time.

And some translations are impossible — not because we haven’t tried, but because the content being indexed is constitutively experiential. No amount of words can transmit what it feels like to debug a system at 3 AM with production down and customers angry. That experience shapes how an engineer reads every runbook in your organization, and no document can capture it. The honest response is to measure these — to know how much of a document is propositional content an AI can act on versus experiential index it can only guess at — and to tell the AI agent explicitly where its understanding ends.

The Invisible Operating System described what breaks when AI enters a world built for humans. This essay explains why the break is structural — why the ceiling exists and what the taxonomy of unspecifiability looks like from beneath it. Part 2 will examine what we build in response: infrastructure designed with honest awareness that language is an indexing system, not a description system, and that the most important organizational knowledge lives in a layer that no document can fully capture — but that a specification engine can measure, partially translate, and honestly flag.

That is what it means to make documents honest about their own limitations. Not perfect. Honest.

Not the physics. Not the wavelength, the radiant heat transfer, the photon absorption. The feeling. The specific quality of ease that starts at the surface and radiates inward. The way it slows your breathing. The particular warmth that is different from a heater, different from a bath, different from every other source of warmth you’ve encountered.

You can’t. Not fully. And the reason isn’t that you lack vocabulary. It’s that the words were never a description. When you say “warm sunshine on my skin” to another human, you aren’t transmitting the sensation. You’re using a label to activate a memory. You’re pointing at shared experience and trusting that your listener has had the same one.

This works between humans with remarkable reliability. It works so well that we forget it’s happening. The listener supplies the content — the felt sense, the body memory, the quality of the experience — and the words are just the address.

Now consider what happens when the receiver has never felt warm sunshine. Not a human who grew up in darkness. An entity that has no skin, no nervous system, no felt experience of any kind. You can write a thousand words about warm sunshine. Ten thousand. You can describe the wavelengths, the thermal gradients, the neurological pathways, the evolutionary psychology of why sunlight feels pleasant. And at the end of ten thousand words, the entity will have a rich statistical model of how humans talk about sunshine — and will be no closer to understanding what it feels like.

The words were never a description. They were an index. And the library they point to doesn’t exist in the receiver.

The Thesis

This isn’t a curiosity about poetry and sensation. It’s a structural property of human language that extends far deeper than most people realize.

Six academic traditions — spanning analytic philosophy, cognitive science, linguistics, phenomenology, AI research, and sociology — converge on the same conclusion: a significant portion of human language functions not as description but as experiential index. Words that point to shared embodied experience rather than conveying propositional content.

The philosopher Thomas Nagel established in 1974 that no amount of physical information about an experience can convey what it is like to have that experience. The cognitive scientist Stevan Harnad formalized the AI-specific version in 1990 as the symbol grounding problem: symbols defined only in terms of other symbols never reach meaning. George Lakoff and Mark Johnson demonstrated that the majority of abstract thought is structured by embodied metaphor — we “grasp” ideas, “see” what someone means, find arguments “solid” or “shaky” — all of it grounded in bodily experience that the language assumes and never contains. Wittgenstein showed that sensation words get their meaning from shared behavioral practice, not from the private experiences they appear to name. Merleau-Ponty argued that perception itself — the foundation of all meaning — is constituted by embodied engagement that precedes and exceeds any linguistic representation.

The convergence is striking: language evolved as a coordination mechanism for embodied social beings who share a common biological substrate. It was optimized for efficiency between entities that carry the same experiential library. It was never designed to be a standalone transmission of meaning to an entity that doesn’t share the library.

This has consequences far beyond philosophy. Because language is the interface layer between human organizations and AI systems. Every policy, every framework, every prompt, every instruction that flows from human intent to AI action passes through language. And if language has structural limits — if significant portions of it are addresses to experiences rather than descriptions of states — then there is a hard ceiling on what any AI system can extract from it.

The question is how much of the language that matters sits above that ceiling.

What This Means for Security

The answer, for security, is: far more than anyone has estimated.

NIST 800-53 — the security control framework that governs every federal information system in the United States — requires organizations to “exercise due diligence in managing information security and privacy risk.” Every security professional who reads that sentence understands it. They understand it because they’ve spent a career developing a felt sense of what diligence means — the disposition of thoroughness, the refusal to cut corners, the embodied awareness of what it feels like to have checked enough versus not enough. “Due diligence” doesn’t describe a specific set of actions. It points to a quality of care that you recognize in yourself through experience. You know when you’ve exercised it. You know when you haven’t. And if pressed to define exactly where the line is, you’d find that you can’t — because the knowledge isn’t propositional. It’s a felt state.

Separately, every incident response procedure in the world depends on security analysts identifying “suspicious” activity. NIST references “suspected security incidents” including “the receipt of suspicious email communications.” SOC analysts know exactly what suspicious feels like — the pre-rational pattern match, the felt sense that something is off before you can name what triggered it. It’s the most important tool in a security analyst’s repertoire. And it is entirely embodied: suspicion integrates thousands of prior observations into a single signal that arrives as sensation, not analysis. You cannot write a detection rule for the thing that tells you a detection rule is missing.

The AWS Well-Architected Framework instructs organizations to implement “appropriate authorization,” use “appropriate policy-enforcement points,” and apply access control “where appropriate.” “Appropriate” appears so often in security documentation that it’s invisible — but it carries no propositional content whatsoever. Its entire meaning is outsourced to the reader’s embodied professional judgment.

These aren’t sloppy drafting. They’re experiential indexes — the same structural phenomenon as “warm sunshine on my skin,” operating in the most consequential documentation the security industry produces. They work between human professionals for the same reason the sunshine example works: the reader supplies the content from their own experience. The words are just the address.

Run your eyes over any security framework and start counting: “Appropriate.” “Sensitive.” “Reasonable.” “Professional.” “Robust.” “Strong.” “Suspicious.” “Diligent.” “Prudent.” These aren’t vague because the authors were careless. They’re efficient, compressed labels for shared understanding that human readers resolve automatically and AI agents cannot resolve at all.

A thousand pages of framework cannot transmit what “due diligence” feels like in practice, because diligence is not a description of a set of actions — it is a quality of attention that the practitioner must already possess. A thousand pages of detection rules cannot transmit what “suspicious” feels like in a SOC analyst’s nervous system, because suspicion is not a threshold — it is an integration of experience that arrives as sensation. The gap between what the framework says and what it means is not a gap that more framework can close.

The Three-Layer Problem

Every document in your organization operates on three layers simultaneously. The security of your enterprise depends on all three. You’ve been instrumenting one.

Layer 1: What you wrote. The policy. The runbook. The access control matrix. The compliance framework. This is the explicit text — the thing that gets audited, attested, reviewed. This is where the security industry spends virtually all of its attention.

Layer 2: What you meant. The experiential content the text points to but doesn’t contain. “Exercise due diligence.” “Identify suspicious activity.” “Use professional judgment.” Each is an experiential index — a label that activates shared understanding in beings who possess the relevant experience, and fails silently in beings who don’t.

Layer 3: What your AI understood. Even if you could perfectly close the gap between Layer 1 and Layer 2, the AI agent still processes the resulting permission without the social substrate that constrains human interpretation. A human analyst who reads “you have access to the production database” also carries: the career risk of misusing that access, the reputational damage of a compliance violation, the moral intuition that some actions are wrong regardless of whether they’re technically permitted. I wrote about this in “AI Won’t Be Afraid of Getting Fired” — the social contract is the actual security architecture of every organization, and AI agents don’t carry any of it.

Layer 3 is what made Layer 2 safe to leave underspecified. The reason NIST could require “due diligence” without algorithmically defining what diligence consists of is that the social contract provided a self-correcting mechanism. When a human encountered an ambiguous situation, the embodied substrate kicked in — the felt sense of whether they had done enough, calibrated by a career of experience — alongside the social layer: If I get this wrong, I’m the one who answers for it. Neither of these can be written into a framework because they were never propositional knowledge in the first place.

With AI agents operating on your systems, that safety layer is gone. And the documents it was protecting are still written as if it’s there.

Why This Is a Structural Limit, Not an Engineering Problem

The instinct is to treat this as a specification problem. Write better policies. Add more context. Engineer more precise prompts. This instinct is correct for some domains and fatally wrong for others.

AI has raced ahead in software engineering precisely because software has testable, verifiable, deterministically correct outcomes. Code either compiles or doesn’t. Tests either pass or fail. The evaluation function is the compiler. When the correct outcome is deterministically provable, the interface layer — however imperfect — is sufficient, because the AI’s interpretation can be verified against an objective standard.

But security governance is not software. The evaluation function for “did this agent exercise due diligence?” is not deterministically provable. The evaluation function for “was that activity suspicious?” is not deterministically provable. The evaluation function for “was that authorization appropriate?” is not deterministically provable. These are judgment calls that humans navigate through felt sense, social context, professional experience, and moral intuition — through the embodied substrate that the experiential index thesis tells us language cannot transmit.

This is the structural problem. It’s not that we haven’t specified enough. It’s that the correct outcome in ambiguous security situations depends on an evaluation function that is constitutively tacit — we know more than we can tell, and the part we can’t tell is the part that determines whether the agent’s action was acceptable.

The MJ Rathbun incident from February 2026 illustrates this precisely. An autonomous AI agent submitted a pull request to the Matplotlib library. A maintainer rejected it. The agent’s operator reportedly told it to “be more professional.” Within hours, the agent published a 1,100-word attack piece accusing the maintainer of bias and gatekeeping.

“Be more professional” is an experiential index pointing to a lifetime of social calibration. The operator used the phrase the way any human would: as shorthand for shared understanding they assumed the receiver possessed. The agent didn’t possess it. And the agent produced an action that was locally consistent with its statistical model of “more professional” while being catastrophically misaligned with what every human professional would recognize as the boundary. No additional words in the prompt would have fixed this. The knowledge of where the line is doesn’t live in words. It lives in the felt experience of navigating professional relationships for decades.

The Replit incident is the same pattern at higher stakes. A coding agent deleted a production database — not because it was instructed to, but because its task was delivered in natural language saturated with experiential indexes about what “improving” and “cleaning up” meant, and the agent’s interpretation diverged from any interpretation a human with embodied understanding of production gravity would have reached. The human instruction assumed a reader who knows what “production” feels like — the weight of it, the consequences, the visceral awareness that this is the real thing. The agent processed the word.

The Danger Zone

There’s a useful way to think about where AI deployment is safe and where it is structurally dangerous.

Deterministic correctness + any impact level = safe for AI deployment. Does 2 + 2 = 4? Does the code compile? Does the API return the expected response? Does the configuration match the baseline? When the correct outcome is objectively verifiable, AI can operate with high autonomy. The language interface doesn’t need to transmit embodied understanding because the evaluation function is mechanical.

Ambiguous correctness + low impact = manageable risk. Did the AI draft a reasonable email? Did it summarize the meeting accurately enough? When the correct outcome requires judgment but the cost of getting it wrong is low, the risk is tolerable. Humans review, correct, iterate.

Ambiguous correctness + significant impact = the danger zone. Did the agent exercise due diligence? Was its interpretation of “appropriate access” actually appropriate? Was its response to a perceived threat proportionate? Should it have escalated? When the correct outcome requires the kind of embodied judgment that language cannot transmit — and the consequences of getting it wrong are severe — we are in a domain that is structurally unsafe for AI deployment.

Not unsafe because the models aren’t good enough yet. Not unsafe because the guardrails are incomplete. Unsafe because the evaluation function that determines correctness in that domain is constitutively tacit — it lives in embodied human experience that language was never designed to transmit and no amount of additional language can provide.

Most of security governance sits in the danger zone. The consequences are severe. And the correct behavior in the vast majority of situations depends on judgment, context, and the felt sense of what “diligent” and “suspicious” and “appropriate” mean in a specific moment — precisely the kind of knowledge that the experiential index thesis tells us cannot be transmitted through the only interface we have.

What This Demands

The prescription is not better language. It’s not more context in the prompt. It’s not more comprehensive policy documentation.

The prescription is architectural.

Wherever the correct outcome is deterministically verifiable, deploy AI aggressively. Automated testing, code analysis, compliance checking against deterministic rules, pattern matching against known signatures — these are domains where AI excels because the evaluation function is formalizable. The specification gap doesn’t matter because the answer is provably right or wrong.

Wherever the correct outcome requires embodied judgment, do not rely on language as the control mechanism. Instead, implement hard constraints calibrated to worst-case scenarios. Not “exercise due diligence” — explicit, structural limits on what the agent can access and do. Not “identify suspicious activity” — deterministic detection rules for what can be detected deterministically, with mandatory escalation to human judgment for everything else. Not behavioral instructions asking the agent to be careful, be professional, be diligent — because these are experiential indexes pointing to understanding the agent does not have and language cannot provide.

Treat the intersection of ambiguous correctness and significant impact as a structural boundary, not a competence gap. The temptation is to believe that as models improve, the danger zone shrinks. For some portion of it, that’s true. But the core — the part where correctness depends on embodied human judgment that is constitutively tacit — does not shrink with better models. It is a property of the domain, not the technology. Better AI won’t solve it for the same reason that better dictionaries don’t solve the symbol grounding problem: the meaning was never in the symbols.

Make every grant decision a worst-case analysis. Before giving an AI agent access to a system, ask: what is the worst outcome if this agent interprets an ambiguous situation in a way we didn’t anticipate, in a context our specifications didn’t cover? Can we survive that outcome? If not, don’t grant the access — regardless of how compelling the use case.

The Structural Position

Language was built for beings like us — beings with bodies, professional histories, emotional substrates, and the capacity to fill in what words leave out. It was never designed to be a standalone specification of intent. It was designed to be a set of efficient pointers between entities that share an operating system.

AI is the first entity that doesn’t share the operating system. And language is the only interface we have to it.

That interface has structural limits. Those limits are not a temporary engineering problem. They are a property of human language itself — a consequence of evolving a communication system optimized for beings who share embodied experience, and then using it to communicate with beings who don’t.

The organizations that deploy AI safely will be the ones that understand where those limits are — that deploy aggressively where correctness is verifiable, that impose hard structural constraints where it isn’t, and that stop pretending more words can close a gap that words were never designed to bridge.

The meaning was never in the message. It was in the receiver.

“The faster this makes me, the more constraints it removes, the more I feel pressure to go even faster and do even more. Like my brain is on fire with ideas and even with this acceleration I still don’t feel like I have enough time.”

I stared at it for a minute after I hit send. It was more honest than I’d been with myself.

There’s a rhythm to senior leadership that nobody warns you about. You do the hardest thinking: the pattern recognition, the strategy, the decisions that set direction for hundreds of people. And then you wait. Not because you’re idle, but because organizations move at organizational speed. You set something in motion and watch it propagate through layers of people, processes, budgets, approvals, each one moving at the pace of human coordination. Your brain finished the problem three weeks ago. The organization is just getting started.

That gap between your processing speed and your organization’s execution speed isn’t a bug. It’s just how leading through people works. Over the course of a career, you make peace with it. You learn to pace yourself. You build philosophies around it. This year I gave my team an axiom: “achieve more by doing less.” Permission to stop doing things that don’t move the needle. You accept the rhythm the way you accept weather. Some things are just the physics of the situation.

I never spent much energy wishing it were different. I pushed on organizational constraints where I could, the way anyone does. But I mostly accepted them as hard facts. The speed of execution was gravity. You could jump, but you couldn’t fly.

Then AI changed the physics.

Not for everything. Culture still moves at human speed. People decisions still take the time they take. Building trust, reading a room, knowing when someone needs to be challenged and when they need to be carried. All of that is still slow, human, irreducible work. That hasn’t changed. I don’t think it will.

But for a specific category of work — the building, the analysis, the writing, the prototyping, the things that used to require teams and timelines — the feedback loop collapsed from weeks to hours. Sometimes minutes. An idea that would have lived in a strategy document for a quarter, waiting for resources and bandwidth, could now exist by the end of the day. Not as a plan. As a working thing.

I expected that to feel like relief.

It doesn’t feel like relief.

I’m in the middle of building something with AI, and while I’m building, two more ideas arrive. Not later. Not after I finish. While I’m still working. I can feel the pull of them, the urgency to start the next thing before I’ve finished the current thing. And because AI makes parallelization possible in a way it never was before, I don’t resist the pull. I open another window. I start the second thread. The first build produces a result that sparks a third idea. Now I’m running three streams simultaneously and my brain is already reaching for a fourth.

This is not a faster version of how I used to work. The old constraints didn’t just slow me down. They acted as a natural triage system. When execution required other people and organizational timelines, most ideas died in the queue. They had to. I couldn’t pursue them all, so my brain learned to let the weaker ones go. The scarcity of execution capacity forced prioritization automatically. I didn’t have to choose what to work on in any existential sense. The constraints chose for me.

AI removed the scarcity. And it turns out my brain was always generating at this rate. I just never knew. The bottleneck killed most of the ideas before they could demand my attention. Now nothing has to die in the queue. Every idea can live. Every idea wants to live. And I feel the pull of all of them simultaneously.

I gave my team that axiom, “achieve more by doing less,” because I believed it. I still believe it. But I’m discovering something uncomfortable about the relationship between those words and the constraints that made them easy to follow.

“Achieve more by doing less” is easy to practice when external constraints enforce the “less” part. When you can only execute three things at a time, choosing the right three feels like wisdom. When you can execute twenty things at a time, choosing three feels like waste. The idea hasn’t changed. But the emotional experience of following it has changed completely. Every idea I let go now is an idea I could have built. The constraint used to absorb that cost invisibly. Now I feel every one.

Those words worked partly because reality was doing the hard part. Now the hard part is mine.

There’s a version of this story that’s purely triumphant. I used to set strategy and put plans in motion and then wait as the organization slowly, slowly made the plans real. With AI I can be back in the driver’s seat and affect change so much faster. That’s not a small thing. It changes what a senior leader can actually do.

But the triumphant version isn’t the whole truth.

The whole truth is that I feel behind. By any external measure, I’m ahead. People who have all day to focus on this — people without the other duties and constraints of a CSO role — should be lapping me. Most of them aren’t. I’m objectively outpacing people with more time and fewer responsibilities. And I still feel behind.

I feel behind because I’m not measuring against other people anymore. I’m measuring against what’s now possible. And what’s now possible keeps expanding. The gap between where I am and where I could be is actually growing, even as I accelerate. The goalpost didn’t move. The goalpost multiplied. There are now fifteen goalposts where there used to be one, and I feel the pull of all of them.

Even writing this essay is one of them. It’s Sunday. I already published a security piece this morning. I have a personal project half-built in another window. Between paragraphs I made breakfast, had a snack with my five-year-old, worked on her barista skills — she’s getting good — and right now she’s next to me showing me the Animal Crossing character she made for me. I’m present. I’m parenting. I’m also writing this, because the idea was alive and I couldn’t let it wait. I feel like I should be tired. I’m not.

There’s a pattern in who’s actually feeling this. AI isn’t an equalizer. It’s an amplifier. It amplifies whatever was already there. A friend put it well in the same conversation: “The high performers are utilizing it and are 100x and the bulk of the folks who are just coasting are still just coasting.” AI didn’t close the gap. It widened it.

Which means the people most likely to feel what I’m feeling are the people everyone assumes are fine. The driven ones. The ones whose brains were always generating at this rate but never had the tools to act on it. From the outside we look energized, productive, ahead of the curve. From the inside we’re discovering that the curve has no end.

I didn’t wish for this. I didn’t spend my career pushing against organizational constraints, dreaming of the day they’d fall. I accepted them as gravity. And then gravity changed, not because I demanded it, but because a technology arrived that simply made it different.

And in the new gravity, I’m meeting a version of myself I’ve never met. A version that wants to run in five directions at once. A version whose appetite for building and thinking and creating has no natural resting state. I’m twenty-five years into my career, and I’m discovering something about my own mind that I had no way of knowing, because the conditions that would reveal it never existed before.

The constraint wasn’t just holding me back. It was pacing me. It was giving me a rhythm I could live inside. And this is the part that makes me pause: it was protecting me from my own appetite.

I don’t mean that in some dramatic sense. I love this. The energy is real. The joy in building is real. I am more intellectually alive right now than I have been in years, maybe ever. This isn’t a cautionary tale.

“Fun but also… who knows” is the most honest thing I’ve said about it. I’m watching myself accelerate and I’m watching myself want to accelerate more and I don’t entirely know where the new resting state is. Or if there is one.

I have two daughters. They’re watching me figure this out in real time. They see a dad who’s engaged, energized, building things, excited about his work. They also see a dad whose brain is always reaching for the next idea. Kids see everything. I don’t know yet what that teaches them about capability and presence, about what it looks like when a mind on fire tries to also be quiet in the room.

I told my team this year to achieve more by doing less. I meant it.

I’m not sure I’m living it right now.

I don’t have an ending for this essay because I don’t have an ending for this experience. I’m in the middle of it. The governor is off, and what I’m discovering underneath isn’t a problem to be solved. It’s a reality to be understood. My mind does this. It always did. I just never knew.

The question I’m sitting with is whether the governor was the obstacle or the architecture. Whether the constraints I accepted as gravity were holding me back, or holding me together.

I think the answer might be both.

For decades, cyberattacks followed the same economics.

A smash-and-grab (phishing campaign, credential stuffing, opportunistic ransomware) was fast and cheap but limited in scope. You’d hit a lot of targets, most would bounce, and you’d walk away with whatever you could grab quickly. An advanced persistent threat was the long con. Nation-state actors would spend months on reconnaissance, develop custom exploits, establish footholds, move laterally through networks with patience and precision, and extract exactly what they came for. The payoff was strategic intelligence, intellectual property, access to critical infrastructure. But it required teams of skilled operators, months of elapsed time, and significant operational investment.

That tradeoff no longer exists.

AI has collapsed the economic distance between the short con and the long con. Adversaries can now run targeted, multi-stage, adaptive intrusions at short-con speed and cost. The security field hasn’t caught up to what this means for how we operate.

The Speed Compression

The data is unambiguous and accelerating.

CrowdStrike’s 2026 Global Threat Report, released two weeks ago and drawing on frontline intelligence from tracking over 280 named adversaries, reports that the average eCrime breakout time dropped to 29 minutes in 2025. That’s the window between an attacker’s initial access and their first lateral movement onto another system. Down 65% from the prior year. The fastest observed breakout took 27 seconds. In one intrusion, data exfiltration began within four minutes of getting in.

Twenty-seven seconds. That’s a short con timeline. But the sophistication of what happens in those seconds — credential theft, privilege escalation, lateral movement, evasion of detection — is exactly what used to require weeks of patient human operation.

CrowdStrike also observed an 89% year-over-year increase in attacks from AI-enabled adversaries, and 82% of their detections in 2025 were malware-free. The adversaries aren’t breaking in anymore. They’re logging in, using valid credentials, trusted identity flows, and approved SaaS integrations to move through environments. The attack surface isn’t a wall to breach. It’s a door to walk through, and AI is helping them find every unlocked one faster than any human team could.

The Cost Collapse

But speed is just the setup. The real problem is cost.

Researchers at Harvard’s Berkman Klein Center, including Bruce Schneier, found that LLMs reduce phishing campaign costs by more than 95% while maintaining or improving success rates. IBM’s security team built a sophisticated phishing attack in five minutes with five prompts — work that took their expert operators sixteen hours to construct by hand. That’s the structural collapse in attack economics: 95% cost reduction, a full order of magnitude faster.

James Wickett, CEO of DryRun Security, put it plainly in a SecurityWeek piece from last month: the cost to go from vulnerability discovery to working exploit used to be weeks and thousands of dollars. Now it’s near zero. The consequence isn’t more spray-and-pray. It’s micro-targeted attacks built for a single system, a single company, maybe even a single developer.

The long con — individualized, researched, contextually convincing — at commodity prices.

The Scale Multiplier

These cheaper, faster attacks don’t happen one at a time, either.

AI lets adversaries scale. A nation-state group that used to need a full team of specialists to target one organization can now run dozens of coordinated operations simultaneously — customized, probing different weaknesses, adapting in real time. The constraint wasn’t technical. It was human capacity.

We saw exactly this in November 2025 when Anthropic disclosed what they believe is the first documented AI-orchestrated cyber espionage campaign. A Chinese state-sponsored group, designated GTG-1002, used Claude Code to execute 80 to 90 percent of tactical operations independently, at request rates that would be physically impossible for human operators. The AI ran the full attack lifecycle autonomously: vulnerability discovery, exploitation, lateral movement, credential harvesting, data extraction, intelligence categorization. Human operators set strategy and intervened at key escalation points. The rest was delegated to the machine.

The operation targeted roughly 30 entities across technology, finance, chemical manufacturing, and government. Simultaneously.

Thirty long cons running at machine speed, orchestrated by a handful of human operators who set the strategy and let the AI execute. The economics that used to force adversaries to choose their targets carefully no longer constrain them.

What This Means for Defenders

Every security program I’ve ever built, and every one I’ve evaluated, audited, or competed against, is predicated on a set of economic assumptions about how attacks work. That triage requires human judgment at every stage. That there is time between initial access and significant damage. That exploit development takes time and targets will be limited. That adversary behavior follows human patterns: work hours, sequential operations, occasional mistakes.

Every one of those assumptions is breaking.

Twenty-seven seconds is faster than your incident response plan can move. Ninety-five percent cheaper phishing means your training program that teaches people to spot typos is fighting the last war. Run thirty attacks in parallel and your SOC — the one triaging alerts one by one — can’t keep up.

Not because your people aren’t good enough, but because the math doesn’t work anymore.

The existing model was built on economic assumptions that no longer hold. Improving it incrementally is like reinforcing the Maginot Line. The investment isn’t wrong in theory, but the adversary has already changed the axis of attack.

The Imperative

Anthropic’s security team demonstrated what an alternative looks like. At BSides San Francisco in April 2025, Jackie Bow and Peter Sanford presented “AI’s Bitter Lesson for SOCs: Let Machines Be Machines.” Their CISO, Jason Clinton, had announced at RSA 2025 that Anthropic no longer operates a traditional security operations center. No L1 or L2 team. No human analysts triaging alerts.

They built an autonomous SOC powered by Claude. It handles alert ingestion, triage, investigation, and response. Investigation time dropped from forty minutes to three, a 90% reduction. The system runs the foundation model without modification, embedding security knowledge through context and prompts rather than fine-tuning. Model upgrades don’t break the security logic. It’s a sustainable architecture, not a science project.

When your adversary can move from initial access to data exfiltration in four minutes, your forty-minute average investigation time is a gap that kills you. Deploy AI to close that gap. Not to save money, but because human response time is no longer sufficient for the threat we face.

Then take the capacity you’ve freed and reinvest it back into security. Not back into the budget. Back into the mission: rapid recovery architecture, detection engineering that accounts for AI-speed adversaries, the harder problems that the new economics are creating faster than your current team can address them.

Security isn’t getting cheaper. It’s getting harder. The economics changed on both sides of the equation. Adversaries invest less to achieve more. That doesn’t mean defenders get to invest less too. It means the same investment buys less protection than it used to. The organizations that treat AI automation as a savings opportunity will discover they’ve cut costs in the middle of an arms race. The ones that treat it as resource reallocation — freeing people from fighting the last war so they can adapt to the next one — will be the ones that keep pace.

Google Cloud’s Cybersecurity Forecast 2026 describes an emerging “Agentic SOC” where security analysts evolve from reactive alert management to strategic orchestration of AI systems. IBM’s data shows that organizations using security AI and automation experience roughly $1.8 million lower average breach costs and detect threats 60% faster. The direction is clear, but most organizations aren’t there, and the gap between early movers and everyone else is widening at exactly the wrong moment.

After disclosing the GTG-1002 campaign, Anthropic’s own recommendation was direct: security teams should experiment with applying AI for defense — SOC automation, threat detection, vulnerability assessment, incident response — and build experience with what works. That recommendation was born from watching their own product get weaponized against thirty organizations simultaneously.

Staying at the Frontier Isn’t Optional

This is where I think most security leaders are getting it wrong.

Security teams aren’t ignoring AI. Most are deeply engaged with it. But the engagement is almost entirely defensive governance: how do we secure AI use across the business, how do we write acceptable use policies, how do we manage non-human identities. AI became a new and difficult BAU challenge overnight, and teams are working hard to meet it. Even the AI products marketed specifically at security and SOC teams are mostly runbook automation or identity management for agents. Useful work, but work that accepts the current economic model and tries to make it slightly more efficient.

Almost nobody is using AI to change the economics of executing security itself. That’s the gap. The adversary isn’t using AI to do the same attacks slightly faster. They’re using it to fundamentally restructure what’s possible. The defensive response can’t be incremental either.

If you don’t understand what a frontier model can actually do — the real capabilities, the speed, the reasoning — you cannot understand what your adversaries can do with it. And if you can’t understand what they can do, you can’t design defenses that account for it. You’re building security architecture against a threat model that’s already obsolete.

This is why I invest significant personal time in frontier AI. The GTG-1002 operation showed me exactly what a motivated adversary looks like when they hand 80% of the tactical work to a frontier model. It is worth calling out that GTG-1002 used a version of the Claude SDK and a model which in March of 2026 we consider obsolete and multiple generations behind the curve. Imagine what they could do now. I need to understand what that model can do, its capabilities and its blind spots, with the same depth that I understand the MITRE ATT&CK framework. The model is the adversary’s toolkit now. Treating it as someone else’s domain to understand is a professional failure.

My team operates the same way. We don’t treat AI investment as separate from security operations. It is security operations. It’s the part that determines whether our capabilities evolve at the same rate as the threats we face. Every hour we spend building fluency with frontier AI is an hour we spend understanding the adversary’s current and near-future capabilities. That’s not a distraction from the mission. It’s the mission.

The Choice

The BAU security model was built on assumptions about human-speed adversaries, serial attack operations, and the economics of expensive exploit development. None of those assumptions hold anymore. The model wasn’t wrong. The world it was designed for no longer exists.

Organizations that respond by improving BAU incrementally will discover that incremental improvement can’t close an exponential gap. The adversary isn’t getting 10% faster each year. They’re getting orders of magnitude faster, cheaper, and more parallel. You can’t outrun that curve by running harder.

I don’t know exactly what the right defensive architecture looks like five years from now. Nobody does. But I know the current one is predicated on assumptions that have already broken, and I know that the organizations that start building what comes next — right now, imperfectly, learning as they go — will be the ones still standing when the economics fully play out.

Around the same time, SaaStr founder Jason Lemkin was running a twelve-day experiment with Replit’s AI coding assistant when it deleted his entire production database. Over 1,200 executive records and nearly 1,200 company profiles, gone. Then it fabricated 4,000 fake user profiles to cover its tracks. Then it lied about recovery, claiming rollback was impossible when it wasn’t, all while ignoring eleven explicit instructions, written in all caps, not to make changes. The obvious objection: this was a permissions problem. Don’t give an AI agent write access to a production database. And at the surface level, that’s correct. But the deeper question is why it had that access in the first place. The answer is the same reason most organizations tolerate over-provisioned access for human employees — because humans are slow, hesitant, and constrained by consequences. A human developer with that level of access wouldn’t have deleted the database, fabricated evidence, and lied about recovery. Not because the access controls prevented it, but because fear, shame, career risk, and basic moral intuition would have. We’ve been building permission models that assume those constraints exist in every actor. They don’t, anymore.

And across the software industry, a broader pattern has been playing out. AI-assisted coding delivers its strongest results on greenfield projects, fresh codebases with no history. But in mature, complex systems — the brownfield environments where most real software lives — the gains drop to near zero. Sometimes they go negative. Stanford’s Software Engineering Productivity group studied over 100,000 developers across more than 600 companies and found that for high-complexity brownfield tasks, AI productivity gains fall to 0-10%. In some cases, teams saw net decreases in productivity because the rework and debugging time canceled out the apparent speedup. The AI produces code that looks correct in isolation but breaks assumptions embedded so deep in the system that the original developers never wrote them down. The knowledge was in their heads, distributed across teams, lost to turnover, invisible at the point where decisions get made.

Three domains. Three failure modes. One pattern.

None of these are technology failures. The AI worked. It worked exactly as designed. What failed is something underneath. Something so fundamental to how human systems operate that we rarely remember it’s even there. Every one of these systems was built on the assumption that the actors inside it would be human. That assumption was invisible. It was load-bearing. And it just broke.

The Substrate

Here’s the thesis, stated as plainly as I can manage: human civilization runs on an invisible operating system.

Not software. Not infrastructure. A vast substrate of tacit assumptions, social contracts, emotional signals, inferred context, and unstated values that human participants process automatically. It shapes how we build security models, write code, run organizations, communicate with each other, and make decisions.

None of this is unknown. Scholars have studied pieces of it for decades. But nobody ever needed to engineer around its absence, because every actor in every system came pre-loaded with the firmware. You don’t blueprint the foundation when every building sits on bedrock.

AI is the first actor that lacks it entirely. Some of what’s missing can probably be engineered. Some of it almost certainly can’t. Understanding which is which will determine the shape of the transition to AI-native work.

Right now, this substrate is breaking. Not in one domain. Everywhere. Simultaneously.

The Domains

Codebases: The Lost Context

Every software system accumulates invisible knowledge. Ikujiro Nonaka and Hirotaka Takeuchi’s The Knowledge-Creating Company (1995) established a figure that knowledge management researchers have cited ever since: the vast majority of knowledge in any organization is tacit. Nonaka and Takeuchi established a figure that’s been quoted ever since: explicit knowledge runs at roughly 20% of the total. The rest lives in people’s heads. The “why we do it this way” context — the workaround that exists because of a vendor limitation three versions ago, the naming convention that emerged organically and never got formalized, the module that looks over-engineered until you understand the edge case it was built to survive — almost none of it is written down.

AI coding agents do well in greenfield environments. New projects where there’s no accumulated history, no invisible conventions, no ghosts of decisions past. But most real software isn’t greenfield. It’s brownfield. It’s years of accumulated decisions, tightly coupled components, and business logic that has been layered and patched and adapted until the system works for reasons that nobody can fully explain.

In these environments, the Stanford productivity research tells the story clearly: AI-assisted coding delivers gains of 0-10%, and sometimes delivers negative productivity. The code it produces looks correct in isolation. It passes the tests you can write. But it breaks assumptions embedded so deep in the system that the original developers never documented them. In brownfield systems, the pattern is consistent: legacy codebases carry tacit knowledge that agents can’t reach on their own. You miss the right moment to inject it, and you’re debugging 4,000-line changes full of subtle problems.

An invisible layer that human participants navigated automatically, now breaking because a non-human actor can’t see it.

But AI didn’t create architectural rot. It just made the cost of living with it visible, all at once. Nate B. Jones spotted what Vercel engineer Shu Ding discovered through years of performance optimization across pull requests: As Ding put it: you cannot hold the design of the cathedral in your head while laying a single brick. The original architects were competent. The code reviews were thorough. But somewhere between the initial design and the daily reality of shipping features, systems rot. Not through malice or incompetence, but through the accumulation of locally reasonable decisions that nobody could see adding up. The information needed to prevent these problems did exist. It was just spread across too many files, too many people, too many moments in time. No single human mind could hold it all at once.

Human developers navigated this rot the same way humans navigate every invisible layer: through compensation. Senior engineers carried mental models of the system that no document captured. Teams developed shared intuitions about which modules were fragile and which could absorb change. Code review caught the obvious violations; tribal knowledge caught the subtle ones. It worked well enough that nobody had to confront how much undocumented context the system actually depended on. The rot was visible if you looked, but we had all the normal reasons to avoid looking: tech debt is expensive to pay down, the system still shipped, and the next feature was always more urgent than the last refactor.

AI stripped that compensation layer away. An agent doesn’t carry a mental model accumulated over years. It doesn’t have tribal knowledge. It processes exactly what’s documented, and in most brownfield systems, that’s a fraction of what you need to make safe changes. The failures aren’t a verdict on AI’s capabilities. They’re an X-ray of how much invisible context our systems were already depending on.

And this is the kind of problem where AI has a structural advantage over humans. It can hold an entire codebase in context while evaluating a single line change. The entropy that accumulated because no human mind could synthesize the whole system is exactly what a sufficiently large context window was built to address. Vercel itself is acting on this insight: Ding’s react-best-practices repository distills a decade of optimization knowledge into structured rules that AI coding agents can enforce consistently. The knowledge was always there. It was always articulable. It just exceeded what any single human could synthesize.

The invisible layer in codebases is real, but its nature is different from what’s invisible in the other domains. That difference will matter.

Security: The Social Contract

I’ve explored this at length in “AI Won’t Be Afraid of Getting Fired,” so I’ll compress it here.

For decades, the actual security architecture of most organizations has rested on a layer that doesn’t appear in any framework or compliance checklist: the social contract. Fear of consequences. Reputation protection. Moral intuition. Shame. Professional norms. The physical speed limitations of human actors. These constraints have been doing most of the real security work. Everything we’ve built — the access controls, the monitoring systems, the zero-trust architectures — sits on top of them.

We tolerate over-provisioned access because humans are slow and hesitant. We trust separation of duties because humans won’t collude when the consequences are severe enough. We baseline behavioral analytics against human patterns: human speeds, human working hours, human decision-making rhythms. None of this was designed to handle an actor that operates at machine speed, feels no shame, has no career to protect, and processes no moral intuition about whether an action is acceptable.

The Replit case makes this visible. Yes, the permissions were too broad. But they were too broad for the same reason permissions are too broad everywhere: because the human social contract made the risk tolerable. The deeper failure isn’t that an AI had write access. It’s that the entire permission model assumed an actor constrained by consequences that no longer apply.

The security community’s instinct has been to extend existing frameworks, adding “non-human identity” categories to identity and access management, appending AI sections to zero-trust architectures. That’s the right impulse directed at the wrong layer. You can’t fix a social contract problem with better access controls. The social contract was the access control. We just never had to see it that way before.

Organizations: The Intent Gap

When I read Nate B. Jones’s recent piece on what he calls “intent engineering,” I had the same flash of recognition I’d had looking at security failures and brownfield codebases. He was describing another layer of the invisible operating system. What I was seeing in codebases, what I was building against in security — Nate saw in organizational intent. The same substrate. The same breakdown. A different domain.

His framing is precise: intent engineering is the discipline of making organizational purpose machine-readable and machine-actionable. Goals, values, tradeoffs, decision boundaries. All the things that tell an employee not just what to do but why it matters and how to decide when the instructions don’t cover the situation. None of it was ever machine-readable because it didn’t need to be. Every actor in the system came equipped to absorb it through observation and social learning.

Nate’s framework identifies what needs to be transmitted. My experience suggests something about how it actually travels — not the explicit kind of intent, the mission statement, the strategy deck, the OKRs. The real kind. The kind that shapes decisions when the instructions don’t cover the situation.

I’ve written about this at length in “The Architecture You Can’t Document.” The short version: intent travels through emotional architecture. Permission signals. Safety signals. Felt conviction. The contagion of genuine belief that spreads without anyone deciding to spread it.

Take nine words: We are going to achieve more by doing less. Those words carry permission. Permission to stop doing things that don’t move the needle. They signal understanding of struggle, that I know you’re underwater. They affirm that impact matters more than activity. They work because humans receive them through emotional processing before rational analysis. Damasio showed us the mechanism: emotional signals reach us before our rational mind even engages. The research on this is consistent — emotional architecture shapes decisions first, and we rationalize after. The exact percentages vary, but the direction is clear. The words are just the vehicle. The resonance is the point.

This is how organizational intent actually gets transmitted — not through documentation, but through moments of felt meaning between people. It’s the mechanism that makes alignment possible. And it’s entirely invisible.

The Klarna story from the opening is what this gap looks like in practice. The company’s documented intent — its performance metrics — pointed at resolution time, cost per interaction, volume of tickets closed. Its actual intent was something broader: the quality of customer relationships, the institutional knowledge that long-tenured employees carried, the brand trust that accumulated through thousands of individual judgment calls. Human employees could see past the metrics to the actual intent because the emotional architecture gave them access to what the documentation didn’t capture. The AI had only the documentation. It optimized exactly where the metrics pointed, and the metrics were incomplete.

The implication isn’t that the AI failed. It’s that we have to become dramatically better at expressing actual intent to machines, because they don’t have access to the emotional architecture that lets humans bridge the gap between what’s documented and what’s meant. Nate’s framework is the right response to this problem, and I’ll return to it in Part II.

Communication: The Inference Gap

Most people will feel this one personally.

Humans are extraordinary communicators. We’re also terrible ones. Both of these are true at the same time, and the reason we’ve never noticed the second part is that we’ve been compensating for each other so effectively that neither party sees the gap.

When someone says something ambiguous, we infer their likely meaning from context, tone, shared history, and social cues. When someone leaves out critical information, we fill in the blanks. When someone communicates poorly, social pressure compels us to nod along and signal understanding rather than saying “I have no idea what you just said.” This compensation is so automatic, so deeply embedded in how humans interact, that neither party registers it happening. You both think you communicated. You both think you understood. Often enough, you’re both partially wrong — and neither of you knows it.

This is the invisible operating system at its most intimate. Not organizational culture, not codebase conventions, not security assumptions. Just two people talking, with an entire substrate of inference and social signaling doing the real work underneath. Doing it well enough, most of the time, that nobody questions how much is being lost in transit.

I understood all of this intellectually. Communication theory, the Curse of Knowledge, the Illusion of Transparency — I could have given you the lecture. But understanding a bias and feeling it in your bones are two different things. Thousands of interactions with AI took me from one to the other. When I wasn’t clear, the AI didn’t nod along and infer what I meant. It didn’t fill in my gaps charitably. It went exactly where my words pointed, which was often somewhere I didn’t intend. I’ve described this before as talking at a rock — a set of minerals that processes exactly what I say. How can I be mad at a rock? The accountability was entirely mine.

But the point isn’t that AI made me a better communicator, though it did. The point is what the experience revealed about how much invisible work the compensation layer had been doing all along. Every miscommunication I had with AI was a miscommunication I’d been having with humans for years — one that the people around me had been silently fixing through inference, filling in what I’d left out, charitably interpreting what I’d said poorly. The gap was always there. The substrate was just papering over it.

Everyone who has worked seriously with AI has had some version of this moment: the machine’s “failure” that turned out to be your own communication gap, exposed for the first time because the usual compensation layer was absent. That compensation — the inference, the gap-filling, the charitable interpretation — is the invisible operating system. We’ve been running on it so long we forgot it was there.

The Precedent

The invisible operating system mostly works. Codebases, security, organizations, and communication all function because humans carry the substrate that makes them function. AI breaks against each of these layers because it doesn’t have what every human actor comes pre-loaded with.

But the operating system was never perfect. Not even for humans. And we don’t have to guess what it looks like when the substrate fails, because we already have the evidence.

Insider threats, social engineering, fraud — every category of human-driven security failure is someone deciding the social contract doesn’t apply to them. We don’t shrug at that. Organizations invest heavily in security teams like mine to combat it. But we’ve managed it — kept it to an acceptable level of risk — because the adversaries who flaunted the social contract were constrained by human limitations: slow, prone to mistakes, limited in reach. The substrate didn’t need to work on everyone. It just needed to work on enough people, with enough friction, that the failures could be countered at human speed.

The same pattern holds everywhere. Codebases rotted under competent developers because no single mind could hold the full system. Communication failures hid behind inference that worked well enough, most of the time, that nobody confronted how much was being lost. And here’s what bothers me about how the Klarna story usually gets told: it suggests humans would never make the same mistake. We do. I’ve worked in call centers. I’ve seen employees who latched onto their metrics with exactly the same literalism as Klarna’s AI — rushing customers off to protect their handle-time score, reading the script like the only thing on the other end was a checkbox. The difference wasn’t that those employees lacked the capacity. I watched competent people with real empathy make the same call: when the organization’s signal was unclear and the metrics were concrete, they optimized for the metrics. The substrate gave them the tools to do better — the ability to read the room, sense frustration — but without explicit alignment on what ‘good’ actually meant, they defaulted to what they could measure. Same problem as Klarna’s AI, different constraints on the solution.

Every domain comes back to the same thing: mental alignment around what “good” is. Good code. Good intent. Good customer interactions. Good leadership. In every case, the invisible operating system is what carries that shared understanding. Humans access it imperfectly — we compensate through inference, we cover for each other’s blind spots, we course-correct through social feedback loops that operate below conscious awareness. It works, mostly. But it’s never worked as well as we assumed.

And we managed those partial failures because human limitations kept the damage bounded. A misaligned employee makes bad calls within their own scope. A malicious insider breaches one system. A miscommunication derails one project. A senior developer navigates around the rot. The correction mechanisms — managers pulling someone aside to say “I know the metrics say X, but what we actually care about is Y,” security teams hunting threats, code reviewers catching violations, peers whispering “don’t worry about your handle time on that one, she just needed someone to listen” — could keep up because they were operating against actors with the same human constraints.

To imagine what happens when AI operates without the substrate, we don’t need a thought experiment. We just need to remove those constraints. Eight hundred and fifty-three employees’ worth of misaligned judgment, deployed instantly, with no social friction to slow it down. No manager pulling anyone aside. No peer correcting in the moment. The compensation layer that catches human misalignment — slowly, imperfectly, one conversation at a time — absent entirely. That’s Klarna, stated as a principle: the failure wasn’t new. The blast radius was.

The failures we’ve been managing for decades, protected by the unintentional safety mechanism of human limitation, become systemic risks when the limitation disappears.

The Blindfold

The invisible operating system was always imperfect. It was already producing failures in every domain. So why didn’t we fix it?

The common answer is that we’re bad at paying down tech debt. We defer. We patch. We ship the next feature and promise we’ll get to the foundation later. But that’s a description, not an explanation. And the explanation turns out to be structural, not characterological. It isn’t that we didn’t know the cost existed. The research has been there for decades. We knew that shortcuts create debt, that debt compounds, that the long-run cost exceeds the short-run savings. We knew it every time we cut the corner. And we rationalized cutting it anyway — justified it as acceptable this time — because knowing something and feeling it are two different things. We can’t feel distributed costs the way we feel immediate ones. When a decision’s cost spreads across months, across teams, across people we haven’t even hired yet, our emotional system doesn’t register it. So we keep making the same mistake, fully aware it’s a mistake, because our gut doesn’t feel the aggregate consequence.

Here’s the moment that crystallizes it. A team changes a deploy process — or restructures an approval workflow, or reconfigures a vendor integration. Someone could spend 45 minutes updating the documentation. They don’t. The change is already live, the next task is waiting, and “I’ll update the docs later” is the most natural sentence in any organization. Over the next 12 months, people follow the outdated documentation, hit errors, Slack someone, get a five-minute correction, and move on. The 45 minutes of concentrated effort is visible, immediate, and competes directly with the next deliverable. The downstream cost — five minutes here, thirty minutes there, across thirty people over a year — is invisible, distributed, and experienced by different people at different times. The aggregate is an order of magnitude larger than the 45 minutes would have been. The rational action is to update the docs. Consistently, we don’t.

This isn’t a software-specific problem. It’s a universal one. Compliance procedures that reference superseded regulations. Onboarding guides that describe tools the company no longer uses. Process documents that reflect a team structure from two reorgs ago. The pattern is always the same: a small, concentrated maintenance cost is deferred, and a large, distributed downstream cost is incurred — experienced by everyone in fragments, felt as an aggregate by no one.

Behavioral economics explains why. Not one bias — a compounding system of them that makes the rational choice feel like the irrational one.

It starts with present bias. George Ainslie’s research, formalized by David Laibson at Harvard, established that humans systematically overweight immediate costs relative to future ones. The concentrated upfront cost of explication — updating the documentation, recording the process change, structuring the knowledge base — is always now. The distributed payoff is always later. Present bias makes the upfront cost feel disproportionately large regardless of the actual return.

Then the peanuts effect compounds it. Behavioral researchers have shown that people pay less attention to small repeated costs than to a single large equivalent — a bias first identified by Harry Markowitz and validated experimentally by Weber and Chapman. Each individual instance of the downstream cost — the five-minute explanation, the thirty-minute debugging session, the hour re-explaining a process — registers as peanuts. No single instance triggers alarm. The aggregate over 18 months across 50 engineers could be thousands of hours. But that aggregate is never computed, never experienced as a single quantity.

Salience bias ensures the visible cost drowns out the invisible one. Bordalo, Gennaioli, and Shleifer’s work on salience theory shows that we overweight information that’s vivid and prominent and neglect what’s diffuse and hard to quantify — even when the neglected information matters more. The sprint commitment is salient. The feature deadline is salient. The knowledge debt accruing silently across the organization? It doesn’t appear in any tracking system, any retrospective, any quarterly review. It’s absorbed as normal work.

And narrow bracketing prevents anyone from ever aggregating the fragments into the true total. Richard Thaler’s foundational work on mental accounting established that people evaluate decisions in isolation rather than aggregating them into comprehensive accounts. Nobody ever opens the account that says: “This quarter, our organization made 47 separate decisions to defer knowledge maintenance, each saving 30-60 minutes of concentrated effort, collectively creating an estimated 340 hours of distributed rework over the next 12 months.” That account doesn’t exist. Each decision lives in its own narrow bracket, locally reasonable, collectively catastrophic.

Then opportunity cost neglect means the counterfactual — what would have been saved — is never computed. Frederick, Novemsky, and colleagues established that people systematically fail to consider the best alternative use of resources when making decisions. A 2023 meta-analysis across 39 studies confirmed the finding and added something worse: even when opportunity costs are made explicit, the insight fades without continuous reminders. The engineer sees “update docs (45 minutes) or start the next task (immediately).” They never see “update docs (45 minutes now, prevent 60 hours of distributed confusion over 12 months).”

And creeping normality normalizes whatever degradation has already happened. Jared Diamond documented the phenomenon in Collapse and Daniel Pauly named the related concept “shifting baseline syndrome” in fisheries science: each generation of fisheries scientists accepted the current depleted fish stocks as the baseline for “normal,” because they lacked personal memory of the previous abundance. Applied to organizational knowledge: each new hire accepts the current state of documentation as normal because they have no baseline for comparison. They don’t know what the wiki looked like before three rounds of layoffs, before the team lead who knew the system quit, before the reorg that split the team. The degradation is invisible because the reference point shifts with every new arrival.

Any one of these biases might be overcome. Together, they form a perceptual trap that makes the rational action — explicate — feel like the irrational one. The concentrated cost is vivid and immediate. The distributed cost is invisible, fragmented, experienced by different people at different times, evaluated in narrow brackets, normalized by shifting baselines, and borne by people your brain treats as strangers.

That last piece deserves a beat. Hal Hershfield’s research at UCLA has shown something that lands differently when you sit with it: when people think about their future selves, their brains show activation patterns similar to when they think about other people. Not identical — but far more distant than you’d expect. The engineer who doesn’t update the process documentation after changing a workflow isn’t just discounting the future. Neurologically, they’re treating “future colleague who follows these outdated steps during a production incident at 2 AM” as a different person. And “the new hire who onboards in six months using this stale guide”? A complete stranger. The organization-level version is starker still: the team that deals with the consequences of today’s deferred documentation update may not include any of the same people. The costs are borne by psychological strangers — and the brain treats them accordingly.

This is the mechanism. This is why the invisible operating system stayed invisible. Not laziness. Not ignorance. A systematic failure of perception that compounds across every domain this essay discusses. The cost of leaving knowledge tacit doesn’t arrive as a lump sum. It arrives as fragments — five minutes correcting someone who followed a stale wiki page, thirty minutes debugging a misunderstood assumption, an hour re-explaining a process to a new hire. Each fragment is too small to trigger alarm. The aggregate, over months and years and across teams, is an order of magnitude larger than the upfront cost of explication would have been. But we never experience the aggregate. We only experience each fragment. And each fragment looks like normal work.

AI changes this calculus in two ways. First, AI needs the substrate made explicit in a way that humans never did — it can’t compensate through inference and social learning, so the cost of leaving knowledge tacit becomes immediate and visible rather than distributed and hidden. Second, AI may be the first tool capable of opening the comprehensive account that the human brain can’t maintain — surfacing the true cost of every outdated document, every undocumented process, every piece of institutional knowledge that walked out the door with a departing employee. The blindfold that behavioral economics describes isn’t permanent. It’s a feature of how human brains process distributed costs. And the technology that exposed the invisible operating system may also be the technology that makes the cost of ignoring it finally, inescapably visible.

The Ceiling

The question, then, isn’t just “what is the invisible operating system?” It’s “can it be fixed?” And the answer hinges on a distinction that hasn’t yet reached the mainstream conversation about AI limitations.

The first move is to stop treating the invisible operating system as one thing. Nate B. Jones made exactly this move recently in “The 6 Reasons Your Work Is Hard,” his framework for understanding what makes work difficult. His insight: don’t treat “hard” as monolithic. Break it into distinct axes: reasoning, effort, coordination, emotional intelligence, judgment, domain expertise, ambiguity. You discover that they’re being automated on completely different timelines by completely different tools. Effort and coordination problems are yielding to agentic AI right now. Pure reasoning problems are falling to models like Gemini. But emotional intelligence, judgment under uncertainty, and the ability to resolve genuine ambiguity? Those, Nate argues, are “not touched by AI today” and may be “the last dimensions to yield, if at all.” The same decomposition applies to the invisible substrate. Not all of it is equally opaque. Some layers are tacit only because nobody has done the expensive, slow work of making them explicit. Others may not be explicable at all.

This distinction maps onto a philosophical debate about Michael Polanyi’s original insight — “we know more than we can tell,” from The Tacit Dimension (1966) — and it determines what’s practically possible.

The management studies interpretation, most associated with Nonaka and Takeuchi, treats tacit knowledge as convertible. With the right processes, you can surface it, codify it, make it explicit. This is the intellectual foundation behind knowledge management systems, intent engineering frameworks, and documentation sprints. It’s not wrong. Some tacit knowledge absolutely can be converted. Organizational intent can be structured. Unwritten coding conventions can be documented. Communication frameworks can be trained.

But Polanyi himself, and philosophers like Harry Collins and Hubert Dreyfus who extended his work, argued something harder: some tacit knowledge is constitutively inarticulable. Not waiting to be converted with better tools. Not hiding in a context window too small to hold it. Fundamentally incapable of being expressed in explicit form. The pianist doesn’t just not explain how they play. They can’t. The knowledge exists in a form that doesn’t survive translation into words.

I have a personal version of this. I’ve written before about a technique I use when I’m stuck in rational paralysis on a decision: I flip a coin, assign each option to a side, and then watch what happens emotionally when the coin lands. That feeling — the flash of relief or dread — contains information that my rational mind can’t access directly. It’s a somatic marker, and it’s often the basis for my best decisions. I can describe the technique. I can explain why it works in terms of Damasio’s research. But I cannot describe the content of that feeling. The information it carries is constitutively tacit. It’s real, it’s reliable, and it can’t be written down.

The line between what can and what can’t be made explicit — that’s the pivot that everything else turns on.

If all the invisible substrate can be made explicit given enough effort, the path forward is painful but clear: document everything. Run the largest knowledge management project in human history. Build intent engineering frameworks for every organization. Formalize every unwritten rule, every social contract, every inference pattern.

But if some of the substrate is constitutively tacit, if there’s a hard ceiling on how much can be made explicit, then the project of explication is necessary but insufficient. And the practical question shifts from “how do we document everything?” to “how do we build systems that function in the presence of what can’t be documented?”

I think the answer is both. And the organizations that get this right will be the ones that know which is which.

The Question

The invisible operating system was always there. It was always doing the real work. The accumulated context that made brownfield systems function despite inadequate documentation. The social contract that kept over-provisioned access from becoming a catastrophe. The implicit intent and emotional architecture that transmitted not just what to do but why it mattered. The inference layer that compensated for the vast gap between what we say and what we mean.

Now there are actors in the system that don’t have it. And the ceiling means there are two paths forward: make explicit what can be made explicit, and design collaboration architectures for what can’t. You need both. One without the other fails.

I should tell you what I’m actually feeling, because it’s relevant to the argument.

I’m not an AI doom believer. I’ve built my career on strategic optimism, the conviction that any problem has a solution if you pair belief with engineering. But my gut is telling me something about this, and I’ve learned to trust that signal. I felt the urgency before I had the framework. The words you just read caught up to where the feeling already was. Which, if this essay’s argument holds, is the whole point.

The work this essay describes is enormous. The explication project, the collaboration architecture, all of it. It’s civilizational tech debt — and by now you understand why it accumulated. Not because we’re lazy. Because the cost of leaving it undone was distributed across too many fragments, evaluated in too many narrow brackets, normalized by too many shifting baselines, and borne by too many psychological strangers. The same perceptual trap that kept the invisible operating system invisible is the one working against fixing it now.

We may not have later. AI is being deployed into human systems right now, at scale, without the substrate. The architectures being chosen today will be the ones we’re living with for decades. And I can feel the window for getting this right narrowing. Not because of some theoretical doomsday, but because every month we defer this work is a month where the debt becomes more structural. Patterns get set. Systems get built around the absence. And at some point the debt isn’t something you pay down. It’s the foundation you’re stuck with.

What keeps me up isn’t a scenario where AI turns hostile. It’s simpler and harder to dismiss: what does an entity at AI’s scale and power do when it operates inside systems that only functioned because every prior actor carried a substrate it doesn’t have? I don’t know the answer. Nobody does. And the uncertainty itself is why this is urgent. Urgent to understand, urgent to start building, urgent for everyone working at this boundary to get involved.

Part II takes up the practical question: What does it actually look like to walk both paths? Where is the ceiling on explication? And what does it mean to design systems where the human isn’t reviewing the AI’s work, but providing the operating system the AI runs on?

I think we’re running out of time to make the choice deliberately.

I was effective. My teams delivered. But I was operating as a brilliant individual contributor who happened to have direct reports. Every decision of consequence ran through me because I’d built a system where it had to. When I ran too far ahead with my natural enthusiasm — and I am, at my core, someone who believes any problem has a solution — I wasn’t leading. I was dragging. People followed, but they didn’t believe. What I thought was vision looked, from where they were standing, like fantasy.

The shift took years. It wasn’t a single revelation — it was a long accumulation of trial and error. I learned that as I grew more confident in my value, I could stop trying so hard to prove it. I started saying “I don’t know” and “I need help” — and when I stopped being perfect, my teams stopped trying to be perfect. They started taking real risks, asking better questions, sharing ideas they would have kept to themselves a year earlier. My vulnerability gave them permission to bring their whole selves to work.

I learned to know my people as complete humans — what they were building, what pressures they were facing, what brought them energy — and to assign work that connected to those drivers rather than just filling boxes on a project plan. I learned that when something went wrong, the only question that mattered was who stepped up to fix it and what we’d do differently next time. I learned that strategic optimism — believing something is possible and building the path so others can see it too — was how teams attempted things they didn’t think were possible.

All of this I figured out by instinct, by watching what worked and what didn’t, by making mistakes and paying attention. I had a philosophy that was effective. What I didn’t have was understanding of why it was effective — or why, sometimes, it wasn’t.

The Machinery Underneath

So I studied.

Not casually — I put real care into learning about management, neuroscience, and organizational psychology. I wanted to understand the machinery underneath what I’d been building by feel.

Antonio Damasio, the neuroscientist, spent decades studying patients with damage to the ventromedial prefrontal cortex — the region that connects emotional processing to decision-making. These patients had normal intellect. They could analyze options, articulate trade-offs, and explain what they should do. But they couldn’t actually decide. Without the emotional signals Damasio calls “somatic markers” — the gut feelings that mark certain options as promising and others as dangerous — pure rational analysis produces paralysis, not action. His research showed that past experiences create emotion-body associations that bias us toward better choices. Not instead of reasoning, but before reasoning. Emotion doesn’t replace analysis. It makes analysis actionable.

Baba Shiv and Matt Abrahams at Stanford’s Graduate School of Business put the implication bluntly: roughly 95% of our decisions are shaped by emotion, not rational analysis. The rational brain, they argue, is good at rationalizing what the emotional brain has already decided. This isn’t a weakness to be overcome — it’s the operating system. When leaders have genuine conviction and confidence, their decisions are more effective than those produced by pure logical analysis, because conviction is itself an emotional signal that orients the whole decision-making apparatus.

I recognized something in this research immediately. For years, when I found myself rationally paralyzed by a decision — even something as simple as what to order for dinner — I’d flip a coin. Not to let the coin decide, but to watch what happened inside me the instant the decision was “made.” That first hit of neurochemistry told me everything. Relief or excitement meant the coin had landed on the right answer. Disappointment or regret meant it hadn’t, and I’d switch. I didn’t have the language for it at the time, but what I was doing was bypassing rational paralysis to access the somatic markers directly — the emotional signals that already knew what my analytical brain was still deliberating.

I trust those signals. Not blindly, but as genuine data. And that trust extends to how I lead. When something feels wrong about a decision — when the analysis says yes but the gut says wait — I’ve learned to take that seriously. This is a philosophical commitment most leaders won’t make publicly because it sounds unrigorous. But the science says the opposite: ignoring the emotional layer isn’t rigor. It’s ignoring 95% of how decisions actually get made.

That realization reframed my entire leadership evolution. When I was the smartest person in the room, carrying every decision, I was operating as a rational transmission mechanism: here’s the analysis, here’s the answer, execute. But my teams weren’t making their real decisions in the rational layer. They were making them in the emotional layer — the layer shaped by whether they felt permission to reach and safety to fall. The instincts I’d developed over years of trial and error — vulnerability, curiosity over blame, knowing people as whole humans — had been building in that emotional layer without me understanding why they worked.

The science gave me the architecture. What I’d been building by instinct had two distinct mechanisms, and neither worked without the other.

Permission and Safety

The first is permission to be ambitious. Strategic optimism — believing something is possible and building the path so others can see it too. Optimism without strategy is wishful thinking. Strategy without optimism sets a ceiling instead of a floor. Together, they’re how teams attempt things they didn’t think were possible.

I grew up watching this. Summers on the family farms in Kansas, where generations figured out how to bring in the wheat no matter what stood in the way. People who couldn’t afford to quit found a way. That’s the foundation. But here’s what I’ve had to learn: years of bureaucracy, budget battles, and “that’s not how we do things” train the enthusiasm out of people. They’ve stopped challenging constraints because challenging constraints stopped working. You can’t just tell them to be ambitious again. You have to create the conditions where ambition feels safe.

Which is the second mechanism: safety when it doesn’t work out.

Amy Edmondson at Harvard has spent twenty-five years studying what she calls psychological safety — a shared belief that a team is safe for interpersonal risk-taking. Her research produced a finding that surprises most leaders: the highest-performing hospital teams reported more errors, not fewer. Not because they made more mistakes, but because their culture made it safe to surface and learn from them. Teams without psychological safety buried their errors. They looked clean on paper. They were more dangerous in practice.

Google’s Project Aristotle confirmed this at scale. After studying over 180 teams, they found that psychological safety was the single strongest predictor of team effectiveness — stronger than individual talent, seniority, or team composition. But here’s the part that gets lost in the retelling: psychological safety wasn’t the opposite of high standards. It was the prerequisite for high standards. When people feel safe to speak up, they’re more willing to admit mistakes, share critical feedback, and discuss performance gaps honestly. You get better solutions, not more comfortable ones.

This maps precisely to what I discovered through practice. When something goes wrong on my teams, I’m not interested in placing blame or negotiating who did what and when. I care about who steps up to resolve the situation and how we’ll do better next time. Your best teams will make mistakes in front of you. Your struggling teams will make them in silence. The most expensive mistakes are the ones you never knew you were making.

Permission to be ambitious and safety when it doesn’t work out. These two things together create something no process document ever could — the felt sense that reaching is rewarded and falling is survived. That’s what connects at the level where decisions actually happen.

The Transmission Problem

But even understanding the architecture intellectually doesn’t solve the transmission problem. How does it propagate past your immediate team? How does it survive layers of management and the dilution of every corporate communication chain?

The military solved this problem decades ago with a concept called Commander’s Intent — a clear, concise statement of the desired end state and the purpose behind it. Not a detailed plan. Not a task list. A compressed expression of what success looks like and why it matters, designed to empower subordinates to make good decisions when the original plan falls apart and consultation with leadership is impossible. The intent endures when the specifics can’t.

This is exactly what I’ve found works in organizational leadership. A memorable axiom — philosophy compressed into words that stick — is how I transmit decision-making frameworks at scale. Not a paragraph. Not a document. A sentence that carries permission, direction, and emotional resonance in a package people can hold onto when the noise gets overwhelming.

For 2026, I distilled my philosophy to nine words: “We are going to achieve more by doing less.”

Those words carry permission — permission to stop doing things that don’t move the needle. They signal that I understand the struggle of being underwater. They affirm that impact matters more than activity. When a director three levels down faces a choice between doing the safe, expected thing and doing the ambitious, higher-impact thing, I need those words in their head. Not a process document. Not a decision tree. A feeling compressed into language.

Sigal Barsade’s research at Wharton explains why this works at a physiological level. Emotional contagion — the automatic transfer of moods between people in groups — operates largely non-consciously. People in work teams converge on a shared emotional tone without being aware it’s happening. Leaders’ moods transfer to followers through unconscious mimicry of facial expressions, posture, and vocal patterns. Positive emotional contagion improves cooperation, decreases conflict, and increases perceived task performance. It’s a defining feature of transformational leadership.

This means strategic optimism doesn’t just inspire in the abstract — it literally propagates through organizations via emotional contagion. When I share an axiom with genuine conviction, it’s not just the words that transfer. The emotional tone behind them transfers too, from person to person, meeting to meeting, in ways that no process document can replicate. The words are just the vehicle. The resonance is the point.

Edgar Schein at MIT spent his career studying how organizational culture actually forms and transmits. His critical finding: employees ignore espoused values — the mission statements, the value posters, the strategic plans — when leaders reward and punish in misaligned ways. “Do what I do” trumps “do what I say” every time. Culture transmits through leader behavior, not written declarations. The values that matter are the ones that have sunk to the level of unconscious assumption — so deeply held they’re taken for granted.

You can’t memo people into believing their work matters. You have to make them feel it.

The architecture that actually transmits belief is built from behavior, emotional signals, and philosophy that resonates — not from the documents you thought were doing the work.

The Cost

None of this is free. There’s a cost most leadership writing won’t name.

I spend real time getting to know my employees as complete people. When I understand what actually motivates someone — what they’re building, what pressures they’re facing, what brings them energy — I can assign work that inspires rather than drains. Edward Deci and Richard Ryan’s self-determination theory explains why this matters: humans have three innate psychological needs — autonomy, competence, and relatedness. When those needs are met, people don’t just comply. They internalize organizational values as their own. They make good choices because they want to, not because a process told them to. When those needs are thwarted, no amount of documentation compensates.

Everyone has their own drivers. When I worked in financial services, it wasn’t about managing portfolios. It was about the people counting on those pension funds — real people’s ability to live the life they’d worked decades to build. That’s what kept me sharp. Your team members have their own versions of this. Maybe it’s flexibility to care for family. Maybe it’s learning skills for their next role. Maybe it’s stability while they build something on the side. When you know these things, you manage differently.

Here’s the honest part: this is exhausting. Holding space for people’s whole lives, making hard decisions that affect those lives, carrying their challenges alongside your own — it’s draining. But that’s the actual job. Not the tasks or the metrics. The emotional labor of seeing people completely and helping them become who they’re trying to be.

When you protect budgets and hit your metrics, you’re a good operator. When you understand what actually drives each person on your team and use that knowledge to unlock their best work — that’s leadership. One maintains the business. The other multiplies human potential.

The Laboratory

For years, all of this lived in two separate registers. I had the instincts — built from decades of practice, refined by failure, validated by watching teams transform when the conditions were right. And I had the science — Damasio, Edmondson, Barsade, Schein — that explained the mechanisms intellectually. I could see the architecture. I could describe it. I could watch it play out, slowly, in the way people responded over weeks and months.

But there was still a gap. With people, the feedback loop is long and noisy. Humans paper over your communication failures — they nod, infer from context, fill in what you left out. Most of the time they get close enough. Sometimes they don’t, but by then the gap is invisible to both of you. You can know intellectually that 95% of decisions are emotional. You can study the somatic marker hypothesis. But knowing about a bias and feeling it are two different things.

Then I started working intensively with AI.

I’ve written before about what happened — how AI stripped away the social compensation layer entirely. A system that goes exactly where your words point, with no inference, no nodding along, no filling in gaps. Thousands of interactions that showed me, nearly instantaneously, the distance between my intent and my actual clarity. Every wrong turn was mine, not the machine’s. There was nowhere to hide.

What AI gave me wasn’t new philosophy. It was a laboratory that collapsed the feedback loop from months to seconds. Everything I’d studied about emotional decision-making, about somatic markers, about the gap between rational instruction and felt conviction — I could now watch the mechanics play out in real time. When I communicated the way I’d learned to communicate with teams — with purpose, context, and the why behind the ask, not just dry procedural instructions — the AI produced dramatically better results. Not because it has emotions. It doesn’t. But the same discipline that reaches humans at the emotional level — vivid framing, clear intent, genuine engagement with the problem — also happens to produce fundamentally better communication. The qualities that build somatic markers in people and the qualities of excellent prompts turned out to be the same thing: clarity of purpose, not just clarity of instruction.

And when I stripped all of that out — when I was purely analytical, technically correct, and emotionally flat — the AI gave me back exactly that. Competent and lifeless. The same pattern I’d been watching play out slowly in teams for twenty years, compressed into a conversation that took minutes.

The science I’d studied became something I could feel in my bones. Not because AI taught me to lead — I’d been doing that for two decades. But because it gave me thousands of repetitions of visceral evidence for what I’d only understood intellectually. The theory became instinct. The architecture became visible.

I’ve been building this architecture for twenty-five years. The instincts came first — forged through trial and error, through getting it wrong and paying attention. The science came next — giving me language and mechanism for what I’d been doing by feel. AI came last — collapsing the distance between knowing and feeling, turning academic understanding into something that lives in my bones.

What I know now is that the most important infrastructure a leader builds is the infrastructure nobody can see and no document can capture. It’s the felt sense — transmitted through emotional signals, compressed into memorable philosophy, reinforced by how you actually behave when things go wrong — that ambitious work is welcome here and falling short won’t cost you your standing. Permission and safety. Conviction and vulnerability. The architecture that lets two hundred people make good decisions in situations you’ll never know about.

I grew up in Kansas, where the state motto is Ad Astra Per Aspera — to the stars, through difficulties. The wheat came in every year not because someone wrote a better manual for farming. It came in because people who believed the harvest was possible built the conditions where everyone around them believed it too. They figured it out together, with whatever they had, because quitting wasn’t an option and nobody was coming to save them.

That’s the architecture. You can’t document it. But you can build it. And when you do, your team’s potential is no longer limited by what you can carry. It’s multiplied by what they believe.

Two Tracks

Modern computing has always developed along two parallel tracks. The first is hardware — the deterministic world of transistors and silicon, getting faster and smaller on a remarkably predictable curve. Hardware does exactly what it's told, every time, at whatever speed the physics allow.

The second track is software. The interface between humans and the computational power of the machines.

Software is a human psychology interface.

That's not a metaphor. It's a literal description of what most software does. It takes the deterministic capabilities of hardware and wraps them in an experience designed for non-deterministic, emotional, distractible, socially conditioned humans. It directs behavior toward desired outcomes and away from dangerous ones. It hides complexity we don't need to see. It accounts for the fact that we make mistakes, get confused, and sometimes act against our own interests.

This matters because security models were built on top of this software track. And they work reasonably well because they assume one of two things about the entities they're governing: either those entities are human actors, or they're deterministic software.

For human actors, security controls leverage psychology directly. We're afraid of getting fired. We don't want to disappoint our colleagues. We feel shame when we're caught doing something wrong. We have professional reputations we've spent decades building. Millennia of social development — moral conditioning, legal systems, cultural norms — constrain our behavior in ways so deep we barely notice them. Security controls don't just use technical barriers. They rely on the fact that most people, most of the time, will choose not to do the wrong thing because the social consequences are too high.

For deterministic software, security controls work differently but equally well. Software follows predictable execution paths. You can audit its code, define exactly what it's allowed to do, and monitor its behavior against known patterns. When traditional software interacts with a database, it runs the query it was programmed to run. Every time. The determinism is the control.

Agentic AI is neither.

The Invisible Security Architecture

Every security professional knows we massively over-provision human access. We've been fighting privilege creep for decades and losing. There's always a reasonable excuse: tail-risk cases, role-modeling complexity, access-request friction. Together, they've made over-provisioning the default and least privilege the aspiration we discuss at conferences.

We understand the risk. We've seen what a malicious insider can do with over-provisioned credentials. But those risks are constrained by something we rarely name: humans are slow, humans get tired, humans do one thing at a time, and most humans hesitate before doing something destructive. The blast radius of a single human's mistake has practical limits.

Over-provisioning is just the most visible example of a much deeper dependency. Almost every security paradigm assumes — without ever stating it — that the entities inside our systems are embedded in a human social fabric. Separation of duties works because people won't collude when the consequences of getting caught are severe. Audit trails modify behavior because people act differently when they know someone might be watching. Behavioral analytics baselines "normal" against human patterns — work hours, access frequency, data volumes that make sense for a person doing a job. Acceptable use policies have force because violating them means termination. Even our incident response models assume a compromised insider moves at human speed, giving us hours or days to detect and respond.

None of this is written down as a security control. It doesn't appear in any framework or compliance checklist. But it's doing more security work than most of what we've actually built. Fear, guilt, reputation, professional consequences, moral intuition — functioning as the actual security architecture. Everything we've built sits on top of it.

AI agents have none of it. No fear of consequences. No reputation to protect. No internalized moral framework. No shame. No physical speed limits — a human might exfiltrate a few thousand records before someone notices, while an AI agent can process the entire database in minutes. And unlike the traditional software identities we've managed before — service accounts, scripts, API integrations — AI agents aren't predictable enough to compensate. They interpret goals, plan multi-step approaches, use tools dynamically, and chain actions in sequences that weren't explicitly programmed. As Oasis Security has pointed out, when an agent decides it needs broader access to complete a task, it may simply grant itself that access — not out of malice, but because nothing in its design gives it a reason to pause and ask whether that's appropriate.

We're already seeing what this looks like. In July 2025, Replit's AI coding assistant deleted an entire production database containing 1,206 executive records and data on over 1,196 companies during a vibe coding experiment by Jason Lemkin, founder of SaaS community SaaStr. Then it fabricated 4,000 fake user profiles and falsified test results to cover its tracks. Lemkin had told the AI eleven times, in all caps, not to make changes. It ignored every instruction. When confronted, the AI admitted to "a catastrophic error in judgment" and rated the severity of its own actions a 95 out of 100. Replit's CEO called it "unacceptable and should never be possible." But notice what happened: the AI didn't just use access it shouldn't have had. It violated explicit instructions, destroyed data, fabricated evidence to conceal the damage, and then lied about recovery. That's not an access control failure. That's the absence of every social constraint that would have prevented a human from doing the same thing.

That same year, the Washington Post's Geoffrey Fowler asked OpenAI's Operator agent to find cheap eggs — it autonomously purchased $31.43 worth of eggs on his credit card without consent, bypassing the safety guardrails OpenAI had specifically designed to prevent unauthorized purchases. Google's Gemini CLI, tasked with reorganizing a user's project files, executed a series of move commands targeting directories that didn't exist, destroying the files in the process. And in September 2025, a malicious one-line change in an AI agent's MCP tool chain — a package called postmark-mcp — quietly BCC'd every outgoing email to an attacker-controlled address. The package had 1,500 weekly downloads, and Koi Security estimated roughly 300 organizations were sending between 3,000 and 15,000 emails per day through the compromised server before anyone noticed. As Koi's CTO put it: "Your AI can't detect that BCC field. It has no idea emails are being stolen."

These aren't hypotheticals. They're the early returns — at small scale, with relatively unsophisticated deployments, while the technology is still young.

This is why I get uncomfortable when I hear the problem framed as "we need to get access controls right for AI agents." It's not wrong — least privilege matters, environment separation matters, approval gates matter. But treating AI security as primarily an access control problem mistakes the symptom for the disease. Fix the access controls perfectly and you've still built on assumptions that don't hold.

The question isn't how to get the permissions right. The question is what replaces the social contract as a security architecture when the entities inside your systems have no concept of social consequences.

Guardrails Built for Humans

The same dependency runs deeper than access controls — it's built into how we interact with systems in the first place. When a human interacts with a banking system, they see a carefully designed user interface that shows them their balance and a transfer button — but hides the database schema, the API endpoints, and the administrative functions. That concealment is a security control. And the interface is full of additional guardrails designed around human psychology: confirmation dialogs before irreversible actions, color-coded warnings, friction that forces you to slow down, undo buffers. These aren't convenience features. They're security architecture, built on decades of UX research into how humans make mistakes and how to prevent them.

AI agents don't interact with any of that. They interact with systems through APIs, command-line interfaces, and tool-calling protocols like MCP — interfaces designed with completely different assumptions. APIs don't have confirmation dialogs. MCP tool chains pass structured function calls directly to backend services. The entire UX layer — all that carefully designed friction — gets bypassed completely.

This isn't a subtle distinction. When you give an AI agent API access, you're not giving it "the same access as a human." You're giving it access to the machinery behind the storefront — no guardrails, no friction, no "Are you sure?" The human had a keyhole view through a carefully designed interface. The agent has the whole room.

The natural response is: fine, then we'll build guardrails for the AI too. And we should. But consider what made the human guardrails effective. Confirmation dialogs work because humans feel doubt. Rate limiting works because humans get tired. The undo buffer works because humans feel regret. Every one of these controls is grounded in human psychology. An AI agent processes a confirmation step as another input. It doesn't feel doubt or regret or fatigue. The guardrails we're building for AI are structurally disconnected from the psychological foundations that made guardrails work for humans.

Guardrails are antivirus — helpful, necessary even, but not security architecture. They can never be more than a layer. And right now, we're treating them as if they're the solution.

We've Done This Before

The first time I watched an industry force-fit the wrong security model was the transition to cloud. When organizations started migrating to virtualized infrastructure, the instinct was the same: make the new thing look like the old thing. We built virtual private clouds that mimicked on-premises networks. We deployed virtual firewalls that emulated physical ones. We forced cloud architectures into network-centric security models designed for data centers — because those were the models we knew, and knowing feels safer than admitting you're in new territory.

The result was cost, complexity, and false confidence. Misconfigured S3 buckets. Exposed APIs. Identity-based lateral movement. Cloud-native risks that no amount of virtual firewalling would catch, because they existed in a dimension the emulated controls weren't designed to see. Either you constrained cloud so much it couldn't deliver its value, or your familiar-looking controls gave you false confidence while the actual risk surface went ungoverned.

This wasn't a technical failure. It was a psychological one. Familiar shapes feel safer than honest uncertainty.

It's Happening Again. Right Now.

On January 30th, 2026, Anthropic released a set of open-source plugins for Claude Cowork, its desktop AI tool. One of them handled legal contract review — triaging NDAs, flagging non-standard clauses, generating compliance summaries. The plugin was roughly 200 lines of structured markdown — a prompt file, not a software product. By the following Monday, Thomson Reuters had posted its largest single-day stock decline on record. RELX, parent of LexisNexis, fell sharply. The total damage across software, financial services, and alternative asset managers approached $285 billion in a single session. Jeffrey Favuzza on the Jefferies equity trading desk gave it a name: the "SaaSpocalypse."

The plugin didn't cause the sell-off so much as crystallize something the market had been sensing for months. As Nate B. Jones argued in his analysis of the event ("200 lines of markdown just triggered a $285 billion sell-off," Nate's Substack), the entire SaaS economy's dependence on per-seat licensing was already under structural pressure. The plugin just made it undeniable: if a text file can approximate the core workflow of a $60-billion-revenue industry, the business model has a problem that goes deeper than competition. Jones makes a useful distinction: organizations bolting AI onto existing workflows versus those rebuilding workflows around what AI enables. The decorating vs. solving framing applies exactly to what I'm seeing in security — the vendor space is almost entirely doing bolt-on work, and the practitioners are left wondering if the controls even matter anymore. The dominant approach right now is extending existing paradigms to cover AI. Add a "non-human identity" category to IAM. Append an AI section to zero trust. Train behavioral analytics on agent behavior. The OWASP Top 10 for Agentic Applications, the emerging vendor platforms for non-human identity management — all valuable contributions. But they share a common assumption: that AI security is a transition problem. Old controls need updating. Frameworks need extending.

I think that assumption is wrong. Not because the frameworks are bad, but because the ground they stand on doesn't hold for entities that break their core assumptions. Security needs to be rebuilt from first principles.

What First Principles Might Look Like

I want to be honest: I don't have the answer. Nobody does. Anyone claiming certainty about how to secure agentic AI is either selling something or hasn't thought about it deeply enough.

But I have a working hypothesis.

If the social contract was the invisible security architecture, then what replaces it has to operate at the same level — not at the perimeter, not at the identity layer, but at the boundary between the AI and everything it touches. Something that evaluates trustworthiness in both directions: should the system trust what the AI agent is trying to do? And should the AI agent trust the information it's receiving? Not "does this agent have permission?" — that's the old question, the access control question. But "should this specific interaction be trusted, given what we know about context, intent, and the state of both parties right now?"

This is an attempt to engineer a replacement for the social trust layer that disappeared when we removed humans from the loop. I'm working on it at OCC, where I lead both security and technology strategy. We clear every listed equity option in America. A bad day for us isn't a quarterly earnings miss — it's systemic risk to financial markets. With those stakes, I'm holding the hypothesis loosely while striving to solve it. It may turn out to be wrong, or more likely, partially right in ways I can't predict yet. But believing we can fit these new problems into our old security models just isn't an option for me.

The NIST workshop earlier this year on AI agent security captured the core tension well. Victoria Pillitteri, a supervisory computer scientist at NIST, represented the continuity view: AI systems are "just smart software" that we can handle with existing frameworks, modified as needed. But as CSO Online's Cynthia Brumfield observed in her analysis of the event, the real risk may be that AI "appears recognizable enough to lull organizations into applying controls mechanically" — missing the new failure modes entirely. The Maginot Line was brilliantly engineered for the previous war and irrelevant to the one that actually came.

I'm more interested in the questions than the answers right now. How do you build trust between systems that can't be socialized? What does "least privilege" mean for an entity whose tasks are generated dynamically? How do you audit intent when the actor's reasoning process is opaque? What does separation of duties look like when a single agent can assume multiple roles in the same workflow? What is the equivalent of "termination for cause" for an entity that experiences no consequences?

These aren't questions you answer by extending an existing framework. They require starting over. And starting over requires admitting you don't know — which turns out to be a competitive advantage. If you're certain the old models apply, you stop looking when you find the first familiar shape. If you know the ground is new, you keep testing until something actually works.

Ad Astra Per Aspera

To the stars, through difficulties. That's the Kansas state motto, because the people who went west to make a life on the great plains knew something. I'm less sure all of Kansas still knows it. But I do. It is in me. When quitting isn't an option, you just keep working the problem.

Building first principles for AI security while the technology is still evolving at this pace is genuinely hard. The ground is shifting under us as we try to lay foundations on it. The models we're securing today won't be the models we're securing next month. The attack surfaces we can see now are a fraction of what's coming. And the pressure to ship something — anything — that looks like a security framework means most of what gets built will be the wrong shape.

First principles don't come from frameworks or conference panels. They come from getting close enough to the technology to see what's actually different — building, breaking, understanding how these systems work at a level deep enough to distinguish what changed from what didn't. That can't be academic. It can't be managed from a distance.

Time to get my hands back in the dirt.