Comprehension in the age of AI: are you managing reality or a convenient fiction?
Executive teams and boards alike assess, decide and provide strategic direction based on their understanding of the situation and capabilities of a company. How do they know it is real and accurate?
Executive teams and boards govern a model of their own organisation. That is their internal representation of an external reality that is multi-faceted and fast changing. An internal representation of an external reality is what cognitive scientist Stanislas Dehaene defines as learning. Hence executive and board leadership operate based on what they have learnt and understood of their company. But how can they be certain that their comprehension is the actual reality of the business? How can they confidently discharge their duties if there is a gap between that comprehension and reality? And how does the new digital reality of business grow that gap in comprehension?
The ability to perform executive and board duties with the appropriate precision and diligence depends on the accuracy and sufficiency of their comprehension of the organisation. That comprehension is shaped by what they see in documents and reports about the capabilities of their company or organisation, but where the capabilities are best documented is not in documents. Documents are just a picture, a painting of reality – and one that depends on the viewpoints as well as on the painter –, not reality itself. Reality is in code. It is in the collection of systems used by the people in the business to run the processes of the business and how these systems are assembled together. Hence, relying on the painting instead of the reality is like driving your car based on a drawing of the road, without ever looking at the road or at other drivers. Simply put, it is dangerous and that becomes increasingly true as systems become more capable of supporting the intelligence behind the way of working of a business. And that’s precisely where what is perhaps the greatest risk of our times lies: the delegation of comprehension to technical or documentary artefacts which may not reflect reality… or the longer term intent of the leadership of a business.
Product managers, product owners, scrum masters, product marketers and other professionals operating at the frontier between the functional and the technical reality of their company know this very well. They all have felt, in some form or other, the pain of the divergence between what they describe or define in terms of needs and functionality and the reality of the tooling that is developed or implemented to perform those functions. There are multiple reasons that explain this divergence and several methods designed to tackle the problem through iterations, tight coupling between business and tech people, retrospectives, pair programming, demos, visibility of impediments… etc. We won’t go into those. There’s ample literature about that topic, but one thing that is quite striking is that most professionals assume that it is quite impossible to understand the code that defines the functionality designed to deliver a business capability. And that certainly looks like a reasonable assumption. Much like it looks rather reasonable for boards to run strategic and governance aspects of a legal entity, that is the container of business value owned by shareholders, based on the strict separation between what is operational and what is of “higher” level.
The time may have come to challenge some of these principles, because, in a digital economy with strong reliance on machine learning capabilities, Board and executive committees cannot afford to be ignorant about code and software that embodies the complete knowledge, expertise and competitive advantage of their business. They should neither indulge in, nor be allowed to relegate these aspects to mere technical implementation status that is not in their purview. Because ultimately they cannot be allowed to operate on the basis of paintings instead of reality. Documentation and reports are just inaccurate representations of specific ways of understanding the business. They are paintings.
Tudor Girba and Simon Wardley make the observation about software architecture diagrams: hand-drawn views of a system are not photographs of reality but “paintings that document the perspectives of the authors,” and in a world of continuous change they are almost guaranteed to be wrong the moment they are made. The observation is more general than the domain it was made in. The picture a leadership team holds of how its own firm works – which capabilities are strong, where value is created, how a decision actually propagates from intent to outcome – is itself such a painting. It is composed from memory, from reporting lines, from the slides that survived the last off-site. And like any painting, it ages, flatters its authors, and quietly diverges from the thing it depicts.
In a connected, tool-mediated economy, comprehension – of strategy and of systems alike – is the one capability a firm must keep under its own control, because every instrument that relieves you of understanding also encodes a decision you did not make and cannot see. Comprehension can be supported, accelerated, instrumented. It cannot be outsourced. The moment it is – to a consultant’s best practice, to a software package’s defaults, to a model’s confident output – the firm has surrendered not a task but its capacity to govern itself.
The gap leaders do not see
Start with what the firm actually is. Not the org chart, which is another painting, but the operating reality: the systems that hold the data and run the transactions, the people who exercise judgement inside and around those systems, and the integration layers – e.g. middleware and API platforms – and processes that bind the two together. That is where the organisation’s behaviour lives. And in 2026 almost none of it is innocent of code. The systems are code by definition. The processes are increasingly encoded – in the ERP configuration, the workflow engine, the approval automation, the routing rules no one has read in three years. Even the people are mediated by code, because the judgement they exercise is shaped by the dashboard they are shown, the field the form will and will not accept, the recommendation the tool surfaces first. To understand how the firm functions is, to a degree that keeps rising, to understand what its code is doing.
This is precisely the comprehension that leadership lacks, and it is a strategic exposure rather than a technical inconvenience. The documented case Girba and Wardley report makes the point with uncomfortable economy: a large corporation spent years and millions trying to improve its central data pipeline, working from a high-level architecture diagram that turned out to be fiction. The diagram omitted an entire third-party system that was processing the company’s data – a fact no one in the organisation knew. They were not optimising the wrong thing. They were optimising a thing that did not exist as drawn. The remedy, when it came, began not with another attempt to fix the pipeline but with a different question: what does the system actually look like? Built into instruments that read the live system rather than the painting of it, that question turned a problem that had defeated hundreds of person-years into one solved in two person-months.
Treat that as a parable, not a product reference. The point is not the tooling that closed the gap; the point is that the gap existed at all, invisibly, at the centre of the firm’s main marketing channel, while competent people made – what they truly believed were – confident decisions on top of it. The number attached to the recovery – a six-hundred-fold improvement – is a single documented case and should be read as one illustration, never as a generalisable yield. What generalises is the structure: leaders governing a representation that had drifted from the reality, with no cheap way to tell.
Decisions made on belief
Software engineering, Girba and Wardley argue, is at root a decision-making activity – every change, every fix, every architectural choice is a decision, and the quality of those decisions turns on how fast and how truthfully one can move from question to answer. The argument is correct, and it does not stop at the engineering door. Strategy is a decision-making activity in exactly the same sense, and it suffers from exactly the same pathology: the prevailing mode makes high-stakes decisions on belief rather than evidence. The board approves an investment on the strength of a representation. The executive commits to a transformation on the strength of a benchmark. Neither has a low-cost way to verify that the representation matches the firm, or that the benchmark matches the firm’s situation. This is the generalisation worth making explicit, because it relocates comprehension from a cost centre in IT to a variable that governs decision quality across the enterprise.
The evidence that this is costly is sturdier than the slogans usually quoted. Large-scale field research finds professional engineers spending on the order of 58% of their time simply comprehending the systems they work on (Xia et al., 2017) — understanding is not a preliminary to the work; for the majority of the time, understanding is the work. At the project level, the empirical distribution of IT cost overruns is heavy-tailed, with only about a third of large public projects finishing within budget (Jørgensen & Moløkken-Østvold, 2022). At the level of strategic change, recent systematic review puts the share of digital transformation initiatives that fully achieve their intended outcomes at roughly 5–30% (ASEJ, 2024), with data quality repeatedly implicated in the failures (Azeroual & Jha, 2021). The other culprit is failure in managing the transition from a discrete, disconnected and slow set of processes to a complex, networked and adaptive system that becomes a totally new work reality shaped more by the practices that leverage the capabilities of the tooling than merely by the tools themselves. That is precisely why isolated ERP, CRM, e-commerce, MES or WMS initiatives are categorically necessary but not sufficient to achieve the kind of profound morphing that a digital transformation brings to companies. Let alone help their people deal with fast continuous change thereafter. Of course, a factor making the situation worse lies in too many system integrators operating as sales agents of makers of software packages, focusing more on their benefits as partners of the software makers than on their clients’ needs, constraints, capabilities and interests.
Girba and Wardley name two metrics for the decision loop – time to answer (how long to answer a specific question about the system / the business) and time to question (how long to formulate a proper one). The useful generalisation is not the vocabulary but the dynamic it captures. When understanding is expensive, an organisation can afford only a few questions, often resorting to cheap proxies instead of hard answers, and governance degrades into hope – a small number of bets placed and prayed over. When understanding becomes cheap, the binding constraint moves: it is no longer the ability to find answers but the imagination to ask the right questions. That is the moment comprehension stops being a technical convenience and becomes a strategic asset, because competitive advantage shifts to whoever can interrogate their own reality faster than rivals can interrogate theirs.
How firms surrender comprehension
If comprehension is the asset, the failure mode is its quiet surrender, which comes into the business through three doors, each of which presents itself as help.
The consultant, and the best practice. A strategy configured to “best practice,” or benchmarked on what competitors are seen to do, is not a neutral shortcut. It is someone else’s strategy, encoded and inherited. Best practice is, by construction, the average of what worked elsewhere, under conditions that were not yours, solving problems you may not have. To adopt it without comprehending it is to import a set of decisions – about where to compete, how to price, what to standardise, what to treat as undifferentiated… – that you never actually made. The work the consultant did was real; the danger is in receiving the conclusion without the comprehension that would let you judge whether it fits, or notice when the ground it assumed has moved. The deliverable transfers; the understanding does not, unless you insist on rebuilding it.
The package, and the default. Enterprise software is where the systems-people-processes point bites hardest, because here the process becomes the configuration. When a firm adopts a packaged system and accepts its defaults, the defaults are not empty – they encode a model of how a business like yours is supposed to run, made by people optimising for the median customer and for the supportability of their own product. Over time, the configuration silently becomes the process: work flows where the system permits it to flow, exceptions are handled the way the screen allows, and the organisation’s actual operating model migrates to wherever the software is path-of-least-resistance. None of this is decided explicitly with the people affected and an authority in the business exerting discernment as to what to apply. It is decided in the implementation by the package consultants, and then it governs the firm. A package you cannot interrogate governs you on belief, not evidence – and what it has quietly decided on your behalf is invisible precisely because it now is “the way we work.”
The model, and the answer. Artificial intelligence is the most powerful instrument yet built for relieving people of the labour of understanding – or at least that’sƒ how it’s sold by AI labs and panicked technology giants of the previous wave of digitalisation. That is exactly what makes it the most powerful vector for this failure. The same capability that collapses the cost of an answer can, trusted blindly, deepen the belief-not-evidence problem rather than cure it, because it manufactures fluency. AI’s competence is frequently performative: an appearance of understanding without the substance, most dangerous on the deep problems where the distance between what you observe and what is actually true is greatest. The independent evidence on even narrow assistance is more modest than the vendor claims – controlled study of code-assistance tools finds gains around 35% on task completion (Shihab et al., 2025), against vendor figures of 50–55% — and the gap between those numbers is itself a small lesson in not accepting a confident answer on trust.
This is the mechanism, not merely the mood. Storey (2026) argues that generative AI redistributes the burden rather than removing it: it can lower the most visible cost – the technical debt in the code – while silently accelerating two less visible ones, the erosion of shared understanding she calls cognitive debt and the loss of captured rationale, of what a system is for, she calls intent debt. The translation to the firm is exact. Every instrument that relieves leadership of understanding lowers a cost that shows – friction, headcount, time-to-answer – and raises one that does not: the distance between the painting and the firm. The trade reads as efficiency precisely because the cost it incurs appears on no report.
The line that separates AI-as-aid from AI-as-substitute is not the model’s sophistication. It is whether its output is explainable and verifiable against the firm’s own ground truth, or accepted because it sounded right. An instrument that shows you how it reached an answer, in terms you can check against reality, augments comprehension. An instrument whose answer you take on faith replaces it – and replacement, here, is the failure. The question is not whether to use the tool. It is where, in the decision, it still matters to have a human who understands.
Partial comprehension and fast action
Past a point, demanding full comprehension before every move is analysis-paralysis dressed as prudence. More often than not we need to move without complete comprehension or maintain a degree of ambiguity to preserve optionality and the time-value dimensions of decisions, especially if the decision is non-reversible. Thus moving without complete comprehension is defensible when the following things are true at the moment of the move:
the decision-maker has a fair estimate of how much they actually understand – the degree of their own comprehension, including the likely size of what they cannot see;
there is an explicit, reckoned expectation of the outcome, weighted across the downside and not merely the case they hope for;
the range of consequences and outcomes does not include catastrophic impacts;
action caused by the decision is very likely to produce one or more lessons or discoveries that can be used to inform the next decision;
an explicit system or mechanism is in place to capture the lessons from the action undertaken, such that it can feed a process of discovery-driven transformation.
Consider the manager who force-releases an enterprise system before it is “ready.” Done in a way that meets the above criteria, the move can break an organisation’s healthy-looking resistance to change and create value that patience would have forfeited – tempo can be worth more than certainty, and the orientation that lets a leader act decisively inside a fast loop is, after Boyd, frequently the decisive move (Osinga, 2007). Done poorly or with a motivation that is not in the interest of the organisation that will bear the consequences of the decision, the same move sends eight people of the finance team into burnout, leaves a car-parts distributor unable to assemble next year’s budget, and renders the FP&A function inoperable for a quarter. Same action, opposite outcomes – and what separates them is not nerve but the ability to discern and understand the structure of the uncertainty if not the underlying reality on which the decision operates. The second manager did not place a comprehended bet; he placed an uncomprehended one that happened to detonate. “Acting to generate information” is a genuine justification, but only when the information is worth its price and the risk taken. An action whose expected value is negative once the tail is included is not a probe, but rather a loss with a narrative attached.
Why does the test turn on the situation as much as on the actor? Because situations differ in kind. Popper’s distinction between clouds and clocks is the useful one here: clouds are systems that are irregular, disorderly and genuinely unpredictable; clocks are orderly mechanisms whose behaviour can be known in advance – and his point was that the world is mostly cloud, with the clock as a limiting case rather than the rule (Popper, 1972). The question before any forced move is which one you face. In a genuine cloud – irreducible uncertainty, where no feasible amount of analysis would yield the answer cheaply – acting to generate information can dominate the patient gathering of evidence (Snowden & Boone, 2007); you learn by probing because there is no other way to learn. But the error runs in both directions, and comprehension is what keeps you from making it. Treat a clock as a cloud and you abandon understanding that was cheaply available, calling the laziness agility. Treat a cloud as a clock and you trust an analysis the situation cannot bear, calling the recklessness rigour. A deep problem made shallow by the right instruments, it is worth adding, is often a clock that merely looked like a cloud from the outside – which is exactly why the cost of comprehension, once it falls, can change whether the trade was ever necessary.
So forcing action ahead of full comprehension is wisdom rather than recklessness when, and only when, the bet is comprehended even though the outcome is not — and that resolves into the conditions we’ve described here, with two important implications. First, the cost of being wrong is bounded, or the move is reversible, so the loop can correct itself before the damage compounds. Second, the missing understanding sits in the shallow zone, where observation and root cause lie close together, rather than the deep zone, where confident action is simply confident ignorance. The manager who killed the budget cycle failed both at once: the cost was neither bounded nor reversible within the operating year, and the comprehension he lacked was deep. The enemy, here as throughout, is not action before certainty. Rather, it is the move taken on the appearance of a reckoning that was never performed – the unwitting bet that felt, from the inside, exactly like decisiveness.
Comprehension as non-specialist right and duty
If comprehension can be surrendered to a vendor or a model, it can also be surrendered to one’s own specialists, and the consequence is the same. An organisation in which only the technical function can read how the firm actually works has not solved its comprehension problem; it has relocated it to a place the board and executives cannot reach. Girba and Wardley’s argument for “universal literacy” – that generated, self-explaining views let people reason about a system without writing it, “democratising the reading of systems in a way that is decoupled from their writing” – is sometimes heard as an engineering nicety. In fact, it is a governance principle. The capacity to interrogate how the firm works must remain accessible to the people accountable for the firm, or accountability is fictional.
This connects comprehension to the theory of the firm directly. Penrose (1959) argued that a firm grows to the limit of its administrative capability – the binding constraint is not capital or demand but the organisation’s own capacity to understand and direct itself. Comprehension is the meta-capability beneath that: the one that bounds all the others, because a firm cannot deploy a capability it cannot see. And in the make-or-buy logic of transaction-cost economics (Coase, 1937; Williamson, 1985), comprehension is the rare thing that resists outsourcing even where execution does not. You can buy the building of the system, the running of the process, the drafting of the strategy. The understanding of what you have bought is not separable from ownership in the same way – outsource it and you have not bought a service, you have ceded control of the asset to whoever holds the understanding. Over time, as systems drift from their documentation and the painting ages, that understanding decays unless the firm actively maintains its own (Meadows, 2008).
The objection to press here is a fair one. Comprehension, the distributed-cognition tradition holds, never lived in a single head; it is a property of people, artefacts and their interactions, and organisations have always run on trust in reports no one person can independently verify (Storey, 2026). How much un-understanding is acceptable is genuinely unsettled. But the objection sets the standard, it does not dissolve it. The claim is not that every director must read code; it is that the firm’s comprehension must stay sufficient and interrogable from where the board sits. Distributed comprehension is fine; inaccessible comprehension is not. A board may rely on others to hold the understanding — not on no one being able to surface it on demand.
A board cannot reasonably hold that it is exercising due diligence and care over a business it does not understand – and the proof is in the two decisions a board most cannot avoid. Capital allocation is the first. To decide where capital expenditure should go is to decide which capabilities are worth owning rather than renting – and that presupposes knowing what the firm already has, what it is worth, and where it is weak. A board that allocates CapEx across a portfolio of capabilities it cannot see is not exercising judgement; it is ratifying a judgement made elsewhere, by whoever does understand. Sourcing is the second. Whether moving a function to an offshore provider creates value or quietly destroys it depends entirely on comprehending what that function actually does – its interdependencies, the tacit knowledge it carries, the failures it silently prevents. The transaction-cost logic that governs the make-or-buy decision (Coase, 1937; Williamson, 1985) cannot be run at all by a board that does not understand the thing being made or bought. Approve the offshoring of a function you do not comprehend and you may be banking a saving or amputating an organ; from the boardroom, without comprehension, the two look identical on the slide.
So the duty of care is not, in substance, discharged by diligence in the procedural sense – meetings held, papers read, votes minuted – when the papers depict a business the board cannot independently interrogate. A board that governs on the say-so of those it governs has delegated not execution but the very understanding of the substance that defines the economic value it manages on behalf of shareholders and, increasingly, other stakeholders. And with that understanding goes the very judgement the duty of care exists to protect.
Genuine, deep and constantly verifiable comprehension is not merely a desirable attribute of good governance. It is a precondition of governance at all.
The capability that cannot be outsourced
Return to the board governing a model, a painting, an approximation of the reality of the substance of the business. Nothing in this argument says the painting is worthless – strategic intent has to be drawn before it can be pursued, and a painting you can revise is better than no destination at all. The argument says only that the firm must keep its own way of checking the painting against the subject, the model against the firm, cheaply and continuously, and must never let any instrument do the checking on faith. The consultant’s best practice, the package’s defaults, the model’s fluent answer: each is useful in exact proportion to whether you can still see through it to your own reality, and dangerous in exact proportion to whether you cannot.
Comprehension is the one capability that, if delegated, undermines the firm’s capacity to choose its strategy, to govern its operations, and to remain recognisably itself rather than a configuration of someone else’s defaults. It is what is left when every executable task has been bought in – and it is the thing that determines whether the bought-in tasks add up to a firm that knows what it is doing or one that merely appears to. Reading your own systems, people and processes – the live state, not the representation of it – tells you where you actually are. Safe passage has always required both, and it has never been safe to let someone else hold the instruments.
Sources / references
ASEJ. (2024). Digital transformation success factors: A systematic review. Annals of Social and Economic Journal. https://asej.eu/index.php/asej/article/download/807/853/1381
Azeroual, O., & Jha, M. (2021). Without data quality, there is no data migration. Big Data and Cognitive Computing, 5(2), 24. https://www.mdpi.com/2504-2289/5/2/24
Coase, R. H. (1937). The nature of the firm. Economica, 4(16), 386–405.
Dehaene, S. (2020). How we learn: Why brains learn better than any machine… for now. Viking. (Original work published 2018 as Apprendre ! Les talents du cerveau, le défi des machines, Odile Jacob)
de Visscher, N. (2025). Architecture in action. Kien.
Girba, T., & Wardley, S. (2024–2025). Rewilding software engineering. feenk / Medium. https://moldabledevelopment.com
Jørgensen, M., & Moløkken-Østvold, K. (2022). The empirical reality of IT project cost overruns. Journal of Management Information Systems. https://www.tandfonline.com/doi/full/10.1080/07421222.2022.2096544
Meadows, D. H. (2008). Thinking in systems: A primer. Chelsea Green Publishing.
Osinga, F. P. B. (2007). Science, strategy and war: The strategic theory of John Boyd. Routledge.
Penrose, E. (1959). The theory of the growth of the firm. Oxford University Press.
Popper, K. R. (1972). Of clouds and clocks: An approach to the problem of rationality and the freedom of man. In Objective knowledge: An evolutionary approach (Ch. 6). Clarendon Press. (Original lecture delivered 1965)
Shihab, M. I. H., Hundhausen, C., Tariq, A., Haque, S., Qiao, Y., & Mulanda, B. W. (2025). The effects of GitHub Copilot on computing students’ programming effectiveness, efficiency, and processes in brownfield coding tasks. In Proceedings of the 2025 ACM Conference on International Computing Education Research, V.1 (pp. 407–420). https://arxiv.org/abs/2506.10051
Snowden, D. J., & Boone, M. E. (2007). A leader’s framework for decision making. Harvard Business Review, 85(11), 68–76.
Storey, M.-A. (2026). From technical debt to cognitive and intent debt: Rethinking software health in the age of AI. ACM Queue.
Williamson, O. E. (1985). The economic institutions of capitalism. Free Press.
Xia, X., Bao, L., Lo, D., Xing, Z., Hassan, A. E., & Li, S. (2017). Measuring program comprehension: A large-scale field study with professionals. IEEE Transactions on Software Engineering, 44(10), 951–976. https://ieeexplore.ieee.org/abstract/document/7997917