Why Software Can't Tell You It's Wrong

Software architecture debates have a problem that most other engineering disciplines don't: the alternative was never built.

When a bridge fails, the failure is physical, attributable, and measurable against every other bridge that didn't. The engineering decisions that caused it can be isolated, traced, and corrected — not just in theory, but in the next bridge, because the material itself produces feedback that no amount of professional opinion can override. Steel deflects. Concrete cracks. Physics doesn't care what the architect believed.

Software produces no equivalent feedback. A system built around the wrong abstractions compiles, runs, ships, and passes its tests just as readily as one built around the right ones. A bug introduced by a misaligned domain model looks identical, from the outside, to a bug introduced by a typo. A feature that took three times longer than it should have, because the structure made it harder than the business logic warranted, produces no artifact that distinguishes it from a feature that was simply difficult. The cost is real. The cause is invisible.

This is the unfalsifiability problem, and it runs deeper than "we can't measure everything." It means that when a system becomes expensive to change, the diagnosis almost always lands on the wrong variable. The domain is complex. The requirements changed. The previous team was careless. Almost never: the structure was wrong, and the structure was wrong because nobody ever built the other version of it to compare against. That version doesn't exist, it never will, and every architectural argument in the industry is conducted in its absence.

This would be a purely philosophical problem if there were nothing to do about it. There is something to do about it — but it requires accepting that the standard metric for software quality, whether it works, is measuring the wrong thing entirely.

The Metric That Hides the Problem

The natural substitute for "is this good engineering" is "does it work," because working is the only property the system demonstrates directly and continuously. A feature either satisfies the requirement or it doesn't. Tests are either green or they aren't. The sprint either shipped or it didn't. These are checkable facts, and in an environment with no physical feedback to measure against, checkable facts become the entire basis for evaluation.

The problem is that working is compatible with almost any underlying structure, good or bad, for a surprisingly long time. A system can satisfy every functional request in its backlog while its essential complexity — the actual rules and relationships of the business — is scattered across service methods, duplicated across layers, encoded differently in three places because three developers made the same inference independently without any structural pressure to put it in one place. None of that appears as a failure. It appears as a working system, until the day a new requirement needs to reach across all three of those places simultaneously, and what should be a small change becomes an investigation into how many versions of the rule actually exist and which ones are authoritative.

By then the cause is undiagnosable. The system is large. The rule has accreted across two years of sprint reviews. The developers who wrote the original three versions are gone. The structure that caused the problem will be attributed to enterprise complexity, changing requirements, accumulated technical debt — anything except the decision, made early and silently, to let working stand in for well-structured.

When behavior is the only metric, behavior is what gets optimized. Frameworks that produce correct behavior fast become the natural answer. Templates that generate working code without requiring structural reasoning become best practice. The question that would actually determine the ten-year cost — what are the mechanics that produce this behavior, and do those mechanics live in a place that makes them findable, correctable, and legible — never gets asked, because asking it requires a comparison that isn't available.

The Mechanics, Not the Behavior

The substitute for physical falsifiability isn't a better way to measure behavior. It's a deliberate shift in what gets examined — from the surface of the system, what it does, to the structure underneath it, what it is.

Every piece of business behavior is produced by mechanics: relationships between concepts, rules that govern those relationships, responsibilities that belong to one concept and not another. A checkout process is behavior. The mechanics underneath it are what actually need to be represented correctly.

Consider a library lending system. The behavior is simple: a borrower takes an item, returns it later. The mechanic, examined carefully, is that a loan is a distinct event — something that happened between a person and an item, at a point in time, with its own start, end, and history. It is not a property of the item itself. A book does not become a different kind of thing when it is borrowed. The loan is a separate concept with a separate lifecycle. That distinction is not an implementation preference. It is what is actually true about the business — and a system that puts loan dates on the Book entity has encoded a false mechanic, one that will prevent the system from answering basic questions about lending history without structural rework.

The behavior can be implemented many ways. The mechanic is either represented correctly or it isn't, and unlike behavior, a misrepresented mechanic produces a specific, locatable kind of wrongness: the rule exists in the wrong place, the concept carries responsibility it was never supposed to own, the assumption is encoded somewhere it can't be found when it turns out to be wrong.

This is not a theoretical exercise. It is a structuring and learning activity. The goal is not to produce a perfect model before a line of code is written — it is to make the structure of the business explicit enough that building it becomes a matter of finding the right place for each piece, rather than making an independent decision every time about where things should go. A user story is not a specification to implement. It is information about what the business is — a piece of evidence that either fits the current understanding of the mechanics or reveals that the understanding was incomplete. Both outcomes are useful. The first confirms the model. The second improves it.

Once the mechanics are explicit, simplicity becomes achievable in a way it never is when the focus stays on behavior alone. The simplest thing that works is not the simplest thing to type — it is the implementation that puts each responsibility exactly where it belongs, with no duplication, no misdirection, no logic living in a place that requires explanation rather than being found by intuition. That kind of simplicity is legible to a new developer on their first day. It is transferable to a team member who didn't write it. It is correctable when an assumption turns out to be wrong, because the assumption lives in one place that can be questioned directly rather than in a diffuse implication scattered across the system.

Essential and Accidental Complexity

Fred Brooks named this distinction decades ago, and it has lost none of its precision: essential complexity is the difficulty that comes from what the problem actually is; accidental complexity is the difficulty introduced by the tools and representations chosen to solve it.

Essential complexity cannot be removed without changing the problem. The fact that a loan involves a lender, an item, and a time period — and that these are three separate things with separate lifecycles — is not a decision anyone made about how to model a library system. It is what a loan is. Strip it out and the system no longer represents what the business does. Every piece of structure that reflects this accurately is essential complexity made explicit.

Accidental complexity is everything else: the framework, the deployment topology, the persistence layer, the test harness, the CI pipeline, the service boundaries drawn for organizational rather than domain reasons. None of this exists in the business. All of it exists to make the essential complexity operational. It is supposed to serve the essential complexity, not the other way around. The moment a framework's conventions, a deployment constraint, or an organizational boundary starts dictating what the domain is allowed to look like, accidental complexity has taken the lead, and the essential complexity is being shaped to fit it rather than the reverse.

This inversion is the likely trajectory of most software projects within a few years of starting — unfalsifiability means it can be neither proven nor disproven with certainty, but a perfect structural alignment between code and business mechanics from day one, sustained without a mechanism to detect and correct drift, requires a precision that makes winning the lottery look achievable. It happens quietly because it produces no visible symptom. The system still works. The tests still pass. The sprints still ship. The essential complexity didn't disappear — it was dissolved into the accidental complexity surrounding it, encoded in service method names and transaction annotations, still doing its job but no longer findable as a thing you can look at, question, or change in one place.

The Cogwheels of the Business

A mechanical watch is a useful image here. The cogwheels of a watch are its essential complexity made physical — the gear ratios, the relationships between wheels, the escapement that governs the release of energy. These cannot be simplified away without the watch losing the ability to tell time. They are what the watch fundamentally is. The casing, the hands, the crystal, the materials — these are accidental complexity: real, necessary, but entirely in service of the mechanism underneath. A watchmaker looking at a malfunctioning watch does not need to guess where the problem is. The mechanism is visible. Each part has a known responsibility, and a part that has the wrong shape for its position will produce a specific, locatable failure rather than a general degradation of the whole. The casing does not malfunction. The mechanism does — and it can be found.

A rich domain model is the cogwheels of the business — and the automation of it. Not data with operations attached, which is what most systems produce, but responsibilities that carry their own data and enforce their own rules. An account is not a row in a table with balance and status fields. An account is something a customer has, that carries a balance, that knows what it means to deposit and withdraw, that enforces the rules governing when a withdrawal is and isn't permitted. Those rules belong on the account — not because of aesthetic preference, but because that is where they would be found by anyone who thought to look for them by asking what an account is. When those rules live on the account, a wrong rule is a local problem, visible and correctable in one place. When those rules live in a WithdrawalService and a BalanceCheckService and a TransactionValidationService distributed across a microservice boundary, a wrong rule is a distributed problem, findable only by someone who already knows which services encode the assumption and can read them all simultaneously.

The practical difference is not academic. On a system where the essential complexity is explicit, a new developer can read the model, understand what the business is, and contribute meaningfully in days — because the structure of the code reflects the structure of the business, and understanding one is understanding the other. On a system where the essential complexity has been dissolved into accidental layers, a new developer learns how to navigate the framework, traces several services to reconstruct a rule nobody wrote down, makes a change, breaks something else that encoded the same rule differently, and starts the cycle again. The business is still there — it is just no longer visible as the thing organizing the code.

Correctability as the Partial Answer

This is where the unfalsifiability problem becomes something other than a purely philosophical concern.

Unfalsifiability means the alternative was never built, so no direct comparison is available. It does not mean nothing can be done. The absence of a physical feedback mechanism can be partially compensated for by a structural property: if wrong assumptions live in one legible place, they can be corrected when they turn out to be wrong. Correctability does not prove the original model was wrong — it cannot, because unfalsifiability is still in effect. But it prevents the cost of being wrong from compounding indefinitely, which is the actual damage the problem causes. A wrong assumption that gets corrected in year two costs a week. The same wrong assumption that is only discoverable by reading fourteen service methods, in year five, after three team changes, costs months — and the correction itself has to find and fix every place the assumption was encoded, which are no longer guaranteed to be consistent with each other.

When essential complexity is kept explicit, a misaligned model is a correctable one. When it has been dissolved into accidental layers, a misaligned model is a permanent one — not because it cannot technically be corrected, but because the correction cost grows faster than anyone is willing to pay it, and the system calcifies around the wrong assumptions instead. Every new feature gets built on top of them. Every new developer inherits them. The system is not failing. It is just becoming progressively more expensive to change, in a way nobody can trace back to a cause because the cause is structural and the structure is no longer visible.

Focusing only on whether the system works skips this entirely. Frameworks that deliver working behavior without requiring the essential complexity to be made explicit appear to win the short-term comparison — they are easier to hire for, and they produce green tests by the end of the sprint. But the appearance is produced by measuring the wrong thing.

A sprint review that demonstrates two completed screens and a passing test suite looks productive. The question it doesn't answer is what those two screens cost in accidental complexity — in configuration, wiring, DTO mapping, coverage mandates, and refinement sessions spent on how to make the technology work rather than on what the business actually needs. A team working from an explicit model of the essential complexity spends that time differently: on understanding what the business is, on placing each new piece of behavior where it belongs, on confirming or correcting the model. The visible output per sprint may look similar. The structural cost is not.

This is worth qualifying honestly. As long as a codebase is small enough to fit inside the working context of the team maintaining it, the benefit of explicit essential complexity is real but modest — the system is comprehensible regardless, because it is simply small enough to hold in one mind. The benefit scales with the size and longevity of the system. As the application grows, without explicit essential complexity, the effort required just to maintain an overview grows faster than the team can absorb it. The common response is distribution — splitting the system into smaller, independently manageable pieces. But distribution assumes precise knowledge of where the business can be cleanly separated, and that knowledge is exactly what the unfalsifiability problem prevents anyone from having with confidence. The separation points get drawn on organizational lines, on deployment convenience, on what seemed coherent at the time — and every cross-cutting rule the business later needs, every requirement that turns out to span the boundary that was drawn too early, pays the price of an integration problem that a single coherent model would never have generated.

The cost of dissolving the essential complexity into accidental layers does not begin at year three. It begins at the first sprint, accumulates without attribution, and will eventually be diagnosed as enterprise complexity or changing requirements or the inevitable entropy of large systems — not as the consequence of a structural choice made at the beginning, when the essential complexity was allowed to dissolve rather than being required to remain visible.

The cogwheels of the business are either explicit or they are not. If they are, the system can tell you when it is wrong. If they are not, it cannot — and the bill for that silence will arrive, as it always does, long after the people who made the choice have moved on.

So What

There are two ways to build enterprise software, and the choice between them is made — usually implicitly, usually early — by whoever decides what success looks like.

The first focuses on what the system does. Tools, frameworks, and delivery velocity are the primary concerns. User stories are work orders. The sprint is the unit of value. A working feature at the end of the iteration is the definition of done. This approach has a ceiling that is invisible until it has already been reached: the essential complexity of the business accumulates inside accidental layers, the system ages with its technology choices, new insights are hard to incorporate because nobody can find where the old ones live, and distribution gets introduced to manage the sprawl — which requires precise knowledge of where the business can be cleanly separated, knowledge that by definition is not yet available. The result is software that works for years while becoming progressively more expensive to change, until the cost of a new feature is indistinguishable from the cost of a small project, and the organisation starts talking about a rewrite.

The second focuses on what the system is. The essential complexity of the business — the actual rules, relationships, and responsibilities — is made explicit, kept in one place, and treated as the primary asset. Technology serves it. Frameworks are evaluated against it. New requirements are treated as information about it. When an assumption turns out to be wrong, it can be found and corrected, because it was never allowed to dissolve into the layers surrounding it. This approach does not age with its technology choices, because the technology was never load-bearing. It does not require distribution to stay manageable, because the model provides the coherence that distribution was trying to restore. New developers find the system comprehensible because the structure of the code reflects the structure of the business, and understanding one is understanding the other.

The first approach feels more immediately productive because it generates visible artefacts — configured services, passing tests, completed screens — while deferring the understanding that would have made those artefacts cheaper to produce. The second does the understanding first, which means the first sprint looks quieter and delivers more.

The cost difference between the two, accumulated over a decade of a business-critical application, is not marginal. It is the kind of number that makes a CFO ask why nobody mentioned this was a choice. The real jewels of enterprise software are not in the framework versions, the language features, or the architectural patterns that fill conference schedules every year. They are in language basics that have not changed in thirty years — an object that knows what it is, owns what belongs to it, and enforces its own rules. Everything else is the casing. The mechanism is what keeps the time.

This article is part of a series on software engineering craft. Other pieces examine the rich domain model as a discipline, the properties of enterprise software that lasts, why frameworks answer structural questions before the domain gets a chance to, and why the practices that prevent structural decay are the same ones that always prevented it.