The practical tests

Each essay ends with something to try against a system you already have. Collected here, they make a rough diagnostic: twelve questions about where your system keeps its meaning, whether the units you have chosen are the units that carry it, and how the work around it is organised.

They are not a substitute for the essays — each one assumes the argument that produced it — but you can run them first and read afterwards.Start with the essays instead.

Essay 1
Fifteen hundred classes and no visible shape
Open your own codebase and try to say out loud what it is made of, in under ten items. Not the technologies and not the layers: the main parts, and what decision or business responsibility each one hides. Then look at one of those parts. How many types does the rest of the program need to see? How many interfaces exist only because two implementation classes were not allowed to call each other directly? How many tests know about classes that ought to be private implementation details? How much wiring disappears if the package is allowed to be a module rather than a folder?

If you cannot describe the shape of the system, that is the finding.
Essay 2
Between deployment and data
Take a system you know well and imagine onboarding somebody to it at a whiteboard. Start with an empty board.

Draw the deployment view and the relevant part of the data model. Then try to name the main parts of the program without using framework layers. If the only honest answer is hundreds or thousands of classes, move up a level and ask whether there are a handful of meaningful modules instead.

Draw those modules and the important relationships between them. Can another engineer point at the picture and ask why one module depends on another? Can you change the internals of a package without invalidating the diagram? Can you explain the overall shape of the program without opening the source code?

If you can, one part of the missing middle may not have needed a new notation at all. It may have needed us to start drawing the program at the right level.
Essay 3
Drawing the transaction
Take one important business process in a system you know well. Start by drawing its database transactions and the interactions between them. Where does an external system become involved? Where does the atomic guarantee end? What happens if the outcome of an external call is unknown?

Choose one transaction and draw its significant statements and decisions over the relevant part of the entity model. Show what it observes, what it changes, and what must remain true afterwards. Record the authority and concurrency assumptions on which it depends.

Put another transaction beside it, choosing one that may inspect or change the same facts. Try to explain what happens when their execution overlaps. Then compare their boundaries with the module view. Look for necessary collaborations and for patterns suggesting that related responsibilities have been scattered.

Finally, compare the drawings with the implementation: the actual statements, commit boundaries, constraints and failure handling. Starting away from the code helps us describe the intended operation; returning to it tells us whether that is what we have built.

The useful result is something two engineers can put between them and reason about. We can point at a read, a change or a commit boundary and ask a precise question. The transaction was always there. Now we have a way to examine it.
Essay 4
Yeah, but does it work?
Take a database-backed application whose test suite is considered good and set the coverage percentage aside.

Choose one meaningful module. Can its behaviour be tested through its intended public surface without depending on its internal class structure?

Choose a transaction and a read or calculation. Starting from small, known business facts, can you exercise the real database behaviour and explain the expected results? For a known concurrency risk, can you reproduce the competing operations and explain the protection the design relies on?

Then choose a larger process. Is there evidence for its important success and failure paths, including what happens across external boundaries?

Explicit transactions and processes would give our tools better things to report: which outcomes, process routes and known interactions have tests, and which do not. A report exposing those omissions would be more useful than a percentage that treats one successful invocation as sufficient evidence.

That brings me back to the application with effectively one hundred percent coverage. Coverage showed us which code the suite exercised. It could not tell us whether the expected answers were sound, the important transactions were examined, or the business process worked. We had made the tests fast and the number impressive while leaving those questions open.

The product owner's question deserved an answer we could demonstrate: does the software perform the work it exists to do?
Essay 5
Why does it only fail in production?
Take an intermittent production problem involving persistent data, or choose an important state-changing operation in a system you know well. Write down the physical transaction as one logical program. What facts does it observe? What decisions depend on them? What does it change? Where does the atomic boundary begin and end? If answering that requires following a large call graph and reconstructing propagation rules, note that as part of the problem.

Then identify the other named transactions that can change facts this transaction relies upon. Pick the most interesting pair and ask what happens if they begin from the same valid state and overlap. Which observations can become stale? Which constraint or isolation guarantee prevents an invalid result? Could the invariant be expressed as a constraint instead? If one transaction must abort, does the retry re-read everything the decision depends on?

Finally, look at the wider properties. Who is authorised to cause the transaction, and is that decision made inside the boundary? Which failures are normal business outcomes? What must be audited with the change? Can you identify the same transaction by name in a production trace?

A bug that appears one time in a thousand is not evidence that the database is unreliable. More often it is evidence that our sequential mental model collided with the concurrent system we had actually built.

Transactions are hard. That is not an argument for making them disappear from the design. It is an argument for making them easier to see.
Essay 6
Why does modularity stop at the database?
Take a database you know well and ignore its current schemas for a moment. Identify a few business areas whose rules and persistent representation tend to change together.

Pick one. Which tables belong to its private physical representation? Which relations, keys or views are deliberately public? Which changes should other parts of the system cause through named transactions rather than direct updates? Which supposedly shared tables are genuinely tiny shared vocabulary, and which are business modules hiding under the label reference?

Then inspect the transactions. Count how many schemas each named transaction reads and writes. Look at the distribution and investigate the outliers. If almost every transaction crosses every proposed boundary, the decomposition is telling you very little.

Finally ask what could be checked automatically. Can tooling report cross-schema reads and writes? Can it flag access to private tables from the wrong application package? Can schema ownership be reflected in migrations and transaction tests? The database may not give us the same compiler boundary as Java, but we do not have to accept having no feedback at all.

The goal is not more schemas.

The goal is to be able to change one business area without first reverse-engineering the whole database.
Essay 7
Why can't I build a simple business screen on my own any more?
Pick a straightforward screen in a system you know: twenty displayed fields, five editable, one grid.

Which of its interactions are unique to this business process, and which are repeated behaviours such as money entry, dates, lookups, dirty-state handling or validation? Then trace what it takes to change one field today. How many files, models, mappings, client states and tests move with it, and which of those exist because of the business rule rather than because the interface is assembled from primitives below the level at which you actually work?

Finally, imagine the markup you would like a developer to write, and give the repeated interactions names. Then make a static version with representative data and put it in front of a user. How much of the interface could you change with them in the room before retreating into framework code, CSS or a design handoff?

The aim is not to make every interface simple. It is to make simple business screens simple enough that developers and users can shape them together.
Essay 8
One role per user
Take a change your team finished recently and trace its journey rather than the effort it took. Where did it stop moving, who was it waiting for, and what question was each wait resolving?

Sort those questions into two piles. Which of them genuinely needed another person's knowledge or authority, and which could have been settled by whoever was holding the work if they had understood more of the business? The second pile is what exclusive ownership costs you.

Then pick one responsibility the organisation bought as a job: testing, analysis, interface design or deployment. Ask what everybody else stopped doing when it arrived, and whether anyone outside that job still practises it often enough to remain good at it.

Finally, try the smallest version of the alternative on the next piece of work. Put the developer, the tester and the person who owns the rule in front of the same rough screen before anything is built. Note which questions are answered in that conversation, and how many of them would otherwise have become a handover.
Essay 9
When Agile becomes process
Take your team's current sprint and try to state its Sprint Goal without looking at the board. If the goal turns out to be the list of tickets, ask what the team would be free to change on Wednesday if it learnt something inconvenient on Tuesday.

Then go through the ceremonies one at a time and ask who outside the team depends on what each one produces. The ones with an external consumer will survive whether or not they help the team. Which of the others would anybody notice if they stopped?

Look back at the last three retrospectives and find something that actually changed as a result. If that change is hard to name, the meeting has outlived its reason while keeping its slot.

Then look at the plan the team sits inside. What is treated as fixed: the scope, the date, or both? When the work produces new knowledge, where does the uncertainty go, and does a discovery arrive as information or as a delay?

Finally, take a recent refactoring in which the business behaviour did not change. Did the tests have to change anyway? A team cannot adapt its implementation freely while its tests are pinned to the implementation it already has.
Essay 10
The network we chose
Pick one business transaction in a system you know, such as placing an order or posting a payment, and trace it from the moment the request arrives to the moment its commit is durable. Count the network hops: load balancers, service calls, database round trips, the SAN. For each one, ask whether somebody decided it was needed, or whether it followed from where the VMs happened to be.

Then measure rather than estimate. Run the transaction's queries from the application host, and again on the database host over a Unix socket. How much of the elapsed time is the database doing work, and how much is waiting on the wire? For how long does the transaction hold its locks?

Next, list the machinery that exists only because of those hops: retries, idempotency keys, sagas, outbox tables, caches, internal certificates. Which of them would still have a job if the components that must agree shared a machine and a database?

Finally, add up the peak load of your systems of record and compare it with one of today's servers. If it would fit several times over, ask what the rest of the estate is buying.
Essay 11
Where did this value come from?
Pick a value in a result produced last month. Find the input version, source record and batch, and explain the checks and decisions that made it eligible for use. Then choose an export and show exactly what left the system and the evidence of its receipt. If processing stopped just after a commit, could a retry safely determine what remained?

If answering those questions requires finding the developer who remembers how the job worked, some of the business system's essential history is still being stored in the wrong place.
Essay 12
The extremely smart intern
Take a change an agent has made that is waiting to be merged. Can the programmer who will merge it explain what changed, why it changed that way, what important alternatives were rejected, and what evidence shows that the result is correct?

If not, the agent may have finished coding, but the programmer has not finished the job.