Drawing the transaction

Why is writing business software so hard? A companion essay on the part of the design we rarely make visible.

Between deployment and data ended with a more human-scale way of drawing the structure of a program. If packages are treated as real modules rather than collections of public classes, a system with fifteen hundred classes can become a diagram of a dozen meaningful parts and the collaborations between them.

That is useful, but for a business system it still leaves out something important. A module view can tell us what the software is made of and which parts work together. It does not tell us what changes together: which persistent facts are inspected, which are changed as one indivisible operation, and where one such change ends and the next begins.

I worked with a colleague who would repeatedly pull design discussions back to the entity model. He was much less interested in the structure of the application code above it than I was. He wanted to understand the data: what facts the system persisted, how those facts related to one another, and how they changed.

I think he was right about something important. Application code is surprisingly temporary. Classes are renamed, split and combined. Frameworks are replaced. A system can be substantially refactored while still representing essentially the same business facts. The persistent data and its relationships are often a more durable description of what the system knows.

It was with that same colleague that I was later discussing the design of a new system. We had the entity model in front of us and were trying to explain how one of the business processes would work. The entities and their relationships told us what state could exist, but they did not tell us how that state changed. I started numbering some of the relationships and annotating the parts of the model that each step acted upon. A query established something about the current state. A later statement updated one part of the model. Another inserted a related record.

One drawing was not enough. We ended up drawing several of these diagrams because the overall business process consisted of several separate transactions. Between them were the other things that happen in a real process: calls to external systems, messages, waits and decisions.

Looking back, I think that was the important shift. We had started with the persistent facts because that was the level my colleague instinctively wanted to discuss. Then, almost accidentally, we started drawing the changes to those facts. What emerged was not another view of the code. It was a way of showing where one indivisible change ended and how a sequence of those changes made up the larger business process.

The process is a chain

A business process is often larger than a single database transaction. Opening an account, settling a trade or onboarding a client may take minutes, hours or days. A person may approve something. A message may be sent to another system. Work may wait in a queue. Another user may complete the next step tomorrow. It would make no sense to hold one database transaction open across the whole thing.

The process is therefore often better understood as a chain of smaller physical transactions with interactions between them: a transaction, an external call, another transaction, perhaps a human decision, a message, then another transaction. Each physical transaction is one durable state change. Together they move the larger process forward.

That distinction matters because it gives us a better question than simply asking what services call what other services. We can ask what changes together at each step, where one atomic state transition ends, and what has to happen before the next one can begin.

The gaps between those transactions are often where the harder design questions live. A local transaction may commit and then require an interaction with an external system that cannot participate in the same atomic commit. Do we write an outgoing message in the transaction and deliver it through an outbox? What happens if the remote system succeeds but our call times out? How is an explicit rejection represented? Where does a retry begin, and how do we avoid repeating an external effect? If a person has to resolve the failure tomorrow, what durable state records that fact?

The point of drawing the process this way is not to prescribe an outbox, a saga or any other pattern in advance. It is to make the boundary visible so that these questions are difficult to miss. A conventional sequence diagram can make an external call look like one more arrow. A business-process view built from physical transactions and the gaps between them makes it clear where the atomic guarantee ends.

This is also where the deployment view becomes relevant. The external systems on the deployment diagram are the things that appear in those gaps. Deployment tells us which boundaries exist. The business-process view tells us which of them a particular process crosses and in what order. The transaction view tells us what state is at stake on either side of each crossing.

What changes together

Once the larger process is drawn as a chain, the next question is what belongs inside each transaction box.

This is where my original whiteboard drawing starts to make more sense. I was not trying to draw the database’s physical read order. I was trying to show the logical sequence of statements and decisions that made one state change atomic.

We talk a great deal about entities, objects, services, APIs, components and modules. We also use database transactions constantly. JDBC supports them. ORMs support them. Frameworks provide annotations for them. The database itself has spent decades developing machinery for making them atomic and controlling what happens when several execute at once.

Yet the transaction is rarely treated as a first-class design object.

Follow an important state-changing operation through a typical layered application and it may pass through a controller, a service, another service, several repositories and an ORM before eventually becoming a sequence of SQL statements inside a database transaction. The transaction undeniably exists at runtime, but there may be nowhere in the design where it is described as a whole.

I increasingly think of that sequence as a small program. It observes relational state, makes decisions and performs mutations. The database optimiser is free to choose how an individual SQL statement is executed: join order, access paths, indexes, parallelism and so on. That is not what the diagram needs to describe. At the design level the interesting sequence is the logical sequence of statements and decisions in the transaction, and the state those statements observe or change.

If a transaction first establishes that an order is in a state that permits approval, then updates the order and inserts an approval record, that sequence is meaningful even though the database may execute the internal reads of the first query however its optimiser chooses. We are not drawing physical table access. We are drawing the transaction program.

The mutations are particularly useful anchors because inserts, updates and deletes are explicit state changes. Reads matter where they establish a condition that drives what happens next, but the interesting question is not usually which table was physically read first. It is what fact had to be established before the next state change was permitted.

For an important transaction I want to be able to ask what state it observes, what it is permitted to change, what must already be true, what must be true when it finishes, who is authorised to cause it to happen, what happens when another transaction runs at the same time, and what should happen if it cannot complete. Those questions describe something much closer to the business operation than the route through controllers, services and repositories.

Drawing the transaction on the data

The entity model may already give us most of the vocabulary we need for that conversation.

Suppose an operation approves an order. The data model shows the order, the account, perhaps a reservation, an approval record and an audit history. Rather than drawing the controller, service and repository calls, we can draw or highlight the parts of the model each statement acts upon. One step may establish a condition from the current state. Another updates the order. Another inserts the approval. We can number those statements where their logical sequence matters.

The picture does not claim that the database physically reads tables in that order, and it does not replace the SQL. It is a shared representation of the transaction program at the level we want to discuss it.

Once it is on the whiteboard, the questions appear naturally. Why does this operation read the account? What prevents two approvals from succeeding concurrently? Is the history record part of the atomic change or a later consequence? Who is allowed to perform this transition? Can a failed attempt be retried safely? The diagram has not answered those questions. It has given the conversation somewhere to happen.

A small description alongside the drawing may be enough:

• what state the transaction inspects
• what state it is permitted to change
• who is allowed to cause it to run
• what must be true when it finishes
• what happens if another legitimate transaction runs at the same time

This representation has another useful property. We can draw one important transaction over six relevant tables, omit two hundred other transactions and two thousand other tables, and the picture can still be true. The entity model tells us what persistent state can exist. The transaction view tells us how that state is allowed to change.

Transactions across modules

There is another boundary that transactions expose.

Module boundaries and transaction boundaries are not the same thing. Often an important transaction will sit largely inside one module, and when that happens the design is comparatively easy to understand. But some business operations genuinely span several responsibilities, so some transactions will cross package or module boundaries even in a well-designed system.

Those cross-module transactions are often the most interesting ones to draw because they make coupling visible in a way the module diagram alone cannot. A structural picture may show orders, credit, inventory and payments as four clean modules. A transaction view may then show that one important operation needs all four.

That does not automatically mean the module boundaries are wrong. It gives us somewhere to ask whether the collaboration is appropriate, whether one module is reaching into another’s state, where coordination belongs, and which invariants are shared across the boundary.

If most important transactions repeatedly cut across the same set of modules, that is evidence worth paying attention to. Perhaps the package boundaries do not match the way the business changes state. But eliminating every cross-module transaction should not be the goal. Business responsibilities do interact, and some operations are inherently collaborative.

The module view and the transaction view therefore describe different decompositions of the same system. The module view asks which responsibilities and decisions belong together over the life of the software. The transaction view asks which observations and state changes must belong together at one moment.

A cross-module transaction can therefore mean several different things. The module boundary may be in the wrong place. The transaction may be doing too much and should become two transactions with an acknowledged gap between them. Or neither may be wrong: the business may genuinely define one invariant across two areas.

That tension is not something I think we should design away. It is something worth making visible.

Transactions may be the more interesting view

This is where I think the argument becomes more radical than package design.

A module diagram is useful because it gives us a human-scale view of the structure of the program. It lets us discuss ownership, dependencies and information hiding. But the transaction view gets closer to the thing the business actually cares about: which valid state transitions the software permits.

A system can have beautifully named packages and still have badly designed transactions. It can have low structural coupling and still permit states the business regards as impossible. It can have elegant interfaces while concurrency, authority and failure are left implicit.

The reverse is also interesting. A cross-module transaction can look untidy from a structural point of view and still be exactly right because the business invariant genuinely spans those modules.

That is why I increasingly think transaction design deserves at least as much attention as package design, and probably more in systems whose main purpose is to maintain important business state.

We have spent decades making the structure of object-oriented programs more elaborate while treating the transaction boundary as something a framework annotation can supply for us. The mechanism survived. The design object largely disappeared.

Artefacts for conversation

The answer is not to create another mandatory documentation method. The value of the transaction drawing is the same as the value of the whiteboard diagrams I used when onboarding people: it gives a conversation somewhere to happen.

Put the larger business process between two engineers and somebody can point at the gap between two transactions and ask what happens if the external result is ambiguous. Put an entity model with one of those transactions drawn over it beside the process and somebody else can point at a state change and ask why it belongs inside the atomic boundary. Another can point at a read and ask what happens if the fact changes concurrently.

The picture matters because those questions become visible before we have to reconstruct them from production behaviour.

The code remains authoritative, but it is a poor starting point for this discussion because the transaction may be scattered across controllers, services, repositories, ORM mappings and framework configuration. A useful artefact leaves all of that detail out and exposes the business state transition instead.

A practical test

Take one important business process in a system you know well. Do not start with the source code.

First draw the process as a chain of physical transactions and the interactions between them. Where does an external system become involved? Where does the atomic guarantee end? What happens if the outcome of an external call is unknown?

Then take one of those physical transactions and draw it over the relevant part of the entity model. Show its significant statements or decisions: what fact it establishes, what state it changes, what must already be true and what must remain true afterwards. Ask who may perform it and what happens when another legitimate transaction runs at the same time.

Finally, put the transaction view beside the module view. Where does the transaction cross a module boundary? Is that because the module design is wrong, because the transaction is too large, or because the business genuinely defines one fact across those areas?

Those are much more interesting questions than which service calls which repository. And in many business systems, they may be closer to the real design than the class structure ever was.