Business Software Is Hard

Essay 8

One role per user

On the difference between a role and a job, and what it costs to confuse them.

One responsibility each

Imagine a business system with a simple rule: each user may hold exactly one role. Not one role at a time, or one primary role. Exactly one. The database enforces it.

The system also insists that each responsibility has its own role, with no way to combine them. Order placement is one. Order cancellation is another. Administration is a third. So the person who places an order cannot cancel it. If they need an order cancelled, somebody else has to do it.

There are good reasons to prevent particular combinations of responsibilities. Anyone who has worked in finance will recognise maker and checker, four eyes, or segregation of duties. Those are deliberate controls. We identify a dangerous combination and decide that one person should not be allowed to do both. That is different from making separation the default.

Give our one-role system a couple of weeks and the effects would be obvious. Someone spots a mistake in an order and has to wait for the person with the cancellation role. A team invents a form to pass work between two desks. Support gets tickets from people who need somebody else to press a button for them.

Nobody is breaking the rules. The delay is produced by everyone obeying them. Eventually people will find workarounds, but even if nobody cheats, we have already made the work slower. Something one person could have done now needs a handover, some communication and usually some waiting.

We would recognise the modelling error. One person can hold several roles. Several people can hold the same role. Sometimes two roles must be kept apart, but that is a rule about those roles, not a reason to pretend that every responsibility needs a different human being. Software organisations seem to find this much harder.

The setting here is internal business software in medium and large organisations, built and maintained by one or more teams of software engineers. These organisations have good reasons to employ specialists. The question is what happens when their job titles become boundaries around the work.

Roles are not jobs

We have analysts, architects, developers, testers, UX designers and DevOps engineers, along with plenty of other job titles. The responsibilities behind those titles are all real. The mistake begins when the job becomes the exclusive owner of the responsibility.

Analysis belongs to the analyst. Architecture belongs to the architect. Testing belongs to QA. Interface design belongs to UX. Development gradually becomes the bit left over after the other responsibilities have been handed out. It looks reasonable on an organisation chart, but it assumes that those responsibilities can be separated cleanly while the software is being built.

Peter Naur’s Programming as Theory Building gives a useful way to think about why that assumption causes trouble. Naur used programming to mean the whole activity of design and implementation. Programmers build a theory of the system: an understanding of how it represents the work it supports, why it has the shape it has, and how its parts fit together. The source code and documentation express some of that understanding, but they do not replace it.

That understanding continues to develop during implementation. You start what sounded like a simple update and discover that two changes have to succeed or fail together. Two business concepts that looked identical in the requirements turn out to behave differently. A screen that looked like a collection of fields turns out to contain three separate business transactions. These discoveries change the analysis and the design. The work moves backwards and forwards between understanding the problem and constructing the solution.

Separating the jobs does not remove that connection. Suppose the analyst understands the business and the developer implements what the analyst has understood. The developer discovers something the analysis did not settle. The question goes back, somebody explains the issue, a decision is made, and the result returns to development. Some questions need another person’s knowledge or authority. Others could have been resolved by the developer if they had enough understanding of the business. Exclusive ownership sends both through the same handover.

We often make this worse while believing we are helping developer productivity. Keep developers out of the business meetings. Let the analyst gather the requirements. Let the architect decide the design. Give development a clean ticket once all that thinking is finished. That sounds efficient if programming is mainly the conversion of a complete description into code. It is much less convincing if programming is one of the places where the description becomes complete enough to be useful.

This does not mean every developer should attend every meeting. Plenty of meetings are coordination, and there is no virtue in filling them with more people. But conversations in which the business problem is still being worked out are different. Saving the developer from those discussions may mean that they later receive a description of an understanding they were never part of building. The saved time then comes back as questions, explanations and rework.

The same applies when specialists join the work later. Code, tickets and documents help, but they cannot transfer the understanding intact. The new person has to build enough of it for themselves, which means questions, explanations and sometimes reopening decisions that appeared to be finished. If you already know that somebody’s expertise will matter, it is often cheaper to involve them while the relevant understanding is being formed.

The production-line model

There is an older idea behind much of this. Frederick Taylor deliberately separated planning from execution. Management would determine how work should be done; workers would perform it. His approach drew on studying repeated tasks whose methods could be measured and standardised. Software development contains routine work too, but the unresolved questions in a change prevent us from treating the whole thing as a predetermined sequence of operations.

Yet that is often how we organise it. Analysis produces requirements. Architecture produces a design. Development turns the design into code. QA verifies the result. Each stage takes the output of the previous stage, performs its specialist transformation, then hands the work onwards.

There is an attractive reason for dividing the work this way. It allows the stages to be pipelined. While developers work on one change, analysts can prepare the next and testers can test the previous one. Several pieces of work can move at once. If the transformations are well understood and the hand-offs are reliable, that can increase throughput considerably.

The trouble is that software keeps sending work backwards. Programming changes the understanding of the requirement. Testing exposes mistakes in the model. Drawing the first version of a screen changes what we think the transaction is. Production constraints alter what looked like a sensible design. Later work keeps discovering things that earlier work was supposed to have finished.

The apparent parallelism is then paid for in queues, hand-offs, explanation and rework. Worse, when work goes backwards, the earlier stage is usually busy with something else. The analyst has moved on to another change, the architect is reviewing something different, or the developer has started another task while waiting for an answer. A question that takes two minutes to resolve can therefore hold up the work for several days. We divided the work partly to make it faster, and can end up taking longer than if the people involved had worked through more of the problem together.

Analysis, architecture, development and QA drawn left to right with a hand-off between each. A bar beneath the row is solid under each stage and absent in the gaps, marking where the change is actually being worked on. The gaps carry what happens instead: waiting for an answer, writing enough down for the next person, reconstructing context somebody already had. An arrow curves back from development to analysis — the question the analysis did not settle — annotated: two minutes to answer, three days of elapsed time.
Every stage is competent and every hand-off is reasonable. The bar marks the only part of the elapsed time in which the change is being worked on. When the question goes back, the stage that has to answer it has already started something else.

Nobody involved needs to be inefficient. Every person can be busy and competent. Every department can meet its targets. The elapsed time accumulates between them: waiting for an answer, arranging a conversation, writing enough down for the next person, reconstructing context that somebody else already had.

That is what slow software delivery often looks like. It does not necessarily look like people working slowly. It looks like a lot of people doing reasonable things while one piece of work spends several weeks travelling between them. The paper form has moved from the application into the organisation.

Buying the capability

These arrangements usually start for sensible reasons. Testing is weak, so hire testers. Screens are poor, so hire UX designers. The system has become incoherent, so appoint architects. Organisations know how to buy jobs. Growing the same capability inside a team is slower and much harder to see. The organisation gets the specialist capability it bought, but can quietly lose the same capability everywhere else.

Consider the first version of a screen. Put a rough form in front of somebody who performs the job and questions appear immediately. Are these two things really changed together? Who is allowed to approve it? Is this one action or two? This is a UX exercise, business analysis and software design at the same time. A UX specialist brings expertise in understanding the user’s work and designing an interaction that supports it. The developer needs to participate because what they learn changes the software. If the existence of the specialist means the developer stops thinking about the interface, they lose part of the process by which the system is understood.

Testing is similar. A good tester will find failures that developers miss. I want that person on the team. What I do not want is the conclusion that can follow from having a QA department: correctness is their responsibility. Developers stop thinking quite so hard about failure because somebody downstream will test it. Testers receive software after many design choices have already been made. Bugs then travel back to development, carrying information that would have been cheaper to use earlier.

DevOps makes the same point unusually clearly. It was supposed to reduce the boundary between development and operations. Turn it into a separate job that owns deployment and operation, and the old boundary can quietly reappear. Production becomes somebody else’s responsibility again.

The analyst programmer

There used to be an ordinary job title that now sounds slightly quaint: analyst programmer. I think there was something important in it.

A common objection to combining these responsibilities is that many programmers are not particularly good analysts. This is plainly true. Plenty are not. But if a developer spends ten years being handed analysis written by somebody else, we should not be surprised that they fail to become good at analysis.

Over time the role and the job become the same thing in everyone’s head. The specialist gets better because they practise the responsibility constantly. Everyone else has fewer opportunities to develop that ability or keep it sharp. Then we look at the difference in ability and conclude that the separation was necessary.

Some programming that appears technically weak is really programming done without enough understanding of what the code is meant to represent. A developer can explain exactly what their code does but cannot explain what it is for. They know that two tables are updated but not why those updates belong together. They know there is an approved flag but not what approval means to the business. They can make the tests pass without having any independent way to tell whether the result makes sense.

Giving that developer more business understanding has not loaded them with an unrelated second job. It has given them information needed to do the first one.

This matters especially in internal business software because so much useful knowledge is peculiar to the organisation. Experience in bonds, insurance or logistics does not tell somebody why your company does one reconciliation at four o’clock, why one status means something different in an old product, or why a process contains an exception introduced after an incident twelve years ago. That knowledge grows inside the organisation. If developers are permanently kept at a distance from it, software development remains dependent on people who translate the business for them.

Roles inside the team

The value of a specialist is that the team becomes better at carrying the responsibility. Different people will be better at different things, and difficult work may need specialist help. What matters is that those strengths do not become boundaries around who is allowed to think about what.

The team between them needs enough understanding to build, test and run the software it is responsible for. Different people can take the lead at different times. Someone with stronger testing instincts may lead the testing of a difficult change; somebody with deeper business knowledge may lead the analysis. A specialist can support several teams without becoming a separate stage through which every change must pass.

Participation does not give everyone the same decision authority. A developer should understand an approval rule and question an inconsistency; the business may still own the policy decision. A team can discuss a design while having a clear person responsible for settling it. Independent assurance can also remain independent. It does not relieve the team of responsibility for producing correct software.

Suppose a developer, a tester and the person responsible for an approval process examine a proposed change together. They use a rough screen to establish what is being approved, which updates must succeed together and what should happen if approval fails. The developer can then implement it without asking somebody else to translate each rule. When a new policy question appears, it still needs a business decision, but the people involved already understand the change. They have agreed how to resolve questions while it is being built. They do not need to sit in a meeting all day.

Some handovers disappear because the developer can answer the question. Others become easier because the people needed have retained enough context to answer it. Where a change spans teams, the same principle applies: involve the relevant people in understanding the change and keep them available as the work develops. Nobody needs to understand every system in the organisation.

Shift-left testing is useful, but moving QA earlier is only part of the change. Responsibility for testing needs to remain with the team throughout development.

You can begin without a reorganisation. Put developers in front of users while the problem is still being understood. Let architects spend time in the code. Bring testers into the design of risky behaviour. Separate queues, priorities and targets may eventually need management attention, but these are practical places to start.

Job titles can stay exactly where they are. The question is whether the responsibilities are allowed to overlap. We already know the answer when we design the permissions system: one user can hold more than one role. A software team should work the same way.

A practical test

Take a change your team finished recently and trace its journey rather than the effort it took. Where did it stop moving, who was it waiting for, and what question was each wait resolving?

Sort those questions into two piles. Which of them genuinely needed another person’s knowledge or authority, and which could have been settled by whoever was holding the work if they had understood more of the business? The second pile is what exclusive ownership costs you.

Then pick one responsibility the organisation bought as a job: testing, analysis, interface design or deployment. Ask what everybody else stopped doing when it arrived, and whether anyone outside that job still practises it often enough to remain good at it.

Finally, try the smallest version of the alternative on the next piece of work. Put the developer, the tester and the person who owns the rule in front of the same rough screen before anything is built. Note which questions are answered in that conversation, and how many of them would otherwise have become a handover.

References