Succeeding with DDD - Documentation
I’m often asked about what teams doing Domain-Driven Design (DDD) should do in the way of documentation. The question What types of Written Design Documents are used in DDD projects? came up on Stack Overflow and I started to write a response, but realized it was getting way too long to post there. So here it is.
We need to understand why we are writing it in the first place: What purpose is each document intending to serve?
This is why agile teams adopt the value of "working software over comprehensive documentation."
However, documentation serves a number of important, and different, purposes. For each documentation artifact, ask: "Is this artifact to support the team now as it develops the software, or is it to support future development?" Depending on the answer to this question, approach the documentation in a different way. Let’s start with supporting future development.
The Problem of Tribal Mythology
Jason Smith in Elemental Design Patterns says the following about kinds of documentation supporting future development:
We know we should document our software; we know we should keep it up to date; we know we should commit to pen or screen the whys, the hows, and the reasons; but we also know it is a pain. It really is, so we don’t do it. What we have instead is a body of knowledge that is locked within the heads of developers, that is passed along in fits and spurts, when prompted and only where necessary, frequently without any comprehensive framework of common understanding among stakeholders.
As Jason points out, Grady Booch has popularized the phrase “tribal knowledge” for this kind of information artifact. Documenting for the future preserves the oral tradition by encoding knowledge that already exists. It supports the later transmission, socializing and sustainability of the "tribal knowledge" of the team.
So one type of documentation we create supports future development by preserving the oral tradition that teams develop along with the software. Without this kind of documentation "…the collected tribal knowledge degrades into “tribal mythology” (Booch). When this happens, no one really knows how the system ended up the way it has, and the knowledge is lost.
This kind of supporting, future-facing documentation is particularly relevant where such knowledge is not immediately apparent by reading the code, supporting tests and other artifacts. Such documentation is typically written after features/modules are implemented/delivered. It can be produced as the software is being built, but then there is the additional maintenance cost of keeping it up-to-date as things change.
Preserving Tribal Wisdom
So we want to avoid tribal mythology by documenting our systems as necessary. We want to capture and preserve for people to come the "tribal wisdom" that has been gained in the rough-and-tumble of developing the system. As Jason points out:
Tribal wisdom, however, is the virtuous flip side of this tribal mythology. It is prescribed action with understanding, how accompanied by why, and is adaptable to new environments, new situations, and new problems. It transcends rote copying, and provides illumination through a comprehensive discussion of the reasons behind the action.
Elemental Design Patterns
At some point in the past, for almost every action or decision in a system, someone knew why it was done that way. Carrying those decisions forward, even the small ones, can be critical. Small decisions accrete into large systems, and small designs build into large designs. By ensuring that we have a strong tradition of knowledge retention that facilitates understanding, we build a tradition of tribal wisdom.
Elemental Design Patterns
Favor Documenting over Documentation
So we support future development by preserving tribal wisdom through documentation, but what about supporting the team as they develop the product?
In the same sense that agile teams favor planning over following a plan, they tend to favor documenting (as an ongoing, just-in-time, activity) over creating a (once-and-for all) document. And in the same manner that their planning is focused around high-fidelity communication, customer collaboration and team interaction, any documenting they do tends to have the same goals and characteristics.
A plan is only useful until it needs to change, which is why agile teams focus on enabling and responding to change. The intention is the same with any documentation they create in service to building a software solution - it should not be painful, but rather serve the team in better understanding the problem space, and helping the team grasp what the solution needs to look like. Let’s look at some important characteristics of this style of documentation:
Characteristics of Useful Documentation
Trustworthy
This should have to go without saying but, like comments in code, much of the documentation that exists cannot be trusted. If you have documents that are supporting your development, make them living documents by keeping them up to date. They must be correct. They must speak the truth about the software and the business domain.
Malleable
Part of keeping documents trustworthy is enabling change. Documentation must be malleable - make it as easy to change as possible. Reduce the friction of having to change it. Documentation that is burdensome to change is less likely to be kept up-to-date.
Makeing it malleable typically means making it as lightweight and informal as possible. Prefer hand-drawn diagrams over created in a tool (such as Visio), prefer electronic over hard-copy. Only include the pertinent details. Indicate which things are tentitive, and which may be harder to change.
The important thing is to understand the purpose of each document, and ensure that it is kept up to date. As much as possible, push the knowledge into the code and the tests.
Accessible
Documentation must be as accessible as necessary. Things that the team is working on right now, I would expect to be on the walls of the team area. Just the same as many teams use information radiators such as burndown charts and task boards to track their delivery progress, I like to see sketches of design diagrams on the walls too.
I like to see a context map on the wall, showing the terrain the team is dealing with. I’ve worked with many teams that were not co-located, so we would put the the documents in shared folders, and on the wiki. Sometimes we would sketch on a whiteboard, and then take a photo of the diagram and put it on the team wiki.
Don’t let your wiki fall prey to the Tragedy of the Commons. Appoint a curator for your documents if necessary. But strive for team-ownership of the documentation, just as you strive for team ownership of the code.
Documentation and Doing DDD
DDD teams often find they have a leg-up with documentation, because they devote so much effort to distilling domain knowledge into the software itself via the domain model. Teams doing DDD are focused capturing the essence of the critical concepts of the core domain in the domain model itself. With DDD the rules, reasoning, assumptions and key business concepts are embedded in the software.
When I start with a team, the first thing we draw together is a context map. This diagram helps set them up for success in terms of knowing what context they are working in, how it relates to their core domain and the other contexts they need to interact with.
For DDD teams, and for software teams in general, the important thing should be not that the domain is documented, it is that it is understood, and that this understanding is shared among everyone connected with developing the software. Good documentation engenders a shared understanding of the business domain. Good documentation for a DDD team captures the essence of the reasoning around the domain model: a rich, expressive software model that enables significant business capabilities in the core domain, supporting the strategic goals of the business. Teams doing DDD accomplish this by simplifying domain complexity enough to provide a shared language and understanding, and embedding it in the code.
DDD is not prescriptive about documentation. What documents are produced usually has more to do with the team’s existing process than doing DDD. However, there are certain kinds of documentation that teams doing DDD do find very helpful. Let’s look at some of these.
Requirements Specification?
Many teams opt for user stories as items in a feature queue, prioritized by value to the business (i.e. "Product Backlog", in Scrum terms). See my earlier blog post on user stories and DDD.
A team doing DDD could use a requirements specification document. But the trap with heavyweight, detailed specification documents is that they tend to separate design from implementation.
The theme running through all of my experience is that the long list of things we have come to call requirements – and the large backlog of things we have come to call stories – are actually the design of the system. Even a list of features and functions is design. And in my experience, design is the responsibility of the technical team developing the system.
I suggest we might get better results if we skip writing lists of requirements and building backlogs of stories. Instead, expect the experienced designers, architects, and engineers on the development team to design the system against a set of high-level goals and constraints – with input from and review by business analysts and product managers, as well as users, maintainers, and other stakeholders.
Agile teams tend to eschew producing detailed requirements specifications, preferring a more light-weight approach to describing what the system needs to do. The problem with such documents is that design decisions are made too early, with insufficient domain and technical knowledge, and having it written up in a specification tends to set that ignorance in concrete.
All too often, detailed requirements lists and backlogs of stories are actually bad system design done by amateurs.
The risk in this approach is that:
Separating design from implementation amounts to outsourcing the responsibility for the suitability of the resulting system to people outside the development team. The team members are then in a position of simply doing what they are told to do, rather than being full partners collaborating to create great solutions to problems that they care about.
Most teams I coach are following some form of agile process (Scrum, XP etc) and thus tend to focus more on rapid feedback loops and incremental development over producing copious amounts of documentation first. This tends to aid with modeling, as the documentation is produced as-needed, rather than to get through some "gate" in a prescribed SDLC process. The code itself is the design, paraphrasing Jack Reeves.
Some teams find it helpful to develop a list of use cases, a list of tasks the program is able to perform or some combination of both. I would experiment with what you find most useful for your team. Use cases have fallen out of vogue recently, but I am still a big fan of them.
Note that I am not against specifying requirements in written form, but rather entombing those requirements (i.e. what features the system should provide to meet the customer’s needs) in a large tome that locks-in the details of how the system should behave. I have utilized use cases in a lightweight, just-in-time way and found them very useful. See Alistair Cockburn’s article on Why I still use use cases for similar reasons to mine.
I would also strongly recommend using mockups and prototypes as much as possible.
Core Elements
I typically create a short document that captures the core domain vision statement and the context map.
Architecture
Architecture is largely orthogonal, but supportive, for DDD. I find the "4+1 architecture" approach to be the most useful. It is useful to keep in mind that, as Grady Booch declared in 2009, architecture is a shared hallucination:
Architecture is just a collective hunch, a shared hallucination, an assertion by a set of stakeholders on the nature of their observable world, be it a world that is or a world as they wish it to be. Architecture therefore serves as a means of anchoring an extended set of stakeholders to a common vision of that world, a vision around which they may rally, to which they are led, and for which they work collectively to make manifest.
Notice that in Krutchen’s approach, scenarios are the unifying thing. Reference scenarios are a more specific form of this. See my presentation on domain scenarios at the DDD Exchange 2012 for a walkthrough of using reference scenarios. In DDD reference scenarios describe the key business problems that the model needs to solve.
Reference scenarios will be the core domain business capabilities that the software, and in particular, the domain model, will enable. They often take the form of a short narrative, with a supporting diagram. Not starting out that way, but the key is capture the significant details that make the problem worth solving for the business.
George Fairbanks book, Just-Enough Software Architecture is the best book I’ve found on characterizing, describing and documenting software archtictures. I love the pragmatic, risk-driven approach to architecture that this book takes (the sections on modeling alone are excellent, though it defines DDD too narrowly for my taste). If you are looking for something more comprehensive in the software engineering tradition, then it’s hard to beat the definitive tome: Documenting Software Architectures.
Ubiquitous language
It can be helpful having a document that explains the Ubiquitous Language. Many teams develop a dictionary of significant business terms early on, and for a team with a business analyst this can be a very significant contribution. However, the same caveats mentioned above relating to separating design from implementation are particularly relevant:
In most software development processes I have encountered, a business analyst or product owner has been assigned the job of writing the requirements or stories or use cases which constitute the design of the system. Quite frankly, people in these roles often lack the training and experience to do good system design, to propose alternative designs and weigh their trade-offs, to examine implementation details and modify the design as the system is being developed.
So as with all the documents described here, the dictionary must be kept up to date to be useful. Such a dictionary can be an important start, but it shouldn’t be the end. I like to see it developed into a document that has diagrams showing important states of the model, and how the terminology of the domain model is used.
As terms change over time, such a document can be a good place to explain why these changes in language were made, since that kind of historical information won’t be obvious by looking at the code etc.
Informal UML diagrams
I am always sketching UML diagrams on whiteboards. It saddens me that many teams don’t see the value in this. I particularly find instance diagrams particularly useful in walking through scenarios with domain experts. I find that when the domain experts see the concrete, pertinent business data values in the "little boxes" in the diagram, it really helps with understanding what the model is expressing.
Many times when I work with a team that has an existing model, one of the first things I will have the developers do is walk me and the domain expert through a reference scenario on the whiteboard, explaining how the model supports solving the important business problem. This activity alone is often enough to show strengths and weaknesses of the domain model. Instance diagrams also really help with understanding aggregate boundaries, since aggregates are runtime artifacts.
Sequence diagrams can be very helpful for understanding the application flow from the UI, API, or context boundary down to the domain model. And also in understanding interactions between sagas, objects, domain services or aggregates (such as via application services or other infrastucture responsible for eventual consistency between aggregates).
To create electronic versions such I often use light-weight UML sketch tools such as [Web Sequence Diagrams](http://www.websequencediagrams.com) and [yUML](http://yuml.me). I like the way these tools produce diagrams that look hand-drawn, which lends them towards being viewed as transient and gives the team permission to change them. One of the problems with producing high-quality UML diagrams is that it tends to communicate that they are "done," and shouldn’t be changed. That they are finished.
Anything else?
I’m a big fan BDD tool such as Cucumber to create living documentation for the system, if the team has the skills and experience with such a tool. For example, the following feature file helps support the ubiquitous language supporting the underlying conceptual model represented in the domain model.
I’m biased towards Cucumber as a tool because I like the separation of steps in feature files and stepdefinitions encourages the separation of ubiquitous language from the technical implementation. The business terminology goes in the feature files, and should be refactored as the ubiquitous language is refined over time.
I am co-authoring the book BDD with Cucumber for Pearson/Addison Wesley. The book will cover doing BDD using Cucumber (Ruby), Cucumber-JVM and SpecFlow.
But it’s not the tool that’s most important, the same thing could be done with other acceptance testing frameworks such as Concordian, Fitnesse or Robot Framework. There’s an interesting discussion going on right now on the Agile Alliance Functional Testing Tools (AA-FTT) mailing list about these frameworks and the various tradeoffs they provide. The important thing is the improvements I see in communication and collaboration when teams use these tools to refine acceptance criteria for user stories.
Standalone vs. Combined Documents
No preference for this. Most teams work this kind of thing out on their own over time. I’m not even sure what the factors are for deciding whether to combine documents or not. My preference is to keep documents short and focused. I find they are more likely to be read and used if they are concise and cohesive - maybe principles of good software module design could be pertinent in structuring documents too.
My preference is for diagrams surrounded by text. If a picture is worth a 1000 words, supporting text that explains the critical aspects of the diagram is a multiplier for this in terms of utility.
Respect Your Audience
Finally, and most importantly, when writing any software documentation consider your audience. Will the readers be coders? testers? domain experts? all of the above? Is this technical documentation, or business-facing documentation? How you answer these questions should factor strongly in terms of what kinds of information you include in the document, particularly how much technical detail you incorporate.
There’s probably a lot of things I’ve missed here. What has been your experience with doing DDD in terms of documentation?
Further Resources
Two excellent resources released since I wrote this article are:
Updates
-
[Updated 24 Oct 2019] - Added Further Resources section
About the Author
Paul is a software design and development coach and mentor. He is available for consulting and training through his company, Virtual Genius LLC. He is the author of The EventStorming Handbook and major contributor to Behavior-Driven Development with Cucumber. He is also the founder and chair for the Explore DDD conference.