When talking audiences who are new to agile software development, we often claim that there is an erroneous belief that agile teams eschew documentation. I was going to put it differently at first, but I have always wanted to use the word “eschew”. It is one of those words that appear in print much more often than in speech. Maybe because it sounds like a sneeze? If you have to look it up like I usually do, I will save you the trouble this time. Eschew means to avoid.
The fact is that I have never actually heard anyone state that belief. I have only heard it said as a straw-man to knock down. I don’t really know if people unfamiliar with agile really have that image. But I still go along with the story and assume a defensive posture most like other agile trainers. Rather than follow my first impulse and backpedal away from it, I learned early on to be more specific about the relationship between “documentation” and “agile software development”. This may be because I was one of those rare developers who actually enjoyed taking a break from C++ to write in English once in a while. That is not to say that the documents I wrote had any value in the long run, just that I didn’t mind writing them.
As for specificity, this is how I like to present it. Documentation is an umbrella term for many different things. To say that an agile project avoids writing documentation is like saying that an agile team avoids pizza. Maybe they avoid anchovy pizza, but that does not mean that they avoid all pizza.
Here are some different kinds of documentation or “artifacts” as some of my CMMI acquaintances call them. Let’s see how agile projects might handle each one of them. I will probably miss a few. Please let me know. Also, note that I am assuming an ideal case here. If I have learned anything in the past few years it is that there is a wide variation in the types of projects that go agile and they all have special needs.
There are three types of documentation that most people think about first and foremost, requirements, specs and design.
Requirements Documentation
Traditionally, requirements are written by the business side of things at the start of a project and then revised a little bit soon after. This is the first thing that people go looking for when they start their first agile project. They might not find it, unless perhaps the project was already planned out before it went agile.
Does an agile team write a requirements document? Well, sure they do. It just takes a different form. And it is done incrementally just like the code. The form is fluid, as well. Requirements are expressed as iteration features or user stories. What ever term the team uses, the intent is the same – what we are building right now. They may exist on paper, stickies, cards, a project management tool, text files, spread sheets. They start out in a high level form and are then elaborated as they are worked. If the team or the organization or the overseers need a record, they should be recorded in some way. Requirements in this form have a different purpose than in a plan-driven project, though. They are a reminder to have a conversation about the details and value of the features, not a contract between parties who may never meet.
Specification Document
A functional specification is usually written by one or two technical leads to describe what the fine details of what the system have to do to meet the requirements. It is often delivered to the business as a promise. Just as often, no one in the business can understand the technical jargon and it just gets approved.
Do we have specification documents in an agile project? Well, sort of. We usually don’t write one up before we start the coding, though. As with requirements, specifications are addressed mainly at the time the associated code is written. They will also take a different form – added to story cards, broken down into task cards or tasks defined digitally in some spreadsheet or agile planning software. More and more specifications are being written in the form of tests – unit tests, integration tests and acceptance tests. So, ideally, an agile team writes specifications but in new ways that may actually be executable. And several people contribute, not just one or two leads.
Design Document
A design document is a description of the system that will achieve the requirements. At least it is intended as such. Usually the design thought of up front is not the design that actually gets implemented. The agile philosophy leverages this fact and trusts in evolutionary or emergent design. So of the big three traditional documents, this is the one that most often “turns up missing”. I am not going to go into the question of “how can that possibly work?” It would be preaching to the choir.
I also happen to believe that bigger projects will benefit from an architectural design done up front and communicated to all necessary parties. But not a big one and not a rigid one. It is a best-guess guide, not a commitment.
Do we have anything to replace it? Well, some people feel strongly that the code itself is the design document – the ultimate artifact – and that any attempt to write a separate text document will lead to problems. I will describe a lightweight approach to this in a future post that I have come to call the “as-built design document”. For the most part, the closest thing most agile teams produce would be a series of whiteboard drawings, possibly preserved with a digital camera.
There are some other documents that projects may want to produce. People new to agile do not always think about these at first.
Project Charter
A project charter is a tool in use beyond the agile world. It describes the project in a few pages – what it is about, the value proposition (expected benefits vs. budget), how it will be done (“markitecture”), the target market, when it needs to ship and who all need to know about it. This is not always done for a traditional project. I think it is a good starting point for an agile project. It serves to align all stakeholders and set expectations all around.
User Documentation
The decision to create user documentation is not an agile decision. It may be treated differently in an agile project, however. You can think of it as a user story to be prioritized along with all other stories. Let the product owner decide how important it is relative to system functionality. The good news is that user documentation usually does not consume much developer time. A tech writer has the right skills and can be a part-time team member. And if the user help is written incrementally in step with functionality, it stands a better chance of being correct when the project ships.
Maintenance Documentation
Pretty much no one likes to do this. Many traditional projects assume that the functional spec and design document are good enough. Always challenge your assumptions. Anyone who has done software product maintenance knows that they are not all that useful. Often they do not match the system that was built. Same goes for comments in the code.
Does that mean that an agile project does not need to write a maintenance document? It is for the team to decide – or perhaps the organizational process rules. The as-built document mentioned above can serve this role. More on that in a later post.
Configuration/Installation Document
I once worked on an 8 month project to revise a large web service. It had a configuration doc that was 60 pages long, all manual steps required to deploy the system into production. I think that the Continuous Integration approach is much better – work to get deployment down to one click through frequent practice and on-going updates to installation scripts. Ideally, an agile team will not need to write one of these documents.
Security Documents
Things like data flow diagrams and threat models may or may not be a standard part of your current practice. Most security reviews are one-offs and not treated as part of incremental development. So for now, do these if the organization requires. Treat them like a feature, planned as part of each release and timed so that the security review does not delay the release. I have more to say about security in an agile project, saved for a later day.
Does an agile project avoid documentation? I don’t think so. They are just more practical about which kinds are needed, what forms are more useful and which are not adding to the value of the work.
