We create plans and estimates so that we can tell the story of the solution and, through the telling of the story, we can practice our craft. When we do it as an ensemble or team, we do it so we can work and practice our craft together, bringing all our individual skills into one cast of characters and producers. We do it to bring joy and amazement to our audience and we use our stories to help solve their problems.
Business needs plans and the tools of plans, such as estimates and budgets. As software story tellers we often sell our skills to make our living and those that manage the money - whether it's clients or employers - need to use it wisely and responsibly. They want a plan to show that they are not wasting the money and are able to account for it responsibly.
Not all software is created for money. Many open source efforts exist simply because of a desire to tell that software's story. Those projects usually have far less elaborate plans. Ironically, they are also some of the most successful stories ever told.
The need to plan, to organize budgets, time and resources is not to be dismissed. Some methods of funding a project cannot happen without an estimate or plan. This is the professional part of being a modern day story teller. The bulk of this message deals with the craft and the means by which the craft accommodates the need for estimates and plans.
To be professional, we need to do our best to help the business with it's plans, estimates, forecasts and resourcing, but it's not what drives us. It's not what fuels our creative juices or helps us hone our craft. For that we leave the profession behind and turn to the work of the craftsmen.
Software producers are modern day storytellers. Our tools are storyboards, designs, UIs, controllers, menus, widgets, prototypes, architecture and code. Our audience is the user and the actors our code, screens, and flows. We guide you through an experience and in the process we attempt to arrive at a resolution to some issue or problem. The stories are sometimes complex and the journey can be complicated but those are the stories we love to tell the most.
Our story starts when someone has a vision, an insight into a story they want told. The vision can be of a solution to a business problem, or the wildly entertaining design such as the experience of a clever advertisement. If the story is simple enough, it can be told with a single page or app, perhaps even a prototype. However, most stories are complicated. Like the stories found in books, movies, and games, most stories have an arc - a beginning, middle, and end.
On the internet, our story often begins when "a user comes to our site wanting to..." On the intranet, the story may be more about a particular business process and the journey through the story might end when an account has been updated. There are many stories and we want to help tell as many of them as we can. And we want to tell our story so well that the audience is inspired and will want to see what we'll do next. We want them to come back. We want them to love what we do.
As any producer, director, or actor will tell you, every story needs a script.
In "studios" or shops that practice waterfall, the script will be deeply developed before putting on the production. Part of that development might be an estimate and before the estimate a detailed work breakdown will be created. A list of producers and directors will be identified. And detailed notes about every screen, movement, and action sequence will be created in advance of the performance.
Estimations of the entire process are given before production starts. The estimates are part of a larger plan that involves a budgeting, staffing, resourcing, and timing for the entire production.
In an Agile "studio" the producers and directors work from short, incomplete stories that describe - in broad strokes - what each scene or collection of scenarios will accomplish. These small stories are subdivided until they can be produced in just a few days or weeks. At the end of each production period, usually one or two weeks, a performance is given in the form of a demo of the latest deployment of working software. Feedback from these "dress rehearsals" then inform the producers and other stakeholders on how to proceed.
Estimates are limited to the weekly or bi-weekly productions. Longer term forecasts are made from the rate, or velocity, at which the weekly or bi-weekly efforts are able to produce the short (user) stories - the scenes - in the form of working software. The forecasts takes this velocity and uses it to predict the release of larger and more complete versions of the entire story. This "release planning" then forms a picture showing when the entire set of stories, the story backlog, will be completed.
For the craftsmen, producing working software from the short incomplete stories offer a chance to inspect how well the story is being received by the audience and to make adjustments on the fly. With each successive release, the working software can be adapted to be more satisfying to the audience, customer, user or client.
Years of experience have shown that the waterfall processes are wasteful and less creative. It's well accepted that waterfall is more expensive and practitioners find it difficult to make changes. Adapting to new or modified requirements is costly and slow. Despite being less efficient waterfall often feels safer and provides more numbers and plans along the way. If you're organization is run by bean-counters and bureaucrats you'll find it hard to follow any other process.
It takes a different kind of director to produce a fully scripted story than it does to produce one that is mostly ad-lib'd. Not everyone is well suited for both processes and many people are clearly more comfortable with one process or the other. It's particularly hard for those invested in waterfall to do Agile. They often feel lost and confused by the uncertainty. They constantly suspect the client will realize they don't have all the answers.
Perhaps the biggest failing of waterfall is that it can take a very long time for the audience (the client) to see the story as working software. Is this the story they imagined? Does it make sense to them? Waterfall can take months to reveal the work of the craftsmen. By the time the audience decides whether they like the story, changes to the production are costly and impact both the schedule and the budget. When the entire project is only a month or two long, these weaknesses are rarely noticed but at an enterprise scale, they can be disastrous.
Waterfall's plans have also proven to be less accurate. Errors in the plan are additive. A schedule slip here and there and dozens of promises get broken. The closer you are to acting out a performance, the more you'll know about changes to the plot, the actors ability to work together, and continuity (does the flow, naming, and UI seem consistent). Agile takes advantage of this by making estimates of the work a week or two at a time right before the work is attempted.
An Agile team should have a very good idea of how much they will accomplish when they get together and plan what they will produced in the next week or two. Agile storytelling teams work hard to ensure that each deployment of working software tells an enjoyable story. Each deployment should stand on it's own and the producers can stop crafting the story at any time the stakeholders decide. This protects both the stakeholders and the story tellers. There is no need for either to lie to the other or make promises they cannot keep.
Agile teams are also pretty good at looking at the list of scenes and small stories that make up the entire production backlog and deciding which stories will be long and complicated and which are short and easy to produce. This allows them to estimate the time needed to produce a story or a scene and to rank them relative to all the others.
On the other hand, Agile postpones answering many important questions until the "last responsible moment." It can be tough to know exactly when that last responsible moment has come. This can make the audience feel like the actor has stepped off the stage and is asking for audience participation. Not all clients can work this way without feeling very nervous. It will almost certainly make the bean-counters nervous.
So why do we estimate and create plans? We do it so we can tell the client's story. We do it so we can practice our craft. We do it so we can help solve the client's problem by taking their concept and creating a solution. We do it to inspire joy, amazement, and the resolution of the story arc (business problem). We do it to produce something beautiful, something needed, something desired, and usually something for which the client will pay.
We estimate and plan so we can tell the story.
No one uses a piece of software because it came in on budget. They use it because they love the story it tells.
I was inspired by these:
Simon Sinek: How great leaders inspire action
The Inside Story
Dan Gilbert: Why we make bad decisions
Julie Taymor: Spider-Man, The Lion King and life on the creative edge
An interesting way of looking at it. No disagreements from me, but a question: How many studios buy a script that is incomplete, rather than fully fleshed out? I don't know, I'm not in the industry.
ReplyDeleteI'd assume they look at the writer and evaluate based on their past work how much they trust them, listen to a high level pitch of a script, agree to pay for more development and then wait to approve the full budget until the script is complete? If that's the case, I'd say that's more waterfall.
But then, how often do we hear about films running over budget?
Chris those are excellent points. To stretch this metaphor a bit more that Agile maps more to an ongoing TV show that has all the creative bits above but smaller deliverables. You would then "order" 12 additional episodes, but you get cost feedback as each one is written, rehearsed, have sets built, and are produced. Agile is that way. Think of each episode as a sprint.
ReplyDeleteI think movies are definitely more of the waterfall model.