Story counting is a technique for estimating and follow up on velocity. The technique is simple, you just count the number of stories completed in a sprint or in a period of time, and the result is your velocity. It’s based on a simple assumption:
Over time the bigger and smaller stories will cancel each other out, hence a simple count ends up the same. – Martin Fowler
In a recent project, we started out with the “strict Scrum way”, we estimated in story points. We took each story, discussed with the product owner,tried to analyze all aspects of it, played planning poker and ended up with a relative number on each story. We spent quite some time discussing the estimates in the beginning, both because it was the first time this team had worked together and because those first stories in a project often are a bit vague. A few sprints later, we started to get better at this. We got a more stable velocity and we used our velocity to see how much to fit in a Sprint. However, we were often a bit too optimistic with the sprint goal and we often had work in progress at the end of the Sprint.
And then someone asked:
“Why are we spending time estimating in Story Points? The discussions we have when estimating are not giving value and we are overcommitting to our Sprints. Can we please stop estimating?”
Good question… The estimation was clearly not giving any value to the team. Planning poker was fun in the beginning, we sat there with our phones (search for “planning poker” in the app store/market), we often had some candy in the room and we tried to have a good time with it. But now we were starting to argue around the estimate itself instead of discovering the important aspect of the story. So why continue to do it? I could not really motivate why to keep doing this. But one thing I was not prepared to give up on was the team velocity metric. I wanted to have that in order to see how much we were delivering each sprint and so we could follow up and use it in retrospectives. But in theory, counting stories would give us the velocity as well? I had never used story counting before so I started to see if the theory were right. I compared the story count with our story points for the first 7 Sprints.
I was convinced… I disregarded the first few sprints, they were a bit special anyway, but from Sprint 3 and onwards it was quite clear that we could skip all the estimating and get a “correct” velocity with using a count rather than the points.
The story count worked well for sprint planning and to track velocity, but would it work for looking ahead more than one sprint? Should be doable right? You just take the number of stories you have left and divide it with the velocity? So if we had 65 stories left and our velocity was 15, we should have somewhere around 5 sprints left. Right? Right?? Well…. There are at least one big dangerous assumptions hidden in there:
- The whole backlog has stories of roughly the same size. And also, it will not grow over time.
We certainly did not have a complete backlog, the last stories were not refined, many stories were removed over time, and new ones were added. And this was mostly intentional. It looked a lot like Henrik Kniberg describes in his “Agile Product Owner in a nutshell” video. The items at the back were fuzzy and large, the items at the front was refined and roughly the same size.
And not only were the items large at the end, we had a constantly growing backlog when measured in terms of stories. Most of it was that we kept splitting stories as they moved through the backlog “funnel”, but also improvements to completed stories showed up there. It was defects, performance improvements, discovering new business needs etc. And that’s natural, new things will always appear and things will always change. But could we say when we were ready to launch the product based on story count? I tried visualizing this growth of the backlog to see if we could get some statistics about the “growth” of our backlog.
It’s quite clear that when we completed 15 stories, around 5 new ones appeared, sometimes more, sometimes less. This is simply a result of the story count technique combined with having a “funnel” shaped backlog. So forecasting a few sprints ahead would work, but more than that one would need to take growth as a parameter as well to the estimate. I never walked down that path and I guess the estimate would end up being a little bit less wrong, but it would still be just an estimate.
The bottom line
This was just a short story of our tests with story points vs. story count. For us, story count certainly replaced the team friction that the estimation created and we got a good velocity measure, but it was quite useless for forecasting long term. I guess the most important thing is to know how to use and not use a metric and especially how that choice benefit (or harm) your team.