On a previous project, I had a long-running discussion with several stakeholders about using story points over time estimates, because how could I know that we were going to deliver without a deadline?
I started using time estimates on a previous project because one release failed to deliver, and so I decided we needed a strong central leader, and I became the lead for the project. Over several releases, we worked hard to reduce the variance between the time estimates and reality from 100% on a fresh new delivery with a fresh new team, to 10% with an experienced team and a well understood domain.
So, with lots of days added to the estimates for the things we’d forgotten (like how long migration scripts took to build and test), a complete signed off set of requirements, a stable team and 4 people in a room estimating for 2 solid days for a 3 month project, we still needed to add 10% variance to cover unknowns. Because when you squeeze down to that much detail, you squeeze out contingency.
With a new team, I tried the same, but we were working in 4 week sprints, so I could review and adjust. Time estimates didn’t work. There was too much variation between the experience levels of the team, both in number of years, and experience with the specific technologies, and the domains, and how much mentoring they were each doing, and how clear the requirements were. After 3 sprints where the accuracy was getting no better than chance, I switched to story points, and started tracking how many points each person was doing (because I had to report it, not because I thought it was useful – it was a concession I had to make).
After 3 Sprints we had numbers to work with, and story points were a better fit for the variance in the team, although they were no more accurate, because we didn’t reset our expectations each sprint, or reset the backlog.
So we started doing that, and we implemented our green and amber lists. Some sprints we completed most of the amber list most sprints we barely scratched it. But we reached a point where we could predict enough that expectations could be set about when to expect features, and what lead times we needed.
But I don’t think story points really did that. By this point, most stories were 1 or 2, and no-one ever knew what a story point meant. Instead we broke each task into 1-2 day chunks.
And now, on my latest project, that’s all we do. If a task is small enough to fit in your head (even if you need several of them to complete a feature), that’s what we measure. Keep doing 2-3 of these a week, and we can set upper and lower bounds of when features will be delivered. It’s not perfect, and still needs tweaking, but assigning time to tasks up front, without knowing who’s doing the work, is backwards and broken, and I won’t be going back.