Software Development as a Christmas Story

Oh Tanenbaum

We decorated the house for Christmas. A smaller project than most of the software I’ve worked on, but a useful reflection.

The Cost of context switching

Sometimes, in the middle of one task, you need to do another. Either because you were interrupted, or because you weren’t prepared. Consider the tree, a tall tree, one you need a ladder to decorate. But you left the in the attic.

You don’t just have the cost of picking up the star, you need to get down the ladder, fold it up, carry it upstairs, open the attic, unfold the ladder, climb to the attic, find the star, then pick it up and unwind all the other steps just to get back where you stated before you can continue the task of putting the Star on the tree.

It’s just a minute to get the star…

Yak shaving

Sometimes it’s more than just one thing. Sometimes to get to where you need to go, there’s a cascade of other tasks.

You want to put the tree up, but it’s the first time you’ve had a real tree, so you need a new base. It’s bigger than last year’s tree, so your lights don’t fit. So you need to go to the shops. But you need to fill up your car. And the tyre pressure warning light comes on so you need to top them up. And you need to check the tread depth whilst you’re there, and so on.

Programming in a nutshell

Performance at scale

Our tree stood up fine when it was delivered. But as it scaled out and the branches widened, it pushed against the wall, making it unstable in the condensed space. It fell over.

Luckily no lights or baubles were using it at the time, but it’s an interesting challenge holding up a heavy tree in one hand, trying to adjust it’s position with the other hand, as I avoided the puddles of water in the floor as my wife mopped them up. If you’ve ever worked support, this may sound familiar.

Turns out that it was harder to stabilise than I anticipated.

The branches were unevenly partitioned, providing more load on one side, so I had to stabilise it against the wall. And the tree was almost a foot taller than expected, which turned it out to be 2 foot taller than the base was rated for.

We upgraded the base to handle the load. It’s bigger than we need.

Technical debt

I have some old pre-LED tree lights and they slowly fail so each year I replace the unlit ones from the packet of spares but I haven’t been replacing the spares. Eventually, they will run out, and the spares will no longer be available. I’ll have to throw them all out and start again.

The new led ones don’t let me replace them individually. But they last longer and are cheaper to run.

Which debt is easier to live with?

With a big tree, those old lights aren’t long enough. So, do I buy another set for the rest of the tree that doesn’t match them, or throw out the existing ones and buy a longer set? The latter looks better, but throws out the sunk cost along with the lights.

You know computers, can you look at this for me?

No, I can’t fix your tree. I can navigate a garden centre. I know enough physics to keep a tree upright, eventually, and safely put the angel on, but I’ve no Idea how to grow one, or what that weird stand with 17 gears you bought does, or how to assemble your plastic one. And I’ve no idea what that bug in the tree is.

Merry Christmas. Catch you all in the New Year.

Sense and Respond by Jeff Gothelf and Josh Seiden

Sense and Respond: How Successful Organizations Listen to Customers and Create New Products Continuously by Jeff Gothelf

My rating: 5 of 5 stars

Having read Lean UX, I was expecting a good book on how organisations could deliver projects. This book is about much more than that. The authors are coming from a point of frustration where agile delivery and Lean UX projects have failed to reach their potential because the organisations surrounding them were not supportive, so this book is about how to reshape an entire organisation to meet the challenged of the modern world, with lots of positive and negative examples, which is refreshing to see in this type of book. If you’ve ever had a strategy meeting, or ever found your project success limited by your organisation, go read this book.

Easily the best book about dysfunctional organisations and organisational change I’ve read since Peopleware.

View all my reviews

Buy on Amazon UK :

Jettison sandbags to focus on your goal

Whether your deadline is near or far, there will always be things on your team list that you don’t need to do. It’s tempting to take ownership of everything, all the servers, all the code, all the admin. As a team of top developers, you could also write a better timesheet management software than the one you’re stuck with.

But should you?

Does it help you with your mission? Does it provide business value? Can you do it better and cheaper than what you can buy in? Do you have a unique circumstance that means you can’t just style an off the shelve solution? Really, are you that unique? Do you want to support it once it’s released?

More important than any plan is a strong purpose, a goal. Something small that everyone understands so that any task can be quickly evaluated against it. If the task doesn’t fit the goal, it’s irrelevant, ditch it. It’s baggage. Even if it’s an existing system.

What’s your team’s purpose? Or at least your 12 month goal?

The Fog

Dream in years.
Plan in months.
Evaluate in weeks.
Ship daily.
— @dpatil

— Mosh (@mishmosh) January 12, 2017

Agile is about the here and the now. The moment, turning on a sixpence as the weather, and the market changes. Adapting to your environment and delivering, delivering, delivering at a consistent high quality.

But what if you’re delivering the wrong thing?

It’s not just about the pivot, it’s about the plan and the dream. How can you decide between prioritising the mobile app or the chat interface unless you know where you’re going?

Yes, we prioritise, reschedule, re-evaluate constantly, and we only ever have a clear path for the next timebox, no matter how much our Gantt charts and managers persuade us otherwise.

But what are you doing next? Do you need to build your storage layer for SSDs or the cloud? Are you optimising for read throughput and fewer writes, or network performance and fewer reads?

Don’t let the immediate goal cloud your thinking. You may not be able to see clearly where you’re going in terms of detail, but you should be clear what direction you’re heading, where the big obstacles are, and who’s nearby.

Out there in the mists are the challenges, the gaps, the shortcuts and the dead-ends. Don’t shoegaze yourself into a corner.

Don’t forget, the feedback loops don’t stop at your retrospective intervals. Look up once in a while to make sure you’re on track.

A task/story is not a feature

Agile is easy. Take the requirements, break them down into tasks, schedule them, do them, review them, test them, ship them.

And then you realise, as many on the agile journey do, that your “spreadsheet import” task is too big. It doesn’t get finished in the same timescale as other tasks, so it sits on its own branch, drifting from the mainline, or it holds up the release until it’s done-done-done.

• As a user
• I want to import spreadsheets
• So I can use data created by other programs

Break it down. Make each task count. The feature fractures into tasks, micro-features, one for reading each format you want to support; one to deal with classes of bad data, those mismatched quotes and uneven lines, one for paging to deal with larger spreadsheets.

Just as we have to break down each interaction into multiple facets, and multiple services, to create the smoothest possible user experience, so we break down each feature into componentized tasks to let us integrate more often, review more often, release more often, and provide the smoothest possible developer experience.

If your process doesn’t flow, fix it, smooth it, deconstruct it.

Story points over times in estimates, and the power of abstraction over teams

On a previous project, I had a long-running discussion with several stakeholders about using story points over time estimates, because how could I know that we were going to deliver without a deadline?

I started using time estimates on a previous project because one release failed to deliver, and so I decided we needed a strong central leader, and I became the lead for the project. Over several releases, we worked hard to reduce the variance between the time estimates and reality from 100% on a fresh new delivery with a fresh new team, to 10% with an experienced team and a well understood domain.

So, with lots of days added to the estimates for the things we’d forgotten (like how long migration scripts took to build and test), a complete signed off set of requirements, a stable team and 4 people in a room estimating for 2 solid days for a 3 month project, we still needed to add 10% variance to cover unknowns. Because when you squeeze down to that much detail, you squeeze out contingency.

With a new team, I tried the same, but we were working in 4 week sprints, so I could review and adjust. Time estimates didn’t work. There was too much variation between the experience levels of the team, both in number of years, and experience with the specific technologies, and the domains, and how much mentoring they were each doing, and how clear the requirements were. After 3 sprints where the accuracy was getting no better than chance, I switched to story points, and started tracking how many points each person was doing (because I had to report it, not because I thought it was useful – it was a concession I had to make).

After 3 Sprints we had numbers to work with, and story points were a better fit for the variance in the team, although they were no more accurate, because we didn’t reset our expectations each sprint, or reset the backlog.

So we started doing that, and we implemented our green and amber lists. Some sprints we completed most of the amber list most sprints we barely scratched it. But we reached a point where we could predict enough that expectations could be set about when to expect features, and what lead times we needed.

But I don’t think story points really did that. By this point, most stories were 1 or 2, and no-one ever knew what a story point meant. Instead we broke each task into 1-2 day chunks.

And now, on my latest project, that’s all we do. If a task is small enough to fit in your head (even if you need several of them to complete a feature), that’s what we measure. Keep doing 2-3 of these a week, and we can set upper and lower bounds of when features will be delivered. It’s not perfect, and still needs tweaking, but assigning time to tasks up front, without knowing who’s doing the work, is backwards and broken, and I won’t be going back.

Divergence: bid estimates vs planning estimates

​Fixed price bids need to control scope, and make assumptions to meet that bid.

Fixed price bids never survive contact with reality.

The first thing I do with those assumptions when delivering a project is pull those assumptions onto the plan, because every one is an unanswered question that needs to be validated, and each one probably has more assumptions behind it. If it’s hosted on the customer site, what version of Windows and SQL server will it run, what ports will be available, what libraries can we install, …? If we host it, What availability and resilience requirements does the customer have? What are the SLAs?

There are ways to model these in bids, but each one represents a waypoint where scope will change, sometimes our assumptions will be correct, often they will need to adapt to unforeseen information.

Ultimately, the public procurement process is not designed for change, despite the improvement GDS is driving. (possibly private too, but I’m in no place to comment). Trying to estimate for everything up front has always been a fools game, but attaching money to them makes estimates even more of a negotiation, turning them into notional numbers dependent on a massive pile of assumptions that only Mulder would believe. 

Treat assumptions as dependencies, and don’t trust any estimate, or requirement that depends on them. Test your assumptions. Always. And test yourself to know what assumptions you’ve made implicitly. 

And stop wasting time estimating. 

Planning: the green and amber list

When doing scrum planning, trying to figure out what can be achieved in a sprint, there’s always variation. Especially in projects where you can’t rely on what came before. You can use yesterday’s weather, and break down tasks, but if you can’t estimate more accurately than “between a day and two weeks”, and you can’t use Kanban, and you have a big enough team, one factor I’ve found useful is to create two backlogs for the sprint. 

In the green list is all the things you know you can complete, and are the most important. The amber list is what you know you can start but can’t guarantee to finish. These might be bigger, exploratory, spiking tasks, or smaller, less important ones that can be done by the developer who finishes the spiking tasks towards the lower end of the estimates. 

Mostly these tasks are the highest priority for the following sprint and will move to the green list, but occasionally they will be smaller tasks that can roll over to the next amber list – such as a task you want a particular developer to handle as a quick win in a new area of code. Importantly, they are never blocked and always ready to start at any point in the sprint.

How do you deal with minimum and maximum caps for scope when planning? 

Bad change : re-opened tickets and the neverending change

It goes on and on and on

One reason I don’t trust change is when that change has no defined end goal. When a change is requested, and the ticket completed, but it then enters a cycle of scope-creep that means the ticket can never be closed.

They often start with something simple e.g. “can you improve the performance of this search”, with a simple tweak – add a JOIN clause to avoid SELECT 1+n queries. In some cases the user acknowledges the improvement, but some users will start pushing new features against the request “because it’s in the same area”. After all, changing the colour of the links on the search page is still part of the same use case, right? And the fact the search is still slow when I use this rare combination of fields means the original ticket wasn’t fixed.

Close it and move on

In some scenarios, it’s easy to be brutal. These are time-bounded sprints, so anything raised must be factored in. This works where features and bugs are interchangeable and represent simply “work to be done”, however they are charged, and let the retrospectives and other analysis worry about whether any particular piece of work is a bug or a feature, so that the team can ensure it’s delivering the best business value.

The definition of done needs to be clear from the ticket (although maybe not to the detail of a SMART objective, it’s worth thinking about that framework). If the provided steps no longer produce an incorrect result, or a speed improvement is recorded, or some other pre-defined success criteria is met, then the ticket is done. Any further discussion is a new ticket. If the objectives aren’t met, it should not be marked as done and the developer should continue work on it.

It’s under warranty

Many software products and projects have a grace period of warranty, where bugs are fixed for free. This is typically useful to resolve conflicts in untested combinations, or volumes, but it can also be used as a stick if the developers aren’t careful. Provided the warranty period is reasonable for the size, complexity and support arrangements within the project, bugs should be fixed to ensure the original scope is delivered. Some customers will however try to extend scope by issuing bug amendments to change scope, under the guise that the bug was raised during the warranty period is therefore should be resolved. After all, bugs should be fixed, but features must be paid for.

Make sure you have a good rapport with your client, and make sure the terms are clear.

Decide now what done means, and how to manage tickets

The last thing you need is to decide this when the tickets that’s blocking the release gets re-opened for the 100th time because the CSS is broken in IE6.

What does it mean to be done?

What constitutes a failure?

Can the software be released as it is?

Just-in-time planning

Is it small, or far away?

I’m thinking about the planning and estimation process deeply, because it’s taken up a lot of my time on fixed-price projects.

I think my biggest problem at the moment is trying to square the circle of the 4 purposes for the planning and estimation we are currently doing, in order of detail : scoping, horizon planning, just-in-time planning and change control.

Scoping and change control are the most important to the project board, horizon planning is most important for setting deadlines for deliverables to and within the project team, and JIT planning allows us to define work at an individual level over a sprint.

Scoping

Scoping has and, I believe, will continue to be a horse trading exercise, allowing us to talk in broad terms about including and dropping functionality, based on workload, risk and timescales.

These scopes are defined mainly from the initial bid and budget and have undergone minor changes.

Horizon Planning

The horizon planning is the key deliverable from last week’s planning meeting and today’s requirements discussions : making sure we set deadlines for delivery of documentation and components to an agreed timescale.

Just-in-time planning

JIT planning is the bit the whole team should be involved in. It’s how we break down the tasks and decide what and how much we do in the next sprint, at a level of granularity that minimises crossover of tasks between developers, and therefore maximises parallel execution.

Change control

Change control requires estimation at a lower level of detail so that we can identify any difference in expected effort between our current plan and the results of any workshop. This must be done before the use cases, but to a similar level of detail, and we will need to add a step during the production or review of those use cases to highlight variations, so that we can suspend delivery of those use cases until the change has been managed.

I understand why, for political reasons with certain customers, the change control detail is necessary, but I prefer to ensure the wider team remain focused on the horizon and JIT planning, to shield them from the politics that may cause disruption within the team.

---

Lots of good quotes in this collection of articles : InfoQ – Agile Planning and Estimation http://www.infoq.com/resource/minibooks/emag-agile-estimation/en/pdf/Agile-Project-Estimation-and-Planning-eMag.pdf

Why Is Scrum So Hard?: http://youtu.be/q3t8twm3aUk