If it hurts, stop doing it: the wrong process

If something is painful, should you do more of it? Not if it’s painful because you have the wrong tool.

Sometimes though, it doesn’t matter what the tool is, there’s something more fundamental at play. A process that exists only because one thing went slightly wrong years ago. And rather than implementing a process to correct mistakes, somebody tried to prevent them.

Just like technical debt, this process debt adds pain to every feature, every bug fix, or every release. That pain can be removed by removing the process.

It’s painful because you shouldn’t be doing it

Does your test plan or your requirements sheet still specify IE as a supported browser? If Microsoft doesn’t support it why should you?

Are you spending all your time monitoring your staff, making sure they’re working when they’re not in the office? Do you struggle getting the right reports? Do you feel resentment from your staff even though it’s in their best interest? Or have you tried trusting them?

Is every release delayed because the database team and the security team have to review all code, and they’re already overstretched? Have you asked them how to provide confidence with less manual intervention? How to minimise the impact any change could make? How to add automation for common areas? How to train the developers to fix common issues upstream before it gets to the frontline teams?

Have you ever thought about deleting a process that isn’t adding anything? About making things simpler?

`The Untapped science of less : Inquiring Minds podcast’

Advertisement

If you truly want people to be creative and innovative, take them off the clock

That doesn’t mean no deadlines, but no timesheets – don’t justify every 15 minutes with a project, because the next ideas aren’t about 1 thing, they’re about connecting multiple things.

They’re about taking time to pause and thinking about the bigger picture: what problems are you seeing in multiple places? Where else would that new thing you’ve built be useful? What are multiple clients asking for?

If it hurts, stop doing it : the wrong tool

There’s a theory under agile, lean and similar methodologies that if something is painful, you should do more of it. If releases are infrequent and error-prone and once a quarter, do them 10 times a day and they’ll get easier.

Same idea with performance reviews, customer feedback, and security audits. If it’s a good idea and it’s painful, practice it and refine it until it’s natural and mostly painless. And the pain that’s left is manageable. Roll back the release, and have another catch-up tomorrow once tempers have dampened.

I’ve seen people make the mistake of assuming that it should apply to everything. Every pain point is a gathering, a thing to be controlled, minimised and made less painful, by repeating it over and over again. After all, if it works over there, it should also work over here.

But not all pain is equal.

Remember, focusing on doing something more means that we deal with the pain by eliminating it. We automate releases so we can throw out that painful checklist. We give small, actionable feedback at the time, rather than a sucker punch that brews for months until it’s released in the appraisal.

But don’t mistake pain for discomfort. Making big improvements will mean transitions that are scary and uncomfortable. And what’s painful for someone else might not be painful for you. That doesn’t mean the pain isn’t real and it still needs to be dealt with.

Here’s a few things that are painful because you shouldn’t be doing them. These are the pebbles in your shoes that you need to remove.

It’s painful because it was never built for that

I know there’s a lot of hate for JIRA. It’s the tool of choice for “Safe Agile” enterprises. And it gets a bad rep for being an overcomplicated monstrosity.

I was a JIRA admin once, bringing the tool into our enterprise. There were things I didn’t like about it on a technical level, but the central tool, with the defaults, isn’t terrible. But it’s so customisable, that you can codify any corporate process you like. And when it causes frustration, people blame the tool, not the admin. When the tool is the process, it makes concrete what people could fudge, and suddenly everyone has to work the way of the manager who needs to show their impact.

Start with the people. Don’t build a process around what people should do. Find out what they actually do and build from there. Some of it might be wrong, but find out why, and help them fall into the pit of success.

Don’t blame the tool for a broken process.

Processes upon processes: the JIRA trap

It is fashionable to hate on JIRA for software developers. Project Management made spaghetti. It has its faults, but the biggest issue is what it allows. It’s not opinionated, so any user can define any process to follow. It’s a perfect machine for generating red tape, or paper clips.

Because every time something goes wrong, the natural instinct is to add a new process, a new safety net, to make sure it doesn’t happen again [see Agile is Dead blog post]. And once added, they’re very difficult to remove.

So we get processes upon processes, the simple rhythm of a ticket lifecycle or of a sprint adorned with Deferents and Epicycles as we try and tame ever-increasing complexity with more text boxes and more statuses.

Complexity cannot fix complexity. But who has time for simplicity? This is the fundamental paradox of enterprise that Agile, and every “new big thing” is meant to resolve: complexity is added to reduce risk, but the complexity itself creates risk, and makes the risk harder to name, harder to spot, and harder to recover from if it is realised.

We have the 5 whys, the blameless retrospectives. And whilst the intention is sound – blame the system, not the individual – the solution is often to add new trinkets around the edges of the system. And reinforce that the system is the only way. They mistakenly put process at the centre, and ask the people to support the process, whereas the process should support the people.

But of course, this creates the shadow IT departments and the “non-compliant” centres. One place I worked had a strict policy that no one has admin rights because that fixed a problem lost to the mists of time. I understand the benefit of the policy, but at the time all our developers were working on IIS and couldn’t develop the websites we were paid for without having admin access on our machines. And so we had dispensations and workarounds until ASP.net core fixed the underlying issue of requiring admin access to serve web content.

Some companies stack procedure on top of procedure because the project is the centre of their universe rather than business value. And every company is in danger of falling into that trap as they treat risk management as risk elimination, instead of mitigation or recovery. They condemn every project to the tarpit of success, sinking below the crushing weight of process where sunlight cannot penetrate.

You will never have a process that prevents the next failure. You need a process to detect and recover, and you need to remove 99% of the “just in case” procedures from your process.

You don’t need to double-check the prime DVD copy before sending it for distribution, because no one has a DVD drive on their servers. You don’t need to change the admin passwords when someone leaves because there should not be an admin account that isn’t attached to a user. Eliminate the process, because every process you have is a process someone can forget. The best process is one you don’t need because the risk it mitigates cannot be represented by the system.

Either accept that you are not the centre of the universe and rewrite your rules to understand that you merely orbit the sun like so many others, or live out the fantasy that you are special, that your problems are unique, and add deferents on top of epicycles when the universe tries to disabuse you of that notion.

You can’t control the universe, only how you react to it. So don’t use JIRA to enforce pi to be 3.2.

Journaling for technologists

I encountered a question online recently about building context quickly, and whilst I thought of the bootstrapping post I made before, I also wanted to take a chance to explore how that plays into continuous practice. I started journaling as a researcher to remind me of all the dead ends and configurations I’d tried. Although I’ve not been entirely consistent in journalling (or sometimes blogging) each day and each new discovery, I think it’s a good practice for technologists to develop. Think out loud, even if it’s to yourself.

When building context on a new project, for example, I often find it useful, as part of discovery, to note what the client (or in very rare circumstances the written requirements) says it does, as well as what it actually does.

And always, always, journal everything. How to get it running locally, how to release, who knows what, who has the admin rights,… Anything that takes more than 2 minutes to figure out.

Sometimes that journal will take the form of shared content to help the next person join the project (and like all good scouts we should leave a place better than we found it), but the important bit is to write it for yourself. 80% of the time future you won’t need it, but that 20% makes the time absolutely worth it.

In the beginning… was the conversation

The history of computing has been an evolution of conversation between us and machines. Single-purpose machines were built to be talked about, not communicated with. Even a hoax like the mechanical Turk.

But then the punched cards of the loom and Ada Lovelace’s ideas of a general-purpose computer started to become real, and we needed a language. At first, it was a language based on mathematics, the language of log tables, the language of angles, and then the language of letter frequencies. But soon we got more advanced, we got to the language of logic. Decision making, and computers, were open to working on anything.

And that’s where software engineers, programmers and developers enter the story. The medium of communication between the world of humans and the world of logic (because there has been no greater fallacy in human endeavour than to assume there is a complete overlap between the two, despite how many Nobel prize winners in economics have tried to prove otherwise).

The story of computers since then has been the story of building layers upon layers on top of machine logic to bring machines closer to humans. The command line, high-level languages (when C was considered a high-level language), graphical user interfaces, and the emerging field of voice and text conversation epitomised by Alexa and but Google and Facebook bots, but also the first line at the call centre, the first responder on live chat.

There has always been a conversation, and there’s always been a mismatch between our words and our intent.

As developers, our job is to derive intent from those words and the actions. Traditionally, offline in workshops and focus groups, but increasingly in real-time, teaching the machines to understand users and adapting conversions, websites, and apps, as we detect errors and confusion, as our data tells us users are unable to complete their goals.

There has always been a conversation. But this new conversation is with the users, and the machines, and the stakeholders, in real-time. With incomplete and biased data.

Be forgiving in your conversations. No one knows what this global conversation is like. But we know from social media and the news that not everyone in the conservation is pursuing truth and simplicity. But that is the only logical route, and behind all the facades is whatever truth you’ve built into the system. Are they the truths you can stand behind? Are those the truths that help and empower people?

Communicate your truths clearly and without favour, because this conversation is never pure or simple, and we have unearned power to control and shape that conversation. I don’t know if anyone would have chosen to give power to the developers and make us responsible for the truth, but in an information economy, that’s where we are. All these conversations are our conversations.

Tech is not neutral.

Speak your truth and recognise your responsibility.

Further reading

Neal Stephenson – in the beginning… was the command line

Conversation is the command line of tomorrow

How to make mistakes

Nobody is perfect. If you tell me you’ve never made a mistake in your career, not only will I not believe you, I won’t be able to trust you.

In a safe and functioning team, we can accept our mistakes, discuss them openly, learn the lessons, improve our system, and move on.

Without that, we’re trapped not by our mistakes but by our fear of making them and our fear of the consequences when our mistakes are inevitably surfaced. And “our” mistakes quickly become “your” mistake. Not a failure of the team, not a failure of the system – the processes, the automation, the communication, and the rest – but a fault of the most vulnerable and the most exposed. “It’s always the intern.”

No one makes the intern pilot responsible for a jet plane. They start in small props and in simulators. They are supported and mentored. They are given space to practice, and to fail. Because what really makes the difference isn’t avoiding mistakes, it’s recovering from them.

Build for failure

Test-first, red-green-refactor: every new feature starts with a failing test which we then fix. Make failures part of the process, small and recoverable. Expected.

Build for backup, rollback, recovery. Test for failure continually. And recover automatically if you can.

Plan for failure

Be agile. Try new things for a day, or a week, or a year. Decide in advance what success and failure look like and monitor them. Adjust course if things aren’t working. Celebrate what you learn either way..

Move fast and bend stuff, don’t break it. Do and succeed, or do, fail and recover.

The longer a decision takes, and the more you invest in it, the harder it will be to change it, and the bigger a mistake it will become. So make it quick and don’t make it permanent.

Lead for failure

A leader needs to set a clear direction. To resolve conflicts by stating strong opinions and sticking to them. It makes conflicts easier to resolve. Until those opinions need to change. Maybe the mistake is yours, maybe it’s wrong because other things have moved on. It’s ok to change your mind, so long as you can explain it.

And then go read about why Cognitive Dissonance makes all of this so hard, by reading “Mistakes Were Made”

If you’re not living on the edge, you take up too much room

.net is a battleship, and it’s taken a long time to change everything to core, and figure out what the Framework/Core future is. In the meantime, you may have found your project crushed in the path as new APIs change or old technology gets deprecated.

Ask Java developers about where Oracle is taking their language and you’ll hear a similar story. The future is different. Maybe better, but in some places definitely worse.

Change is inevitable. That’s why our industry has embraced agile, so we’re ready to change on weekly or monthly cycles, not yearly ones. The longer it takes to make the decision to change, the more baggage you have, and the harder that change will be. That’s why Jez Humble recommends “if it hurts, do it more often”. That change may come from the business, from the competition, from the platform, or from the environment. How did you deal with Heartbleed, or Spectre? How many of your customers are still vulnerable, and are reducing herd immunity for the rest? Are other companies carrying your baggage without knowing it? Are you the reason that IE6 VMs are still a thing?

The bleeding edge is painful. .net Core broke things, Angular 2 broke things, Python 3 broke things, Edge broke things. But not keeping up breaks more.

How do you keep your tech contemporary?

When the old guard are not heroes

When I started programming, there were some key figures identified as the experts. They wrote the books that laid out how my code should look, how I should behave, and what the purpose of a software developer was. And there’s a new guard, also steeped in that mythos, publishing their hot takes and simple books.

I looked to my peers and seniors for guidance, and they looked to the agile manifesto, the gang of four and others. Those who packaged up existing ideas and best practices, set against a growing trend of Taylorism, and wrapped themselves in a punk ethos fighting the power, wielding keyboards like axes.

And there was value in what they were selling, small pieces that fed my hunger for “something better”.

But for all the bravado, the facade of Woody Guthrie and Bruce Springsteen they presented, deep down they were the establishment, tweaking the system slightly for their own benefit, re-engaging the bro culture of the “frontier spirit” of the early internet. Rules and processes are dictated by “the man” to restrict our freedom to innovate, and to work in the best way.

Companies formed and rose on the counter-culture ethic, buoyed by the vacuum left as union power fell, blinding us all that tech was the new utopia, and it brought a meritocracy that would bring equality.

And the gods saw agile, and it was good.

We moved fast and broke stuff, re-wrote the rules. The industry didn’t take time to consider why those rules existed, just that they were roadblocks.

So when women and people of colour, and anyone else who wasn’t a bro, stepped into the arena, they were vulnerable. Because there were still rules, just no process. And the rules aren’t made for them. And the gatekeepers, clear in their self-image that they weren’t racist, or sexist, and only discriminated on merit, could not comprehend that processes and written rules exist to limit privilege, because equality means nothing without equity.

They asked all of us, the squirrels, the rhinos, the fish, and the peacocks, to climb trees and collect nuts to demonstrate our worthiness to join their club, without regard for our skills or our backgrounds. They bully, they fight, they protect their own, because rules weigh us down, man.

---

• People over process, so long as the people are the right people.
• Working software, so long as it works for me. Anything else is externalities and not my problem.
• Collaboration over contracts, we’re all on the same team, so forgive me as I forgive you.
• Respond to change over following a plan, but is there a vision that informs what changes are acceptable?

---

Agile still has its place, but what would it look like if the manifesto and the guiding principles had been laid out by a more representative group? The antecedents, stretching right back through NASA, Bletchly Park and The Difference Engine, remain the same, and the key lessons are independent of those teachers, but what did we miss by not having others at the table?

• What does process that protects people look like? Process that keeps people safe and secure to deliver the best solution?
• What does software that works for everyone look like? Beyond documentation on accessibility and anti-discrimination.
• What does proper, honest, compassionate collaboration look like? Within teams, between teams and across disciplines?
• What does meaningful, people-centered change look like?

---

Software is too important to be left to the swamp that is this libertarian mess. The current ways are driving great developers out of tech, and users are not fairly represented. Some of the old guard recognise this, but many don’t, and they have plenty of followers who still believe the meritocracy myth.

There is no level playing field in tech.

Not everyone has spare time for technical exams outside work hours. Not everyone is comfortable pair programming. Not everyone can follow a spoken stand-up. Not everyone feels safe to bring their full self to work. Everyone is not treated the same. Not everyone can have alcohol and pizza. Not everyone has the experience of standing up to authority figures. Not everyone can be themselves without being judged or mocked by the cis straight white male gatekeepers, and their supporters.

There’s precious few unions and the biggest tech companies are all struggling with human rights, fair employment and treating their users fairly. And it’s not just them.

Unlike the 20th century, it’s not a battle between management and worker. Agile has cast management and worker in the same side, but some workers are far more equal than others.

People are not supported by the process. Software doesn’t work for everyone. Collaboration is limited because contracts don’t protect and promote safety. The plan for meritocracy to end discrimination isn’t working. How are you going to change it?

Becoming a technical lead: Trust your team

When I first became a technical lead, one of my biggest challenges was giving up control. It wasn’t a matter of not trusting my team technically but I didn’t want to overload them, so I took on all the planning, all the customer interaction, and all the other jobs that I felt had distracted me from the job of writing code.

I was wrong.

I needed to let my team in, because they had ideas that I didn’t, they had capacity whereas I was a bottleneck. I was holding up my team.

But I needed to be sure that coding standards were followed and quality was maintained. You know who else cares about that? My team. I made it their responsibility to ensure that the code met standards, and pushed out code reviews. I put internal documents at the same level as code. Anyone can edit, but everyone can review, or veto.

I asked for honest feedback in retrospectives, and implemented changes, so they could trust that I was looking out for them, and I set them goals to improve, alongside the feature work : “can we release twice a week?” (challenge – automate the pain points); “can we halve the bugs found in testing” (challenge – better unit tests)

And the more I trusted them, with support, and understanding their limits, the better they performed. I’ve also worked with managers who don’t trust their teams. Those teams don’t work so well.

If you don’t trust your team, they’re not your team. They’re just people who work beside you. What are you doing today to foster trust?