Mise en place architecture

I love working with smart people. I learn a lot and it gives me energy.

I hate working with smart people who aren’t motivated. They’ll either get sloppy, get a new job, or get creative with their code design. The kind of creativity that makes you curse when you’re debugging a production incident at 3am.

The best creativity happens in a constrained environment, which also happens to make the easiest debugging.

Sure, we could let the developers figure out the best way to do something for every component, and there’s sometimes a benefit, but for every hour they’re spending figuring out how to solve a problem that didn’t need to be solved, or figuring out an unusual design, or evaluating a logging package, or writing boilerplate, there’s an hour not delivering value.

When an architecture is designed to put everything right where it should be, where decisions that have already been made are baked into the code and the tools, where a developer doesn’t have to think about how to structure their solution, the code is easier to write, easier to review and easier to debug.

Chefs like to follow mise en place. Everything in its right place. Before preparing a dish, prepare the workspace, the knives, and the food. Everything you need for the task and nothing you don’t. Everything is in a predictable place. Because then you can concentrate on the dish, instead of the kitchen. Good preparation helps every task fall into the pit of success, and makes it easier to recover if something goes wrong.

The more steps you have to complete a subtask, the easier it is to make mistakes. You might forget what the previous step was, you might walk to the fridge and then have to return to your workspace to remember what you need. Multitasking adds friction and adds opportunities for error.

That’s why we want encapsulated classes and single responsibility. One change updates one file, as far as possible. Although one feature may cover many changes in order to make it possible. Isolate your code from the data store, isolate the public API from your code, parse don’t validate.

Keep smart people working on solving new problems, and keep them consistent, because that’s the way to get the best from the team at all times, especially when you have a Priority 1 to update a logging framework at 3am.

Advertisement

If it hurts, stop doing it : the wrong tool

There’s a theory under agile, lean and similar methodologies that if something is painful, you should do more of it. If releases are infrequent and error-prone and once a quarter, do them 10 times a day and they’ll get easier.

Same idea with performance reviews, customer feedback, and security audits. If it’s a good idea and it’s painful, practice it and refine it until it’s natural and mostly painless. And the pain that’s left is manageable. Roll back the release, and have another catch-up tomorrow once tempers have dampened.

I’ve seen people make the mistake of assuming that it should apply to everything. Every pain point is a gathering, a thing to be controlled, minimised and made less painful, by repeating it over and over again. After all, if it works over there, it should also work over here.

But not all pain is equal.

Remember, focusing on doing something more means that we deal with the pain by eliminating it. We automate releases so we can throw out that painful checklist. We give small, actionable feedback at the time, rather than a sucker punch that brews for months until it’s released in the appraisal.

But don’t mistake pain for discomfort. Making big improvements will mean transitions that are scary and uncomfortable. And what’s painful for someone else might not be painful for you. That doesn’t mean the pain isn’t real and it still needs to be dealt with.

Here’s a few things that are painful because you shouldn’t be doing them. These are the pebbles in your shoes that you need to remove.

It’s painful because it was never built for that

I know there’s a lot of hate for JIRA. It’s the tool of choice for “Safe Agile” enterprises. And it gets a bad rep for being an overcomplicated monstrosity.

I was a JIRA admin once, bringing the tool into our enterprise. There were things I didn’t like about it on a technical level, but the central tool, with the defaults, isn’t terrible. But it’s so customisable, that you can codify any corporate process you like. And when it causes frustration, people blame the tool, not the admin. When the tool is the process, it makes concrete what people could fudge, and suddenly everyone has to work the way of the manager who needs to show their impact.

Start with the people. Don’t build a process around what people should do. Find out what they actually do and build from there. Some of it might be wrong, but find out why, and help them fall into the pit of success.

Don’t blame the tool for a broken process.

Stop working when you’re ill

Want to stop people from working when they’re ill?

1. Pay them.
2. Warn them if you find they are working. The culture should expect everyone to recover before working.
3. Encourage everyone to have an illness plan – where can anyone find what work needs to be covered, and who’s best placed to cover it.

See also: working when children or parents are ill.

Thinking outside the box: the difference between constraints and perceptions

You need an app.

The constraint is that it has to be accessible.
The perception is that it has to be screen reader friendly.

But… That excludes deaf people who miss the audio cues in the app
… That misses the option to add voice control. If it’s a booking app, why not talk through the booking?

---

The constraint is that users need to be notified (it’s a legal requirement)
The perception is that only one channel is acceptable (must be post, must be a tracked email)

But… each user has their own preference, or accessibility baseline. Post only may help, or hinder, victims of abuse. I can’t guarantee that you’ve read a letter, but I can guarantee you’ve hit the “I read this” button.

---

The constraint is that you need to mark and control personal and sensitive data because of GDPR, and you need informed consent to do that

The perception is that making users click “I agree” is informed consent
The perception is that gaining consent absolves you of responsibility to mark and control personal data “because they agreed to our terms and conditions”
The perception is that location data isn’t sensitive, even though Google knows your home and work address, and the address of the abortion clinic, the LGBT nightclub, the local mosque, the local love hotel

Processes upon processes: the JIRA trap

It is fashionable to hate on JIRA for software developers. Project Management made spaghetti. It has its faults, but the biggest issue is what it allows. It’s not opinionated, so any user can define any process to follow. It’s a perfect machine for generating red tape, or paper clips.

Because every time something goes wrong, the natural instinct is to add a new process, a new safety net, to make sure it doesn’t happen again [see Agile is Dead blog post]. And once added, they’re very difficult to remove.

So we get processes upon processes, the simple rhythm of a ticket lifecycle or of a sprint adorned with Deferents and Epicycles as we try and tame ever-increasing complexity with more text boxes and more statuses.

Complexity cannot fix complexity. But who has time for simplicity? This is the fundamental paradox of enterprise that Agile, and every “new big thing” is meant to resolve: complexity is added to reduce risk, but the complexity itself creates risk, and makes the risk harder to name, harder to spot, and harder to recover from if it is realised.

We have the 5 whys, the blameless retrospectives. And whilst the intention is sound – blame the system, not the individual – the solution is often to add new trinkets around the edges of the system. And reinforce that the system is the only way. They mistakenly put process at the centre, and ask the people to support the process, whereas the process should support the people.

But of course, this creates the shadow IT departments and the “non-compliant” centres. One place I worked had a strict policy that no one has admin rights because that fixed a problem lost to the mists of time. I understand the benefit of the policy, but at the time all our developers were working on IIS and couldn’t develop the websites we were paid for without having admin access on our machines. And so we had dispensations and workarounds until ASP.net core fixed the underlying issue of requiring admin access to serve web content.

Some companies stack procedure on top of procedure because the project is the centre of their universe rather than business value. And every company is in danger of falling into that trap as they treat risk management as risk elimination, instead of mitigation or recovery. They condemn every project to the tarpit of success, sinking below the crushing weight of process where sunlight cannot penetrate.

You will never have a process that prevents the next failure. You need a process to detect and recover, and you need to remove 99% of the “just in case” procedures from your process.

You don’t need to double-check the prime DVD copy before sending it for distribution, because no one has a DVD drive on their servers. You don’t need to change the admin passwords when someone leaves because there should not be an admin account that isn’t attached to a user. Eliminate the process, because every process you have is a process someone can forget. The best process is one you don’t need because the risk it mitigates cannot be represented by the system.

Either accept that you are not the centre of the universe and rewrite your rules to understand that you merely orbit the sun like so many others, or live out the fantasy that you are special, that your problems are unique, and add deferents on top of epicycles when the universe tries to disabuse you of that notion.

You can’t control the universe, only how you react to it. So don’t use JIRA to enforce pi to be 3.2.

Journaling for technologists

I encountered a question online recently about building context quickly, and whilst I thought of the bootstrapping post I made before, I also wanted to take a chance to explore how that plays into continuous practice. I started journaling as a researcher to remind me of all the dead ends and configurations I’d tried. Although I’ve not been entirely consistent in journalling (or sometimes blogging) each day and each new discovery, I think it’s a good practice for technologists to develop. Think out loud, even if it’s to yourself.

When building context on a new project, for example, I often find it useful, as part of discovery, to note what the client (or in very rare circumstances the written requirements) says it does, as well as what it actually does.

And always, always, journal everything. How to get it running locally, how to release, who knows what, who has the admin rights,… Anything that takes more than 2 minutes to figure out.

Sometimes that journal will take the form of shared content to help the next person join the project (and like all good scouts we should leave a place better than we found it), but the important bit is to write it for yourself. 80% of the time future you won’t need it, but that 20% makes the time absolutely worth it.

Don’t be a speed shaver

CW: description of blood.

There’s a deadline looming, and your estimates show the project coming in late. Very late.

You need to claw it back. Drop the ballast and become a lean mean coding machine.

---

You have a plane to catch. You look at your watch and you’re going to be late. Very late.

You need to cut corners. Shave as you shower. No time for shaving foam in the mirror.

The first cut is annoying. Your cheek stings in the heat, but you keep going.

A second cut, right on the chin. That one hurt. And it’s dripping rather than seeping.

But you need to finish. You don’t have time for this. Keep going.

Missed a bit, back under the chin.

4 cuts. 3 seeping and one dripping.

And the heat keeps them open.

So you patch it over with tissue once you’re out of the shower, try and clean it up to see what you’ve missed. And then you have to go back in and finish the job with cold skin, so more nicks, and more time is taken.

Aftershave to help seal and clean. That’s sore. That’ll sting until you get on the plane.

More tissues, and then plasters. Trying to get dressed without getting any blood stains, so that’s slower too.

---

Every shortcut you take has consequences. Technical debt pays interest, and sometimes it’s eye-watering. The debt exceeds the functionality and the project takes longer, and even longer than that. And so you have to make more cuts and ensure more pain. And still, your clients can see it’s held together with plasters that are already starting to peel.

Don’t be a speed shaver. You don’t have enough time to do it quickly.

How to make mistakes

Nobody is perfect. If you tell me you’ve never made a mistake in your career, not only will I not believe you, I won’t be able to trust you.

In a safe and functioning team, we can accept our mistakes, discuss them openly, learn the lessons, improve our system, and move on.

Without that, we’re trapped not by our mistakes but by our fear of making them and our fear of the consequences when our mistakes are inevitably surfaced. And “our” mistakes quickly become “your” mistake. Not a failure of the team, not a failure of the system – the processes, the automation, the communication, and the rest – but a fault of the most vulnerable and the most exposed. “It’s always the intern.”

No one makes the intern pilot responsible for a jet plane. They start in small props and in simulators. They are supported and mentored. They are given space to practice, and to fail. Because what really makes the difference isn’t avoiding mistakes, it’s recovering from them.

Build for failure

Test-first, red-green-refactor: every new feature starts with a failing test which we then fix. Make failures part of the process, small and recoverable. Expected.

Build for backup, rollback, recovery. Test for failure continually. And recover automatically if you can.

Plan for failure

Be agile. Try new things for a day, or a week, or a year. Decide in advance what success and failure look like and monitor them. Adjust course if things aren’t working. Celebrate what you learn either way..

Move fast and bend stuff, don’t break it. Do and succeed, or do, fail and recover.

The longer a decision takes, and the more you invest in it, the harder it will be to change it, and the bigger a mistake it will become. So make it quick and don’t make it permanent.

Lead for failure

A leader needs to set a clear direction. To resolve conflicts by stating strong opinions and sticking to them. It makes conflicts easier to resolve. Until those opinions need to change. Maybe the mistake is yours, maybe it’s wrong because other things have moved on. It’s ok to change your mind, so long as you can explain it.

And then go read about why Cognitive Dissonance makes all of this so hard, by reading “Mistakes Were Made”

return Task vs return await Task

The asychronous pattern in C# is great for simplifying the boilerplate of dealing with threads. It makes things look simple, but as with all abstractions, there are leaks.

In most cases, when an async method calls another and there’s no chain (e.g. public method calling a private method directly) return Task is fine, and avoids the overhead of the await state machine for that method.

BUT never do inside a using, or other scoped construct where the object required by the inner call requires context to successfully complete.

In the following example, the first variation, without the await, returns a database call that escapes from the context of the database connection at the end of the method, so when the call is made, the database connection has been closed. You need the await in the second example to postpone the disposal of the database connection until the GetUserAsync call returns.

In this example it’s fairly obvious, but the same pattern can cause threads to unravel far from where this fray was introduced.

class UserDetailsRepository
{
    public UserDetailsRepository(IDbConnectionFactory dbConnectionFactory)
    {
        _dbConnectionFactory = dbConnectionFactory;
    }

    public Task<UserDetailsResult> GetUserDetails(string userId, CancellationToken cancellationToken = default)
    {
        using (var dbConnection = _dbConnectionFactory.CreateConnection())
        {
            return GetUserAsync(userId, dbConnection, cancellationToken);
        }
    }

    public async Task<UserDetailsResult> GetUserDetailsAsync(string userId, CancellationToken cancellationToken = default)
    {
        using (var dbConnection = _dbConnectionFactory.CreateConnection())
        {
            return await GetUserAsync(userId, dbConnection,  cancellationToken);
        }
    }
}

Get uncomfortable

This blog is about technical topics and being a technical lead. Understanding architecture is part of it, but if you don’t reflect on understanding people then you will never be a leader. I don’t have all the answers but it starts with accepting you can learn from others, and some of what they share will not fit with your current world view. Be ready to tear down and rebuild some foundations if you’re a technical person willing to become a leader. In the spirit of learning I’ve included the original tweets at the bottom for context so you can see the rest of the conversation.

Data needs the right questions

I trust science, so data beats anecdotes, but once I started to listen to people’s stories, I realized the people collecting the data weren’t asking important questions. “How many women applied for this job?” “How many targets of violent crime are gay?” “Are black men targeted by stop and search?” Some of these questions now have better answers, but there’s still a lot of questions that don’t even occur to people to ask.

I learned a lot from my wife in that respect – the importance of qualitative research to make sure you’re asking the right questions. But qualitative is “soft science” so isn’t as well respected, despite being fundamental to getting it right. Sound familiar?

And then you have to ask why the data isn’t collected? Is it a blind spot, or do people earnestly not want to know because then they’d have to face up to uncomfortable truths that their image of themselves does not match how others see them? Our egos are fragile, which is why I have to work hard and compassionately with new developers to understand ego-less code and collective ownership. Vulnerability is hard, especially for men in tech, and that manifests itself in many defensive micro-agressions.

I’m not going to talk about toxic masculinity here, but please go watch The Mask You Live In to hear a US perspective on how “manning up” is creating a toxic environment.

Code needs the right questions

Yeah, code is for solving the problems you know about, but how do you solve the problems you don’t know about?

If someone calls you a snowflake, or an sjw, just for asking a question, there’s a very important reason they don’t want you to ask that question : they’re scared of the answer.

Let’s be clear, science and data matter, otherwise the opinion of some white guy who can’t keep his job at Google is worth as much as someone who has collected, reviewed and summarised all the data. But we all need to be sure that the right data is collected and the right questions are asked.

To be honest, I started down this path because I saw my friends were hurting, and they helped me understand homophobia, and then I started seeing where everyone else was disadvantaged. Having a friend to guide you is the best way to open your eyes.

Why should you change? Because “we’ve always done it this way” is the worst justification for anything. Because if you find out something is broken, it should make you uncomfortable. And if you think nothing is broken then you shouldn’t be writing software and fixing problems.

Make space

This isn’t about feelings, or political correctness or any of that. This is about you doing your job, understanding the domain that the technology you create sits within. It’s about bringing your full self to work, and making sure everyone else on the team has that opportunity too. And if they don’t want to talk about what they did at the weekend, that’s their choice too.

If you can’t make space to accept that other points of view are valid, that technology mediates access and knowledge, and your code will directly impact someone’s ability to access that, you should not be in this industry. Make space for someone who gives a shit about the users, and the wider community affected by every decision you make in every line of code you write and review, and every interaction associated with that code.

Is it exhausting? It can be. Especially at first. But you know what’s really exhausting? Fighting… technology…

Every

Step

…

Of

…

The

….

…

…

Way.

Don’t accept the status quo

Don’t be the developer that makes their colleagues rage quit. Or makes the users curse their every day stuck because you didn’t ask the right questions.

Did you test your facial recognition on black faces?

Can blind people order pizza on your website?

Is every woman on your team made of regular polygons and has regular periods?

Question everything. The truth is out there if you care to look. Other people should not be alien to you.

---

I just talked about @jsconfeu. The value of the mixed methods (quantitative and qualitative) research approach to collecting and evaluating data to guide business decisions. It takes both to ensure that products and services meet customers and client needs #causeascene https://t.co/Zgxl2Hjmss

— Kim Crayton 🏢 💻🎙#causeascene (@KimCrayton1) June 5, 2018

Learned a lot from my wife in that respect – the importance of qualitative research to make sure you're asking the right questions. But qualitative is "soft science" so isn't as well respected, despite being fundamental to getting it right. Sound familiar?

— craignicol (@craignicol) June 5, 2018

I trust science, so data > anecdotes, but once I started to listen to people's stories, I realized the people collecting the data weren't asking important questions. "How many women applied?" "How many targets of violent crime are gay?" "Are black men targeted by stop and search?

— craignicol (@craignicol) June 5, 2018

Yeah, code is for solving the problems you know about, but how do you solve the problems you don't know about?

— craignicol (@craignicol) June 5, 2018

If someone calls you a snowflake, or an sjw, just for asking a question, there's a very important reason they don't want you to ask that question : they're scared of the answer.

— craignicol (@craignicol) June 5, 2018

Let's be clear, science and data matter, otherwise the opinion of some white guy who can't keep his job at Google is worth as much as someone who can raise a massive crowd. But we all need to be sure that the right data is collected and the right questions are asked.

— craignicol (@craignicol) June 5, 2018

To be honest, I started down this path because I saw my friends were hurting, and they helped me understand homophobia, and then I started seeing where everyone else was disadvantaged. Having a friend to guide you is the best way to open your eyes.

— craignicol (@craignicol) June 5, 2018