Categories
development

Blank slate

I’m a fan of the original Sherlock Holmes books, and there are a few things in them that talk about an enlightenment way of thinking that’s useful for a developer when gathering requirements (whether tomes, user stories, backlog ideas,etc), writing code or approaching debugging.

Today I want to talk a bit about A Study in Scarlet, and the early conversations between Holmes and Watson before their first case together, because they capture the essence of his enlightenment thinking very well.

Ignore Irrelevant detail

Watson tells Holmes that the sun rotates around the earth, as he is surprised that an educated man does not know this. In reply, Holmes says,
“That’s very interesting, I will now do my best to forget it”

It’s not an important factor in any of his cases, it doesn’t affect his work, so it’s not a fact he needs to record. He accepts it but has no way to act on that information, so he dismisses it.

It’s something that is often hard to do in requirements gathering and may need to be done retrospectively after you find out whether Andy from finance’s love for Richard Branson is just infatuation, or it means that every call to action across your site will need to be red.

It’s also something to apply more generally, choose what to ignore, don’t learn every new JS framework. Don’t expect to be an expert in everything. Be content with being T-shaped, and become an expert in solving problems and focusing on the right details.

The Book of Life

” it attempted to show how much an observant man might learn by an accurate and systematic examination of all that came in his way.”

The truth is never simple. Whatever you build will be part of an ecosystem comprised of other software, of manual processes, of fallible humans, and the winds of fate.

Gather whatever information you can, in whatever detail you can. Organise it and understand the bigger context.

The Phoenix Project has a great understanding of this in the discussion of the SOX-404 audit, where the IT department are busy worrying about controls on software without understanding the manual processes surrounding the software and how that fits into the compliance picture.

Speculation

On their journey to their first case together, Watson is keen to speculate about the motives and means that led to the crime, extrapolating from what little information they got from the initial introduction.

Holmes quickly and sternly cut Watson off from that line of thinking. He was clear that he would also base hypotheses on the causes and consequences once he had evidence before him to narrow down the possibilities, removing the impossible.

Speculate make hypotheses on why the system failed, on what will improve sales, on what the performance bottleneck is really going to be. Without data or a way to test them, they are useless for reaching understanding. Sometimes the beauty of a forest can only be appreciated by the birds.

Remove your preconceptions and bias from judgement by understanding what they are. Follow the data instead of your gut.

Categories
code cosmosdb data development

Cosmosdb and Heterogeneous data

A selection of different watches. They all tell the time, but some are analogues, some are digital, some are branded, and some are not.
Same, but different

CosmosDb, in common with other NoSQL databases, is schema-free. In other words, it doesn’t validate incoming data by default. This is a feature, not a bug. But it’s a dramatic change in thinking, akin to moving to a dynamically typed language from a statically typed one (and not, as it might first appear, moving from a strongly typed to a weakly typed one).

For those of us coming from a SQL or OO background, it’s tempting to use objects, possibly nested, to represent and validate the data, and hence encourage all the data within a collection to have the same structure (give or take some optional fields). This works, but it doesn’t provide all the benefits of moving away from a structured database. And it inherits from classic ORMs the migration problem when the objects and schema need to change. It can very easily lead to a fragile big-bang deployment.

For those of us used to dynamic languages and are comfortable with Python’s duck typing or the optional-by-default sparse mapping required to use continuously-versioned JSON-based RESTful services, there’s an obvious alternative. Be generous in what you accept.

If I have a smart home, packed with sensors, I could create a subset of core data with time, sensor identifier and a warning flag. So long as the website knows if that identifier is a smoke alarm or a thermostat, it can alert the user appropriately. But on top of that, the smoke alarm can store particle count, battery level, mains power status, a flag for test mode enabled, and the thermostat can have a temperature value, current programme state, boiler status, etc, both tied into the same stream.

Why would I want to do this?

Versioning

Have historic and current data from a device/user in one place, recorded accurately as how it was delivered (so that you can tweak the algorithm to fix that timedrift bug) rather than having to reformat all your historical data when you know only a small subset will ever be read again.

Data siblings

Take all the similar data together for unified analysis – such as multiple thermostat models with the same base properties but different configurations. This allows you to generate a temperature trend across devices, even as the sensors change, if sensors are all from different manufacturers, and across anything with a temperature sensor.

Co-location

If you’re making good use of cosmosdb partitions you may want to keep certain data within a partition to optimise queries. For example, a customer, all of their devices, and aggregated summaries of their activity. You can do this by partitioning on the customer id, and collecting the different types of data into one collection.

Conclusion

NoSQL is not 3NF, so throw put those textbooks and start thinking of data as more dynamic and freeform. You can still enforce structure if you want to, but think about if you’re causing yourself pain further down the road.

Check out @craignicol’s Tweet: https://twitter.com/craignicol/status/1122224379658633217?s=09

Categories
development leadership

Unsuccessful Teams

Are you making space?

Game Outcomes

In a previous post, I looked at how to create successful teams, and looked at the Game Outcomes project as a useful formulation.

Some of these points are about avoiding negatives and that’s what I want to focus on here.

The most important indicators for success from the Game Outcomes project are:

  1. Great game development teams have a clear, shared vision of the game design and the development plan and an infectious enthusiasm for that vision.
  2. Great game development teams carefully manage the risks to the design vision and the development plan.
  3. Members of great game development teams buy into the decisions that are made.
  4. Great game development teams avoid crunch (overtime).
  5. Great gamedev teams build an environment where it’s safe to take a risk and stick your neck out to say what needs to be said.
  6. Great gamedev teams do everything they can to minimize turnover and avoid changing the team composition except for growing it when needed. This includes avoiding disruptive re-organizations as much as possible.
  7. Great gamedev teams resolve interpersonal conflicts swiftly and professionally.
  8. Great gamedev teams have a clearly-defined mission statement and/or set of values, which they genuinely buy into and believe in. This matters FAR more than you might think.
  9. Great gamedev teams keep the feedback loop going strong. No one should go too long without receiving feedback on their work.
  10. Great gamedev teams celebrate novel ideas, even if they don’t achieve their intended result. All team members need the freedom to fail, especially creative ones.

Confounding factors

Overtime and crunch

Deadlines are good, to a point. It helps focus. With a clear goal and a timebox it’s much easier to discard sandbags and maintain motivation. Many personal productivity schemes rely on setting yourself deadlines.

When those deadlines are too restrictive however the product will suffer. Teams will work late and produce lower quality work. They will cut corners. If time is fixed then either scope or quality or both need to be cut.

Unplanned or persistent overtime is a critical bug and needs to be prioritized as such. There’s no such thing as completely bug-free, but you should always be aiming for zero.

Mono-cultures and silos

Cross-functional teams make better decisions faster. That’s a lesson I learned the hard way. The consumer of the API should sit in the same room as the producer. Even better, at the same desk, or in the same chair.

It’s not just technology silos that cause problems. If your team is a straight cis able-bodied English-speaking white male silo, or functionally equivalent to one, then it will fail at every interface with someone outside that group. Widening your team doesn’t stop those failures, but if you manage the team properly, the failures are fixed within the team (with the goal of fixing them before the code is written) rather than experienced by consumers.

Diverse teams are also more creative.

Inter-personal conflict

Teams don’t shy away from conflict. Open discussion, even a heated one, clears the air rather than letting micro-vexations and micro-aggressions become the norm and harm the team, one papercut at a time.

Successful teams solve disagreements in the open. People first, then process after, to remind everyone of decisions made.

Punishment driven development

Feedback is great. Tracking progress is useful. But please be sure you are tracking the right thing.

I can’t say this any better than Louise Elliot, who talks about all the ways measuring the wrong thing can seriously affect a team. Video is below, but you can also listen to her talking about Punishment Driven Development on the .Net Rocks podcast, if that’s more your style.

Are you unsuccessful?

What dysfunctions have you seen in your teams now or in the past? How have you fixed them, or how will you?

Categories
development google leadership

Successful teams

Successful teams deliver successful projects. As a lead, how do you build a successful team?

There are many factors to build a successful team, but the foundation of them all is safety. Can problems be discussed openly? Does everyone trust everyone else? And once you have that, the team can build. Build diversity, build towards a common goal, and build something that matters.

Successful Google team

Google defines successful teams according to its research at https://rework.withgoogle.com/blog/five-keys-to-a-successful-google-team/

Psychological safety: Can we take risks on this team without feeling insecure or embarrassed?
Dependability: Can we count on each other to do high quality work on time?
Structure & clarity: Are goals, roles, and execution plans on our team clear?
Meaning of work: Are we working on something that is personally important for each of us?
Impact of work: Do we fundamentally believe that the work we’re doing matters?

I accept, given multiple ongoing accusations against them about defending toxic managers and culture, that Google may not be living these values. However, these are clear statements that are supported by other studies such as The Game Outcomes project.

Penguins at Edinburgh Zoo

So how do we build a team like that?

Number one thing, and the only way I’ve found success, is to empower the team and everyone within it to make changes. Without that ownership, nothing matters.

Once you have that, you as the leader have to own the rest. Delegate where you need to, but own your team’s safety, support, direction, purpose and motivation.

Safety

Are you free to take risks and try something new?

Not everything you do will be a success, so do you celebrate knowledge and learning as a goal? Yes, that cost us time and money, but we learned not to do that again

Are team members supported? When someone mansplains your tech lead, do you correct them, and ensure her voice is heard? When a deaf developer joins the team, do you ask whether they prefer lip reading or sign, and help the team adapt appropriately? Do you recognise colour? Do you use preferred pronouns?

When mistakes are made, do you find someone to blame or do you all accept responsibility to address it? If the production database can be deleted by the graduate on their first day, and there are no backups, that is never their fault.

Creating Psychological Safety in the Workplace https://hbr.org/ideacast/2019/01/creating-psychological-safety-in-the-workplace

High Quality

Do you always have an high standard?

Everyone has their code reviewed, especially the lead. Is every line of code, and every process open to review and improvement? Great that you’re agile, but if you really value people over process, write the process down, and follow it. It doesn’t mean no process, it means that process serves the people, not the other way around. It means you change it when it no longer supports the people or the product.

What are your quality standards for code, for user experience, for security, and most importantly for behaviour? How are they enforced? And are they always enforced on time, every time?

Have policies. Do not have a daily fight over tabs vs spaces.

Direction

Ask everyone on your team what the team is building. If you get more than one answer, that’s a bug.

Ask everyone which part everyone else on the team plays towards that. Does that match how they see their role? Are there any gaps in responsibility?

Ask everyone what their priority is and why. Is anyone blocked? Ask them what their next priority is and if they have everything they need to fulfil it. If not, do they know where to get it?

Purpose

Is everyone bringing their whole self to work? Do office politics make them wary? Are they in a marginalized group and they have to bring representation as well as talent, and they are having to do both jobs at once?

At the office, is this the number one thing for them to be doing? Are your developers feeling stuck in support or BA? Are they frustrated that they aren’t allowed to refactor a gnarly piece of code that’s very open to be improved because “it works, don’t touch it”.

Does everyone on the team feel empowered to speak up and to fix things where they interfere with the goal of the team?

Motivation

Ask everyone why the team is building what they’re building, and why their part is important.

How will this change the user’s day? How will it affect the company? What’s the net improvement?

The Game Outcomes formulation

If you don’t like the Google formulation, try the game outcomes one. There’s plenty that applies to non-game projects. There’s a few negatives to avoid, and I’ll revisit them in a later post.

The most important indicators for success from the Game Outcomes project are:

  1. Great game development teams have a clear, shared vision of the game design and the development plan and an infectious enthusiasm for that vision.
  2. Great game development teams carefully manage the risks to the design vision and the development plan.
  3. Members of great game development teams buy into the decisions that are made.
  4. Great game development teams avoid crunch (overtime).
  5. Great gamedev teams build an environment where it’s safe to take a risk and stick your neck out to say what needs to be said.
  6. Great gamedev teams do everything they can to minimize turnover and avoid changing the team composition except for growing it when needed. This includes avoiding disruptive re-organizations as much as possible.
  7. Great gamedev teams resolve interpersonal conflicts swiftly and professionally.
  8. Great gamedev teams have a clearly-defined mission statement and/or set of values, which they genuinely buy into and believe in. This matters FAR more than you might think.
  9. Great gamedev teams keep the feedback loop going strong. No one should go too long without receiving feedback on their work.
  10. Great gamedev teams celebrate novel ideas, even if they don’t achieve their intended result. All team members need the freedom to fail, especially creative ones.

How do you keep your team on the right path?

We all want to work on successful projects, and there’s been a couple of times in my career I’ve been lucky enough to work in a team where everyone is delivering 10x. 10x developers don’t work in isolation, they work on teams where all the above needs are met, and they thrive off each other.

It’s great to have that dream team, but start by thinking about how to make your team reliably successful, and you’ll be doing better than most software teams.

Categories
development

Your API Sucks – The ‘default’ API : REST and repeat

I’ve recently got a Fitbit, and I use the website for logging food. I noticed something about the URL scheme that I’ve seen elsewhere and it made me think about how to design default points in URL schemes.

On Fitbit, ‘/foods/log’ opens the page for today’s food, and there are controls to navigate to yesterday and beyond, when the URL changes to ‘/foods/log/{date}’. So far, so simple. But when I navigate back to today, the URL still contains the date. My browser is set to reload pages from last time, so the first URL always gives me today. The second, doesn’t.

Should we set defaults that can change based on context? Or should the default redirect to the appropriate context? Or should we always be explicit? Should default URLs automatically redirect?

If I load the page before midnight but submit after, the screen and the URL indicate different dates. What is the right date?

UPDATE: The new Fitbit logging page, which appears occasionally at `/foods/log` takes the date as an explicit parameter (it’s a drop-down on the page), so no longer has a default. This is a much better strategy for submitting data

Categories
code

Live coding during a presentation

I’ve done a few talks in my time, and some of them have involved live coding to help take the audience on a journey to understand the lesson I want to teach. It’s a technique I use rarely however as it can easily get problematic.

Live coding for a presentation is not the same thing as coding for your job or a mob session. Live coding for a presentation has to be as slick and as polished as the rest of the talk. No googling the API docs, no stumbling over syntax errors (although don’t worry if you don’t spot them, someone will.)

If you’re nervous with slides, live coding can help relieve some of the pressure. It certainly helped me when I was starting out, as I could pretend I was having a conversation with my computer for some of the talk. Fortunately, it was a conference with microphones.

Live coding is also great for presenting something novel (or at least novel for most of the audience) as it gets to demonstrate that it’s real and it works. I used it, in one presentation, to demonstrate TDD.

Be careful though. Technology can let you down, so have a think about how you might demonstrate another way. Do you have a video you can show on someone else’s laptop? Can you demonstrate with screenshots? Could you whiteboard a solution?

If you don’t feel comfortable writing code in front of an audience, don’t. Can the code (I use VS code snippets, but there’s plenty of macro choices), so you can reveal it gradually. Or pre-build it and uncomment as you go.

If you are preparing code in advance, to distribute with the slides, heavily comment the code so that when people download it afterwards they understand the context.

I do love to see live coding done well in a presentation, but unless you’ve prepared it meticulously, it won’t go well. Think about the story, and how and when you’re introducing new concepts. Don’t be afraid to go back to something later (“Here’s how to connect, but I want to show you search first. I’ll come back to the parameters here once you’ve seen the search results.”) It’s OK to refer to your notes, especially if you’ve just thrown up a screenful of code for the audience to read. Make every interaction useful.

It’s OK to backtrack. Show the mistakes you made, the error messages you saw, and how you corrected them. You won’t be the last person to make that mistake.

Relax and enjoy it. If code is what you do, be comfortable with it, even if the nerves are trying to turn you inside out from your stomach.

Categories
code development programming

How I approach coding challenges

This time last year I was doing a lots of tests for job interviews, and I was practicing with https://adventofcode.com

I know there’s various data structures and algorithms you can learn to make these easier (and I hope to come back to some of my favourites in a later post), but there’s a general approach I take to these that helps me, and might help some of you too.

Use TDD

I’ve always been a fan of Test-Driven Development, and these type of puzzles lend themselves well to TDD thinking. Often examples are provided that will form the initial test suite. The algorithm is often broken into steps (sometimes explicitly “On every clock tick…”)

Get into the habit of checking as you go to make the most of the information provided to you. Think about the problem first, and the mapping between the input and the output, before you write any code.

Check everything on simple solutions.

Have templates

Most challenges involve a few common tasks. Read input from a file (ignoring blank lines), create a list of…, output as CSV, etc.

Some of these will have standardised patterns in your chosen language, and you should use them. Some will likely require some error handling. Some will require loading from standard or 3rd party libraries. And all will require a test framework.

As you go, build up a utilities list of functions, classes and packages that you know how to use, are robust enough and have minimal clutter. They’re not meant to be usable by anyone but you, but you will use them a lot. Collect and cherish them.

Think simple

Technical tests and coding challenges, in general, are designed to have short, easy solutions. If you need to solve P == NP to make your code fast, it’s likely you’re using an algorithm that’s a poor fit. Find 2 other ways to solve the problem, and pick the simpler one.

Simpler than that

Think about the simplest solution that could possibly work and write it simply.
Make the code readable. This is not the time for clever solutions. When it fails on an input, make it easy to change.

But not that simple

You still need to think about edge cases, and memory usage, and a myriad of other resource constraints. Think about what could go wrong, within the confines of the puzzle. Advent of Code has a particular knack for detecting edge cases in your code for Part 2 of each day’s challenge.

Think fast

Beware of premature optimisation, but expect long inputs and many loops if you use the naive implementation. Learn new algorithms and data structures.

Categories
development programming

Somebody else’s problem

Sometimes dividing tasks creates inefficiency. When one person both cooks and cleans, they tend to create less washing up than if the tasks are divided, because washing becomes “somebody else’s problem”. However, when one person washes and the other dries, the load is equivalent. This setup can be prone to stoppages when the drainer is empty and the washer is trying to get that stubborn burnt bit off the pan, at which point any good ToC practitioner will ask the drier-upper to help with the washing to maintain throughput.

When teams are divided by layer instead of features, the interface between them becomes a fracture because each team has context missing from the other. Data has the wrong shape, validation requires fields that can’t be captured. If you build both sides, the transition is smooth.

When developers test, they can focus on smaller, independent chunks, which means fewer tests with greater coverage. Writing integration tests for complex calculations is painful and slow. Testing those same calculations as an xUnit Theory is cleaner and faster.

Specialists are great, but where is specialisation causing fractures and more work?

Categories
code development programming security

This is raw

This is raw chicken : 🐤

If you eat it like that, you may get hurt immediately, by its beak, or its claws. It may grab your money and run off with it.

If you want to eat it, better to kill it first. 💀

If you eat it like that, you may get hurt or die, in a few hours, or days. Washing it won’t help.

Cook it. Cook it well. If there’s any sign of pink, cook it some more. 🔥

It might still kill you, but at least you’re a lot safer than when you started.


This is raw data : 🐤

If you display it, users will get hurt immediately, whether by cross-site scripting, cookie sniffing, crypt-currency mining, or something else. If you’re lucky, it will be something your user’s see immediately and leave your site never to return. Otherwise they may get infected.

If you want to use it, better to validate it first. 💀

If you save it like that, your users are still vulnerable. It might appear on the front end in a different form. It might be a string of unicode characters that crashes your phone. It might be a link to somewhere they can’t trust.

Encapsulate it. Sandbox it. Never trust it, in or out . HTML encode, whitelist the output as well as the input.

And if you need to avoid spam, or incitement, or solicitation, maybe you need editors. Computers can’t fix all the social problems. 🔥

Categories
code leadership

CodeCraft – Architecting Teams Notes and follow-up

Thanks to everyone who came to the CodeCraft guided conversation last week. I’ve added some notes to the questions below to carry on the discussion, and there’s a few topics I want to revisit in this blog, and in DDD Scotland if my talk is accepted.

The code craft team did a great job of reviewing and improving the questions so thanks to them, but I think in the very busy session itself, some of the questions were too ambiguous. For example, when I was thinking about goals, I was thinking about project goals and strategy goals rather than just “Make more money”, but I suspect that’s as much to do with the ambiguity many people have about their company strategy and culture in general.

It was also interesting to me to hear how agile is interpreted. The agile manifesto is clear that people take priority over procedures, and software over documentation, but explicitly the authors “still value the things on the right”. So it’s not about no procedures, and no documentation (please tell me you have coding standards), it’s about the right ones to support the people, the delivery of working software, etc. rather than documentation in its own right. Indeed, there are a number of explicit rules that an agile team needs in order to function. Is there a framework? What is the lifecycle of a unit of work? What language are we using? Not all of it needs to be documented, but some of it absolutely does, especially if you want to understand and improve it, and remove the rules you no longer need.

Be wary of anything that interrupts transparency, whether it’s Chinese walls, secret tasks, hidden agendas or ninja developers who work in the shadows and surprise everyone when they’re least expecting it.

And it’s never just enough to say who you are and what you do. Promote, encourage, and challenge. Be proud to be a diverse employer, to be wheelchair accessible, to have staff with more than 2 years real world experience, and welcoming to fresh faces with new ideas.

Trust. Respect. Communicate.


As a general point, I’m thinking of a team as “the people you work regularly with, usually daily”.

  1. How do you keep track of your team and the company goals?
  2. How does your team manage risk to and changes to those goals?
    • Pre-mortem
    • Chaos monkey
    • Agile “threw away risk register and other important documents”
  3. What makes a great team?
    • People
    • Feel comfortable
    • Trust
    • Communication
    • Balance
    • Good Leader
    • Sustainable development
    • Transparency
    • Healthy Debate
    • Growth and change
    • Common Goals
    • Motivation tailored to each person
    • Rules
      • Raise concerns
      • Feedback should be timely and structured
      • Coding standards
      • Process rules (e.g. how do we report a story can’t be implemented, what’s the weekly schedule)
  4. What would you change about your current team?
    • Management
      • Decisions made outside the team, not taking input from the team
      • Coaching developers to talk to management
      • Wrong person promoted to lead
    • Too many rules
      • “Because we’ve always done that”
      • Needs the context of why that rule exists
    • Clear roles
    • Respect for decisions the team has made
    • No secret tasks
    • Co-located, or remote, not mixed.
    • No “them and us”
      • Manager vs developer
      • Techie vs non-techie
  5. What makes you feel unsafe in a team?
    • Lack of:
      • Support
      • Communication
      • Respect
      • Goals
    • Secret genius / hero developer who does their own thing
    • Decisions made without buy-in
    • Teams without source control
    • No tests
    • Secrets
    • Success at the expense of others
    • Developers are not “resources”
      • And managers are not “overhead”
    • Fear
    • Stagnation
  6. How well would your current team survive a conflict?
    • To survive, have Strong Opinions, Weakly Held
    • Have a “naughty step” project for someone who doesn’t follow the rules, or doesn’t respect the team
    • Avoid imbalance in workloads that lead to loading stress onto key individuals
  7. Should teams change when projects change?
    • Teams should be mixed to avoid stagnation, but not completely break them up
    • Good teams can inspire others
    • Every good team cares about the product they produce
  8. How does the culture of a team change as it grows?
    • Negatively. Easily leads to “Us vs Them” (e.g. backend vs frontend vs DBAs, developers vs testers)
    • Lines of communication fail, especially as distance increases.
    • Old vs new – “I prefered the company when it was smaller”
    • T-shaped individuals are better collaborators
    • Tribe structures can help by allowing multiple communication lines
  9. What kinds of diversity should we seek in the teams we work in?
    • Diverse teams build diverse products
  10. Should we recruit to enforce the current culture or diversify it?
    • Put accessibility positively on ad when true (wheelchair accessible, BSL-friendly)
    • Do you know the current culture?
    • What is “culture fit”?
    • Does anyone know your company values? (not just the 5 Apprentice team names on the posters around the office)
    • Get job adverts reviewed by as diverse a team as possible
    • Diversify where the jobs are advertised
    • Not every candidate has 20 hours for a coding exercise, or wants to give up 2 holidays to pair with you
    • Coding review should be blind
  11. What one thing can a leader do to make a team great?
    • Trust
    • Delegate authority and empower the team
    • Protect the team
    • “Reading the room” understand what’s not being said so you can investigate
    • Push everyone to improve (including yourself)
    • Empathy
    • Happy people
    • Communicate
  12. Are effective teams a democracy or a dictatorship?

codecraftuk-sessions/architecting-teams.md at master · craignicol/codecraftuk-sessions · GitHub