Inflection points

An inflection point comes where there is a dramatic change in direction. In a typical business or software process, this is the fulcrum around which the rest of the process rotates.

They’re often easy to spot by looking at the before and after. For a typical shopping scenario, the inflection point is the buy now point. Everything before that is sales, everything after is fulfillment.

Some processes may also have secondary inflection points. Broadband provisioning for example has one inflection point where the order is confirmed, a fulfillment process that may involve site visits, transfer of your number from your previous provider and other preparatory work, and then a second inflection point once everything’s ready and you start paying line rental and usage fees.

Inflection points in software

The reason I’m interested in this at the moment is that these can be unnecessarily complex things to model in software.

We have a natural tendency, especially in domain driven thinking, to simplify the mapping between the business and software domains, so that we can reduce the cognitive load switching between the two.

For inflection points, this simplicity of mapping can easily lead to additional code complexity.

For a typical physical sale, most of the following will occur at the point the user hits the “buy” button :

• Verify payment
• Decrement stock count
• Send confirmation email
• Trigger warehouse workflow
• Trigger delivery workflow
• Redirect user to confirmation page

Like all good developers, we’ll break these down into functions so we can test each step individually, we’ll put the payment options, the warehouse and delivery systems and email behind interfaces so we can mock them out. But we’re still doing too much.

Decoupling inflection points

We have a single user interaction with a lot of side effects, making it harder to reason about, delaying the user response and coupling the success of the interaction to multiple external systems. Yes, the payment provider is essential in creating the order, but does it break the order if the email, warehouse and delivery components don’t start until several minutes later?

There is an opportunity here to use event listeners or scheduled tasks or other asynchronous processes to decouple the success of the order from the other systems. It means that the steps in the business process aren’t as easy to follow, so the triggers for the consequentials need to be clear (“all orders without a delivery workflow should have one created”), and this approach definitely prefers pull conditions rather than push conditions, so each step knows what it depends on, but not what depends on it – the payment process now only cares that it creates an order, not who fulfils that order.

How do you deal with your inflection points?

Thinking in Code

What is coding? The practice of breaking down big problems into very small problems. Turning everything into logic : recipes and decisions. Taking the fuzziness and ambiguity of the world and human language and turning it into something definitive and concrete, something that can be reasoned about.

And then you start to notice things about human behaviour and the processes they generate. With the exception of a few professions, human processes are hard to reason about. They’re designed with ambiguity built in because it’s not human nature to dive deep into every problem.

Have a principle, not a process “we need to be more secure than that dating site”, “users need to be able to manage their privacy settings easily”. Every process you bake into the code will support the principles, if you have the right conversations. But how does it deal with the outliers, the user with 1 name, the one organisation with 1 million staff, the one day the stock market price changes by 60% rather than 0.6%?

For Software to be repeatable and reliable, we have to remove flexibility. How do you delegate to humans?

I’ll come back to that in a later post.

The Constant Gardener : Knowing your team

One of the most interesting challenges in becoming a lead is understanding your team, and balancing project tasks between them. Whilst experience, both technical and domain, is a valuable indicator of how to assign tasks, there’s a few other things to take into account.

Having worked with a few people over the years, I can see 2 main types of developers. Managers may be asking which of the 16 Myers-Briggs personality types these represent, but I’m trying to give a persona sketch rather than a detailed breakdown.

Innovators are the ones I see most often – developers who want to work on the hot new thing, keep seeking out new ideas, and often like to throw things out and start again. They can pick up new projects quickly, by focussing on certain areas but are likely to get bored if they stay on one project for too long and may sometimes miss the bigger picture. That’s great for learning, and pushing forward the boundaries, but may not provide the stability that some projects need, and may need closer code reviews to ensure they maintain the house style for the project. If you’ve ever needed a white knight on your project, they’ll probably be an innovator.

Gardeners will take longer to get up to speed with a new project, because they prefer to dig in and find out how everything works, and build a detailed knowledge of the system before making changes. They will tend to evolve systems over extended periods. If you get someone like that on your project, you’ll want to keep them for several months, as their value to the project increases over time. Long lived projects with several maintenance releases benefit most from this type of individual. You may well find that they know the business domain better than the customer.

I think everyone I have worked with has qualities of both, but there are individuals who are clearly more innovators, and others that are clearly more gardeners. A good project should have both, with innovators pushing ideas and bringing new technologies to the tables, and the gardeners bringing the domain knowledge to make sure the right changes are made for the long-term health of the project. As a lead, your job should be to understand the way your team things, so that you can take the right ideas on board at the right time, and bring your team along with you. And, most importantly, know yourself. The most important thing the others on the team bring is knowledge of things you don’t know, or at least questions that force you to think about them. Embrace it. Inspire your innovators and nurture your gardeners.

Handling disagreements as a team

Following my code review post, I added some more thoughts regarding code reviews, but there was one comment that I wanted to come back to later:

“Social aspect: How do we handle disagreements? As a team.” – peitor

There are a number of aspects to successfully dealing with disagreements, some of which are about mitigation and avoiding the problem in advance, and others about how you deal with the problem once the disagreement has started to grow, after all, it’s hard to avoid the vim vs emacs, “that’s not RESTful”, brace placement, or other developer arguments.

Collective Ownership

The first key to avoiding unnecessary disagreements is to leave your ego at the door, because teams full of egos will fight each other to be seen. As Peopleware: Productive Projects and Teams so elegantly describes, any situation that asks developers to compete against each other, whether under performance review,or by dividing functional teams geographically, will inevitably lead to conflict and a blame culture. If you are all on the same team, with the same goal, it’s in everyone’s interest to resolve disagreements before they become conflicts.

Constructive Criticism

One of my colleagues has been writing a series on becoming a technical lead, it’s all worth a read, but his latest post, about constructive criticism, is particularly relevant to this discussion.

Constructive Criticism is a reality of doing code reviews, it’s arguably the whole point of doing them, but make sure that you are able to articulate clearly what you expected in a friendly manner, a Blame Culture is bad.

This doesn’t just apply to code reviews, but that’s an important example of where code is judged. Firstly, constructive criticism should be about the behaviour or the outcome, rather than the person (although even at that level, it’s wise to avoid abusive language if you want to avoid alienating your team, just ask Linux Torvalds), and secondly it should offer practical, achievable improvements, so that the person receiving them can use them as an opportunity to improve.

Lone Wolf

There are 2 main categories for individuals who have a detrimental effect on the team. One is the black hole who needs additional support or encouragement and sucks time away from the rest of the team, and any work they produce is marginal. They might be just a poor team player, or a square peg in a round hole (maybe they’d be great in another job on the team or in the company), but they breed resentment in the rest of the team having to compensate.

As a leader, you either need to find a way to help the person to improve and fit into the role they are doing, via mentoring, training or other means; or you need to find a role they are better suited to, by looking at other tasks that need doing.

If you can’t do either of these things, you will need to make the tough decision to get that person off your team so that the rest of the team become more productive and do not degenerate into dysfunction.

Rockstar

The other key individual threat to team coherence is the rockstar developer. The one who plays by their own rules, and always knows best. Unlike the black hole, they get the job done, but they often do so at the expense of the team.

Key traits of Rockstar code is code that doesn’t fit the rest of the team, code that is too clever for its own good, and therefore much harder to test and debug.

Everyone knows that debugging is twice as hard as writing a program in the first place. So if you’re as clever as you can be when you write it, how will you ever debug it?

• Brian Kernighan

Just like the black hole, the rockstar developer causes addition work for the rest of the team, but often leaves behind code that the rest of the team don’t understand in order to be able to refactor it. This resentment can be seen by the rockstar developer as jealousy, as their non-stick ego prevents them from taking responsibility for dysfunctions in the team, and it is clear, particularly to those outside the team, that the individual is achieving results (indeed, the rockstar often makes it very clear).

A rockstar does have many of the qualities needed to be a good developer, so long as they can be convinced to simplify and work with the team rather than steamrolling past. Unlike black holes, they are probably doing the right tasks, but need help to understand their impact on the rest of the team and any problems that result from their code, as they may have trouble accepting it.

Duelling banjos

Sometimes, there are individuals that work well with the rest of the team but not each other. Sometimes this can be caused by jealousy or resentment, particularly if one was recently promoted or given extra responsibility when they were previously equal, or one of them is more likely to have their ideas accepted by the team.

The slighted individual in this scenario needs support from the other individual and the rest of the team to accept the situation, and build resilience so they can move on, and, where appropriate, a clear explanation for why the differential occurs.

Hopefully these disagreements are on the ideas rather than the individuals, because when things get personal, it gets a lot harder for people to back away from a disagreement. However, the technology world is notorious for bug fights over little things.

Which text editor do you use? (whichever one you like, so long as you can set the format to the project coding standards)

Where do you put the opening braces? (for Javascript, always use K&R style, on the same line, for everything else, whatever the project says)

If you’re fighting over something that doesn’t really matter, ask yourself why, and if you can just agree to disagree, and accept that you’re not going to convert everyone to Dvorak.

If you see others on your team fighting, look out for Logical Fallacies that can drag the argument down into personal attacks, or unproductive sniping rather than a stimulating argument on improving the code. Be prepared to step in and separate the individuals. Give them a chance to cool down and reset, or space to think, talk and reflect.

Red eyed monster

The tired, stressed team are far more likely to argue and those arguments are more likely to get personal. Don’t overwork. Find your work-life balance.

You

You are a member of the team. Exhibit the behaviour you want to see.

Google Inbox and Inbox Zero

With a broken phone and Inbox by Google I went from 5000 unread messages to Inbox Zero in a week with some email bankruptcy and clearing out some tasks.

It’s not about being anal and keeping things neat, it’s about recognising that you’re never going to reply to that email from 2 years ago, but it’s hiding that email you should respond to.

Be ruthless. Archive, delete in bulk. That sale 3 years ago is over, you’re never going to fix that NHibernate bug, and someone else is looking at it, it’s too late to sign the petition to bless the rain down in Africa.

And as I found when my phone broke, and I got thrown back to a version of Gmail without bundles, stop hiding things behind labels. If you’re not going to read it, unsubscribe or filter it into the bin. Lose the haystack, keep the needles.

Up to speed with Git with recommended tools and books

On my current project, I introduced Git to a team used to Subversion (apart from the odd few), so I’ve had to research a few good resources for the team. I wanted to share these with you, for those of you new to Git, or wanted to get your team up to speed.

The best resource I found to start with was Version Control by Example by Eric Sink, which walks through a typical workflow in Subversion, Git and a couple of other version control systems. It’s great for picking up the basics for those coming from another version control system, or none.

I recently read Pro Git, by Scott Chacon and Ben Straub, which has given me a much better understanding of what’s going on under the hood, and has been a great resource to help me understand some of the problems the team have had with Git, and the patterns and principles required to fix them. It’s also got some great information for administering servers and repos, including setting up hooks, transport protocols, and various other details. I’m not sure I’d recommend it to a novice, but if you really want to get under the skin of the key-value datastore and directory structure underneath Git and want to know how and why it does what it does, it’s a great resource.

Pro Git recommends HTTPS as the transport layer, and that’s probably the easiest to set up for corporate networks with tough firewalls, but I have had problems with Github’s 2-factor authentication, and I tend to switch between a number of tools on the desktop, primarily SourceTree and Visual Studio 2013’s Git Flow Extension by Jakob Ehn for the Git Flow process, TortoiseGit for the merge UI, and the command line for everything else, which means the credential cache is often out of date, whereas SSH keys are much easier to keep in sync via Pageant, as installed by Tortoise. If you want to set up SSH keys yourself in Windows, one of my colleagues wrote this post.

For the members of the team less comfortable with curly braces, I’ve pointed them towards GitHub Desktop for the easiest way to stay in sync.

What are your favourite Git tools and reading?

Dynamic language and semantics

Given the opportunity to revisit my Genetic Algorithms library, I wanted to revisit my open letter to Jon Skeet, about semantics and dynamic languages, and compare how the code looks in C++ vs Python. I accept that C++ is an overly verbose language but the problem I want to discuss is similar in C# and Java. I’m just going to pick a few examples, but feel free to explore the code yourself, either as C++ templates or as Python examples. These examples all expect a certain knowledge of Genetic Algorithms, so I’d recommend reviewing my previous post first if you are unfamiliar with the concepts.

If you’re trying to compare them directly from the repositories, please note that I have removed some comments and logging, and changed the formatting in this post for reasons of clarity, and have yet to apply those changes back to the repository. I have also added some functionality to the python code that doesn’t currently exist in the repository to ensure the examples are comparable.

Fitness

In C++

  /***********************************************************************
   *            SAMPLE FITNESS FUNCTIONS                                 *
   ***********************************************************************/

  template<class _Ctype, int _Csize>
  struct fitness_base {
    virtual double operator()(_Ctype chrom[_Csize]) = 0;
  } ;

  template<class _Ctype, int _Csize>
  struct maxones : public fitness_base<_Ctype,_Csize>{
    double operator()(_Ctype chrom[_Csize]) {
      return std::count(chrom, chrom+_Csize, 1);    
    };
  } ;

The fitness function for a chromosome returns a number that is used to rank the chromosome such that fitter solutions are more likely to survive and breed. For the example below, note that by default, the chromosome is an array of type _Ctype (default: bool), of length _CSize (default: 32)

The fitness function must be passed as a parameter to the constructor so that different problems can be explored using the same representation. This requires a type for fitness functions fitness_base which is defined as virtual to ensure it can be overridden, and contains an () operator so it can be called. In C#, this would be a delegate type, and we would be able to use the function/action types from the library as the base class for maxones. The fitness function shown simply counts the number of ones in the chromosome.

In Python

    def fitness(self):
        return sum(self.genome) 

Functions are first-class objects in Python, so we don’t need anything special to pass them as arguments, or require overloading the () operator. However, Python also allows patching classes post-initialisation, so we can overwrite the fitness function, or we can use Python’s multiple inheritance mechanism to subclass 2 different versions of chromosome, one of which overrides the fitness function, and the other overriding the crossover function.

Note that the fitness function is different between the two fitness functions is simple, but I now prefer the cleanliness of sum to a conditional count, but the overall character of the code will not be altered by the 2-character difference.

Run one generation

In C++

    void run_once_replace() { // one generation = psize crossovers
      pop_t newpop;
      newpop.reserve(_population_size);

      for(int i = 0; i < _population_size; ++i) {
          chrom_t child = select_chromosome() + select_chromosome();
          newpop.push_back(child);
      }

      copy(newpop.begin(),newpop.end(),_population.begin());

    };

In Python

    def run_once(self):
        "Run one iteration of the genetic algorithm"
        # Use replacement algorithm
        new_pop = []
        for i in range(len(self.pop)):
            a, b = self.select_two_members()
            new_pop.append(a + b)
        self.pop = new_pop

In these examples, apart from some small syntactic sugar that allows us to return 2 values from a function, the C++ and Python code are just as clear as each other.

Conclusion

In contrast to my letter, there isn’t as big a difference as I believed when I originally wrote the letter, at least in terms of the resultant code, although I remember the pain trying to get the types lining up properly in the first place. Semantics still matter, and types definitely got in the way designing the chromosomes, but actually, outside some specific cases, with well structured code, static and dynamic languages can both make semantic meaning clear.

DRM and plastic knifes

I tried to do something I wasn't allowed to do but they refused to tell me what I'm not allowed to do.

I thought Digital Rights Management was dying when iTunes no longer required it. I was wrong. It’s sneaking into HTML, it’s in coffee machines, it’s being discussed for JPEG, it’s led to the introduction of boot lockers to prevent users from modifying the operating system on their machine.

So what? Surely copyright holders have the right to protect their content, no matter that the entire music industry could be bought out with Google or Apple’s spare change. Surely they need the money to support the poor struggling up and coming artists, who they can’t afford to pay from streaming revenues. Surely we should just let Hollywood have their way?

And anyway, doesn’t restricting operating systems and the software we can install protect us from the bad guys? Apple will keep us safe in their protected iOS and App Store, right?

When you restrict tools via drm, only the professionals can use the full tools, but where do the next professionals come from? We’re suffering from a shortage of people who want to learn to develop software, and we’re putting extra barriers in place, so that we need specialist devices for teaching development in the form of the Raspberry Pi and others, because the main machines we use are locked down and restricted.

Knives are dangerous and they can kill, so Scotland has restrictions on their sale and carrying them, but we still teach children to cook, sharp knives and tools for sharpening them are still widely available, and you don’t need to be a licenced chef to use a real knife.

If we can trust society enough that we don’t need plastic knives outside preschool, fast food vendors and cheap airlines, why can’t we trust them with access to a fully functional computer?