code development programming

Abstractions are scaffolding

All software is an abstraction. It’s human-like language with a logical structure to abstract over ill defined business processes, and gates and transistors, and assembly language, and often 7 network layers, and memory management and databases and a myriad of other things that abstractions paper over to help you deliver business value.

Abstractions are the scaffolding you need to get your project running, but they’re another dependency you need to own. You need to understanding what’s under the abstraction. Or you start thinking network traffic is free. The network is not your friend, it’s slow and unreliable.

Abstractions provide some structure to make what they support easier, but they in turn rely on structures underneath. If you’re going to own an abstraction like any other dependency, you need to understand the structure it’s built on and what it’s designed to support. I understand what ORMs do because I’ve written some of that code in a less performant, less reliable way, before I realised someone had done it a lot better. Indeed, that was the realisation that drove me to alt.Net and NHibernate, but it meant that I understood there was SQL underneath, and the SELECT N+1 problem was a mismatch in the queries that could be resolved, and not just an unexplainable performance hit caused by a network spike.

Abstractions make more sense if you understand what they’re abstracting over. They’re not there for you to forget about what’s underneath, just to save you having to write that code for every class in every project you work on, and to benefit from a wider pool of developers working on battle-tested code. If you don’t know TCP from HTTP, don’t write web applications. If you don’t understand databases or SQL, don’t use an ORM.

All abstractions are imperfect. Learn what’s under the cracks.


Moving house and denormalised data 

There’s a lot of abstract talk about data normalisation and having a single source of truth within your organisation for each type of data. Sometimes this goes very wrong. 

I moved house this year, and whilst I accept the extra administration that comes from each organisation needing to be updated individually (after all, why should DVLA need to know who I bank with?), I do expect each organisation to make it easy to change my address. 

And it was easy, except for my bank. Current accounts, credit cards and savings are all managed separately, the address I see online is unrelated to any of these, and I have to change my address on joint and single accounts separately. 

I don’t care what your backend system is like. My interface should be one call and you sort it out, not keep redirecting me to queue after queue. And sending me a letter to the new address telling me it’s fixed, followed by another letter dated a week later to the old address with marketing in it. 

And then, I tried to phone them about that letter, and I couldn’t pass their security questions, because my address was updated everywhere except the phone validation system. 

How easy is it for your data subjects to update their data? One call or many? Is their security tied to that data – could they lose access if a bad agent adds a new address or credit card? How confident are you that you know all the places in your user, marketing, communication and other databases that need updating? 

How can you make it easier? 

code data development programming quickfix

The Getting Better Moment

At the stroke of midnight, your code will turn into a pumpkin.

Inspired by The Getting Better Moment – You’ve Been Haacked , this is my story of my first job and a getting better moment.

During my summer break in university, back in 1999, my local council, who part funded my university education, back in the days before student loans, found me a placement at a local charity to develop a CPD database to track the exams their members took every year.

It was an Access database, installed on one machine on the network. With manual backups. It stored the pass/fail data in a set of bit fields in the database, requiring some complex sql to answer the common “who hasn’t done this course yet” mail merge query, it used VBA to parse those SQL queries into a screen full of check boxes to allow users to search.

None of that was my getting better moment though. I went from being a very shy geek at the start of the placement to a slightly less shy geek with a lot of confidence in my code. It was the first thing I’d written that was out in the real world and I was proud of it.

Until the first work day of 2000, when it stopped working. Despite the initial suspicions of the customer, it wasn’t a y2k bug. There was a bug in the end of year routine. What it was meant to do was create an empty exam results table for the new year, then rewrite the rest of the system to point to that table do that all the queries, screens and reports were up to date. It failed because it wasn’t properly tested. And I spent a whole day in their office, feeling sheepish, rebuilding the queries, screens and reports by hand to fix the mess.

But that wasn’t my big learning moment. That came exactly a year later, with another call, and another broken database. A lot of pressure now, trying to remember a fix from 12 months before on 18 month old code. It took longer than the first time, but I spent some time afterwards documenting the fix, testing the end of year rollover, and simplifying the code so I was capable of debugging it.

Luckily the customer was reasonably happy, as they didn’t run many exams in the first week of the year, and they understood you don’t get perfect solutions when you pay student placement wages, but I have tried hard since then to keep the customer happy and keep bugs that are fixed, fixed. All software has bugs in it, it’s how you deal with them, and the customer, that separates a professional from a beginner. The experience set me on the path towards TDD, open and honest communication, and not wanting to pick up support calls out of hours, which is a great motivator of quality. If you can lie on a beach, knowing you won’t get called because your code just broke, you’re doing it right.

What’s your getting better story?