code development

Excellent Export, IE and Security

Following on from supporting large Client-side spreadsheets in Chrome, I’ve extended my pull request to also support IE, which uses its own proprietary Javascript method. Because, Microsoft.

And all was good with the world.

Until a Microsoft update broke HTML-formatted XLS files. Because, Microsoft. If you support XLS downloads on your site, and your users are getting a blank Excel window when they try and open them, thank Microsoft, and a security fix. Then go back to using CSV. Or ODS.

Thanks Microsoft.

development programming security ux

Your API sucks : illegality

No human is illegal, especially your users. I know you were told to make it secure, so you’re filtering input, but some users (so long as you cover Scotland) live in Flat 1/2, so let me put the slash in their address. And let Shaun O’Malley have an apostrophe. Not only does this make for a poor user experience that the developers using your API either have to pass on to their users, or find a way to deal with – if you let them know what encodings you support – but that might be against security policy.

Worse than that however, a policy like that is a red flag to hackers. If you don’t allow /, do you allow \, or do you filter DROP? It’s a sign that there’s a wormhole through your code with a single line of defence against SQL Injection, or script injection. As a developer, it worries me, from a usability and a security perspective.


Stop sending the wrong message and making my users illegal.

code development

Your API sucks : documentation

Your API is perfect. It just needs a little finesse on the documentation. It’s fine, just farm it out to Johnny Clueless. After all, good APIs, like all good code, is self documenting. It’s just those damn users that keep asking what you mean when you said that, or wanting to know what all the options are. They’re very demanding.

Documentation out of sync with the code

getPostcode(double opt)
This method takes returns the current longitude and latitude of the user based on the information provided by their device.

Teaching Granny to suck eggs

Bad API documentation is like bad comments, it tells you everything except what you need to know, such as the return format.

getAddressFromPostcode(string postcode)
This method takes a postcode and returns the associated address.

Be explicit

However, don’t use this as an excuse to avoid documentation. Good documentation helps users access your API without using up your time. Be explicit about options, about which standards you support, and where necessary, how you support them. Even for simple things.

explicit-documentationThis image represents an authentication call to retrieve an API token that can be used when making further calls to an API. Each of the arrows represents an email discussion where the user needed clarification. This can be simplified with the right explicit statement:

To retrieve your API token, a POST request must be made to the HTTPS URL provided, with the following key-value pairs x-www-form-urlencoded into the Body of the request:

Key                       Value

client_id              your_username

client_secret      your_password

grant_type          client_credentials

If successful, this will return a JSON body in the response. The value associated with the key access_token should be appended to all subsequent requests as described below.

The use of italics is defined appropriately, and user specific information is supplied in a format to match the documented table.

code development

Excellent Export and the Chrome URL limit

One key difference between junior and senior developers is that when they encounter a bug, senior developers are much more likely to blame themselves before others, because experience moves you up the Dunning-Kruger curve.

I was given a bug report recently where Chrome was unable to download a particular spreadsheet, and IE didn’t work either. Other spreadsheets worked OK. It was fairly obvious that one feature of this spreadsheet was the likely candidate : it was by far the largest spreadsheet in the system.

I tried to recreate the bug, and I use Firefox, because I prefer the developer tools. And the spreadsheet download worked as expected. So it’s a difference between browsers. It might still be my fault, but it’s looking less likely.

The library we used for the export is a Javascript library called excellentexport, which converts an HTML table into an Excel-compatible spreadsheet. It uses the data: URI scheme to encode the spreadsheet directly into the page, so that the spreadsheet can be generated entirely server-side. Internet Explorer does not support data: so displays the link, but clicking on it has no effect. Chrome and Firefox both support it, but it turns out Chrome reduced the URI limit from (what I think is) 256Mb to 2Mb, and I wasn’t able to see why. This originally caused the tab to crash, and still can under the right conditions, but for URIs just bigger than 2Mb (in our case 3.5Mb), the click is processed, but the data URI is rendered empty, so a null file is downloaded.

This can be a tricky one to diagnose, hence this post. If you ever encounter this, the solution is to switch to CreateObjectUrl, which is supported in all recent browsers, including Edge, but not IE (but as it doesn’t support data: you’re not losing out there). I’ve submitted a Pull Request to excellentexport demonstrating the change required. Worth remembering if you ever need to create a download link client side.


code development ux

Your API sucks : verbosity

The more complicated your API, the easier it is for developers to make a mistake. Make it easy to do the right thing. If you have to create an order before submitting it, make the order itself the required context for submission. In a library, make Submit() a method on the class. In a RESTful web service, make the submission POST against the URI of the order you are submitting.

Limit user options. Choose sensible defaults that can be overridden if, and only if, requested. Think about sending an email. By default it’s not important, there’s no BCC, the reply-to address is your address. You don’t have to think about them unless you need them.

Compare that to the C++ Win32 API, which is partly a C++ problem, and partly a Windows problem. Look at the prerequisites you need to follow, look at how many NULL arguments there are, look at how much error checking is required.

Walkthrough: Creating Windows Desktop Applications (C++)

And then compare it with the Python TkInter Hello World example, which leans on a lot of defaults, and packages up most of the prerequisites inside the API.

Python TkInter – A Simple Hello World Program

code development programming

Your API sucks : standards

Standards are great. They help users understand what to do, they allow developers to write libraries that support multiple packages. Use them when you get the chance.

Not invented here

Sometimes companies can’t help themselves. They need to use something they control, to lock developers in, or because they’re agrophobic, or just to be different. It’s very annoying.

If you really want to ruin a developer’s year, however, take a standard, and almost implement it. Ask anyone using CSS or JavaScript in IE6. Or Netscape Navigator. Ask anyone who uses K&R braces in C# code. Or ask these guys:

At one time I had the pleasure of using WorldPay’s api. It was XML based but instead of just saying ‘it uses XML’ they spent a lot of the documentation describing what they meant by XML, e.g. “an attribute value is enclosed in double quotes”, or “the ‘>’ character must be escaped with >”. This always left me worried that they had written their own parser that might reject valid XML if it didn’t match the subset they described in the documents and if whatever library I happened to be using decided to put an attribute in single quotes or use a CDATA section to avoid escaping content it might go horribly wrong. – Comment from Duncan Booth

Use the platform

Once you have a standard, stick to it. Don’t mix your styles, don’t invent new ways of doing things. Users should only have to deal with one library at a time, not mix-and-match like Frankenstein’s monster.

    Name : "Your Woman"
    Address : "<Line1>My House</Line1>
            <Line2>My Street</Line2>
            <Town>White Town</Town>
development programming

Your API sucks : notice period

Based on a true story, although as one of the clients in question, I can’t guarantee the validity of the conversation. Any similarity to any developer alive or dead is purely coincidental.

“Dave, we need to make a change to our API.”

“Is it a zero-day security issue?”

“Nah, but it’s pretty important.”

“Will it break existing clients?”

“Yeah, probably, we used to recommend 128-bit keys, and we need to jump to 512-bit keys.”

“Do you think we should tell our users?”

“Yeah, probably. Want to let marketing know?”

“Shall we tell them it’s important?”

“Nah, they’ll figure it out.”

2 weeks pass

“Hey, support  have started getting a lot of calls about clients who cannot connect to our API, any idea why?”


“Did we make a change and not tell our users?”


“Do you think we should tell them?”

“Do we have to?”

Remember, even if you do give a positive notice period, your clients will need lead time to update their code.

code development programming

Your API sucks : foolish inconsistency

There are many ways that an API can be inconsistent. Different methods can follow different conventions, one method may have different results, or maybe a change introduces inconsistency between versions.

Spatial Inconsistency

When calling 2 different methods in what appear to be related domains require 2 very different calls and get 2 very different results.

GET /cat

{ Name : “Garfield” }

GET /food/index.php/fooditem/1


Random Inconsistency

When calling the same method multiple times can lead to a result with a significantly different format.

GET /cat

{ Name : “Garfield” }

GET /cat

{ Name : “Jatt” }

GET /cat

{ Name : “Lion-o” }

GET /cat


Temporal Inconsistency

When versioning an API breaks existing clients without warning. Speak to Seb Lambla if you want to do it without versioning, or Troy Hunt if you want to version it right, in all the wrong ways.



  API-version: “v1.1”
  Search-type: “postcode-lookup”
  Input: “EH28 8PL”
development lifehacks

Take your laptop home

The modern workplace demands flexibility. Sometimes to accommodate work, because international companies need to work to international timezones. Sometimes to accommodate life. Don’t discount good staff because they have childcare duties, or other demands on their non-work time. 9 to 5 isn’t the only way, so long as people do the work and are involved in the right conversations.

As a worker though, that flexibility in hours means that sometimes you need to work after bedtime, or you need to learn about a new technology in an evening or weekend event. You don’t stop thinking when you leave the office, but you can stop taking every call.
The modern workplace is noisy. It’s full of distractions. Go home and disconnect so you can have the peace you need to get in the zone. But put it in your calendar, in advance, so no one is surprised.
But only take what you need. Prepare your equipment and your space. Make it productive.
development leadership lifehacks

Don’t take your laptop home

To fit a good work life balance, your work should stay at work. My previous job supplied me with a laptop and rules, when working at certain desks, that I had to take my laptop home “for security”, and it became second nature to carry a heavy bag with heavy laptop, heavy notebook, and several printouts, because I’d need them.

It made sense for the job. I was on client sites a lot, and didn’t know when I’d have a network connection, so I had to take what I thought I’d need with me. And I’d check email on my personal phone because 3g/4g is far more available than WiFi.
I told myself it was OK because I was using webmail so I didn’t get notifications on my phone. But then I got phone calls on my mobile. At night. On holiday. Boundaries crumbled, and yet my bosses had it worse. They’d give me battle stories of weekend working and lack of sleep, as if poor planning at a company level was something to be proud of.
Let’s be clear. A culture of antisocial hours is a failing culture. A culture that bleeds onto personal devices when I pick up the cost is a failing culture. A culture where people brag about it is a failing culture. A culture that asks for a hackathon to work on things no-one can get a budget to fix is a failing culture.
I’ve heard horror stories of companies set up for one or more of the above. It happened to me occasionally on the odd project, and when I was leading, I made it my responsibility to identify and fix the root causes. Failing once is a good opportunity to learn. Twice is a warning. If you fail the same way 3 times in a row, it’s a dysfunction you should stop everything to correct.
I don’t have a laptop bag in my new job. I have a place to leave it overnight. I don’t need papers to visit clients. It’s lightweight, and technology has moved on so WiFi is available. Taking a laptop home is an exception, not an expectation.
To those who need their laptop out of the office:
  • Travel light. Small notebooks are better, tearaway pages are better still.
  • Don’t bring your own device. If a company needs you to access work outside the office, they will pay you to do so. Or at least give you reasonable compensation. They’ll have a sane home-working policy, or a laptop-installed communication channel. If you can’t use Unified communications, or at least VOIP, things should change.
  • Treat long hours as a bug. Fix it. If the company can’t fix it, the culture is wrong.
  • Antisocial hours may be required, but they must be compensated for in whatever reward suits the worker best. And planned ahead whenever possible.