Your API sucks : illegality

No human is illegal, especially your users. I know you were told to make it secure, so you’re filtering input, but some users (so long as you cover Scotland) live in Flat 1/2, so let me put the slash in their address. And let Shaun O’Malley have an apostrophe. Not only does this make for a poor user experience that the developers using your API either have to pass on to their users, or find a way to deal with – if you let them know what encodings you support – but that might be against security policy.

Worse than that however, a policy like that is a red flag to hackers. If you don’t allow /, do you allow \, or do you filter DROP? It’s a sign that there’s a wormhole through your code with a single line of defence against SQL Injection, or script injection. As a developer, it worries me, from a usability and a security perspective.

 

Stop sending the wrong message and making my users illegal.

Fear of failure, and risk-takers

We all make mistakes. We should learn from them, whether it’s because we didn’t know enough, or because we’re taking a risk. People are afraid of mistakes. We’re afraid of being found out. We need to fight imposter syndrome, and tell the world we are not phonies. So we don’t want to fail, we don’t to make mistakes, so we avoid taking risks.

But what if, when we ask Is it OK to fail? we start asking instead, “is it OK to learn?” or “Is it OK to to take risks?” If you feel like an imposter, that you’re out of your depth, you have 2 options:

1. Decide you need to learn. I’m currently in charge of a team that’s moving to a more devops culture, and I don’t know enough to know if we’re doing it right. So I’m reading, and testing, and building out my knowledge, so that I can make some mistakes and learn from them, to help me understand the mistakes bigger teams made so I can learn from them too.
2. Decide you don’t need to learn. Trust your team, or a 3rd party to deal with it for you. Learn how to detect bullshit, and let them deal with the detail.

I loved reading Risk taking and imposter syndrome at Google where Google talk about battling imposter syndrome head-on. When you are surrounded by the brightest and best, you are always going to struggle with confidence. So ask everyone what risks they are taking, and what they’re learning. What do you know today that you didn’t at the last stand-up?

When you are adding value, no-one will think you’re an imposter. And hopefully, neither will you, but when you do, know you’re not alone.

Go learn.

 

 

Your API sucks : documentation

Your API is perfect. It just needs a little finesse on the documentation. It’s fine, just farm it out to Johnny Clueless. After all, good APIs, like all good code, is self documenting. It’s just those damn users that keep asking what you mean when you said that, or wanting to know what all the options are. They’re very demanding.

Documentation out of sync with the code

getPostcode(double opt)
This method takes returns the current longitude and latitude of the user based on the information provided by their device.

Teaching Granny to suck eggs

Bad API documentation is like bad comments, it tells you everything except what you need to know, such as the return format.

getAddressFromPostcode(string postcode)
This method takes a postcode and returns the associated address.

Be explicit

However, don’t use this as an excuse to avoid documentation. Good documentation helps users access your API without using up your time. Be explicit about options, about which standards you support, and where necessary, how you support them. Even for simple things.

This image represents an authentication call to retrieve an API token that can be used when making further calls to an API. Each of the arrows represents an email discussion where the user needed clarification. This can be simplified with the right explicit statement:

To retrieve your API token, a POST request must be made to the HTTPS URL provided, with the following key-value pairs x-www-form-urlencoded into the Body of the request:

Key                       Value

client_id              your_username

client_secret      your_password

grant_type          client_credentials

If successful, this will return a JSON body in the response. The value associated with the key access_token should be appended to all subsequent requests as described below.

The use of italics is defined appropriately, and user specific information is supplied in a format to match the documented table.

Excellent Export and the Chrome URL limit

One key difference between junior and senior developers is that when they encounter a bug, senior developers are much more likely to blame themselves before others, because experience moves you up the Dunning-Kruger curve.

I was given a bug report recently where Chrome was unable to download a particular spreadsheet, and IE didn’t work either. Other spreadsheets worked OK. It was fairly obvious that one feature of this spreadsheet was the likely candidate : it was by far the largest spreadsheet in the system.

I tried to recreate the bug, and I use Firefox, because I prefer the developer tools. And the spreadsheet download worked as expected. So it’s a difference between browsers. It might still be my fault, but it’s looking less likely.

The library we used for the export is a Javascript library called excellentexport, which converts an HTML table into an Excel-compatible spreadsheet. It uses the data: URI scheme to encode the spreadsheet directly into the page, so that the spreadsheet can be generated entirely server-side. Internet Explorer does not support data: so displays the link, but clicking on it has no effect. Chrome and Firefox both support it, but it turns out Chrome reduced the URI limit from (what I think is) 256Mb to 2Mb, and I wasn’t able to see why. This originally caused the tab to crash, and still can under the right conditions, but for URIs just bigger than 2Mb (in our case 3.5Mb), the click is processed, but the data URI is rendered empty, so a null file is downloaded.

This can be a tricky one to diagnose, hence this post. If you ever encounter this, the solution is to switch to CreateObjectUrl, which is supported in all recent browsers, including Edge, but not IE (but as it doesn’t support data: you’re not losing out there). I’ve submitted a Pull Request to excellentexport demonstrating the change required. Worth remembering if you ever need to create a download link client side.

 

Your API sucks : verbosity

The more complicated your API, the easier it is for developers to make a mistake. Make it easy to do the right thing. If you have to create an order before submitting it, make the order itself the required context for submission. In a library, make Submit() a method on the class. In a RESTful web service, make the submission POST against the URI of the order you are submitting.

Limit user options. Choose sensible defaults that can be overridden if, and only if, requested. Think about sending an email. By default it’s not important, there’s no BCC, the reply-to address is your address. You don’t have to think about them unless you need them.

Compare that to the C++ Win32 API, which is partly a C++ problem, and partly a Windows problem. Look at the prerequisites you need to follow, look at how many NULL arguments there are, look at how much error checking is required.

Walkthrough: Creating Windows Desktop Applications (C++)

And then compare it with the Python TkInter Hello World example, which leans on a lot of defaults, and packages up most of the prerequisites inside the API.

Python TkInter – 24.1.2.2. A Simple Hello World Program

Your API sucks : standards

Standards are great. They help users understand what to do, they allow developers to write libraries that support multiple packages. Use them when you get the chance.

Not invented here

Sometimes companies can’t help themselves. They need to use something they control, to lock developers in, or because they’re agrophobic, or just to be different. It’s very annoying.

If you really want to ruin a developer’s year, however, take a standard, and almost implement it. Ask anyone using CSS or JavaScript in IE6. Or Netscape Navigator. Ask anyone who uses K&R braces in C# code. Or ask these guys:

At one time I had the pleasure of using WorldPay’s api. It was XML based but instead of just saying ‘it uses XML’ they spent a lot of the documentation describing what they meant by XML, e.g. “an attribute value is enclosed in double quotes”, or “the ‘>’ character must be escaped with >”. This always left me worried that they had written their own parser that might reject valid XML if it didn’t match the subset they described in the documents and if whatever library I happened to be using decided to put an attribute in single quotes or use a CDATA section to avoid escaping content it might go horribly wrong. – Comment from Duncan Booth

Use the platform

Once you have a standard, stick to it. Don’t mix your styles, don’t invent new ways of doing things. Users should only have to deal with one library at a time, not mix-and-match like Frankenstein’s monster.

{
    Name : "Your Woman"
    Address : "<Line1>My House</Line1>
            <Line2>My Street</Line2>
            <Town>White Town</Town>
            <Postcode>EE11EE</Postcode>"
}

CodeCraftConf : call for guides

The CodeCraftConf call for guides is open, and I’d recommend either applying to be a guide or buying a ticket to attend. Last year’s CodeCraftConf was a great event, and I learned a lot.

For those of you unfamiliar with the guided conversations format, the conference website has a good write-up, but it’s a way to keep the conversation going by injecting new questions and riffing of old ones, to give everyone a chance to contribute.

If you want to be a good guide, be sure to share the conversation, and not let one or two individuals control it, especially if they’re in conflict.

If you want to apply, and you’re not sure what sort of questions to ask, have a look at my list from last year, and my write-up of the answers.

I hope to see you there.

Cognitive load: Default to success

​Process impedes progress. It’s the hurdles, and the red tape, that stop you from doing what you want to do. It’s why you can’t have administration rights on your machine, it’s why you can’t commit directly to master. It’s why developers don’t get logins to production. It limits your agility. 

Sometimes that’s a good thing. So long as it’s a straightforward, time efficient workaround. You don’t commit directly to master, but every green, code reviewed pull request does. You don’t get a login to production, but the deployment server does, and you can quickly get access to production data on your machine when required to recreate a bug. And your live servers are protected. 

Make it easy to do the right thing. Process is part of it, and so is tooling. Don’t make me think about the right way to do it, direct me, without getting in my way. Make the process implicit, but open for investigation and improvement. 

We believe in people over process, but when machines can automate the process, the people are free to think about the business problem.

Your API sucks : notice period

Based on a true story, although as one of the clients in question, I can’t guarantee the validity of the conversation. Any similarity to any developer alive or dead is purely coincidental.

“Dave, we need to make a change to our API.”

“Is it a zero-day security issue?”

“Nah, but it’s pretty important.”

“Will it break existing clients?”

“Yeah, probably, we used to recommend 128-bit keys, and we need to jump to 512-bit keys.”

“Do you think we should tell our users?”

“Yeah, probably. Want to let marketing know?”

“Shall we tell them it’s important?”

“Nah, they’ll figure it out.”

2 weeks pass

“Hey, support  have started getting a lot of calls about clients who cannot connect to our API, any idea why?”

“Erm….”

“Did we make a change and not tell our users?”

“Erm….”

“Do you think we should tell them?”

“Do we have to?”

Remember, even if you do give a positive notice period, your clients will need lead time to update their code.