Categories
development

Your API Sucks – The ‘default’ API : REST and repeat

I’ve recently got a Fitbit, and I use the website for logging food. I noticed something about the URL scheme that I’ve seen elsewhere and it made me think about how to design default points in URL schemes.

On Fitbit, ‘/foods/log’ opens the page for today’s food, and there are controls to navigate to yesterday and beyond, when the URL changes to ‘/foods/log/{date}’. So far, so simple. But when I navigate back to today, the URL still contains the date. My browser is set to reload pages from last time, so the first URL always gives me today. The second, doesn’t.

Should we set defaults that can change based on context? Or should the default redirect to the appropriate context? Or should we always be explicit? Should default URLs automatically redirect?

If I load the page before midnight but submit after, the screen and the URL indicate different dates. What is the right date?

UPDATE: The new Fitbit logging page, which appears occasionally at `/foods/log` takes the date as an explicit parameter (it’s a drop-down on the page), so no longer has a default. This is a much better strategy for submitting data

Categories
code development programming

Translations with React, Redux and Asp.Net

In a recent project, which was the first time I’ve used React and Redux in anger, we had a requirement to support 2 different languages, in the .Net backend for emails and PDFs, and in the React frontend.

As the translators we used were used to resx files, we wanted to use those as the master source. In other projects we’ve done the javascript translations via a pre-compilation step into static javascript files, but since redux has it’s own store, I decided to see if we could use that to store and process the translations.

This approach has the advantage that we can use the redux state to translate the site automatically when a new language is selected, without having to reload any pages. For an application that depends on up-to-date data, and had to operate on por data connections, avoiding reloads is essential.

The redux examples shown here use TypeScript, which certainly helped in our development process, but there’s a lot of gotchas getting the dotnet new react-redux template and many 3rd party react and redux libraries working nicely with TypeScript. You can make it work, but it’s definitely not an out-of-the-box solution in most cases.

The solution consists of 3 parts:

  1. The Asp.Net Core 2.0 controller to translate the resx data into JSON (in this example, the resource file is called UserFacingStrings.resx and the default is en-US,
  2. The Redux configuration to retrieve the data and populate it into the store; and
  3. A React function that reads the translation from the store and presents it to the user.

The solution below attempts to get the localisation from the user headers, so you’ll need to enable localisation in your Startup.cs, but depending on your use case, you may want save user language settings in the user’s account data, or in a cookie or other storage in their browser.

ASP.Net Core API controller

[Produces("application/json")]
[Route("api/Localisation")]
public class LocalisationController : Controller
{
    private static Dictionary _cultureCache = new Dictionary();

    /// Get all known translations
    /// Choose a culture (e.g. `en-US` )
    /// will default to browser culture if not specified
    [HttpGet("[action]/{culture?}")]
    public IActionResult Translations(string culture)
    {
        try
        {
            var cultureInfo = CultureInfo.CurrentUICulture;
            try
            {
                string preferredLanguage = HttpContext.GetPreferredLanguage();
                if (!string.IsNullOrWhiteSpace(culture))
                {
                    cultureInfo = CultureInfo.GetCultureInfo(culture);
                }
                else if (!string.IsNullOrWhiteSpace(preferredLanguage))
                {
                    cultureInfo = CultureInfo.GetCultureInfo(preferredLanguage);
                }
            }
            catch (CultureNotFoundException)
            {
                // Fallback to en-US
            }

            if (_cultureCache.Keys.Contains(cultureInfo.ToString()))
            {
                return new OkObjectResult(new { t = _cultureCache[cultureInfo.ToString()] });
            }

            var translations = new Dictionary();

            // Insert all resources, to then be overwritten
            if (cultureInfo.TwoLetterISOLanguageName != "en")
            {
                ExtractResources(translations, UserFacingStrings.ResourceManager.GetResourceSet(CultureInfo.GetCultureInfo("en-US"), true, true));
            }

            ExtractResources(translations, UserFacingStrings.ResourceManager.GetResourceSet(cultureInfo, true, true));

            _cultureCache[cultureInfo.ToString()] = translations;

            return new OkObjectResult(new { t = translations });
    }
    catch (Exception)
    {
        return NotFound();
    }
}

private static void ExtractResources(Dictionary translations, ResourceSet resourceSet)
{
    foreach (DictionaryEntry res in resourceSet)
    {
        translations[res.Key.ToString()] = res.Value.ToString();
    }
}

Browser Extension

public static class BrowserDetailsExtensions
{
    public static string GetPreferredLanguage(this HttpContext http)
    {
        return http?.Request
            ?.Headers["Accept-Language"].ToString()
            ?.Split(',').FirstOrDefault()
            ?.Split(';').FirstOrDefault();
    }
}

Redux plumbing

CultureActions.ts

import { AppThunkAction } from '../store/index';
import { fetch } from 'domain-task';
import { CALL_API } from 'redux-api-middleware';
import { actionCreatorFactory } from 'typescript-fsa';
import { Translations } from '../reducers/CultureState';
import { Dictionary } from 'lodash';

const API_ADDRESS: string = "/api/Localisation/";
const API_GET_TRANSLATIONS: string = "Translations/";
const API_LANGUAGE_CODE: string = ""; // "pt-BR", "en-GB"
const API_REQUEST_TYPE_GET: string = "GET";

export interface ApiError {
    name: 'ApiError',
    status: number,
    statusText: string,
    response: string
}

export interface GetTranslationsAction {
    type: 'RECEIVE_TRANSLATIONS';
    t: Dictionary;
};

export interface TranslationsApiPosted {
    type: 'Translations_REQUEST_POSTED';
    payload: never;
};

export interface TranslationsApiFailed {
    type: 'Translations_FAILURE';
    payload: ApiError;
    error: true;
};

export interface TranslationsApiSuccess {
    type: 'Translations_SUCCESS';
    t: Dictionary;
}

// Declare a 'discriminated union' type. This guarantees that all references to 'type' properties contain one of the
// declared type strings (and not any other arbitrary string).
export type CultureAction = GetTranslationsAction;

const actionCreator = actionCreatorFactory();
export const translationsAwaitingResponse = actionCreator('Translations_REQUEST_POSTED');
export const translationsSuccess = actionCreator('Translations_SUCCESS');
export const translationsFailure = actionCreator('Translations_FAILURE');

// ACTION CREATORS don't directly mutate state, but they can have external side-effects (such as loading data).
export const cultureActions = {
    getTranslations: (): AppThunkAction => (dispatch, getState) => {
        console.log("REQUEST TRANSLATIONS ACTION")
        const rsaaRequestAuctions = {
            [CALL_API]: {
                credentials: 'same-origin',
                endpoint: API_ADDRESS + API_GET_TRANSLATIONS + API_LANGUAGE_CODE,
                method: API_REQUEST_TYPE_GET,
                types: ['Translations_REQUEST_POSTED', 'Translations_SUCCESS', 'Translations_FAILURE']
            }
        }
        dispatch(rsaaRequestAuctions);
    },
};

CultureStore.ts and the tr() method

import { Dictionary } from "lodash";
import * as He from 'he';
import { Decimal } from "decimal.js";

export type CultureState = {
    strings: Translations
};

export interface Translations {
    t: Dictionary
};

export const emptyCulture: CultureState = {
    strings: {
        t: {}
    }
};

export function tr(culture: CultureState, key: string): string {
    try {
        return He.decode(culture.strings.t[key]) || key;
    } catch (error) {
        console.log("translation for '" + key + "' not found");
        if (key == undefined || key == null) {
            return "- -";
        }
        return "-".repeat(key.length);
    }
}

// Returns the key value itself if no match is found in the resx
export function trFallback(culture: CultureState, key: string): string {
    try {
        return He.decode(culture.strings.t[key]) || key;
    } catch (error) {
        return key;
    }
}

export function trFormat(culture: CultureState, key: string, args: string[]): string {
    return formatString(tr(culture, key), args);
}

export function formatDecimalAmount(subject: Decimal): string {
    return formatAmount(subject.toNumber());
};

export function formatAmount(subject: number): string {
    return subject.toLocaleString(navigator.language, { maximumFractionDigits: 2 });
};

function formatString(subject: string, args: string[]): string {
    if (subject === undefined) {
        return "";
    }
    return subject.replace(/{(\d+)}/g, function (match, number) {
        return typeof args[number] != 'undefined'
            ? args[number]
            : match
        ;
     });
};

CultureReducer.ts

import { Action, Reducer } from 'redux';
import { isType } from 'typescript-fsa';
import { Translations, emptyCulture, CultureState } from './CultureState';
import { ApiError, CultureAction, translationsAwaitingResponse, translationsFailure, translationsSuccess } from '../actions/CultureActions';
import { Dictionary } from 'lodash';

export interface CultureStateR extends CultureState { }

const unloadedState: CultureStateR = emptyCulture;

export const reducer: Reducer = (state: CultureStateR, incomingAction: Action) => {
    const action = incomingAction as CultureAction;

    if (typeof state === undefined) {
        return unloadedState;
    }

    if (isType(incomingAction, translationsAwaitingResponse)) {
        return {
            strings: state.strings
        };
    }
    if (isType(incomingAction, translationsFailure)) {
        return {
            strings: state.strings
        };
    }
    if (isType(incomingAction, translationsSuccess)) {
        const rawTranslations: Dictionary = incomingAction.payload.t;
        return {
            strings: { t: rawTranslations }
        };
    }

    switch (action.type) {
        case 'RECEIVE_TRANSLATIONS':
            return state;

        default:
            break;
    }

    return state || unloadedState;
}

Use in account controller

import { CultureState, tr } from '../reducers/CultureState';
/// Some imports removed for clarity

/// Add CultureState into props for this component
type AccountProps = AccountState & CultureState & typeof accountActions & RouteComponentProps;

export class Account extends React.Component {
}

Use tr() in view tsx file

</pre>
<h2>{tr(this.props, "PersonalDetails")}</h2>
<table>
<tbody>
<tr>
<td>{tr(this.props, "Name")}:</td>
<td>{this.props.user.userName}</td>
<td><a href="/account/profile">{tr(this.props, "Edit")}</a></td>
</tr>
<tr>
<td>{tr(this.props, "Company")}:</td>
<td>{this.props.user.companyName}</td>
</tr>
<tr>
<td>{tr(this.props, "Email")}:</td>
<td>{this.props.user.emailAddress}</td>
</tr>
</tbody>
</table>
<pre>
Categories
development

New Job : Welcome to Screenmedia

Following my request across my network looking for a new job, I started at Screenmedia 3 weeks ago. For those who don’t know, it’s a digital design practice, which means I’m back to consulting, and I get to work with a lot of smart people, covering technology and design. I’m a Technical Architect in the integrations team, so that’s APIs, voice assistants, serverless, analytics, so should be a good wee adventure. I’ve got a few thoughts on the job hunt which I’m sure will come up at ddd.scot and future blog posts. But if you’re currently looking, we’re hiring. If you’ve got any burning career questions, the DDD Scotland panel survey is still open.

I’ve been working on lots of interesting projects already, so groking multiple domains, sometimes multiple for the same customer, Checking the checklists, and reviewing the onboarding process. Sometimes the best change is the one that lets you re-evaluate what you think you know.

 

 

Categories
development programming security

You own your dependencies 

I mentioned as part of my Your API Sucks series that I don’t want your API to be the weak point in my application. But it runs deeper than that.

Every dependency you add to your project is a codebase whose maintenance schedule you need to know, whose security vulnerabilities you and your customers are exposed to, whose existence you depend on – whether it’s a long established company or a guy who wrote 12 lines of Javascript that everyone uses. Know how you’re going to keep using it when the existing support isn’t there.

Because it’s not just one dependency, it’s dependencies all the way down.

Of course, modern software can’t be built without collaboration, without using dependencies written by others, but where you have a choice, always choose the dependency that works closest to how you’d do it yourself, just in case you have to.

Categories
code development programming

Usable APIs follow-up

Following the Usable APIs guided conversation at CodeCraftConf, I wanted to capture some of the thoughts that came out.

Starting an API (as a user or a developer)

  • Does the API documentation include examples of usage (i.e. have they thought about the client)
  • How mature is the API?
  • How well maintained is it?
  • How long does it take to get to the first success (e.g. 200 OK – assuming success doesn’t mean error).
  • What’s the versioning policy?
  • What’s the contract?
  • What’s the shape of the data?

Changing and retiring APIs

  • Never, ever, ever, change the endpoint.
  • Give as much notice as possible of changes (and never negative notice).
  • Provide migration guides to clients, or automation scripts, such as the Python 2to3 migration scripts and guide.
  • Be proactive – there was an example given of a web API (I can’t remember which one, sorry) that changed their endpoint, and created a bot that searched Github for usages of the old URL, and submitted pull requests for the new usage.
  • Deprecate, then kill, once usage falls below a threshold.

Foolproof APIs

Isolation and proxies

  • Log everything to detect unreliability.
  • Make sure proxies are kept up to date with the underlying API, and fail gracefully when the API changes.
  • Expect failure.
  • Data is key – don’t give up more than you have to.
  • Make sure, as a server writer you understand the client, and the network.
Categories
code development

Usable APIs @CodeCraftConf 

Thanks for those of you who came to the APIs conversation. If you want to continue the discussion, these are the questions I had on my cards. 

  1. What’s the worst thing an API has done to you?
  2. What’s the first thing you check when evaluating the use of a new API?
  3. What’s the first thing you do when developing a new API?
  4. Do you approach developing APIs internally differently to one for the public?
  5. How do you make an API foolproof?
  6. How much structure should an API have? Does it need a contract? (see SOAP vs REST)
  7. How should you change an API?
  8. How should you retire an API?
  9. Which stakeholders need input to API upgrades and changes, and how do you engage them?
  10. How do you keep an API consistent if you grow it via tests?
  11. Do bug fixes necessitate a new version?
  12. How do you isolate your application from an insecure API?
  13. How do you isolate your application from an unreliable API?
Categories
development programming ux

Developers are Users Too : Why the User Experience of Your API sucks #yourapisucks

Many thanks to those of you who came along to my talk on why your API sucks. There were some great discussions during and after, and I hope I’ll be seeing slightly fewer reasons to tear my hair out in the near future.

A few things that people mentioned that I want to discuss again, as they’re not on the slides.

APIs still need a view model

There was a question about the API view of your data vs the database view. A good API still follows MVC principles. Just because a View is written in JSON instead of HTML doesn’t make it a model. Isolate it, because otherwise a database change is an API change. You can use automappers to save you having to write convertors, but always create a view for your API so you can handle versions and create consistency.

There was also a big discussion on how deep the hierarchy should be, particularly relating to data obesity. And it’s a different tradeoff if you’re optimising for poor latency, where larger packets with redundant informatio make more sense, or timeliness where small packets that can be built and parsed quickly are preferred. It all depends on your users. It may be the case, as the Trello API does, that returning the deep hierarchy is a request option so that the user can decide.

User Experience doesn’t always mean asking the user

Developers will ask for lots of things that aren’t useful. Sometimes you need to discover what the user needs are. Maybe it’s a test-first design. Maybe it’s a loosely typed API to help you discover what users want to do (think Google search vs Yahoo’s hierarchy – Google is more loosely types so was more likely to return odd results, but Google had lots of data on what people were searching for that it could learn from).

Link suggestions from the audience

Thanks for these suggestions. I’ll have to try them out myself, so I’m putting them here without warranty.

Kong – to consolidate your APIs, and create the polished surface that doesn’t expose the dirty, inconsistent history behind it.

RAML – to design a typed RESTful API up front so you know what it will look like.

Heroku’s HTTP API design guide

Slides from Google Docs

Developers are users too

Slides from Slideshare

Categories
development test

Your API sucks : testing

Finally, you’re ready to launch, you’ve integrated the API, you’ve decoded the documentation, you have passing integration tests, and FAT passed against their test server, with their test data. You go live. And everything breaks.

Strawman testing

Sometimes the test data or the API out of sync with live, and you find you’ve been testing the wrong thing. Maybe the test server is a version ahead of the live API (if they’ve ignored the advice not to version HTTP APIs).

More frustrating is when the data is plain wrong. When the returned record has { Firstname = “Lincoln”, Lastname = “Abraham” }, or an invalid postcode (did you know, The final two letters do not use the letters CIKMOV) which means the test site may require a postcode that your validator will reject.

Tests doomed to succeed

Sometimes the test data is correct, but incomplete. Imagine a postcode lookup where test addresses are all SW… which means you can’t test for:

  • Flat x/y addresses
  • BFPO addresses
  • Irish or other EU addresses

Or imagine a payment processor test stub that always returns success – so you can’t test for declined cards, or 3DS cards, or check that you only accept debit cards.

If your API supports it, put it in your test, and make it available to users for their tests. If it’s not tested, it’s not complete.

 

Categories
development programming security ux

Your API sucks : illegality

No human is illegal, especially your users. I know you were told to make it secure, so you’re filtering input, but some users (so long as you cover Scotland) live in Flat 1/2, so let me put the slash in their address. And let Shaun O’Malley have an apostrophe. Not only does this make for a poor user experience that the developers using your API either have to pass on to their users, or find a way to deal with – if you let them know what encodings you support – but that might be against security policy.

Worse than that however, a policy like that is a red flag to hackers. If you don’t allow /, do you allow \, or do you filter DROP? It’s a sign that there’s a wormhole through your code with a single line of defence against SQL Injection, or script injection. As a developer, it worries me, from a usability and a security perspective.

 

Stop sending the wrong message and making my users illegal.

Categories
code development

Your API sucks : documentation

Your API is perfect. It just needs a little finesse on the documentation. It’s fine, just farm it out to Johnny Clueless. After all, good APIs, like all good code, is self documenting. It’s just those damn users that keep asking what you mean when you said that, or wanting to know what all the options are. They’re very demanding.

Documentation out of sync with the code

getPostcode(double opt)
This method takes returns the current longitude and latitude of the user based on the information provided by their device.

Teaching Granny to suck eggs

Bad API documentation is like bad comments, it tells you everything except what you need to know, such as the return format.

getAddressFromPostcode(string postcode)
This method takes a postcode and returns the associated address.

Be explicit

However, don’t use this as an excuse to avoid documentation. Good documentation helps users access your API without using up your time. Be explicit about options, about which standards you support, and where necessary, how you support them. Even for simple things.

explicit-documentationThis image represents an authentication call to retrieve an API token that can be used when making further calls to an API. Each of the arrows represents an email discussion where the user needed clarification. This can be simplified with the right explicit statement:

To retrieve your API token, a POST request must be made to the HTTPS URL provided, with the following key-value pairs x-www-form-urlencoded into the Body of the request:

Key                       Value

client_id              your_username

client_secret      your_password

grant_type          client_credentials

If successful, this will return a JSON body in the response. The value associated with the key access_token should be appended to all subsequent requests as described below.

The use of italics is defined appropriately, and user specific information is supplied in a format to match the documented table.