Categories
code development ux

Blank spaces : how to deal with nothing in API design

Consider the following HTTP endpoint:

GET /{user}/appointments

How do we deal with nothing?

Nothingman

đŸš« If the user doesn’t currently exist, return 404 – this informs the caller that nothing can be done with this resource until it is created or recreated.

🙈 If the user exists, but we want to protect against username enumeration, return 404 – this removes a route for malicious agents to identify actual users, perhaps prior to a password brute force. They may decide this endpoint is less likely to have the full protections afforded to the login endpoint. This endpoint should also avoid indirect enumeration, for example returning immediately for “user doesn’t exist” and delayed for “user exists but we’re pretending because security”

🔒 If the user exists but the caller doesn’t have permission to see their appointments, return 403 – caller will have to login, or ask someone who has access.

Empty time

Given the selected user exists

🔐 If this user does not support appointments, return 404 – these resources can’t be found.

🗓 If this user does support appointments, but there are none, return an empty list.

note : some APIs will return 204 No Content in this scenario. 204 should only be used for POST or PUT requests to indicate server action was a success, and there’s no data to send back

Empty space

Given the selected user exists
And they have at least 1 valid appointment (see business for what “valid” means)

đŸ“ș If the appointment has no location (because online conference links are saved elsewhere in the appointment body) then no location property should be returned

❔ If the appointment has no location (because it is unknown) then the location property should be returned with no data (the empty string)

Advertisement
Categories
development

Your API Sucks – The ‘default’ API : REST and repeat

I’ve recently got a Fitbit, and I use the website for logging food. I noticed something about the URL scheme that I’ve seen elsewhere and it made me think about how to design default points in URL schemes.

On Fitbit, ‘/foods/log’ opens the page for today’s food, and there are controls to navigate to yesterday and beyond, when the URL changes to ‘/foods/log/{date}’. So far, so simple. But when I navigate back to today, the URL still contains the date. My browser is set to reload pages from last time, so the first URL always gives me today. The second, doesn’t.

Should we set defaults that can change based on context? Or should the default redirect to the appropriate context? Or should we always be explicit? Should default URLs automatically redirect?

If I load the page before midnight but submit after, the screen and the URL indicate different dates. What is the right date?

UPDATE: The new Fitbit logging page, which appears occasionally at `/foods/log` takes the date as an explicit parameter (it’s a drop-down on the page), so no longer has a default. This is a much better strategy for submitting data

Categories
code development programming

Translations with React, Redux and Asp.Net

In a recent project, which was the first time I’ve used React and Redux in anger, we had a requirement to support 2 different languages, in the .Net backend for emails and PDFs, and in the React frontend.

As the translators we used were used to resx files, we wanted to use those as the master source. In other projects we’ve done the javascript translations via a pre-compilation step into static javascript files, but since redux has it’s own store, I decided to see if we could use that to store and process the translations.

This approach has the advantage that we can use the redux state to translate the site automatically when a new language is selected, without having to reload any pages. For an application that depends on up-to-date data, and had to operate on por data connections, avoiding reloads is essential.

The redux examples shown here use TypeScript, which certainly helped in our development process, but there’s a lot of gotchas getting the dotnet new react-redux template and many 3rd party react and redux libraries working nicely with TypeScript. You can make it work, but it’s definitely not an out-of-the-box solution in most cases.

The solution consists of 3 parts:

  1. The Asp.Net Core 2.0 controller to translate the resx data into JSON (in this example, the resource file is called UserFacingStrings.resx and the default is en-US,
  2. The Redux configuration to retrieve the data and populate it into the store; and
  3. A React function that reads the translation from the store and presents it to the user.

The solution below attempts to get the localisation from the user headers, so you’ll need to enable localisation in your Startup.cs, but depending on your use case, you may want save user language settings in the user’s account data, or in a cookie or other storage in their browser.

ASP.Net Core API controller

[Produces("application/json")]
[Route("api/Localisation")]
public class LocalisationController : Controller
{
    private static Dictionary _cultureCache = new Dictionary();

    /// Get all known translations
    /// Choose a culture (e.g. `en-US` )
    /// will default to browser culture if not specified
    [HttpGet("[action]/{culture?}")]
    public IActionResult Translations(string culture)
    {
        try
        {
            var cultureInfo = CultureInfo.CurrentUICulture;
            try
            {
                string preferredLanguage = HttpContext.GetPreferredLanguage();
                if (!string.IsNullOrWhiteSpace(culture))
                {
                    cultureInfo = CultureInfo.GetCultureInfo(culture);
                }
                else if (!string.IsNullOrWhiteSpace(preferredLanguage))
                {
                    cultureInfo = CultureInfo.GetCultureInfo(preferredLanguage);
                }
            }
            catch (CultureNotFoundException)
            {
                // Fallback to en-US
            }

            if (_cultureCache.Keys.Contains(cultureInfo.ToString()))
            {
                return new OkObjectResult(new { t = _cultureCache[cultureInfo.ToString()] });
            }

            var translations = new Dictionary();

            // Insert all resources, to then be overwritten
            if (cultureInfo.TwoLetterISOLanguageName != "en")
            {
                ExtractResources(translations, UserFacingStrings.ResourceManager.GetResourceSet(CultureInfo.GetCultureInfo("en-US"), true, true));
            }

            ExtractResources(translations, UserFacingStrings.ResourceManager.GetResourceSet(cultureInfo, true, true));

            _cultureCache[cultureInfo.ToString()] = translations;

            return new OkObjectResult(new { t = translations });
    }
    catch (Exception)
    {
        return NotFound();
    }
}

private static void ExtractResources(Dictionary translations, ResourceSet resourceSet)
{
    foreach (DictionaryEntry res in resourceSet)
    {
        translations[res.Key.ToString()] = res.Value.ToString();
    }
}

Browser Extension

public static class BrowserDetailsExtensions
{
    public static string GetPreferredLanguage(this HttpContext http)
    {
        return http?.Request
            ?.Headers["Accept-Language"].ToString()
            ?.Split(',').FirstOrDefault()
            ?.Split(';').FirstOrDefault();
    }
}

Redux plumbing

CultureActions.ts

import { AppThunkAction } from '../store/index';
import { fetch } from 'domain-task';
import { CALL_API } from 'redux-api-middleware';
import { actionCreatorFactory } from 'typescript-fsa';
import { Translations } from '../reducers/CultureState';
import { Dictionary } from 'lodash';

const API_ADDRESS: string = "/api/Localisation/";
const API_GET_TRANSLATIONS: string = "Translations/";
const API_LANGUAGE_CODE: string = ""; // "pt-BR", "en-GB"
const API_REQUEST_TYPE_GET: string = "GET";

export interface ApiError {
    name: 'ApiError',
    status: number,
    statusText: string,
    response: string
}

export interface GetTranslationsAction {
    type: 'RECEIVE_TRANSLATIONS';
    t: Dictionary;
};

export interface TranslationsApiPosted {
    type: 'Translations_REQUEST_POSTED';
    payload: never;
};

export interface TranslationsApiFailed {
    type: 'Translations_FAILURE';
    payload: ApiError;
    error: true;
};

export interface TranslationsApiSuccess {
    type: 'Translations_SUCCESS';
    t: Dictionary;
}

// Declare a 'discriminated union' type. This guarantees that all references to 'type' properties contain one of the
// declared type strings (and not any other arbitrary string).
export type CultureAction = GetTranslationsAction;

const actionCreator = actionCreatorFactory();
export const translationsAwaitingResponse = actionCreator('Translations_REQUEST_POSTED');
export const translationsSuccess = actionCreator('Translations_SUCCESS');
export const translationsFailure = actionCreator('Translations_FAILURE');

// ACTION CREATORS don't directly mutate state, but they can have external side-effects (such as loading data).
export const cultureActions = {
    getTranslations: (): AppThunkAction => (dispatch, getState) => {
        console.log("REQUEST TRANSLATIONS ACTION")
        const rsaaRequestAuctions = {
            [CALL_API]: {
                credentials: 'same-origin',
                endpoint: API_ADDRESS + API_GET_TRANSLATIONS + API_LANGUAGE_CODE,
                method: API_REQUEST_TYPE_GET,
                types: ['Translations_REQUEST_POSTED', 'Translations_SUCCESS', 'Translations_FAILURE']
            }
        }
        dispatch(rsaaRequestAuctions);
    },
};

CultureStore.ts and the tr() method

import { Dictionary } from "lodash";
import * as He from 'he';
import { Decimal } from "decimal.js";

export type CultureState = {
    strings: Translations
};

export interface Translations {
    t: Dictionary
};

export const emptyCulture: CultureState = {
    strings: {
        t: {}
    }
};

export function tr(culture: CultureState, key: string): string {
    try {
        return He.decode(culture.strings.t[key]) || key;
    } catch (error) {
        console.log("translation for '" + key + "' not found");
        if (key == undefined || key == null) {
            return "- -";
        }
        return "-".repeat(key.length);
    }
}

// Returns the key value itself if no match is found in the resx
export function trFallback(culture: CultureState, key: string): string {
    try {
        return He.decode(culture.strings.t[key]) || key;
    } catch (error) {
        return key;
    }
}

export function trFormat(culture: CultureState, key: string, args: string[]): string {
    return formatString(tr(culture, key), args);
}

export function formatDecimalAmount(subject: Decimal): string {
    return formatAmount(subject.toNumber());
};

export function formatAmount(subject: number): string {
    return subject.toLocaleString(navigator.language, { maximumFractionDigits: 2 });
};

function formatString(subject: string, args: string[]): string {
    if (subject === undefined) {
        return "";
    }
    return subject.replace(/{(\d+)}/g, function (match, number) {
        return typeof args[number] != 'undefined'
            ? args[number]
            : match
        ;
     });
};

CultureReducer.ts

import { Action, Reducer } from 'redux';
import { isType } from 'typescript-fsa';
import { Translations, emptyCulture, CultureState } from './CultureState';
import { ApiError, CultureAction, translationsAwaitingResponse, translationsFailure, translationsSuccess } from '../actions/CultureActions';
import { Dictionary } from 'lodash';

export interface CultureStateR extends CultureState { }

const unloadedState: CultureStateR = emptyCulture;

export const reducer: Reducer = (state: CultureStateR, incomingAction: Action) => {
    const action = incomingAction as CultureAction;

    if (typeof state === undefined) {
        return unloadedState;
    }

    if (isType(incomingAction, translationsAwaitingResponse)) {
        return {
            strings: state.strings
        };
    }
    if (isType(incomingAction, translationsFailure)) {
        return {
            strings: state.strings
        };
    }
    if (isType(incomingAction, translationsSuccess)) {
        const rawTranslations: Dictionary = incomingAction.payload.t;
        return {
            strings: { t: rawTranslations }
        };
    }

    switch (action.type) {
        case 'RECEIVE_TRANSLATIONS':
            return state;

        default:
            break;
    }

    return state || unloadedState;
}

Use in account controller

import { CultureState, tr } from '../reducers/CultureState';
/// Some imports removed for clarity

/// Add CultureState into props for this component
type AccountProps = AccountState & CultureState & typeof accountActions & RouteComponentProps;

export class Account extends React.Component {
}

Use tr() in view tsx file

</pre>
<h2>{tr(this.props, "PersonalDetails")}</h2>
<table>
<tbody>
<tr>
<td>{tr(this.props, "Name")}:</td>
<td>{this.props.user.userName}</td>
<td><a href="/account/profile">{tr(this.props, "Edit")}</a></td>
</tr>
<tr>
<td>{tr(this.props, "Company")}:</td>
<td>{this.props.user.companyName}</td>
</tr>
<tr>
<td>{tr(this.props, "Email")}:</td>
<td>{this.props.user.emailAddress}</td>
</tr>
</tbody>
</table>
<pre>
Categories
development

New Job : Welcome to Screenmedia

Following my request across my network looking for a new job, I started at Screenmedia 3 weeks ago. For those who don’t know, it’s a digital design practice, which means I’m back to consulting, and I get to work with a lot of smart people, covering technology and design. I’m a Technical Architect in the integrations team, so that’s APIs, voice assistants, serverless, analytics, so should be a good wee adventure. I’ve got a few thoughts on the job hunt which I’m sure will come up at ddd.scot and future blog posts. But if you’re currently looking, we’re hiring. If you’ve got any burning career questions, the DDD Scotland panel survey is still open.

I’ve been working on lots of interesting projects already, so groking multiple domains, sometimes multiple for the same customer, Checking the checklists, and reviewing the onboarding process. Sometimes the best change is the one that lets you re-evaluate what you think you know.

 

 

Categories
development programming security

You own your dependencies 

I mentioned as part of my Your API Sucks series that I don’t want your API to be the weak point in my application. But it runs deeper than that.

Every dependency you add to your project is a codebase whose maintenance schedule you need to know, whose security vulnerabilities you and your customers are exposed to, whose existence you depend on – whether it’s a long established company or a guy who wrote 12 lines of Javascript that everyone uses. Know how you’re going to keep using it when the existing support isn’t there.

Because it’s not just one dependency, it’s dependencies all the way down.

Of course, modern software can’t be built without collaboration, without using dependencies written by others, but where you have a choice, always choose the dependency that works closest to how you’d do it yourself, just in case you have to.

Categories
code development programming

Usable APIs follow-up

Following the Usable APIs guided conversation at CodeCraftConf, I wanted to capture some of the thoughts that came out.

Starting an API (as a user or a developer)

  • Does the API documentation include examples of usage (i.e. have they thought about the client)
  • How mature is the API?
  • How well maintained is it?
  • How long does it take to get to the first success (e.g. 200 OK – assuming success doesn’t mean error).
  • What’s the versioning policy?
  • What’s the contract?
  • What’s the shape of the data?

Changing and retiring APIs

  • Never, ever, ever, change the endpoint.
  • Give as much notice as possible of changes (and never negative notice).
  • Provide migration guides to clients, or automation scripts, such as the Python 2to3 migration scripts and guide.
  • Be proactive – there was an example given of a web API (I can’t remember which one, sorry) that changed their endpoint, and created a bot that searched Github for usages of the old URL, and submitted pull requests for the new usage.
  • Deprecate, then kill, once usage falls below a threshold.

Foolproof APIs

Isolation and proxies

  • Log everything to detect unreliability.
  • Make sure proxies are kept up to date with the underlying API, and fail gracefully when the API changes.
  • Expect failure.
  • Data is key – don’t give up more than you have to.
  • Make sure, as a server writer you understand the client, and the network.
Categories
code development

Usable APIs @CodeCraftConf 

Thanks for those of you who came to the APIs conversation. If you want to continue the discussion, these are the questions I had on my cards. 

  1. What’s the worst thing an API has done to you?
  2. What’s the first thing you check when evaluating the use of a new API?
  3. What’s the first thing you do when developing a new API?
  4. Do you approach developing APIs internally differently to one for the public?
  5. How do you make an API foolproof?
  6. How much structure should an API have? Does it need a contract? (see SOAP vs REST)
  7. How should you change an API?
  8. How should you retire an API?
  9. Which stakeholders need input to API upgrades and changes, and how do you engage them?
  10. How do you keep an API consistent if you grow it via tests?
  11. Do bug fixes necessitate a new version?
  12. How do you isolate your application from an insecure API?
  13. How do you isolate your application from an unreliable API?
Categories
development programming ux

Developers are Users Too : Why the User Experience of Your API sucks #yourapisucks

Many thanks to those of you who came along to my talk on why your API sucks. There were some great discussions during and after, and I hope I’ll be seeing slightly fewer reasons to tear my hair out in the near future.

A few things that people mentioned that I want to discuss again, as they’re not on the slides.

APIs still need a view model

There was a question about the API view of your data vs the database view. A good API still follows MVC principles. Just because a View is written in JSON instead of HTML doesn’t make it a model. Isolate it, because otherwise a database change is an API change. You can use automappers to save you having to write convertors, but always create a view for your API so you can handle versions and create consistency.

There was also a big discussion on how deep the hierarchy should be, particularly relating to data obesity. And it’s a different tradeoff if you’re optimising for poor latency, where larger packets with redundant informatio make more sense, or timeliness where small packets that can be built and parsed quickly are preferred. It all depends on your users. It may be the case, as the Trello API does, that returning the deep hierarchy is a request option so that the user can decide.

User Experience doesn’t always mean asking the user

Developers will ask for lots of things that aren’t useful. Sometimes you need to discover what the user needs are. Maybe it’s a test-first design. Maybe it’s a loosely typed API to help you discover what users want to do (think Google search vs Yahoo’s hierarchy – Google is more loosely types so was more likely to return odd results, but Google had lots of data on what people were searching for that it could learn from).

Link suggestions from the audience

Thanks for these suggestions. I’ll have to try them out myself, so I’m putting them here without warranty.

Kong – to consolidate your APIs, and create the polished surface that doesn’t expose the dirty, inconsistent history behind it.

RAML – to design a typed RESTful API up front so you know what it will look like.

Heroku’s HTTP API design guide

Slides from Google Docs

Developers are users too

Slides from Slideshare

Categories
development test

Your API sucks : testing

Finally, you’re ready to launch, you’ve integrated the API, you’ve decoded the documentation, you have passing integration tests, and FAT passed against their test server, with their test data. You go live. And everything breaks.

Strawman testing

Sometimes the test data or the API out of sync with live, and you find you’ve been testing the wrong thing. Maybe the test server is a version ahead of the live API (if they’ve ignored the advice not to version HTTP APIs).

More frustrating is when the data is plain wrong. When the returned record has { Firstname = “Lincoln”, Lastname = “Abraham” }, or an invalid postcode (did you know, The final two letters do not use the letters CIKMOV) which means the test site may require a postcode that your validator will reject.

Tests doomed to succeed

Sometimes the test data is correct, but incomplete. Imagine a postcode lookup where test addresses are all SW… which means you can’t test for:

  • Flat x/y addresses
  • BFPO addresses
  • Irish or other EU addresses

Or imagine a payment processor test stub that always returns success – so you can’t test for declined cards, or 3DS cards, or check that you only accept debit cards.

If your API supports it, put it in your test, and make it available to users for their tests. If it’s not tested, it’s not complete.

 

Categories
development programming security ux

Your API sucks : illegality

No human is illegal, especially your users. I know you were told to make it secure, so you’re filtering input, but some users (so long as you cover Scotland) live in Flat 1/2, so let me put the slash in their address. And let Shaun O’Malley have an apostrophe. Not only does this make for a poor user experience that the developers using your API either have to pass on to their users, or find a way to deal with – if you let them know what encodings you support – but that might be against security policy.

Worse than that however, a policy like that is a red flag to hackers. If you don’t allow /, do you allow \, or do you filter DROP? It’s a sign that there’s a wormhole through your code with a single line of defence against SQL Injection, or script injection. As a developer, it worries me, from a usability and a security perspective.

 

Stop sending the wrong message and making my users illegal.