Tag: API

API driven development: eating your own dog food

Alex Bilbie / / Talk

On Tuesday I presented at the first ever PHP North East conference in Newcastle. Hosted in the fantastic Tyneside Cinema (which is the last surviving Newsreel theatre still operating as a cinema full-time in the UK), it was a very well organised event and I’m incredibly grateful to the PHP North East group for accepting my talk proposal.

My topic was “API driven development: Eating your own dog food”, I’ve embedded my slides below:

Unfortunately they weren’t able to video my talk so my slides may seem a little out of context but hopefully you can get the overall gist of what I talked about.

Once again, I’d like to say thank you to Anthony Sterling and the PHPNE crew for putting on a wonderful event.

A guide to OAuth grants

Alex Bilbie / / In depth

OAuth by it’s nature is a very flexible standard and can adapted to work in many different scenarios. The core specification describes four authorisation grants:

  • Authorisation code grant
  • Implicit grant
  • Resource owner credentials grant
  • Client credentials grant

The specification also details another grant called the refresh token grant.

Furthermore there are a number of other grants that have gone through the IETF ratification process (none of which at the time of writing have been formally standardised):

  • Message authentication code (MAC) tokens
  • SAML 2.0 Bearer Assertion Profiles
  • JSON web token grant

The end goal of each of these grants (except the refresh token grant) is for the client application to have an access token (which represents a user’s permission for the client to access their data) which it can use to authenticate a request to an API endpoint.

In this post I’m going to describe each of the above grants and their appropriate use cases.

As a refresher here is a quick glossary of OAuth terms (taken from the core spec):

  • Resource owner (a.k.a. the User) – An entity capable of granting access to a protected resource. When the resource owner is a person, it is referred to as an end-user.
  • Resource server (a.k.a. the API server) – The server hosting the protected resources, capable of accepting and responding to protected resource requests using access tokens.
  • Client – An application making protected resource requests on behalf of the resource owner and with its authorisation. The term client does not imply any particular implementation characteristics (e.g. whether the application executes on a server, a desktop, or other devices).
  • Authorisation server – The server issuing access tokens to the client after successfully authenticating the resource owner and obtaining authorisation.

Authorisation code grant (section 4.1)

The authorisation code grant is the grant that most people think of when OAuth is described.

If you’ve ever signed into a website or application with your Twitter/Facebook/Google/(insert major Internet company here) account then you’ll have experienced using this grant.

Essentially a user will click on a “sign in with Facebook” (or other IdP) and then be redirected from the application/website (the “client”) to the IdP authorisation server. The user will then sign in to the IdP with their credentials, and then – if they haven’t already – authorise the client to allow it to use the user’s data (such as their name, email address, etc). If they authorise the request the user will be redirected back to the client with a token (called the authorisation code) in the query string (e.g. http://client.com/redirect?code=XYZ123) which the client will capture and exchange for an access token in the background.

This grant is suitable where the resource owner is a user and they are using a client which is allows a user to interact with a website in a browser. An obvious example is the client being another website, but desktop applications such as Spotify or Reeder use embedded browsers.

Some mobile applications use this flow and again use an embedded browser (or redirect the user to the native browser and then are redirected back to the app using a custom protocol).

In this grant the access token is kept private from the resource owner.

If you have a mobile application that is for your own service (such as the official Spotify or Facebook apps on iOS) it isn’t appropriate to use this grant as the app itself should already be trusted by your authorisation server and so the _resource owner credentials grant would be more appropriate.

Implicit grant (section 4.2)

The implicit grant is similar to the authentication code grant described above. The user will be redirected in a browser to the IdP authorisation server, sign in, authorise the request but instead of being returned to the client with an authentication code they are redirected with an access token straight away.

The purpose of the implicit grant is for use by clients which are not capable of keeping the client’s own credentials secret; for example a JavaScript only application.

If you decide to implement this grant then you must be aware that the access token should be treated as “public knowledge” (like a public RSA key) and therefore it must have a very limited permissions when interacting with the API server. For example an access token that was granted using the authentication code grant could have permission to be used to delete resources owned by the user, however an access token granted through the implicit flow should only be able to “read” resources and never perform any destructive operations.

Resource owner credentials grant (section 4.3)

When this grant is implemented the client itself will ask the user for their username and password (as opposed to being redirected to an IdP authorisation server to authenticate) and then send these to the authorisation server along with the client’s own credentials. If the authentication is successful then the client will be issued with an access token.

This grant is suitable for trusted clients such as a service’s own mobile client (for example Spotify’s iOS app). You could also use this in software where it’s not easy to implement the authorisation code – for example we bolted this authorisation grant into OwnCloud so we could retrieve details about a user that we couldn’t access over LDAP from the university’s Active Directory server.

Client credentials grant (section 4.4)

This grant is similar to the resource owner credentials grant except only the client’s credentials are used to authenticate a request for an access token. Again this grant should only be allowed to be used by trusted clients.

This grant is suitable for machine-to-machine authentication, for example for use in a cron job which is performing maintenance tasks over an API. Another example would be a client making requests to an API that don’t require user’s permission.

When someone visits a member of staff’s page on the University of Lincoln staff directory the website uses it’s own access token (that was generated using this grant) to authenticate a request to the API server to get the data about the member of staff that is used to build the page. When a member of staff signs in to update their profile however their own access token is used to retrieve and update their data. Therefore there is a good separation of concerns and we can easily restrict permissions that each type of access token has.

Refresh token grant (section 1.5)

The OAuth 2.0 specification details a fifth grant which can be used to “refresh” (i.e. renew) an access token which has expired.

Authorisation servers which support this grant will also issue a “refresh token” when it returns an access token to a client. When the access token expires instead of sending the user back through the authorisation code grant the client can use to the refresh token to retrieve a new access token with the same permissions as the old one.

My problem with the grant is that it means the client has to maintain state of each token and then either on a cron job keep access tokens up to date or when it tries to make a request and it fails then go and update the access token and repeat the request.

I personally prefer to issue access tokens that last longer than the user’s session cookie so that when they next sign in they’ll be issued a new token anyway.

The extension grants

The following grants are currently going through the standardisation process.

N.B. My explanation from here on is how ** I ** (as a developer) am interpreting the specifications – all three of the following grants specs need further work in my opinion to better explain their purpose and how they compare to the core grants.

MAC token grant (draft 3 at time of writing)

The MAC token grant can be used alongside another grant (the specification describes using it alongside the authentication code grant) to further improve the security of requests to the API server by including the addition of a MAC (message authentication code) signature along with the access token in order to both authenticate the request and prove the identity of the client making the request (because it prevents tampering and forgery). In some ways it is similar to the OAuth 1.0 request process.

When the authorisation server returns an access token to a client it also includes a key which the client uses to generate the MAC signature. The MAC signature is generated by combining the parameters of the request and then hashing them against the key.

This grant could be used when additional security measures are needed to ensure that the client querying an API is definitely who it is identifying itself as. It can also prevent access tokens being used by unauthorised clients (if an access token has been compromised by a man in the middle attack) because a valid signature (which can only be generated by the client which knows the MAC key associated with the access token) is required for each request.

SAML 2.0 Bearer Assertion Profiles (draft 15 at time of writing)

A SAML assertion is an XML payload which contains a security token. They are issued by identity providers and consumed by a service provider who relies on its content to identify the assertion’s subject for security-related purposes. It is quite literally an assertion as to who someone (or what something) is.

This grant allows a client to exchange an existing SAML assertion for an access token, meaning the user doesn’t have to authenticate again with an authorisation server. In some ways it resembles the concept of the refresh token grant.

An example use case of this grant would be a publishers website receiving an assertion from the UK Federation (after a user from a UK education institution has authenticated with their institution’s own resource server), converting the assertion into an access token and then querying the users institution’s API server to request additional details about the user.

JSON web token grant (draft 6 at time of writing)

The JSON web token grant is similar to the SAML assertion grant but uses JSON as the serialisation format instead of XML. This means that a JWT (JSON web token) can comfortable fit inside a HTTP authorization header.

A potential use case could be a web based client application that also has a mobile version, the web version could somehow share the existing access token with the mobile application which in turn requests its own access token. Possibly? Maybe? I’ll admit I can’t think of a better example.

Securing your API with OAuth 2.0

Alex Bilbie / / Code, In depth

In a previous post I announced my new OAuth 2.0 PHP libraries.

In this post I will show you how to use the server library to secure a simple API with OAuth 2.0.

Install the library

The recommended way of installing the library is via Composer.

If you already have a composer.json file in your root then add ”lncd/oauth2”: “*” in the require object. Then run composer update.

Otherwise create a new file in your project root called composer.json add set the contents to:

{
    "require": {
        "lncd/OAuth2": "*"
    }
}

Now, assuming you have installed Composer run composer install.

Ensure now that you’ve set up your project to autoload composer packages.

You could alternatively add the library as a git submodule or download a zip.

Set up the database

To setup the database just import sql/mysql.sql

Create the storage models

In order to retrieve data from the database you should create classes which implement the following interfaces:

  • \OAuth2\Storage\ScopeInterface
  • \OAuth2\Storage\SessionInterface

Hooking it all up

Setting up the library is simple, just create a new instance of \OAuth2\ResourceServer and pass in your storage models.

// Include the storage models
include 'model_scope.php';
include 'model_session.php';

// Initiate the Request handler
$request = new \OAuth2\Util\Request();

// Initiate the auth server with the models
$server = new \OAuth2\ResourceServer(new SessionModel, new ScopeModel);

Checking for valid access tokens

Before your API responds you need to check that an access token has been presented with the request (either in the query string ?access_token=abcdef or as an authorization header Authorization: bearer abcdef).

If you’re using a framework such as Laravel or CodeIgniter you could use a route filter to do this, or have a custom controller which other controllers extend from. In this example I’m using the Slim framework and I’m going to create a simple route middleware which is run before each endpoint function.

$checkToken = function () use ($server) {

    return function() use ($server)
    {
        // Test for token existance and validity
        try {
            $server->isValid();
        }

        // The access token is missing or invalid...
        catch (\OAuth2\Exception\InvalidAccessTokenException $e)
        {
            $app = \Slim\Slim::getInstance();
            $res = $app->response();
            $res['Content-Type'] = 'application/json';
            $res->status(403);

            $res->body(json_encode(array(
                'error' =>  $e->getMessage()
            )));
        }
    };

};

When $server->isValid() is called the library will run the following tasks:

  • Check if an access token is present in the query string
    • If not, check if a base64 encoded access token is contained in an authorization header.
      • If not, throw \OAuth2\Exception\InvalidAccessTokenException
  • Check if the access token is valid with the database
    • If not, throw \OAuth2\Exception\InvalidAccessTokenException
  • If the access token is valid:
    • Get the owner type (e.g. “user” or “client”) and their ID
    • Get a list of any scopes that are associated with the access token

Assuming an exception isn’t thrown you can then use the following functions in your API code:

  • getOwnerType() – This will return the type of the owner of the access token. For example if a user has authorized another client to use their resources the owner type would be “user”.
  • getOwnerId() – This will return the ID of the access token owner. You can use this to check if the owner has permission to do take some sort of action (such as retrieve a document or upload a file to a folder).
  • hasScope() – You can use this function to see if a specific scope (or several scopes) has been associated with the access token. You can use this to limit the contents of an API response or prevent access to an API endpoint without the correct scope.

A simple example

This example endpoint will return a user’s information if a valid access token is present. If the access token has the user.contact it will return additional information.

$app->get('/user/:id', $checkToken(), function ($id) use ($server, $app) {

    $user_model = new UserModel();

    $user = $user_model->getUser($id);

    if ( ! $user)
    {
        $res = $app->response();
        $res->status(404);
        $res['Content-Type'] = 'application/json';
        $res->body(json_encode(array(
            'error' => 'User not found'
        )));
    }

    else
    {
        // Basic response
        $response = array(
            'error' => null,
            'result'    =>  array(
                'user_id'   =>  $user['id'],
                'firstname' =>  $user['firstname'],
                'lastname'  =>  $user['lastname']
            )
        );

        // If the acess token has the "user.contact" access token include
        //  an email address and phone numner
        if ($server->hasScope('user.contact'))
        {
            $response['result']['email'] = $user['email'];
            $response['result']['phone'] = $user['phone'];
        }

        // Respond
        $res = $app->response();
        $res['Content-Type'] = 'application/json';

        $res->body(json_encode($response));
    }

});

Limiting an endpoint to a specific owner type

In this example, only a user’s access token is valid:

$app->get('/user', $checkToken(), function () use ($server, $app) {

    $user_model = new UserModel();

    // Check the access token's owner is a user
    if ($server->getOwnerType() === 'user')
    {
        // Get the access token owner's ID
        $userId = $server->getOwnerId();

        $user = $user_model->getUser($userId);

        // If the user can't be found return 404
        if ( ! $user)
        {
            $res = $app->response();
            $res->status(404);
            $res['Content-Type'] = 'application/json';
            $res->body(json_encode(array(
                'error' => 'Resource owner not found'
            )));
        }

        // A user has been found
        else
        {
            // Basic response
            $response = array(
                'error' => null,
                'result'    =>  array(
                    'user_id'   =>  $user['id'],
                    'firstname' =>  $user['firstname'],
                    'lastname'  =>  $user['lastname']
                )
            );

            // If the acess token has the "user.contact" access token include
            //  an email address and phone numner
            if ($server->hasScope('user.contact'))
            {
                $response['result']['email'] = $user['email'];
                $response['result']['phone'] = $user['phone'];
            }

            // Respond
            $res = $app->response();
            $res['Content-Type'] = 'application/json';

            $res->body(json_encode($response));
        }
    }

    // The access token isn't owned by a user
    else
    {
        $res = $app->response();
        $res->status(403);
        $res['Content-Type'] = 'application/json';
        $res->body(json_encode(array(
            'error' => 'Only access tokens representing users can use this endpoint'
        )));
    }

});

You might use an API function like this to allow a client to discover who a user is after they’ve signed into your authorization endpoint (see an example of how to do this here).

Limiting an endpoint to a specific owner type and scope

In this example, the endpoint will only respond to access tokens that are owner by client applications and that have the scope users.list.

$app->get('/users', $checkToken(), function () use ($server, $app) {

    $user_model = new UserModel();

    $users = $user_model->getUsers();

    // Check the access token owner is a client
    if ($server->getOwnerType() === 'client' && $server->hasScope('users.list'))
    {
        $response = array(
            'error' => null,
            'results'   =>  array()
        );

        $i = 0;
        foreach ($users as $k => $v)
        {
            // Basic details
            $response['results'][$i]['user_id'] = $v['id'];
            $response['results'][$i]['firstname'] = $v['firstname'];
            $response['results'][$i]['lastname'] = $v['lastname'];

            // Include additional details with the right scope
            if ($server->hasScope('user.contact'))
            {
                $response['results'][$i]['email'] = $v['email'];
                $response['results'][$i]['phone'] = $v['phone'];
            }

            $i++;
        }

        $res = $app->response();
        $res['Content-Type'] = 'application/json';

        $res->body(json_encode($response));
    }

    // Access token owner isn't a client or doesn't have the correct scope
    else
    {
        $res = $app->response();
        $res->status(403);
        $res['Content-Type'] = 'application/json';
        $res->body(json_encode(array(
            'error' => 'Only access tokens representing clients can use this endpoint'
        )));
    }

});

You might secure an endpoint in this way to only allow specific clients (such as your applications’ main website) access to private APIs.

Hopefully you will see how easy it is to secure an API with OAuth 2.0 and how you can use scopes to limit response contents or access to endpoints.

You can download a complete working example here – https://github.com/lncd/oauth2-example-resource-server.

Hawk: a new HTTP authentication scheme

Alex Bilbie / / Uncategorized

Eran Hammer (formerly editor of the OAuth specifications) has introduced a new HTTP authentication scheme called Hawk.

Hawk is described as a HTTP authentication scheme using a message authentication code (MAC) algorithm to provide partial HTTP request cryptographic verification. It is related to his other new project Oz which is a web authorisation protocol promoted as an alternative to OAuth. Eran has made it clear that he doesn’t want to write a new specification after the leaving the OAuth working group and so he is intending to describe Hawk (and Oz) through code and leave it up to someone else to write a formal specification.

What is Hawk?

An overview of Hawk is provided in the README file in the repository:

Hawk is an HTTP authentication scheme providing a method for making authenticated HTTP requests with partial cryptographic verification of the request, covering the HTTP method, request URI, and host.

Similar to the HTTP Basic access authentication scheme, the Hawk scheme utilizes a set of client credentials which include an identifier and key. However, in contrast with the Basic scheme, the key is never included in authenticated requests but is used to calculate a request MAC value which is included instead.

The Hawk scheme requires the establishment of a shared symmetric key between the client and the server, which is beyond the scope of this module. Typically, the shared credentials are established via an initial TLS-protected phase or derived from some other shared confidential information available to both the client and the server.

The primary design goals of this mechanism are to:

  • simplify and improve HTTP authentication for services that are unwilling or unable to employ TLS for every request,
  • secure the shared credentials against leakage when sent over a secure channel to the wrong server (e.g., when the client uses some form of dynamic configuration to determine where to send an authenticated request), and
  • mitigate the exposure of credentials sent to a malicious server over an unauthenticated secure channel due to client failure to validate the server’s identity as part of its TLS handshake.

Unlike the HTTP Digest authentication scheme, Hawk provides limited protection against replay attacks which does not require prior interaction with the server. Instead, the client provides a timestamp which the server can use to prevent replay attacks outside a narrow time window. Also unlike Digest, this mechanism is not intended to protect the key itself (user’s password in Digest) because the client and server both have access to the key material in the clear.

To clarify, MAC (Message Authentication Code) is a cryptographic string that is sent alongside a message (such as an HTTP request) to detect tampering and forgery. Both the sender and the receiver will use the same shared secret to generate and verify the MAC. The OAuth 1.0 specification used MAC signatures but these were removed in OAuth 2.0.

Protocol Example

To demonstrate how Hawk authentication works there is a protocol example:

The client attempts to access a protected resource without authentication, sending the following HTTP request to
the resource server:

GET /resource/1?b=1&a=2 HTTP/1.1
Host: 127.0.0.1:8000

The resource server returns the following authentication challenge:

HTTP/1.1 401 Unauthorized
WWW-Authenticate: Hawk

The client has previously obtained a set of Hawk credentials for accessing resources on the “http://example.com/”
server. The Hawk credentials issued to the client include the following attributes:

  • Key identifier: dh37fgj492je
  • Key: werxhqb98rpaxn39848xrunpaw3489ruxnpa98w4rxn
  • Algorithm: hmac-sha-256

The client generates the authentication header by calculating a timestamp (e.g. the number of seconds since January 1,
1970 00:00:00 GMT) and constructs the normalized request string (newline separated values):

1353832234
GET
/resource/1?b=1&a=2
127.0.0.1
8000
some-app-data

The request MAC is calculated using the specified algorithm “hmac-sha-256” and the key over the normalized request string.
The result is base64-encoded to produce the request MAC:

/uYWR6W5vTbY3WKUAN6fa+7p1t+1Yl6hFxKeMLfR6kk=

The client includes the Hawk key identifier, timestamp, and request MAC with the request using the HTTP “Authorization”
request header field:

GET /resource/1?b=1&a=2 HTTP/1.1
Host: 127.0.0.1:8000
Authorization: Hawk id="dh37fgj492je", ts="1353832234", ext="some-app-data", mac="/uYWR6W5vTbY3WKUAN6fa+7p1t+1Yl6hFxKeMLfR6kk="

The server validates the request by calculating the request MAC again based on the request received and verifies the validity and scope of the Hawk credentials. If valid, the server responds with the requested resource.

Security Considerations

There are a number of security considerations to consider which can be seen here https://github.com/hueniverse/hawk#security-considerations.

Hawk vs. OAuth

The Hawk documentation says that is can be used for client to server authentication but I think it would also make an easier to implement (than OAuth) authentication protocol if you are implementing machine to machine authentication.

For delegating access to protected resources then the documentation recommends Oz as alternative to OAuth. I will review Oz in another post.

PHP Implementation

I’ve had a go at implementing Hawk in PHP – https://github.com/alexbilbie/PHP-Hawk.

Client Usage

Assume you’re hitting up the following endpoint:

https://api.example.com/user/123?foo=bar

And the API server has given you the following credentials:

  • Key – ghU3QVGgXM
  • Secret – 5jNP12yT17Hx5Md3DCZ5pGI5sui82efX

To generate the header run the following:

$key = 'ghU3QVGgXM';
$secret = '5jNP12yT17Hx5Md3DCZ5pGI5sui82efX';
$hawk = Hawk::generateHeader($key, $secret, array(
                'host'    =>  'api.example.com', // you must set this
                `port`  =>  443, // you must set this
                'path'    =>  '/user/123',
                'method'  =>  'GET' // could be POST/DELETE/etc
            ));

You can also pass in additional application specific data with an ext key in the array.

Once you’ve got the Hawk string include it in your HTTP request as an Authorization header.

Server Usage

On your API endpoint if the incoming request is missing an authorization header then return the following two headers:

HTTP/1.1 401 Unauthorized
WWW-Authenticate: Hawk

If the request does contain a Hawk authorization header then process it like so:

$hawk = ''; // the authorisation header

// First parse the header to get the parts from the string
$hawk_parts = Hawk::parseHeader($hawk);

// Then with your own function, get the secret for the key from the database
$secret = getSecret($hark_parts['id']);

// Now validate the request
$valid = Hawk::verifyHeader($hawk, array(
        'host'    =>  'api.example.com',
        'port'    =>  443,
        'path'    =>  '/user/123',
        'method'  =>  'GET'
    ), $secret); // return true if the request is valid, otherwise false

Update w/c 3rd September

Alex Bilbie / / News

I’ve spent the last few weeks working away on the OAuth PHP library which now includes a resource server as well as an authentication server. I’ve also started merging in Phil Sturgeon’s OAuth 2.0 client code library, which when I’ve finished, will result in a mean, lean PHP library for working with any aspect of OAuth 2.0 (authentication, resource sharing or client side). Both server classes are now fully unit tested and I’m at 90% code coverage for all of the methods. I’ve started writing documentation for the library too and I’m going to write a tutorial on how to build an OAuth secured API server in the CodeIgniter framework.

On the UAG side, both Tim and I have been reading Mastering Microsoft Forefront UAG 2010 Customisation (Amazon link). I’ve now got some ideas about how we can easily integrate the university’s OAuth server that I developed with our UAG install. More on this soon.