Chapter 9

Authentication

Authentication is an important part of any modern web application. Being able to identify who is using your application and to see if they have the authorization to do so is invaluable.

On most websites and web applications, authentication is handled with the use of cookies, defined in the RFC 6265 titled ‘HTTP State Management Mechanism’.

The name says it all: cookies were created to let a server store states. That’s the complete opposite of what Fielding offered in the REST architectural style. The Stateless constraint states that all client-server interactions should be stateless, i.e. a request should not depend upon another one. The idea behind this is to allow web applications to scale more easily and cache to be more effective.

By default, HTTP authentication schemes, Basic and Digest, are stateless. But nowadays, most businesses need to know who their users are and want to lower the barrier to use their products. This means avoiding asking a user for his credentials too often and, if possible, only the first time.

For web APIs, there are multiple approaches depending on who will be the final user.

If your web API is meant to offer tools for other web applications, using a long, unique, secret key is the best practice and is known as an API key or an API secret token. It could be described as a password that’s impossible to remember, due to its length and complexity, and that needs to be transmitted with every request. It can be associated with an access token that acts as the identifier for that password, just like the email for an email/password combination. Web APIs like the one provided by Google Maps follow this model, and those identifying keys can be used to set usage quota and limit access to specific resources.

Box 9.1. SSL, everywhere, all the time.

Whatever authentication scheme you choose, you should always set up an SSL certificate for your applications in order to protect the sensitive data being exchanged between your clients and your server. With Let’s Encrypt and their free SSL certificates, you have no excuse to avoid having SSL configured on your web applications.

It’s a different story when the clients of your web APIs also need to handle users. If you’re building a Rails API to go with an Ember.js front-end, for example, you need to not only authenticate your clients (the Ember.js application), but also the users using it. To authenticate clients, using API keys as we’ve seen above is good, but it doesn’t work for the users. They need to be able to login with the usual credential combination (email/password). Sending those credentials in every request, even hashed and with SSL, is far from ideal. Instead, another solution is to generate a unique access token when a user signs in and uses that access token for all the following requests.

We will learn how to do this in the second module when we actually implement this solution.

9.1. Identification vs. Authentication vs. Authorization

In a perfect world, authentication probably wouldn’t be needed. But we aren’t living in one, so it’s important to define who is who and what they can do to provide the best “developer-experience” for the people who will use our API.

Authentication and authorization are two fundamental processes for securing web applications, including web APIs. The difference between them is not always clear, so let’s explain what each one is.

Identity is knowing who claims to be making an API request. Authentication is the process of checking that they really are who they claim to be. Authorization is the set of rules used to ensure that they only do what they are allowed to do.

Your API might not need the three of them and this will affect which authentication scheme you should use.

9.1.1. Identity

Google Maps provides developers with only API keys. That’s all they need to start looking up addresses. They are used by the Google Maps’ API to identify who is making the request, in order to limit the usage for example. But I can give my key to one of my friends and he will be able to use the API as well. The Google Maps API is not authenticating its users.

9.1.2. Authentication

On the other hand, if you take Twitter as an example, most API calls require authentication. To access more than the public information for a user, you need to login using either a username/password combination or with OAuth.

9.1.3. Authorization

In Twitter’s case, even once you’re authenticated, you still cannot post tweets under someone else’s identity, unless you have their credentials or an OAuth key for their account. You are not authorized to do so.

Twitter does all three while Google Maps’ API only cares about identification.

9.2. Understanding The “Stateless” Constraint

I’m not supposed to talk about REST until the third module, but it is important to define what a stateless application is. First, you need to know that the HTTP protocol is stateless, i.e. it doesn’t keep any state between requests, meaning that each request is independent.

The stateless constraint is part of the REST architectural style and its goals are, as we’ve said before, scalability and caching. To do so, the server shouldn’t keep track of where the user is or what he is doing. The state should not be maintained on the server.

While great on paper, there are very few stateless applications on the web today. Pretty much anything using cookies or access tokens breaks this constraint.

An application can be considered stateless only if each request is independent and does not rely on previously sent requests. For example, any application with a mandatory login cannot be considered stateless, whatever the mechanisms used, e.g. cookies, access tokens or others.

Building a stateful application is fine as long as you’re prepared for the extra-cost coming with it since handling 10k signed-in users won’t be the same as handling 10 million.

9.3. Authentication with HTTP

The default authentication mechanisms for HTTP are defined in the RFC 2617 as Basic and Digest. Those mechanisms were designed following the REST constraints, which means they are stateless. The username/password couple is included in every request, either in clear or encoded in Base64 (Basic Auth) or hashed with the MD5 hash function (Digest Auth).

Following the RFC, in order to authenticate, a client needs to send the Authorization header formatted as:

Authorization: auth-scheme hashed-credentials

If the server receives the adequate values, the request proceeds and the server returns the requested representation. However, in case of an unauthenticated request, the server should respond with the 401 Unauthorized and set the WWW-Authenticate header specifying what authentication scheme should be used and in which realm.

Here is an example:

WWW-Authenticate: Basic realm="User Realm"

The realm directive, which is actually optional, indicates the protection space. With it, the same server application can have different protected areas using different authentication schemes.

9.3.1. Basic

The Basic authentication is, by default, non-secure. The use of SSL is mandatory if one wishes to securely use this authentication mechanism. However, as we will see later, there are better options.

The main advantage of Basic authentication is that it’s stateless. However, it is highly non-secure, since the credentials are as easily decoded as they are encoded. Plus, due to the stateless constraint, the credentials have to be sent with every request. Using SSL is mandatory in order to provide a minimum amount of security, but it’s still not entirely safe.

Implementation

Let’s implement basic authentication with Sinatra. First, create the following folder and file in the module_01 folder:

mkdir chapter_09 && touch chapter_09/webapi_basic_auth.rb

Next, let’s fill the file we just created with the minimum code from our previous Sinatra web API. We will also set up HTTP Basic auth with the Rack::Auth::Basic middleware.

# webapi_basic_auth.rb
require 'sinatra'

use Rack::Auth::Basic, 'User Area' do |username, password|
  username == 'john' && password == 'pass'
end

get '/' do
  'Master Ruby Web APIs - Chapter 9'
end

Start the server…

ruby webapi_http_auth.rb

and let’s give it a try:

curl -i http://localhost:4567

As expected, it doesn’t work. Good news is, our authentication is working.

HTTP/1.1 401 Unauthorized
Content-Type: text/plain
Content-Length: 0
WWW-Authenticate: Basic realm="User Area"
X-Content-Type-Options: nosniff
Connection: keep-alive
Server: thin

To make it work, we need to pass the credentials and send them with the request.

curl -v -u john:pass http://localhost:4567

Since we are using the verbose mode, we can see the sent and received requests.

* Rebuilt URL to: http://localhost:4567/
*   Trying ::1...
* connect to ::1 port 4567 failed: Connection refused
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 4567 (#0)
* Server auth using Basic with user 'john'
> GET / HTTP/1.1
> Host: localhost:4567
> Authorization: Basic am9objpwYXNz
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: text/html;charset=utf-8
< Content-Length: 32
< X-XSS-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< X-Frame-Options: SAMEORIGIN
< Connection: keep-alive
< Server: thin
<
* Connection #0 to host localhost left intact
Master Ruby Web APIs - Chapter 9

Notice the Authorization header which contains the base64 encoded credentials:

Authorization: Basic am9objpwYXNz

9.3.2. Digest

Digest works pretty much like Basic, but is more secure since the credentials are hashed using MD5. Additionally, it is also stateless, with all its subsequent advantages and inconveniences. And yet, it’s still not the best in terms of security.

Implementation

Let’s see how to implement Digest auth with Sinatra. First, create two new files in the module_01 folder. To make it work within Sinatra, we need to have a config.ru file and run it with the rackup command.

touch webapi_digest_auth.rb && touch config.ru

We don’t need much in our API to test the Digest authentication. Most of the code for the job will go in the config.ru file.

# webapi_digest_auth.rb
require 'sinatra'

get '/' do
  'Master Ruby Web APIs - Chapter 9'
end

In order to use Digest authentication, we need to pass the Sinatra application through a rack middleware (Rack::Auth::Digest::MD5), as you can see in the code below:

# config.ru
require File.expand_path '../webapi_digest_auth.rb', __FILE__

app = Rack::Auth::Digest::MD5.new(Sinatra::Application) do |username|
  # Here we need to return the password for a specific username
  {'john' => 'pass'}[username]
end

app.realm = 'User Area'
app.opaque = 'secretkey'

run app

Then, start the server with:

rackup config.ru

If you’re having trouble with the rackup command, double-check that you have rack installed.

gem install rack

Once the server is running, let’s try to access the restricted endpoint.

curl -i http://localhost:9292
HTTP/1.1 401 Unauthorized
Content-Type: text/plain
Content-Length: 0
WWW-Authenticate: Digest realm="User Area",
  nonce="MTQ2NDUxNDkzNyA4MmNiMTc4ZjVjZGM1MTNlMDg4YjNlNzRlMDQyODZiMA==",
  opaque="f2f555205f367f5b51faee7ebb8dcc1b",
  qop="auth"
Connection: keep-alive
Server: thin

It didn’t work since we didn’t supply any credentials. Notice how we are getting details about the authentication scheme in the WWW-Authenticate header. The nonce present in those directives is needed in order to send a successful request. Let’s use curl to send a valid request and see what happens.

curl -v --digest -u john:pass http://localhost:9292

First, curl sends an unauthenticated request in order to get more information about the authentication scheme.

* Rebuilt URL to: http://localhost:9292/
*   Trying ::1...
* connect to ::1 port 9292 failed: Connection refused
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 9292 (#0)
* Server auth using Digest with user 'john'
> GET / HTTP/1.1
> Host: localhost:9292
> User-Agent: curl/7.43.0
> Accept: */*
>

It ends up failing, but contains the valuable information that curl needs to send a successful request.

< HTTP/1.1 401 Unauthorized
< Content-Type: text/plain
< Content-Length: 0
< WWW-Authenticate: Digest realm="User Area",
   nonce="MTQ2NDUxNDk0OSBjMzNiODQ5NGQwODEzYTBjMzBjYmYwMzQzMDkyZGZjNQ==",
   opaque="f2f555205f367f5b51faee7ebb8dcc1b",
   qop="auth"
< Connection: keep-alive
< Server: thin
<

curl is getting ready to send the second request using the Digest authentication with the user “john.”

* Connection #0 to host localhost left intact
* Issue another request to this URL: 'http://localhost:9292/'
* Found bundle for host localhost: 0x7fc98bc0f090
* Re-using existing connection! (#0) with host localhost
* Connected to localhost (127.0.0.1) port 9292 (#0)
* Server auth using Digest with user 'john'

The request below is sent with all the necessary directives to authenticate the request.

> GET / HTTP/1.1
> Host: localhost:9292
> Authorization: Digest username="john", realm="User Area",
  nonce="MTQ2NDUxNDk0OSBjMzNiODQ5NGQwODEzYTBjMzBjYmYwMzQzMDkyZGZjNQ==",
  uri="/",
  cnonce="M2M1ZTVmYjdiZDFjOTM1MmQ0YWZlYzBhNmJlMDMyODg=", nc=00000001,
  qop=auth,
  response="117ce0f57aa4b552e5ce364902826b56",
  opaque="f2f555205f367f5b51faee7ebb8dcc1b"
> User-Agent: curl/7.43.0
> Accept: */*
>

Finally, the server allows the resource to be accessed and returns the expected content.

< HTTP/1.1 200 OK
< Content-Type: text/html;charset=utf-8
< Content-Length: 32
< X-XSS-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< X-Frame-Options: SAMEORIGIN
< Connection: keep-alive
< Server: thin
<
* Connection #0 to host localhost left intact
Master Ruby Web APIs - Chapter 9

9.3.3. Token

Token-based authentication is the usual choice for web APIs. It allows a user to enter their username and password in order to get a token that will then be used in every request. This authentication approach is similar to the one using cookies, and cannot be considered stateless.

Here is an example of the Authorization header for this mechanism.

Authorization: Token l07oVEphj8EIgq1ZH60l5AxigxPk5oRA

Implementation

Create the webapi_token_auth.rb file in the module_01/chapter_09 folder.

touch webapi_token_auth.rb

Below you will find a very simplistic implementation of the token-based authentication mechanism. Read through the code and the comments and copy/paste it in the file created previously.

# webapi_token_auth.rb
require 'sinatra'
require 'json'

# This is our database of users
users = { 'thibault@samurails.com' => 'supersecret' }

# We will store tokens in this hash
tokens = {}

helpers do
  def unauthorized!
    response.headers['WWW-Authenticate'] = 'Token realm="Token Realm"'
    halt 401
  end

  def authenticate!(tokens)
    auth = env['HTTP_AUTHORIZATION']
    # We check if the Authorization header was provided
    # and if it matches the format we want: Token lfkdsfkdsjfsf
    unauthorized! unless auth && auth.match(/Token .+/)
    _, access_token = auth.split(' ')
    # Then we check in the tokens hash if there
    # is a token with the value sent by the client
    unauthorized! unless tokens[access_token]
  end
end

get '/' do
  authenticate!(tokens)
  'Master Ruby Web APIs - Chapter 9'
end

post '/login' do
  params = JSON.parse(request.body.read)
  email = params['email']
  password = params['password']

  content_type 'application/json'
  # If the email and password are correct
  if users[email] && users[email] == params['password']
    # We generate a token
    token = SecureRandom.hex
    # Store it in the tokens hash with a way
    # to get the user from that token
    tokens[token] = email
    { 'access_token' => token }.to_json
  else
    # If not, we send back a generic error message
    # To prevent attackers from knowing when they got
    # an email or password correctly.
    halt 400, { error: 'Invalid username or password.' }.to_json
  end
end

delete '/logout' do
  authenticate!(tokens)
  tokens.delete(access_token)
  halt 204
end

Let’s see if our authentication actually works and prevents unauthenticated users from using it.

curl -i http://localhost:4567

Yes, looks good!

HTTP/1.1 401 Unauthorized
Content-Type: text/html;charset=utf-8
WWW-Authenticate: Token realm="Token Realm"
Content-Length: 0
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
Connection: keep-alive
Server: thin

Now let’s try to log in.

curl http://localhost:4567/login \
  -i -d '{"email":"thibault@samurails.com", "password": "supersecret"}'

And we get an access token back, great!

HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 51
X-Content-Type-Options: nosniff
Connection: keep-alive
Server: thin

{"access_token":"6afc7f5db9eaaf7eab423e35458b12be"}

Let’s use this token to send a request. This time, we should get back what we want instead of a 401 Unauthorized.

curl -i -H 'Authorization: Token 6afc7f5db9eaaf7eab423e35458b12be' \
  http://localhost:4567

We received what we wanted!

HTTP/1.1 200 OK
Content-Type: text/html;charset=utf-8
Content-Length: 32
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
Connection: keep-alive
Server: thin

Master Ruby Web APIs - Chapter 9

Let’s try the logout.

curl -i -X DELETE -H 'Authorization: Token 6afc7f5db9eaaf7eab423e35458b12be' \
  http://localhost:4567/logout
HTTP/1.1 204 No Content
X-Content-Type-Options: nosniff
Connection: close
Server: thin

We just implemented a simple token-based authentication!

9.3.4. API Keys

It can also be used to authenticate clients automatically, as in the case of a mobile application. The client needs to contain the secret key before being deployed and will send that key with every request. Often, developers implement API keys in a way that allows people to simply append the key at the end of the URL, as a query parameter: ?api_key=fkdjslfkjdslfkjdfsdfdsfdsf.

It’s usually a better idea to reuse the Authorization header with a custom authentication scheme. The only reason to add the API key as a query parameter is if you really, really want to be able to copy/paste the URL around.

Implementation

The implementation of the API key-based authentication mechanism is going to be pretty simple; all we have to do is check that the client provided the expected key.

First, let’s create a new file to hold this new API.

touch webapi_apikey_auth.rb

And here is the entire code. Everything happens in the before filter, where we check that the Authorization header contains the correct API key.

# webapi_apikey_auth.rb
require 'sinatra'
require 'json'

API_KEY = 'ZmvhBBpb4RlbyblpKoj9F716CoONTOtr'

before do
  auth = env['HTTP_AUTHORIZATION']
  unless auth && auth.match(/Key #{API_KEY}/)
    response.headers['WWW-Authenticate'] = 'Key realm="User Realm"'
    halt 401
  end
end

get '/' do
  'Master Ruby Web APIs - Chapter 9'
end

Start the server.

ruby webapi_apikey_auth.rb

Give it a try with an unauthenticated request to see how the server reacts.

curl -i http://localhost:4567

Unauthorized request - it’s working correctly!

HTTP/1.1 401 Unauthorized
Content-Type: text/html;charset=utf-8
WWW-Authenticate: Key realm="User Realm"
Content-Length: 0
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
Connection: keep-alive
Server: thin

Now, let’s add the Authorization header with the key we hard-coded. For real-world applications, it’s better to keep these kind of secret values in environment variables.

curl -i -H "Authorization: Key ZmvhBBpb4RlbyblpKoj9F716CoONTOtr" \
  http://localhost:4567
HTTP/1.1 200 OK
Content-Type: text/html;charset=utf-8
Content-Length: 32
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
Connection: keep-alive
Server: thin

Master Ruby Web APIs - Chapter 9%

9.4. A Note About OAuth 2.0

I wanted to quickly talk about the OAuth protocol here. It’s out of the context of this chapter, since OAuth 2.0 is not an authentication protocol.

9.5. Wrap Up

In this chapter, we learned more about HTTP authentication schemes. We also discovered that we can make our own schemes and document them with our application, even if it’s not ideal compared to re-using standards. In the second module, we will implement a custom authentication scheme in order to easily identify clients and authenticate users.