Chapter 15

Building Representations

In this chapter, we are going to build tools that will help us generate representations for our resources.

Our first option would be to use one of the third-party solutions that are available. For example, we could use ActiveModelSerializers, which is a neat little gem that lets you create simple serializers for your models. Or we could just do it ourselves using POROs.

ActiveModelSerializers

class BookSerializer < ActiveModel::Serializer
  attribute :id
  attribute :title
  attribute :subtitle
  attribute :released_on

  belongs_to :author
  belongs_to :publisher
end

Plain-Old Ruby Object

class BookSerializer

  def initialize(book)
    @book = book
  end

  def as_json(*)
    {
      id: @book.id,
      title: @book.title,
      subtitle: @book.subtitle,
      released_on: @book.released_on
      author: @book.author
      publisher: @book.publisher
    }
  end

end

If you don’t need total control over your representations, I think ActiveModelSerializers is a nice option. However, in this book we are going to write some custom tools since we want to be able to handle the serialization differently.

Instead of using an existing solution, we are going to make our own.

15.1. Feature Requirements

Wondering why we need to make our own? Well, the first reason is that I think you will learn more if I show you how to make everything from scratch, and you will have a better idea of how things work underneath some famous gems. That way, you can start using them with more confidence.

The second reason is that to have a powerful and flexible API, there are some features that we need to have. To create something simple and efficient, it is better if we are in full control of those features.

Filtering

We want clients to be able to query records based on some specific attributes. For example, clients should be able to ask for all the books that have a title starting with Rails or get all the books released after 01/01/2010.

Sorting

Allowing clients to sort any list of entities is a pretty important part of any server application. Plus, it’s pretty easy to implement something decent!

Pagination

Trying to get 3000 books with one request is not a good idea. Most of the users will only look at the first 5 anyway, so why send so much data? Pagination is the best option to avoid this, as it helps you to save some bandwidth and time.

Field Picking

This one is less commonly seen in web APIs. Some big companies, like Google, actually offer it with their web API. The idea is to let the client decide which fields are needed for this specific call. For example, a client could say it want a list of books, but only need the id and title for each of them. This feature can be used to optimize the requests, once again saving bandwidth and time.

Embed Picking

Just like field picking, embed picking lets clients decide which associated entities they want to see embedded inside the representation. For a client, using this could be the difference between receiving a book with the author_id field or a hash named author containing the given_name and family_name properties.

Serialization

Serialization to JSON will be pretty much left to Rails. However, we will need to implement the method as_json(*) to ensure entities get properly serialized.

15.2. Crafting Our Own Tools

To have all those features and make them work well together, we need to create our own tools. That’s why we are going to create presenters, query builders and representation builders.

Presenters

Presenters should allow us to define how the representation for a model can be built. Each model should have a presenter to answer the following two questions: Which fields can be used to build the representation? And which ones can be used can be used for sorting and filtering?

Presenters will be used by the query builders and the representation builders.

Query Builders

Query builders will be used mostly for resources where a list of entities is expected. They will contain the logic for sorting, filtering and pagination. Together, they will produce a query to execute against the database. In Rails terms, the responsibility of those classes will be to generate and return scoped data depending on the passed parameters.

We will be able to reuse query builders for different resources.

Representation Builders

Finally, before we can send back the data, we will need to to ensure that only the requested fields and associations are inserted in the representation. The representation builders will take care of field picking, embed picking and serialization.

We will be able to reuse these representation builders for different resources.

15.3. Creating Presenters

If what we are going to do sounds abstract, don’t worry, it’s about to get real. We are now going to create the BasePresenter from which other presenters will inherit. Then we will create the first real presenter, BookPresenter. Finally, we will implement the first query builder, FieldPicker and use it in the BooksController class.

For now, we will store all those custom classes in the app/ folder. While they will probably end up being quite generic, we can’t know for sure yet.

Let’s go!

The `BasePresenter` class

Presenters should be easy to create and configure. They should look something like the ActiveModelSerializer we saw earlier.

For example, I’d like the book presenter to be something like this:

class BookPresenter < BasePresenter
  build_with    :id, :title, :subtitle, :isbn_10, :isbn_13, :description,
                :released_on, :publisher_id, :author_id, :created_at,
                :updated_at
  related_to    :publisher, :author
  sort_by       :id, :title, :released_on, :created_at, :updated_at
  filter_by     :id, :title, :isbn_10, :isbn_13, :released_on, :publisher_id,
                :author_id

end

Actually, that’s exactly how it should look! You might not like the names I chose, so feel free to change them. As long as you also change them in the BasePresenter class, you can call them whatever you like.

First, we need to write some tests. Let’s create the folder spec/presenters and put the file base_presenter_spec.rb in there.

mkdir spec/presenters && touch spec/presenters/base_presenter_spec.rb

Then create the presenters folder and the base_presenter.rb file inside.

mkdir app/presenters && touch app/presenters/base_presenter.rb

We have to create an empty presenter class to avoid getting exceptions when we run our tests.

# app/presenters/base_presenter.rb
class BasePresenter
end

Let’s talk about the logic behind the presenters. The presenter instances will be passed to the representation builders. They need to hold the entity for which the representation is being built and the parameters received in the controller. We can define this behavior in the BasePresenter and simply inherit from it in all our presenters.

Three tests should be enough to ensure that the instantiation happens without any issue. We can just check that the instance variables were correctly set with the passed parameters. We also want a small check for the as_json method to ensure that the data attribute gets serialized.

# spec/presenters/base_presenter_spec.rb
require 'rails_helper'

RSpec.describe BasePresenter do
  # Let's create a subclass to avoid polluting
  # the BasePresenter class
  class Presenter < BasePresenter; end
  let(:presenter) { Presenter.new('fake', { something: 'cool' }) }

  describe '#initialize' do
    it 'sets the "object" variable with "fake"' do
      expect(presenter.object).to eq 'fake'
    end

    it 'sets the "params" variable with { something: "cool" }' do
      expect(presenter.params).to eq({ something: 'cool' })
    end

    it 'initializes "data" as a HashWithIndifferentAccess' do
      expect(presenter.data).to be_kind_of(HashWithIndifferentAccess)
    end
  end

  describe '#as_json' do
    it 'allows the serialization of "data" to json' do
      presenter.data = { something: 'cool' }
      expect(presenter.to_json).to eq '{"something":"cool"}'
    end
  end
end

Let’s add more red to this book.

rspec spec/presenters/base_presenter_spec.rb

Failure (RED)

...

Finished in 0.05305 seconds (files took 2.26 seconds to load)
4 examples, 4 failures

...

Failing, as expected. Let’s implement the minimum amount of code to make those tests pass.

# app/presenters/base_presenter.rb
class BasePresenter
  attr_accessor :object, :params, :data

  def initialize(object, params, options = {})
    @object = object
    @params = params
    @options = options
    @data = HashWithIndifferentAccess.new
  end

  def as_json(*)
    @data
  end

end

rspec spec/presenters/base_presenter_spec.rb

Success (GREEN)

...

BasePresenter
  #initialize
    sets the "object" variable with "fake"
    sets the "params" variable with { something: "cool" }
    initializes "data" as a HashWithIndifferentAccess
  #as_json
    allows the serialization of "data" to json

Finished in 0.06144 seconds (files took 2.6 seconds to load)
4 examples, 0 failures

Alright, that’s great - but we still have no way to define which fields can be used to build the representation. To do this, we will add a class method (build_with) to the base presenter that will fill an array and store it in the class as build_attributes. With this, any representation builder will be able to access this list of attributes when they need it.

# spec/presenters/base_presenter_spec.rb
require 'rails_helper'

RSpec.describe BasePresenter do

  class Presenter < BasePresenter; end

  describe '#initialize' # hidden code
  describe '#as_json' # hidden code

  describe '.build_with' do
    it 'stores ["id","title"] in "build_attributes"' do
      Presenter.build_with :id, :title
      expect(Presenter.build_attributes).to eq ['id', 'title']
    end
  end
end

Failure (RED)

rspec spec/presenters/base_presenter_spec.rb

Output

...

1) BasePresenter.build_with stores ["id","title"] in "build_attributes"
   Failure/Error: Presenter.build_with :id, :title
   NoMethodError:
     undefined method `build_with' for Presenter:Class
...

Finished in 0.16387 seconds (files took 2.8 seconds to load)
5 examples, 1 failure

We need to create a method at the class level and to do it we have two options. Either we use:

class << self
  def method_name
    # do something great
  end
end

Or:

def self.method_name
  # do something great
end

I personally prefer the first syntax and, since we will add a class attribute accessor, it’s actually the easiest option. To go with this class method, we also need a class level instance variable to store the list of attributes.

Box 15.1. Class Variables vs Class Level Instance Variables

Class Variables are variables defined on a class using the double @ notation.

class MyClass
  @@my_variable = 'something'
end

The problem with these variables is that they are shared with all the classes that inherit from the class implementing it.

class MySubClass < MyClass
  @@my_variable = 'something else'
end

If we use MyClass.my_variable, it now returns something_else instead of something.

Class Variables are very rarely useful in Ruby.

Instead, you usually need the behavior of class level instance variables. In Ruby, everything is an object, that includes classes. This means we can do the following:

class MyClass
  @my_variable = 'something'
end

class MySubClass < MyClass
  @my_variable = 'something else'
end

And we will have the behavior we need.

MyClass.my_variable # => 'something'
MySubClass.my_variable # => 'something else'

In the end, we end up with the code below, full of useful comments.

# app/presenters/base_presenter.rb
class BasePresenter
  # Define a class level instance variable
  @build_attributes = []

  # Open the door to class methods
  class << self
    # Define an accessor for the class level instance
    # variable we created above
    attr_accessor :build_attributes

    # Create the actual class method that will
    # be used in the subclasses
    # We use the splash operation '*' to get all
    # the arguments passed in an array
    def build_with(*args)
      @build_attributes = args.map(&:to_s)
    end
  end

  attr_accessor :object, :params, :data

  def initialize(object, params, options = {})
    @object = object
    @params = params
    @options = options
    @data = HashWithIndifferentAccess.new
  end

  def as_json(*)
    @data
  end

end

rspec spec/presenters/base_presenter_spec.rb

Success (GREEN)

...

Finished in 0.07152 seconds (files took 3.46 seconds to load)
5 examples, 0 failures

The first version of our BasePresenter looks good to go. In the next chapters, we will get back to it to add more class methods to handle filtering, sorting and more!

The `BookPresenter` class

With the BasePresenter class ready, let’s now create the first class that will inherit from it: BookPresenter. Create the file app/presenters/book_presenter.rb and add the code below inside.

touch app/presenters/book_presenter.rb

In this code, we call the build_with class method and give it the list of attributes that can be used to build the Book model. The client will also be able to pick any field in this list to build a custom representation.

# app/presenters/book_presenter.rb
class BookPresenter < BasePresenter
  build_with    :id, :title, :subtitle, :isbn_10, :isbn_13, :description,
                :released_on, :publisher_id, :author_id, :created_at,
                :updated_at, :cover

  def cover
    @object.cover.url.to_s
  end
end

Notice how we added the method cover on the representer. That’s because we want to be able to override or add some fields in the presenter that are not necessarily present on the model. This gives us more flexibility and lets us define that we want the cover to be a simple URL instead of a hash, as it would be by default.

The basic logic for presenters is now ready for the Book model. It’s time to create our first representation builder: FieldPicker.

15.4. The `FieldPicker` class

This class is going to be more complex than anything we’ve built before. To avoid spending too much time on it, we are only going to write a few tests. Let’s discuss the logic behind this representation builder.

To be able to build the representation for a model, this class needs two things: the model presenter and the parameters sent by the client. Once the builder has done its job, it will return the presenter with an updated data attribute.

Create the test file.

mkdir spec/representation_builders && \
  touch spec/representation_builders/field_picker_spec.rb

And the actual file that will contain the FieldPicker class.

mkdir app/representation_builders && \
  touch app/representation_builders/field_picker.rb

Here is the basic content for the FieldPicker class.

# app/representation_builders/field_picker.rb
class FieldPicker

  def initialize(presenter)
    @presenter = presenter
    @fields = @presenter.params[:fields]
  end

  def pick
    # Return presenter with updated data
  end

end

We are only going to write 3 tests for the pick method. Those tests are available below but before you go through them, let met explain what’s in there.

The first test is about testing that the FieldPicker works as intended when given the fields parameter, with no overriding methods in the presenter.
The second test is like the first one, but with an override defined in the presenter.
The last one tests the default behavior when no fields parameter is given.

Here is the file. I added comments on some specific parts so don’t just copy/paste the code… read it too! ;)

Since we are creating these tools to be part of the application, we are going to use our existing Rails models to test them. If we decide to extract the builders into a gem, we will need to rewrite these tests and create dummy models.

# spec/representation_builders/field_picker_spec.rb
require 'rails_helper'

RSpec.describe 'FieldPicker' do
  # We define 'let' in cascade where each one of them is used by the
  # one below. This allows us to override any of them easily in a
  # specific context.
  let(:rails_tutorial) { create(:ruby_on_rails_tutorial) }
  let(:params) { { fields: 'id,title,subtitle' } }
  let(:presenter) { BookPresenter.new(rails_tutorial, params) }
  let(:field_picker) { FieldPicker.new(presenter) }

  # We don't want our tests to rely too much on the actual implementation of
  # the book presenter. Instead, we stub the method 'build_attributes'
  # on BookPresenter to always return the same list of attributes for
  # the tests in this file
  before do
    allow(BookPresenter).to(
      receive(:build_attributes).and_return(['id', 'title', 'author_id'])
    )
  end

  # Pick is the main method of the FieldPicker. It's meant to be used
  # as 'FieldPicker.new(presenter).pick and should return the same presenter'
  # with updated data
  describe '#pick' do
    context 'with the "fields" parameter containing "id,title,subtitle"' do
      it 'updates the presenter "data" with the book "id" and "title"' do
        expect(field_picker.pick.data).to eq({
          'id'    => rails_tutorial.id,
          'title' => 'Ruby on Rails Tutorial'
        })
      end

      context 'with overriding method defined in presenter' do
        # In this case, we want the presenter to have the method 'title'
        # in order to test the overriding system. To do this, the simplest
        # solution is to meta-programmatically add it.
        before { presenter.class.send(:define_method, :title) { 'Overridden!' } }

        it 'updates the presenter "data" with the title "Overridden!"' do
          expect(field_picker.pick.data).to eq({
            'id' => rails_tutorial.id,
            'title' => 'Overridden!'
          })
        end

        # Let's not forget to remove the method once we're done to
        # avoid any problem with other tests. Always clean up after your tests!
        after { presenter.class.send(:remove_method, :title) }
      end
    end

    context 'with no "fields" parameter' do
      # I mentioned earlier how we can easily override any 'let'.
      # Here we just override the 'params' let which will be used in place
      # of the one we created earlier, but only in this context
      let(:params) { {} }

      it 'updates "data" with the fields ("id","title","author_id")' do
        expect(field_picker.send(:pick).data).to eq({
          'id' => rails_tutorial.id,
          'title' => 'Ruby on Rails Tutorial',
          'author_id' => rails_tutorial.author.id
        })
      end
    end
  end
end

rspec spec/representation_builders/field_picker_spec.rb

Failure (RED)

...

Finished in 0.13876 seconds (files took 2.44 seconds to load)
3 examples, 3 failures

...

Here is the FieldPicker to make those tests pass. Before I give you the entire file, let me explain each one of the methods we need: initialize, pick, validate_fields, fields and pickable.

We’ve already talked about how this class get initialized. It takes a presenter as the only parameter and assigns it to an instance variable. For convenience, we also extract the fields from the presenter params attribute to an instance variable.

# app/representation_builders/field_picker.rb
class FieldPicker

  def initialize(presenter)
    @presenter = presenter
    @fields = @presenter.params[:fields]
  end

  def pick # Hidden Code

  private

  def validate_fields # Hidden Code
  def pickable # Hidden Code

end

Next we have the pickable method which is going to extract the build_attributes from the presenter and store them in the @pickable instance variable.

# app/representation_builders/field_picker.rb
class FieldPicker

  def initialize # Hidden Code
  def pick # Hidden Code

  private

  def validate_fields # Hidden Code

  def pickable
    @pickable ||= @presenter.class.build_attributes
  end

end

The function of validate_fields is to ensure that only allowed fields go through the filtering process. It uses pickable and the reject method to ensure that. If no specific fields were requested by the client, it will just return nil.

# app/representation_builders/field_picker.rb
class FieldPicker

  def initialize # Hidden Code
  def pick # Hidden Code

  private

  def validate_fields
    return nil if @fields.blank?
    validated = @fields.split(',').reject { |f| !pickable.include?(f) }
    validated.any? ? validated : nil
  end

  def pickable # Hidden Code

end

Finally, the pick method will either get the list of fields from validate_fields, if it didn’t get nil back, or from pickable which is basically the entire list of attributes. After this, it will loop through each field, check if it’s defined on the presenter and either call it on the presenter or on the model. It will then add the key/value inside the presenter data attribute.

# app/representation_builders/field_picker.rb
class FieldPicker

  def initialize # Hidden Code

  def pick
    (validate_fields || pickable).each do |field|
      value = (@presenter.respond_to?(field) ? @presenter :
                                               @presenter.object).send(field)
      @presenter.data[field] = value
    end

    @presenter
  end

  private

  def validate_fields # Hidden Code
  def pickable # Hidden Code

end

Here is how it all comes together:

# app/representation_builders/field_picker.rb
class FieldPicker

  def initialize(presenter)
    @presenter = presenter
    @fields = @presenter.params[:fields]
  end

  def pick
    (validate_fields || pickable).each do |field|
      value = (@presenter.respond_to?(field) ? @presenter :   
                                               @presenter.object).send(field)
      @presenter.data[field] = value
    end

    @presenter
  end

  private

  def validate_fields
    return nil if @fields.blank?
    validated = @fields.split(',').reject { |f| !pickable.include?(f) }
    validated.any? ? validated : nil
  end

  def pickable
    @pickable ||= @presenter.class.build_attributes
  end

end

rspec spec/representation_builders/field_picker_spec.rb

Success (GREEN)

...

FieldPicker
  #pick
    with the "fields" parameter containing "id,title,subtitle"
      updates the presenter "data" with the book "id" and "title"
      with overriding method defined in presenter
        updates the presenter "data" with the title "Overridden!"
    with no "fields" parameter
      updates "data" with the fields ("id","title","author_id") in presenter

Finished in 0.1924 seconds (files took 2.19 seconds to load)
3 examples, 0 failures

The FieldPicker is ready, congratulations!

15.5. Playing With Our New Toy

We’ve just built the first representation builder and it’s now time to start using it! We didn’t have any control earlier on how the books were returned from the /api/books URL. Now, that’s no longer the case.

The first thing we want to do is add some tests in order to ensure that we can pick the fields we want when sending requests to /api/books.

# spec/requests/books_spec.rb
require 'rails_helper'

RSpec.describe 'Books', type: :request do
  # Hidden Code: let definitions

  describe 'GET /api/books' do
    before { books }

    context 'default behavior' # Hidden Code

    describe 'field picking' do
      context 'with the fields parameter' do
        before { get '/api/books?fields=id,title,author_id' }

        it 'gets books with only the id, title and author_id keys' do
          json_body['data'].each do |book|
            expect(book.keys).to eq ['id', 'title', 'author_id']
          end
        end
      end

      context 'without the "fields" parameter' do
        before { get '/api/books' }

        it 'gets books with all the fields specified in the presenter' do
          json_body['data'].each do |book|
            expect(book.keys).to eq BookPresenter.build_attributes.map(&:to_s)
          end
        end
      end
    end # End of describe 'field picking'
  end
end

As usual with the TDD cycle, let’s run the tests and see them fail.

rspec spec/requests/books_spec.rb

Failure (RED)

...

Finished in 0.45282 seconds (files took 2.55 seconds to load)
5 examples, 1 failure

...

Fixing them is actually pretty simple, we just need to start using the book presenter and the field picker! In the code below, we map our list of books and replace the records by BookPresenter instances after they went through the FieldPicker builder.

# app/controllers/books_controller.rb
class BooksController < ApplicationController

  def index
    books = Book.all.map do |book|
      FieldPicker.new(BookPresenter.new(book, params)).pick
    end

    render json: { data: books }.to_json
  end

end

The tests are now working properly!

rspec spec/requests/books_spec.rb

Success (GREEN)

...

Books
  GET /api/books
    gets HTTP status 200
    receives a json with the "data" root key
    receives all 3 books
    field picking
      with the fields parameter
        gets books with only the id, title and author_id keys
      without the field parameter
        gets books with all the fields specified in the presenter

Finished in 0.37697 seconds (files took 1.75 seconds to load)
5 examples, 0 failures

Our tests are still running well. Automated testing is good, but just for good measure let’s give it a manual try to see the result of our efforts.

rails s

I’m using a Google Chrome extension named JSON Formatted to automatically prettify JSON documents.

Now try some of the following URLs:

http://localhost:3000/api/books: See Figure 1.

https://s3.amazonaws.com/devblast-mrwa-book/images/figures/15/01

http://localhost:3000/api/books?fields=id,title,subtitle: See Figure 2.

https://s3.amazonaws.com/devblast-mrwa-book/images/figures/15/02

http://localhost:3000/api/books?fields=weird,stuff: See Figure 3.

https://s3.amazonaws.com/devblast-mrwa-book/images/figures/15/03

15.6. Pushing Our Changes

It’s now time to push our changes to GitHub. The flow is the same as in the previous chapter. First, however, let’s run our test suite to ensure that everything is still working.

rspec

Success (GREEN)

...

Finished in 0.7135 seconds (files took 2.58 seconds to load)
32 examples, 0 failures

Great! Here is the list of steps to push the code.

Check the changes.

git status

On branch master
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

	modified:   app/controllers/books_controller.rb
	modified:   spec/requests/books_spec.rb

Untracked files:
  (use "git add <file>..." to include in what will be committed)

	app/presenters/
	app/representation_builders/
	spec/presenters/
	spec/representation_builders/

no changes added to commit (use "git add" and/or "git commit -a")

Stage them.

git add .

Commit the changes.

git commit -m "Add BooksController, BookPresenter and FieldPicker"

[master 108269a] Add BooksController, BookPresenter and FieldPicker
 7 files changed, 217 insertions(+), 2 deletions(-)
 create mode 100644 app/presenters/base_presenter.rb
 create mode 100644 app/presenters/book_presenter.rb
 create mode 100644 app/representation_builders/field_picker.rb
 create mode 100644 spec/presenters/base_presenter_spec.rb
 create mode 100644 spec/representation_builders/field_picker_spec.rb

Push to GitHub.

git push origin master

15.7. Wrap Up

We now have a working books resource and we can let the client decide which fields are needed thanks to the FieldPicker class.

By creating the BasePresenter class, we have also laid out the foundation for the next query builders and representation builders that we are about to implement in the next chapter.

Previous - Next

Master Ruby Web APIs

Chapter 15

Building Representations