Modularity in Elixir

As you may or may not know, I’ve written a book on the topic of Modularity with Ruby on Rails. The idea behind the approach presented in the book is to divide your applications into logical (and potentially reusable) components.

Modularity is basically the separation of concerns principle applied at the level of your entire application.

Modular Rails explores how modularity can be implemented using a feature that came with Rails 3: Engines. From the Ruby on Rails guides:

Engines can be considered miniature applications that provide functionalities to their host applications.

It’s unfortunate that this feature, making the whole modular approach possible, is only present at the framework level, and not in the language itself.

Elixir

About 2 years ago, I started playing around with Elixir. I re-built Devblast using it, but barely scratched the surface of its powers. A few months later, I joined OmiseGO as Lead Engineer and started building a more advanced Elixir application.

Through research, test implementation and help from my first team member (who had a background in Erlang), I was able to wrap my head around a lot of new concepts: functional programming, immutability, process management, GenServers, etc. But the thing I got most excited about was the Umbrella feature.

It’s basically everything that was hacked together to make Rails engines work, but cleanly implemented in the language itself and offered with proper dependencies management. You can adjust how you want your sub-apps (as we call the individual applications in Umbrella apps) to interact and define if you want them to actually start and manage their own processes.

It was love at first sight.

I decided right away that we should build the eWallet as an Umbrella app, in order to properly structure all the moving parts. It was so easy to get everything working that I’m not sure I want to go through the pain of doing it in Ruby anymore.

You can look at the sub-applications inside an Umbrella app as internal micro-services, each dealing with its own set of responsibilities and working with the others to provide an actually useful set of features to your users.

An example

Now, let’s see how it looks like in practice, by making a very simple Umbrella app. We won’t go into the details, I’m just trying to share the whole idea here, not how to actually code it.

We’re going to be creating an example structure for a CRM application, that will offer the following features:

• Users & Authentication management
• Contacts / Leads management
• All interactions with the CRM should be available through a regular web interface, as well as through a web API.

We can actually structure it in two different ways (and probably more, but I found these two to be the most interesting ones). First, let’s generate the Umbrella app and we’ll go through the two approaches:

mix new crm_umbrella --umbrella

* creating README.md
* creating .formatter.exs
* creating .gitignore
* creating mix.exs
* creating apps
* creating config
* creating config/config.exs

Your umbrella project was created successfully.
Inside your project, you will find an apps/ directory
where you can create and host many apps:

    cd crm_umbrella
    cd apps
    mix new my_app

Commands like "mix compile" and "mix test" when executed
in the umbrella project root will automatically run
for each application in the apps/ directory.

cd crm_umbrella/apps

The “Business Logic Modules” / “Silos” approach

With this approach, we split the business logic of the CRM into modules, where each one contains everything it needs to function. In short, we will build a set of silos that connects the end user to the data layer.

To make things simpler, we start off with a core module where we will initialize a shared Ecto Repo:

mix new core

We can then create two more sub-applications, one for the users and one for the contacts:

mix new users

mix new contacts

We will have the entire stack in each one of them:

• User / Contact Ecto schemas
• The usual Phoenix stuff: router, controllers, views, and templates.
• Any needed modules between the data layer and the web layer.

The last thing we’ll probably want to add is a dispatcher sub-application. Since we have more than one application that can handle web requests (both the users and contacts apps can), we want to have one sub-application that receives all web request and dispatch them to the right place:

mix new dispatcher

In the end, the dependency tree for our sub-apps will look like this:

    dispatcher
   |         |       
 users     contacts   
    \        /        
       core

This approach is interesting if you’re looking for something similar to usual micro-services, you can just add more sub-apps in the middle layer and have different teams working on each one of them.

Note that I usually dislike micro-services architecture, and I’m only referencing it as an example here. I don’t recommend going that route. My simple rule of thumb to define if micro-services are needed is that if one developer is going to work on more than one micro-service, then you don’t need micro-services.

The “Functional Layer Modules” approach

This approach focuses more on cutting your application horizontally, into a stack of layers.

First, we create a sub-application where we will put all our database schemas. This will represent the data layer to the rest of the sub-apps, making it easier to store and retrieve records.

mix new db

On top, we create another sub-app to hold the modules that will contain the business logic. To keep this simple, I’m only creating one, but nothing prevents you from having more:

mix new crm

Finally, we create sub-apps that can handle HTTP request and will be accessible by anyone. We create one for our website and one for our API. For the website app, we can either build a frontend using Phoenix templating system, or we could have a JavaScript “Single Page Application” that connects to the API sub-app. That’s actually how the Admin Panel in the OmiseGO eWallet works.

mix new web

mix new api --module API

We end up with an Umbrella application structured like this:

web   API
 |     |
 \     /
   CRM
    |
   DB

I personally tend to prefer this approach lately, especially for smaller teams.

The End

I originally worked with the “Silos” approach for my first modular application, simply because the company I was working for wanted to be able to take those siloed features (encapsulated in modules) and add them to other applications. For example, a booking module could be used in an app for restaurants, for doctors or for massages. Another good example is Devise, which encapsulates the whole Authentication stack inside a Rails engine.

At OmiseGO, we wanted to build a flexible, all-in-one, and easy to deploy application. We weren’t planning to reuse modules anywhere else or run the sub-apps as micro-services. That’s why it made sense for us to go with the Layered approach.

There is no right or wrong, both of those approaches are valid and there are definitely more ways to structure your modular applications. It really comes down to your needs.

Let me know which approach you prefer (or are currently using) and why in the comments :)