Creating Reusable Ecto Code

Creating a highly reusable Ecto API is one of the ways we can create long-term sustainable code for ourselves, while growing it with our application to allow for infinite combination possibilites and high code reusability. If we write our Ecto code correctly, we can not only have a very well defined split between query definition and combination/execution using our context but also have the ability to re-use the queries we design individually, together with others to create larger complex queries.

Splitting our definitions from glue

Our first step is to split out our Ecto query definitions from the schema, we can do this by defining all our queries inside schema files and banning usage of the Ecto.Query and Ecto.Query.API functions from within our contexts.

For example, imagine we had an accounts context with a user schema, we’d like to have an api for querying for the user by ID, first name and last name.

To set this up, we can first define our queries in our User schema:

defmodule MyApp.Accounts.User do
  use Ecto.Schema
  import Ecto.Query

  schema "users" do
    field :first_name, :string
    field :last_name, :string
  end

  def by_id(id), do: where(User,  id: ^id)
  def by_first_name(first_name), do: where(User,  first_name: ^first_name)
  def by_last_name(last_name), do: where(User,  last_name: ^last_name)
end

Then we can define some interfacing functions in our Accounts context:

defmodule MyApp.Accounts do
  alias MyApp.{Repo, Accounts.User}

  def find_user_by_id(id) do
    id |> User.by_id |> Repo.all
  end

  def find_user_by_first_name(name) do
    name |> User.by_first_name |> Repo.all
  end

  def find_user_by_last_name(name) do
    name |> User.by_last_name |> Repo.all
  end
end

This creates a clear split in responsibilites, the Schemas define the queries, the Contexts glue the queries together and call Repo. Sticking to this pattern allows us to scale our code and keep it simple in a highly complex system.

Reusing Queries

To enable us to re-use queries, we need to be able to chain them together. We can do this by passing in the current query as the first parameter, and then adding onto it. As an extra step we’ll set a default so they can be used either as the initial part of a query, by itself, or as part of a bigger query.

In our User schema module, we can change our functions like so:

defmodule MyApp.Accounts.User do
  alias MyApp.Accounts.User

  ...

  def by_id(query \\ User, id) do
    where(query, id: ^id)
  end

  def by_first_name(query \\ User, first_name) do
    where(query, first_name: ^first_name)
  end

  def by_last_name(query \\ User, last_name) do
    where(query, last_name: ^last_name)
  end
end

In turn we can utilize pipes to now filter for first names and last names:

defmodule MyApp.Accounts do
  ...

  def find_by_first_and_last_name(first_name, last_name) do
    first_name |> User.by_first_name |> User.by_last_name(last_name) |> Repo.all
  end
end

Reusability Over Joins Using `as`

Now that we’ve made queries more re-usable what about when we need to do a join. For example what if we had a Role schema like so:

defmodule MyApp.Accounts.Role do
  use Ecto.Schema
  import Ecto.Query

  schema "roles" do
    field :code, :string
  end
end

And our user could have roles like so:

defmodule MyApp.Accounts.User do
  schema "users" do
    many_to_many :roles, MyApp.Accounts.Role, join_through: "user_roles"
  end
end

Now, say we wanted to query a user by a specific role, we could define a join and then filter by the role.code:

defmodule MyApp.Accounts.User do
  ...

  def join_roles(query \\ User) do
    join(query, :inner, [u], r in assoc(u, :roles))
  end
end

This is where it get’s a bit weird, we would need to define a specific function for the query with joined roles since the binding position would be in the second position and would go like this:

defmodule MyApp.Accounts.User do
  def by_roles(query \\ join_roles(), role_codes) do
    where(query, [user, role], role in ^role_codes)
  end
end

While this is great for this situation, it’s not very reusable, instead we can use the as keyword to create a reusable function for roles like so:

defmodule MyApp.Accounts.User do
  ...

  def join_roles(query \\ User) do
    join(query, :inner, [u], r in assoc(u, :roles), as: :role)
  end
end

In our Role schema we could then define a by_code function that looks for a role binding:

defmodule MyApp.Accounts.Role do
  ...

  def by_code(query \\ Role, code) do
    where(query, [role: r], r.code == ^code)
  end
end

But wait there’s an error?!

Yes, we have an error, if we use this as the first function in our query chain, we’re going to pass in Role which will not have a binding for [role: r] since we didin’t define an as keyword. To get around this we should strive to define all our default query parameters like so:

defmodule MyApp.Accounts.Role do
  ...

  def from(query \\ User), do: from(query, as: :user)

  # This allows us to do the following
  def by_code(query \\ from(), code) do
    where(query, [role: r], r.code == ^code)
  end
end

By doing this we can now use Role.by_code() as the first function in a query chain and call this all together like so:

def find_user_by_code(code) do
  User.join_roles() |> Role.by_code(code) |> Repo.all
end

# We can also use it to filter on the Role table itself
def find_role_by_code(code) do
  code |> Role.by_code |> Repo.one
end

By using the as keyword in combination with a default query \\ from() parameter, we can define all our filtering functions in our original schema module, and re-use the functions across any joins we may have in the future. This allows us to re-use all our query definitions in different combinations without creating many variants.

Reusable Contexts

Our final goal, is to be able to re-use the functions from within our MyApp.Accounts context. After all our work our current context function currently looks like the following:

defmodule MyApp.Accounts do
  alias MyApp.{Repo, Accounts.User}

  def find_user_by_code(code) do
    User.join_roles() |> Role.by_code(code) |> Repo.all
  end

  def find_role_by_code(code) do
    code |> Role.by_code |> Repo.one
  end

  def find_user_by_id(id) do
    id |> User.by_id |> Repo.all
  end

  def find_user_by_first_name(name) do
    name |> User.by_first_name |> Repo.all
  end

  def find_user_by_last_name(name) do
    name |> User.by_last_name |> Repo.all
  end

  def find_by_first_and_last_name(first_name, last_name) do
    first_name |> User.by_first_name |> User.by_last_name(last_name) |> Repo.find
  end
end

We can start refactoring this by building ourselves a more abstract api to use, and making it so each parameter is combine into a query part, then we can combine them in any way we want, without defining a new function like how we had to for find_by_first_and_last_name.

To do this we can combine the parameters and add to our query like so:

def find_user(query \\ User.from(), params)

def find_user(query, %{first_name: first_name} = params) do
  query
    |> User.by_first_name(first_name)
    |> find_user(Map.delete(params, :first_name))
end

def find_user(query, %{last_name: last_name} = params) do
  query
    |> User.by_last_name(last_name)
    |> find_user(Map.delete(params, :last_name))
end

def find_user(query, %{id: id} = params) do
  query
    |> User.by_id(id)
    |> find_user(Map.delete(params, :id))
end

def find_user(query, %{role_code: role_code} = params) do
  query
    |> User.join_roles
    |> Role.by_code(role_code)
    |> find_user(Map.delete(params, :role_code))
end

def find_user(query, _params) do
  Repo.one(query)
end

Doing this we can now call our find_user function in a few ways:

Accounts.find_user(%{id: 1})
Accounts.find_user(%{first_name: "Bilbo", last_name: "Baggins"})
Accounts.find_user(%{first_name: "Bilbo", role_code: "ADMIN"})

As we can see above, because of the fact we’re adding onto each query based off the available parameters, we’re able to construct a query unique to our parameter set, and build a simple API for ourselves via our Context.

Review

We’ve come a long way from our first naive implementation of schemas and context modules, to sum it up there are a few things we should do for re-usable Ecto code:

Keep schemas for query definitions and contexts for query glue and Repo execution
Always default query functions to from(MySchema, as: :my_schema) so that we can re-use our functions across joins
Use as bindings from joins and inside query functions such as where so they can be reused across different joins
Use a default query in our context functions so we can build ouselves an abstracted api and have parameter combinations to create queries
This doesn’t cover every case, sometimes we still need a very specific query and we need a seperate function, this is totally fine to do!

Extra Bonus

To encapsulate the majority of this reusability, we can utilize a little library I created named EctoShorts. After installing it we can redefine our schema module like so:

defmodule MyApp.Accounts do
  def find_user(params), do: EctoShorts.Actions.find(User, params)
end

Now we can do all the same queries but without all the manual query definitions

Accounts.find_user(%{id: 1})
Accounts.find_user(%{first_name: "Bilbo", last_name: "Baggins"})
# Our role code is slightly different in syntax
Accounts.find_user(%{first_name: "Bilbo", role: %{code: "ADMIN"}})

We can also define an all_users function like so:

defmodule MyApp.Accounts do
  alias EctoShorts.Actions

  def find_user(params), do: Actions.find(User, params)
  def all_users(params), do: Actions.all(User, params)
end

Ontop of simplifying our code, we gain access to filtering on all fields that are present on the schema, as well as all relations ontop of the schema. EctoShorts also contains a few extras that allow us to filter using limit, offset as well as before or after a specific time. We can also pass in preloads. To put this all together could look like:

MyApp.Accounts.all_users(%{
  first_name: ["Frodo", "Bilbo", "Merry", "Pippin"], # Find any of these
  last_name: %{!=: "Baggins"} # Last name not baggins,
  role: %{code: "ADMIN"},
  last: 5, # Latest 5 items
  end_date: NaiveDateTime.utc_now() # Created before today,
  preload: [:roles]
})

As a bonus for any filters that don’t fit within the confines of EctoShorts, we can also pass in custom queries as our first paramater:

def find_user(%{top_performer: true} = params) do
  Actions.all(User.by_top_performing_user(), Map.delete(params, :top_performer))
end

While we could manually define all these queries and create an api to pull it all together, it’s much easier to have it done for us dynamically, which is where EctoShorts shines! Using this library allows us to create basic CRUD actions with easy and easily utilize the code in the rest of our application in a reusable fashion! One of the uses I personally find the best, is to utilize this MyApp.Accounts.find_user from within an absinthe resolver, and just pass the params through directly, this allows us to shortcut the majority of creating basic CRUD interfaces via GraphQL!

Start Your Free Preview Today!

Creating Reusable Ecto Code in Elixir

Creating Reusable Ecto Code

Splitting our definitions from glue

Reusing Queries

Reusability Over Joins Using `as`

But wait there’s an error?!

Reusable Contexts

Review

Extra Bonus

Links

Start Your Free Preview Today!

Creating Reusable Ecto Code in Elixir

Creating Reusable Ecto Code

Splitting our definitions from glue

Reusing Queries

Reusability Over Joins Using as

But wait there’s an error?!

Reusable Contexts

Review

Extra Bonus

Links

Reusability Over Joins Using `as`