Enum Part 2

In Part 1, you were introduced to the Enum module. We also covered 2 really important Enum functions.

• Enum.each/2
• Enum.map/2

Now we will continue looking at Enum. We’ll start by looking the powerhouse Enum.reduce/3 function. We then re-visiting anonymous functions because they are used so frequently with Enum.

Using Enum.reduce/3

Enum.reduce/3 is a powerful function. It may feel a little foreign to people new to Elixir at first. Particularly if you haven’t use a “reduce” function in other languages or frameworks. However, this function is definitely worth understanding.

Reduce executes the function for each item in the enumerable. The special power is that it has an accumulator and the result of the reduce function is the accumulated value. Remember the recursion exercises? Passing an accumulator solves a number of looping situations. Internally, reduce is implemented using the recursion pattern. So using reduce can be a handy shortcut!

Let’s review the recursive way we summed a list of numbers:

defmodule Playing do
  def sum([num | rest], acc) do
    sum(rest, acc + num)
  end

  def sum([], acc), do: acc
end

data = [1, 2, 3, 4, 5]
Playing.sum(data, 0)
#=> 15

Now let’s see the same solution using Enum.reduce/3.

data = [1, 2, 3, 4, 5]
Enum.reduce(data, 0, fn(num, acc) ->
  num + acc
end)
#=> 15

The 1st argument is the list being enumerated. The 2nd argument is the initial value of the accumulator or acc. The anonymous function receives 2 arguments. The first is a value from the list (ie num) and the second is the current accumulated value (ie acc). The result of the anonymous function becomes the new accumulator value.

We could further condense this using the anonymous function shorthand.

data = [1, 2, 3, 4, 5]
Enum.reduce(data, 0, &(&1 + &2))
#=> 15

Exercise #4 – Compute Order Total

This is re-implementing a recursion exercise. Using Enum.reduce/3, process a list of OrderItem structs to compute an order total. Refer to the following structs for details as needed:

• lib/schemas/order_item.ex
• lib/schemas/item.ex

The order total is computed as an Item’s price * the quantity being ordered. Summing all of those together for the total order price.

Review the tests to see the data samples being tested.

mix test test/enum_shortcut_test.exs --only describe:"order_total/1"

  def order_total(order_items) do
    Enum.reduce(order_items, 0, fn %OrderItem{} = order_item, total ->
      order_item.quantity * order_item.item.price + total
    end)
  end

Anonymous function clauses

When working with the Enum module, we are frequently also working with anonymous functions. We talked about using the shorthand version which can be very condensed. There is a benefit to using the longer form version of anonymous functions as well. It supports multiple clauses which we can use for pattern matching. Yes. You read that right. Let’s see it to understand it.

The structure of it looks like this:

fn
  pattern_clause_1 ->
    expression_when_clause_1_matches
  pattern_clause_2 ->
    expression_when_clause_2_matches
  pattern_clause_3 ->
    expression_when_clause_3_matches
end

Here’s a working version that you can edit and play with.

fun = 
  fn
    0 ->
      :ok
    nil ->
      "You passed nil!"
    num when is_number(num) ->
      "#{num} + 1 = #{num + 1}"
  end

fun.(0)
#=> :ok

fun.(1)
#=> "1 + 1 = 2"

fun.(2)  
#=> "2 + 1 = 3"

fun.(nil)
#=> "You passed nil!"

With this new understanding, let’s apply it to conditionally increment an accumulator in a reduce function.

Exercise #5 – Counting Active

One of the recursion exercises we did previously used function clause pattern matching to identify when we should conditionally increment the accumulator. Using 2 different ways, let’s do that with Enum.reduce/3 just to get the practice. In both cases, the goal is the same. Given a list of Customers, count how many are active. A Customer has an active boolean flag. Conditionally increment the accumulator counter only when a Customer is active. Refer to the following as needed:

• lib/schemas/customer.ex – Defines the customer struct
• test/recursion_test.exs – Shows data examples your code needs to process

There are tests for this exercise here:

mix test test/enum_shortcut_test.exs --only describe:"count_active/1"

Using an anonymous function with clauses

Using the approach we just discussed with anonymous functions supporting multiple pattern clauses, write the Elixir code that makes the tests pass.

  def count_active(customers) do
    Enum.reduce(customers, 0, fn
      %Customer{active: true}, total -> total + 1
      _customer, total -> total
    end)
  end

Using module function clauses

Let’s solve the same problem in a different way. Write private defp do_count_active function clauses that use pattern matching to conditionally increment the accumulator. Your call to Enum.reduce/3 can pass the function by reference (meaning the version where you specify the arity).

  def count_active(customers) do
    Enum.reduce(customers, 0, &do_count_active/2)
  end

  defp do_count_active(%Customer{active: true}, total), do: total + 1
  defp do_count_active(_customer, total), do: total

Reduce accumulates more than a number

Up to this point, all the reduce examples have been focused on accumulating a number like a count or a total value. It is important to realize that you can accumulate any data structure that makes sense for your problem.

As an example, let’s take a list of maps where each map has a :status key. We want to group all the maps by their status values. Let’s look at the example.

data = [
  %{name: "Tammy", status: "pending"},
  %{name: "Joan", status: "active"},
  %{name: "Timothy", status: "closed"},
  %{name: "Alan", status: "active"},
  %{name: "Nick", status: "active"}
]

results = 
  Enum.reduce(data, %{}, fn(%{status: status} = item, acc) ->
    case Map.fetch(acc, status) do
      {:ok, items} ->
        Map.put(acc, status, [item | items])
        
      :error ->
        Map.put(acc, status, [item])
    end
  end)
  
results
#=> %{
#=>   "active" => [
#=>     %{name: "Nick", status: "active"},
#=>     %{name: "Alan", status: "active"},
#=>     %{name: "Joan", status: "active"}
#=>   ],
#=>   "closed" => [%{name: "Timothy", status: "closed"}],
#=>   "pending" => [%{name: "Tammy", status: "pending"}]
#=> }

Notice that the initial value for the accumulator (the 2nd argument in Enum.reduce/3) is an empty map %{}. The current accumulated value is passed to our function for each item. If the the status key already exists, we add our item to the list.

The message here is that you can use “reduce” to build and transform data structures in powerful ways. You can “reduce” into other data structures, not just to count up totals.

Enumerate over non-lists

We’ve covered lots of examples of enumerating over a list or a range. Did you know you can also enumerate over a map? When you do this, the “item” passed to your function is a tuple of {key, value}. Let’s see an example:

data = %{name: "Howard", age: 32, email: "howard@example.com"}

Enum.each(data, fn({key, value}) ->
  IO.puts("The #{key} = #{inspect(value)}")
end)
#=> The age = 32
#=> The email = "howard@example.com"
#=> The name = "Howard"
#=> :ok

You can also make a custom struct work with Enum. To do this, you need to implement the Enumerable protocol. This means you implement 4 functions for your data structure and then it can mapped, reduced, and more.

Enum.map/2 and pipelines

The Pipe Operator takes in “data” or an “expression” on the left. A list is data too! Let’s see an example of processing a small list using a pipeline. This could be a list of products to order, customers to bill, etc.

This sample is already decorated with several IO.inspect calls to help see into the transformations at each step.

defmodule PipePlay do
  
  def lists do
    [1, 2, 3, 4, 5, 6]
    |> IO.inspect()
    |> Enum.map(fn(num) -> num * 10 end)
    |> IO.inspect()
    |> Enum.map(fn(num) -> num + 1 end)
    |> IO.inspect()
    |> Enum.map(fn(num) -> to_string(num) end)
  end

end
PipePlay.lists()
#=> [1, 2, 3, 4, 5, 6]
#=> [10, 20, 30, 40, 50, 60]
#=> [11, 21, 31, 41, 51, 61]
#=> ["11", "21", "31", "41", "51", "61"]

The list is piped through 3 different calls to Enum.map/2 where different functions are being applied to the list at each step. In this case we are using anonymous functions. When we re-write the anonymous functions to use the abbreviated & syntax, it makes our example code easier to look at. It removes a lot of the visual noise of parenthesis, arrows, and end‘s.

defmodule PipePlay do
  
  def lists do
    [1, 2, 3, 4, 5, 6]
    |> IO.inspect()
    |> Enum.map(& &1 * 10)
    |> IO.inspect()
    |> Enum.map(& &1 + 1)
    |> IO.inspect()
    |> Enum.map(&to_string(&1))
  end

end
PipePlay.lists()
#=> [1, 2, 3, 4, 5, 6]
#=> [10, 20, 30, 40, 50, 60]
#=> [11, 21, 31, 41, 51, 61]
#=> ["11", "21", "31", "41", "51", "61"]

Many Enum convenience functions

There are many functions in the Enum module. Nearly all of them are convenience functions and you should turn to them first. Please note that many of the Enum functions are actually implemented using recursion. That’s how powerful the recursive list processing pattern is.

The Enum module is an essential module to get comfortable with. Here are a few highlights beyond what we already covered:

• Enum.all?/2
• Enum.any?/2
• Enum.filter/3
• Enum.find/3
• Enum.sort/2

Enum Recap

We covered a lot through Part 1 and 2! One important point to take is that we did everything here using immutable data! That’s so cool! I hope you see that you can still accomplish what you need to do in a non-mutating way. You may need to change the way you think about things, but now you know you can do it!

Let’s review the three Enum module functions that are commonly used when needing to loop.

• Enum.each/2 – You will likely end up using Enum.each/2 the least of the three. Since each has no meaningful return value and you can’t mutate anything inside the loop, there aren’t as many opportunities to actually use it. It is only used for creating other system side-effects.
• Enum.map/2 – When you want to “modify” a list of things, this is your go-to function. It returns a new list where each item had your function applied to it. It is probably the function you want if you are most familiar with mutating and imperative looping.
• Enum.reduce/3 – A real power house function. It iterates an enumerable applying a function. However, rather than returning a new list, it returns the accumulator. Your function defines what to accumulate and how to do it.

Other topics covered

• Ranges – A range is a number sequence expressed like this: 1..10. It is a good tool for looping a desired number of times. It works best when the bounds are developer defined.
• Anonymous functions – All the Enum functions we looked at take something to enumerate and a function to apply to each item. Anonymous functions are very commonly used for these. We covered more on how we can declare and use anonymous functions in Elixir.
  • Shorthand syntax – Reminder of the shorthand you can use for anonymous functions. fun = &IO.puts(&1)
  • Pass by reference – Can pass a function by it’s reference fun = &IO.puts/1. Works great for local functions that do more pattern matching or handle more complex logic.
  • Multiple clauses – Anonymous functions support multiple pattern matching clauses! Work great for a small number of clauses and simple logic.
• Enum.reduce/3 – Can accumulate more than a number. You can create lists, maps, and other things.
• Enumerate non-lists – You can enumerate over maps and more, not just lists.
• Lists can be pipelined – Enum.map/2 is a great tool for that.