Thinking outside the pipe

Recently there have been an exhausting number of posts on the elixir-lang mailing list about the shortcomings of the pipe operator.

What is the pipe operator?

For those not familiar, the pipe operator is |> and is used like so:

a = foo(5)  
b = bar(a, 4)  
c = qux(b, 3)  
# or
qux(bar(foo(5), 4), 3)  
# or
foo(5) |> bar(4) |> qux(3)  

It's a macro that adds the expression on the left as the first argument to the call on the right. Let's look at the AST:

iex(1)> quote do: foo(5)  
{:foo, [], [5]}
iex(2)> quote do: bar(4)  
{:bar, [], [4]}
iex(3)> quote do: bar(foo(5), 4)  
{:bar, [], [{:foo, [], [5]}, 4]}

Pretty straightforward. We see what the AST looks like for each of these calls.

iex(4)> quote do: foo(5) |> bar(4)  
{:|>, [context: Elixir, import: Kernel], [{:foo, [], [5]}, {:bar, [], [4]}]}

Here we have a bigger tree in place, but if we expand the |> macro, we get a familiar result.

iex(5)> Macro.expand((quote do: foo(5) |> bar(4)), __ENV__)  
{:bar, [], [{:foo, [], [5]}, 4]}

So again, nothing special. Just syntactic sugar.

Why is the pipe operator useful?

There is a technique present in most OO languages called method chaining. Consider the following Ruby code:

list.uniq.count  

Look at this example in Elixir without pipes:

uniq_list = Enum.uniq(list)  
Enum.count(uniq_list)  

That's fine. Nothing wrong with that. With the pipe operator it cleans up a bit.

list |> Enum.uniq |> Enum.count  

Sometimes we break it up into multiline statements. This makes it easier to add or remove code (as well as making for easier to follow diffs).

# Ruby
list  
  .uniq
  .count

# Elixir
list  
  |> Enum.uniq
  |> Enum.count

And that's about it. That's the pipe operator.

When not to use the pipe operator

Here's where the confusion is coming. Elixir developers love the pipe operator because it makes a lot of common situations easier to follow. Many Elixir functions end up looking a bit like this (pulled from a production app I'm running):

  def get_people(params) do
    params
      |> group_params
      |> filter_params
      |> transform_params
      |> build_queries
      |> validate_query
      |> make_request("people_current", "people", limit, offset)
      |> produce_result
  end

This captures the flow of data pretty well. However, let's say validate_query doesn't always return a successful value. It doesn't make sense to always call make_request after validate_query. So what's the solution? Less pipes.

  query = params
      |> group_params
      |> filter_params
      |> transform_params
      |> build_queries
      |> validate_query

  case query do
    {:ok, valid_query} ->
      query
        |> make_request("people_current", "people", limit, offset)
        |> produce_result
    {:error, errors} ->
      produce_error_result(errors)
  end

Be explicit about branching in your application. If you need to branch on the result of a function, don't use pipe. It's perfectly fine to break up a series of pipe calls.

Idiomatic Pipe Usage

For people new to Elixir, the pipe operator is usually met with a positive reaction. There's an impression they get that the pipe operator is central to programming in Elixir. To see if this is truly the case,I took a look at some of the most popular Elixir libraries to see how they use pipes.

Project Total Lines Pipes Pipe Frequency
Elixir 38214 165 1: 89, 2: 18, 3: 9, 4: 2, 5: 1
Phoenix 10985 308 1: 32, 2: 65, 3: 25, 4: 7, 5: 6, 8: 1
Plug 6202 121 1: 22, 2: 31, 3: 6, 4: 2, 5: 1, 6: 1
Poison 780 8 1: 6, 2: 1
Total Lines
Total lines in lib
Pipes
Total number of pipes in lib
Pipe Frequency
How pipes are chained together ("4: 5" means there are 5 counts of using 4 pipes in a single statement)

While pipes aren't uncommon, they aren't used as often as I expected. Phoenix uses them the most with 3% of lines using a pipe operator. Additionally, long pipelines are quite rare. Usually only one or two pipes is used in a single statement. Phoenix pushes it a bit and has quite a few cases of three pipes in a statement. It even has a pipeline that is 8 calls long! But overall the usage of the pipe operator is pretty conservative.

Conclusion

Pipes are there to make code more readable, but they don't always make sense to use. In practice, the largest elixir projects use pipes sparingly and avoid large pipelines. That doesn't mean you should avoid using pipes. Just recognize that they aren't central to most Elixir applications.

comments powered by Disqus