Clojure macros for beginners
 |
| Bjørvika |
Imagine you are about to write an assertions library for Clojure, similar to FEST Assertions, ScalaTest assertions or Hamcrest. Of course there are such existing but this is just for educational purposes. What we essentially need first is a
assert-equals function used like this:(assert-equalsOf course this is more than trivial:
(count (filter even? primes)) 1)
(defn assert-equals [actual expected]Quick test with incorrectly defined
(when-not (= actual expected)
(throw
(AssertionError.
(str "Expected " expected " but was " actual)))))
primes vector:user=> (def primes [0 2 3 5 7 11])Cool, but imagine this test failing on CI server or seeing this in your terminal. There is no context, maybe you’ll get test name if you’re lucky. “
#'user/primes
user=> (assert-equals (count (filter even? primes)) 1)
AssertionError Expected 1 but was 2
Expected 1 but was 2” tells us nothing about the nature or root cause of the problem. Wouldn’t it be great to see:AssertionError Expected '(count (filter even? primes))' to be 1 but was 2You see this? Assertion error now gives us full expression that yielded incorrect result. We can see from the very first second what the issue can be. However, there is a problem. Big one. By the time we are throwing
AssertionError, original expression is lost. We got actual value as an argument and we have no idea where did that value came from. It could have been a constant, result of expression like (count (filter even? primes)) or even a random value. Function arguments are computed eagerly and there is no way to access code that produced these arguments.Entering macros
Macros and functions in Clojure are not independent or orthogonal. In fact, they are almost the same:- Functions execute at run time, they take and produce data (values). Conceptually one can replace every (pure) function invocation with its value.
- Macros execute at compile time, they take and produce code. Conceptually one can replace (expand) every occurrence of macro with its value.
What does it all mean and how can it help us? Let’s jump straight into writing our first (incorrect) macro and improve it step-by-step to finally achieve desired result. To keep samples focused I skip throwing an
AssertionError and leave only equality condition:user=> (defmacro assert-equals [actual expected]Works? In fact we are very far from having a correct version:
(= expected actual))
#'user/assert-equals
user=> (assert-equals 2 2)
true
user=> (assert-equals 2 3)
false
user=> (assert-equals (inc 5) 6)
false
user=> (def x 1)
#'user/x
user=> (assert-equals (+ x 2) 3)
false
1 + 2 is definitely equal to 3, yet it returns false. In order to appreciate this behaviour and call it “feature” rather than “bug” we must deeply understand what just happened. Remember, macros are executed at compile time, right? And they are almost ordinary functions. So, the compiler executes assert-equals. However during compilation it can’t possibly know the values of variables like x, therefore it can’t eagerly evaluate macro arguments. We don’t even want that, as you see later.Instead the compiler passes Clojure code, literally. The
actual parameter is (inc 5) - literally, Clojure list holding two elements: inc symbol and 5 number. That’s all there is to it. expected is just a number. This means that inside macro we have full access to Clojure source code enclosed by that macro.So maybe you can now guess what happens. Clojure compiler executes macro definition, that is
(= expected actual). As far as the compiler is concerned, actual is a list (inc 5) while expected is a number 6. List can never possibly be equal to a number. Thus macro returns false, just like any other function can return it. Later on Clojure compiler replaces (assert-equals (inc 5) 6) expression with the outcome of macro, which happens to be… false. We said before that macro should return valid Clojure code (represented using Clojure data structures). false is valid Clojure code!Now we know that instead of evaluating
(= expected actual) by the compiler (after all, we don’t want the compiler to run our assertions, we only want to compile them!) we simply want to return code that represents this assertion. It’s not that hard!(defmacro assert-equals [actual expected] (list '= expected actual))Now our macro returns result of evaluating
(list '= expected actual) expression. The result happens to be… (= expected actual). That’s right, it looks like valid Clojure code, again. Extra quote ('=) was added so that = is interpreted as raw symbol rather than a function reference. Let’s take it for a test drive:user=> (assert-equals (inc 5) 6)
true
user=> (macroexpand '(assert-equals (inc 5) 6))
(= 6 (inc 5))
macroexpand and macroexpand-1 are your weapons of choice when debugging macros. Here you see that (assert-equals (inc 5) 6) is actually being replaced by (= 6 (inc 5)). This process happens at compile time, macros don’t exist at runtime. In your compiled code you are left with (= 6 (inc 5)). OK, so let’s restore the full functionality of throwing AssertionError. As you know by now, our macro should return Clojure code that includes equality check and throwing an exception. This becomes a bit unwieldy:(defmacro assert-equals [actual expected]Notice how every single symbol has to be escaped (
(list 'when-not (list '= actual expected)
(list 'throw
(list 'AssertionError.
(list 'str "Expected " expected " but was " actual)))))
'when-not, 'throw, 'AssertionError., …), otherwise compiler will try to evaluate it at compile time. Moreover list in Clojure denotes function call so we must proceed every list literal with (list ...) function call. If you are not that familiar with Clojure: (list 1 2) returns list of (1 2) while (1 2) will throw an exception since 1 number is not a function.Ugly or not, it works:
user=> (assert-equals (inc 5) 6)We barely reproduced what original
nil
user=> (assert-equals 5 6)
AssertionError Expected 6 but was 5
assert-equals function was doing and the first commandment of writing macros is: don’t write macros if function is sufficient. But before we go further, let us clean up what we have so far. Typical macro definition consists of lots of Clojure code that has to be escaped and not that much live values like actual and expected in our case. So there is a smart default - instead of quoting everything except few items, quote everything upfront and selectively unquote things. This is called syntax-quoting (using ` character) and unquoting is done via ~ operator. Look carefully: we syntax quote whole result and selectively unquote what was previously not quoted:(defmacro assert-equals [actual expected]This is equivalent to previous definition but looks much better, almost entirely like valid Clojure code. Let’s employ
`(when-not (= ~actual ~expected)
(throw
(AssertionError.
(str "Expected " ~expected " but was " ~actual)))))
macroexpand-1 to see how our macro is expanded during compilation. macroexpand would work as well, but since when-not is also a macro (!) it would be recursively expanded, cluttering output:user=> (macroexpand-1 '(assert-equals (inc 5) 6))It’s like templating language embedded within that language! Notice how
(when-not
(= (inc 5) 6)
(throw
(java.lang.AssertionError.
(str "Expected " 6 " but was " (inc 5)))))
(inc 5) piece of code was inserted instead of ~actual twice. Keep that in mind. Also experiment by removing unquote (~) symbol here or there. Use macroexpand-1 to figure out what is going on.Remember, our ultimate goal was to show
actual expression in its full glory, not only its value. (AssertionError.What should we put in place of
(str "Expected '???' to be " ~expected " but was " actual-value#))))))
??? to print “(inc 5)” string. We know that value of actual is not 6 but a list with two items: (inc 5). Can we somehow quote that list again so that it no longer evaluates at run-time but instead is treated as a data structure? Of course, we know how to quote things!(defmacro assert-equals [actual expected]
`(let [~'actual-value ~actual]
(when-not (= ~'actual-value ~expected)
(throw
(AssertionError.
(str "Expected '" '~actual "' to be " ~expected " but was " ~'actual-value))))))
'~actual, oh dear! quote unquote actual. This translates to '(inc 5). And that’s it! Look how descriptive assertion error messages are:user=> (assert-equals (inc 5) 5)Expanding this macro manually reveals how it is translated by the compiler (edited to improve readability):
AssertionError Expected '(inc 5)' to be 5 but was 6
user=> (assert-equals (count (filter even? primes)) 1)
AssertionError Expected '(count (filter even? primes))' to be 1 but was 2
user=> (macroexpand-1 '(assert-equals (inc 5) 5))There is really no magic here, we could have written that ourselves. But macros avoid lots of repetitive work.
(when-not
(= (inc 5) 5)
(throw
(java.lang.AssertionError.
(str "Expected '" (quote (inc 5)) "' to be " 5 " but was " (inc 5)))))
Bindings in macros
Our solution so far has one major issue. Imagine we are testing impure or slow function like this:(def question "Answer to the Ultimate Question of Life, The Universe, and Everything")As you can see it returns wrong result, which can be easily proved in a unit test:
(defn answer [q]
(do
(println "Computing for 7½ million years...")
41))
user=> (assert-equals (answer question) 42)The error message is fine, but notice that “
Computing for 7½ million years...
Computing for 7½ million years...
AssertionError Expected '(answer question)' to be 42 but was 41
Computing...” statement was printed twice. Clearly because impure answer function was called twice as well. Macro expansion reveals why:user=> (macroexpand-1 '(assert-equals (answer question) 42))
(when-not
(= (answer question) 42)
(throw (java.lang.AssertionError.
(str "Expected '" (quote (answer question)) "' to be " 42 " but was "
(answer question)))))
(answer question) appears twice (not counting quoted one), once during comparison and second time when we generate assertion message. This is rarely desired, especially when function under test has side effects. The solution is simple: precompute (answer question) once, store it somewhere and reference when needed. But there is a twist: declaring let bindings inside macros is tricky. Sometimes you might hit unexpected name shadowing and overriding when names of variables inside macro collide with the ones used in user code. Not going into much detail, using (gensym) or convenient # suffix is enough to keep our macros safe. In both cases Clojure compiler will produce unique names making sure they don’t collide. Our final solution looks like this:(defmacro assert-equals [actual expected]This time
`(let [actual-value# ~actual]
(when-not (= actual-value# ~expected)
(throw
(AssertionError.
(str "Expected '" '~actual "' to be " ~expected
" but was " actual-value#))))))
actual-value# binding is used to compute actual only once:user=> (macroexpand-1 '(assert-equals (answer question) 42))Extra suffix replacing
(let [actual-value__264__auto__ (answer question)]
(when-not (= actual-value__264__auto__ 42)
(throw
(java.lang.AssertionError.
(str "Expected '" (quote (answer question)) "' to be " 42 "
but was " actual-value__264__auto__)))))
# symbol makes sure actual-value is not colliding with any other symbol.Summary
Ourassert-equals macro is not the most comprehensive one, just like this tutorial. But it gives you some impression of what macros can do and how they work. If you need further resources, check out this great macro tutorial (part 2 and 3). If you like the idea of enhanced assertions, Power Assertions in Groovy are even more comprehensive. But I bet this behaviour can be reproduced in Clojure macros!Tags: clojure, functional programming