Erlang EUnit – continuation 1 – Fixtures

This time we will learn how to create a specific environment for our specific tests. As usual, all this can be found in the “EUnit User’s Guide”.

Why would I want to create specific environments?

Consider the case that we wish to perform EUnit tests on on a stateful system (a system with state that is), it could be a server, an fsm, whatever holds and updates a state.

How do I create these environments?

Meet your friend the fixture! There are three kind of major fixtures, Setup / Node / Foreach, this blog entry will only cover the basics of Setup. A fixture is the EUnit name for test-environment. By using fixture statements, we can control how a state (fixture) is created and destroyed. In EUnit terms, this is called

setup/0
cleanup/1

Normally, our EUnit tests run in the same basic environment, as was the case for the mylist EUnit tests, as we shall se next, this is often not desirable when testing a server which contains state (a stateful process).

An example stateful server!

Before (a-hah!) I show some fixture examples, I will show the example code that will be tested using fixtures! It will become clear why the fixture is needed.

-module(numberserver).
-export([start/0,stop/0,op/2,get/0,init/0]).
start() ->
    Pid = spawn_link(?MODULE,init,[]),
    register(?MODULE,Pid),
    ok.
stop() -> ?MODULE ! stop, unregister(?MODULE).
op(Op,Num) -> ?MODULE ! {Op,Num}, ok.
get() ->
    ?MODULE ! {get_result,self()},
    receive
        X -> X
    end.
init() -> loop(basic).
loop(E) ->
    receive
        stop -> ok;
        {get_result,From} ->
            From ! E, loop(E);
        {Op,Num} -> loop(result(Op,E,Num))
    end.
result(_,basic,X) -> X;
result('+',X,Y) -> X + Y;
result('*',X,Y) -> X * Y;
result('-',X,Y) -> X - Y;
result('/',X,Y) -> X / Y.

The code above gives us a small server which performs basic binary operations (operations that take two operands), based on which operator and number is sent with the op/2 function. Compilation , start , usage and stop(ing) is shown below. As usual, compilation and execution is performed from the root directory of my “project”.

zen:EunitBasic2 zenon$ tree .
.
├ ── compile
├── ebin
├── src
│ └── numberserver.erl
└── test

zenon:EunitBasic2$ erlc -o ebin/ src/*.erl
zenon:EunitBasic2$ erl -pa ebin/
Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.5 (abort with ^G)
1> numberserver:start().
ok
2> numberserver:op('+',10).
ok
3> numberserver:op('*',2).
ok
4> numberserver:op('-',5).
ok
5> numberserver:op('/',3).
ok
6> numberserver:get().
5.0
7> numberserver:stop().
true
8>
(note the 5.0, this is due to the division which produced a non-integer result).

As you could see, this process obviously maintain some kind of state (hint: the previous computation result), and the subsequent calls use the state as second operand for each binary operation. Thus, when testing the different computations, we wish a clean untouched server for each run. (To exemplify further: assume we are testing basic addition,in one test generating function and later wish to test multiplication, now, in between each test we would ideally want the server to be reset in some way). This is where the fixture comes in.

Basic fixturing (state setup / cleanup)

The most basic fixture example does setup only (=creation only). This would be achieved with the 3-tuple

{ setup,
  SETUP-FUNCTION/0 ,
  TESTSET / INSTANTIATOR/1 }

SETUP-FUNCTION must be a function with arity 0 (zero) and shall perform all necessary operations for creating the encompassing environment.

TEST can be a an EUnit Control operator (more on this in next post), or a test primitive, an INSTANTIATOR/1 is a function receiving the result from the SETUP-FUNCTION and generated instantiated test-sets. I shall use the basic _test(Expr) macro here as is customary (note: there is nothing special about the _test/1 macro, it just returns the the given Expr as a test-function.

first_additon_test_() ->
     { setup,
       fun setup/0,
       fun cleanup/1,
       ?_test(
          begin
              numberserver:op('+',2),
              numberserver:op('*',3),
              ?assertEqual(6,numberserver:get())
          end)}.

This extremely simple test could as well have been written as

-module(numberserver_tests).
-include_lib("eunit/include/eunit.hrl").
first_additon_test_() ->
    numberserver:start(),
    numberserver:op('+',2),
    numberserver:op('*',3),
    ?assertEqual(6,numberserver:get()).

Let’s test this, save the code in numberserver_tests.erl inside the test/ directory, compile and run.

zenon:EunitBasic2$ erlc -o ebin/ src/*.erl test/*.erl
zenon:EunitBasic2$ erl -pa ebin/
zenon:EunitBasic2$ erl

Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.5 (abort with ^G)
1> eunit:test(numberserver,[verbose]).
======================== EUnit ========================
module 'numberserver'
module 'numberserver_tests'
numberserver_tests:6: first_additon_test_...ok
[done in 0.004 s]
[done in 0.004 s]
=======================================================
Test passed.
ok

Great, but what happens if we run it again?

2> eunit:test(numberserver,[verbose]).
======================== EUnit ========================
module 'numberserver'
module 'numberserver_tests'
*** context setup failed ***
::error:badarg
in function erlang:register/2
called as register(numberserver,<0.48.0>)
in call from numberserver:start/0

=======================================================
Failed: 0. Skipped: 0. Passed: 0.
One or more tests were cancelled.
error

Uh oh, obviously we can’t register the same numberserver in the same environment again. Seems like we need to do some kind of cleanup (hint hint). The cleanup/1 function is automatically given one argument, the result from the setup.

setup/0 -> X :: any()
cleanup( X :: any() ) -> any()

So, if your setup returns a Pid, you could use that pid in the cleanup to perform stops / kills, whatnot. As is, setup/0 is excuted before any tests are performed, and cleanup/1 is executed after all tests have been performed regardless of test errors or crashes. This would use the 4 tuple setup

{ setup,
  SETUP-FUNCTION/0,
  CLEANUP-FUNCTION/1,
  TEST/INSTANTIATOR }

CLEANUP-FUNCTION/1 is the arity-1 function that reverses the effect of SETUP, and TEST is as usual a test-primitive or an EUnit control term, the INSTANTIATOR/1 receives the same argument as cleanup, namely the result from SETUP.

Setup – Cleanup Fixture Example

-module(numberserver_tests).
-include_lib("eunit/include/eunit.hrl").
first_additon_test_() ->
     { setup,
       fun setup/0,
       fun cleanup/1,
       ?_test(
          begin
              numberserver:op('+',2),
              numberserver:op('*',3),
              ?assertEqual(6,numberserver:get())
          end)}.
first_multiply_test_() ->
     { setup,
       fun setup/0,
       fun cleanup/1,
       ?_test(
          begin
              numberserver:op('*',1),
              numberserver:op('*',3),
              numberserver:op('*',5),
              ?assertEqual(15,numberserver:get())
          end)}.

setup() ->
    numberserver:start().

cleanup(_Pid) ->
    numberserver:stop().

Let’s run it!

zen:EunitBasic2 zenon$ erlc -o ebin/ src/*.erl test/*.erl
zen:EunitBasic2 zenon$ erl -pa ebin
Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.5 (abort with ^G)
1> eunit:test(numberserver).
All 2 tests passed.
ok

What is so special about this? Well, for starters, we added one more tests, this second test will now succeed but would have failed previously. Why? Well, because we have a teardown and setup of the numberserver.

This was the basic setup and cleanup, you can find more in the manual pages, next time we will look at test representations.

Posted by gianfrancoalongi on August 30, 2010

https://erlcode.wordpress.com/2010/08/30/erlang-eunit-continuation-1-fixtures/