Knowing Basics of Setup, Cleanup understanding test representation, it’s time to look at EUnit test control. Test control encompassed the ability to specify
- If the TestSet should be run in a specific subprocess
- If the TestSet should be run in a specific subprocess on a specific node
- What timeout a TestSet should have
- If the STO’s in the TestSet should be run in a specific predetermined order
- If the STO’s in the TestSet should be run in parallell (if possible)
- If the STO’s in the TestSet should be run in parallell (if possible), but with the added control that no more than N of them may be run at the same time in parallell
Each of these control specifications are designated by tuples (like most things in EUnit), and may replace any TestSet (single test or deeplist) in the same place where the testset whas previousy placed: But, the replaced testset should then be placed inside the TestSet holder of the tuple.
Replacing a single direct testset as first element in a test generating function (remember the _test_())
testgenerator_test_() -> TESTSET. testgenerator_test_() -> {CONTROL-TUPLE,TESTSET}.
Replacing a deep list TestSet inside another TestSet.
testgenerator_test_() -> [ TESTSET1, TESTSET2 ]. testgenerator_test_() -> [ TESTSET1, {CONTROL, TESTSET2} ]
without further ado, here comes the examples!
Subprocess Specification for single process and single process on node.
{spawn, TestSet } {spawn, Node::atom(), TestSet }
Then, may be used as follows
testgeneratorA_test_() -> {spawn, nullarySTO1() }. testgeneratorB_test_() -> {spawn, [nullarySTO1(),tupleSTO1(),tupleSTO4()]}. testgeneratorC_test_() -> {spawn, 'eunit@zen', nullarySTO1() }. testgeneratorD_test_() -> {spawn, 'eunit@zen', [ nullarySTO1() , tupleSTO1(), tupleSTO4()]}. testgeneratorE_test_() -> [ nullarySTO1(), nullarySTO2(), {spawn, [ {spawn, 'eunit2@zen', tupleSTO1()}, {spawn, 'eunit2@zen', tupleSTO2()}, tupleSTO3() ] }, tupleSTO4() ].
As can be seen, the Control Tuple (CT) can be placed in any place of a TestSet, and puts that replaced testset into itself. Conceptually, it can be thought of as a function, CT(TestSet) which returns the CT with the testset inside.
To prove my point, I shall compile and run it.
zen:EunitBasic4 zenon$ tree . . ├── ebin ├── src │ └── mylist.erl └── test └── mylist_tests.erl 3 directories, 2 files
The mylists_tests.erl contains
-module(mylist_tests). -include_lib("eunit/include/eunit.hrl"). -compile(export_all). % Basics nullarySTO1() -> fun() -> ?assertEqual(6,mylist:sum([1,2,3])) end. nullarySTO2() -> fun nullarySTO1/0. nullarySTO3() -> fun mylist_tests:nullarySTO1/0. tupleSTO1() -> { mylist_tests, nullarySTO1 }. tupleSTO2() -> { mylist_tests, nullarySTO2 }. tupleSTO3() -> { mylist_tests, nullarySTO3 }. tupleSTO4() -> ?_test( fun() -> ?assertEqual(6,mylist:sum([1,2,3])) end ). tupleSTO5() -> ?_test( begin ?assertEqual(6,mylist:sum([1,2,3])) end ). tupleSTO6() -> ?_test( nullarySTO2() ). tupleSTO7() -> { 16, nullarySTO3() }. % Proof of concept testgeneratorA_test_() -> {spawn, nullarySTO1() }. testgeneratorB_test_() -> {spawn, [ nullarySTO1() , tupleSTO1(), tupleSTO4()]}. testgeneratorC_test_() -> {spawn, 'eunit@zen', nullarySTO1() }. testgeneratorD_test_() -> {spawn, 'eunit@zen', [ nullarySTO1() , tupleSTO1(), tupleSTO4()]}. testgeneratorE_test_() -> [ nullarySTO1(), nullarySTO2(), {spawn, [ {spawn, 'eunit2@zen', tupleSTO1()}, {spawn, 'eunit2@zen', tupleSTO2()}, tupleSTO3() ] }, tupleSTO4() ].
So, starting 3 shells (one eunit node, and two other nodes), as follows [ note: the other nodes should have the path to the ebins : AND have the code loaded by some means, please see this post to the fun problem issue ]
In orde to automate the code loading, the nl(Module) function is used
nl(Module) -- load module on all nodes
First shell
zen:EunitBasic4 zenon$ erl -pa ebin/ -sname eunit@zen -setcookie eunit Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.5 (abort with ^G) (eunit@zen)1>
Second Shell
zen:EunitBasic4 zenon$ erl -pa ebin/ -sname eunit2@zen -setcookie eunit Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.5 (abort with ^G) (eunit2@zen)1>
Third shell, starts, connects, loads on all nodes and runs
zen:EUnitBasic4 zenon$ erlc -o ebin/ src/*.erl test/*.erl zen:EUnitBasic4 zenon$ erl -pa ebin/ -sname base@zen -setcookie eunit Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.5 (abort with ^G) (base@zen)1> net_kernel:connect_node(eunit@zen). true (base@zen)2> net_kernel:connect_node(eunit2@zen). true (base@zen)3> nl(mylist_tests). abcast (base@zen)4> eunit:test(mylist,[verbose]). ======================== EUnit ======================== module 'mylist' module 'mylist_tests' mylist_tests: nullarySTO1...ok mylist_tests: nullarySTO1...ok mylist_tests: nullarySTO1...ok mylist_tests:13: tupleSTO4...ok mylist_tests: nullarySTO1...[0.003 s] ok mylist_tests: nullarySTO1...ok mylist_tests: nullarySTO1...ok mylist_tests:13: tupleSTO4...ok mylist_tests: nullarySTO1...ok mylist_tests: nullarySTO2...ok mylist_tests: nullarySTO1...ok mylist_tests: nullarySTO2...ok mylist_tests: nullarySTO3...ok mylist_tests:13: tupleSTO4...ok [done in 0.070 s] [done in 0.070 s] ======================================================= All 14 tests passed. ok (base@zen)5>
This concludes spawn, as can be seen, all the tests pass, even the node spawned ones. Don’t forget to load the code on the remote nodes first!
Timeout Control
{timeout, Time::number(), Tests}
The effect is that all tests in the TestSet “argument” are given the total time of Time seconds to complete. If the TestSet has not finished during the Time seconds, the TestSet is stopped abruptly without cleanup. Any Setup and Cleanup is also considered during this time.
testgeneratorA_test_() -> {timeout, 1, nullarySTO1() }. testgeneratorB_test_() -> {timeout, 1, [ nullarySTO1() , tupleSTO1(), tupleSTO4() ] }. testgeneratorC_test_() -> [ nullarySTO1(), {timeout, 5, [ {timeout, 1, tupleSTO1()}, {timeout, 2, tupleSTO2()}, tupleSTO3() ] }, tupleSTO4() ].
A cool detail is that it is fully legal to do as I did, that is, to nest levels of timeout, which makes total sense if you know some parts may hang indefinitely.
Thus my mylist_tests module now contains
-module(mylist_tests). -include_lib("eunit/include/eunit.hrl"). -compile(export_all). % Basics nullarySTO1() -> fun() -> ?assertEqual(6,mylist:sum([1,2,3])) end. nullarySTO2() -> fun nullarySTO1/0. nullarySTO3() -> fun mylist_tests:nullarySTO1/0. tupleSTO1() -> { mylist_tests, nullarySTO1 }. tupleSTO2() -> { mylist_tests, nullarySTO2 }. tupleSTO3() -> { mylist_tests, nullarySTO3 }. tupleSTO4() -> ?_test( fun() -> ?assertEqual(6,mylist:sum([1,2,3])) end ). tupleSTO5() -> ?_test( begin ?assertEqual(6,mylist:sum([1,2,3])) end ). tupleSTO6() -> ?_test( nullarySTO2() ). tupleSTO7() -> { 16, nullarySTO3() }. % % Proof of concept testgeneratorA_test_() -> {timeout, 1, nullarySTO1() }. testgeneratorB_test_() -> {timeout, 1, [ nullarySTO1() , tupleSTO1(), tupleSTO4() ] }. testgeneratorC_test_() -> [ nullarySTO1(), {timeout, 5, [ {timeout, 1, tupleSTO1()}, {timeout, 2, tupleSTO2()}, tupleSTO3() ] }, tupleSTO4() ].
And the tests are run as
zen:EUnitBasic4 zenon$ erlc -o ebin/ src/*.erl test/*.erl zen:EUnitBasic4 zenon$ erl -pa ebin/ Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.5 (abort with ^G) 1> eunit:test(mylist,[verbose]). ======================== EUnit ======================== module 'mylist' module 'mylist_tests' mylist_tests: nullarySTO1...ok mylist_tests: nullarySTO1...ok mylist_tests: nullarySTO1...ok mylist_tests:13: tupleSTO4...ok mylist_tests: nullarySTO1...ok mylist_tests: nullarySTO1...ok mylist_tests: nullarySTO2...ok mylist_tests: nullarySTO3...ok mylist_tests:13: tupleSTO4...ok [done in 0.027 s] [done in 0.027 s] ======================================================= All 9 tests passed. ok 2>
This concludes the Timeout demonstration as there is not more to say, sadly 🙂
Strict Order Of Tests
As is, EUnit is not forced to follow the in-module order or even in-test order of a TestSet, so it’s handy to have a control structure for this.
{inorder, Tests}
As basic as it seems, I shall supply a test for you, with the following module
-module(mylist_tests). -include_lib("eunit/include/eunit.hrl"). -compile(export_all). nullarySTO1() -> fun() -> ?assertEqual(6,mylist:sum([1,2,3])) end. nullarySTO2() -> fun nullarySTO1/0. nullarySTO3() -> fun mylist_tests:nullarySTO1/0. tupleSTO1() -> { mylist_tests, nullarySTO1 }. tupleSTO2() -> { mylist_tests, nullarySTO2 }. tupleSTO3() -> { mylist_tests, nullarySTO3 }. tupleSTO4() -> ?_test( fun() -> ?assertEqual(6,mylist:sum([1,2,3])) end ). tupleSTO5() -> ?_test( begin ?assertEqual(6,mylist:sum([1,2,3])) end ). tupleSTO6() -> ?_test( nullarySTO2() ). tupleSTO7() -> { 16, nullarySTO3() }. generator_test_() -> {inorder, [ nullarySTO1(), nullarySTO2(), nullarySTO3(), tupleSTO1(), tupleSTO2(), tupleSTO3(), tupleSTO4(), tupleSTO5(), tupleSTO6(), tupleSTO7() ]}.
And compilation, running, etc (you know the drill by now).
zen:EUnitBasic4 zenon$ erlc -o ebin/ src/*.erl test/*.erl zen:EUnitBasic4 zenon$ erl -pa ebin/ Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.5 (abort with ^G) 1> eunit:test(mylist). All 10 tests passed. ok 2> eunit:test(mylist,[verbose]). ======================== EUnit ======================== module 'mylist' module 'mylist_tests' mylist_tests: nullarySTO1...ok mylist_tests: nullarySTO2...ok mylist_tests: nullarySTO1...ok mylist_tests: nullarySTO1...ok mylist_tests: nullarySTO2...ok mylist_tests: nullarySTO3...ok mylist_tests:13: tupleSTO4...ok mylist_tests:14: tupleSTO5...ok mylist_tests:15: tupleSTO6...ok mylist_tests:16: nullarySTO1...ok [done in 0.030 s] [done in 0.030 s] ======================================================= All 10 tests passed. ok 3>
Now we are dead sure all tests where run in the strict appropriate order.
Parallel Test execution (if possible)
What if we have thousands of tests, which test single part of library routines? Like mylist tests with add, etc? Well, it sure makes sense to run them in parallel! Well, inparallel to the resque!
{inparallel, Tests}
Examples are more talkative and I will supply with such an example
-module(mylist_tests). -include_lib("eunit/include/eunit.hrl"). -compile(export_all). nullarySTO1() -> fun() -> ?assertEqual(6,mylist:sum([1,2,3])) end. nullarySTO2() -> fun nullarySTO1/0. nullarySTO3() -> fun mylist_tests:nullarySTO1/0. tupleSTO1() -> { mylist_tests, nullarySTO1 }. tupleSTO2() -> { mylist_tests, nullarySTO2 }. tupleSTO3() -> { mylist_tests, nullarySTO3 }. tupleSTO4() -> ?_test( fun() -> ?assertEqual(6,mylist:sum([1,2,3])) end ). tupleSTO5() -> ?_test( begin ?assertEqual(6,mylist:sum([1,2,3])) end ). tupleSTO6() -> ?_test( nullarySTO2() ). tupleSTO7() -> { 16, nullarySTO3() }. generator_test_() -> {inparallel, [ nullarySTO1(), nullarySTO2(), nullarySTO3(), tupleSTO1(), tupleSTO2(), tupleSTO3(), tupleSTO4(), tupleSTO5(), tupleSTO6(), tupleSTO7() ]}.
Lo and behold the run-timeth!
zen:EUnitBasic4 zenon$ erlc -o ebin/ src/*.erl test/*.erl zen:EUnitBasic4 zenon$ erl -pa ebin/ Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.5 (abort with ^G) 1> eunit:test(mylist,[verbose]). ======================== EUnit ======================== module 'mylist' module 'mylist_tests' mylist_tests: nullarySTO1...ok mylist_tests: nullarySTO2...ok mylist_tests: nullarySTO1...ok mylist_tests: nullarySTO1...ok mylist_tests: nullarySTO2...ok mylist_tests: nullarySTO3...ok mylist_tests:13: tupleSTO4...ok mylist_tests:14: tupleSTO5...ok mylist_tests:15: tupleSTO6...ok mylist_tests:16: nullarySTO1...ok [done in 0.004 s] [done in 0.004 s] ======================================================= All 10 tests passed. ok 2>
Inparallel but with upper bound on parallelism
Maybe you don’t wish more processes to be running in parallel than you have cores, or whatnot, thus, use inparallel with a maximum value of N.
-module(mylist_tests). -include_lib("eunit/include/eunit.hrl"). -compile(export_all). nullarySTO1() -> fun() -> ?assertEqual(6,mylist:sum([1,2,3])) end. nullarySTO2() -> fun nullarySTO1/0. nullarySTO3() -> fun mylist_tests:nullarySTO1/0. tupleSTO1() -> { mylist_tests, nullarySTO1 }. tupleSTO2() -> { mylist_tests, nullarySTO2 }. tupleSTO3() -> { mylist_tests, nullarySTO3 }. tupleSTO4() -> ?_test( fun() -> ?assertEqual(6,mylist:sum([1,2,3])) end ). tupleSTO5() -> ?_test( begin ?assertEqual(6,mylist:sum([1,2,3])) end ). tupleSTO6() -> ?_test( nullarySTO2() ). tupleSTO7() -> { 16, nullarySTO3() }. generator_test_() -> {inparallel,2, [ nullarySTO1(), nullarySTO2(), nullarySTO3(), tupleSTO1(), tupleSTO2(), tupleSTO3(), tupleSTO4(), tupleSTO5(), tupleSTO6(), tupleSTO7() ]}.
Running and compilation
zen:EUnitBasic4 zenon$ erlc -o ebin/ src/*.erl test/*.erl zen:EUnitBasic4 zenon$ erl -pa ebin/ Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.5 (abort with ^G) 1> eunit:test(mylist,[verbose]). ======================== EUnit ======================== module 'mylist' module 'mylist_tests' mylist_tests: nullarySTO1...ok mylist_tests: nullarySTO2...ok mylist_tests: nullarySTO1...ok mylist_tests: nullarySTO1...ok mylist_tests: nullarySTO2...ok mylist_tests: nullarySTO3...ok mylist_tests:13: tupleSTO4...ok mylist_tests:14: tupleSTO5...ok mylist_tests:15: tupleSTO6...ok mylist_tests:16: nullarySTO1...ok [done in 0.015 s] [done in 0.015 s] ======================================================= All 10 tests passed. ok 2>
Alas, the runtime is slightly higher, but all went well.
Removing the last question marks
If you where wondering whether all these control tuples play together, then good for you, since here is the answer: YES.
And to prove it, comes the monster module below.
-module(mylist_tests). -include_lib("eunit/include/eunit.hrl"). -compile(export_all). % Basics nullarySTO1() -> fun() -> ?assertEqual(6,mylist:sum([1,2,3])) end. nullarySTO2() -> fun nullarySTO1/0. nullarySTO3() -> fun mylist_tests:nullarySTO1/0. tupleSTO1() -> { mylist_tests, nullarySTO1 }. tupleSTO2() -> { mylist_tests, nullarySTO2 }. tupleSTO3() -> { mylist_tests, nullarySTO3 }. tupleSTO4() -> ?_test( fun() -> ?assertEqual(6,mylist:sum([1,2,3])) end ). tupleSTO5() -> ?_test( begin ?assertEqual(6,mylist:sum([1,2,3])) end ). tupleSTO6() -> ?_test( nullarySTO2() ). tupleSTO7() -> { 16, nullarySTO3() }. % % Proof of concept mixedA_test_() -> {spawn, nullarySTO1() }. mixedB_test_() -> {spawn, [ {timeout, 1, nullarySTO1()} , {inparallel, [tupleSTO1(), tupleSTO4()]}]}. mixedC_test_() -> {spawn, 'eunit@zen', nullarySTO1() }. mixedD_test_() -> {spawn, 'eunit@zen', {inorder, [ nullarySTO1() , tupleSTO1(), tupleSTO4()]}}. mixedE_test_() -> [ nullarySTO1(), nullarySTO2(), {inparallel, 2, [ {spawn, 'eunit2@zen', tupleSTO1()}, {spawn, 'eunit2@zen', tupleSTO2()}, tupleSTO3() ] }, {timeout, 1, tupleSTO4()} ].
As can be seen, the CT can be mixed wildly and are allowed wherever a TestSet is allowed, compiling and running in three shells.
zen:EUnitBasic4 zenon$ erl -pa ebin/ -sname eunit2@zen -setcookie eunit Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.5 (abort with ^G) (eunit2@zen)1>
And the other one
zen:EUnitBasic4 zenon$ erl -pa ebin/ -sname eunit@zen -setcookie eunit Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.5 (abort with ^G) (eunit@zen)1>
And the main shell doing all the fun
zen:EUnitBasic4 zenon$ erlc -o ebin/ src/*.erl test/*.erl zen:EUnitBasic4 zenon$ erl -pa ebin/ -sname base@zen -setcookie eunit Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.5 (abort with ^G) (base@zen)1> net_kernel:connect_node(eunit2@zen). true (base@zen)2> net_kernel:connect_node(eunit@zen). true (base@zen)3> nl(mylist_tests). abcast (base@zen)4> eunit:test(mylist,[verbose]). ======================== EUnit ======================== module 'mylist' module 'mylist_tests' mylist_tests: nullarySTO1...ok mylist_tests: nullarySTO1...ok mylist_tests: nullarySTO1...ok mylist_tests:13: tupleSTO4...ok mylist_tests: nullarySTO1...[0.003 s] ok mylist_tests: nullarySTO1...ok mylist_tests: nullarySTO1...ok mylist_tests:13: tupleSTO4...ok mylist_tests: nullarySTO1...ok mylist_tests: nullarySTO2...ok mylist_tests: nullarySTO1...ok mylist_tests: nullarySTO2...ok mylist_tests: nullarySTO3...ok mylist_tests:13: tupleSTO4...ok [done in 0.063 s] [done in 0.063 s] ======================================================= All 14 tests passed. ok (base@zen)5>
This concludes the Control of EUnit tests, next post will be about testing gen_fsms with EUnit.
Erlang, Testing and TDD 