EUnit Testing gen_fsm – Part 4 Automated Test Machinery


Using the idea from the last post, we have a machinery which takes a set of Rules as input, a gen_fsm, setup and cleanup. To develop this, there is once again a big question, will this be done with parse transforms or using macros and functions? To try and answer this question along with how it should work, let’s write some mock-code. This will be the first iteration of the ATG (Automated Test Generator).

First Iteration – Brainstorm

For this first iteration, it gets easier if we break it down into some use-cases, first off, the battery of Rules,

% ------------------------------------------------------------------------------
% List of all rules - mock code!
% ------------------------------------------------------------------------------
rules() ->
    [
     % Seller and buyer registration,
     % The postcondition needs to access the password which was generated in
     % the program so that we can verify that the tradepost holds the correct
     % registration.
     { not reg(seller), do_reg(seller), reg(seller,generated_pwd)},
     { not reg(buyer),  do_reg(buyer),  reg(buyer,generated_pwd)},

     % Insert item / cash, should insert random amount of cash and random
     % items. The PostCond must be able to test the existence of that item / cash
     % and thus must have access to the generated item / cash.
     { reg(seller), insert(item), item_is(inserted_item)},
     { reg(buyer),  insert(cash),   cash_is(inserted_amount)},

     % Removal of cash or item, remove needs to use the password which
     % the tradepost currently has.
     { tradepost_has(cash), remove(cash), not tradepost_has(cash) },
     { tradepost_has(item), remove(item), not tradepost_has(item) },

     % Close a deal, we must check that the _returned value_ is the correct one.
     % We must pass on the returned value to the postcond. The Postcond must
     % then also have access to the original item and cash.
     { tradepost_has(cash)
       andalso tradepost_has(item), closedeal(), return_value_is(??)}
    ].

As can be seen, I added some comments on thoughts that deserve discussion. It will often be the case that the postcondition needs to refer to a value which is the result of the Program. Such an example can be the test for correct registration. For that case, we do not only need access to the state, but also the generated password which was used. How to fix it? Bind the Program result to a special atom ‘$PROGRAM_RESULT’, also, bind the state to a special atom ‘$STATE’. Problem solved. Please note that such a solution , is leaning towards parse-transforms.

% The State is accessed through the special  atom
% '$STATE', and the module must include the state record.
reg(seller) -> '$STATE'#state.seller =/= undefined;
reg(buyer) ->  '$STATE'#state.buyer =/= undefined.

% Passwd can be result from Program, thus the call in
% the Rule may then be { ..... , reg(seller,'$PROGRAM_RESULT')}
reg(seller,Passwd) -> '$STATE'#state.seller == Passwd;
reg(buyer,Passwd) -> '$STATE'#state.buyer == Passwd.

Another special binding added to this machinery, is the identifier for the gen_fsm, this could be a registered name or a Pid. The usage of this is shown below, where a registration is performed, and the password is returned for the Postcondition.

% The identifier of the gen_fsm is accessed through '$ID'
do_reg(seller) ->
    Password = generate_passwd(),
    tradepost:seller_identify('$ID',Password),
    Password. % This line ensures that '$PROGRAM_RESULT' is bound to
             % the generated password, for the Postcondition to access.

How the ‘$ID’ is bound, is unclear for now, but that can be taken care of later, remember that we are just writing mock code to get an idea of what we possibly want. Going from the top to bottom, how would we love to see the insertion?

% This of course also uses the $ID, but also the state to access the password.
% seems only fair to return the inserted item for this function as that
% probably is what we wish, ... always.
insert(item) ->
    Password = '$STATE'#state.seller,
    Item = generate_item(),
    tradepost:seller_insert('$ID',Item,Password),
    Item; % Ensure '$PROGRAM_RESULT is Item
insert(cash) ->
    Password = '$STATE'#state.buyer,
    Cash = generate_cash(),
    tradepost:buyer_insert('$ID',Cash,Password),
    Cash; % Ensure '$PROGRAM_RESULT is Cash

Coupled to this, Now, how to test that the cash is the amount we wish, and that the item is the sought one?

% Check that item is the specified item (argument).
item_is(GivenItem) -> '$STATE'#state.item == GivenItem.

% Check that cash is specified amount (argument)
cash_is(GivenAmount) -> '$STATE'#state.cash == GivenAmount.

Or simpler, just test existence?

% Testing state to contain cash xor item
tradepost_has(cash) -> '$STATE'#state.cash =/= undefined;
tradepost_has(item) -> '$STATE'#state.item =/= undefined.

Likewise, removal should be easy with the help of ‘$ID’ and ‘$STATE’

% Utilizing both state and id, but nice thing is that result
% of withdraw function will be bound to $PROGRAM_RESULT in this mockup.
remove(cash) ->
    Passwd = '$STATE'#state.buyer,
    tradepost:buyer_withdraw('$ID',Passwd);
remove(item) ->
    Passwd = '$STATE'#state.seller,
    tradepost:seller_withdraw('$ID',Passwd);

Nearing the end of the first iteration brainstorming, what remains is a way to program the deal closing and a way to test the return value of this (a way to test the return value of the deal closing), ‘$PROGRAM_RESULT’ is still our friend, but it would be good to have some kind of parallelism. Once again, how would we like to see this written?

% Deal closing, is a parallelized action.
% Return value should still be bound to '$PROGRAM_RESULT'
% Could be bound in the form [ A , B ] for this example.
% Inspiration from the parallel keyword in EUnit.
closedeal() ->
    {'$PARALLEL',
     [ begin
           Pass = '$STATE'#state.buyer,
           ItemAndCash = tradepost:get_contents('$ID',Pass),
           tradepost:buyer_deal('$ID',Pass,ItemAndCash)
       end,
       begin
           Pass = '$STATE'#state.seller,
           ItemAndCash = tradepost:get_contents('$ID',Pass),
           tradepost:seller_deal('$ID',Pass,ItemAndCash)
       end]}.

This seems to do the trick,  the idea is to pass a list of  functions which are executed in parallel, the result of each one is bound to an element in the generated list which is bound to ‘$PROGRAM_RESULT’. As this seems that this covers the first iteration of brainstorming, it could be nice to sum it up.

Summary of First Brainstorming session

The Automated Test Generator machinery has a set of Rules, each Rule is modeled as a 3-tuple, The first element in the 3-tuple is the Precondition that has to be met for the Rule to possibly take action. The second element is a Program which is executed, iff the Precondition holds and the Rule is chosen. The third element is a Postcondition that must hold once the Program of the Rule is executed.

The different special syntax elements which have been identified now are

  • ‘$STATE’,  the internal state of the gen_fsm being tested.
  • ‘$ID’,  identifier of the gen_fsm, can be a registered name, a registered name on a node or a Pid.
  • ‘$PROGRAM_RESULT’, the result of the Program (middle element in the Rule)
  • ‘$PARALLEL’, keyword for marking that functions are to be executed in parallel, and the joint result is bound to ‘$PROGRAM_RESULT’

Besides these special syntactical elements, we should try structuring the thoughts a bit more.

rule() = { precondition(), program(), postcondition() }

precondition() = expr() -> true | false.
program()    = expr() -> any().
postcondition()= expr() -> true | false.

Maybe this definition feels a bit weak and lame, but as for now, it captures what we wish, next brainstorming session should focus on the machinery of our automated test generator.

Cheers

EUnit Testing gen_fsm – Part 3


This third part will add the deal closure API as well as refactoring the code, we will also see the inception of the Automated Test Generating machinery.

Sixth Iteration – Deal closing API and the beginning of the ATG machinery

Buyer and Seller can now insert their items, and wish to close the deal, for this we need some kind of “security”. To implement this, the buyer and seller must both agree on the item and the cash. They both query the tradepost for it’s contents, get the item name and the cash amount, then if they agree, they can send an okay to the tradepost with the same item name and the same cash and their Pid. If the tradepost contains an item with that name, and that cash, the tradepost sends the item to the buyer and the cash to the seller. The tradepost then terminates.

For this parallelised receive, the syntax of the symbolic language is extended with

{parallel,call,MODULE,FUNCTION,ARGLIST,REPLYMATCH}

Which gives the new code in eunit_fsm.erl as

%%% @author Gianfranco <zenon@zen.home>
%%% @copyright (C) 2010, Gianfranco
%%% Created : 19 Sep 2010 by Gianfranco <zenon@zen.home>
-module(eunit_fsm).
-export([translateCmd/2,get/2]).
-define(Expr(X),??X).
translateCmd(Id,{state,is,X}) ->
    case get(Id,"StateName") of
        X -> true;
        _V ->  .erlang:error({statename_match_failed,
                              [{module, ?MODULE},
                               {line, ?LINE},
                               {expected, X},
                               {value, _V}]})
    end;
translateCmd(_Id,{call,M,F,A,X}) ->
    case apply(M,F,A) of
        X -> ok;
        _V ->  .erlang:error({function_call_match_failed,
                              [{module, ?MODULE},
                               {line, ?LINE},
                               {expression, "apply("++atom_to_list(M)++","++atom_to_list(F)++","++
                                ?Expr(A)},
                               {expected, X},
                               {value, _V}]})
    end;
translateCmd(Id,{loopdata,is,X}) ->
    case tl(tuple_to_list(get(Id,"StateData"))) of
        X    -> true;
        _V ->    .erlang:error({loopdata_match_failed,
                                [{module, ?MODULE},
                                 {line, ?LINE},
                                 {expected, X},
                                 {value, _V}]})
    end;
translateCmd(Id,{parallel,call,M,F,A,X}) ->
    spawn_link(fun() -> translateCmd(Id,{call,M,F,A,X}) end).             

% StateName or StateData
get(Id,Which) ->
    {status,_Pid,_ModTpl, List} = sys:get_status(Id),
    AllData = lists:flatten([ X || {data,X} <- lists:last(List) ]),
    proplists:get_value(Which,AllData).

To pull this off, we need to extend the state with a holder for the agreed parts Pids, this will reveal a flaw with the current syntax that has not been so evident until now (bu surely nagging) that will need to be countered.

closedeal(Pid) ->
    ?fsm_test(Pid,
              "Seller Identifies, Buyer Identifies, Seller Inserts Item "
              ", Buyer Inserts Cash, Seller Gets Contents, Buyer Gets Contents "
              ", Buyer Agrees, Seller Agrees ",
              [{state,is,pending},
               {call,tradepost,seller_identify,[Pid,s],ok},
               {call,tradepost,buyer_identify,[Pid,b],ok},
               {call,tradepost,seller_insertitem,[Pid,ring,s],ok},
               {call,tradepost,buyer_insertcash,[Pid,100,b],ok},
               {call,tradepost,get_contents,[Pid,s],{ring,100}},
               {call,tradepost,get_contents,[Pid,b],{ring,100}},
               {parallel,call,tradepost,seller_deal,[Pid,s,{ring,100}],100},
               {parallel,call,tradepost,buyer_deal,[Pid,b,{ring,100}],ring}
              ]).

Writing the code and also modifying the state record, the result is

1> ======================== EUnit ========================
module 'tradepost'
  module 'tradepost_tests'
    tradepost_tests: started_properly (Started Properly Test)...[0.001 s] ok
    tradepost_tests: identify_seller (Identify Seller Test)...ok
    tradepost_tests: insert_item (Insert Item Test)...ok
    tradepost_tests: withdraw_item (Withdraw Item Test)...ok
    tradepost_tests: identify_buyer (Identify Buyer Test)...ok
    tradepost_tests: insert_cash (Insert Cash Test)...ok
    tradepost_tests: withdraw_cash (Withdraw Cash Test)...ok
    tradepost_tests: interleaving1 (Seller Identifies, Inserts Item, then Buyer Identiefies)...ok
    tradepost_tests: interleaving2 (Buyer Identifies, Inserts Item, then Seller Identiefies)...ok
    tradepost_tests: interleaving3 (Buyer Identifies, Seller Identifies, buyer inserts cash)...ok
    tradepost_tests: interleaving4 (Seller Identifies, Buyer Identifies, Seller inserts item)...ok
    tradepost_tests: interleaving5 (Seller Identifies, Seller inserts item, Buyer Identifies,Buyer Inserts Cash)...ok
    tradepost_tests: interleaving6 (Seller Identifies, Seller inserts item, Buyer Identifies,Seller Withdraws Item, Buyer Inserts Cash, Seller Inserts ItemBuyer Withdraws Cash, Seller Withdraws Item)...ok
    tradepost_tests: closedeal (Seller Identifies, Buyer Identifies, Seller Inserts Item , Buyer Inserts Cash, Seller Gets Contents, Buyer Gets Contents , Buyer Agrees, Seller Agrees )...ok
    [done in 0.045 s]
  [done in 0.045 s]
=======================================================
  All 14 tests passed.

1>

Greatness, but, in order for this to work, some modification had to be done to all previous tests! We had to extend the state-record with new fields, and the result is that the loopdata assertion had to be changed in each test! What if we had 1000 previous tests? *pulls hair in desperation*. Also, after this success, I made some refactoring, something that _should_ be done after each tests + code + success. This is also a great thing, once you have a nice big set of tests, refactoring can always be tested to preserve the logic by running your tests *thumbs up*.

Anyway, here is the cut and paste version of  tradepost.erl

%%%-------------------------------------------------------------------
%%% @author Gianfranco <zenon@zen.home>
%%% @copyright (C) 2010, Gianfranco
%%% Created :  2 Sep 2010 by Gianfranco <zenon@zen.home>
%%%-------------------------------------------------------------------
-module(tradepost).
-behaviour(gen_fsm).
% ------------------------------------------------------------------------------
% API
-export([start_link/0,stop/1,seller_identify/2,seller_insert/3,seller_withdraw/2,
         buyer_identify/2,buyer_insert/3,buyer_withdraw/2,get_contents/2,
         seller_deal/3,buyer_deal/3]).
% ------------------------------------------------------------------------------
% States
-export([pending/3,item_received/3,cash_received/3]).
% ------------------------------------------------------------------------------
% gen_fsm callbacks
-export([init/1, handle_event/3, handle_sync_event/4, handle_info/3,terminate/3,
         code_change/4]).
% ------------------------------------------------------------------------------
% State representation
-record(state, {item,cash,seller,buyer,seller_accept,buyer_accept}).

% ------------------------------------------------------------------------------
% API
start_link() -> gen_fsm:start_link(?MODULE, [], []).
stop(Pid) -> gen_fsm:sync_send_all_state_event(Pid,stop).
seller_identify(TP,Pwd) ->
    gen_fsm:sync_send_all_state_event(TP,{identify,seller,Pwd}).
seller_insert(TP,Item,Pwd) ->
    gen_fsm:sync_send_event(TP,{insert,item,Item,Pwd}).
seller_withdraw(TP,Pwd) ->
    gen_fsm:sync_send_event(TP,{withdraw,item,Pwd}).
seller_deal(TP,Pwd,{Item,Cash}) ->
    gen_fsm:sync_send_all_state_event(TP,{accept,seller,Pwd,{Item,Cash}}).
buyer_deal(TP,Pwd,{Item,Cash}) ->
    gen_fsm:sync_send_all_state_event(TP,{accept,buyer,Pwd,{Item,Cash}}).
buyer_identify(TP,Pwd) ->
    gen_fsm:sync_send_all_state_event(TP,{identify,buyer,Pwd}).
buyer_insert(TP,Amount,Pwd) ->
    gen_fsm:sync_send_event(TP,{insert,cash,Amount,Pwd}).
buyer_withdraw(TP,Pwd) ->
    gen_fsm:sync_send_event(TP,{withdraw,cash,Pwd}).
get_contents(TP,Pwd) ->
    gen_fsm:sync_send_all_state_event(TP,{get_contents,Pwd}).
% ------------------------------------------------------------------------------
% Internal States
pending({insert,item,Item,Pwd},_,LoopD) when Pwd == LoopD#state.seller,
                                             LoopD#state.seller =/= undefined ->
    {reply,ok,item_received,LoopD#state{item=Item}};
pending({insert,cash,Amount,Pwd},_,LoopD) when Pwd == LoopD#state.buyer;
                                             LoopD#state.buyer =/= undefined ->
    {reply,ok,cash_received,LoopD#state{cash=Amount}};
pending(_E,_From,LoopD) -> {reply,error,pending,LoopD}.

item_received({withdraw,What,Pwd},_,LoopD) ->
    withdraw(What,Pwd,LoopD,item_received);
item_received({insert,cash,Amount,Pwd},_,LoopD = #state{buyer=Pwd})->
    {reply,ok,cash_received,LoopD#state{cash=Amount}};
item_received({insert,item,Item,Pwd},_,LoopD = #state{seller=Pwd})->
    {reply,ok,item_received,LoopD#state{item=Item}};
item_received(_E,_From,LoopD) -> {reply,error,item_received,LoopD}.

cash_received({withdraw,What,Pwd},_,LoopD) ->
    withdraw(What,Pwd,LoopD,cash_received);
cash_received({insert,cash,Amount,Pwd},_,LoopD = #state{buyer=Pwd})->
    {reply,ok,cash_received,LoopD#state{cash=Amount}};
cash_received({insert,item,Item,Pwd},_Frm, LoopD = #state{seller=Pwd})->
    {reply,ok,item_received,LoopD#state{item=Item}};
cash_received(_E,_From,LoopD) -> {reply,error,cash_received,LoopD}.
% ------------------------------------------------------------------------------
% Callbacks with Logic
handle_sync_event({identify,Who,Pwd},_From,StateName,LoopD) ->
    {Reply,NextState} =
        case {Who , LoopD#state.seller, LoopD#state.buyer } of
            {seller, undefined, _ } -> {ok,LoopD#state{seller=Pwd}};
            {buyer, _ , undefined } -> {ok,LoopD#state{buyer=Pwd}};
            _ -> {error,LoopD}
        end,
    {reply,Reply,StateName,NextState};
handle_sync_event(stop,_From,_,LoopData) -> {stop,normal,ok,LoopData};
handle_sync_event({get_contents,Pwd},_Frm,StateName,LoopData) ->
    Reply =
        case {LoopData#state.seller,LoopData#state.buyer} of
            {Pwd,_} -> {LoopData#state.item,LoopData#state.cash};
            {_,Pwd} -> {LoopData#state.item,LoopData#state.cash}
        end,
    {reply,Reply,StateName,LoopData};
handle_sync_event({accept,seller,Pwd,{Item,Cash}},From,StateName,LoopD)
  when Pwd == LoopD#state.seller; LoopD#state.seller =/= undefined ->
    accept(LoopD#state.buyer_accept, {Item,Cash,From}, LoopD#state.item,
           LoopD#state.cash, seller, StateName, LoopD);
handle_sync_event({accept,buyer,Pwd,{Item,Cash}},From,StateName,LoopD)
  when Pwd == LoopD#state.buyer; LoopD#state.buyer =/= undefined ->
    accept(LoopD#state.seller_accept, {Item,Cash,From}, LoopD#state.item,
           LoopD#state.cash, buyer, StateName, LoopD);
handle_sync_event(_E,_From,StateName,LoopData) ->
    {reply,error,StateName,LoopData}.
% ------------------------------------------------------------------------------
% Obligatory others
init([]) -> {ok, pending, #state{}}.
handle_event(_Event, StateName, State) ->{next_state, StateName, State}.
handle_info(_Info, StateName, State) -> {next_state, StateName, State}.
terminate(_Reason, _StateName, _State) -> ok.
code_change(_OldVsn, StateName, State, _Extra) -> {ok, StateName, State}.

% ------------------------------------------------------------------------------
% Withdraw cash xor item - grossly compressed  :(
-spec withdraw(item | cash, any(), #state{}, atom()) ->
    {reply,ok|error,atom(),#state{}}.
withdraw(What,Pwd,LoopD=#state{seller=S,buyer=B,item = I,cash=C},State) ->
  {Reply,NextState,NewLoopData} =
   case { State, What, S, B, I, C} of
    {item_received, item, Pwd, _ , _, undefined } -> {ok, pending, undef(item,LoopD)};
    {item_received, item, Pwd, _ , _, _V } -> {ok, cash_received, undef(item,LoopD)};
    {item_received, cash, _ , Pwd, _, _} -> {ok, item_received, undef(cash,LoopD)};
    {cash_received, item, Pwd, _, _, undefined} -> {ok, cash_received, undef(item,LoopD)};
    {cash_received, cash, _, Pwd, undefined, _ } -> {ok, pending, undef(cash,LoopD)};
    {cash_received, cash, _ , Pwd, _V, _} -> {ok, item_received, undef(cash,LoopD)};
     _ -> {error,State,LoopD}
  end,
    {reply,Reply,NextState,NextLoopData}.
% ------------------------------------------------------------------------------
% Accept a deal
-spec accept({atom(), integer(), {reference(),pid()}},
             {atom(), integer(), {reference(),pid()}},
             atom(), integer(), seller | buyer, atom(),#state{}) ->
    {reply, atom() | integer | {error,not_same},  atom(), #state{}} |
    {next_state, atom(), #state{}}.
accept(AgreeOther, {ItemSelf,CashSelf,Self}, TradePostItem, TradePostCash,
       Who,StateName,LoopD) ->
    % Check if Other and Self have now agreed on the same thing
    AgreeThis  = {ItemSelf, CashSelf, Self},
    Contains   = {TradePostItem, TradePostCash},
    case { AgreeOther , AgreeThis,  Contains} of
        % Same Object And Cash, send cash to buyer And item to seller
        { {Item, Cash, Other}, {Item, Cash, Self}, {Item, Cash} } ->
            {SelfEnd,OtherEnd} =
                case Who of
                    seller -> {Cash,Item};
                    buyer -> {Item,Cash}
                end,
            gen_fsm:reply(Other,OtherEnd),
            {reply,SelfEnd,StateName,LoopD};
        % Not agreed on same
        { {_A,_B,Other}, {_C,_D,Self}, _ } ->
            gen_fsm:reply(Other,{error,not_same}),
            {reply,{error,not_same},StateName,LoopD};
        % Only self has agreed so far
        { undefined, _ , _ } ->
            NewLoopD =
            case Who of
                seller -> LoopD#state{seller_accept={ItemSelf,CashSelf,Self}};
                buyer -> LoopD#state{buyer_accept={ItemSelf,CashSelf,Self}}
            end,
            {next_state,StateName,NewLoopD}
    end.

-spec undef(item | cash, #state{}) -> #state{}.
undef(item,LoopData) -> LoopData#state{item=undefined};
undef(cash,LoopData) -> LoopData#state{cash=undefined}.

And the modified test module tradepost_tests.erl

% @author Gianfranco <zenon@zen.home>
% @copyright (C) 2010, Gianfranco
% Created :  6 Sep 2010 by Gianfranco <zenon@zen.home>
-module(tradepost_tests).
-include_lib("eunit/include/eunit.hrl").
-include("include/eunit_fsm.hrl").

% This is the main point of "entry" for my EUnit testing.
% A generator which forces setup and cleanup for each test in the testset
main_test_() ->
    {foreach,
     fun setup/0,
     fun cleanup/1,
     % Note that this must be a List of TestSet or Instantiator
     [
      % First Iteration
      fun started_properly/1,
      % Second Iteration
      fun identify_seller/1,
      fun insert_item/1,
      fun withdraw_item/1,
      % Fourth iteration
      fun identify_buyer/1,
      fun insert_cash/1,
      fun withdraw_cash/1,
      % Fifth iteration
      fun interleaving1/1,
      fun interleaving2/1,
      fun interleaving3/1,
      fun interleaving4/1,
      fun interleaving5/1,
      fun interleaving6/1,
      % Sixth Iteration
      fun closedeal/1
     ]}.

% Setup and Cleanup
setup()      -> {ok,Pid} = tradepost:start_link(), Pid.
cleanup(Pid) -> tradepost:stop(Pid).

% Pure tests below
% ------------------------------------------------------------------------------
% Let's start simple, I want it to start and check that it is okay.
% I will use the introspective function for this
started_properly(Pid) ->
    ?fsm_test(Pid,"Started Properly Test",
              [{state,is,pending},
               {loopdata,is,[undefined,undefined,undefined,undefined,
                            undefined,undefined]}
              ]).

% Now, we are adding the Seller API tests
identify_seller(Pid) ->
    ?fsm_test(Pid,"Identify Seller Test",
              [{state,is,pending},
               {call,tradepost,seller_identify,[Pid,seller_password],ok},
               {state,is,pending},
               {loopdata,is,[undefined,undefined,seller_password,undefined,
                             undefined,undefined]}
              ]).

insert_item(Pid) ->
    ?fsm_test(Pid,"Insert Item Test",
              [{state,is,pending},
               {call,tradepost,seller_identify,[Pid,seller_password],ok},
               {call,tradepost,seller_insert,[Pid,playstation,seller_password],ok},
               {state,is,item_received},
               {loopdata,is,[playstation,undefined,seller_password,undefined,
                             undefined,undefined]}
              ]).

withdraw_item(Pid) ->
    ?fsm_test(Pid,"Withdraw Item Test",
              [{state,is,pending},
               {call,tradepost,seller_identify,[Pid,seller_password],ok},
               {call,tradepost,seller_insert,[Pid,button,seller_password],ok},
               {state,is,item_received},
               {call,tradepost,seller_withdraw,[Pid,seller_password],ok},
               {state,is,pending},
               {loopdata,is,[undefined,undefined,seller_password,undefined,
                             undefined,undefined]}
              ]).

identify_buyer(Pid) ->
    ?fsm_test(Pid,"Identify Buyer Test",
              [{state,is,pending},
               {call,tradepost,buyer_identify,[Pid,buyer_password],ok},
               {state,is,pending},
               {loopdata,is,[undefined,undefined,undefined,buyer_password,
                            undefined,undefined]}
              ]).

insert_cash(Pid) ->
    ?fsm_test(Pid,"Insert Cash Test",
              [{state,is,pending},
               {call,tradepost,buyer_identify,[Pid,buyer_password],ok},
               {call,tradepost,buyer_insert,[Pid,100,buyer_password],ok},
               {state,is,cash_received},
               {loopdata,is,[undefined,100,undefined,buyer_password,
                            undefined,undefined]}
              ]).

withdraw_cash(Pid) ->
    ?fsm_test(Pid,"Withdraw Cash Test",
              [{state,is,pending},
               {call,tradepost,buyer_identify,[Pid,buyer_password],ok},
               {call,tradepost,buyer_insert,[Pid,100,buyer_password],ok},
               {call,tradepost,buyer_withdraw,[Pid,buyer_password],ok},
               {loopdata,is,[undefined,undefined,undefined,buyer_password,
                            undefined,undefined]}
              ]).

interleaving1(Pid) ->
    ?fsm_test(Pid,"Seller Identifies, Inserts Item, then Buyer Identiefies",
              [{state,is,pending},
               {call,tradepost,seller_identify,[Pid,s],ok},
               {call,tradepost,seller_insert,[Pid,ring,s],ok},
               {call,tradepost,buyer_identify,[Pid,b],ok},
               {loopdata,is,[ring,undefined,s,b,undefined,undefined]}
              ]).

interleaving2(Pid) ->
    ?fsm_test(Pid,"Buyer Identifies, Inserts Item, then Seller Identiefies",
              [{state,is,pending},
               {call,tradepost,buyer_identify,[Pid,b],ok},
               {call,tradepost,buyer_insert,[Pid,100,b],ok},
               {call,tradepost,seller_identify,[Pid,s],ok},
               {loopdata,is,[undefined,100,s,b,undefined,undefined]}
              ]).

interleaving3(Pid) ->
    ?fsm_test(Pid,"Buyer Identifies, Seller Identifies, buyer inserts cash",
              [{state,is,pending},
               {call,tradepost,buyer_identify,[Pid,b],ok},
               {call,tradepost,seller_identify,[Pid,s],ok},
               {call,tradepost,buyer_insert,[Pid,100,b],ok},
               {loopdata,is,[undefined,100,s,b,undefined,undefined]}
              ]).

interleaving4(Pid) ->
    ?fsm_test(Pid,"Seller Identifies, Buyer Identifies, Seller inserts item",
              [{state,is,pending},
               {call,tradepost,seller_identify,[Pid,s],ok},
               {call,tradepost,buyer_identify,[Pid,b],ok},
               {call,tradepost,seller_insert,[Pid,ring,s],ok},
               {loopdata,is,[ring,undefined,s,b,undefined,undefined]}
              ]).

interleaving5(Pid) ->
    ?fsm_test(Pid,"Seller Identifies, Seller inserts item, Buyer Identifies,"
              "Buyer Inserts Cash",
              [{state,is,pending},
               {call,tradepost,seller_identify,[Pid,s],ok},
               {call,tradepost,seller_insert,[Pid,ring,s],ok},
               {call,tradepost,buyer_identify,[Pid,b],ok},
               {call,tradepost,buyer_insert,[Pid,100,b],ok},
               {loopdata,is,[ring,100,s,b,undefined,undefined]}
              ]).

interleaving6(Pid) ->
    ?fsm_test(Pid,"Seller Identifies, Seller inserts item, Buyer Identifies,"
              "Seller Withdraws Item, Buyer Inserts Cash, Seller Inserts Item"
              "Buyer Withdraws Cash, Seller Withdraws Item",
              [{state,is,pending},
               {call,tradepost,seller_identify,[Pid,s],ok},
               {call,tradepost,seller_insert,[Pid,ring,s],ok},
               {call,tradepost,buyer_identify,[Pid,b],ok},
               {call,tradepost,seller_withdraw,[Pid,s],ok},
               {call,tradepost,buyer_insert,[Pid,100,b],ok},
               {call,tradepost,seller_insert,[Pid,ring,s],ok},
               {call,tradepost,buyer_withdraw,[Pid,b],ok},
               {call,tradepost,seller_withdraw,[Pid,s],ok},
               {loopdata,is,[undefined,undefined,s,b,undefined,undefined]}
              ]).

closedeal(Pid) ->
    ?fsm_test(Pid,
              "Seller Identifies, Buyer Identifies, Seller Inserts Item "
              ", Buyer Inserts Cash, Seller Gets Contents, Buyer Gets Contents "
              ", Buyer Agrees, Seller Agrees ",
              [{state,is,pending},
               {call,tradepost,seller_identify,[Pid,s],ok},
               {call,tradepost,buyer_identify,[Pid,b],ok},
               {call,tradepost,seller_insert,[Pid,ring,s],ok},
               {call,tradepost,buyer_insert,[Pid,100,b],ok},
               {call,tradepost,get_contents,[Pid,s],{ring,100}},
               {call,tradepost,get_contents,[Pid,b],{ring,100}},
               {parallel,call,tradepost,seller_deal,[Pid,s,{ring,100}],100},
               {parallel,call,tradepost,buyer_deal,[Pid,b,{ring,100}],ring}
              ]).

With this success, it’s time to hunt the real cornercase-bugs using Automated Test Generation, a very strong piece of machinery, that requieres some setup. Oh, and just in case, if you forgot how it all fits together…

zen:EUnitFSM zenon$ tree .
.
├── ebin
│   ├── eunit_fsm.beam
│   ├── tradepost.beam
│   └── tradepost_tests.beam
├── include
│   └── eunit_fsm.hrl
├── src
│   └── tradepost.erl
└── test
    ├── eunit_fsm.erl
    └── tradepost_tests.erl

4 directories, 7 files
zen:EUnitFSM zenon$ 

Automated Test Generation – inception

As we wish to generate complex (almost the same as long) test sequences, and do not wish to enter them by hand, we want to have a piece of machinery which makes this for us. An immediate question that comes to mind is then, how does the machinery know which sequences are actually valid? Well, that is for us to know and express. Here we can draw inspiration from from Hoare Logic and use that notion for our automated test generator. A good candidate for this would be a syntax similar to something as

Rule = { PreCondPred, Program, PostCondPred }

PreCondPred = fun FSM-State x SetupResult -> Boolean

Program = [Action]

Action = {call,M,F,A,ExpectedResult} |
        {parallel, call, M, F, A, ResultExpectedResult}

PostCondPred = fun FSM-State x SetupResult -> Boolean

We would thus want the machinery to behave as follows. Given a current gen_fsm state, and the list of all rules, return a list of all rules for which the PreCondPred holds. From this list of rules, choose one randomly, and apply execute Program. After execution, check if PostCondPred holds true. If not, record failure. On Success, repeat process with random selection of valid rule and application, etc.

For this to work, we need to write our own generator, for this, we can peek at the EUnit User’s Guide section on Lazy Generators. However, this should be discussed in the next part of this series. For this purpose, the iterations have ended.

Ps: I apologize for the long delay before this post came, but I had a lot to do at the sidelines.

EUnit Testing gen_fsm – Part 2


Last post we saw the symbolic (somewhat DSL) for the gen_fsm testing, using that as help we shall continue with the Buyer API. Technically, we are now doing the fourth iteration.

Fourth Iteration – Buyer API

The buyer wishes to be identified like the seller, to deposit cash and to withdraw cash. Thus his/her usage is similar. By good TDD, we will write the tests first, adding them to tradepost_tests.erl

The new added instantiators

% This is the main point of "entry" for my EUnit testing.
% A generator which forces setup and cleanup for each test in the testset
main_test_() ->
    {foreach,
     fun setup/0,
     fun cleanup/1,
     % Note that this must be a List of TestSet or Instantiator
     [
      % First Iteration
      fun started_properly/1,
      % Second Iteration
      fun identify_seller/1,
      fun insert_item/1,
      fun withdraw_item/1,
      % Fourth iteration
      fun identify_buyer/1,
      fun insert_cash/1,
      fun withdraw_cash/1
     ]}.

And their implementations

identify_buyer(Pid) ->
    ?fsm_test(Pid,"Identify Buyer Test",
       [{state,is,pending},
        {call,tradepost,buyer_identify,[Pid,buyer_password],ok},
        {state,is,pending},
        {loopdata,is,[undefined,undefined,undefined,buyer_password,undefined]}
      ]).

insert_cash(Pid) ->
    ?fsm_test(Pid,"Insert Cash Test",
       [{state,is,pending},
        {call,tradepost,buyer_identify,[Pid,buyer_password],ok},
        {call,tradepost,buyer_insertcash,[Pid,100,buyer_password],ok},
        {state,is,cash_received},
        {loopdata,is,[undefined,100,undefined,buyer_password,undefined]}
       ]).

withdraw_cash(Pid) ->
    ?fsm_test(Pid,"Withdraw Cash Test",
       [{state,is,pending},
        {call,tradepost,buyer_identify,[Pid,buyer_password],ok},
        {call,tradepost,buyer_insertcash,[Pid,100,buyer_password],ok},
        {call,tradepost,buyer_withdrawcash,[Pid,buyer_password],ok},
        {loopdata,is,[undefined,undefined,undefined,buyer_password,undefined]}
       ]).

The changes in tradepost.erl

%% API
-export([start_link/0,stop/1,seller_identify/2,seller_insertitem/3,
         seller_withdraw_item/2,buyer_identify/2,buyer_insertcash/3,
         buyer_withdrawcash/2]).
%% States
-export([pending/2,pending/3,item_received/3,cash_received/3]).

buyer_identify(TradePost,Password) ->
    gen_fsm:sync_send_event(TradePost,{identify_buyer,Password}).
buyer_insertcash(TradePost,Amount,Password) ->
    gen_fsm:sync_send_event(TradePost,{insert_cash,Amount,Password}).
buyer_withdrawcash(TradePost,Password) ->
    gen_fsm:sync_send_event(TradePost,{withdraw_cash,Password}).    

pending({identify_buyer,Password},_Frm,LoopD = #state{buyer=Password}) ->
    {reply,ok,pending,LoopD};
pending({identify_buyer,Password},_Frm,LoopD = #state{buyer=undefined}) ->
    {reply,ok,pending,LoopD#state{buyer=Password}};
pending({identify_buyer,_},_,LoopD) ->
    {reply,error,pending,LoopD};

pending({insert_cash,Amount,Password},_Frm,LoopD = #state{buyer=Password}) ->
    {reply,ok,cash_received,LoopD#state{cash=Amount}};
pending({insert_cash,_,_},_Frm,LoopD) ->
    {reply,error,pending,LoopD}.

cash_received({withdraw_cash,Password},_From,LoopD = #state{buyer=Password}) ->
    {reply,ok,pending,LoopD#state{cash=undefined}};
cash_received({withdraw_cash,_},_From,LoopD) ->
    {reply,error,cas_received,LoopD}.

Compiling and running

zen:EUnitFSM zenon$ erlc -o ebin/ src/*.erl test/*.erl
zen:EUnitFSM zenon$ erl -pa ebin/ -eval 'eunit:test(tradepost,[verbose]).'
Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4]
[async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.5  (abort with ^G)
1> ======================== EUnit ========================
module 'tradepost'
  module 'tradepost_tests'
    tradepost_tests: started_properly (Started Properly Test)...ok
    tradepost_tests: identify_seller (Identify Seller Test)...ok
    tradepost_tests: insert_item (Insert Item Test)...[0.001 s] ok
    tradepost_tests: withdraw_item (Withdraw Item Test)...[0.001 s] ok
    tradepost_tests: identify_buyer (Identify Buyer Test)...ok
    tradepost_tests: insert_cash (Insert Cash Test)...[0.001 s] ok
    tradepost_tests: withdraw_cash (Withdraw Cash Test)...ok
    [done in 0.024 s]
  [done in 0.024 s]
=======================================================
  All 7 tests passed.

1>

Wow. Great. So the buyer and the seller can now deposit and retract their respective parts. Awesome. However, there are some intentionally left out parts (and yes, I assume more than one of you has been thinking and cringing about it) – the interleaving of the actions. That is left for the fifth iteration.

Fifth Iteration – Interleaving of Actions

Much straight forward  – what if the buyer identifies and inserts the item, and the buyer wishes to identify after this?

Why, let’s write a test scenario for it (and of course it will fail).

interleaving1(Pid) ->
    ?fsm_test(Pid,"Seller Identifies, Inserts Item, then Buyer Identiefies",
              [{state,is,pending},
               {call,tradepost,seller_identify,[Pid,a],ok},
               {call,tradepost,seller_insertitem,[Pid,ring,a],ok},
               {call,tradepost,buyer_identify,[Pid,b],ok}
              ]).

And just as we knew, it would blow

zen:EUnitFSM zenon$ erlc -o ebin/ src/*.erl test/*.erl
zen:EUnitFSM zenon$ erl -pa ebin/ -eval 'eunit:test(tradepost,[verbose]).'
Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4]
[async-threads:0] [hipe] [kernel-poll:false]Eshell V5.7.5  (abort with ^G)
1> ======================== EUnit ========================
module 'tradepost'
  module 'tradepost_tests'
    tradepost_tests: started_properly (Started Properly Test)...[0.001 s] ok
    tradepost_tests: identify_seller (Identify Seller Test)...ok
    tradepost_tests: insert_item (Insert Item Test)...ok
    tradepost_tests: withdraw_item (Withdraw Item Test)...ok
    tradepost_tests: identify_buyer (Identify Buyer Test)...ok
    tradepost_tests: insert_cash (Insert Cash Test)...ok
    tradepost_tests: withdraw_cash (Withdraw Cash Test)...ok
    tradepost_tests: interleaving1 (Seller Identifies, Inserts Item, then Buyer Identiefies)...*skipped*
undefined
*unexpected termination of test process*
::{function_clause,[{tradepost,item_received,
                               [{identify_buyer,b},
                                {<0.69.0>,#Ref<0.0.0.113>},
                                {state,ring,undefined,a,undefined,...}]},
                    {gen_fsm,handle_msg,7},
                    {proc_lib,init_p_do_apply,3}]}

The issue is of course that it is not possible to identify oneself in any other state than the pending one. Do we consider this a flaw or as part of the system design?  For this example, we shall regard it to be a flaw. And the true design should be that either buyer or seller must be able to identify themselves once before inserting their part and closing the deal.

Let’s add some more tests that we know should pass, like reversing the roles, and adding more interleaving of actions. As will be seen, this causes a lot of code, and we discover the need for some Automated Test Generation.

interleaving1(Pid) ->
    ?fsm_test(Pid,"Seller Identifies, Inserts Item, then Buyer Identiefies",
              [{state,is,pending},
               {call,tradepost,seller_identify,[Pid,s],ok},
               {call,tradepost,seller_insertitem,[Pid,ring,s],ok},
               {call,tradepost,buyer_identify,[Pid,b],ok},
               {loopdata,is,[ring,undefined,s,b]}
              ]).

interleaving2(Pid) ->
    ?fsm_test(Pid,"Buyer Identifies, Inserts Item, then Seller Identiefies",
              [{state,is,pending},
               {call,tradepost,buyer_identify,[Pid,b],ok},
               {call,tradepost,buyer_insertcash,[Pid,100,b],ok},
               {call,tradepost,seller_identify,[Pid,s],ok},
               {loopdata,is,[undefined,100,s,b]}
              ]).

interleaving3(Pid) ->
    ?fsm_test(Pid,"Buyer Identifies, Seller Identifies, buyer inserts cash",
              [{state,is,pending},
               {call,tradepost,buyer_identify,[Pid,b],ok},
               {call,tradepost,seller_identify,[Pid,s],ok},
               {call,tradepost,buyer_insertcash,[Pid,100,b],ok},
               {loopdata,is,[undefined,100,s,b]}
              ]).

interleaving4(Pid) ->
    ?fsm_test(Pid,"Seller Identifies, Buyer Identifies, Seller inserts item",
              [{state,is,pending},
               {call,tradepost,seller_identify,[Pid,s],ok},
               {call,tradepost,buyer_identify,[Pid,b],ok},
               {call,tradepost,seller_insertitem,[Pid,ring,s],ok},
               {loopdata,is,[ring,undefined,s,b]}
              ]).

interleaving5(Pid) ->
    ?fsm_test(Pid,"Seller Identifies, Seller inserts item, Buyer Identifies,"
              "Buyer Inserts Cash",
              [{state,is,pending},
               {call,tradepost,seller_identify,[Pid,s],ok},
               {call,tradepost,seller_insertitem,[Pid,ring,s],ok},
               {call,tradepost,buyer_identify,[Pid,b],ok},
               {call,tradepost,buyer_insertcash,[Pid,100,b],ok},
               {loopdata,is,[ring,100,s,b]}
              ]).

These are not all the tests (some 6 more are hidden). For the future (for some future iteration), we would like to specify which state transitions are legal, which functions cause these transitions, and ultimately let the machine generate them for us, run the sequences and test whether all is good.

Also, while failing, it would be tremenduously nice if the automatic test generation would show us a trace of the failing run.

Fixing up the problem with identification, a lot of the tests run through, however a new problem is evident.

zen:EUnitFSM zenon$ erlc -o ebin/ src/*.erl test/*.erl
zen:EUnitFSM zenon$ erl -pa ebin/ -eval 'eunit:test(tradepost,[verbose]).'
Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4]
[async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.5  (abort with ^G)
1> ======================== EUnit ========================
module 'tradepost'
  module 'tradepost_tests'
    tradepost_tests: started_properly (Started Properly Test)...ok
    tradepost_tests: identify_seller (Identify Seller Test)...ok
    tradepost_tests: insert_item (Insert Item Test)...ok
    tradepost_tests: withdraw_item (Withdraw Item Test)...ok
    tradepost_tests: identify_buyer (Identify Buyer Test)...ok
    tradepost_tests: insert_cash (Insert Cash Test)...ok
    tradepost_tests: withdraw_cash (Withdraw Cash Test)...ok
    tradepost_tests: interleaving1 (Seller Identifies, Inserts Item, then Buyer Identiefies)...ok
    tradepost_tests: interleaving2 (Buyer Identifies, Inserts Item, then Seller Identiefies)...ok
    tradepost_tests: interleaving3 (Buyer Identifies, Seller Identifies, buyer inserts cash)...ok
    tradepost_tests: interleaving4 (Seller Identifies, Buyer Identifies, Seller inserts item)...ok
    tradepost_tests: interleaving5 (Seller Identifies, Seller inserts item, Buyer Identifies,Buyer Inserts Cash)...*skipped*
undefined
*unexpected termination of test process*
::{function_clause,[{tradepost,item_received,
                               [{insert_cash,100,a},
                                {<0.85.0>,#Ref<0.0.0.153>},
                                {state,ring,undefined,a,a,...}]},
                    {gen_fsm,handle_msg,7},
                    {proc_lib,init_p_do_apply,3}]}

=ERROR REPORT==== 5-Sep-2010::22:33:23 ===
** State machine <0.83.0> terminating
** Last message in was {'$gen_sync_event',
                           {<0.85.0>,#Ref<0.0.0.153>},
                           {insert_cash,100,a}}
** When State == item_received
**      Data  == {state,ring,undefined,a,a,undefined}
** Reason for termination =
** {function_clause,[{tradepost,item_received,
                                [{insert_cash,100,a},
                                 {<0.85.0>,#Ref<0.0.0.153>},
                                 {state,ring,undefined,a,a,undefined}]},
                     {gen_fsm,handle_msg,7},
                     {proc_lib,init_p_do_apply,3}]}
=======================================================
  Failed: 0.  Skipped: 0.  Passed: 11.
One or more tests were cancelled.

1>

It is not possible to insert the cash after the item has been inserted!(?) Clearly, there is an interleaving problem between item insertion and cash insertion. The true design  should be that either buyer or seller must be able to insert xor withdraw their item / cash before closing the deal, irrespective of the other parts item / cash.

An interesting thing to note is that we got this failing tests because this test was longer. It triggered more transitions, and was in a sense, more complex. This is another thing we wish to get for free from an automated test generating engine.

Wishing to test what we just discussed, we add a longer test that should serve as a green light once it goes through.

interleaving6(Pid) ->
    ?fsm_test(Pid,"Seller Identifies, Seller inserts item, Buyer Identifies,"
              "Seller Withdraws Item, Buyer Inserts Cash, Seller Inserts Item"
              "Buyer Withdraws Cash, Seller Withdraws Item"
              [{state,is,pending},
               {call,tradepost,seller_identify,[Pid,s],ok},
               {call,tradepost,seller_insertitem,[Pid,ring,s],ok},
               {call,tradepost,buyer_identify,[Pid,b],ok},
               {call,tradepost,seller_withdraw_item,[Pid,s],ok},
               {call,tradepost,buyer_insertcash,[Pid,100,b],ok},
               {call,tradepost,seller_insertitem[Pid,ring,s],ok},
               {call,tradepost,buyer_withdrawcash,[Pid,b],ok},
               {call,tradepost,seller_withdraw_item,[Pid,s],ok},
               {loopdata,is,[undefined,undefined,s,b]}
              ]).

A little later …

zen:EUnitFSM zenon$ erlc -o ebin/ src/*.erl test/*.erl
zen:EUnitFSM zenon$ erl -pa ebin/ -eval 'eunit:test(tradepost,[verbose]).'
Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.5  (abort with ^G)
1> ======================== EUnit ========================
module 'tradepost'
  module 'tradepost_tests'
    tradepost_tests: started_properly (Started Properly Test)...ok
    tradepost_tests: identify_seller (Identify Seller Test)...ok
    tradepost_tests: insert_item (Insert Item Test)...[0.001 s] ok
    tradepost_tests: withdraw_item (Withdraw Item Test)...[0.001 s] ok
    tradepost_tests: identify_buyer (Identify Buyer Test)...[0.001 s] ok
    tradepost_tests: insert_cash (Insert Cash Test)...[0.001 s] ok
    tradepost_tests: withdraw_cash (Withdraw Cash Test)...[0.001 s] ok
    tradepost_tests: interleaving1 (Seller Identifies, Inserts Item, then Buyer Identiefies)...ok
    tradepost_tests: interleaving2 (Buyer Identifies, Inserts Item, then Seller Identiefies)...ok
    tradepost_tests: interleaving3 (Buyer Identifies, Seller Identifies, buyer inserts cash)...[0.001 s] ok
    tradepost_tests: interleaving4 (Seller Identifies, Buyer Identifies, Seller inserts item)...ok
    tradepost_tests: interleaving5 (Seller Identifies, Seller inserts item, Buyer Identifies,Buyer Inserts Cash)...[0.001 s] ok
    tradepost_tests: interleaving6 (Seller Identifies, Seller inserts item, Buyer Identifies,Seller Withdraws Item, Buyer Inserts Cash, Seller Inserts ItemBuyer Withdraws Cash, Seller Withdraws Item)...[0.001 s] ok
    [done in 0.041 s]
  [done in 0.041 s]
=======================================================
  All 13 tests passed.

1>

That done, the code of tradepost.erl is now

%%%-------------------------------------------------------------------
%%% @author Gianfranco <zenon@zen.home>
%%% @copyright (C) 2010, Gianfranco
%%% Created :  2 Sep 2010 by Gianfranco <zenon@zen.home>
%%%-------------------------------------------------------------------
-module(tradepost).
-behaviour(gen_fsm).

%% API
-export([start_link/0,stop/1,seller_identify/2,seller_insertitem/3,
         seller_withdraw_item/2,buyer_identify/2,buyer_insertcash/3,
         buyer_withdrawcash/2]).

%% States
-export([pending/3,item_received/3,cash_received/3]).

%% gen_fsm callbacks
-export([init/1, handle_event/3, handle_sync_event/4, handle_info/3,
         terminate/3, code_change/4]).
-record(state, {object,cash,seller,buyer}).

%%%===================================================================
%%% API
start_link() -> gen_fsm:start_link(?MODULE, [], []).

stop(Pid) -> gen_fsm:sync_send_all_state_event(Pid,stop).

seller_identify(TradePost,Password) ->
    gen_fsm:sync_send_all_state_event(TradePost,{identify_seller,Password}).
seller_insertitem(TradePost,Item,Password) ->
    gen_fsm:sync_send_event(TradePost,{insert,Item,Password}).
seller_withdraw_item(TradePost,Password) ->
    gen_fsm:sync_send_event(TradePost,{withdraw,Password}).

buyer_identify(TradePost,Password) ->
    gen_fsm:sync_send_all_state_event(TradePost,{identify_buyer,Password}).
buyer_insertcash(TradePost,Amount,Password) ->
    gen_fsm:sync_send_event(TradePost,{insert_cash,Amount,Password}).
buyer_withdrawcash(TradePost,Password) ->
    gen_fsm:sync_send_event(TradePost,{withdraw_cash,Password}).    

%%--------------------------------------------------------------------
pending({insert,Item,Password},_Frm,LoopD = #state{seller=Password}) ->
    {reply,ok,item_received,LoopD#state{object=Item}};
pending({insert_cash,Amount,Password},_Frm,LoopD = #state{buyer=Password}) ->
    {reply,ok,cash_received,LoopD#state{cash=Amount}};
pending(_E,_From,LoopD) ->
    {reply,error,pending,LoopD}.

item_received({withdraw,Password},_Frm,LoopD = #state{seller=Password}) ->
    NewState = case LoopD#state.cash of
                   undefined -> pending;
                   _ -> cash_received
               end,
    {reply,ok,NewState,LoopD#state{object=undefined}};
item_received({withdraw_cash,Password},_Frm,LoopD = #state{buyer=Password}) ->
    NewState = case LoopD#state.object of
                   undefined -> pending;
                   _ -> item_received
               end,
    {reply,ok,NewState,LoopD#state{cash=undefined}};
item_received({insert_cash,Amount,Password},_Frm,LoopD = #state{buyer=Password})->
    {reply,ok,cash_received,LoopD#state{cash=Amount}};
item_received({insert,Item,Password},_Frm,LoopD = #state{seller=Password})->
    {reply,ok,item_received,LoopD#state{object=Item}};
item_received(_E,_From,LoopD) ->
    {reply,error,item_received,LoopD}.

cash_received({withdraw_cash,Password},_From,LoopD = #state{buyer=Password}) ->
    NewState = case LoopD#state.object of
                   undefined -> pending;
                   _ -> item_received
               end,
    {reply,ok,NewState,LoopD#state{cash=undefined}};
cash_received({withdraw,Password},_Frm,LoopD = #state{seller=Password}) ->
    NewState = case LoopD#state.cash of
                   undefined -> pending;
                   _ -> cash_received
               end,
    {reply,ok,NewState,LoopD#state{object=undefined}};
cash_received({insert_cash,Amount,Password},_Frm,LoopD = #state{buyer=Password})->
    {reply,ok,cash_received,LoopD#state{cash=Amount}};
cash_received({insert,Item,Password},_Frm,LoopD = #state{seller=Password})->
    {reply,ok,item_received,LoopD#state{object=Item}};
cash_received(_E,_From,LoopD) ->
    {reply,error,cash_received,LoopD}.

%%--------------------------------------------------------------------
handle_sync_event({identify_seller,Pass},_From,StateName,
                  LoopData=#state{seller=undefined}) ->
    {reply,ok,StateName,LoopData#state{seller=Pass}};
handle_sync_event({identify_buyer,Pass},_From,StateName,
                  LoopData=#state{buyer=undefined}) ->
    {reply,ok,StateName,LoopData#state{buyer=Pass}};
handle_sync_event(stop,_From,_StateName,LoopData) ->
    {stop,normal,ok,LoopData};
handle_sync_event(_E,_From,StateName,LoopData) ->
    {reply,error,StateName,LoopData}.

%%--------------------------------------------------------------------
init([]) -> {ok, pending, #state{}}.
handle_event(_Event, StateName, State) ->{next_state, StateName, State}.
handle_info(_Info, StateName, State) -> {next_state, StateName, State}.
terminate(_Reason, _StateName, _State) -> ok.
code_change(_OldVsn, StateName, State, _Extra) -> {ok, StateName, State}.

And the time has inevitably come for the deal closure API (next up – in the third part), and for your convenience, the full tradepost_tests.erl

%%% @author Gianfranco <zenon@zen.home>
%%% @copyright (C) 2010, Gianfranco
%%% Created :  6 Sep 2010 by Gianfranco <zenon@zen.home>
-module(tradepost_tests).
-include_lib("eunit/include/eunit.hrl").
-include("include/eunit_fsm.hrl").

% This is the main point of "entry" for my EUnit testing.
% A generator which forces setup and cleanup for each test in the testset
main_test_() ->
    {foreach,
     fun setup/0,
     fun cleanup/1,
     % Note that this must be a List of TestSet or Instantiator
     [
      % First Iteration
      fun started_properly/1,
      % Second Iteration
      fun identify_seller/1,
      fun insert_item/1,
      fun withdraw_item/1,
      % Fourth iteration
      fun identify_buyer/1,
      fun insert_cash/1,
      fun withdraw_cash/1,
      % Fifth iteration
      fun interleaving1/1,
      fun interleaving2/1,
      fun interleaving3/1,
      fun interleaving4/1,
      fun interleaving5/1,
      fun interleaving6/1
     ]}.

% Setup and Cleanup
setup()      -> {ok,Pid} = tradepost:start_link(), Pid.
cleanup(Pid) -> tradepost:stop(Pid).

% Pure tests below
% ------------------------------------------------------------------------------
% Let's start simple, I want it to start and check that it is okay.
% I will use the introspective function for this
started_properly(Pid) ->
    ?fsm_test(Pid,"Started Properly Test",
              [{state,is,pending},
               {loopdata,is,[undefined,undefined,undefined,undefined]}
              ]).

% Now, we are adding the Seller API tests
identify_seller(Pid) ->
    ?fsm_test(Pid,"Identify Seller Test",
              [{state,is,pending},
               {call,tradepost,seller_identify,[Pid,seller_password],ok},
               {state,is,pending},
               {loopdata,is,[undefined,undefined,seller_password,undefined]}
              ]).

insert_item(Pid) ->
    ?fsm_test(Pid,"Insert Item Test",
              [{state,is,pending},
               {call,tradepost,seller_identify,[Pid,seller_password],ok},
               {call,tradepost,seller_insertitem,[Pid,playstation,seller_password],ok},
               {state,is,item_received},
               {loopdata,is,[playstation,undefined,seller_password,undefined]}
              ]).

withdraw_item(Pid) ->
    ?fsm_test(Pid,"Withdraw Item Test",
              [{state,is,pending},
               {call,tradepost,seller_identify,[Pid,seller_password],ok},
               {call,tradepost,seller_insertitem,[Pid,button,seller_password],ok},
               {state,is,item_received},
               {call,tradepost,seller_withdraw_item,[Pid,seller_password],ok},
               {state,is,pending},
               {loopdata,is,[undefined,undefined,seller_password,undefined]}
              ]).

identify_buyer(Pid) ->
    ?fsm_test(Pid,"Identify Buyer Test",
              [{state,is,pending},
               {call,tradepost,buyer_identify,[Pid,buyer_password],ok},
               {state,is,pending},
               {loopdata,is,[undefined,undefined,undefined,buyer_password]}
              ]).

insert_cash(Pid) ->
    ?fsm_test(Pid,"Insert Cash Test",
              [{state,is,pending},
               {call,tradepost,buyer_identify,[Pid,buyer_password],ok},
               {call,tradepost,buyer_insertcash,[Pid,100,buyer_password],ok},
               {state,is,cash_received},
               {loopdata,is,[undefined,100,undefined,buyer_password]}
              ]).

withdraw_cash(Pid) ->
    ?fsm_test(Pid,"Withdraw Cash Test",
              [{state,is,pending},
               {call,tradepost,buyer_identify,[Pid,buyer_password],ok},
               {call,tradepost,buyer_insertcash,[Pid,100,buyer_password],ok},
               {call,tradepost,buyer_withdrawcash,[Pid,buyer_password],ok},
               {loopdata,is,[undefined,undefined,undefined,buyer_password]}
              ]).

interleaving1(Pid) ->
    ?fsm_test(Pid,"Seller Identifies, Inserts Item, then Buyer Identiefies",
              [{state,is,pending},
               {call,tradepost,seller_identify,[Pid,s],ok},
               {call,tradepost,seller_insertitem,[Pid,ring,s],ok},
               {call,tradepost,buyer_identify,[Pid,b],ok},
               {loopdata,is,[ring,undefined,s,b]}
              ]).

interleaving2(Pid) ->
    ?fsm_test(Pid,"Buyer Identifies, Inserts Item, then Seller Identiefies",
              [{state,is,pending},
               {call,tradepost,buyer_identify,[Pid,b],ok},
               {call,tradepost,buyer_insertcash,[Pid,100,b],ok},
               {call,tradepost,seller_identify,[Pid,s],ok},
               {loopdata,is,[undefined,100,s,b]}
              ]).

interleaving3(Pid) ->
    ?fsm_test(Pid,"Buyer Identifies, Seller Identifies, buyer inserts cash",
              [{state,is,pending},
               {call,tradepost,buyer_identify,[Pid,b],ok},
               {call,tradepost,seller_identify,[Pid,s],ok},
               {call,tradepost,buyer_insertcash,[Pid,100,b],ok},
               {loopdata,is,[undefined,100,s,b]}
              ]).

interleaving4(Pid) ->
    ?fsm_test(Pid,"Seller Identifies, Buyer Identifies, Seller inserts item",
              [{state,is,pending},
               {call,tradepost,seller_identify,[Pid,s],ok},
               {call,tradepost,buyer_identify,[Pid,b],ok},
               {call,tradepost,seller_insertitem,[Pid,ring,s],ok},
               {loopdata,is,[ring,undefined,s,b]}
              ]).

interleaving5(Pid) ->
    ?fsm_test(Pid,"Seller Identifies, Seller inserts item, Buyer Identifies,"
              "Buyer Inserts Cash",
              [{state,is,pending},
               {call,tradepost,seller_identify,[Pid,s],ok},
               {call,tradepost,seller_insertitem,[Pid,ring,s],ok},
               {call,tradepost,buyer_identify,[Pid,b],ok},
               {call,tradepost,buyer_insertcash,[Pid,100,b],ok},
               {loopdata,is,[ring,100,s,b]}
              ]).

interleaving6(Pid) ->
    ?fsm_test(Pid,"Seller Identifies, Seller inserts item, Buyer Identifies,"
              "Seller Withdraws Item, Buyer Inserts Cash, Seller Inserts Item"
              "Buyer Withdraws Cash, Seller Withdraws Item",
              [{state,is,pending},
               {call,tradepost,seller_identify,[Pid,s],ok},
               {call,tradepost,seller_insertitem,[Pid,ring,s],ok},
               {call,tradepost,buyer_identify,[Pid,b],ok},
               {call,tradepost,seller_withdraw_item,[Pid,s],ok},
               {call,tradepost,buyer_insertcash,[Pid,100,b],ok},
               {call,tradepost,seller_insertitem,[Pid,ring,s],ok},
               {call,tradepost,buyer_withdrawcash,[Pid,b],ok},
               {call,tradepost,seller_withdraw_item,[Pid,s],ok},
               {loopdata,is,[undefined,undefined,s,b]}
              ]).

EUnit Testing gen_fsm – Part 1


This post will be TDD based and shows how the Tradepost program is developed and is tested.

First iteration – Start and Stop

I start writing the test! This gives me the opportunity to crystallize my thoughts about the program.

The module implementing the tests of the Tradepost

-module(tradepost_tests).
-include_lib("eunit/include/eunit.hrl").
% This is the main point of "entry" for my EUnit testing.
% A generator which forces setup and cleanup for each test in the testset
main_test_() ->
    {foreach,
     fun setup/0,
     fun cleanup/1,
     % Note that this must be a List of TestSet or Instantiator
     % (I have instantiators == functions generating tests)
     [
      % First Iteration
      fun started_properly/1,
     ]}.

% Setup and Cleanup
setup()      -> {ok,Pid} = tradepost:start_link(), Pid.
cleanup(Pid) -> tradepost:stop(Pid).

% Pure tests below
% ------------------------------------------------------------------------------
% Let's start simple, I want it to start and check that it is okay.
% I will use the introspective function for this
started_properly(Pid) ->
    fun() ->
            ?assertEqual(pending,tradepost:introspection_statename(Pid)),
            ?assertEqual([undefined,undefined,undefined,undefined,undefined],
                         tradepost:introspection_loopdata(Pid))
    end.

Compilation and running should fail as the gen_fsm module is barely minimal

zen:EUnitFSM zenon$ tree .
.
├── ebin
├── include
├── src
│   └── tradepost.erl
└── test
    └── tradepost_tests.erl

4 directories, 2 files
zen:EUnitFSM zenon$ 

zen:EUnitFSM zenon$ erlc -o ebin/ src/*.erl test/*.erl
zen:EUnitFSM zenon$ erl -pa ebin/
Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4]
[async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.5  (abort with ^G)
1> eunit:test(tradepost).
tradepost_tests: started_properly...*failed*
::error:{assertEqual_failed,
          [{module,tradepost_tests},
           {line,33},
           {expression,"tradepost : introspection_statename ( Pid )"},
           {expected,pending},
           {value,ok}]}
  in function tradepost_tests:'-started_properly/1-fun-0-'/2
  in call from tradepost_tests:'-started_properly/1-fun-2-'/1

module 'tradepost_tests'
*** context cleanup failed ***
::error:undef
  in function tradepost:stop/1
    called as stop(<0.39.0>)

=======================================================
  Failed: 1.  Skipped: 0.  Passed: 0.
One or more tests were cancelled.
error
2>

Clearly the tradepost does not handle the introspective functions and stop, let’s write them first.

The interface

introspection_statename(TradePost) ->
    gen_fsm:sync_send_all_state_event(TradePost,which_statename).
introspection_loopdata(TradePost) ->
    gen_fsm:sync_send_all_state_event(TradePost,which_loopdata).
stop(Pid) -> gen_fsm:sync_send_all_state_event(Pid,stop).

And the handling

handle_sync_event(which_statename, _From, StateName, LoopData) ->
    {reply, StateName, StateName, LoopData};
handle_sync_event(which_loopdata, _From, StateName, LoopData) ->
    {reply,tl(tuple_to_list(LoopData)),StateName,LoopData};
handle_sync_event(stop,_From,_StateName,LoopData) ->
    {stop,normal,ok,LoopData}.

Running

zen:EUnitFSM zenon$ erl -pa ebin/
Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4]
[async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.5  (abort with ^G)
1> eunit:test(tradepost,[verbose]).
======================== EUnit ========================
module 'tradepost'
  module 'tradepost_tests'
    tradepost_tests: started_properly...ok
    [done in 0.004 s]
  [done in 0.005 s]
=======================================================
  Test passed.
ok
2>


Wow, I’m glad we got that sorted out. Now,  as we have set the first nail in the mountain and hooked us to it, the climb begins in a series of cycles. Next up, the Seller API.

Second Iteration – Seller API

What the seller needs, is a way to insert an item, and a way to remove an item. Also to identify him/her self with a naive password approach. Once identified, the seller (and only the seller) should be able to add and retract items.

The main_test_() has now been expanded to

% This is the main point of "entry" for my EUnit testing.
% A generator which forces setup and cleanup for each test in the testset
main_test_() ->
    {foreach,
     fun setup/0,
     fun cleanup/1,
     % Note that this must be a List of TestSet or Instantiator
     % (I have instantiators)
     [
      % First Iteration
      fun started_properly/1,
      % Second Iteration
      fun identify_seller/1,
      fun insert_item/1,
      fun withdraw_item/1
     ]}.

And the definition of the tests themselves

% Now, we are adding the Seller API tests
identify_seller(Pid) ->
    fun() ->
            % From Pending, identify seller, then state should be pending
            % loopdata should now contain seller_password
            ?assertEqual(pending,tradepost:introspection_statename(Pid)),
            ?assertEqual(ok,tradepost:seller_identify(Pid,seller_password)),
            ?assertEqual(pending,tradepost:introspection_statename(Pid)),
            ?assertEqual([undefined,undefined,seller_password,undefined,
                       undefined],tradepost:introspection_loopdata(Pid))
    end.

insert_item(Pid) ->
    fun() ->
            % From pending and identified seller, insert item
            % state should now be item_received, loopdata should now contain itm
            tradepost:introspection_statename(Pid),
            tradepost:seller_identify(Pid,seller_password),
            ?assertEqual(ok,tradepost:seller_insertitem(Pid,playstation,
                                                  seller_password)),
            ?assertEqual(item_received,tradepost:introspection_statename(Pid)),
            ?assertEqual([playstation,undefined,seller_password,undefined,
                       undefined],tradepost:introspection_loopdata(Pid))
    end.

withdraw_item(Pid) ->
    fun() ->
            % identified seller and inserted item, withdraw item
            % state should now be pending, loopdata should now contain only password
            tradepost:seller_identify(Pid,seller_password),
            tradepost:seller_insertitem(Pid,playstation,seller_password),
            ?assertEqual(ok,tradepost:withdraw_item(Pid,seller_password)),
            ?assertEqual(pending,tradepost:introspection_statename(Pid)),
            ?assertEqual([undefined,undefined,seller_password,undefined,
                       undefined],tradepost:introspection_loopdata(Pid))
    end.

Now, this is a lot of code duplication per test, and  I shall start rewriting the code into something more wieldy, but first: the failed compilation and the code making it pass.

zen:EUnitFSM zenon$ erlc -o ebin/ src/*.erl test/*.erl
zen:EUnitFSM zenon$ erl -pa ebin/
Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4]
[async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.5  (abort with ^G)
1> eunit:test(tradepost).
tradepost_tests: identify_seller...*failed*
::error:undef
  in function tradepost:seller_identify/2
    called as seller_identify(<0.39.0>,seller_password)
  in call from tradepost_tests:'-identify_seller/1-fun-1-'/2
  in call from tradepost_tests:'-identify_seller/1-fun-4-'/1

tradepost_tests: insert_item...*failed*
::error:undef
  in function tradepost:seller_identify/2
    called as seller_identify(<0.39.0>,seller_password)
  in call from tradepost_tests:'-insert_item/1-fun-3-'/1

tradepost_tests: withdraw_item...*failed*
::error:undef
  in function tradepost:seller_identify/2
    called as seller_identify(<0.39.0>,seller_password)
  in call from tradepost_tests:'-withdraw_item/1-fun-3-'/1

=======================================================
  Failed: 3.  Skipped: 0.  Passed: 1.
error
2>

Some writing and enjoying later, the module

%%%-------------------------------------------------------------------
%%% @author Gianfranco <zenon@zen.home>
%%% @copyright (C) 2010, Gianfranco
%%% Created :  2 Sep 2010 by Gianfranco <zenon@zen.home>
%%%-------------------------------------------------------------------
-module(tradepost).
-behaviour(gen_fsm).

%% API
-export([start_link/0,introspection_statename/1,introspection_loopdata/1,
         stop/1,seller_identify/2,seller_insertitem/3,withdraw_item/2]).

%% States
-export([pending/2,pending/3,item_received/3]).

%% gen_fsm callbacks
-export([init/1, handle_event/3, handle_sync_event/4, handle_info/3,
         terminate/3, code_change/4]).
-record(state, {object,cash,seller,buyer,time}).

%%% API
start_link() -> gen_fsm:start_link(?MODULE, [], []).

introspection_statename(TradePost) ->
    gen_fsm:sync_send_all_state_event(TradePost,which_statename).
introspection_loopdata(TradePost) ->
    gen_fsm:sync_send_all_state_event(TradePost,which_loopdata).
stop(Pid) -> gen_fsm:sync_send_all_state_event(Pid,stop).

seller_identify(TradePost,Password) ->
    gen_fsm:sync_send_event(TradePost,{identify_seller,Password}).
seller_insertitem(TradePost,Item,Password) ->
    gen_fsm:sync_send_event(TradePost,{insert,Item,Password}).

withdraw_item(TradePost,Password) ->
    gen_fsm:sync_send_event(TradePost,{withdraw,Password}).

%%--------------------------------------------------------------------
pending(_Event,LoopData) -> {next_state,pending,LoopData}.

pending({identify_seller,Password},_Frm,LoopD = #state{seller=Password}) ->
    {reply,ok,pending,LoopD};
pending({identify_seller,Password},_Frm,LoopD = #state{seller=undefined}) ->
    {reply,ok,pending,LoopD#state{seller=Password}};
pending({identify_seller,_},_,LoopD) ->
    {reply,error,pending,LoopD};

pending({insert,Item,Password},_Frm,LoopD = #state{seller=Password}) ->
    {reply,ok,item_received,LoopD#state{object=Item}};
pending({insert,_,_},_Frm,LoopD) ->
    {reply,error,pending,LoopD}.

item_received({withdraw,Password},_Frm,LoopD = #state{seller=Password}) ->
    {reply,ok,pending,LoopD#state{object=undefined}};
item_received({withdraw,_},_Frm,LoopD) ->
    {reply,error,item_received,LoopD}.

%%--------------------------------------------------------------------
handle_sync_event(which_statename, _From, StateName, LoopData) ->
    {reply, StateName, StateName, LoopData};
handle_sync_event(which_loopdata, _From, StateName, LoopData) ->
    {reply,tl(tuple_to_list(LoopData)),StateName,LoopData};
handle_sync_event(stop,_From,_StateName,LoopData) ->
    {stop,normal,ok,LoopData};
handle_sync_event(_E,_From,StateName,LoopData) ->
    {reply,ok,StateName,LoopData}.

%%--------------------------------------------------------------------
init([]) -> {ok, pending, #state{}}.
handle_event(_Event, StateName, State) ->{next_state, StateName, State}.
handle_info(_Info, StateName, State) -> {next_state, StateName, State}.
terminate(_Reason, _StateName, _State) -> ok.
code_change(_OldVsn, StateName, State, _Extra) -> {ok, StateName, State}.

And proof for the functioning of it.

zen:EUnitFSM zenon$ erlc -o ebin/ src/*.erl test/*.erl
zen:EUnitFSM zenon$ erl -pa ebin/ -eval 'eunit:test(tradepost,[verbose]).'
Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4]
[async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.5  (abort with ^G)
1> ======================== EUnit ========================
module 'tradepost'
  module 'tradepost_tests'
    tradepost_tests: started_properly...ok
    tradepost_tests: identify_seller...ok
    tradepost_tests: insert_item...ok
    tradepost_tests: withdraw_item...ok
    [done in 0.015 s]
  [done in 0.015 s]
=======================================================
  All 4 tests passed.
1>

Awesome! And we got a bonus as well: how to run eunit directly from the command line with the -eval command. Cool, oneliners always make you feel more “1337”.

It all looks very good so far, but I will use the third iteration to fixing up the eunit module, it has way to much duplication and could be made more declarative (specifying what should be computed – not how [ which is kind of more abstract <of course someone will oppose>]).

Third Iteration –  Eunit GenFSM DSL (do not worry)

Testing gen_fsm’s should ideally be all about testing state transitions, in-state computations and variables. For this purpose, I would like to have my DSL that handles all of the pesky details for me.

To exemplify, I would like to write the first test

started_properly(Pid) ->
    fun() ->
            ?assertEqual(pending,tradepost:introspection_statename(Pid)),
            ?assertEqual([undefined,undefined,undefined,undefined,undefined],
                         tradepost:introspection_loopdata(Pid))
    end.

as something along these lines

started_properly(Pid) ->
    {"Proper startup test",
     [{statename,is,pending},
      {loopdata,is,[undefined,undefined,undefined,undefined,undefined]}
      ]}.

And, this test

insert_item(Pid) ->
    fun() ->
        % From pending and identified seller, insert item
        % state should now be item_received, loopdata should now contain itm
        tradepost:introspection_statename(Pid),
        tradepost:seller_identify(Pid,seller_password),
        ?assertEqual(ok,tradepost:seller_insertitem(Pid,playstation,
                                              seller_password)),
        ?assertEqual(item_received,tradepost:introspection_statename(Pid)),
        ?assertEqual([playstation,undefined,seller_password,undefined,
                   undefined],tradepost:introspection_loopdata(Pid))
    end.

as something along these lines

insert_item(Pid) ->
    {"Insert Item Test",
      [{state,is,pending},
       {call,tradepost,seller_identify,[Pid,seller_password],ok},
       {call,tradepost,seller_insertitem,[Pid,playstation,seller_password]},
       {state,is,item_received},
       {loopdata,is,[playstation,undefined,seller_password,undefined,undefined]}
      ]}.

This is a symbolic method that defines our DSL in a yet readable way, hiding the logic. For this to work, we need a translation from our DSL to actual EUnit syntax without losing the EUnit machinery. We will also drop the intrusive introspective functions for the usage of the better sys:get_status/1 (thank you Ulf).

Thus, let the journey begin, first with the translation of the Test. This can be done in two ways, either at runtime or at compile time with parse-transforms. I choose the runtime one with translation functions and macros. Ulf Wiger has a neat library for cooler parse transforms, but I shall not use this for now, (the code will probably be revised many times)

The test module has now been changed to

-module(tradepost_tests).
-include_lib("eunit/include/eunit.hrl").
-include("include/eunit_fsm.hrl").

% This is the main point of "entry" for my EUnit testing.
% A generator which forces setup and cleanup for each test in the testset
main_test_() ->
    {foreach,
     fun setup/0,
     fun cleanup/1,
     % Note that this must be a List of TestSet or Instantiator
     [
      % First Iteration
      fun started_properly/1,
      % Second Iteration
      fun identify_seller/1,
      fun insert_item/1,
      fun withdraw_item/1
     ]}.

% Setup and Cleanup
setup()      -> {ok,Pid} = tradepost:start_link(), Pid.
cleanup(Pid) -> tradepost:stop(Pid).

% Pure tests below
% ------------------------------------------------------------------------------
% Let's start simple, I want it to start and check that it is okay.
% I will use the introspective function for this
started_properly(Pid) ->
    ?fsm_test(tradepost,Pid,"Started Properly Test",
      [{state,is,pending},
       {loopdata,is,[undefined,undefined,undefined,undefined,undefined]}
     ]).

% Now, we are adding the Seller API tests
identify_seller(Pid) ->
    ?fsm_test(Pid,"Identify Seller Test",
      [{state,is,pending},
       {call,tradepost,seller_identify,[Pid,seller_password],ok},
       {state,is,pending},
       {loopdata,is,[undefined,undefined,seller_password,undefined,undefined]}
      ]).

insert_item(Pid) ->
    ?fsm_test(Pid,"Insert Item Test",
       [{state,is,pending},
        {call,tradepost,seller_identify,[Pid,seller_password],ok},
        {call,tradepost,seller_insertitem,[Pid,playstation,seller_password],ok},
        {state,is,item_received},
        {loopdata,is,[playstation,undefined,seller_password,undefined,undefined]}
       ]).

withdraw_item(Pid) ->
    ?fsm_test(Pid,"Withdraw Item Test",
       [{state,is,pending},
        {call,tradepost,seller_identify,[Pid,seller_password],ok},
        {call,tradepost,seller_insertitem,[Pid,button,seller_password],ok},
        {state,is,item_received},
        {call,tradepost,seller_withdraw_item,[Pid,seller_password],ok},
        {state,is,pending},
        {loopdata,is,[undefined,undefined,seller_password,undefined,undefined]}
       ]).

And the supporting logic is found in eunit_fsm.hrl

-define(fsm_test(Id,Title,CmdList),
  {Title,fun() -> [ eunit_fsm:translateCmd(Id,Cmd) || Cmd <- CmdList] end}).

together with eunit_fsm.erl

-module(eunit_fsm).
-export([translateCmd/2,get/2]).
-define(Expr(X),??X).
translateCmd(Id,{state,is,X}) ->
    case get(Id,"StateName") of
        X -> true;
        _V ->  .erlang:error({statename_match_failed,
                              [{module, ?MODULE},
                               {line, ?LINE},
                               {expected, X},
                               {value, _V}]})
    end;
translateCmd(_Id,{call,M,F,A,X}) ->
    case apply(M,F,A) of
        X -> ok;
        _V ->  .erlang:error({function_call_match_failed,
                              [{module, ?MODULE},
                               {line, ?LINE},
                               {expression, ?Expr(apply(M,F,A))},
                               {expected, X},
                               {value, _V}]})
    end;
translateCmd(Id,{loopdata,is,X}) ->
    case tl(tuple_to_list(get(Id,"StateData"))) of
        X    -> true;
        _V ->    .erlang:error({loopdata_match_failed,
                                [{module, ?MODULE},
                                 {line, ?LINE},
                                 {expected, X},
                                 {value, _V}]})
    end.

% StateName or StateData
get(Id,Which) ->
    {status,_Pid,_ModTpl, List} = sys:get_status(Id),
    AllData = lists:flatten([ X || {data,X} <- lists:last(List) ]),
    proplists:get_value(Which,AllData).

Now, we have a method of expressing the tests more clearly, next up is the buyer API. Testing if of the essence!

zen:EUnitFSM zenon$ tree .
.
├── ebin
├── include
│   └── eunit_fsm.hrl
├── src
│   └── tradepost.erl
└── test
    ├── eunit_fsm.erl
    └── tradepost_tests.erl

4 directories, 4 files
zen:EUnitFSM zenon$ 

Compile and run

zen:EUnitFSM zenon$ erlc -o ebin/ src/*.erl test/*.erl
zen:EUnitFSM zenon$ erl -pa ebin/ -eval 'eunit:test(tradepost,[verbose]).'
Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4]
[async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.5  (abort with ^G)
1> ======================== EUnit ========================
module 'tradepost'
  module 'tradepost_tests'
    tradepost_tests: started_properly (Started Properly Test)...[0.001 s] ok
    tradepost_tests: identify_seller (Identify Seller Test)...ok
    tradepost_tests: insert_item (Insert Item Test)...ok
    tradepost_tests: withdraw_item (Withdraw Item Test)...ok
    [done in 0.014 s]
  [done in 0.014 s]
=======================================================
  All 4 tests passed.

1>

And so, it works. Next part is adding buyer API and extending the DSL where needed, also maybe rewriting it with Ulf’s awesome parse transform libs.

Erlang EUnit – continuation 3 – Test Control


Knowing Basics of Setup, Cleanup understanding test representation, it’s time to look at EUnit test control. Test control encompassed the ability to specify

  1. If the TestSet should be run in a specific subprocess
  2. If the TestSet should be run in a specific subprocess on a specific node
  3. What timeout a TestSet should have
  4. If the STO’s in the TestSet should be run in a specific predetermined order
  5. If the STO’s in the TestSet should be run in parallell (if possible)
  6. If the STO’s in the TestSet should be run in parallell (if possible), but with the added control that no more than N of them may be run at the same time in parallell

Each of these control specifications are designated by tuples (like most things in EUnit), and may replace any TestSet (single test or deeplist) in the same place where the testset whas previousy placed: But, the replaced testset should then be placed inside the TestSet holder of the tuple.

Replacing a single direct testset as first element in a test generating function (remember the _test_())

testgenerator_test_() -> TESTSET.
testgenerator_test_() -> {CONTROL-TUPLE,TESTSET}.

Replacing a deep list TestSet inside another TestSet.

testgenerator_test_() -> [ TESTSET1, TESTSET2 ].
testgenerator_test_() -> [ TESTSET1, {CONTROL, TESTSET2} ]

without further ado, here comes the examples!

Subprocess Specification for single process and single process on node.

{spawn, TestSet }
{spawn, Node::atom(), TestSet }

Then, may be used as follows

testgeneratorA_test_() -> {spawn, nullarySTO1() }.
testgeneratorB_test_() -> {spawn, [nullarySTO1(),tupleSTO1(),tupleSTO4()]}.
testgeneratorC_test_() -> {spawn, 'eunit@zen', nullarySTO1() }.
testgeneratorD_test_() -> {spawn, 'eunit@zen', [ nullarySTO1() ,
                                          tupleSTO1(),
                                          tupleSTO4()]}.
testgeneratorE_test_() ->
    [ nullarySTO1(),
      nullarySTO2(),
      {spawn, [ {spawn, 'eunit2@zen', tupleSTO1()},
                {spawn, 'eunit2@zen', tupleSTO2()},
                tupleSTO3() ] },
      tupleSTO4() ].

As can be seen, the Control Tuple (CT) can be placed in any place of a TestSet, and puts that replaced testset into itself. Conceptually, it can be thought of as a function, CT(TestSet) which returns the CT with the testset inside.

To prove my point, I shall compile and run it.

zen:EunitBasic4 zenon$ tree .
.
├── ebin
├── src
│   └── mylist.erl
└── test
    └── mylist_tests.erl

3 directories, 2 files

The mylists_tests.erl contains

-module(mylist_tests).
-include_lib("eunit/include/eunit.hrl").
-compile(export_all).
% Basics
nullarySTO1() -> fun() -> ?assertEqual(6,mylist:sum([1,2,3])) end.
nullarySTO2() -> fun nullarySTO1/0.
nullarySTO3() -> fun mylist_tests:nullarySTO1/0.
tupleSTO1() -> { mylist_tests, nullarySTO1 }.
tupleSTO2() -> { mylist_tests, nullarySTO2 }.
tupleSTO3() -> { mylist_tests, nullarySTO3 }.
tupleSTO4() -> ?_test( fun() -> ?assertEqual(6,mylist:sum([1,2,3])) end ).
tupleSTO5() -> ?_test( begin ?assertEqual(6,mylist:sum([1,2,3])) end ).
tupleSTO6() -> ?_test( nullarySTO2() ).
tupleSTO7() -> { 16, nullarySTO3() }.

% Proof of concept
testgeneratorA_test_() -> {spawn, nullarySTO1() }.
testgeneratorB_test_() -> {spawn, [ nullarySTO1() , tupleSTO1(), tupleSTO4()]}.
testgeneratorC_test_() -> {spawn, 'eunit@zen', nullarySTO1() }.
testgeneratorD_test_() -> {spawn, 'eunit@zen', [ nullarySTO1() ,
                                          tupleSTO1(),
                                          tupleSTO4()]}.
testgeneratorE_test_() ->
    [ nullarySTO1(),
      nullarySTO2(),
      {spawn, [ {spawn, 'eunit2@zen', tupleSTO1()},
                {spawn, 'eunit2@zen', tupleSTO2()},
                tupleSTO3() ] },
      tupleSTO4() ].

So, starting 3 shells (one eunit node, and two other nodes), as follows [ note: the other nodes should have the path to the ebins : AND have the code loaded by some means, please see this post to the fun problem issue ]

In orde to automate the code loading, the nl(Module) function is used

nl(Module) -- load module on all nodes

First shell

zen:EunitBasic4 zenon$ erl -pa ebin/ -sname eunit@zen -setcookie eunit
Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4]
[async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.5  (abort with ^G)
(eunit@zen)1>

Second Shell

zen:EunitBasic4 zenon$ erl -pa ebin/ -sname eunit2@zen -setcookie eunit
Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4]
[async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.5  (abort with ^G)
(eunit2@zen)1>

Third shell, starts, connects, loads on all nodes and runs

zen:EUnitBasic4 zenon$ erlc -o ebin/ src/*.erl test/*.erl
zen:EUnitBasic4 zenon$ erl -pa ebin/ -sname base@zen -setcookie eunit
Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4]
[async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.5  (abort with ^G)
(base@zen)1> net_kernel:connect_node(eunit@zen).
true
(base@zen)2> net_kernel:connect_node(eunit2@zen).
true
(base@zen)3> nl(mylist_tests).
abcast
(base@zen)4> eunit:test(mylist,[verbose]).
======================== EUnit ========================
module 'mylist'
  module 'mylist_tests'
    mylist_tests: nullarySTO1...ok
    mylist_tests: nullarySTO1...ok
    mylist_tests: nullarySTO1...ok
    mylist_tests:13: tupleSTO4...ok
    mylist_tests: nullarySTO1...[0.003 s] ok
    mylist_tests: nullarySTO1...ok
    mylist_tests: nullarySTO1...ok
    mylist_tests:13: tupleSTO4...ok
    mylist_tests: nullarySTO1...ok
    mylist_tests: nullarySTO2...ok
    mylist_tests: nullarySTO1...ok
    mylist_tests: nullarySTO2...ok
    mylist_tests: nullarySTO3...ok
    mylist_tests:13: tupleSTO4...ok
    [done in 0.070 s]
  [done in 0.070 s]
=======================================================
  All 14 tests passed.
ok
(base@zen)5>


This concludes spawn, as can be seen, all the tests pass, even the node spawned ones. Don’t forget to load the code on the remote nodes first!

Timeout Control

{timeout, Time::number(), Tests}

The effect is that all tests in the TestSet “argument” are given the total time of Time seconds to complete. If the TestSet has not finished during the Time seconds, the TestSet is stopped abruptly without cleanup. Any Setup and Cleanup is also considered during this time.

testgeneratorA_test_() -> {timeout, 1, nullarySTO1() }.
testgeneratorB_test_() -> {timeout, 1, [ nullarySTO1() ,
                                   tupleSTO1(),
                                   tupleSTO4() ] }.
testgeneratorC_test_() ->
    [ nullarySTO1(),
     {timeout, 5, [ {timeout, 1, tupleSTO1()},
                   {timeout, 2, tupleSTO2()},
                   tupleSTO3() ] },
     tupleSTO4() ].

A cool detail is that it is fully legal to do as I did, that is, to nest levels of timeout, which makes total sense if you know some parts may hang indefinitely.

Thus my mylist_tests module now contains

-module(mylist_tests).
-include_lib("eunit/include/eunit.hrl").
-compile(export_all).

% Basics
nullarySTO1() -> fun() -> ?assertEqual(6,mylist:sum([1,2,3])) end.
nullarySTO2() -> fun nullarySTO1/0.
nullarySTO3() -> fun mylist_tests:nullarySTO1/0.
tupleSTO1() -> { mylist_tests, nullarySTO1 }.
tupleSTO2() -> { mylist_tests, nullarySTO2 }.
tupleSTO3() -> { mylist_tests, nullarySTO3 }.
tupleSTO4() -> ?_test( fun() -> ?assertEqual(6,mylist:sum([1,2,3])) end ).
tupleSTO5() -> ?_test( begin ?assertEqual(6,mylist:sum([1,2,3])) end ).
tupleSTO6() -> ?_test( nullarySTO2() ).
tupleSTO7() -> { 16, nullarySTO3() }.

% % Proof of concept
testgeneratorA_test_() -> {timeout, 1, nullarySTO1() }.
testgeneratorB_test_() -> {timeout, 1, [ nullarySTO1() ,
                                   tupleSTO1(),
                                   tupleSTO4() ] }.
testgeneratorC_test_() ->
    [ nullarySTO1(),
     {timeout, 5, [ {timeout, 1, tupleSTO1()},
                   {timeout, 2, tupleSTO2()},
                   tupleSTO3() ] },
     tupleSTO4() ].

And the tests are run as

zen:EUnitBasic4 zenon$ erlc -o ebin/ src/*.erl test/*.erl
zen:EUnitBasic4 zenon$ erl -pa ebin/
Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4]
[async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.5  (abort with ^G)
1> eunit:test(mylist,[verbose]).
======================== EUnit ========================
module 'mylist'
  module 'mylist_tests'
    mylist_tests: nullarySTO1...ok
    mylist_tests: nullarySTO1...ok
    mylist_tests: nullarySTO1...ok
    mylist_tests:13: tupleSTO4...ok
    mylist_tests: nullarySTO1...ok
    mylist_tests: nullarySTO1...ok
    mylist_tests: nullarySTO2...ok
    mylist_tests: nullarySTO3...ok
    mylist_tests:13: tupleSTO4...ok
    [done in 0.027 s]
  [done in 0.027 s]
=======================================================
  All 9 tests passed.
ok
2>

This concludes the Timeout demonstration as there is not more to say, sadly :)

Strict Order Of Tests

As is, EUnit is not forced to follow the in-module order or even in-test order of a TestSet, so it’s handy to have a control structure for this.

{inorder, Tests}

As basic as it seems, I shall supply a test for you, with the following module

-module(mylist_tests).
-include_lib("eunit/include/eunit.hrl").
-compile(export_all).

nullarySTO1() -> fun() -> ?assertEqual(6,mylist:sum([1,2,3])) end.
nullarySTO2() -> fun nullarySTO1/0.
nullarySTO3() -> fun mylist_tests:nullarySTO1/0.

tupleSTO1() -> { mylist_tests, nullarySTO1 }.
tupleSTO2() -> { mylist_tests, nullarySTO2 }.
tupleSTO3() -> { mylist_tests, nullarySTO3 }.

tupleSTO4() -> ?_test( fun() -> ?assertEqual(6,mylist:sum([1,2,3])) end ).
tupleSTO5() -> ?_test( begin ?assertEqual(6,mylist:sum([1,2,3])) end ).
tupleSTO6() -> ?_test( nullarySTO2() ).
tupleSTO7() -> { 16, nullarySTO3() }.

generator_test_() ->
    {inorder,
     [
      nullarySTO1(),
      nullarySTO2(),
      nullarySTO3(),
      tupleSTO1(),
      tupleSTO2(),
      tupleSTO3(),
      tupleSTO4(),
      tupleSTO5(),
      tupleSTO6(),
      tupleSTO7()
     ]}.

And compilation, running, etc (you know the drill by now).

zen:EUnitBasic4 zenon$ erlc -o ebin/ src/*.erl test/*.erl
zen:EUnitBasic4 zenon$ erl -pa ebin/
Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4]
[async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.5  (abort with ^G)
1> eunit:test(mylist).
  All 10 tests passed.
ok
2> eunit:test(mylist,[verbose]).
======================== EUnit ========================
module 'mylist'
  module 'mylist_tests'
    mylist_tests: nullarySTO1...ok
    mylist_tests: nullarySTO2...ok
    mylist_tests: nullarySTO1...ok
    mylist_tests: nullarySTO1...ok
    mylist_tests: nullarySTO2...ok
    mylist_tests: nullarySTO3...ok
    mylist_tests:13: tupleSTO4...ok
    mylist_tests:14: tupleSTO5...ok
    mylist_tests:15: tupleSTO6...ok
    mylist_tests:16: nullarySTO1...ok
    [done in 0.030 s]
  [done in 0.030 s]
=======================================================
  All 10 tests passed.
ok
3>

Now we are dead sure all tests where run in the strict appropriate order.

Parallel Test execution (if possible)

What if we have thousands of tests, which test single part of library routines? Like mylist tests with add, etc? Well, it sure makes sense to run them in parallel! Well, inparallel to the resque!

{inparallel, Tests}

Examples are more talkative and I will supply with such an example

-module(mylist_tests).
-include_lib("eunit/include/eunit.hrl").
-compile(export_all).

nullarySTO1() -> fun() -> ?assertEqual(6,mylist:sum([1,2,3])) end.
nullarySTO2() -> fun nullarySTO1/0.
nullarySTO3() -> fun mylist_tests:nullarySTO1/0.

tupleSTO1() -> { mylist_tests, nullarySTO1 }.
tupleSTO2() -> { mylist_tests, nullarySTO2 }.
tupleSTO3() -> { mylist_tests, nullarySTO3 }.

tupleSTO4() -> ?_test( fun() -> ?assertEqual(6,mylist:sum([1,2,3])) end ).
tupleSTO5() -> ?_test( begin ?assertEqual(6,mylist:sum([1,2,3])) end ).
tupleSTO6() -> ?_test( nullarySTO2() ).
tupleSTO7() -> { 16, nullarySTO3() }.

generator_test_() ->
    {inparallel,
     [
      nullarySTO1(),
      nullarySTO2(),
      nullarySTO3(),
      tupleSTO1(),
      tupleSTO2(),
      tupleSTO3(),
      tupleSTO4(),
      tupleSTO5(),
      tupleSTO6(),
      tupleSTO7()
     ]}.

Lo and behold the run-timeth!

zen:EUnitBasic4 zenon$ erlc -o ebin/ src/*.erl test/*.erl
zen:EUnitBasic4 zenon$ erl -pa ebin/
Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4]
[async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.5  (abort with ^G)
1> eunit:test(mylist,[verbose]).
======================== EUnit ========================
module 'mylist'
  module 'mylist_tests'
    mylist_tests: nullarySTO1...ok
    mylist_tests: nullarySTO2...ok
    mylist_tests: nullarySTO1...ok
    mylist_tests: nullarySTO1...ok
    mylist_tests: nullarySTO2...ok
    mylist_tests: nullarySTO3...ok
    mylist_tests:13: tupleSTO4...ok
    mylist_tests:14: tupleSTO5...ok
    mylist_tests:15: tupleSTO6...ok
    mylist_tests:16: nullarySTO1...ok
    [done in 0.004 s]
  [done in 0.004 s]
=======================================================
  All 10 tests passed.
ok
2>

Inparallel but with upper bound on parallelism

Maybe you don’t wish more processes to be running in parallel than you have cores, or whatnot, thus, use inparallel with a maximum value of N.

-module(mylist_tests).
-include_lib("eunit/include/eunit.hrl").
-compile(export_all).

nullarySTO1() -> fun() -> ?assertEqual(6,mylist:sum([1,2,3])) end.
nullarySTO2() -> fun nullarySTO1/0.
nullarySTO3() -> fun mylist_tests:nullarySTO1/0.

tupleSTO1() -> { mylist_tests, nullarySTO1 }.
tupleSTO2() -> { mylist_tests, nullarySTO2 }.
tupleSTO3() -> { mylist_tests, nullarySTO3 }.

tupleSTO4() -> ?_test( fun() -> ?assertEqual(6,mylist:sum([1,2,3])) end ).
tupleSTO5() -> ?_test( begin ?assertEqual(6,mylist:sum([1,2,3])) end ).
tupleSTO6() -> ?_test( nullarySTO2() ).
tupleSTO7() -> { 16, nullarySTO3() }.

generator_test_() ->
    {inparallel,2,
     [
      nullarySTO1(),
      nullarySTO2(),
      nullarySTO3(),
      tupleSTO1(),
      tupleSTO2(),
      tupleSTO3(),
      tupleSTO4(),
      tupleSTO5(),
      tupleSTO6(),
      tupleSTO7()
     ]}.

Running and compilation

zen:EUnitBasic4 zenon$ erlc -o ebin/ src/*.erl test/*.erl
zen:EUnitBasic4 zenon$ erl -pa ebin/
Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4]
[async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.5  (abort with ^G)
1> eunit:test(mylist,[verbose]).
======================== EUnit ========================
module 'mylist'
  module 'mylist_tests'
    mylist_tests: nullarySTO1...ok
    mylist_tests: nullarySTO2...ok
    mylist_tests: nullarySTO1...ok
    mylist_tests: nullarySTO1...ok
    mylist_tests: nullarySTO2...ok
    mylist_tests: nullarySTO3...ok
    mylist_tests:13: tupleSTO4...ok
    mylist_tests:14: tupleSTO5...ok
    mylist_tests:15: tupleSTO6...ok
    mylist_tests:16: nullarySTO1...ok
    [done in 0.015 s]
  [done in 0.015 s]
=======================================================
  All 10 tests passed.
ok
2>

Alas, the runtime is slightly higher, but all went well.

Removing the last question marks

If you where wondering whether all these control tuples play together, then good for you, since here is the answer: YES.

And to prove it, comes the monster module below.

-module(mylist_tests).
-include_lib("eunit/include/eunit.hrl").
-compile(export_all).

% Basics
nullarySTO1() -> fun() -> ?assertEqual(6,mylist:sum([1,2,3])) end.
nullarySTO2() -> fun nullarySTO1/0.
nullarySTO3() -> fun mylist_tests:nullarySTO1/0.
tupleSTO1() -> { mylist_tests, nullarySTO1 }.
tupleSTO2() -> { mylist_tests, nullarySTO2 }.
tupleSTO3() -> { mylist_tests, nullarySTO3 }.
tupleSTO4() -> ?_test( fun() -> ?assertEqual(6,mylist:sum([1,2,3])) end ).
tupleSTO5() -> ?_test( begin ?assertEqual(6,mylist:sum([1,2,3])) end ).
tupleSTO6() -> ?_test( nullarySTO2() ).
tupleSTO7() -> { 16, nullarySTO3() }.

% % Proof of concept
mixedA_test_() -> {spawn, nullarySTO1() }.
mixedB_test_() -> {spawn, [ {timeout, 1, nullarySTO1()} ,
                            {inparallel, [tupleSTO1(),
                                          tupleSTO4()]}]}.
mixedC_test_() -> {spawn, 'eunit@zen', nullarySTO1() }.
mixedD_test_() -> {spawn, 'eunit@zen', {inorder,
                                        [ nullarySTO1() ,
                                          tupleSTO1(),
                                          tupleSTO4()]}}.
mixedE_test_() ->
    [ nullarySTO1(),
      nullarySTO2(),
      {inparallel, 2, [ {spawn, 'eunit2@zen', tupleSTO1()},
                        {spawn, 'eunit2@zen', tupleSTO2()},
                        tupleSTO3() ] },
      {timeout, 1, tupleSTO4()} ].

As can be seen, the CT can be mixed wildly and are allowed wherever a TestSet is allowed, compiling and running in three shells.

zen:EUnitBasic4 zenon$ erl -pa ebin/ -sname eunit2@zen -setcookie eunit
Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4] 
[async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.5  (abort with ^G)
(eunit2@zen)1>

And the other one

zen:EUnitBasic4 zenon$ erl -pa ebin/ -sname eunit@zen -setcookie eunit
Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4] 
[async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.5  (abort with ^G)
(eunit@zen)1>

And the main shell doing all the fun

zen:EUnitBasic4 zenon$ erlc -o ebin/ src/*.erl test/*.erl
zen:EUnitBasic4 zenon$ erl -pa ebin/ -sname base@zen -setcookie eunit
Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4] 
[async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.5  (abort with ^G)
(base@zen)1> net_kernel:connect_node(eunit2@zen).
true
(base@zen)2> net_kernel:connect_node(eunit@zen).
true
(base@zen)3> nl(mylist_tests).
abcast
(base@zen)4> eunit:test(mylist,[verbose]).
======================== EUnit ========================
module 'mylist'
  module 'mylist_tests'
    mylist_tests: nullarySTO1...ok
    mylist_tests: nullarySTO1...ok
    mylist_tests: nullarySTO1...ok
    mylist_tests:13: tupleSTO4...ok
    mylist_tests: nullarySTO1...[0.003 s] ok
    mylist_tests: nullarySTO1...ok
    mylist_tests: nullarySTO1...ok
    mylist_tests:13: tupleSTO4...ok
    mylist_tests: nullarySTO1...ok
    mylist_tests: nullarySTO2...ok
    mylist_tests: nullarySTO1...ok
    mylist_tests: nullarySTO2...ok
    mylist_tests: nullarySTO3...ok
    mylist_tests:13: tupleSTO4...ok
    [done in 0.063 s]
  [done in 0.063 s]
=======================================================
  All 14 tests passed.
ok
(base@zen)5>

This concludes the Control of EUnit tests, next post will be about testing gen_fsms with EUnit.

EUnit – Spawn on Node – Missing Name Problem


This post is actually a prequel without the first one being posted yet (it’s in draft mode), anyway, if you try doing a spawn with node control, things seem to get a bit out of hand

Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4]
[async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.5  (abort with ^G)
(t@zen)1>
=ERROR REPORT==== 1-Sep-2010::09:06:20 ===
Error in process <0.56.0> on node 't@zen' with exit value: 
{badarg,[{erlang,atom_to_list,[[]]},
        {eunit_lib,fun_parent,1},
        {eunit_data,parse_function,1},
        {eunit_data,next,1},
        {eunit_data,iter_next,1},
        {eunit_proc,get_next_item,1},
        {eunit_proc,tests_inorder,3},{eunit_proc,with_timeout...

So, what is the cause for this error?

According to the error message, there is a bad usage with the erlang BIF atom_to_list, from the module eunit_lib, in the function fun_parent

{eunit_lib,fun_parent,1}

Now, if you go and read the source (eunit_lib),

%% This library is free software; you can redistribute it and/or modify
%% it under the terms of the GNU Lesser General Public License as
%% published by the Free Software Foundation; either version 2 of the
%% License, or (at your option) any later version.
%%
%% This library is distributed in the hope that it will be useful, but
%% WITHOUT ANY WARRANTY; without even the implied warranty of
%% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
%% Lesser General Public License for more details.
%%
%% You should have received a copy of the GNU Lesser General Public
%% License along with this library; if not, write to the Free Software
%% Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
%% USA
%%
%% $Id: eunit_lib.erl 339 2009-04-05 14:10:47Z rcarlsson $
%%
%% @copyright 2004-2007 Mickaël Rémond, Richard Carlsson
%% @author Mickaël Rémond <mickael.remond@process-one.net>
%%   [http://www.process-one.net/]
%% @author Richard Carlsson <richardc@it.uu.se>
%%   [http://user.it.uu.se/~richardc/]
%% @private
%% @see eunit
%% @doc Utility functions for eunit
%% ---------------------------------------------------------------------
%% Get the name of the containing function for a fun. (This is encoded
%% in the name of the generated function that implements the fun.)
fun_parent(F) ->
    {module, M} = erlang:fun_info(F, module),
    {name, N} = erlang:fun_info(F, name),
    case erlang:fun_info(F, type) of
        {type, external} ->
            {arity, A} = erlang:fun_info(F, arity),
            {M, N, A};
        {type, local} ->
            [$-|S] = atom_to_list(N),
            C1 = string:chr(S, $/),
            C2 = string:chr(S, $-),
            {M, list_to_atom(string:sub_string(S, 1, C1 - 1)),
             list_to_integer(string:sub_string(S, C1 + 1, C2 - 1))}
    end.

Where the second line fun_info(F, name) is the culprit. It seems as if the fun does not retain the name for some reason, but fortunately, after discussing this with my senior friend Nicolas, it turns out this is a problem caused by the module from where the fun is sent code not being loaded on the other node where the tests are to be evaluated.

Thus, the simple solution is to make sure the remote node has loaded the code from the tests a priori. This can be achieved with a module_info() or code:load_file/1.

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: