Last story to play on the workernet job layer,
“I want to be able to delete a job once it’s done, through any node.”
To design this – the test is written first (as usual)
done_job_deleted() -> wn_resource_layer:register(#wn_resource{name = "Laptop", type = [{perl,1}], resides = node() }), Path = create_file_at(?NODE_ROOT), File1 = #wn_file{id = "File1",file = Path,resides = node()}, Job1 = #wn_job{id = "JobId", files = [File1], resources = [perl], commands = ["perl -e 'print(\"HelloWorld\n\")'"], timeout = 1000 }, ok = wn_job_layer:register(Job1), ok = wn_job_layer:stream(user,"JobId"), timer:sleep(500), ok = wn_job_layer:delete("JobId"), ?assertMatch([#wn_file{id="File1"}],wn_file_layer:list_files()), ?assertEqual([],wn_job_layer:list_all_jobs()), ok.
As usual, this will not even compile first as stuff is missing and dialyzer should get a headache about it. Now, everything is there except the wn_job_layer:delete/1 function.
Therefore, the first implementation goes into wn_job_layer.erl
-spec(delete(string()) -> ok | {error,term()}). delete(Id) -> gen_server:call(?MODULE,{delete,Id}).
next the internal handle_call/3 clause for handling the delete request
handle_call({delete,Id},_,State) -> Result = try_delete(Id,State), {reply,Result,State};
this raises the need to implement the internal try_delete/2 function
try_delete(Id,State) -> case ets:lookup(State#state.jobs,Id) of [{Id,JobKeeperPid,_}] -> case wn_job_keeper:get_stored_result(JobKeeperPid) of {ok,Result} -> ok = wn_job_keeper:delete(JobKeeperPid), ets:delete(State#state.jobs,Id), wn_file_layer:delete_file(node(),Result); X -> X end; [] -> {error,no_such_job} end.
That was all the code needed inside wn_job_layer.erl . However, the last internal function brought up the need for a new function; wn_job_keeper:delete/1 so the function is to be implemented next!
In wn_job_keeper.erl
-spec(delete(pid()) -> ok | {error,term()}). delete(Pid) -> gen_fsm:sync_send_all_state_event(Pid,delete).
try_delete(Id,State) -> case ets:lookup(State#state.jobs,Id) of [{Id,JobKeeperPid,_}] -> case wn_job_keeper:get_stored_result(JobKeeperPid) of {ok,Result} -> ok = wn_job_keeper:delete(JobKeeperPid), ets:delete(State#state.jobs,Id), wn_file_layer:delete_file(node(),Result); X -> X end; [] -> {error,no_such_job} end.
With a handle_call clause inside it for the delete request
handle_sync_event(delete,_,done,State) -> {stop,normal,ok,State}; handle_sync_event(delete,_,X,State) -> {reply,{error,not_done},X,State};
That would be all! Since so much had already been written in prior tests, this one turned out to be quite a breeze. Of course this passes both compilation, dialyzation and testing. The judge ‘make full’ turns it’s mighty eye on this
zen:worker_net-0.1 zenon$ make full erlc -pa . -o ebin/ src/*.erl test/*.erl erl -pa ebin/ -eval 'eunit:test(wn_resource_layer,[verbose]), init:stop().' Erlang R14B (erts-5.8.1) [source] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.1 (abort with ^G) 1> ======================== EUnit ======================== module 'wn_resource_layer' module 'wn_resource_layer_tests' wn_resource_layer_tests: local_resource_test_ (Can register resources locally)...[0.001 s] ok wn_resource_layer_tests: register_distributed (Can Register Distributed)...[0.006 s] ok wn_resource_layer_tests: register_restart_register (Can Register, Restart and Register)...[0.015 s] ok wn_resource_layer_tests: register_deregister (Can Register, Deregister and Register)...[0.013 s] ok [done in 6.324 s] [done in 6.325 s] ======================================================= All 4 tests passed. erl -pa ebin/ -eval 'eunit:test(wn_file_layer,[verbose]), init:stop().' Erlang R14B (erts-5.8.1) [source] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.1 (abort with ^G) 1> ======================== EUnit ======================== module 'wn_file_layer' module 'wn_file_layer_tests' wn_file_layer_tests: file_layer_local_test_ (Can store file locally)...[0.326 s] ok wn_file_layer_tests: file_layer_local_test_ (Can retrieve files locally)...[0.003 s] ok wn_file_layer_tests: file_layer_local_test_ (Can delete files locally)...[0.002 s] ok wn_file_layer_tests: can_store_distributed (Can store file distributed)...[0.024 s] ok wn_file_layer_tests: can_retrieve_distributed (Can retrieve file distributed)...[0.018 s] ok wn_file_layer_tests: can_delete_distributed (Can delete file distributed)...[0.020 s] ok wn_file_layer_tests: must_retain (Must retain information between node kill and node restart)...[0.391 s] ok [done in 2.348 s] [done in 2.348 s] ======================================================= All 7 tests passed. erl -pa ebin/ -eval 'eunit:test(wn_job_layer,[verbose]), init:stop().' Erlang R14B (erts-5.8.1) [source] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.1 (abort with ^G) 1> ======================== EUnit ======================== module 'wn_job_layer' module 'wn_job_layer_tests' wn_job_layer_tests: local_test_ (Can register locally)...[0.005 s] ok wn_job_layer_tests: local_test_ (Executed locally)...{1299,338969,204767} : file_fetching_done {1299,338969,204771} : executing_commands {1299,338969,218209} : {executing,"more EUnitFile"} {1299,338969,225651} : "1,2,3" {1299,338969,226358} : no_more_commands {1299,338969,226362} : building_result_tgz {1299,338969,226462} : done [1.018 s] ok wn_job_layer_tests: local_test_ (Executed queue)...{1299,338970,229138} : file_fetching_done {1299,338970,229141} : executing_commands {1299,338970,229539} : {executing,"file EunitFile"} {1299,338970,239052} : "EunitFile: ASCII text" {1299,338970,239301} : no_more_commands {1299,338970,239304} : building_result_tgz {1299,338970,239476} : done {1299,338970,243238} : file_fetching_done {1299,338970,243250} : executing_commands {1299,338970,243360} : {executing,"cat EUnitFile"} {1299,338970,250730} : "1,2,3" {1299,338970,250782} : no_more_commands {1299,338970,250785} : building_result_tgz {1299,338970,250856} : done [1.105 s] ok wn_job_layer_tests: local_test_ (Queueus on resource type amount)...[0.001 s] ok wn_job_layer_tests: local_test_ (Canceled in queue)...[0.001 s] ok wn_job_layer_tests: local_test_ (Done Job Stored in file layer)...[0.606 s] ok wn_job_layer_tests: local_test_ (Done Job canceled)...{1299,338971,966369} : file_fetching_done {1299,338971,966377} : executing_commands {1299,338971,966651} : {executing,"perl -e 'print(\"HelloWorld\n\")'"} {1299,338972,3168} : "HelloWorld" {1299,338972,3405} : no_more_commands {1299,338972,3408} : building_result_tgz {1299,338972,3508} : done [0.504 s] ok [done in 3.305 s] [done in 3.305 s] ======================================================= All 7 tests passed. dialyzer src/*.erl test/*.erl Checking whether the PLT /Users/zenon/.dialyzer_plt is up-to-date... yes Proceeding with analysis... Unknown functions: eunit:test/1 done in 0m6.92s done (passed successfully) zen:worker_net-0.1 zenon$
The end
This concludes the TDD hands on project for the WorkerNet – the source can now be found through my github repository for this.
git://github.com/Gianfrancoalongi/WorkerNet.git
itamoeba
/ October 11, 2011Just wanted to say thanks for this. I’ve started converting an application written in Python / PYRO to Erlang and I’m sure your blog will speed up the process, it’s great to see a worked TDD based example as a reference point.
All the best
Steve