Datest

Top

Datest is a data oriented unit testing framework for clojure. This page will serve as its documentation.

Distinguishing features

  1. Designed to be used from REPL
  2. Test suites and test results are clojure values that you can manipulate in the REPL
  3. Cleans stack traces so that they only contain frames from your module
  4. Works with both clojure and clojurescript

Defining a test suite

testing macro is used to define the test suite tree. The leaf nodes in this tree will contain the testing logic and the inner nodes will be used to group multiple test suites. An invocation of the testing macro returns a test suite as a value:

(def suite1 (testing :test [] {:result :OK :message "A simple test suite"}))

The first argument to the macro is the name of the test suite (:test in this case). How the rest of the arguments are interpreted depends on whether the test suite is a leaf node or an inner node.

Leaf Nodes

Leaf nodes must have a vector after the name. The above example is a leaf node. As you can see, it looks like a function call. You are supposed to put all the testing logic inside these "functions". It must return a clojure map with :result as a key. It can throw an exception as well. The test result will have the stack trace with unnecessary stack frames filtered out (clojure.core, clojure.lang, etc).
The empty vector isn't there just for marking the leaf nodes. I'll describe how to use them later.

Inner Nodes

Inner nodes must contain nested testing macro calls following the name:

(def suite2 (testing :a (testing :b [] {:result :OK}) (testing :c [] {:result :ERR :message "failing test"}) (testing :d [] (throw (new Error)) {:result :OK :message "ignored"}) ))

Manipulating test suites

You can combine multiple test suites into a single suite using combine-tests function:

(def combined (combine-tests [suite1 suite2]))

You can define parts of test suite in different modules and combine them into one big suite in the main testing module.

You can also pull out parts of the test suite because it's a regular clojure map:

(def suite3 (get combined :test)) (def suite4 (get-in combined [:a :c]))

This can be used to run a subset of the full test suite.

Running test suites

run-test function can be used to execute a test suite. It returns a result tree whose structure matches that of the given test suite.

(run-test combined) => {:a {:b {:result :OK, :state {}}, :c {:result :ERR, :message "failing test", :state {}}, :d {:result :EXCEPTION, :exception #error{}, :state {}}}, :test {:result :OK, :message "A simple test suite", :state {}}}(run-test (get-in combined [:a :c])) => {:result :ERR, :message "failing test", :state {}}

Stack traces cleanup

The framework uses first word of the namespace as filter for stack traces by default. For example, for tests defined in datest.core, run-test will filter out all stack frames that don't have datest in them.

If this default behavior doesn't work for you, you can rebind datest.core/EXCEPTION_FILTER to a regex of your choice and it will be used for cleaning up exceptions.

(binding [datest.core/EXCEPTION_FILTER #"clojure\.lang"] (testing :my-test [])

Test state

Consider the following test unit:

(testing :suite [] (let [a (get_a) b (get_b a)] (if (valid_value? b) {:result :OK} {:result :ERR :cause "Invalid result" :a a :b b})))

What if get-b throws an exception. You'll get the following result when you execute this test:

{:result :EXCEPTION :exception #error}

The exception info will contain the stack trace, but not the value of local variable a that caused get-b to throw an exception.

Datest has the concept of state variables that can help with this. The state variables for a test unit are declared in the argument vector of the "function" for that test unit:

(testing :suite [val_a] (let [a (get_a) _ (reset! val_a a) b (get_b a)] (if (valid_value? b) {:result :OK} {:result :ERR :cause "Invalid result" :a a :b b})))

The execution context of the test unit will have an atom for each state variable that is declared. You can store intermediate state of the test unit inside these atoms, and this state will be available in the test result:

{:suite {:result :EXCEPTION, :exception #error, :state {val_a 1}}}

Utilities

get-failed : Return test results with (not= status :OK):

(get-failed (run-test combined)) {:a {:c {:result :ERR, :message "failing test"}, :d {:result :EXCEPTION, :exception #error}}}

flatten-result/treefy-result : Can be used to iterate over test results:

(->> (run-test combined) flatten-result (filter my-result-filter) treefy-result)(def failed (flatten-result (get-failed (run-test combined)))) ; number of failed results (count failed) ; first failed test: (first failed) ; stats about test results (frequencies (map :result (flatten-result (run-test combined)))) {:OK 2, :ERR 1, :EXCEPTION 1}

return-comparison : Compare two objects and return a result with their comparison:

(run-test (testing :test [] (let [expected {:a {:b 1 :c 3}} actual {:a {:b 1 :c 4}}] (return-comparison expected actual)))) => {:test {:main {:result :ERR, :expected {:a {:b 1, :c 3}}, :actual {:a {:b 1, :c 4}}, :diff ({:a {:c 3}} {:a {:c 4}} {:a {:b 1}}), :state {}}}}

:diff sequence is the result of clojure.data/diff. First element is data that is only in expected response, second element is data only in actual response, third element is common data.