Module Osnap

OCaml random snapshot testing

The library takes inspiration from ppx_expect, we add random generators to store random application of functions. A snapshot will be the random applications on a function, the snapshot will be latter use to look for diffs between old and new version of a function.

• Spec is used to give a function specification. Spec provides combinator to describe function generators and printer.

Example:

let spec_add = Osnap.Spec.(int ^> int ^>> string_of_int)

• Test is used to describe a single test, that is, where the snapshot will be stored and the function under test with its specification.

Example:

let test =
  let spec = Osnap.Spec.(int ^> int ^>> int) in
  Osnap.Test.make ~path:".osnap/add" ~spec (+)

• Runner is used to run tests.

• see https://github.com/janestreet/ppx_expect

Specifying function specification

Spec takes inspiration from OCaml's property-based testing library Monolith. The idea is that the programmer describes the function specification through the set of generators from QCheck and a printer.

Examples:

• Addition specification:

let add = (+)
let spec_add = Spec.(int ^> int ^>> string_of_int)

• List sum specification:

let sum = List.fold_left ( + )
let spec_sum = list int ^>> string_of_int

Test creation

Runner

Runner has tree mode:

• Interactive:

Interactive mode provides an interactive runner, displaying differences between old and new snapshots. Promoting new version is proposed at every diff, exit when new diff is not promoted, considered as an error.

Example:

let test =
  let spec = OSnap.Spec.(int ^> int ^>> string_of_int) in
  Osnap.Test.make ~count:1 ~path:".osnap/add" ~spec (+)

(* .osnap/add:

   f 5 6 = 11 *)

(* Then, the function under test is updated *)
let test =
  let spec = OSnap.Spec.(int ^> int ^>> int) in
  Osnap.Test.make ~count:1 ~path:".osnap/add" ~spec (fun _ _ -> 0)

let _ = Runner.(run_tests ~mode:Interactive [test])

(* .osnap/add
   -| f 5 6 = 11;
   +| f 5 6 = 0

   Promote changes ? [Y\n] *)

• Promote:

Promote mode provides an automatic promotion of diffs.

Example:

let test =
  let spec = OSnap.Spec.(int ^> int ^>> int) in
  Osnap.Test.make ~count:1 ~path:".osnap/add" ~spec (+)

(* .osnap/add:

   f 5 6 = 11; *)

(* Then, the function under test is updated *)
let test =
  let spec = OSnap.Spec.(int ^> int ^>> int) in
  Osnap.Test.make ~count:1 ~path:".osnap/add" ~spec (fun _ _ -> 0)

let _ = Runner.(run_tests ~mode:Promote [test])

(* .osnap/add has been promoted
   -| f 5 6 = 11;
   +| f 5 6 = 0
*)

• Error:

Error mode will raise errors on diffs.

let test =
  let spec = OSnap.Spec.(int ^> int ^>> int) in
  Osnap.Test.make ~count:1 ~path:".osnap/add" ~spec (+)

(* .osnap/add:

   f 5 6 = 11; *)

(* Then, the function under test is updated *)
let test =
  let spec = OSnap.Spec.(int ^> int ^>> int) in
  Osnap.Test.make ~count:1 ~path:".osnap/add" ~spec (fun _ _ -> 0)

let _ = Runner.(run_tests ~mode:Promote [test])

(* .osnap/add has differences, exit
   -| f 5 6 = 11;
   +| f 5 6 = 0
*)