A “dev” image of the ansible-runner, with the newly created test playbooks needs to be built, pushed to the desired repo & updated in the litmus job spec (litmus book) before deploying/testing it. However, users who wish to refrain from pushing development images into public/private repositories can instead download this “dev” image on the cluster nodes with imagePullPolicy of the custom ansible-runner set to “Never”.
Also, it is necessary to have the environment (platform, application configured with PVCs of desired storage classes) for which the test is built, in place, before trialing the litmus book.
It is recommended to perform negative testing of the litmusbook too, in order to gauge how test failures at various points impacts the cluster. The test playbooks (and by extension, the utils) make use of ansible’s version of the try-catch construct, i.e., the block-rescue-always flow to define the error-handling logic.