1. Running Acceptance Tests
    1. Installing TextTest
    2. Running the Acceptance Tests Manually
    3. Updating the 'Comparison' results

Running Acceptance Tests

Acceptance tests are tests of the compiled Chaste executable, where we check that for a given input parameter file, it generates all of the same output files as it used to.

The results that are to be compared with are stored in the Chaste trunk under 

apps/texttest/chaste

for nightly tests, and 

apps/texttest/weekly

for weekly tests.

Installing TextTest

TextTest is both the testing framework, and a little gui to help visualize results, and replace reference files when there are intended differences.

Not all the build machines have this installed. userpc58 has at the moment, see InstallTextTest if you want to set it up elsewhere.

Running the Acceptance Tests Manually

  1. Do an svn up to the required version, and build a fresh executable (i.e. scons cl=1 b=IntelProduction exe=1 apps/src).
  2. Check that the environment variable TEXTTEST_HOME is set to either (the full path to) apps/texttest/chaste or apps/texttest/weekly as appropriate (see below).
  3. Navigate to the folder that contains the Chaste executable (e.g. cd ~/eclipse/workspace/Chaste/apps/src)
  4. Run TextTest from here (on userpc58 this is currently at ~/texttest-3.19/source/bin/texttest.py)
  5. Select all the available tests and press Run.
    • They should now go green or red one-by-one as the tests are run.

Updating the 'Comparison' results

catalogue.chaste is created for each individual test, and this lists the directory structure and files that are created when the test runs. If you want any of this to change, the reference catalogue.chaste file will need updating for that test.

By clicking on each 'red' result, the differences will be displayed in the panel on the bottom right.

  • If you decide these are intended, then you can update the reference results in the repository by right-clicking the filename and selecting 'Save' (then commit as usual from command line or eclipse).
  • If these results are not intended, then something is broken and the code should be fixed!

Note:

  • Texttest keeps a list of files to compare with reference versions for each acceptance test in the file apps/texttest/chaste/config.chaste. If any of the files that you want to compare are found in a different place in the output, or you want to compare new ones (or remove some), you'll need to edit this file too.