FitBenchmarking Tests

The tests for FitBenchmarking require pytest>=3.6. We have split the tests into three categories:

  • default: denotes tests involving pip installable software packages,

  • all: in addition to default, also runs tests on external packages, with the exception of matlab.

  • matlab: Runs tests for matlab fitting software. Please note that these tests can currently only be run locally through pytest.

Unit tests

Each module directory in FitBenchmarking (e.g. controllers) contains a test folder which has the unit tests for that module. One can run the tests for a module by:

pytest fitbenchmarking/<MODULE_DIR> --test-type <TEST_TYPE>

where <TEST_TYPE> is either default or all. If --test-type argument is not given the default is all

System tests

System tests can be found in the systests directory in FitBenchmarking. As with the unit tests, these can be run via:

pytest fitbenchmarking/systests --test-type <TEST_TYPE>

Warning

The files in the expected results subdirectory of the systests directory are generated to check consistency in our automated tests via GitHub Actions. They might not pass on your local operating system due to, for example, different software package versions being installed.

GitHub Actions tests

The scripts that are used for our automated tests via GitHub Actions are located in the ci folder. These give an example of how to run both the unit and system tests within FitBenchmarking.