Test output changes every time some tests are run

Each time a new run of the test suites is performed, the test output of IDAxiOddBrillBH and QuasiLocalMeasures changes a little. These changes are below the tolerances set in the test.ccl files, so the tests pass, but there should be no change at all when the tests are run on the same machine.

An example of the difference from one test run to the next is shown at http://git.barrywardell.net/EinsteinToolkitTestResults.git/blobdiff/95fe088e38298cac66df15223daaef5836e42356..d78c6b784027cf218ee51644004f50fe7cc3e1e5:/test_1/IDAxiOddBrillBH/E2xeon_test_axioddbh/gxx.xl.

Another example, from QuasiLocalMeasures, is at http://git.barrywardell.net/EinsteinToolkitTestResults.git/blobdiff/95fe088e38298cac66df15223daaef5836e42356..d78c6b784027cf218ee51644004f50fe7cc3e1e5:/test_2/QuasiLocalMeasures/qlm-ks-shifted/admbase::metric.average.asc.

The only thorns to exhibit this behaviour are IDAxiOddBrillBH and QuasiLocalMeasures. All other test data remains unchanged from one test run to the next. Note that the two test runs will be performed with different executables, since a new configuration is created for each run. I think I remember testing that simply re-running the same executable resulted in the same test output.

Is it possible that there are a small number of points which access uninitialised memory which changes on each run for these thorns?

Keyword: testsuites
Keyword: QuasiLocalMeasures
Keyword: IDAxiOddBrillBH

Comments (6)