CarpetProlongateTest/test_o9 is failing on several machines

CarpetProlongateTest/test_o9 is failing on Gordon, Mike and Redshift. On Gordon, the differences are

   carpetprolongatetest::difference.d.asc: substantial differences
      significant differences on 1 (out of 206) lines
      maximum absolute difference in column 13 is 1.07288360595703e-06
      maximum absolute difference in column 14 is 6.98491930961609e-10
      maximum relative difference in column 13 is 1.12625729345722e-12
      maximum relative difference in column 14 is 3
      (insignificant differences on 19 lines)
   carpetprolongatetest::difference.x.asc: differences below tolerance on 26 lines
   carpetprolongatetest::difference.y.asc: differences below tolerance on 26 lines
   carpetprolongatetest::errornorm..asc: differences below tolerance on 1 lines
   carpetprolongatetest::scalar.d.asc: differences below tolerance on 13 lines
   carpetprolongatetest::scalar.x.asc: differences below tolerance on 6 lines
   carpetprolongatetest::scalar.y.asc: differences below tolerance on 4 lines

and on the other machines the results are similar. test.ccl contains

TEST test_o7
{
  ABSTOL 2.0e-11
}

TEST test_o9
{
  ABSTOL 5.0e-10
}

TEST test_o11
{
  ABSTOL 3.0e-8
}

Higher order prolongation probably leads to more amplification of roundoff differences, which is why the absolute tolerances listed here increase with prolongation order.

On Redshift, which uses -Ofast with gcc, the maximum absolute differences in columns 13 and 14 are just marginally above the tolerance of 5e-10:

      maximum absolute difference in column 13 is 9.31322574615479e-10
      maximum absolute difference in column 14 is 6.98491930961609e-10
      maximum relative difference in column 13 is 9.77653900570505e-16
      maximum relative difference in column 14 is 3

However on Gordon and Mike, the column 13 absolute difference is 1e-6, which presumably means the data is large, so the relative tolerance will come into play. The default relative tolerance is 1e-12, and the difference in column 13 is marginally above this.

Gordon:

      maximum absolute difference in column 13 is 1.07288360595703e-06
      maximum absolute difference in column 14 is 6.98491930961609e-10
      maximum relative difference in column 13 is 1.12625729345722e-12
      maximum relative difference in column 14 is 3

Mike:

      maximum absolute difference in column 13 is 1.07288360595703e-06
      maximum absolute difference in column 14 is 6.98491930961609e-10
      maximum relative difference in column 13 is 1.12625729345722e-12
      maximum relative difference in column 14 is 3

Should we increase both the relative and absolute tolerances for this test to 1e-11? I believe that would make the test pass on all three machines.

Keyword:

Comments (10)