compiling Carpet fails with intel 17 and optimisation

Issue #2021 on hold
Wolfgang Kastaun created an issue

Compiling carpet with intel 17.0.1 and -O2 or -O3 triggers an internal compiler error ("internal error: 0_76"). It only affects the file bbox.cc. Happened on two different clusters. ET version is Payne.

Keyword: backport

Comments (26)

  1. Zach Etienne
    • removed comment

    I suggest re-opening this ticket, as it is more serious than previously thought: Intel compiler version 18.0.1 20171018 produces the same failure, and the fix only applies if Intel 17.0.0 is used. Should be an easy fix to include Intel 18 as well. I would be surprised if all Intel compilers from 17.0.0 through 18.0.1 are not affected by this.

  2. Ian Hinder
    • removed comment

    We never reported it to Intel, so we shouldn't be too surprised. "anonymous" above claimed that reporting the problem to Intel required us to reproduce the problem with a small code. Is this true? These days, that seems like an unrealistic requirement. It should be possible to just give clear simple instructions for how to reproduce the problem (e.g. run GetComponents with a minimal thornlist, and compile). Does anybody know how to report the bug to Intel? 60 seconds with Google didn't lead to a clear answer; it seems you might have to have a support contract and contact them through that. I don't know how to do that here, and it would be much easier to just submit a form on a website.

  3. Roland Haas
    • removed comment

    Maybe anonymous was me. The AEI has a support contract with Intel :-P

    NCSA likely has through Cray and I think that we have at least some vulnerable compiler versions on BW.

    If it really is a compile time ICE then a "big" code should also be ok. Plus one can likely strip out everything but Carpet, make a tarball of the pre-configured source code and hand that one to Intel.

  4. Roland Haas
    • changed status to open
    • removed comment

    This also affects Intel 18. I suggest to for now test for all intel compilers >= 17.00 and add the upper bound once we find a compiler that works again. I would also backport this to the release branch.

    Namely use:

    // intel-17.0.0 fails with internal error 0_76 if vectorization is not disabled
    #if __INTEL_COMPILER >= 1700
    #pragma GCC optimization_level 1
    #endif
    

    and would like to apply this very soon as stampede2's intel 18 compiler requires this and the next round of xrac allocation will come online on Jan 1st 2018.

  5. Roland Haas
    • removed comment

    Any objections to the proposed patch? If none are voiced I will push the change on or after Jan 19 2018.

  6. Ian Hinder
    • removed comment

    The comment in master, and in the patch proposed in comment:14, state that //vectorisation// needs to be disabled, and then go on to reduce the //optimisation// level to 1. Is this just a typo, and it should be "optimisation" in each case?

  7. Roland Haas
    • removed comment

    I unfortunately do not know exactly how vectorization affects bbox.cc, though it could: Integer operations vectorize, not just floating point operations. Low optimization levels will not auto-vectorize code, which is independent of Cactus'/Carpet's/Vectors' vectorization which uses the vector builtins. So my best guess is that "vectorization" here means the auto-vectorization that the compiler does on higher levels of optimization (the size of the ivects etc in a bbox are known at compile time so they seem like good candidates for vectorization).

    The original patch and comment is David Radice's.

  8. David Radice
    • removed comment

    It was meant to be "optimization" not "vectorization". It might be that the issue is indeed related to vectorization at the end.... I don't know.

  9. Roland Haas
    • removed comment

    Ok, I'll fix the comment and push the commit later than. Nothing voiced so far sounds like an "objection" to me.

  10. Roland Haas
    • changed status to resolved
    • removed comment

    Applied as git commit 1378c7df "CarpetLib: fix comment about optimization in bbox.cc" and " db84c98c - CarpetLib: use O1 in bbox.cc for all Intel >17.0 compilers" of Carpet for the master branch, applied as git commit a15fa600 "CarpetLib: fix comment about optimization in bbox.cc" and 703ac0d3 "CarpetLib: use O1 in bbox.cc for all Intel >17.0 compilers" for the ET_2017_06 branch of carpet.

  11. Roland Haas
    • changed status to open

    This should be revisited to determine an Intel compiler version that can handle optimization levels >O1 in the file (assuming there is such a one starting from Intel 17 or higher).

  12. Log in to comment