ML_BSSN: other_timelevels Parameter Not Respected

Issue #1932 open
Zach Etienne created an issue

At the Jun 27, 2016 ET telecon, we found the following bug in McLachlan/ML_BSSN:

Inside the ET_2016_05 ML_BSSN/schedule.ccl, you'll notice the following lines: STORAGE: ML_Ham[timelevels] STORAGE: ML_mom[timelevels] STORAGE: ML_cons_detg[timelevels] STORAGE: ML_cons_Gamma[timelevels] STORAGE: ML_cons_traceA[timelevels]

in all of these lines, "timelevels" should be replaced by "other_timelevels".

This should result in significantly increased memory usage in the latest ML_BSSN (possibly at the 10-20% level), particularly in vacuum evolutions.

Related to this problem, I noticed that in a previous version of McLachlan (2015_05, where the above issue does not exist), all constraints are being stored in checkpoint files, despite having only one timelevel. ML_BSSN_Helper is supposed to overwrite the ML_BSSN/interface.ccl request to set the Checkpoint="no" tag.

Keyword:

Comments (10)

  1. Frank Löffler
    • removed comment

    #1960 was marked as duplicate of this bug. It looks like a reasonable proposal to me to replace 'timelevels' by 'other_timelevels' in these cases. Concerning checkpointing: if other_timelevels is indeed 1, there is no need to store them. Would it be conceivable that someone might actually want to store them (and other_timelevels is set to a high enough value)?

  2. Roland Haas
    • removed comment

    Currently, when other_timelevels is 1 the code in ML_BSSN_Helper will disable checkpointing, prolongation and persistency. Though looking at the code at least the prolongation part is stated to be handled by Kranc itself (Erik and Ian would know details it seems).

  3. Erik Schnetter
    • removed comment

    Kranc should use other_timelevels for some grid functions, but it doesn't. This broke at some point, and should be corrected. Kranc's logic for which parameter to use for which grid functions is located at a rather low-level place, and I don't know how to pass the respective information there.

  4. Erik Schnetter
    • removed comment

    This looks good; we should merge this. There might be fall-out in terms of parameter files that don't set other_timelevels correctly, so we should merge this now, not before the release.

  5. Roland Haas
    • removed comment

    Replying to [comment:2 rhaas]:

    Currently, when other_timelevels is 1 the code in ML_BSSN_Helper will disable checkpointing, prolongation and persistency. Though looking at the code at least the prolongation part is stated to be handled by Kranc itself (Erik and Ian would know details it seems).

    I just tried this with the attached qc0_mclachlan.par file and Kranc (at least in the release branch where the comment appears) does not seem to turn off prolongation:

    ERROR from host 127.0.0.1 process 0
      while executing schedule bin (none), routine (no thorn)::(no routine)
      in thorn CarpetLib, file configs/sim/build/CarpetLib/ggf.cc:334:
      -> The variable "ML_BSSN::cS" has only 1 active time levels, which is not enough for boundary prolongation of order 2
    

    for this to happen one has to by hand modify schedule.ccl (for the release branch) and use "other_timelevels" for ML_Ham etc.

  6. Erik Schnetter
    • removed comment

    Apparently Kranc does not yet handle this. We need to remove the respective "#if 0" in "mclachlan/m/prototype/ML_BSSN_Helper/src/SetGroupTags.c" to re-activate the mechanism there, then regenerate McLachlan.

  7. Roland Haas
    • removed comment

    Hmm it seems https://bitbucket.org/einsteintoolkit/mclachlan/pull-requests/3/ml_bssn-use-other_timelevels-parameter-for/diff has already been applied (by me? I don't think I intended to).

    Using it and the reactivated code in ML_BSSN_Helper the testsuite fails in four tests: * magnetizedTOV (from IllinoisGRMHD) * ml-gw1d-small (from PeriodicCarpet) * ml-gw1d-small-amr (from PeriodicCarpet) * testperiodicinterp (from PeriodicCarpet)

    The CarpetPeriodic ones fail with a SEGFAULT while the IllinoisGRMHD one fails reporting missing time levels for time prolongation. I attach the log files of the 4 tests.

  8. Log in to comment