Prolongation operators are only parallelized in one direction

Issue #1332 new
Frank Löffler created an issue

Currently, Carpet's prolongation operators are only parallelized in one direction (the largest). This happens before calling the actual operator in call_operator() (CarpetLib). With increasing core(thread)-counts this is a problem. There are hardly enough points in any single direction to make this efficient.

Wouldn't it be much better (in terms of openmp efficiency) to let the operators handle openmp-parallelization themselves? I currently see quite a large overhead for real-world parfiles with 32 threads just because of this.

Keyword:

Comments (1)

  1. Erik Schnetter
    • removed comment

    Instead of moving parallelization to a lower level, it would probably be more efficient to move it even higher up. For example, given a single refined region on a single MPI process, its six boundaries should be prolongated in parallel. Doing this efficiently and dynamically requires an infrastructure that is more flexible than a typical OpenMP implementation.

    In the short term, using LoopControl (where applicable) in the prolongation operators' loops is probably the best, as you suggest.

  2. Log in to comment