Compilation on comet failed to due to out of memory errors

Issue #2204 closed
Roland Haas created an issue

Compiling on Comet's login node failed for me when compiling

arrangements/Carpet/CarpetLib/src/restrict_3d_cc_o5_rf2.cc

due to an out of memory error. ulimit -a indicates that there is a limit of 4GB per user

rhaas@comet-ln2:~$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 257194
max locked memory       (kbytes, -l) 524288
max memory size         (kbytes, -m) 4194304
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) unlimited
cpu time               (seconds, -t) 10800
max user processes              (-u) 1024
virtual memory          (kbytes, -v) 4194304
file locks                      (-x) unlimited

which can be avoided by going to a compute node

srun --partition=debug --pty --nodes=1 --ntasks-per-node=24 -t 00:30:00 --wait=0 --export=ALL /bin/bash

and compiling via

simfactory/bin/sim build --machine comet

This is (hopefully) a temporary measure on comet.

Keyword: comet

Comments (4)

  1. Roland Haas reporter
    • removed comment

    This does likely not affect the release code but was observed using the code in the rhaas/openmp-tasks branch.

    I am trying to find out the amount of memory required to compile to report it back to SDSC support who may be willing to raise the vmem limit.

  2. Roland Haas reporter
    • changed status to resolved
    • removed comment

    The admins have bumped the memory limit to 8GB which is enough to compile again (it needs almost that much, namely 7929856 kiByte fail).

    For reference this is in XSEDE ticket "#96792 : XUP: vmem limit on SDSC comet login nodes".

  3. Log in to comment