- removed comment
CarpetIOHDF5 commit 69c73fb1 " CarpetIOHDF5: Add "active" attribute to grid function dataset describing which points are active" breaks 'IO::out_mode = "onefile"'
Issue #2117
resolved
Setting
IO::out_mode = "onefile"
in a parfile and running with more than 1 MPI ranks causes and out-of-range array access in Output.cc in line 913:
ostringstream buf; buf << (vdd.at(Carpet::map) ->local_boxes.at(mglevel) .at(refinementlevel) ----> .at(local_component) .active);
due to local_component
being -1. This is the backtrace:
#6 0x00007ffff3a03355 in std::__throw_out_of_range_fmt(char const*, ...) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #7 0x00005555578f17ee in std::vector<dh::local_dboxes, std::allocator<dh::local_dboxes> >::_M_range_check (__n=<optimized out>, this=<optimized out>) at /usr/include/c++/7/bits/stl_vector.h:825 #8 std::vector<dh::local_dboxes, std::allocator<dh::local_dboxes> >::at (__n=<optimized out>, this=<optimized out>) at /usr/include/c++/7/bits/stl_vector.h:846 #9 CarpetIOHDF5::AddAttributes (cctkGH=cctkGH@entry=0x55555f829ab0, fullname=fullname@entry=0x5555634efb10 "TMUNUBASE::eTxx", vdim=3, refinementlevel=<optimized out>, refinementlevel@entry=0, request=request@entry=0x5555635801b0, bbox=..., dataset=360287970189640065, is_index=false) at /data/rhaas/postdoc/gr/cactus/ET_vanilla/configs/sim/build/CarpetIOHDF5/Output.cc:913 #10 0x00005555578f2a45 in CarpetIOHDF5::WriteVarChunkedSequential (cctkGH=cctkGH@entry=0x55555f829ab0, outfile=outfile@entry=72057594037928056, io_bytes=@0x7fffffffb960: 62212, request=0x5555635801b0, called_from_checkpoint=called_from_checkpoint@entry=true, indexfile=indexfile@entry=-1) at /data/rhaas/postdoc/gr/cactus/ET_vanilla/configs/sim/build/CarpetIOHDF5/Output.cc:513 #11 0x00005555578d9bdf in CarpetIOHDF5::Checkpoint (cctkGH=0x55555f829ab0, called_from=0) at /data/rhaas/postdoc/gr/cactus/ET_vanilla/configs/sim/build/CarpetIOHDF5/CarpetIOHDF5.cc:1218
To reproduce:
wget https://einsteintoolkit.org/gallery/ns/tov.par OMP_NUM_THREADS=1 mpirun -n 2 exe/cactus_sim tov.par
Keyword:
Comments (10)
-
reporter -
reporter - changed milestone to ET_2018_02
- removed comment
-
reporter - changed title to CarpetIOHDF5 commit 69c73fb1 " CarpetIOHDF5: Add "active" attribute to grid function dataset describing which points are active" breaks 'IO::out_mode = "onefile"'
- removed comment
-
reporter Pull request is here: https://bitbucket.org/eschnett/carpet/pull-requests/27/rhaas-onefile-active/diff
The reproduce need "old" version of parfile like so:
wget https://bitbucket.org/einsteintoolkit/www/raw/433c62e91b7bf638e2f0df104774af5a22b37d33/gallery/ns/tov.par OMP_NUM_THREADS=1 mpirun -n 2 exe/cactus_sim tov.par
-
reporter - edited description
- changed status to open
-
reporter Unless objected I will apply this patch after 2019-04-29.
-
reporter - changed status to resolved
Applied as git hash a2e72a97 “CarpetIOHDF5: send active string to IO rank” or carpet.
-
reporter - changed status to closed
-
reporter - changed status to open
This causes a out-of-range error when checkpointing grid scalar and grid arrays b/c I was not careful enough checking under which circumstances the active region string was added before.
-
reporter - changed status to resolved
- Log in to comment