1. Einstein Toolkit Avatar Einstein Toolkit
  2. Einstein Toolkit
  3. tickets
  4. Issues

Deal with Fortran module files in standard C include paths (where they don't belong, but might have been placed by the system)

Issue #1769 closed
Bruno Mundim created an issue

Note: the description changed, as the ticket evolved. The original description can be found below.

New description:

Only as short summary: Some systems install Fortran module files into standard paths for C include files. In order to find them, these paths need to be given to the Fortran compiler using the -I flag. They are not searched by default (by some compilers), since these paths, while C standard include paths, are not Fortran standard include paths. On the other hand, adding this -I option to all compilations in Cactus would interfere with C/C++, as these directories should not be added in that case.

One possible solution, not implemented right now, might be to use the relatively new INC_DIRS_F variable in the detect script.

Original description:

Several Linux distributions have already started to move their libraries to a new directory structure that reflects the target architecture and allows the installation of packages from multiple architectures in the same system. The following links provide more details of this change:

https://wiki.debian.org/Multiarch/TheCaseForMultiarch
https://wiki.ubuntu.com/MultiarchSpec

This change potentially affects several of external library scripts shipped with ET. For example the HDF5/src/detect.sh script is not able to detect the hdf5 libraries installed in the directory /usr/lib/x86_64-linux-gnu for Ubuntu 14.04.2 LTS.

I have attached a simple patch to remedy this issue and open a discussion on the best way to proceed here. This patch does depend on the availability of gcc on the system. It currently works fine for me, but we might need a better solution for systems without gcc.

Keyword: ExternalLibraries
Keyword: HDF5
Keyword: Multiarch

Comments (41)

  2. Roland Haas
    • removed comment

    The idea looks good. Nice to learn how to generate those machine strings. I suspect that MACHINE needs to also be taken into account in the actual search phase earlier on in detect.sh when libdir is set.

  3. Bruno Mundim reporter
    • removed comment

    Doing more tests with the modified patch I submitted to the HDF5 thorn I think it would be safer to revert it for the release and let the HDF5 thorn build the libraries instead on ubuntu. The reason is that I am facing problems to find the hdf5.mod present in the ubuntu system at /usr/include (it doesn't affect ET thorns, but it does for private ones). With the hershel release I had to comment out the line stripping the system include path, but that is not working anymore for the development. If you are ok with that I will revert that patch for HDF5 thorn until I figure out what is going wrong on ubuntu.

  4. Ian Hinder
    • removed comment

    Does this only affect Fortran? We have other machines were Fortran support in HDF5 is disabled. Nothing in the ET requires it, and we only decided to require it because it was "easy" and wouldn't cause any problems (#1158). Or maybe Cactus will make use of (broken) Fortran support if it finds it, even if it is told not to require it? In any case; if building HDF5 ourselves is the easiest solution, then I'm happy to go with that.

  5. Bruno Mundim reporter
    • removed comment

    Replying to [comment:4 hinder]:

    Does this only affect Fortran?

    Apparently yes, but I am not confident we avoid problems for other thorns completely. If I set HDF5_DIR to /usr in my option list and the detect.sh script finds out the library is at /usr/lib/TARGET, where TARGET is the host architecture like x86_64-linux-gnu for example, it sets HDF5_LIB_DIRS to that directory. The potential problem is that other libraries might be there as well and we might not want them. Maybe the solution to the problem I am facing is adding the /usr/lib/TARGET to the system default library search directory list.

    We have other machines were Fortran support in HDF5 is disabled.

    We do need the fortran support.

    Nothing in the ET requires it, and we only decided to require it because it was "easy" and wouldn't cause any problems (#1158). Or maybe Cactus will make use of (broken) Fortran support if it finds it, even if it is told not to require it?

    In this case we tell cactus we need fortran support, the library does support fortran, but Cactus doesn't find the fortran modules offered by the library in standard places such as /usr/include/hdf5.mod. I am not sure why.

    In any case; if building HDF5 ourselves is the easiest solution, then I'm happy to go with that.

    At least for the release it might be safer to not introduce this TARGET library path when detecting and configuring cactus. If we add it to the default system search directories then it might solve this problem. I am not sure where this is hard coded on flesh though.

  6. Frank Löffler
    • removed comment

    Replying to [comment:5 bmundim]:

    Replying to [comment:4 hinder]:

    We have other machines were Fortran support in HDF5 is disabled. We do need the fortran support.

    The ET does not need it. That doesn't mean we shouldn't make it work, but it lowers priority a bit.

    Nothing in the ET requires it, and we only decided to require it because it was "easy" and wouldn't cause any problems (#1158). Or maybe Cactus will make use of (broken) Fortran support if it finds it, even if it is told not to require it? In this case we tell cactus we need fortran support, the library does support fortran, but Cactus doesn't find the fortran modules offered by the library in standard places such as /usr/include/hdf5.mod. I am not sure why.

    Ok, then this would need to debugged.

    In any case; if building HDF5 ourselves is the easiest solution, then I'm happy to go with that.

    We shouldn't make this the default just because some users might require Fortran HDF5 for some of their private thorns, as long as it works otherwise well for everybody else that is.

    We could mention in the release notes that, if you are going to need HDF5 Fortran support in one of your private thorns, and you are using one of the problematic library installations, you might be better off building HDF5 using Cactus.

    At least for the release it might be safer to not introduce this TARGET library path when detecting and configuring cactus. If we add it to the default system search directories then it might solve this problem. I am not sure where this is hard coded on flesh though.

    In the long run: couldn't we find standard paths (where possible) using something like (for C):

    echo | cpp -Wp,-v 2>&1 | perl -p0e 's/.*include <...> search starts here:.(.*)End of search list.*/\1/sm; s/(\n|^) /:/g'

    In my case, that list includes

    /usr/lib/gcc/x86_64-linux-gnu/4.9/include
/usr/local/include
/usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed
/usr/include/x86_64-linux-gnu
/usr/include
  7. Bruno Mundim reporter
    • removed comment

    To be more precise where and how this failure is happening this is the standard output:

    echo ________________________________________________________________________
________________________________________________________________________
echo Preprocessing /path/to/my/thorn/src/file.F90
Preprocessing /path/to/my/thorn/src/file.F90
{ if test yes = 'yes'; then echo '#line 1 "'/path/to/my/thorn/src/file.F90'"'; fi; cat /path/to/my/thorn/src/file.F90; } | perl -p -e 's.//.CCTK_AUTOMATICALLY_GENERATED_CONCATENATION_PROTECTION.g' | cpp -traditional  -fopenmp -Wall -I"/usr/lib/openmpi/include" -I"/usr/lib/openmpi/include/openmpi" -I"/usr/lib/openmpi/include" -I"/usr/lib/openmpi/include/openmpi" -I"/usr/lib/x86_64-linux-gnu" -I"/usr/lib" -I"/path/to/my/thorn/src" -I"/path/to/my/thorn/src/include" -I"/home/bruno/tmp/ET_git_dev/Cactus/configs/thc/config-data" -I"/home/bruno/tmp/ET_git_dev/Cactus/configs/thc/bindings/include" -I"/home/bruno/tmp/ET_git_dev/Cactus/src/include" -I"/home/bruno/tmp/ET_git_dev/Cactus/arrangements" -I"/home/bruno/tmp/ET_git_dev/Cactus/configs/thc/bindings/Configuration/Thorns" -I"/home/bruno/tmp/ET_git_dev/Cactus/configs/thc/bindings/include/mythorn" -I"/usr/lib/openmpi/include" -I"/usr/lib/openmpi/include/openmpi" -I"/usr/lib/openmpi/include" -I"/usr/lib/openmpi/include/openmpi" -I"/usr/lib/x86_64-linux-gnu" -I"/usr/lib" -I"/path/to/my/thorn/src" -I"/path/to/my/thorn/src/include" -I"/home/bruno/tmp/ET_git_dev/Cactus/configs/thc/bindings/include/mythorn"  -DFCODE -DF90CODE | perl -p -e 's.CCTK_AUTOMATICALLY_GENERATED_CONCATENATION_PROTECTION.//.g' | perl -p -e 's/__FORTRANFILE__/\"file.F90\"/g' | perl -s /home/bruno/tmp/ET_git_dev/Cactus/lib/sbin/f_file_processor.pl -free_format -line_directives=yes -source_file_name=/path/to/my/thorn/src/file.F90 > file.f90
echo Compiling /path/to/my/thorn/src/file.F90
Compiling /path/to/my/thorn/src/file.F90
if test "gfortran" = "none" ; then echo "There is no Fortran 90 compiler available.  Aborting." ; exit 1 ; fi ; current_wd=`pwd` ; cd /home/bruno/tmp/ET_git_dev/Cactus/configs/thc/scratch ; gfortran -g3 -march=native -fcray-pointer -m128bit-long-double -ffixed-line-length-none -fno-range-check -O2 -ffast-math -fopenmp -Wall -I"/usr/lib/openmpi/include" -I"/usr/lib/openmpi/include/openmpi" -I"/usr/lib/openmpi/include" -I"/usr/lib/openmpi/include/openmpi" -I"/usr/lib/x86_64-linux-gnu" -I"/usr/lib" -I"/path/to/my/thorn/src" -I"/path/to/my/thorn/src/include" -I"/home/bruno/tmp/ET_git_dev/Cactus/configs/thc/config-data" -I"/home/bruno/tmp/ET_git_dev/Cactus/configs/thc/bindings/include" -I"/home/bruno/tmp/ET_git_dev/Cactus/src/include" -I"/home/bruno/tmp/ET_git_dev/Cactus/arrangements" -I"/home/bruno/tmp/ET_git_dev/Cactus/configs/thc/bindings/Configuration/Thorns" -I"/home/bruno/tmp/ET_git_dev/Cactus/configs/thc/bindings/include/mythorn" -I"/usr/lib/openmpi/include" -I"/usr/lib/openmpi/include/openmpi" -I"/usr/lib/openmpi/include" -I"/usr/lib/openmpi/include/openmpi" -I"/usr/lib/x86_64-linux-gnu" -I"/usr/lib" -I"/path/to/my/thorn/src" -I"/path/to/my/thorn/src/include" -I"/home/bruno/tmp/ET_git_dev/Cactus/configs/thc/bindings/include/mythorn" -c -o $current_wd/file.F90.o $current_wd/file.f90
Warning: Nonexistent include directory "/path/to/my/thorn/src/include"
Warning: Nonexistent include directory "/path/to/my/thorn/src/include"
/path/to/my/thorn/src/file.F90:2.6:

  use hdf5
      1
Fatal Error: Can't open module file 'hdf5.mod' for reading at (1): No such file or directory

    If I add -I"/usr/include" immediately before "-c -o $current_wd/file.F90.o $current_wd/file.f90" then the compilation succeeds. Can we at least add this path explicitly and maybe the "/usr/local/include" for this release?

  8. Frank Löffler
    • removed comment

    Can you please post a little more: your option list, and preferably a complete log of the configure stage (HDF5)?

  9. Erik Schnetter
    • removed comment

    Adding -I/usr/include is most likely a bad idea, for just the reasons discussed earlier. This shortly before a release I'd rather favour a work-around. Would building HDF5 yourself work? If so, then I suggest to do this for the release, and then we have time to discuss and test a proper fix afterwards that can also be backported to the release.

  10. Bruno Mundim reporter
    • removed comment

    In the long run: couldn't we find standard paths (where possible) using something like (for C): echo | cpp -Wp,-v 2>&1 | perl -p0e 's/.*include <...> search starts here:.(.*)End of search list.*/\1/sm; s/(\n|^) /:/g' In my case, that list includes /usr/lib/gcc/x86_64-linux-gnu/4.9/include /usr/local/include /usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed /usr/include/x86_64-linux-gnu /usr/include

    This command, as it stands, doesn't work on a 'CentOS release 6.4' system (gordon). It returns 'Ambiguous output redirect.'. However I think it is on the right track of parsing the system include paths.

  11. Bruno Mundim reporter
    • removed comment

    Replying to [comment:8 knarf]:

    Can you please post a little more: your option list, and preferably a complete log of the configure stage (HDF5)?

    I used the ubuntu.cfg optionlist as it is in the repository. My config-info file is the following:

    # CONFIGURATION  : thc
# CONFIG-DATE    : Sat May  9 14:03:14 2015 (GMT)
# CONFIG-HOST    : bhlap
# CONFIG-STATUS  : 0
# CONFIG-OPTIONS :
BOOST_DIR=/usr
CC=gcc
CFLAGS=-g3 -march=native -std=gnu99
CPP=cpp
CPPFLAGS=-DMPICH_IGNORE_CXX_SEEK
CPP_DEBUG_FLAGS=-DCARPET_DEBUG
CPP_OPENMP_FLAGS=-fopenmp
CPP_OPTIMISE_FLAGS=-DKRANC_VECTORS
CPP_PROFILE_FLAGS=
CPP_WARN_FLAGS=-Wall
CXX=g++
CXXFLAGS=-g3 -march=native -std=gnu++0x
CXX_DEBUG_FLAGS=-O0
CXX_OPENMP_FLAGS=-fopenmp
CXX_OPTIMISE_FLAGS=-O2 -ffast-math
CXX_PROFILE_FLAGS=-pg
CXX_WARN_FLAGS=-Wall
C_DEBUG_FLAGS=-O0
C_LINE_DIRECTIVES=yes
C_OPENMP_FLAGS=-fopenmp
C_OPTIMISE_FLAGS=-O2 -ffast-math
C_PROFILE_FLAGS=-pg
C_WARN_FLAGS=-Wall
DEBUG=no
F77=gfortran
F77FLAGS=-g3 -march=native -fcray-pointer -m128bit-long-double -ffixed-line-length-none -fno-range-check
F77_DEBUG_FLAGS=-O0
F77_OPENMP_FLAGS=-fopenmp
F77_OPTIMISE_FLAGS=-O2 -ffast-math
F77_PROFILE_FLAGS=-pg
F77_WARN_FLAGS=-Wall
F90=gfortran
F90FLAGS=-g3 -march=native -fcray-pointer -m128bit-long-double -ffixed-line-length-none -fno-range-check
F90_DEBUG_FLAGS=-O0
F90_OPENMP_FLAGS=-fopenmp
F90_OPTIMISE_FLAGS=-O2 -ffast-math
F90_PROFILE_FLAGS=-pg
F90_WARN_FLAGS=-Wall
FPP=cpp
FPPFLAGS=-traditional
FPP_DEBUG_FLAGS=-DCARPET_DEBUG
FPP_OPENMP_FLAGS=-fopenmp
FPP_OPTIMISE_FLAGS=
FPP_PROFILE_FLAGS=
FPP_WARN_FLAGS=-Wall
F_LINE_DIRECTIVES=yes
HDF5_DIR=/usr
LDFLAGS=-rdynamic
LIBDIRS=
MPI_DIR=/usr
OPENMP=yes
OPTIMISE=yes
PROFILE=no
PTHREADS=yes
VERSION=2012-09-28
WARN=yes

    config.log file is attached. The make.HDF5.defn file at Cactus/<config>/bindings/Configuration/Capabilities is as follows:

    include /home/bruno/tmp/ET_git_dev/Cactus/configs/thc/bindings/Configuration/Capabilities/make.ZLIB.defn
include /home/bruno/tmp/ET_git_dev/Cactus/configs/thc/bindings/Configuration/Capabilities/make.MPI.defn
INC_DIRS +=  $(HDF5_INC_DIRS)
HDF5_ENABLE_CXX     = no
HDF5_ENABLE_FORTRAN = yes
LIBSZ_DIR           =
LIBZ_DIR            =
HDF5_INSTALL_DIR    =
HDF5_DIR            = /usr
HDF5_ENABLE_CXX     = no
HDF5_ENABLE_FORTRAN = yes
HDF5_INC_DIRS       = /usr/lib/x86_64-linux-gnu /usr/lib
HDF5_LIB_DIRS       = /usr/lib/x86_64-linux-gnu
HDF5_LIBS           =  hdf5hl_fortran hdf5_fortran hdf5_hl hdf5 z m z
HAVE_CAPABILITY_HDF5 = 1
  12. Bruno Mundim reporter
    • removed comment

    Replying to [comment:9 eschnett]:

    Adding -I/usr/include is most likely a bad idea, for just the reasons discussed earlier.

    Note that I didn't propose to add this path when we detect the external library, but at the end, after all external libraries have been configured. Also it might only be really necessary for Fortran compilation. There seems to have a bug in gfortran in which the system include paths are not searched. Take a look at this bug report thread: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=35707. Despite the documentation saying -Idir works the same as for cpp or gcc case, that's not true. We do need to add -Idir for system directories in Fortran if we hope to find system *.mod Fortran module files.

    This shortly before a release I'd rather favour a work-around. Would building HDF5 yourself work? If so, then I suggest to do this for the release, and then we have time to discuss and test a proper fix afterwards that can also be backported to the release.

    Adding -I/usr/include at the end of the command line compiling Fortran source codes seemed to me a workaround. Yes, we can always build the library, but I don't want to do so and I had hoped a more general solution would have surfaced by now. Unfortunately it is trickier than what I anticipated.

    My personal workaround for Ubuntu, and Ubuntu only, is this patch together with commenting out the following lines of the detect.sh script:

    HDF5_INC_DIRS="$(${CCTK_HOME}/lib/sbin/strip-incdirs.sh ${HDF5_INC_DIRS})"
HDF5_LIB_DIRS="$(${CCTK_HOME}/lib/sbin/strip-libdirs.sh ${HDF5_LIB_DIRS})"

    this way both system include path, /usr/include, and the library path, /usr/libx86_64-linux-gnu, would be searched. This works well for me because a) I don't plan on building any external library on ubuntu; b) I don't plan on using any library that is not shipped with the distribution. This is obviously not an workaround for everyone or on every system. Said that, maybe the best is to just revert these patchs on HDF5 and Boost external library thorns for the release. I will happily apply these patches on my system, everybody else builds these libraries when configuring their ubuntu system until we come up with a better solution. To me the long term solution is to identify the system include and library paths and add them at the end of the command line for compiling and linking, respectively.

  13. Frank Löffler
    • removed comment

    Replying to [comment:10 bmundim]:

    In the long run: couldn't we find standard paths (where possible) using something like (for C): echo | cpp -Wp,-v 2>&1 | perl -p0e 's/.*include <...> search starts here:.(.*)End of search list.*/\1/sm; s/(\n|^) /:/g'

    This command, as it stands, doesn't work on a 'CentOS release 6.4' system (gordon). It returns 'Ambiguous output redirect.'. However I think it is on the right track of parsing the system include paths.

    Could it be that you are using tcsh? The redirect is bash syntax (and that script is bash).

  14. Bruno Mundim reporter
    • removed comment

    Replying to [comment:13 knarf]:

    Replying to [comment:10 bmundim]:

    In the long run: couldn't we find standard paths (where possible) using something like (for C): echo | cpp -Wp,-v 2>&1 | perl -p0e 's/.*include <...> search starts here:.(.*)End of search list.*/\1/sm; s/(\n|^) /:/g'

    This command, as it stands, doesn't work on a 'CentOS release 6.4' system (gordon). It returns 'Ambiguous output redirect.'. However I think it is on the right track of parsing the system include paths.

    Could it be that you are using tcsh? The redirect is bash syntax (and that script is bash).

    My bad... yes, you are right!

  16. Roland Haas
    • removed comment

    Once this is up for proper review: * these seem very useful, thank you for gettting this started * the functions in bash_utils.sh (I think the name could be improved to be more descriptive to eg refer to the external libraries) namely find_files and find_libs use a global variable FOUND to communicate the fact that find_files suceeded. Instead it may be better to have find_files return 0 or 1 depending on success or failure. * it reports "Could not find tar command." if the TAR variable is not set. This is slightly incorrect since TAR may well be set to an incorrect path in which case it also "does not find tar" later on (the behaviour matches what the current detect.sh scripts do however). * a possible inprovement above using set_make_vars would be to have the functions use echo to output the variable definition along the lines "HDF5_LIBS=hdf5" so that the caller can use eval to activate them (same as eg ssh-agent does). This avoids using global variables to transfer function results. One can use if ! vars=function ; then echo "Failure" ; fi to assign the returned string to vars and catch errors at the same time.

  17. Frank Löffler
    • removed comment

    That file is meant to contain useful bash scripts, not necessarily only for libraries, but I agree that it does at the moment. The functions could return something else than 0, but then the calling scripts have to check that if they use '-e'. Good point. I am not sure about your last point now. The functions already echo failure messages.

  18. Bruno Mundim reporter
    • removed comment

    Hi Frank,

    I built a fresh configuration today on Kubuntu 14.04.3 LTS with your most current fixes for bash_utils.sh and HDF5 external library detect.sh. Unfortunately hdf5 library installed in my system wasn't found and the library was built. I think the usual problem of finding the library in the multiarch directory layout persists. The verbose output of the script is attached in the file hdf.debug.

  19. Frank Löffler
    • removed comment

    Thanks for the helpful log. It appears that Ubuntu didn't package the necessary .pc package config file in trusty (14.04), but did starting in utopic (14.10) and vivid (15.04).

    Utopic is EOL already (Trusty isn't), Vivid is alive, and Wily (15.10) is about to release tomorrow. Updating might already resolve the issue. In the meantime - would it be feasible to either let it build the bundled version, or set specific LIBS/LIB_DIRS/INC_DIRS on Trusty, until you update to the one of the current releases?

  21. Frank Löffler
    • removed comment

    Replying to [comment:20 rhaas]:

    Isn't 14.04 the LTS release? We should then still support it.

    True. The next LTS is scheduled only for 1Q2016, and even then ...

  22. Bruno Mundim reporter
    • removed comment

    I agree with Roland. Should we have a specific ubuntu.cfg file for Trusty? I will set the proper paths for ubuntu, test again and report here later on.

  23. Frank Löffler
    • removed comment

    Is a solution using the 'gcc' workaround still viable? It doesn't sound too bad, assuming it also works if gcc isn't present.

  24. Frank Löffler

    I've added a 'multiarch' patch, which adds the functionality Bruno already provided in his patch to the flesh helper function: Look for output of 'gcc -dumpmachine'. Ignore errors (if any), and ignore if there is no output. If there is, also search in lib/OUTPUT.

    I've tested that this works on Trusty (where it is needed) and Jessie (where it is not). When gcc isn't available, the script runs through, without error. It might, of course, in that case not find the library but build the bundled version.

    Please review.

  27. Bruno Mundim reporter
    • changed status to open
    • removed comment

    Hi Frank,

    the issue with using 'gcc --dumpmachine' is that we end up having a system directory in the middle of a list of directories in the linking stage. Then it could pick up other system libraries we might not want to use. As long as the system libraries are linked last then it should be ok. The same for the include directories. As long as the system include directories are last in the list then we are fine.

    Anyways, I am reopening this ticket because the script is not able to find /usr/include/hdf5.mod which is in my system and hdf5 library installed does support fortran. In addition your solution was applied only for HDF5 external library. There are others that need to be patched such as Boost, PAPI and pthreads. They were also moved to multiarch layout on ubuntu.

  28. Frank Löffler

    Replying to [comment:27 bmundim]:

    the issue with using 'gcc --dumpmachine' is that we end up having a system directory in the middle of a list of directories in the linking stage. Then it could pick up other system libraries we might not want to use. As long as the system libraries are linked last then it should be ok. The same for the include directories. As long as the system include directories are last in the list then we are fine.

    This is not handled, true. We'll leave this for after the release now. The code stripping the paths would need to know about this (lib/sbin/strip-libdirs.sh) - and that code is not new, so that code wouldn't have stripped this in the old version as well.

    Anyways, I am reopening this ticket because the script is not able to find /usr/include/hdf5.mod which is in my system and hdf5 library installed does support fortran.

    Assuming this is Ubuntu (trusty, and for reference): no pkg-config

    /use/include/hdf5.h
/usr/include/hdf5.mod
/usr/lib/x86_64-linux-gnu/libhdf5.so

    The script (on my Ubuntu VM) finds hdf5.h and libhdf5.so. It never checks for hdf5.mod (only for the relevant libraries). It does search for the include path, and finds it as /usr/include. Because that is a system include path, it does not get added to the INC_DIRS, for the reason you stated. It goes on marking the library as found and starts to compile. It never fails since there is no Fortran HDF5 code in the ET.

    Now, gfortran decided to not search standard include paths when looking for .mod files (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=35707). This means, to get things to compile, you would have to add -I/usr/include - which obviously is a bad idea in general. That's why distributions started to put these into separate directories, which is the only sensible way - but doesn't help us here now. We cannot add /usr/include in INC_DIRS in case Fortran is requested, because that might just break the next library. We also do not sort the list of library/include directories according to their 'is system path' status for each compiler command. We probably could dabble into that mess - but is that what you want? One possible easier way would be to detect such situation, and to copy the .mod file to somewhere inside Cactus, and reference that path. This would separate the mod file from the system include path. But this would require checking that this file is always up2date with it's original location. We could use a symlink to make sure this is true, but would still need to check for it dangling.

    In any case - I would like to see this issue with Fortran module files and standard include paths be rather in a new ticket, or things get too confusing here.

    In addition your solution was applied only for HDF5 external library. There are others that need to be patched such as Boost, PAPI and pthreads. They were also moved to multiarch layout on ubuntu.

    Indeed. We didn't want to change all this for all libraries so short before a release. Eventually all libraries should use the flesh scripts.

  29. Roland Haas
    • removed comment

    On danger of exposing myself to friendly fire: unless we can make sure that the ExternalLibraries scripts and the changes for multi-arch work properly on (almost) all machine we need to support (I am exlucing all clusters since there we almost always provide explicit paths in simfactory option lists) that is at least OSX (macports and homebrew), long term supported Ubuntu, current Ubuntu, Fedora, Debian (stable, testing?), Windows Cygwin (?).

    If not we should revert all changes and restore the state of the last release which, while undesirable in that it build from source too often, at least otherwise works.

  30. Frank Löffler
    • removed comment

    Replying to [comment:29 rhaas]:

    On danger of exposing myself to friendly fire: unless we can make sure that the ExternalLibraries scripts and the changes for multi-arch work properly on (almost) all machine we need to support

    If not we should revert all changes and restore the state of the last release which, while undesirable in that it build from source too often, at least otherwise works.

    It did not really work either. It either also didn't add system include paths to HDF5_INC_DIRS for system libraries, or it did. Both decisions are incorrect. It only worked on Ubuntu, because the actual detection failed, and no system paths got involved. On systems without multi-arch we would still see the same problem we see now. The issue with Fortran now is not really multi-arch related, it just got exposed by fixing the latter. Reverting would provide a 'workaround' for Ubuntu, but only there.

    The only simple "fix" (simple because of the close release) would be to always build if Fortran is requested. Even if a library location was specified by the user it might be a system one, and show the current problem, and we don't have a good way of telling one from another (as pointed out earlier).

  31. Roland Haas
    • removed comment

    Let me restate the question then: is the current state of the scripts going to make the success rate of configuring ExternalLibraries without user intervention (and not fail in any of the machines in simfactory where we do provide explicit settings) higher than if we went back to the state of the last release? If not then we should go back to the state of the last release since then at least we get a known bad thing rather than an unknown bad thing. Basically I don't want to make things worse than before (obviously) but also do not want to make them only a little bit better but introduce larger, mostly non-battle tested scripts.

  32. Frank Löffler
    • removed comment

    Until we test on all machines we don't know, and we only do this in a coordinated fashion before a release. Instead of reverting a lot of unrelated changed, we could revert only disabling looking in multi-arch directories. This should fix it on the old Ubuntu - the only machine we know of where this was reported.

    Also, since the ET does not contain any HDF5 Fortran code, the current problem was not "battle" tested anyway. We only have this problem if users happen to have HDF5 Fortran code around, and because of that explicitly enable it, which by itself is already a diversion from the option lists in simfactoy you mention.

  33. Roland Haas
    • removed comment

    I don't think of just the fortran code but the whole set of scripts currently on master on the different thorns. They were developed on master, and my question is basically: if these changes were proposed (at the point in time of the last commit) for inclusion in the release, would you review them as "please apply"? If so then, fine, they can stay. If not then they are rejected and should be reverted. This is unfortunately the tension we have in the statements "development happens on master" and "changes to the ET require review", both of which we subscribe to. Note that I am not suggesting at all to abandon the effort, I only want to check carefully whether we find it is in a state that is sufficient for the release.

  34. Frank Löffler
    • removed comment

    Replying to [comment:33 rhaas]:

    They were developed on master, and my question is basically: if these changes were proposed (at the point in time of the last commit) for inclusion in the release, would you review them as "please apply"?

    I would. And of course they are developed for the development version. Where else would they be?

    "development happens on master" and "changes to the ET require review", both of which we subscribe to.

    Indeed. This is a fine line to walk: as stable as possible, but without hindering progress.

    Note that I am not suggesting at all to abandon the effort, I only want to check carefully whether we find it is in a state that is sufficient for the release.

    I agree with you. In order to provide a short-cut for others who do not want to read the entire thread I attempt a summary:

    Current state: - multi-arch libraries are now found, and correctly used, but only two libraries use this mechanism so far. More would be nice but has to happen after the release. No problems with that mechanism itself are currently known (to Frank). - In general, there exists a problem with Fortran modules in system include paths. This is independent of multi-arch, but happened to be found after the multi-arch change because Bruno both had the issue of multi-arch without pkg-config (ubuntu), and he uses the HDF5 Fortran bindings outside of the ET. The problem in short here is that in order to compile, the paths containing the Fortran modules have to be added to the INC_DIRS, but if these also contain headers from other libraries (as they often do for system paths) this may break other libraries. So, both adding and not adding them to INC_DIRS is problematic. Fortran module files should simply not be put in directories that contain also files from other libraries. Ideas for workarounds exist, but most are likely to invasive for the release.

    Frank thinks one of the following could be done about the Fortran module problem, if tested by someone who actually uses the Fortran bindings (again, nothing in the ET depends on them, so the ET itself will compile just fine): - Always build HDF5 if Fortran bindings are requested. This makes sure that the HDF5 library is in a location by itself. This is also a pretty small change. It would, however, build 'too often' in general. Frank could live with that for the release. - If Fortran is requested, add the HDF5 include directory, regardless whether this might be a system path or not. This is also a pretty small change in the script, but might break other libraries within the ET, on some machines. Frank thinks that this has the greater potential for breakage.

    In the long run we might just have to detect if the modules files are in an "inproper" location, and build ourself in that case.

  35. Bruno Mundim reporter
    • removed comment

    Replying to [comment:28 knarf]:

    Anyways, I am reopening this ticket because the script is not able to find /usr/include/hdf5.mod which is in my system and hdf5 library installed does support fortran.

    Assuming this is Ubuntu (trusty, and for reference): no pkg-config

    There is pkg-config installed on Ubuntu 14.04.3 LTS. I guess what you meant is that hdf5 doesn't have a *.pc on trusty, right?

    /use/include/hdf5.h /usr/include/hdf5.mod /usr/lib/x86_64-linux-gnu/libhdf5.so

    The script (on my Ubuntu VM) finds hdf5.h and libhdf5.so. It never checks for hdf5.mod (only for the relevant libraries). It does search for the include path, and finds it as /usr/include. Because that is a system include path, it does not get added to the INC_DIRS, for the reason you stated. It goes on marking the library as found and starts to compile. It never fails since there is no Fortran HDF5 code in the ET.

    That's true. It fails because one of our private codes use 'use hdf5' statement.

    Now, gfortran decided to not search standard include paths when looking for .mod files (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=35707). This means, to get things to compile, you would have to add -I/usr/include - which obviously is a bad idea in general. That's why distributions started to put these into separate directories, which is the only sensible way - but doesn't help us here now. We cannot add /usr/include in INC_DIRS in case Fortran is requested, because that might just break the next library. We also do not sort the list of library/include directories according to their 'is system path' status for each compiler command. We probably could dabble into that mess - but is that what you want?

    One idea would be to define a new variable, let's say CCTK_APPEND_SYSTEM_LIBS_INCLUDES, and when set it to "yes" it would find out what the system library and include paths are and add them only after all libraries and flags have already been set. To avoid breaking other configurations this would default to "no". We could then play with it by setting to "yes" only on ubuntu.cfg and smooth the rough edges of the new system there.

    Also note that there is a discussion on standardize these new directories in the case of fortran:

    https://bugs.linuxfoundation.org/show_bug.cgi?id=757

    but progress is very slow on that matter.

    One possible easier way would be to detect such situation, and to copy the .mod file to somewhere inside Cactus, and reference that path. This would separate the mod file from the system include path. But this would require checking that this file is always up2date with it's original location. We could use a symlink to make sure this is true, but would still need to check for it dangling.

    ok, but I am not sure this would be a good idea. How would you check for that? What would trigger this checking?

    In any case - I would like to see this issue with Fortran module files and standard include paths be rather in a new ticket, or things get too confusing here.

    Ok, do you want to create a new ticket then?

    In addition your solution was applied only for HDF5 external library. There are others that need to be patched such as Boost, PAPI and pthreads. They were also moved to multiarch layout on ubuntu.

    Indeed. We didn't want to change all this for all libraries so short before a release. Eventually all libraries should use the flesh scripts.

    Until then, I suggest to keep this ticket open. What do you think?

  36. Frank Löffler
    • removed comment

    Replying to [comment:35 bmundim]:

    There is pkg-config installed on Ubuntu 14.04.3 LTS. I guess what you meant is that hdf5 doesn't have a *.pc on trusty, right?

    Right.

    One idea would be to define a new variable, let's say CCTK_APPEND_SYSTEM_LIBS_INCLUDES, and when set it to "yes" it would find out what the system library and include paths are and add them only after all libraries and flags have already been set. To avoid breaking other configurations this would default to "no". We could then play with it by setting to "yes" only on ubuntu.cfg and smooth the rough edges of the new system there.

    Ubuntu isn't the issue here. This can happen on any system. If we were to use something like CCTK_APPEND_SYSTEM_LIBS_INCLUDES, it would need to be enabled by HDF5, if Fortran is requested - on all systems.

    Also note that there is a discussion on standardize these new directories in the case of fortran: https://bugs.linuxfoundation.org/show_bug.cgi?id=757 but progress is very slow on that matter.

    Yes - nothing at all for the last four years, unfortunately. Maybe the hope was that the issue resolves itself in another way. In any case, there doesn't seem to be enough momentum to hope for anything in the near future.

    ok, but I am not sure this would be a good idea. How would you check for that? What would trigger this checking?

    We could trigger it the same time the library is already checked now (was that re-build?). That's not ideal, but also not worse than the situation now (where an external library could 'vanish' without Cactus noticing right away).

    Ok, do you want to create a new ticket then?

    We could - or given that the multi-arch issue is resolved, mis-use this here. It would avoid copying all comments over.

    Until then, I suggest to keep this ticket open. What do you think?

    I agree.

  39. Roland Haas
    • removed comment

    We no longer seem to have issues with the module files not being found. As far as I can tell some support for multi-arch is present in bash_utils.sh so this ticket seems to be fixed / no longer needed?

  40. Roland Haas
    • edited description
    • changed status to resolved

    Fortran modules are now handled (more) correctly.

    The multi-arch issue that came up during discussion is addressed (but not yet resolved) in #2133

  42. Log in to comment
Assignee
–
Type
bug
Priority
minor
Status
closed
Component
EinsteinToolkit thorn
Milestone
–
Version
development version
Votes
0
Watchers
0
Jira Software: the preferred issue tracker for Bitbucket. Join the team!