compile ExternalLibraries with other thorns rather than when CST runs

Issue #1474 new
Roland Haas created an issue

Compiling external libraries takes a long time by now. Currently this happens very early in the compilation phase, before the other thorns are compiled.

It would be better if the compilation happened at the same time as that of other thorns.

This will entail splitting the configuration ie. detecting whether to build the included source or use a system wide copy from the compilation, most likely either using Cactus makefile variables or temporary files to transfer the decisions from the configuration stage to the build stage.

Keyword:

Comments (5)

  1. Erik Schnetter
    • removed comment

    Note that the configuration stage already remembers things like INSTALL_DIR etc. If another variable needs to be remembered, then this should be done the same way.

    Yes, moving the actual build to build time is a very good idea.

  2. Erik Schnetter
    • removed comment

    I have implemented this for zlib, one of the simpler external libraries. The implementation looks as follows:

    • the build instructions are taken out of configure.sh and moved to a new file build.sh
    • a new make variable BUILD_ZLIB remembers whether zlib is found on the system or needs to be built
    • make.code.deps states that the thorn depends onto the zlib library, if BUILD_ZLIB is set
    • make.code.deps defines how to build zlib, and calls build.sh

    This is too disruptive for the release. I will test this more, and propose a patch after the release.

  3. Erik Schnetter
    • removed comment

    Instead of building ExternalLibraries with thorns, they should rather be built ahead of time, and be independent of Cactus configurations. Simfactory3 supports this already.

  4. Roland Haas reporter
    • removed comment

    In view of #1755 we may want to move the configure stage of the ExternalLibraries into detect.sh rather than in build.sh. This way we can know in detect.sh which additional link libraries are required.

  5. Roland Haas reporter
    • removed comment

    I retract comment:4 due to typical configure scripts of library A that depend on library B expecting that B has been fully compiled before the configure script is run, eg to look for include files or to compile a test code against it. This means we cannot run the configure script for A until after B has been fully compiled thus needing to run both configure and make stage of the ExternalLibrary at the same time.

    We could either do both at CST time but then would need a full parallel make+dependency tracking for it or move it into the Cactus build stage with the addition that the build stage must not be able to set INC_DIRS, LIB_DIRS and LIBS (or whatever they are called) the way the CST stage does since for some ExternalLibraries (notable the Fortran bindings of MPI) the name of the link library produced is determined at its configure stage (eg it could be libopenmpif90 or libopenmpi08 depending on the Fortran compiler found).

    I remember that there was a discussion to provide this capability (likely with Erik and I being the main participants) but cannot find it in trac or the mailing list right now.

  6. Log in to comment