Use subdirectories when generating Formaline tarballs

Issue #561 closed
Erik Schnetter created an issue

When there are many thorn, the directory used to generate the Formaline tarballs can contain many files, likely reducing I/O performance. This patch uses a subdirectory for each thorn, which improves the directory layout and should also improve performance.

This patch also adds all thorns to the git repository at once, instead of using a single make rule for each thorn. Since git repositories cannot be accessed for writing in parallel, using multiple make rules leads to long wait times for the git repository lock, which is avoided if one explicitly serialises this.

Keyword:

Comments (5)

  1. Frank Löffler
    • removed comment

    The patch looks fine, please apply. One question though: after updating to this within an existing configuration, would old-style-named files and directories stay around, not ever being deleted? If this would be the case, how difficult would it be to automatically trigger a full removal of old files followed by a full regeneration using the new scheme if old-style files are found?

  2. Erik Schnetter reporter
    • removed comment

    The easiest way to clean up the previous files is to issue a "make clean". We have mechanisms in Cactus to enforce a rebuild (rerun the CST stage) and to enforce a reconfiguration (rerun configure), but we don't have a corresponding mechanism to enforce cleaning.

    I could add code to look for old-style files, and if so, delete the Formaline directory, which would then probably lead to a make failure, but the next make would succeed. It is difficult to do anything else, because make examines the directory structure first and runs commands later, so we cannot remove old files before make examines what is there. I'm sure there is a trick to do it, but this would require debugging, and would add complexity necessary only for the transition.

  3. Log in to comment