- changed status to open
- removed comment
Add support for parameter file scripts
Issue #473
closed
It is often useful to specify simulations by higher-level descriptions than Cactus parameter files. For example, when performing a convergence test, many parameters might change between simulations at different resolutions, and changing them all manually is tedious and error-prone. Similarly, it can be very useful to set parameters to values computed from simple expressions. For this reason, I usually use a "parameter file script" as my basic description of a simulation.
A parameter file script is a file with a ".rpar" extension which, when executed, generates a file in the same place but with a ".par" extension. The resulting file should be a valid simfactory parameter file. The attached patch adds support in SimFactory for these scripts. Use a script in place of a parameter file when invoking SimFactory. When a simulation is run, the script will be executed and the resulting parameter file will be used by Cactus. Also attached is an example of a parameter file script written in Perl. It looks exactly like a Cactus parameter file apart from a header and footer, and $-replacements in the main body.
Keyword:
Comments (8)
-
reporter
-
- removed comment
There is one script generating multiple parameter files -- how do you choose which parameter file to use? Do you use SimFactoy's @ variables?
-
reporter
- removed comment
The parameter script is first run, and SimFactory's @...@ variables are expanded in the resulting file. It would be slightly cleaner for this to happen in the opposite order, but it would break compatibility with a huge number of parameter file scripts at the AEI which have to escape the @s in @WALLTIME_HOURS@ with a backslash. We could add a temporary workaround which dealt with this if we decided that expanding in the script was better.
At the moment, I only support the case where a script generates a single parameter file. This has the advantage of having a 1-1 mapping between scripts and simulations. This is good because the simulation name is conveniently derived from the script name. It is also a disadvantage, as it means that I need N scripts when I do N resolutions in a convergence test, which currently leads to a lot of copy commands and manual editing.
One method which would work with the current patch is to create a symbolic link to the script for each resolution that you want to run. The script could determine which resolution to run from the name it is invoked with. e.g. you could have bbh_24.rpar, bbh_28.rpar, bbh_32.rpar as your symbolic links to bbh.rpar, and bbh.rpar looks at $0 to determine "n", which it uses to set the required number of grid points.
Another method would be to have a "variant" option to simfactory. This would be a string passed on the "sim create" command line which would be expanded using @VARIANT@ in the parameter file. This would need the expansion to happen in the script rather than in the generated file though. The script could then look at the value of @VARIANT@ and decide how to interpret it. The simulation name could have @VARIANT@ appended by default, when it is derived from the parameter file. For example,
sim create-submit --variant 24 par/bbh.rpar 256 24:00:00
would give a simulation called "bbh_24" and @VARIANT@ would be expanded as "24" in the script. Alternatively, the variant string could be passed as an argument to the script, rather than using the substitution method.
-
reporter
- removed comment
The first version of the patch doesn't actually work - I must not have tested it sufficiently after reorganising it. The v2 version has an error corrected.
-
- removed comment
Running a script seems like a useful enough functionality to have. Please apply.
Of course, in an ideal world you would "import Simfactory" in the beginning, and then use Python...
-
reporter
- removed comment
I had tried to do this in the past, but the Python syntax for "here-documents" never seemed to be as convenient as the Perl one. You need to provide a dictionary containing all your definitions, and if you want to compute one definition using a previous definition, this requires a lot of duplication. However, I have found a way to work around this by using the locals() function which returns a dictionary containing all the locally defined variables. This can then be used in the substitution. I'm attaching a Python version of test_rpar.
-
reporter
- changed status to resolved
- removed comment
Committed in r1395.
-
- edited description
- changed status to closed
- Log in to comment
