Improve consistency and performance of sync/build/create

One of the design goals of SimFactory is to make it very difficult for a user to make mistakes or end up with inconsistent results. Currently, if a user modifies a source file and submits a simulation without remembering to rebuild the configuration, the old version of the configuration will be used. SimFactory could scan all the source files which would be used for a build and warn the user if there are new source files. Optionally it could also rebuild the configuration automatically. This check might be better implemented in Cactus, which in principle has the dependency information already. It is functionally equivalent to performing a build before every create, so maybe the best approach would be to make the Cactus build process much faster in the case where no files have been modified. This may not be possible.

A related idea is to ensure that remote configurations are synced before being built, and parameter files are synced before simulations are created. One could record the time of the last successful sync (separately of the source tree and the parameter files) to each machine, and look for files which have been modified since that time. If there are such files, a sync could be initiated, or the user warned. Statting files on a laptop should be very fast. Statting files on Kraken can take minutes or hours. This information could also be used to restrict the list of files synced, to avoid having to stat every file in the source tree on the remote system. This assumes that the only modifications of the source tree on the remote system are due to simfactory on the local system. For me, this is the case 99.999% of the time.

I attach a script that I wrote to interact conveniently with remote systems. It performs a sync of either the source tree or the parameter files depending on the simfactory command you want to run. It allows you to use the syntax

<machine> <command> <args>

instead of

sim --remote <machine> <command> <args>

e.g.

damiana list-simulations

The script should be somewhere on your path, and symlinks named after machines should be created pointing to it. For example,

cp machine .../bin chmod u+x .../bin/sim/machine cd .../bin ln -s machine datura ln -s machine damiana ln -s machine kraken

cd /Cactus

datura sync datura create-submit ... datura build

Keyword: