Display prompts of executed commands

Issue #1396 open
Roland Haas created an issue

the attached patch goes to some lengths to capture both stdout and stderr when executing commands and displays each line of output as it is generated by the command. This is useful for programs that output prompts to stdout and wait for user input afterwards.

Keyword:

Comments (13)

  1. Erik Schnetter
    • removed comment

    The obvious question: did you check whether the usual suspects have the respective Perl capabilities installed and available? Are they standard on a fresh Linux box?

  2. Roland Haas reporter
    • removed comment

    It works on my workstation and on queenbee. Given that GetComponents is the first point of contact between new users and Cactus, I'll give this some more testing and test it on a virtual machine, my macbook and all machines in simfactory that I have a access to. Preferably with a thornlist that actually exercises all capabilities.

  3. Erik Schnetter
    • removed comment

    We need to make sure that these

    use IPC::Open3;
    use IO::Select;
    use POSIX ":sys_wait_h";
    use FileHandle;
    

    are available on all systems. I seem to recall that Eric experimented with IPC parallelization, but had to give up since it was not supported on some systems. The same may stop us here.

  4. Ian Hinder
    • removed comment

    This cannot be committed without additional testing. Erik, did you review the code? I'm not sure about the process here; should the ticket then go into "reviewed OK", but not be committed until it has been tested in more places? Or does it stay in "accepted" or "reviewed" while the testing is done?

  5. Erik Schnetter
    • removed comment

    Should we apply the patch to the development version give it more testing? This would be with the understanding that issues that cannot be quickly resolved would require us to revert the patch.

    Otherwise -- can we use "tee" for this? For example, wrapping the command into

    "cd $dir; { $cmd ; } | tee $outfile"
    

    may do the trick, and leaves the gory details to tee.

  6. Roland Haas reporter
    • removed comment

    I would like to apply the patch to the development versin to see what happens. This can be achieved with a suitably constructed tee line if we have access to bash and are willing to use a temporary file to capture output. The required line look something like this (this separates stderr and stdout into ifferent file which we don't really need but which the script from which I lifted the code does):

    exec 3>&1
    { exec 4>&1 ; mpirun -n $ncpus $new_executable $cactus_opts -logdir $outdir --redirect=oe $parfile 2>&4 4>&- 3>&- | tee $outdir/stdout >&3 ; } | tee $outdir/stderr
    

    where all the complication comes from wanting to reproduce the commands stderr stream on stderr but also to capture it.

  7. Frank Löffler
    • removed comment

    Are you sure you replied to the correct ticket? This ticket is about GetComponents. There shouldn't be an mpirun, nor do we have a cactus executable at this point. The solution is interesting though. :)

  8. Roland Haas reporter
    • removed comment

    Yes, correct ticket. The comment is in reply to https://trac.einsteintoolkit.org/ticket/1396#comment:7 which suggests using "tee" instead of the original Perl code. The pasted code (the mpirun using fragment) is just and example on what is required to capture and propagate both stderr and stdout. I did not want to rewrite it and possibly introduce bugs just for an example.

  9. Ian Hinder
    • changed status to open
    • removed comment

    The issues raised in Comment:3 are still present. Somebody needs to run GetComponents on a representative selection of machines to test that the features are available. Note that this will not happen automatically for the release, as people will generally run GetComponents on their own machine and then sync. Further, the automated system run by Erik does not run GetComponents on the remote machines either.

    Alternatively, if bash and tee are present (which I assume they will be), the patch could be modified to use those, and this probably doesn't need such extensive testing as there is little reason to expect problems.

    I don't see a reason to commit the patch before one of the above options is implemented.

  10. Roland Haas reporter
    • removed comment

    Using only bash and tee you need something like this:

    exec 3>&1
    {
      exec 4>&1
      ls -lR 2>&4 | tee stdout >&3
    } | tee stderr >&2
    

    which will capture ls's stdout in the file stdout and its stderr in the file stderr and output each stream to the console as well. I have used something like this for quite a while to capture make's output.

  11. Log in to comment