Opened 6 years ago

Last modified 14 months ago

#1396 reopened enhancement

Display prompts of executed commands

Reported by: Roland Haas Owned by: Eric Seidel
Priority: optional Milestone:
Component: GetComponents Version: development version
Keywords: Cc:


the attached patch goes to some lengths to capture both stdout and stderr when executing commands and displays each line of output as it is generated by the command. This is useful for programs that output prompts to stdout and wait for user input afterwards.

Attachments (1)

0001-rewrite-run_command-routine-to-capture-stdout-and-st.patch (5.2 KB) - added by Roland Haas 6 years ago.

Download all attachments as: .zip

Change History (14)

comment:1 Changed 6 years ago by Roland Haas

Status: newreview

comment:2 Changed 6 years ago by Erik Schnetter

The obvious question: did you check whether the usual suspects have the respective Perl capabilities installed and available? Are they standard on a fresh Linux box?

comment:3 Changed 6 years ago by Roland Haas

It works on my workstation and on queenbee. Given that GetComponents is the first point of contact between new users and Cactus, I'll give this some more testing and test it on a virtual machine, my macbook and all machines in simfactory that I have a access to. Preferably with a thornlist that actually exercises all capabilities.

comment:4 Changed 5 years ago by Erik Schnetter

We need to make sure that these

use IPC::Open3;
use IO::Select;
use POSIX ":sys_wait_h";
use FileHandle;

are available on all systems. I seem to recall that Eric experimented with IPC parallelization, but had to give up since it was not supported on some systems. The same may stop us here.

comment:5 Changed 5 years ago by Ian Hinder

This cannot be committed without additional testing. Erik, did you review the code? I'm not sure about the process here; should the ticket then go into "reviewed OK", but not be committed until it has been tested in more places? Or does it stay in "accepted" or "reviewed" while the testing is done?

comment:6 Changed 5 years ago by Erik Schnetter

No, I did not review the code. I only briefly looked at it.

comment:7 Changed 5 years ago by Erik Schnetter

Should we apply the patch to the development version give it more testing? This would be with the understanding that issues that cannot be quickly resolved would require us to revert the patch.

Otherwise -- can we use "tee" for this? For example, wrapping the command into

"cd $dir; { $cmd ; } | tee $outfile"

may do the trick, and leaves the gory details to tee.

comment:8 Changed 5 years ago by Roland Haas

I would like to apply the patch to the development versin to see what happens. This *can* be achieved with a suitably constructed tee line if we have access to bash and are willing to use a temporary file to capture output. The required line look something like this (this separates stderr and stdout into ifferent file which we don't really need but which the script from which I lifted the code does):

exec 3>&1
{ exec 4>&1 ; mpirun -n $ncpus $new_executable $cactus_opts -logdir $outdir --redirect=oe $parfile 2>&4 4>&- 3>&- | tee $outdir/stdout >&3 ; } | tee $outdir/stderr

where all the complication comes from wanting to reproduce the commands stderr stream on stderr but also to capture it.

comment:9 Changed 5 years ago by Frank Löffler

Are you sure you replied to the correct ticket? This ticket is about GetComponents. There shouldn't be an mpirun, nor do we have a cactus executable at this point. The solution is interesting though. :)

comment:10 Changed 5 years ago by Roland Haas

Yes, correct ticket. The comment is in reply to which suggests using "tee" instead of the original Perl code. The pasted code (the mpirun using fragment) is just and example on what is required to capture and propagate both stderr and stdout. I did not want to rewrite it and possibly introduce bugs just for an example.

comment:11 Changed 4 years ago by Ian Hinder

Status: reviewreopened

The issues raised in Comment:3 are still present. Somebody needs to run GetComponents on a representative selection of machines to test that the features are available. Note that this will not happen automatically for the release, as people will generally run GetComponents on their own machine and then sync. Further, the automated system run by Erik does not run GetComponents on the remote machines either.

Alternatively, if bash and tee are present (which I assume they will be), the patch could be modified to use those, and this probably doesn't need such extensive testing as there is little reason to expect problems.

I don't see a reason to commit the patch before one of the above options is implemented.

comment:12 Changed 4 years ago by Roland Haas

Using only bash and tee you need something like this:

exec 3>&1
  exec 4>&1
  ls -lR 2>&4 | tee stdout >&3
} | tee stderr >&2

which will capture ls's stdout in the file stdout and its stderr in the file stderr and output each stream to the console as well. I have used something like this for quite a while to capture make's output.

comment:13 Changed 14 months ago by Roland Haas

This is handled by the change proposed in .

Modify Ticket

Change Properties
Set your email in Preferences
as reopened The owner will remain Eric Seidel.
Next status will be 'review'.
as The resolution will be set.
to The owner will be changed from Eric Seidel to the specified user.
The owner will be changed from Eric Seidel to anonymous.

Add Comment

E-mail address and name can be saved in the Preferences.

Note: See TracTickets for help on using tickets.