Make Carpet timers hierarchical

Issue #445 open
Ian Hinder created an issue

With the current flat structure of timers in Carpet, it is difficult to identify which timers are contained in which other timers, and hence to avoid double-counting when adding up the times.

This series of patches modifies the timer infrastructure in Carpet to generate a tree of timers where the hierarchy reflects the call-graph of the program. This makes it much easier to interpret the timer output than with the previous flat structure, where it was not possible to see which timers "contained" which others. More implementation details are given at the top of TimerNode.hh.

Note that the Timer source and header files have been renamed as CactusTimer and a new Timer file and object has been created. This is because the Timer object now only provides a wrapper around the Cactus timer mechanism which was contained in the old Timer object.

New parameters output_initialise_timer_tree and output_timer_tree_every control output of a new "timer tree diagram" to standard output for the Initialise and Evolve timer trees respectively. These diagrams indicate:

  1. the value of each timer;
  2. the percentage of the given tree taken by each timer;
  3. which timers are contained in which other timers;
  4. any untimed code

for any timer which takes more than 1% of the tree time.

Making the timers hierarchical means that the ad-hoc methods used before to identify the hierarchy (such as naming the timer Evolve::Sync, for example, to indicate that the Sync timer was a child of the Evolve timer) are no longer necessary and have been removed. Additionally, the construction of timers in a "dynamic" manner is now handled automatically for all timers, so special-case code is no longer needed and has been removed.

Additionally some previously-untimed parts of the code are now timed, and timer names have been made more consistent in some places.

There is code in the patches to output the entire timer tree as an XML file, but it is not enabled.

Ideally the timer tree printed to standard output would contain reductions across processes, but at the moment it contains only the timer from the current process.

The attached "tree-example.txt" file shows an example of the timer tree that is printed for a simulation using the qc0-mclachlan parameter file from the Einstein Toolkit.

Keyword:

Comments (13)

  1. Ian Hinder reporter
    • removed comment

    The v2 version of the patches has several timers added and two bugs fixed in the "untimed" output.

  2. Ian Hinder reporter
    • removed comment

    These patches have been committed on approval by Erik. Leaving ticket open because there is no reduction across processes at the moment.

  3. Ian Hinder reporter
    • removed comment

    Yes, this has been committed. We left the ticket open because the timers are not yet reduced across processes.

  4. Ian Hinder reporter
    • removed comment

    I don't see any reduction code in TimerNode.cc; the reduction should be applied to stdout and the XML timer output. Can you point me to where you think this is done?

  5. Erik Schnetter
    • removed comment

    I assume that these timers are output as Cactus timers, and these reductions are in CactusUtils/TimerReport.

  6. Ian Hinder reporter
    • removed comment

    These are output as Cactus timers, but that format is not hierarchical. The comment above was about the hierarchical output that Carpet does, in TimerNode.cc. This prints the timer tree to stdout and to an XML file. Neither of these include reductions, and they should.

  7. Ian Hinder reporter
    • marked as
    • removed comment

    The reduction has been added to the standard output, but not to the XML output. This means that we have one XML file per process, and these then need to be reduced in post-processing. I think there should be just one XML file containing reduced values, otherwise the number of files becomes awkward, especially when benchmarking on large numbers of processes.

  8. Log in to comment