Modify

Opened 7 years ago

Last modified 3 years ago

#641 confirmed enhancement

Parameter files and thornlists could be tested as part of the release process

Reported by: Ian Hinder Owned by:
Priority: major Milestone:
Component: Other Version:
Keywords: Cc:

Description

There are a number of parameter files and thornlists included as examples with thorns in the ET, and in Cactus. Would it be feasible to have these tested as part of the release process? For example, they should run without crashing or missing thorns, should not trigger NaNs etc, and we should minimise any warnings that are emitted. It would be nice to automate this, though it probably requires a cluster, so would be distinct from the usual test suite procedure, due to requiring more memory than tests.

Something to think about for the next release.

Attachments (0)

Change History (10)

comment:1 Changed 7 years ago by Ian Hinder

Milestone: ET_2012_05ET_2012_11

There are currently 172 example parameter files (arrangements/*/*/par/*.par). Testing these is only feasible automatically. For this, we would need information on how much memory and run time is needed. This could be added as comments to the parameter file (users will also want to know this) which are parsed by whatever system does the testing. We could then test that they at least run without giving any errors. We can't check correctness; for that we hope that the regression tests are sufficient.

There are 27 thornlists (find arrangements/ -name "*.th" -follow). We should test that these build successfully with the ET checkout. What should we do about thornlists which require non-ET thorns?

I think that this task is too much to do before the upcoming release, so I am moving the milestone to the next one.

comment:2 Changed 7 years ago by Frank Löffler

I only find 8 thornlists within an ET checkout:

arrangements/CactusIO/IOJpeg/par/jpeg.th
arrangements/CactusIO/IOJpeg/par/jpeg_amr.th
arrangements/EinsteinEvolve/GRHydro/par/GRHydro.th
arrangements/Carpet/Carpet/par/test_rad_ref.th
arrangements/Carpet/Carpet/par/carpet-test.th
arrangements/Carpet/Carpet/par/test_rad.th
arrangements/McLachlan/doc/mclachlan-public.th
arrangements/McLachlan/doc/qc0-mclachlan-public.th

I don't think it is a problem to include thornlists which include thorns outside of the ET. It is perfectly ok to have thorns being part of the ET, but also usable and interesting in connections with thorns outside of the ET. The same in theory applies to testsuites, but there the issue is size. This isn't an issue with a thornlist.

As for the example parameter files - yes, they should be

  • marked as being important for an ET release (if so)
  • include information on
    • how much memory they need (could we have Carpet give a 'good guess' and use that?)
    • categorize them regarding their runtime (giving an exact runtime isn't possible or usefull, but it would be nice to have something like ({quick, moderate, long}).

Concerning format, Cactus already has a '!DESC' tag, so following this we could use for example:

!DESC Short description
!TAGS ET_2012_05, ET_2012_11
!MEM  2048MB
!TIME quick

comment:3 Changed 7 years ago by Ian Hinder

Yes you are correct - only 8 thornlists. I was using the checkout on my laptop which has some additional arrangements added. We also have some Cactus thornlists (http://svn.cactuscode.org/www/download/thorns/) though these are no longer linked from cactuscode.org, as far as I can tell.

I agree about allowing thornlists which contain non-ET thorns, but we should try to modify the thornlists to only include ET thorns if possible.

It would be nice to be able to run Cactus on a parameter file and have it give a good guess about the memory needed. This would need to be done on the head node before qsub is run, which means Cactus would need to be able to run there. It might not be possible to run Cactus on the head node due to mpirun not working there. So for the purpose of this ticket, I would include the approximate size of the run in the parameter file.

What does "TAGS" mean in your example?

comment:4 Changed 7 years ago by Frank Löffler

I would include in TAGS whatever the thorn author finds that parameter file interesting for. The ET release tags might be one option (to indicate that this file should be usable with that release), but others could be added if desired. We could use it to find out which of the parameter files should be tested with the ET in the first place.

comment:5 Changed 6 years ago by Frank Löffler

Since we don't have the infrastructure for that in place yet and no time now: milestone bump.

comment:6 Changed 6 years ago by Frank Löffler

Milestone: ET_2012_11ET_2013_05

comment:7 Changed 6 years ago by Frank Löffler

Milestone: ET_2013_05

Same argument as last time.

comment:8 Changed 3 years ago by Ian Hinder

Milestone: ET_2015_11
Priority: minormajor
Status: newconfirmed

As discussed at the ET workshop 2015, this is actually a serious issue for new users. They need examples to copy, and if many of them are broken, they are not the best people to fix them, and may be unwilling to ask on the mailing list. Increasing priority to major. I propose that we put effort into this ticket so that all the examples which don't use non-ET thorns can run before the November release.

comment:9 Changed 3 years ago by Ian Hinder

I have started a wiki page for this project. Status will be reported there.

comment:10 Changed 3 years ago by Frank Löffler

Milestone: ET_2015_11

Modify Ticket

Change Properties
Set your email in Preferences
Action
as confirmed The ticket will remain with no owner.
Next status will be 'review'.
as The resolution will be set.
to The owner will be changed from (none) to the specified user.
The owner will be changed from (none) to anonymous.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.