Test "Poisson equation" example

Issue #2173 resolved
Roland Haas created an issue

Before each release, check that http://einsteintoolkit.org/gallery/poisson/index.html still works and produces correct output.

Then upload the results file to https://bitbucket.org/einsteintoolkit/www/downloads/

Keyword: None

Comments (77)

  1. Roland Haas reporter
    • removed comment

    I re-ran the example using current master code and got this graph for the error (terr value, plotted as line number vs. column 2):

    However looking at the values of the coefficient (or the solution in psi for that matter) reveals that not all is well:

    namely of the 5 Gaussian blobs mentioned in the parfile only 2 show up.

    Some digging reveals that this is b/c of a not-backwards compatible change to CT_Analytic in d07b160 - (tag: ET_2015_11_v0) Reduced number of terms of gaussian[] superposition in Analytic.m in the first version of the thorn in the toolkit.

    This is in principle ok since it never supported more than 2 Gaussian blobs while in the ET but we likely should

    1. remove the non-functional parameters from param.ccl
    2. regenerate the gallery example with the correct data
  2. anonymous
    • removed comment

    We agreed in the ET call today to: 1. remove the non-functional parameters from the thorn 1. regenerate the example data using the current thorn. Will compare that the result are comparable to what was produced by the original code when using the same number of Gaussians.

  3. Roland Haas reporter
    • removed comment

    In the call on 2018-0802 a suggestion was made to mabye re-instantiate the 5 Gaussians. param.ccl right now claims that 20 gaussians are supported, so supporting 5 is not actually fixing the "bug" but instead only making the gallery example work.

    I new pull request enabling 20 gaussians is here:

    I have quickly checked on runtime and find that run time of the generated routine is:

    number of Gaussians run time total run time (s)
    2 139 287
    5 305 582
    20 869 1030

    So basically the CT_Analytic function's runtime is scaling almost linearly with the number of Gaussians. Note that this is a version of the poisson.par file where the max. number of iterations in the ML scheme has been reduced to 2 from 200 so presumably in a real run of the parfile CT_Analytic would not take >50% of the runtime since it only runs once and not once per grid sweep.

    I will thus prepare and test a commit allowing up to 5 Gaussians which lets one run the gallery example as shown.

    Only relative numbers matter as the absolute numbers would change with processor speed etc.

  4. Roland Haas reporter
    • removed comment

    Commands to create tarball and upload to bitbucket:

    tar czf poisson-$(date +%Ym%d).tar.gz --exclude cactus-source --exclude SIMFACTORY --exclude 'checkpoint*.h5' --exclude 'ct_multilevel-err*.h5' --exclude 'ct_multilevel-residual*.h5' err.p?? poisson
    
    curl -u 'rhaas@illinois.edu' -X POST https://api.bitbucket.org/2.0/repositories/einsteintoolkit/www/downloads -F files=@"poisson-$(date +%Ym%d).tar.gz"
    

    removing some of the output files was required as the upload otherwise fails (for the 750MB full tarball).

  5. Roland Haas reporter
    • changed status to resolved
    • removed comment

    I pushed a change to support 5 amp[] values in git hash 2563eba "CT_Analytic: regenrate code" of ctthorns. This makes the result look like what the gallery shows. I have updated the gallery in git hash 71f3c7a "gallery/ns: update plot and data after re-run" of wwww.

  6. Matthew

    I obtained the above results from running the poisson gallery example, using the “terr_norm_eqn0.asc” and “ct_analytic-ct_testc0.file_*.h5” files respectively. Would you still like me to upload all the results onto bitbucket?

  7. helvi witek

    Done.

    We uploaded the data of the latest test and the corresponding information on the website.

  8. Roland Haas reporter

    I updated the link on the Poisson gallery page to point to the newest tarball, and added Matthew's plot to the page as well.

    Marking as resolved.

  9. helvi witek

    The gallery example ”poisson” was last successfully tested on 11 October 2019. The data is uploaded and the website description updated.

  10. William Gabella

    Tested several times and once I used 1 node, 8 mpi tasks, and 1 cpu per task, with 48 GB it ran in 20-25 minutes. Did not use simfactory, so the output tarball reflects that. Removed big files and added some plot files, a python script to make the err.png graphic, a Readme, and left in the 11 Oct 2019 run script and err.png with _20191011 appended to them. The command file might be useful to other HPCs. Included our Slurm script and the Lmod script it uses to load modules.

    Used a more manual curl command to add to the Downloads directory, see Roland’s above,

    curl -u 'b.gabella@vanderbilt.edu' -X POST https://api.bitbucket.org/2.0/repositories/einsteintoolkit/www/downloads -F files=@"poisson_20200529.tar.gz"
    

    which is useful getting the output director from the HPC to the Bitbucket Downloads folder. Must have an account and permissions to do this.

    I used git to edit the index.html and to change out the err.png file:

    git clone https://bitbucket.org/einsteintoolkit/www/src/master/
    

    Find the /master/gallery/poisson folder and update the index.html and any graphics (at least the err.png) you change or add. Then just git after that, always with care,

    git status
    git add index.html
    git add err.png
    git commit -m "some comment"
    git push
    <with login>
    

    Then clear your browser history and update that Poisson gallery page

    http://einsteintoolkit.org/gallery/poisson/index.html

    and check your changes were taken.

    I found it useful to download my tarball with wget (my browser seems to gunzip it but leave the name XXX.tar.gz, but it is no longer gzipped. Checked it was right with

    wget https://bitbucket.org/einsteintoolkit/www/downloads/poisson_20200529.tar.gz
    

    and that downloaded a gzipped tarball. Go figure.

    Could not update the VisIt graphic, did not have access, will still try. At the top of this issue is a *.session file for VisIt that should be useful.

  11. William Gabella

    [Warning save your long entry before attaching any files…just lost a ton of text when I attached psi2.session. Ack!]

    Have been unable to install Visit either on my laptop or on our HPC cluster. Found a docker image for version 2.12.3 (current as of now is 3.1.2) at https://github.com/symerio/visit-docker . Cannot run docker on my laptop because Fedora 32 uses CGRoups V2 and currently runc from docker throws an error. I can run podman as

    podman run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix:rw symerio/visit

    or with more -v local directories mapped to container directories.

    From the HPC cluster, where we use singularity, I use the command

    singularity run docker://symerio/visit

    from within the Poisson data directory, the one containing the two plotted data files ct_analytic-ct_testc0.file_0.h5 and ct_multilevel-psi.file_0.h5 . Copy the psi2.session file, and run above command, and restore the session from File > Restore Session and select the psi2.session file. This should recreate the images with the sources in white and the slice through the Psi potential with color contours. With the new versions of the Toolkit, running this gallery example, I get an image that is indistinguishable to the eye from the previous one.

  12. William Gabella

    Re-ran Poisson example today and then did a diff of the *.asc files and they agree but maybe for the last digit. Run looks fine with new ET.

    Updated the *.png files and they look identical. Used Visit via singularity for psi.png and used the plotErr.py python script for err.png (had to rename it).

    Was able to git push the web pages but cannot do the curl command from the hpc:

    gw345/gabe :<esims/poissFreeze3 370 >curl -u 'b.gabella@vanderbilt.edu' -X POST https://api.bitbucket.org/2.0/repositories/einsteintoolkit/www/downloads -F files=@"poisson_20201119.tar.gz"
    Enter host password for user 'b.gabella@vanderbilt.edu':
    
    Because your Atlassian account admin requires single sign-on, you'll need to authenticate with an app password. Create an app password at https://bitbucket.org/account/admin/app-passwords
    

    Host password was NOT my user password. Thought this worked seamlessly last time:)

    Trying to upload the 300 MB poisson_20201119.tar.gz via the web page after I am logged into ET bitbucket.

  13. William Gabella

    Logging out and back on did not help.

    Tried also using the web page at the Downloads folder to “Add Files” and it takes a long time and then shows a “You must solve the captcha.” without every showing me a captcha.

  14. William Gabella

    Oops, forgot that psi_4x3.png was used by the gallery page. Just added that to poisson directory via git. Cropped the psi.png to be 1200x900 pixels.

  15. William Gabella

    Okay the “host password” using my userid and not my SSO email, worked today. Maybe something was buggered yesterday evening or this is the magic incantation. Curl command was

    curl -u 'wegabella' -X POST https://api.bitbucket.org/2.0/repositories/einsteintoolkit/www/downloads -F files=@"poisson_20201119.tar.gz"
    Enter host password for user 'wegabella':
    

    And I used the “app password” made with my Bitbucket, SSO account with full email “b.gabella@vanderbilt.edu” and selecting all permissions…maybe too much.

    In “Your profile and settings” > Personal Settings> App Passwords. Create one and in this case because of the difficulty I had I selected all permissions. It warns you but, you cannot look at the password after the one time it shows it to you, so copy it down.

  16. Miguel Gracia

    I have updated the parfile with the Cactus::presync_mode = "mixed-error" option.

    I ran the example using a machine configured with Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz and 64Gb of ram.

  17. Miguel Gracia

    I updated the website and uploaded the results. I ran the example using a machine configured with Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz and 64Gb of ram.

  18. Nadine Kuo

    Just uploaded the results under https://bitbucket.org/einsteintoolkit/www/downloads/. It includes the following error plot:

    I was able to upload my tar.gz file via Chrome, despite the total size of 262 MB…

    Curl was giving me an error message similar to the one described earlier by @William Gabella .

    nadinekuo@DESKTOP-CFJ01HF:~/simulations$ curl -u 'hnkuo2@illinois.edu' -X POST https://api.bitbucket.org/2.0/repositories/einsteintoolkit/www/downloads -F files=@"poisson_20221015.tar.gz"
    Enter host password for user 'hnkuo2@illinois.edu':
    Bitbucket Cloud recently stopped supporting account passwords for API authentication.
    See our community post for more details: https://atlassian.community/t5/x/x/ba-p/1948231
    App passwords are recommended for most use cases and can be created in your Personal settings:
    https://bitbucket.org/account/settings/app-passwords/
    For more details on API authentication methods see our documentation:
    https://developer.atlassian.com/cloud/bitbucket/rest/intro/#authentication
    

    However, I think curl would still work after creating an app password with write permission (https://bitbucket.org/account/settings/app-passwords/) and as mentioned, use your Bitbucket username rather than e-mail address.

  19. Nadine Kuo

    Moreover, the machine I used to run the gallery example is configured as follows:

    Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz with 16.0GB of RAM.

  20. Leonardo Werneck

    @Roland Haas could you please change this milestone to ET_2023_05 and assign it to @Bing-Jyun Tsao ? I don’t know how to change the assignee.

  21. Bing-Jyun Tsao

    2023_05 test
    Ran with 1 thread and 1 node for 72 minutes, or 8 threads and 1 node for 10 minutes. Machine uses Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GH. RAM is 64GB.
    Both error plot and 3D psi plot are consistent with previous tests.
    Visualized with Visit3.2.1, change source path from psi2.session and saved as psi4.session:

    <Field name="SOURCE00" type="string">"ct_analytic-ct_testc0.file_* database"</Field>
    <Field name="SOURCE01" type="string">"ct_multilevel-psi.file_* database"</Field>
    

    Note for future testers using just 1 thread: h5 files will only be saved in ct_analytic-ct_testc0.h5 and ct_multilevel-psi.h5 so the path above will need to be changed accordingly.

  22. Samuel Cupp

    Thanks! @Roland Haas Do we need to do anything with the data? Or just change the ‘last tested’ line on the website?

  23. Log in to comment