Opened 16 months ago

Last modified 21 hours ago

#2064 review enhancement

Add GiRaFFE to the Einstein Toolkit

Reported by: zachetie@… Owned by: Roland Haas
Priority: major Milestone: ET_2018_08
Component: EinsteinToolkit thorn Version: development version
Keywords: GiRaFFE, GRFFE, IllinoisGRMHD Cc:


We (the authors of GiRaFFE) hereby request that GiRaFFE be added to the Einstein Toolkit. In short, GiRaFFE is the first open source, dynamical-spacetime code for modeling plasma flows in general relativistic force-free electrodynamics (GRFFE). It will be the best code in the Toolkit for modeling black hole and neutron star magnetospheres, as these magnetically-dominated regions are best modeled numerically by a GRFFE code (GRMHD codes break down when modeling very highly magnetized plasmas).

Here is the commit ID for code reviewer comments:

Here is the original release announcement to the mailing list, including link to the code release paper:

We are pleased to announce the public release of GiRaFFE, an open-source General Relativistic Force-Free Electrodynamics (GRFFE) code for dynamical spacetimes.

-= Code Release Paper =-
Our code release paper--including a full description of the GiRaFFE code, a discussion of potential applications of GiRaFFE, and code validation test results--has been uploaded to the arXiv: .

-= Executive Summary =-
GiRaFFE adopts the strategy pioneered by McKinney [1] and modified by Paschalidis and Shapiro [2] to convert a general relativistic magnetohydrodynamic (GRMHD) code into a GRFFE code. In short, GiRaFFE exists as an extensive modification to IllinoisGRMHD [3]. Both GiRaFFE and IllinoisGRMHD leverage the Einstein Toolkit’s highly-scalable infrastructure [4,5] to make possible large-scale simulations of magnetized plasmas in strong, dynamical spacetimes on adaptive-mesh refinement (AMR) grids.

As you are aware, IllinoisGRMHD is an open-source rewrite of the Illinois Numerical Relativity Group's GRMHD code. GiRaFFE, on the other hand, is not based on any existing GRFFE code and is designed only to solve the equations of GRFFE in the context of fixed or dynamically-generated spacetimes. GiRaFFE has passed a large suite of GRFFE code validation tests passed by other state-of-the-art GRFFE codes, leading us to conclude that GiRaFFE is now ready for large-scale, state-of-the-art GRFFE simulations in dynamical spacetimes.

-= New Thorns for the Einstein Toolkit =-
All GiRaFFE thorns are listed below. Next to the name of each thorn, we include the number of lines of code to highlight the fact that GiRaFFE is quite compact and designed to be welcoming to new users. In addition, the comment-to-line-of-code ratio is very nearly 50%, and the code release paper provides a clean, broad-brush overview of the formalism and algorithms adopted. To become acquainted with any given thorn, first take a look at its README file.

  • GiRaFFE (~1,600 lines of code): Solves the GRFFE equations in the formalism of Paschalidis & Shapiro [2] using the same reconstruction (PPM) and staggered vector potential methods as IllinoisGRMHD.
  • GiRaFFEfood (~1,300 lines of code): Code validation initial data and basic diagnostics. Contains

Five 1D tests in flat spacetime (fast wave, Alfvén wave, degenerate Alfvén wave, "three-wave", FFE breakdown),
One 3D test in flat spacetime (aligned rotator), and
Three 3D tests in curved spacetimes ("Exact Wald" solution, "Magnetospheric Wald" solution, and split monopole).

  • ShiftedKerrSchild (~230 lines of code): Sets up all 3+1 quantities for a Kerr-Schild spacetime with an arbitrary radial shift, to mitigate effects of extreme spacetime curvature near r_{KS}=0. See Appendix of code release paper for full discussion.
  • VolumeIntegrals_slab_Gfood (~310 lines of code): Performs volume integrals over arbitrary rectangular prisms as specified in parameter files.
  • GiRaFFE_to_HydroBase & ID_converter_GiRaFFE (~230 lines of code): Acts as a translation layer to convert 3-velocities in HydroBase's Valencia formalism to/from the 3-velocity found in the induction equation (vi = ui/u0; native to GiRaFFE).

-= License =-
To maximize compatibility with existing thorns in ET, all of the aforementioned GiRaFFE thorns are either licensed GPLv2+ or 2-clause BSD (FreeBSD). See LICENSE file in each.

-= Download GiRaFFE! =-
The collection of thorns comprising GiRaFFE may be downloaded here:


The GiRaFFE team

-Zach Etienne
-Mew-Bing Wan
-Maria Babiuc
-Sean McWilliams
-Ashok Choudhary

[1] J.C. McKinney. Mon. Not. R. Astron. Soc., 367:1797, 2006.
[2] V. Paschalidis and S. L. Shapiro. Phys. Rev. D, 88:104031, 2013.
[3] Z. B. Etienne, V. Paschalidis, R. Haas, P. M¨osta, and S. L. Shapiro. Class. Quantum Grav., 32(17):175009, 2015.

Attachments (1)

ILGRMHD_GiRaFFE.diff (165.8 KB) - added by Roland Haas 3 months ago.
a patch file comparing GiRaFFE to IllinoisGRMHD

Download all attachments as: .zip

Change History (17)

comment:1 Changed 16 months ago by Roland Haas

Owner: set to Roland Haas
Priority: unsetmajor
Status: newassigned

comment:2 Changed 16 months ago by Roland Haas

Status: assignedreview

comment:3 Changed 15 months ago by Roland Haas

comment:4 Changed 10 months ago by Ian Hinder

Milestone: ET_2018_02

comment:5 Changed 10 months ago by Ian Hinder

Milestone: ET_2018_02ET_2018_08

Unfortunately, the code is not yet reviewed, so we have to bump it to the next release. We will try very hard to get this reviewed shortly after the release, added to master, and then included in the next release.

comment:6 Changed 3 months ago by Roland Haas

This is taking me a very long time to review, sorry. I think I will try and see if I can review just the differences to IllinosGRMHD (which has already been reviewed).

Changed 3 months ago by Roland Haas

Attachment: ILGRMHD_GiRaFFE.diff added

a patch file comparing GiRaFFE to IllinoisGRMHD

comment:7 Changed 3 months ago by Roland Haas

I added a diff of IllinoisGRMHD to GiRaFFE comparing either files of the same name (small changes only indeed) or names that are similar but one contains GRFFE in it (larger changes), and excluding the TeX documentation and README files. That brings the diff down to ~3200 lines.

comment:8 Changed 3 months ago by Roland Haas

Status: reviewreopened

Done with review of GiRaFFE proper, no "Please apply" since there are too many (small) issues that should at least be looked at. Nothing that I can point a finger at as being definitely wrong, though in some places comments and code contradict each other.

comment:9 Changed 3 months ago by Roland Haas

This was discussed in today's ET call and Roland has no objections about including GiRaFFE, GiRaFFE_food, ID_converter_GiRaFFE

comment:10 Changed 3 months ago by zachetie@…

GiRaFFE_to_HydroBase and ShiftedKerrSchild are also required thorns. The former populates HydroBase variables based on GiRaFFE variables -- needed for certain diagnostics. The latter is needed by GiRaFFEfood.

comment:11 Changed 3 months ago by anonymous

No objections against those two either (though I think some may be missing explicit documentation in their .tex files).

comment:12 Changed 22 hours ago by Roland Haas

The thorns were included in git hash 62866e1 "GiRaFFE review:" of wvuthorns and added to the thornlist in b963885 "Add GiRaFFE thorn family" of the maninfest.

Documentation tex files are still missing as far as I can tell.

comment:13 Changed 21 hours ago by Roland Haas

Zach says (offline due to not yet having an external CCT account):

Documentation for GiRaFFE is located in the GiRaFFE/doc directory (you'll
see documentation.tex and references.bib). It can be built with the usual:

pdflatex documentation.tex
pdflatex documentation.tex
bibtex documentation.aux
pdflatex documentation.tex

The documentation build system currently does not support BibTeX, which
might explain why the GiRaFFE documentation isn't appearing on your radar.

comment:14 Changed 21 hours ago by Roland Haas

Are you ok if I copy and paste you email with "Zach says" to the ticket?

The GiRaFFE docs are fine (or at least there is nothing in there that I
did not already comment on during the actual review for GiRaFFE). The
docs I was referring to in the ticket at those for the helper thorns:

  • GiRaFFE_to_HydroBase
  • ShiftedKerrSchild

Please note that I have not checked right now if there is actual docs
for those two. In both cases the docs should be very short, for the
ShiftedKerrSchild one should be able to just copy&paste from the
GiRaFFE paper and for GiRaFFE_to_HydroBase this is really something
that one may not even need since most likley you already have the info
in the actual GiRaFFE docs (the separate thorn is only there so that
one can run without HydroBase analysis and get a bit more speed?).

These are *very* minor and should *only* be done *after* the WVUThorns
issues you wanted to work on this week.

comment:15 Changed 21 hours ago by Roland Haas

Zach says:

I was tempted to mention this during today's telecon, but
GiRaFFE_to_HydroBase is destined for extinction. One of the coauthors of
the GiRaFFE paper is working to incorporate this thorn into GiRaFFE itself,
so that GiRaFFE uses the HydroBase variables whenever possible, and when it
is not possible, to update the HydroBase variables continuously (with very
little additional cost, it turns out). It's still a WIP, but here's the
pull request:

Documentation for ShiftedKerrSchild should be a matter of simply
copy-pasting some of the text from our paper (basic equations) into the
doc/ directory. I'll put this on my to-do list after the outstanding
WVUThorns_diagnostics items.

comment:16 Changed 21 hours ago by Roland Haas

Status: reopenedreview

Modify Ticket

Change Properties
Set your email in Preferences
as review The owner will remain Roland Haas.
Next status will be 'reviewed_ok'.
as The resolution will be set.
The resolution will be deleted.
to The owner will be changed from Roland Haas to the specified user.
to The owner will be changed from Roland Haas to the specified user.
The owner will be changed from Roland Haas to anonymous.

Add Comment

E-mail address and name can be saved in the Preferences.

Note: See TracTickets for help on using tickets.