Overall flow control of the simulation. More...
#include <ParallelGravity.h>
Public Member Functions | |
Main (CkArgMsg *m) | |
Main routine to start simulation. More... | |
Main (CkMigrateMessage *m) | |
Restart Main constructor. More... | |
void | niceExit () |
entry method to cleanly shutdown. Only used for debugging. | |
void | setupICs () |
Load particles into pieces. More... | |
void | initialForces () |
Initial calculation of forces. More... | |
void | doSimulation () |
Principal method which does all the coordination of the simulation over timesteps. More... | |
void | restart (CkCheckpointStatusMsg *msg) |
Callback to restart simulation after a checkpoint or after writing a checkpoint. More... | |
void | waitForGravity (const CkCallback &cb, double startTime, int activeRung) |
wait for gravity in the case of concurrent SPH | |
void | advanceBigStep (int) |
Take one base timestep of the simulation. More... | |
void | domainDecomp (int iPhase) |
Perform domain decomposition. More... | |
void | loadBalance (int iPhase) |
Perform load balance. More... | |
void | buildTree (int iPhase) |
Build Tree. More... | |
void | startGravity (const CkCallback &cbGravity, int iActiveRung, double *startTime) |
Routine to start self gravity; if gravity is not being calculated, then clear the accelerations. More... | |
void | externalGravity (int iActiveRung) |
Apply external gravitational field. More... | |
void | updateuDot (int iActiveRung, const double duKick[], const double dStartTime[], int bUpdateState, int bAll) |
Update time derivative of thermal energy. More... | |
void | kick (bool bClosing, int iActiveRung, int nextMaxRung, const CkCallback &cbGravity, double gravStartTime) |
Update velocities. More... | |
int | adjust (int iKickRung) |
Calculate timesteps of particles. More... | |
void | rungStats () |
Count and print out the number of particles in each timestep bin. More... | |
void | countActive (int activeRung) |
determine number of active particles in the given rung More... | |
void | emergencyAdjust (int iRung) |
Change timesteps of particles experiencing sudden gas forces. More... | |
void | starCenterOfMass () |
Main::starCenterOfMass Calculates the total mass and center of mass of all the star particles and saves them to all the available COOL structs. More... | |
void | calcEnergy (double, double, const char *) |
Calculate various energy and momentum quantities, and output them to a log file. | |
void | getStartTime () |
determine start time of simulation More... | |
void | getOutTimes () |
Read in desired output times and reshifts from a file. More... | |
int | bOutTime () |
Return true if we need to write an output. More... | |
void | writeOutput (int iStep) |
Output a snapshot. More... | |
void | outputBinary (OutputParams ¶ms, int bParaWrite, const CkCallback &cb) |
Output a Tipsy or NChilada XDR binary float array file. | |
void | cbOpen (Ck::IO::FileReadyMsg *msg) |
void | cbIOReady (Ck::IO::SessionReadyMsg *msg) |
Session is ready; write my data. | |
void | cbIOComplete (CkMessage *msg) |
All IO has completed. Close the file. | |
void | cbIOClosed (CkMessage *msg) |
File is closed. Update header for NChilada. Resume main program. More... | |
std::string | getNCNextOutput (OutputParams ¶ms) |
void | writeNCXML (std::string filename) |
void | NCXMLattrib (ofstream *desc, CkVec< std::string > *names, std::string family) |
void | updateSoft () |
Change the softening in comoving coordinates. More... | |
void | growMass (double dTime, double dDelta) |
Slowly increase mass of a subset of particles. | |
void | initSph () |
initialize SPH quantities More... | |
void | initCooling () |
Initialize cooling constants and integration data structures. | |
void | initStarLog () |
int | ReadASCII (char *extension, int nDataPerLine, double *dDataOut) |
function from PKDGRAV to read an ASCII table More... | |
void | restartGas () |
Read in array files for complete gas information. | |
void | doSph (int activeRung, int bNeedDensity=1) |
Perform the SPH force calculation. More... | |
void | AGORAfeedbackPreCheck (double dTime, double dDelta, double dTimeToSF) |
This routine is called when AGORA feedback is enabled. It checks for any star particles that will have a feedback event in the next timestep and puts the neighboring gas particles onto a smaller timestep. Because the amount of energy injected is very large, particles need to be placed on a small timestep BEFORE the first force calculation is done to avoid significant integration errors. More... | |
void | FormStars (double dTime, double dDelta) |
void | StellarFeedback (double dTime, double dDelta) |
void | outputBlackHoles (double dTime) |
Output black hole orbit information. More... | |
void | SetSink () |
Initial identify sinks. | |
void | FormSinks (double dTime, double dDelta, int iKickRung) |
Form sink particles; main routine. More... | |
void | doSinks (double dTime, double dDelta, int iKickRung) |
Process sink particles. | |
int | DumpFrameInit (double dTime, double dStep, int bRestart) |
void | DumpFrame (double dTime, double dStep) |
int | nextMaxRungIncDF (int nextMaxRung) |
void | addDelParticles () |
Coalesce all added and deleted particles and update global quantities. More... | |
void | memoryStats () |
void | memoryStatsCache () |
void | pup (PUP::er &p) |
void | liveVizImagePrep (liveVizRequestMsg *msg) |
void | doSIDM (double dTime, double dDelta, int activeRung) |
Main method to perform Self Interacting Dark Matter interactions. More... | |
Overall flow control of the simulation.
As well as controlling the overall flow of the simulation, the constructors are the main entry points into the program. The sequence of tasks is: read the simulation parameters (Main()), read in the initial conditions (setupICs()), calculate the initial forces (initialForces()), then iterate across timesteps and write the final output (doSimulation()).
Main::Main | ( | CkArgMsg * | m | ) |
Main routine to start simulation.
This routine parses the command line and file specified parameters, and allocates the charm structures. The charm "readonly" variables are writable in this method, and are broadcast globally once this method exits. Note that the method finishes with an asynchronous call to setupICs().
Main::Main | ( | CkMigrateMessage * | m | ) |
Restart Main constructor.
This is only called when restarting from a checkpoint.
void Main::addDelParticles | ( | ) |
Coalesce all added and deleted particles and update global quantities.
This is large, but no larger than the counts message
int Main::adjust | ( | int | iKickRung | ) |
Calculate timesteps of particles.
Particles on the KickRung and shorter have their timesteps adjusted. Calls TreePiece::adjust().
iKickRung | Rung (and above) about to be kicked. |
void Main::advanceBigStep | ( | int | iStep | ) |
Take one base timestep of the simulation.
iStep | The current step number. |
This method implements the standard "Kick Drift Kick" (Quinn et al 1997) hierarchical timestepping algorithm. It assumes that the forces for the first opening kick have already been calculated.
void Main::AGORAfeedbackPreCheck | ( | double | dTime, |
double | dDelta, | ||
double | dTimeToSF | ||
) |
This routine is called when AGORA feedback is enabled. It checks for any star particles that will have a feedback event in the next timestep and puts the neighboring gas particles onto a smaller timestep. Because the amount of energy injected is very large, particles need to be placed on a small timestep BEFORE the first force calculation is done to avoid significant integration errors.
dTime | The current simulation time (in years) |
dDelta | The size of the next timestep (in years) |
dTimeToSF | The time until the next star formation event (in years) |
int Main::bOutTime | ( | ) |
Return true if we need to write an output.
Advances iOut attribute, therefore this can only be called once per timestep.
void Main::buildTree | ( | int | iPhase | ) |
Build Tree.
iPhase | Active rung (or phase). |
void Main::cbIOClosed | ( | CkMessage * | msg | ) |
File is closed. Update header for NChilada. Resume main program.
Continue to next particle type
void Main::cbOpen | ( | Ck::IO::FileReadyMsg * | msg | ) |
Determine offsets for all pieces, then start the write session. This needs to be a threaded entry method.
void Main::countActive | ( | int | activeRung | ) |
determine number of active particles in the given rung
Calls TreePiece::countActive() and sets Main::nActiveGrav and Main::nActiveSPH.
void Main::domainDecomp | ( | int | iPhase | ) |
Perform domain decomposition.
Active | rung (or phase). |
void Main::doSIDM | ( | double | dTime, |
double | dDelta, | ||
int | activeRung | ||
) |
Main method to perform Self Interacting Dark Matter interactions.
dTime | current simulation time |
dDelta | timestep over which to calculate interactions |
activeRung | timestep rung corresponding to dDelta |
void Main::doSimulation | ( | ) |
Principal method which does all the coordination of the simulation over timesteps.
This routine calls advanceBigStep() for each step, logs statistics, determines if output is needed, and halts the simulation when done.
void Main::doSph | ( | int | activeRung, |
int | bNeedDensity = 1 |
||
) |
Perform the SPH force calculation.
activeRung | Timestep rung (and above) on which to perform SPH |
bNeedDensity | Does the density calculation need to be done? Defaults to 1 |
void Main::emergencyAdjust | ( | int | iRung | ) |
Change timesteps of particles experiencing sudden gas forces.
iRung | The rung on which we are calculating forces. |
For gas simulations, find particles who are are in the middle of too large a timestep and adjust their velocities to a smaller timestep.
void Main::externalGravity | ( | int | iActiveRung | ) |
Apply external gravitational field.
iActiveRung | Rung on which to apply forces. |
void Main::FormSinks | ( | double | dTime, |
double | dDelta, | ||
int | iKickRung | ||
) |
Form sink particles; main routine.
dTime | Current time. |
dDelta | Current timestep. |
iKickRung | Rung being kicked. |
Calls TreePiece::formSinks.
void Main::FormStars | ( | double | dTime, |
double | dDelta | ||
) |
form stars main method
std::string Main::getNCNextOutput | ( | OutputParams & | params | ) |
Return the file name of the next nchilada output to write (gas, dark or star). Returns empty if we are done. Also advances the iTypeWriting in params.
void Main::getOutTimes | ( | ) |
Read in desired output times and reshifts from a file.
Fills in the vdOutTime vector by reading the .red file
void Main::getStartTime | ( | ) |
determine start time of simulation
Function to determine the start time of the simulation. Modifies dTime member of main.
void Main::initialForces | ( | ) |
Initial calculation of forces.
This is called both when starting or restarting a run. It concludes by calling doSimulation(), the main simulation loop.
void Main::initSph | ( | ) |
initialize SPH quantities
Initial calculation of densities and internal energies, and cooling rates.
void Main::kick | ( | bool | bClosing, |
int | iActiveRung, | ||
int | nextMaxRung, | ||
const CkCallback & | cbGravity, | ||
double | gravStartTime | ||
) |
Update velocities.
bClosing | Is this a closing kick? |
iActiveRung | Rung (and higher) which to update |
nextMaxRung | Rung of smallest timestep to be taken |
cbGravity | Callback to wait for gravity (close only) |
gravStartTime | Timing start for gravity |
Based on bClosing the velocities of particles on rung iActiveRung through nextMaxRung are either moved to a 1/2 step based on their rung (open) or moved from the 1/2 step to the end of the step (close).
void Main::loadBalance | ( | int | iPhase | ) |
Perform load balance.
iPhase | Active rung (or phase). -1 indicates initial LB. |
void Main::memoryStats | ( | ) |
Diagnostic function to summmarize memory usage across all processors
void Main::memoryStatsCache | ( | ) |
Diagnostic function to summmarize cache memory usage across all processors
void Main::outputBlackHoles | ( | double | dTime | ) |
Output black hole orbit information.
int Main::ReadASCII | ( | char * | extension, |
int | nDataPerLine, | ||
double * | dDataOut | ||
) |
function from PKDGRAV to read an ASCII table
extension | Appended to outName to determine file name to read. |
nDataPerLine | Number of columns in the table. |
dDataOut | pointer to array in which to store the table. Note if dDataOut is NULL it just counts the number of valid input lines. |
void Main::restart | ( | CkCheckpointStatusMsg * | msg | ) |
Callback to restart simulation after a checkpoint or after writing a checkpoint.
A restart looks like we've just finished writing a checkpoint. We can tell the difference by the bIsRestarting flag set in the Main CkMigrate constructor. In that case we do some simple parameter parsing and go to InitialForces. Otherwise we return to the doSimulation() loop.
void Main::rungStats | ( | ) |
Count and print out the number of particles in each timestep bin.
This routine is for information only.
void Main::setupICs | ( | ) |
Load particles into pieces.
Reads the particle data in from a file. Since the full information about the run (including starting time) isn't known until the particles are loaded, this routine also completes the specification of the run details and writes out the log file entry. It concludes by calling initialForces()
void Main::starCenterOfMass | ( | ) |
Main::starCenterOfMass Calculates the total mass and center of mass of all the star particles and saves them to all the available COOL structs.
Requires that cooling for planets be enabled at compile time.
void Main::startGravity | ( | const CkCallback & | cbGravity, |
int | iActiveRung, | ||
double * | startTime | ||
) |
Routine to start self gravity; if gravity is not being calculated, then clear the accelerations.
cbGravity | Callback if we overlapping gravity with SPH. |
iActiveRung | Rung (and higher) on which to calculate forces. |
pointer | to start time |
void Main::StellarFeedback | ( | double | dTime, |
double | dDelta | ||
) |
Feedback main method
void Main::updateSoft | ( | ) |
Change the softening in comoving coordinates.
When compiled with -DCHANGESOFT, and bPhysicalSoft is set, the (comoving) softening is changed so that it is constant in physical units.
void Main::updateuDot | ( | int | iActiveRung, |
const double | duKick[], | ||
const double | dStartTime[], | ||
int | bUpdateState, | ||
int | bAll | ||
) |
Update time derivative of thermal energy.
iActiveRung | Rung (and higher) which to update |
duKick | Array of timesteps per rung |
dStartTime | Array of beginning times (simulation units) per rung |
bUpdateState | Whether to update the ionization fractions |
bAll | Whether to update all rungs below iActiveRung |
void Main::writeOutput | ( | int | iStep | ) |
Output a snapshot.
iStep | Timestep we are outputting, used for file name. |