You can interact with this notebook online: Launch interactive version

Storing Simulations to HDF

You can ask TARDIS to store the state of each iteration of the simulation you are running. We show examples of how this is done:

Initialize the simulation with the tardis_example.yml configuration file.

from tardis import run_tardis
from tardis.io.atom_data.util import download_atom_data

# We download the atomic data needed to run the simulation
download_atom_data('kurucz_cd23_chianti_H_He')

# We run the simulation
simulation = run_tardis('tardis_example.yml')

/usr/share/miniconda3/envs/tardis/lib/python3.7/importlib/_bootstrap.py:219: QAWarning: pyne.data is not yet QA compliant.
  return f(*args, **kwds)

[py.warnings         ][WARNING]  /usr/share/miniconda3/envs/tardis/lib/python3.7/site-packages/traitlets/traitlets.py:3050: FutureWarning: --rc={'figure.dpi': 96} for dict-traits is deprecated in traitlets 5.0. You can pass --rc <key=value> ... multiple times to add items to a dict.
  FutureWarning,
 (warnings.py:110)
[py.warnings         ][WARNING]  /usr/share/miniconda3/envs/tardis/lib/python3.7/site-packages/tardis-2021.12.21.0.dev129+g098e24fc-py3.7.egg/tardis/plasma/properties/radiative_properties.py:92: RuntimeWarning: invalid value encountered in true_divide
  (g_lower * n_upper) / (g_upper * n_lower)
 (warnings.py:110)
[py.warnings         ][WARNING]  /usr/share/miniconda3/envs/tardis/lib/python3.7/site-packages/tardis-2021.12.21.0.dev129+g098e24fc-py3.7.egg/tardis/plasma/properties/radiative_properties.py:92: RuntimeWarning: invalid value encountered in true_divide
  (g_lower * n_upper) / (g_upper * n_lower)
 (warnings.py:110)

[py.warnings         ][WARNING]  /usr/share/miniconda3/envs/tardis/lib/python3.7/site-packages/tardis-2021.12.21.0.dev129+g098e24fc-py3.7.egg/tardis/plasma/properties/radiative_properties.py:92: RuntimeWarning: invalid value encountered in true_divide
  (g_lower * n_upper) / (g_upper * n_lower)
 (warnings.py:110)

[py.warnings         ][WARNING]  /usr/share/miniconda3/envs/tardis/lib/python3.7/site-packages/tardis-2021.12.21.0.dev129+g098e24fc-py3.7.egg/tardis/plasma/properties/radiative_properties.py:92: RuntimeWarning:

invalid value encountered in true_divide

 (warnings.py:110)

You can now use the to_hdf method, to save properties to a HDF file.

Parameters

file_path: Path where the HDF file should be stored. (Required)

path: Path inside the HDF store to store the elements. (Optional)

name: Name of the group inside HDF store, under which properties will be saved.(Optional) overwrite: If the HDF file already exists, do you overwrite the existing file (Optional, default False)

Note

Throughout this notebook, we set overwrite=True so that the notebook can be run repeatedly if needed.

simulation.to_hdf('/tmp/full_example.hdf', overwrite=True)

# The commented out code below shows an example of to_hdf with more parameters
#simulation.to_hdf(file_path='/tmp/full_example.hdf', path='/', name='simulation')

[py.warnings         ][WARNING]  /usr/share/miniconda3/envs/tardis/lib/python3.7/site-packages/pandas/core/generic.py:2505: PerformanceWarning:


your performance may suffer as PyTables will pickle object types that it cannot
map directly to c-types [inferred_type->mixed,key->block0_values] [items->Int64Index([0], dtype='int64')]


 (warnings.py:110)
[py.warnings         ][WARNING]  /usr/share/miniconda3/envs/tardis/lib/python3.7/site-packages/pandas/core/generic.py:2505: PerformanceWarning:


your performance may suffer as PyTables will pickle object types that it cannot
map directly to c-types [inferred_type->mixed,key->values] [items->None]


 (warnings.py:110)

Open the stored HDF file with pandas and print a list of its entries using the keys() method:

import pandas as pd

data = pd.HDFStore('/tmp/full_example.hdf', overwrite=True)

data.keys()

['/simulation/iterations_electron_densities',
 '/simulation/iterations_t_inner',
 '/simulation/iterations_t_rad',
 '/simulation/iterations_w',
 '/simulation/runner/emitted_packet_mask',
 '/simulation/runner/j_estimator',
 '/simulation/runner/last_interaction_in_nu',
 '/simulation/runner/last_interaction_type',
 '/simulation/runner/last_line_interaction_in_id',
 '/simulation/runner/last_line_interaction_out_id',
 '/simulation/runner/last_line_interaction_shell_id',
 '/simulation/runner/montecarlo_virtual_luminosity',
 '/simulation/runner/nu_bar_estimator',
 '/simulation/runner/output_energy',
 '/simulation/runner/output_nu',
 '/simulation/runner/packet_luminosity',
 '/simulation/runner/scalars',
 '/simulation/runner/spectrum_virtual/_frequency',
 '/simulation/runner/spectrum_virtual/luminosity',
 '/simulation/runner/spectrum_virtual/luminosity_density_lambda',
 '/simulation/runner/spectrum_virtual/scalars',
 '/simulation/runner/spectrum_virtual/wavelength',
 '/simulation/runner/spectrum_reabsorbed/_frequency',
 '/simulation/runner/spectrum_reabsorbed/luminosity',
 '/simulation/runner/spectrum_reabsorbed/luminosity_density_lambda',
 '/simulation/runner/spectrum_reabsorbed/scalars',
 '/simulation/runner/spectrum_reabsorbed/wavelength',
 '/simulation/runner/spectrum/_frequency',
 '/simulation/runner/spectrum/luminosity',
 '/simulation/runner/spectrum/luminosity_density_lambda',
 '/simulation/runner/spectrum/scalars',
 '/simulation/runner/spectrum/wavelength',
 '/simulation/plasma/abundance',
 '/simulation/plasma/atomic_mass',
 '/simulation/plasma/beta_rad',
 '/simulation/plasma/beta_sobolev',
 '/simulation/plasma/continuum_interaction_species',
 '/simulation/plasma/density',
 '/simulation/plasma/electron_densities',
 '/simulation/plasma/excitation_energy',
 '/simulation/plasma/f_lu',
 '/simulation/plasma/g',
 '/simulation/plasma/g_electron',
 '/simulation/plasma/general_level_boltzmann_factor',
 '/simulation/plasma/ion_number_density',
 '/simulation/plasma/ionization_data',
 '/simulation/plasma/j_blues',
 '/simulation/plasma/level_boltzmann_factor',
 '/simulation/plasma/level_number_density',
 '/simulation/plasma/levels',
 '/simulation/plasma/lines',
 '/simulation/plasma/lines_lower_level_index',
 '/simulation/plasma/lines_upper_level_index',
 '/simulation/plasma/metastability',
 '/simulation/plasma/nu',
 '/simulation/plasma/number_density',
 '/simulation/plasma/partition_function',
 '/simulation/plasma/phi',
 '/simulation/plasma/scalars',
 '/simulation/plasma/selected_atoms',
 '/simulation/plasma/stimulated_emission_factor',
 '/simulation/plasma/t_electrons',
 '/simulation/plasma/t_rad',
 '/simulation/plasma/tau_sobolevs',
 '/simulation/plasma/transition_probabilities',
 '/simulation/plasma/w',
 '/simulation/plasma/wavelength_cm',
 '/simulation/model/r_inner',
 '/simulation/model/scalars',
 '/simulation/model/t_radiative',
 '/simulation/model/v_inner',
 '/simulation/model/v_outer',
 '/simulation/model/w',
 '/simulation/model/homologous_density/density_0',
 '/simulation/model/homologous_density/scalars']

Access model.homologous_density.density_0 under simulation, which is a one-dimensional array

print(data['/simulation/model/homologous_density/density_0'])

0     1970.527174
1       13.360318
2       10.146658
3        7.786621
4        6.033444
5        4.717122
6        3.718946
7        2.954982
8        2.365191
9        1.906156
10       1.546154
11       1.261789
12       1.035646
13       0.854653
14       0.708918
15       0.590901
16       0.494811
17       0.416168
18       0.351490
19       0.298047
20       0.253691
dtype: float64

Scalars are stored in a scalars pandas.Series for every module. For example to access model.t_inner under simulation, one would need to do the following.

print(data['/simulation/model/scalars']['t_inner'])

10650.463255529794

Breakdown of the various to_hdf methods

Every module in TARDIS has its own to_hdf method responsible to store its own data to an HDF file.

Plasma

The following call will store every plasma property to /tmp/plasma_output.hdf under /parent/plasma

simulation.plasma.to_hdf('/tmp/plasma_output.hdf', path='parent', overwrite=True)

import pandas

plasma_data = pandas.HDFStore('/tmp/plasma_output.hdf')

plasma_data.keys()

['/parent/plasma/abundance',
 '/parent/plasma/atomic_mass',
 '/parent/plasma/beta_rad',
 '/parent/plasma/beta_sobolev',
 '/parent/plasma/continuum_interaction_species',
 '/parent/plasma/density',
 '/parent/plasma/electron_densities',
 '/parent/plasma/excitation_energy',
 '/parent/plasma/f_lu',
 '/parent/plasma/g',
 '/parent/plasma/g_electron',
 '/parent/plasma/general_level_boltzmann_factor',
 '/parent/plasma/ion_number_density',
 '/parent/plasma/ionization_data',
 '/parent/plasma/j_blues',
 '/parent/plasma/level_boltzmann_factor',
 '/parent/plasma/level_number_density',
 '/parent/plasma/levels',
 '/parent/plasma/lines',
 '/parent/plasma/lines_lower_level_index',
 '/parent/plasma/lines_upper_level_index',
 '/parent/plasma/metastability',
 '/parent/plasma/nu',
 '/parent/plasma/number_density',
 '/parent/plasma/partition_function',
 '/parent/plasma/phi',
 '/parent/plasma/scalars',
 '/parent/plasma/selected_atoms',
 '/parent/plasma/stimulated_emission_factor',
 '/parent/plasma/t_electrons',
 '/parent/plasma/t_rad',
 '/parent/plasma/tau_sobolevs',
 '/parent/plasma/transition_probabilities',
 '/parent/plasma/w',
 '/parent/plasma/wavelength_cm']

Plasma’s to_hdf method can also accept a collection parameter which can specify which types of plasma properties will be stored. For example if we wanted to only store Input plasma properties, we would do the following:

from tardis.plasma.properties.base import Input
simulation.plasma.to_hdf('/tmp/plasma_input_output.hdf', collection=[Input], overwrite=True)

import pandas

plasma_input_data = pandas.HDFStore('/tmp/plasma_input_output.hdf')

plasma_input_data.keys()

['/plasma/abundance',
 '/plasma/continuum_interaction_species',
 '/plasma/density',
 '/plasma/scalars',
 '/plasma/t_rad',
 '/plasma/w']

Model

The following call will store properties of the Radial1DModel to /tmp/model_output.hdf under /model.

simulation.model.to_hdf('/tmp/model_output.hdf', overwrite=True)

import pandas

model_data = pandas.HDFStore('/tmp/model_output.hdf')

model_data.keys()

['/model/r_inner',
 '/model/scalars',
 '/model/t_radiative',
 '/model/v_inner',
 '/model/v_outer',
 '/model/w',
 '/model/homologous_density/density_0',
 '/model/homologous_density/scalars']

MontecarloRunner

The following call will store properties of the MontecarloRunner to /tmp/runner_output.hdf under /runner.

simulation.runner.to_hdf('/tmp/runner_output.hdf', overwrite=True)

import pandas

runner_data = pandas.HDFStore('/tmp/runner_output.hdf')

runner_data.keys()

['/runner/emitted_packet_mask',
 '/runner/j_estimator',
 '/runner/last_interaction_in_nu',
 '/runner/last_interaction_type',
 '/runner/last_line_interaction_in_id',
 '/runner/last_line_interaction_out_id',
 '/runner/last_line_interaction_shell_id',
 '/runner/montecarlo_virtual_luminosity',
 '/runner/nu_bar_estimator',
 '/runner/output_energy',
 '/runner/output_nu',
 '/runner/packet_luminosity',
 '/runner/scalars',
 '/runner/spectrum_virtual/_frequency',
 '/runner/spectrum_virtual/luminosity',
 '/runner/spectrum_virtual/luminosity_density_lambda',
 '/runner/spectrum_virtual/scalars',
 '/runner/spectrum_virtual/wavelength',
 '/runner/spectrum_reabsorbed/_frequency',
 '/runner/spectrum_reabsorbed/luminosity',
 '/runner/spectrum_reabsorbed/luminosity_density_lambda',
 '/runner/spectrum_reabsorbed/scalars',
 '/runner/spectrum_reabsorbed/wavelength',
 '/runner/spectrum/_frequency',
 '/runner/spectrum/luminosity',
 '/runner/spectrum/luminosity_density_lambda',
 '/runner/spectrum/scalars',
 '/runner/spectrum/wavelength']