wepy.reporter.reporter module

exception wepy.reporter.reporter.ReporterError

Bases: Exception

args

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

class wepy.reporter.reporter.Reporter(**kwargs)

Bases: object

Abstract base class for wepy reporters.

All reporters must customize and override minimally the ‘report’ method. Optionally the ‘init’ and ‘cleanup’ can be overriden.

See also

wepy.sim_manager

details of calls to reporter methods.

Construct a reporter.

Void constructor for the Reporter base class.

Parameters

**kwargs (key-value pairs) – Ignored kwargs, but accepts them from subclass calls for compatibility.

init(**kwargs)

Initialization routines for the reporter at simulation runtime.

Initialize I/O connections including file descriptors, database connections, timers, stdout/stderr etc.

Void method for reporter base class.

Reporters can expect to have the following key word arguments passed to them during a simulation by the sim_manager in this call.

Parameters

• init_walkers (list of Walker objects) – The initial walkers for the simulation.

• runner (Runner object) – The runner that will be used in the simulation.

• resampler (Resampler object) – The resampler that will be used in the simulation.

• boundary_conditions (BoundaryConditions object) – The boundary conditions taht will be used in the simulation.

• work_mapper (WorkMapper object) – The work mapper that will be used in the simulation.

• reporters (list of Reporter objects) – The list of reporters that are in the simulation.

• continue_run (int) – The index of the run that is being continued within this same file.

report(**kwargs)

Given data concerning the main simulation components state, perform I/O operations to persist that data.

Void method for reporter base class.

Reporters can expect to have the following key word arguments passed to them during a simulation by the sim_manager.

Parameters

• cycle_idx (int) –

• new_walkers (list of Walker objects) – List of walkers that were produced from running their dynamics by the runner.

• warp_data (list of dict of str : value) – List of dict-like records for each warping event from the last cycle.

• bc_data (list of dict of str : value) – List of dict-like records specifying the changes to the state of the boundary conditions in the last cycle.

• progress_data (dict str : list) – A record indicating the progress values for each walker in the last cycle.

• resampling_data (list of dict of str : value) – List of records specifying the resampling to occur at this cycle.

• resampler_data (list of dict of str : value) – List of records specifying the changes to the state of the resampler in the last cycle.

• n_segment_steps (int) – The number of dynamics steps that were completed in the last cycle

• worker_segment_times (dict of int : list of float) – Mapping worker index to the times they took for each segment they processed.

• cycle_runner_time (float) – Total time runner took in last cycle.

• cycle_bc_time (float) – Total time boundary conditions took in last cycle.

• cycle_resampling_time (float) – Total time resampler took in last cycle.

• resampled_walkers (list of Walker objects) – List of walkers that were produced from the new_walkers from applying resampling and boundary conditions.

cleanup(**kwargs)

Teardown routines for the reporter at the end of the simulation.

Use to cleanly and safely close I/O connections or other cleanup I/O.

Use to close file descriptors, database connections etc.

Reporters can expect to have the following key word arguments passed to them during a simulation by the sim_manager.

Parameters

• runner (Runner object) – The runner at the end of the simulation

• work_mapper (WorkeMapper object) – The work mapper at the end of the simulation

• resampler (Resampler object) – The resampler at the end of the simulation

• boundary_conditions (BoundaryConditions object) – The boundary conditions at the end of the simulation

• reporters (list of Reporter objects) – The list of reporters at the end of the simulation

class wepy.reporter.reporter.FileReporter(file_paths=None, modes=None, file_path=None, mode=None, **kwargs)

Bases: wepy.reporter.reporter.Reporter

Abstract reporter that handles specifying file paths for a reporter.

This abstract class doesn’t perform any operations that involve actually opening file descriptors, but only the validation and organization of file paths.

This provides a uniform API for retrieving file paths from all reporters inheriting from it.

Additionally, FileReporter implements an interface for performing a so-called reparametrization of the relevant values associated with each file specification (i.e. file path and mode).

A reparametrization can be performed by calling the ‘reparametrize’ method, and can be customized.

Additionally, there are some customizable class constants than can be used in subclasses to control this process including: DEFAULT_MODE, SUGGESTED_FILENAME_TEMPLATE, DEFAULT_SUGGESTED_EXTENSION, FILE_ORDER, and SUGGESTED_EXTENSIONS.

The intention is to allow the redefinition of file paths dynamically to adapt to changing runtime requirements. Such as execution on a separate subtree of a directory hierarchy.

Constructor for FileReporter.

This constructor allows the specification of either a list of file names (and modes) via ‘file_paths’ and ‘modes’ key-word arguments or a single ‘file_path’ and ‘mode’.

The access API though is always a list of file paths and modes where order is important for associating other features.

Parameters

• file_paths (list of str) – The list of file paths (in order) to use.

• modes (list of str) – The list of mode specs (in order) to use.

• file_path (str) – If ‘file_paths’ not specified, the single file path to use.

• mode (str) – If ‘file_path’ option used, this is the mode for that file.

MODES = ('x', 'w', 'w-', 'r', 'r+')

Valid modes accepted for files.

DEFAULT_MODE = 'x'

The default mode to set for opening files if none is specified (create if doesn’t exist, fail if it does.)

SUGGESTED_FILENAME_TEMPLATE = '{config}{narration}{reporter_class}.{ext}'

Template to use for dynamic reparametrization of file path names.

The fields in the template are:

config : indicator of the runtime configuration used

narration : freeform description of the instance

reporter_classthe name of the class that produced the

output. When no specific name is given for a file report generated from a reporter this is used to disambiguate, along with the extension.

ext : The file extension, for multiple files produced from one reporter this should be sufficient to disambiguate the files.

The ‘config’ and ‘narration’ should be the same across all reporters in the same simulation manager, and the ‘narration’ is considered optional.

DEFAULT_SUGGESTED_EXTENSION = 'report'

The default file extension used for files during dynamic reparametrization, if none is specified

FILE_ORDER = ()

Specify an ordering of file paths. Should be customized.

SUGGESTED_EXTENSIONS = ()

Suggested extensions for file paths for use with the automatic reparametrization feature. Should be customized.

_validate_mode(mode)

Check if the mode spec is a valid one.

Parameters

mode (str) –

Returns

valid

Return type

bool

property mode

For single file path reporters the mode of that file.

property file_path

For single file path reporters the file path to that file spec.

cleanup(**kwargs)

Teardown routines for the reporter at the end of the simulation.

Use to cleanly and safely close I/O connections or other cleanup I/O.

Use to close file descriptors, database connections etc.

Reporters can expect to have the following key word arguments passed to them during a simulation by the sim_manager.

Parameters

• runner (Runner object) – The runner at the end of the simulation

• work_mapper (WorkeMapper object) – The work mapper at the end of the simulation

• resampler (Resampler object) – The resampler at the end of the simulation

• boundary_conditions (BoundaryConditions object) – The boundary conditions at the end of the simulation

• reporters (list of Reporter objects) – The list of reporters at the end of the simulation

property file_paths

The file paths for this reporter, in order.

init(**kwargs)

Initialization routines for the reporter at simulation runtime.

Initialize I/O connections including file descriptors, database connections, timers, stdout/stderr etc.

Void method for reporter base class.

Reporters can expect to have the following key word arguments passed to them during a simulation by the sim_manager in this call.

Parameters

• init_walkers (list of Walker objects) – The initial walkers for the simulation.

• runner (Runner object) – The runner that will be used in the simulation.

• resampler (Resampler object) – The resampler that will be used in the simulation.

• boundary_conditions (BoundaryConditions object) – The boundary conditions taht will be used in the simulation.

• work_mapper (WorkMapper object) – The work mapper that will be used in the simulation.

• reporters (list of Reporter objects) – The list of reporters that are in the simulation.

• continue_run (int) – The index of the run that is being continued within this same file.

report(**kwargs)

Given data concerning the main simulation components state, perform I/O operations to persist that data.

Void method for reporter base class.

Reporters can expect to have the following key word arguments passed to them during a simulation by the sim_manager.

Parameters

• cycle_idx (int) –

• new_walkers (list of Walker objects) – List of walkers that were produced from running their dynamics by the runner.

• warp_data (list of dict of str : value) – List of dict-like records for each warping event from the last cycle.

• bc_data (list of dict of str : value) – List of dict-like records specifying the changes to the state of the boundary conditions in the last cycle.

• progress_data (dict str : list) – A record indicating the progress values for each walker in the last cycle.

• resampling_data (list of dict of str : value) – List of records specifying the resampling to occur at this cycle.

• resampler_data (list of dict of str : value) – List of records specifying the changes to the state of the resampler in the last cycle.

• n_segment_steps (int) – The number of dynamics steps that were completed in the last cycle

• worker_segment_times (dict of int : list of float) – Mapping worker index to the times they took for each segment they processed.

• cycle_runner_time (float) – Total time runner took in last cycle.

• cycle_bc_time (float) – Total time boundary conditions took in last cycle.

• cycle_resampling_time (float) – Total time resampler took in last cycle.

• resampled_walkers (list of Walker objects) – List of walkers that were produced from the new_walkers from applying resampling and boundary conditions.

set_path(file_idx, path)

Set the path for a single indexed file.

Parameters

• file_idx (int) – Index in the listing of files.

• path (str) – The new path to set for this file

property modes

The modes for the files, in order.

set_mode(file_idx, mode)

Set the mode for a single indexed file.

Parameters

• file_idx (int) – Index in the listing of files.

• mode (str) – The new mode spec.

reparametrize(file_paths, modes)

Set the file paths and modes for all files in the reporter.

Parameters

• file_paths (list of str) – New file paths for each file, in order.

• modes (list of str) – New modes for each file, in order.

class wepy.reporter.reporter.ProgressiveFileReporter(file_paths=None, modes=None, file_path=None, mode=None, **kwargs)

Bases: wepy.reporter.reporter.FileReporter

Super class for a reporter that will successively overwrite the same file over and over again. The base FileReporter really only supports creation of file one time.

Constructor for FileReporter.

This constructor allows the specification of either a list of file names (and modes) via ‘file_paths’ and ‘modes’ key-word arguments or a single ‘file_path’ and ‘mode’.

The access API though is always a list of file paths and modes where order is important for associating other features.

Parameters

• file_paths (list of str) – The list of file paths (in order) to use.

• modes (list of str) – The list of mode specs (in order) to use.

• file_path (str) – If ‘file_paths’ not specified, the single file path to use.

• mode (str) – If ‘file_path’ option used, this is the mode for that file.

DEFAULT_MODE = 'x'

The default mode to set for opening files if none is specified (create if doesn’t exist, fail if it does.)

DEFAULT_SUGGESTED_EXTENSION = 'report'

The default file extension used for files during dynamic reparametrization, if none is specified

FILE_ORDER = ()

Specify an ordering of file paths. Should be customized.

MODES = ('x', 'w', 'w-', 'r', 'r+')

Valid modes accepted for files.

SUGGESTED_EXTENSIONS = ()

Suggested extensions for file paths for use with the automatic reparametrization feature. Should be customized.

SUGGESTED_FILENAME_TEMPLATE = '{config}{narration}{reporter_class}.{ext}'

Template to use for dynamic reparametrization of file path names.

The fields in the template are:

config : indicator of the runtime configuration used

narration : freeform description of the instance

reporter_classthe name of the class that produced the

output. When no specific name is given for a file report generated from a reporter this is used to disambiguate, along with the extension.

ext : The file extension, for multiple files produced from one reporter this should be sufficient to disambiguate the files.

The ‘config’ and ‘narration’ should be the same across all reporters in the same simulation manager, and the ‘narration’ is considered optional.

_validate_mode(mode)

Check if the mode spec is a valid one.

Parameters

mode (str) –

Returns

valid

Return type

bool

cleanup(**kwargs)

Teardown routines for the reporter at the end of the simulation.

Use to cleanly and safely close I/O connections or other cleanup I/O.

Use to close file descriptors, database connections etc.

Reporters can expect to have the following key word arguments passed to them during a simulation by the sim_manager.

Parameters

• runner (Runner object) – The runner at the end of the simulation

• work_mapper (WorkeMapper object) – The work mapper at the end of the simulation

• resampler (Resampler object) – The resampler at the end of the simulation

• boundary_conditions (BoundaryConditions object) – The boundary conditions at the end of the simulation

• reporters (list of Reporter objects) – The list of reporters at the end of the simulation

property file_path

For single file path reporters the file path to that file spec.

property file_paths

The file paths for this reporter, in order.

property mode

For single file path reporters the mode of that file.

property modes

The modes for the files, in order.

reparametrize(file_paths, modes)

Set the file paths and modes for all files in the reporter.

Parameters

• file_paths (list of str) – New file paths for each file, in order.

• modes (list of str) – New modes for each file, in order.

report(**kwargs)

Given data concerning the main simulation components state, perform I/O operations to persist that data.

Void method for reporter base class.

Reporters can expect to have the following key word arguments passed to them during a simulation by the sim_manager.

Parameters

• cycle_idx (int) –

• new_walkers (list of Walker objects) – List of walkers that were produced from running their dynamics by the runner.

• warp_data (list of dict of str : value) – List of dict-like records for each warping event from the last cycle.

• bc_data (list of dict of str : value) – List of dict-like records specifying the changes to the state of the boundary conditions in the last cycle.

• progress_data (dict str : list) – A record indicating the progress values for each walker in the last cycle.

• resampling_data (list of dict of str : value) – List of records specifying the resampling to occur at this cycle.

• resampler_data (list of dict of str : value) – List of records specifying the changes to the state of the resampler in the last cycle.

• n_segment_steps (int) – The number of dynamics steps that were completed in the last cycle

• worker_segment_times (dict of int : list of float) – Mapping worker index to the times they took for each segment they processed.

• cycle_runner_time (float) – Total time runner took in last cycle.

• cycle_bc_time (float) – Total time boundary conditions took in last cycle.

• cycle_resampling_time (float) – Total time resampler took in last cycle.

• resampled_walkers (list of Walker objects) – List of walkers that were produced from the new_walkers from applying resampling and boundary conditions.

set_mode(file_idx, mode)

Set the mode for a single indexed file.

Parameters

• file_idx (int) – Index in the listing of files.

• mode (str) – The new mode spec.

set_path(file_idx, path)

Set the path for a single indexed file.

Parameters

• file_idx (int) – Index in the listing of files.

• path (str) – The new path to set for this file

init(**kwargs)

Construct a ProgressiveFileReporter.

This is exactly the same as the FileReporter.

Parameters

• file_paths (list of str) – The list of file paths (in order) to use.

• modes (list of str) – The list of mode specs (in order) to use.

• file_path (str) – If ‘file_paths’ not specified, the single file path to use.

• mode (str) – If ‘file_path’ option used, this is the mode for that file.

See also

wepy.reporter.reporter.FileReporter