wepy.orchestration.orchestrator module

exception wepy.orchestration.orchestrator.OrchestratorError

Bases: Exception

args

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

class wepy.orchestration.orchestrator.Orchestrator(orch_path=None, mode='x', append_only=False)

Bases: object

HASH_PICKLE_PROTOCOL = 3

DEFAULT_WORKDIR = '/home/salotz/tree/lab/devel/wepy/sphinx'

DEFAULT_CONFIG_NAME = 'root'

DEFAULT_NARRATION = ''

DEFAULT_MODE = 'x'

DEFAULT_CHECKPOINT_FILENAME = 'checkpoint.orch.sqlite'

ORCH_FILENAME_TEMPLATE = '{config}{narration}.orch.sqlite'

DEFAULT_ORCHESTRATION_MODE = 'x'

KV_MODE = 'r+'

SQLITE3_DEFAULT_TIMEOUT = 5

RUN_SELECT_FIELDS = ('last_cycle_idx', 'config_hash')

property mode

property append_only

close()

property db_uri

property orch_path

classmethod serialize(snapshot)

Serialize a snapshot to a compressed, encoded, pickle string representation.

Currently uses the dill module for pickling because the base pickle module is inadequate. However, it is mostly compatible and can be read natively with pickle but this usage is officially not supported. Instead use the deserialize_snapshot.

Also compresses with default zlib compression and is encoded in base64.

The object will always have a deepcopy performed on it so that all of the extraneous references to it are avoided since there is no (AFAIK) way to make sure all references to an object are deleted.

NOTE: Perhaps there is a way and that should be done (and tested) to see if it provides stable pickles (i.e. pickles that always hash to the same value). To avoid the overhead of copying large objects.

Parameters

snapshot (SimSnapshot object) – The snapshot of the simulation you want to serialize.

Returns

serial_str – Serialized string of the snapshot object

Return type

str

classmethod deserialize(serial_str)

Deserialize an unencoded string snapshot to an object.

Parameters

serial_str (str) – Serialized string of the snapshot object

Returns

snapshot – Simulation snapshot object

Return type

SimSnapshot object

set_default_sim_apparatus(sim_apparatus)

set_default_init_walkers(init_walkers)

set_default_configuration(configuration)

set_default_snapshot(snapshot)

gen_default_snapshot()

get_default_sim_apparatus()

get_default_init_walkers()

get_default_configuration()

get_default_configuration_hash()

get_default_snapshot()

get_default_snapshot_hash()

classmethod hash_snapshot(serial_str)

Parameters

serial_str –

get_snapshot(snapshot_hash)

Returns a copy of a snapshot.

Parameters

snapshot_hash –

get_configuration(config_hash)

Returns a copy of a snapshot.

Parameters

config_hash –

property snapshot_hashes

property configuration_hashes

add_snapshot(snapshot)

Parameters

snapshot –

add_serial_snapshot(serial_snapshot)

gen_start_snapshot(init_walkers)

Parameters

init_walkers –

property default_snapshot_hash

property default_snapshot

snapshot_registered(snapshot)

Check whether a snapshot is already in the database, based on the hash of it.

This serializes the snapshot so may be slow.

Parameters

snapshot (SimSnapshot object) – The snapshot object you want to query for.

snapshot_hash_registered(snapshot_hash)

Check whether a snapshot hash is already in the database.

Parameters

snapshot_hash (str) – The string hash of the snapshot.

configuration_hash_registered(config_hash)

Check whether a snapshot hash is already in the database.

Parameters

snapshot_hash (str) – The string hash of the snapshot.

add_configuration(configuration)

add_serial_configuration(serial_configuration)

property create_run_table_query

property add_run_record_query

property update_run_record_query

property delete_run_record_query

_add_run_record(start_hash, end_hash, configuration_hash, cycle_idx)

_delete_run_record(start_hash, end_hash)

_update_run_record(start_hash, end_hash, new_config_hash, new_last_cycle_idx)

register_run(start_hash, end_hash, config_hash, cycle_idx)

Parameters

• start_hash –

• end_hash –

• config_hash –

• cycle_idx (int) – The cycle of the simulation run the checkpoint was generated for.

get_run_records()

get_run_record(start_hash, end_hash)

run_last_cycle_idx(start_hash, end_hash)

run_configuration(start_hash, end_hash)

run_configuration_hash(start_hash, end_hash)

run_hashes()

run_continues(start_hash, end_hash)

Given a start hash and end hash for a run, find the run that this continues.

Parameters

• start_hash –

• end_hash –

Returns

Return type

run_id

_init_checkpoint_db(start_hash, configuration, checkpoint_dir, mode='x')

_save_checkpoint(checkpoint_snapshot, config_hash, checkpoint_db_path, cycle_idx)

Parameters

• checkpoint_snapshot –

• config_hash –

• checkpoint_db_path –

• mode – (Default value = ‘wb’)

static gen_sim_manager(start_snapshot, configuration)

Parameters

• start_snapshot –

• configuration –

run_snapshot_by_time(start_hash, run_time, n_steps, checkpoint_freq=None, checkpoint_dir=None, configuration=None, configuration_hash=None, checkpoint_mode='x')

For a finished run continue it but resetting all the state of the resampler and boundary conditions

Parameters

• start_hash –

• run_time –

• n_steps –

• checkpoint_freq – (Default value = None)

• checkpoint_dir – (Default value = None)

• configuration – (Default value = None)

• configuration_hash – (Default value = None)

• checkpoint_mode – (Default value = None)

orchestrate_snapshot_run_by_time(snapshot_hash, run_time, n_steps, checkpoint_freq=None, checkpoint_dir=None, orchestrator_path=None, configuration=None, work_dir=None, config_name=None, narration=None, mode=None, **kwargs)

Parameters

• snapshot_hash –

• run_time –

• n_steps –

• checkpoint_freq – (Default value = None)

• checkpoint_dir – (Default value = None)

• orchestrator_path – (Default value = None)

• configuration – (Default value = None)

• these can reparametrize the paths# for both the orchestrator produced# files as well as the configurationwork_dir (#) – (Default value = None)

• config_name – (Default value = None)

• narration – (Default value = None)

• mode – (Default value = None)

• extra kwargs will be passed to the# configuration.reparametrize method**kwargs (#) –

wepy.orchestration.orchestrator.reconcile_orchestrators(host_path, *orchestrator_paths)

Parameters

• template_orchestrator –

• *orchestrators –