wepy.orchestration.orchestrator module

exception wepy.orchestration.orchestrator.OrchestratorError

Bases: Exception

args

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

class wepy.orchestration.orchestrator.Orchestrator(orch_path=None, mode='x', append_only=False)

Bases: object

HASH_PICKLE_PROTOCOL = 3

DEFAULT_WORKDIR = '/Users/berkatik/Desktop/wepy/sphinx'

DEFAULT_CONFIG_NAME = 'root'

DEFAULT_NARRATION = ''

DEFAULT_MODE = 'x'

DEFAULT_CHECKPOINT_FILENAME = 'checkpoint.orch.sqlite'

ORCH_FILENAME_TEMPLATE = '{config}{narration}.orch.sqlite'

DEFAULT_ORCHESTRATION_MODE = 'x'

KV_MODE = 'r+'

SQLITE3_DEFAULT_TIMEOUT = 5

RUN_SELECT_FIELDS = ('last_cycle_idx', 'config_hash')

property mode

property append_only

close()

property db_uri

property orch_path

classmethod serialize(snapshot)

Serialize a snapshot to a compressed, encoded, pickle string representation.

Currently uses the dill module for pickling because the base pickle module is inadequate. However, it is mostly compatible and can be read natively with pickle but this usage is officially not supported. Instead use the deserialize_snapshot.

Also compresses with default zlib compression and is encoded in base64.

The object will always have a deepcopy performed on it so that all of the extraneous references to it are avoided since there is no (AFAIK) way to make sure all references to an object are deleted.

NOTE: Perhaps there is a way and that should be done (and tested) to see if it provides stable pickles (i.e. pickles that always hash to the same value). To avoid the overhead of copying large objects.

Parameters:

snapshot (SimSnapshot object) – The snapshot of the simulation you want to serialize.

Returns:

serial_str – Serialized string of the snapshot object

Return type:

str

classmethod deserialize(serial_str)

Deserialize an unencoded string snapshot to an object.

Parameters:

serial_str (str) – Serialized string of the snapshot object

Returns:

snapshot – Simulation snapshot object

Return type:

SimSnapshot object

set_default_sim_apparatus(sim_apparatus)

set_default_init_walkers(init_walkers)

set_default_configuration(configuration)

set_default_snapshot(snapshot)

gen_default_snapshot()

get_default_sim_apparatus()

get_default_init_walkers()

get_default_configuration()

get_default_configuration_hash()

get_default_snapshot()

get_default_snapshot_hash()

classmethod hash_snapshot(serial_str)

Parameters:

serial_str

get_snapshot(snapshot_hash)

Returns a copy of a snapshot.

Parameters:

snapshot_hash

get_configuration(config_hash)

Returns a copy of a snapshot.

Parameters:

config_hash

property snapshot_hashes

property configuration_hashes

add_snapshot(snapshot)

Parameters:

snapshot

add_serial_snapshot(serial_snapshot)

gen_start_snapshot(init_walkers)

Parameters:

init_walkers

property default_snapshot_hash

property default_snapshot

snapshot_registered(snapshot)

Check whether a snapshot is already in the database, based on the hash of it.

This serializes the snapshot so may be slow.

Parameters:

snapshot (SimSnapshot object) – The snapshot object you want to query for.

snapshot_hash_registered(snapshot_hash)

Check whether a snapshot hash is already in the database.

Parameters:

snapshot_hash (str) – The string hash of the snapshot.

configuration_hash_registered(config_hash)

Check whether a snapshot hash is already in the database.

Parameters:

snapshot_hash (str) – The string hash of the snapshot.

add_configuration(configuration)

add_serial_configuration(serial_configuration)

property create_run_table_query

property add_run_record_query

property update_run_record_query

property delete_run_record_query

_add_run_record(start_hash, end_hash, configuration_hash, cycle_idx)

_delete_run_record(start_hash, end_hash)

_update_run_record(start_hash, end_hash, new_config_hash, new_last_cycle_idx)

register_run(start_hash, end_hash, config_hash, cycle_idx)

Parameters:

• start_hash

• end_hash

• config_hash

• cycle_idx (int) – The cycle of the simulation run the checkpoint was generated for.

get_run_records()

get_run_record(start_hash, end_hash)

run_last_cycle_idx(start_hash, end_hash)

run_configuration(start_hash, end_hash)

run_configuration_hash(start_hash, end_hash)

run_hashes()

run_continues(start_hash, end_hash)

Given a start hash and end hash for a run, find the run that this continues.

Parameters:

• start_hash

• end_hash

Return type:

run_id

_init_checkpoint_db(start_hash, configuration, checkpoint_dir, mode='x')

_save_checkpoint(checkpoint_snapshot, config_hash, checkpoint_db_path, cycle_idx)

Parameters:

• checkpoint_snapshot

• config_hash

• checkpoint_db_path

• mode – (Default value = ‘wb’)

static gen_sim_manager(start_snapshot, configuration)

Parameters:

• start_snapshot

• configuration

run_snapshot_by_time(start_hash, run_time, n_steps, checkpoint_freq=None, checkpoint_dir=None, configuration=None, configuration_hash=None, checkpoint_mode='x')

For a finished run continue it but resetting all the state of the resampler and boundary conditions

Parameters:

• start_hash

• run_time

• n_steps

• checkpoint_freq – (Default value = None)

• checkpoint_dir – (Default value = None)

• configuration – (Default value = None)

• configuration_hash – (Default value = None)

• checkpoint_mode – (Default value = None)

orchestrate_snapshot_run_by_time(snapshot_hash, run_time, n_steps, checkpoint_freq=None, checkpoint_dir=None, orchestrator_path=None, configuration=None, work_dir=None, config_name=None, narration=None, mode=None, **kwargs)

Parameters:

• snapshot_hash

• run_time

• n_steps

• checkpoint_freq – (Default value = None)

• checkpoint_dir – (Default value = None)

• orchestrator_path – (Default value = None)

• configuration – (Default value = None)

• configurationwork_dir (# these can reparametrize the paths# for both the orchestrator produced# files as well as the) – (Default value = None)

• config_name – (Default value = None)

• narration – (Default value = None)

• mode – (Default value = None)

• method**kwargs (# extra kwargs will be passed to the# configuration.reparametrize)

wepy.orchestration.orchestrator.reconcile_orchestrators(host_path, *orchestrator_paths)

Parameters:

• template_orchestrator

• *orchestrators