ExecuteWorkflowApp#

ExecuteWorkflowApp with inherited methods#

class pydidas.apps.ExecuteWorkflowApp(*args, **kwargs)#

Bases: BaseApp

Inherits from pydidas.apps.BaseApp.

The ExecuteWorkflowApp is used to run workflows which can be any operation which can be written as Plugin.

Parameters can be passed either through the argparse module during command line calls or through keyword arguments in scripts.

The names of the parameters are similar for both cases, only the calling syntax changes slightly, based on the underlying structure used. For the command line, parameters must be passed as -<parameter name> <value>. For keyword arguments, parameters must be passed during instantiation using the keyword argument syntax of standard <parameter name>=<value>.

Notes

The full list of Parameters used by the ExecuteWorkflowApp:

autosave_resultsbool, optional: Use this flag to control whether result data should be automatically saved to disk. The default is False.
autosave_directoryUnion[pathlib.Path, str], optional: The directory for autosave_files. If autosave_results is True, the directory must be set.
autosave_formatstr: The formats for saving the results. This must be chosen from the list of available formats.
live_processingbool, optional: Flag to enable live processing. This will implement checks on file existance before processing starts. The default is False.

The “sig_results_updated” signal will be emitted upon a new update of the stored result and can be used

Parameters:

*args (tuple) – Any number of Parameters. These will be added to the app’s ParameterCollection.
**kwargs (dict) – Parameters supplied with their reference key as dict key and the Parameter itself as value.

add_param(param: Parameter)#

Add a parameter to the ParameterCollection.

This is a wrapper for the ParameterCollection.add_parameter method.

Parameters:: param (Parameter) – An instance of a Parameter object.

add_params(*params: tuple[Parameter | dict | ParameterCollection])#

Add parameters to the object.

This method adds Parameters to the ParameterCollection of the object. Parameters can be either supplies as args or a ParameterCollection or dictionary in the form of <ref_key>: <Parameter>. This method is explicitly separate from the __init__ method to allow subclasses full control over args and kwargs.

Parameters:: *params (Tuple[Union[Parameter, dict, ParameterCollection]]) – Any Parameter or ParameterCollection

copy(slave_mode: bool = False) → Self#

Get a copy of the App.

Parameters:: slave_mode (bool, optional) – Keyword to toggle creation of a slave app which does not include attributes marked in the classes slave_mode attribute.
Returns:: App – A copy of the App instance.
Return type:: BaseApp

deepcopy() → Self#

Get a deepcopy of the object.

Returns:: The object’s deepcopy.
Return type:: ObjectWithParameterCollection

export_state() → dict#

Get the sanitized app Parameters and configuration for export.

Returns:: The state dictionary.
Return type:: dict

get_config() → dict#

Get the App configuration.

Returns:: The App configuration stored in the _config dictionary.
Return type:: dict

classmethod get_default_params_copy() → ParameterCollection#

Get a copy of the default ParameterCollection.

Returns:: A copy of the default ParameterCollection.
Return type:: ParameterCollection

get_param(param_key: str) → Parameter#

Get a parameter.

Note: This method returns the Parameter itself, not a copy.

Parameters:: param_key (str) – The key name of the Parameter.
Returns:: The Parameter object.
Return type:: Parameter

get_param_keys() → list[str]#

Get the keys of all registered Parameters.

Returns:: The keys of all registered Parameters.
Return type:: list

get_param_value(param_key: str, *default: object, dtype: type | None = None, for_export: bool = False) → object#

Get a Parameter value.

Parameters:

param_key (str) – The key name of the Parameter.
default (object) – The default value if the param_key does not exist.
dtype (type, optional) – A datatype to convert the value into. If None, the native datatype is returned. The default is None.
for_export (bool, optional) – An optional flag to force converting the Parameter value to an export-compatible format. This flag is not compatible with a specific dtype. The default is False.

Returns:

The value of the Parameter.

Return type:

object

get_param_values_as_dict(filter_types_for_export: bool = False) → dict#

Get a dictionary with Parameter names and values only.

Parameters:: filter_types_for_export (bool) – Flag to return objects in types suitable for exporting (i.e. pickleable).
Returns:: name_val_pairs – The dictionary with Parameter <name>: <value> pairs.
Return type:: dict

get_params(*param_keys: tuple[str, ...]) → list[Parameter]#

Get multiple parameters based on their reference keys.

Parameters:: *param_keys (Tuple[str, ...]) – Any number of reference keys.
Returns:: A list with the Parameter instances referenced by the supplied keys.
Return type:: list

import_state(state: dict)#

Import a stored state including Parameters and configuration.

Parameters:: state (dict) – The stored state.

initialize_shared_memory()#: Initialize the shared arrays from the buffer size and result shapes.

multiprocessing_carryon() → bool#

Get the flag value whether to carry on processing.

By default, this Flag is always True. In the case of live processing, a check is done whether the current file exists.

Note: This method is a dummy which will be overwritten in the prepare_run method depending on the settings for the live processing.

Returns:: Flag whether the processing can carry on or needs to wait.
Return type:: bool

multiprocessing_func(index: int) → int | Tuple[int, dict]#

Perform key operation with parallel processing.

Parameters:: index (int) – The task index to be executed.
Returns:: _image – The (pre-processed) image.
Return type:: pydidas.core.Dataset

multiprocessing_get_tasks() → ndarray#

Return all tasks required in multiprocessing.

Returns:: The specified input tasks.
Return type:: np.ndarray

multiprocessing_post_run()#

Perform operations after running main parallel processing function.

The generic method simple performs no operation and subclasses only need to re-implement it when they explicitly need it.

multiprocessing_pre_cycle(index: int)#

Store the reference to the frame index internally.

Parameters:: index (int) – The index of the image / frame.

multiprocessing_pre_run()#: Perform operations prior to running main parallel processing function.

multiprocessing_store_results(index: int, *data: tuple)#

Store the results of the multiprocessing operation.

Parameters:

index (int) – The index of the processed task.
data (tuple) – The results from multiprocessing_func. This can be either a tuple with (buffer_pos, metadata) or the integer buffer_pos.

property param_keys: list[str]#

Get the keys of all stored Parameters.

Returns:: The keys of all stored Parameters.
Return type:: list[str]

property param_values: dict#

Get the values of all stored Parameters along with their refkeys.

Returns:: The refkey, value pairs for all stored Parameters.
Return type:: Dict

parse_args_and_set_params()#: Parse the command line arguments and update the corresponding Parameter values.

parse_func() → dict#

Parse the command line arguments for the ExecuteWorkflowApp.

Parameters:: caller (object, optional) – If this function is called by a class as method, it requires a single argument which corresponds to the instance.
Returns:: A dictionary with the parsed arugments which holds all the entries and entered values or - if missing - the default values.
Return type:: dict

prepare_run()#

Prepare running the workflow execution.

For the main App (i.e. running not in slave_mode), this involves the following steps:

Get the shape of all results from the WorkflowTree and store them for internal reference.

Get all multiprocessing tasks from the ScanContext.

Calculate the required buffer size and verify that the memory requirements are okay.

Initialize the shared memory arrays.

Both the slaved and the main applications then initialize local numpy arrays from the shared memory.

print_param_values()#: Print the name and value of all Parameters.

q_settings_get(key: str, dtype: type | None = None, default: object | None = None) → object#

Get the value from a QSetting key.

Parameters:

key (str) – The QSetting reference key.
dtype (Union[type, None], optional) – A return datatype. If not None, the output will be returned as dtype(value), otherwise, the generic string/int will be returned. The default is None.
default (type, optional) – The default value which is returned if the key defaults to None. The default is None.

Returns:

value – The value, converted to the type associated with the Parameter referenced by param_key or dtype, if given.

Return type:

object

q_settings_set(key: str, value: object)#

Set the value of a QSettings key.

Parameters:

key (str) – The name of the key.
value (object) – The value to be stored.

reset_runtime_vars()#: Reset the runtime variables for a new run.

restore_all_defaults(confirm: bool = False)#

Restore the default values to all entries.

Parameters:: confirm (bool) – Confirmation flag as safety feature.

run()#: Run the app serially without multiprocessing support.

set_default_params()#

Set default entries.

This method will go through the supplied defaults iterable. If there are no entries for the Parameter keys, it will add a Parameter with default value.

set_param_value(param_key: str, value: object)#

Set a parameter value.

Parameters:

param_key (str) – The key name of the Parameter.
value (object) – The value to be set. This has to be the datatype associated with the Parameter.

set_param_values(**kwargs: dict)#

Set multiple parameter values at once.

Parameters:: **kwargs (dict) – The reference key and value pairs for all Parameters to be set.

set_param_values_from_dict(value_dict: dict)#

Set the Parameter values from a dict with name, value paris.

Parameters:: value_dict (dict) – The dictionary with the stored information.

update_param_values_from_kwargs(**kwargs: dict)#

Update the Parameter values corresponding to the given keys.

Parameters:: **kwargs (dict) – The dictionary with Parameter refkeys and values.

update_params_from_init(*args: tuple, **kwargs: dict)#

Update the Parameters from the given init args and kwargs.

Parameters:

*args (Tuple) – The input arguments.
**kwargs (Dict) – The input keyword arguments.