itwinai.parser

Provide functionalities to manage configuration files, including parsing, execution, and dynamic override of fields.

itwinai.parser.add_replace_field(config: Dict, key_chain: str, value: Any) → None

Replace or add (if not present) a field in a dictionary, following a path of dot-separated keys. Adding is not supported for list items. Inplace operation.

Parameters:

• config (Dict) – dictionary to be modified.

• key_chain (str) – path of nested (dot-separated) keys to specify the location of the new value (e.g., ‘foo.bar.line’ adds/overwrites the value located at config[‘foo’][‘bar’][‘line’]).

• value (Any) – the value to insert.

class itwinai.parser.ConfigParser(config: str | Dict, override_keys: Dict[str, Any] | None = None)

Bases: object

Parses a pipeline from a configuration file. It also provides functionalities for dynamic override of fields by means of nested key notation.

Parameters:

• config (Union[str, Dict]) – path to YAML configuration file or dict storing a configuration.

• override_keys (Optional[Dict[str, Any]], optional) – dict mapping nested keys to the value to override. Defaults to None.

Example:

>>> # pipeline.yaml file
>>> pipeline:
>>>   class_path: itwinai.pipeline.Pipeline
>>>   init_args:
>>>     steps:
>>>       - class_path: dataloader.MNISTDataModuleTorch
>>>         init_args:
>>>           save_path: .tmp/
>>>
>>>       - class_path: itwinai.torch.trainer.TorchTrainer
>>>         init_args:
>>>           model:
>>>             class_path: model.Net
>>>
>>> from itwinai.parser import ConfigParser
>>>
>>> parser = ConfigParser(
>>>     config='pipeline.yaml',
>>>     override_keys={
>>>         'pipeline.init_args.steps.0.init_args.save_path': /save/path
>>>     }
>>> )
>>> pipeline = parser.parse_pipeline()
>>> print(pipeline)
>>> print(pipeline.steps)
>>>
>>> dataloader = parser.parse_step(0)
>>> print(dataloader)
>>> print(dataloader.save_path)

pipeline: Pipeline

Pipeline object instantiated from the configuration file.

config: Dict

Configuration to parse.

parse_pipeline(pipeline_nested_key: str = 'pipeline', verbose: bool = False) → Pipeline

Merges steps into pipeline and parses it.

Parameters:

• pipeline_nested_key (str, optional) – nested key in the configuration file identifying the pipeline object. Defaults to “pipeline”.

• verbose (bool) – if True, prints the assembled pipeline to console formatted as JSON.

Returns:

instantiated pipeline.

Return type:

Pipeline

parse_step(step_idx: str | int, pipeline_nested_key: str = 'pipeline', verbose: bool = False) → BaseComponent

class itwinai.parser.ArgumentParser(*args, env_prefix: bool | str = True, formatter_class: ~typing.Type[~jsonargparse._formatters.DefaultHelpFormatter] = <class 'jsonargparse._formatters.DefaultHelpFormatter'>, exit_on_error: bool = True, logger: bool | str | dict | ~logging.Logger = False, version: str | None = None, print_config: str | None = '--print_config', parser_mode: str = 'yaml', dump_header: ~typing.List[str] | None = None, default_config_files: ~typing.List[str | ~os.PathLike] | None = None, default_env: bool = False, default_meta: bool = True, **kwargs)

Bases: ArgumentParser

Wrapper of jsonargparse.ArgumentParser. Initializer for ArgumentParser instance. It can parse arguments from a series of configuration files. Example:

>>> python main.py --config base-conf.yaml --config other-conf.yaml \
>>> --param OVERRIDE_VAL

All the arguments from the initializer of argparse.ArgumentParser are supported. Additionally it accepts:

Parameters:

• env_prefix (Union[bool, str], optional) – Prefix for environment variables. True to derive from prog.. Defaults to True.

• formatter_class (Type[DefaultHelpFormatter], optional) – Class for printing help messages. Defaults to DefaultHelpFormatter.

• exit_on_error (bool, optional) – Defaults to True.

• logger (Union[bool, str, dict, logging.Logger], optional) – Configures the logger, see LoggerProperty. Defaults to False.

• version (Optional[str], optional) – Program version which will be printed by the –version argument. Defaults to None.

• print_config (Optional[str], optional) – Add this as argument to print config, set None to disable. Defaults to “–print_config”.

• parser_mode (str, optional) – Mode for parsing config files: 'yaml', 'jsonnet' or ones added via set_loader().. Defaults to “yaml”.

• dump_header (Optional[List[str]], optional) – Header to include as comment when dumping a config object. Defaults to None.

• default_config_files – (Optional[List[Union[str, os.PathLike]]], optional): Default config file locations, e.g. ['~/.config/myapp/*.yaml']. Defaults to None.

• default_env (bool, optional) – Set the default value on whether to parse environment variables. Defaults to False.

• default_meta (bool, optional) – Set the default value on whether to include metadata in config objects. Defaults to True.