kedro.config.ConfigLoader¶
-
class
kedro.config.
ConfigLoader
(conf_paths)[source]¶ Recursively scan the directories specified in
conf_paths
for configuration files with ayaml
,yml
,json
,ini
,pickle
,xml
orproperties
extension, load them, and return them in the form of a config dictionary.When the same top-level key appears in any 2 config files located in the same
conf_path
(sub)directory, aValueError
is raised.When the same key appears in any 2 config files located in different
conf_path
directories, the last processed config path takes precedence and overrides this key.For example, if your
conf_path
looks like this:. `-- conf |-- README.md |-- base | |-- catalog.yml | |-- logging.yml | `-- experiment1 | `-- parameters.yml `-- local |-- catalog.yml |-- db.ini |-- experiment1 | |-- parameters.yml | `-- model_parameters.yml `-- experiment2 `-- parameters.yml
You can access the different configurations as follows:
import logging.config from kedro.config import ConfigLoader conf_paths = ['conf/base', 'conf/local'] conf_loader = ConfigLoader(conf_paths) conf_logging = conf_loader.get('logging*') logging.config.dictConfig(conf_logging) # set logging conf conf_catalog = conf_loader.get('catalog*', 'catalog*/**') conf_params = conf_loader.get('**/parameters.yml')
Methods
get
(*patterns)Recursively scan for configuration files, load and merge them, and return them in the form of a config dictionary.
-
__init__
(conf_paths)[source]¶ Instantiate a ConfigLoader.
- Parameters
conf_paths (
Union
[str
,Iterable
[str
]]) – Non-empty path or list of paths to configuration directories.- Raises
ValueError – If
conf_paths
is empty.
-
get
(*patterns)[source]¶ Recursively scan for configuration files, load and merge them, and return them in the form of a config dictionary.
- Parameters
*patterns – Glob patterns to match. Files, which names match any of the specified patterns, will be processed.
- Raises
ValueError – If 2 or more configuration files inside the same config path (or its subdirectories) contain the same top-level key.
MissingConfigException – If no configuration files exist within a specified config path.
BadConfigException – If configuration is poorly formatted and cannot be loaded.
- Returns
- A Python dictionary with the combined
configuration from all configuration files. Note: any keys that start with _ will be ignored.
- Return type
Dict[str, Any]
-