kedro.config.ConfigLoader¶
-
class
kedro.config.
ConfigLoader
(conf_paths)[source]¶ Bases:
object
Recursively scan the directories specified in
conf_paths
for configuration files with ayaml
,yml
,json
,ini
,pickle
,xml
orproperties
extension, load them, and return them in the form of a config dictionary.When the same top-level key appears in any 2 config files located in the same
conf_path
(sub)directory, aValueError
is raised.When the same key appears in any 2 config files located in different
conf_path
directories, the last processed config path takes precedence and overrides this key.For example, if your
conf_path
looks like this:. `-- conf |-- README.md |-- base | |-- catalog.yml | |-- logging.yml | `-- experiment1 | `-- parameters.yml `-- local |-- catalog.yml |-- db.ini |-- experiment1 | |-- parameters.yml | `-- model_parameters.yml `-- experiment2 `-- parameters.yml
You can access the different configurations as follows:
import logging.config from kedro.config import ConfigLoader conf_paths = ['conf/base', 'conf/local'] conf_loader = ConfigLoader(conf_paths) conf_logging = conf_loader.get('logging*') logging.config.dictConfig(conf_logging) # set logging conf conf_catalog = conf_loader.get('catalog*', 'catalog*/**') conf_params = conf_loader.get('**/parameters.yml')
Methods
ConfigLoader.__init__
(conf_paths)Instantiate a ConfigLoader. ConfigLoader.get
(*patterns)Recursively scan for configuration files, load and merge them, and return them in the form of a config dictionary. -
__init__
(conf_paths)[source]¶ Instantiate a ConfigLoader.
Parameters: conf_paths ( Union
[str
,Iterable
[str
]]) – Non-empty path or list of paths to configuration directories.Raises: ValueError
– Ifconf_paths
is empty.
-
get
(*patterns)[source]¶ Recursively scan for configuration files, load and merge them, and return them in the form of a config dictionary.
Parameters: patterns (
str
) – Glob patterns to match. Files, which names match any of the specified patterns, will be processed.Raises: ValueError
– If 2 or more configuration files inside the same config path (or its subdirectories) contain the same top-level key.MissingConfigException
– If no configuration files exist within a specified config path.
Returns: - A Python dictionary with the combined
configuration from all configuration files. Note: any keys that start with _ will be ignored.
Return type: Dict[str, Any]
-