kedro.config.ConfigLoader¶

class kedro.config.ConfigLoader(conf_paths)[source]¶

Bases: object

Recursively scan the directories specified in conf_paths for configuration files with a yaml, yml, json, ini, pickle, xml or properties extension, load them, and return them in the form of a config dictionary.

When the same top-level key appears in any 2 config files located in the same conf_path (sub)directory, a ValueError is raised.

When the same key appears in any 2 config files located in different conf_path directories, the last processed config path takes precedence and overrides this key.

For example, if your conf_path looks like this:

.
`-- conf
    |-- README.md
    |-- base
    |   |-- catalog.yml
    |   |-- logging.yml
    |   `-- experiment1
    |       `-- parameters.yml
    `-- local
        |-- catalog.yml
        |-- db.ini
        |-- experiment1
        |   |-- parameters.yml
        |   `-- model_parameters.yml
        `-- experiment2
            `-- parameters.yml

You can access the different configurations as follows:

import logging.config
from kedro.config import ConfigLoader

conf_paths = ['conf/base', 'conf/local']
conf_loader = ConfigLoader(conf_paths)

conf_logging = conf_loader.get('logging*')
logging.config.dictConfig(conf_logging)  # set logging conf

conf_catalog = conf_loader.get('catalog*', 'catalog*/**')
conf_params = conf_loader.get('**/parameters.yml')

Methods

`ConfigLoader.__init__`(conf_paths)	Instantiate a ConfigLoader.
`ConfigLoader.get`(*patterns)	Recursively scan for configuration files, load and merge them, and return them in the form of a config dictionary.

__init__(conf_paths)[source]¶

Instantiate a ConfigLoader.

Parameters:	conf_paths (`Union`[`str`, `Iterable`[`str`]]) – Non-empty path or list of paths to configuration directories.
Raises:	`ValueError` – If `conf_paths` is empty.

get(*patterns)[source]¶

Recursively scan for configuration files, load and merge them, and return them in the form of a config dictionary.

Parameters:	patterns (`str`) – Glob patterns to match. Files, which names match any of the specified patterns, will be processed.
Raises:	`ValueError` – If 2 or more configuration files inside the same config path (or its subdirectories) contain the same top-level key. `MissingConfigException` – If no configuration files exist within a specified config path.
Returns:	A Python dictionary with the combined configuration from all configuration files. Note: any keys that start with _ will be ignored.
Return type:	Dict[str, Any]