kedro.config.ConfigLoader

class kedro.config.ConfigLoader(conf_source, env=None, runtime_params=None, *, base_env='base', default_run_env='local')

Recursively scan directories (config paths) contained in conf_source for configuration files with a yaml, yml, json, ini, pickle, xml or properties extension, load them, and return them in the form of a config dictionary.

The first processed config path is the base directory inside conf_source. The optional env argument can be used to specify a subdirectory of conf_source to process as a config path after base.

When the same top-level key appears in any 2 config files located in the same (sub)directory, a ValueError is raised.

When the same key appears in any 2 config files located in different (sub)directories, the last processed config path takes precedence and overrides this key.

For example, if your conf_source looks like this:

.
`-- conf
    |-- README.md
    |-- base
    |   |-- catalog.yml
    |   |-- logging.yml
    |   `-- experiment1
    |       `-- parameters.yml
    `-- local
        |-- catalog.yml
        |-- db.ini
        |-- experiment1
        |   |-- parameters.yml
        |   `-- model_parameters.yml
        `-- experiment2
            `-- parameters.yml

You can access the different configurations as follows:

import logging.config
from kedro.config import ConfigLoader
from kedro.framework.project import settings

conf_path = str(project_path / settings.CONF_SOURCE)
conf_loader = ConfigLoader(conf_source=conf_path, env="local")

conf_logging = conf_loader.get('logging*')
logging.config.dictConfig(conf_logging)  # set logging conf

conf_catalog = conf_loader.get('catalog*', 'catalog*/**')
conf_params = conf_loader.get('**/parameters.yml')

Attributes

conf_paths

Property method to return deduplicated configuration paths.

Methods

get(*patterns)

Required method to get all configurations.

__init__(conf_source, env=None, runtime_params=None, *, base_env='base', default_run_env='local')

Instantiates a ConfigLoader.

Parameters

• conf_source (str) – Path to use as root directory for loading configuration.

• env (Optional[str]) – Environment that will take precedence over base.

• runtime_params (Optional[Dict[str, Any]]) – Extra parameters passed to a Kedro run.

• base_env (str) – Name of the base environment. Defaults to “base”. This is used in the conf_paths property method to construct the configuration paths.

• default_run_env (str) – Name of the base environment. Defaults to “local”. This is used in the conf_paths property method to construct the configuration paths. Can be overriden by supplying the env argument.

property conf_paths

Property method to return deduplicated configuration paths.

get(*patterns)

Required method to get all configurations.

Return type

Dict[str, Any]