kedro.config.ConfigLoader¶
-
class
kedro.config.
ConfigLoader
(conf_source, env=None, runtime_params=None, *, base_env='base', default_run_env='local')[source]¶ Recursively scan directories (config paths) contained in
conf_source
for configuration files with ayaml
,yml
,json
,ini
,pickle
,xml
orproperties
extension, load them, and return them in the form of a config dictionary.The first processed config path is the
base
directory insideconf_source
. The optionalenv
argument can be used to specify a subdirectory ofconf_source
to process as a config path afterbase
.When the same top-level key appears in any 2 config files located in the same (sub)directory, a
ValueError
is raised.When the same key appears in any 2 config files located in different (sub)directories, the last processed config path takes precedence and overrides this key.
For example, if your
conf_source
looks like this:. `-- conf |-- README.md |-- base | |-- catalog.yml | |-- logging.yml | `-- experiment1 | `-- parameters.yml `-- local |-- catalog.yml |-- db.ini |-- experiment1 | |-- parameters.yml | `-- model_parameters.yml `-- experiment2 `-- parameters.yml
You can access the different configurations as follows:
import logging.config from kedro.config import ConfigLoader from kedro.framework.project import settings conf_path = str(project_path / settings.CONF_SOURCE) conf_loader = ConfigLoader(conf_source=conf_path, env="local") conf_logging = conf_loader.get('logging*') logging.config.dictConfig(conf_logging) # set logging conf conf_catalog = conf_loader.get('catalog*', 'catalog*/**') conf_params = conf_loader.get('**/parameters.yml')
Attributes
Property method to return deduplicated configuration paths.
Methods
get
(*patterns)Required method to get all configurations.
-
__init__
(conf_source, env=None, runtime_params=None, *, base_env='base', default_run_env='local')[source]¶ Instantiates a
ConfigLoader
.- Parameters
conf_source (
str
) – Path to use as root directory for loading configuration.env (
Optional
[str
]) – Environment that will take precedence over base.runtime_params (
Optional
[Dict
[str
,Any
]]) – Extra parameters passed to a Kedro run.base_env (
str
) – Name of the base environment. Defaults to “base”. This is used in the conf_paths property method to construct the configuration paths.default_run_env (
str
) – Name of the base environment. Defaults to “local”. This is used in the conf_paths property method to construct the configuration paths. Can be overriden by supplying the env argument.
-
property
conf_paths
¶ Property method to return deduplicated configuration paths.
-