kedro.runner.AbstractRunner¶
- class kedro.runner.AbstractRunner(is_async=False, extra_dataset_patterns=None)[source]¶
AbstractRunner
is the base class for allPipeline
runner implementations.Methods
run
(pipeline, catalog[, hook_manager, ...])Run the
Pipeline
using the datasets provided bycatalog
and save results back to the same objects.run_only_missing
(pipeline, catalog, hook_manager)Run only the missing outputs from the
Pipeline
using the datasets provided bycatalog
, and save results back to the same objects.- __init__(is_async=False, extra_dataset_patterns=None)[source]¶
Instantiates the runner class.
- Parameters:
is_async (bool) – If True, the node inputs and outputs are loaded and saved asynchronously with threads. Defaults to False.
extra_dataset_patterns (dict[str, dict[str, Any]] | None) – Extra dataset factory patterns to be added to the DataCatalog during the run. This is used to set the default datasets on the Runner instances.
- run(pipeline, catalog, hook_manager=None, session_id=None)[source]¶
Run the
Pipeline
using the datasets provided bycatalog
and save results back to the same objects.- Parameters:
pipeline (Pipeline) – The
Pipeline
to run.catalog (DataCatalog) – The
DataCatalog
from which to fetch data.hook_manager (PluginManager | None) – The
PluginManager
to activate hooks.session_id (str | None) – The id of the session.
- Raises:
ValueError – Raised when
Pipeline
inputs cannot be satisfied.- Return type:
- Returns:
Any node outputs that cannot be processed by the
DataCatalog
. These are returned in a dictionary, where the keys are defined by the node outputs.
- run_only_missing(pipeline, catalog, hook_manager)[source]¶
Run only the missing outputs from the
Pipeline
using the datasets provided bycatalog
, and save results back to the same objects.- Parameters:
pipeline (
Pipeline
) – ThePipeline
to run.catalog (
DataCatalog
) – TheDataCatalog
from which to fetch data.hook_manager (
PluginManager
) – ThePluginManager
to activate hooks.
- Raises:
ValueError – Raised when
Pipeline
inputs cannot be satisfied.- Return type:
- Returns:
Any node outputs that cannot be processed by the
DataCatalog
. These are returned in a dictionary, where the keys are defined by the node outputs.