kedro.io¶
Description
kedro.io
provides functionality to read and write to a
number of data sets. At core of the library is AbstractDataSet
which allows implementation of various AbstractDataSet
s.
Data Catalog¶
kedro.io.DataCatalog ([data_sets, feed_dict, …]) |
DataCatalog stores instances of AbstractDataSet implementations to provide load and save capabilities from anywhere in the program. |
Data Sets¶
kedro.io.LambdaDataSet (load, save[, exists, …]) |
LambdaDataSet loads and saves data to a data set. |
kedro.io.MemoryDataSet ([data, copy_mode]) |
MemoryDataSet loads and saves data from/to an in-memory Python object. |
kedro.io.PartitionedDataSet (path, dataset[, …]) |
PartitionedDataSet loads and saves partitioned file-like data using the underlying dataset definition. |
kedro.io.IncrementalDataSet (path, dataset[, …]) |
IncrementalDataSet inherits from PartitionedDataSet , which loads and saves partitioned file-like data using the underlying dataset definition. |
kedro.io.CachedDataSet (dataset[, version, …]) |
CachedDataSet is a dataset wrapper which caches in memory the data saved, so that the user avoids io operations with slow storage media. |
kedro.io.DataCatalogWithDefault ([data_sets, …]) |
A DataCatalog with a default DataSet implementation for any data set which is not registered in the catalog. |
Errors¶
kedro.io.DataSetAlreadyExistsError |
DataSetAlreadyExistsError raised by DataCatalog class in case of trying to add a data set which already exists in the DataCatalog . |
kedro.io.DataSetError |
DataSetError raised by AbstractDataSet implementations in case of failure of input/output methods. |
kedro.io.DataSetNotFoundError |
DataSetNotFoundError raised by DataCatalog class in case of trying to use a non-existing data set. |
Base Classes¶
kedro.io.AbstractDataSet |
AbstractDataSet is the base class for all data set implementations. |
kedro.io.AbstractVersionedDataSet (filepath, …) |
AbstractVersionedDataSet is the base class for all versioned data set implementations. |
kedro.io.AbstractTransformer |
Transformers will be deprecated in Kedro 0.18.0 in favour of the Dataset Hooks. |
kedro.io.Version |
This namedtuple is used to provide load and save versions for versioned data sets. |