kedro.io

Description

kedro.io provides functionality to read and write to a number of data sets. At core of the library is AbstractDataSet which allows implementation of various AbstractDataSets.

Data Catalog

kedro.io.DataCatalog([data_sets, feed_dict, …]) DataCatalog stores instances of AbstractDataSet implementations to provide load and save capabilities from anywhere in the program.

Data Sets

kedro.io.LambdaDataSet(load, save[, exists, …]) LambdaDataSet loads and saves data to a data set.
kedro.io.MemoryDataSet([data, copy_mode]) MemoryDataSet loads and saves data from/to an in-memory Python object.
kedro.io.PartitionedDataSet(path, dataset[, …]) PartitionedDataSet loads and saves partitioned file-like data using the underlying dataset definition.
kedro.io.IncrementalDataSet(path, dataset[, …]) IncrementalDataSet inherits from PartitionedDataSet, which loads and saves partitioned file-like data using the underlying dataset definition.
kedro.io.CachedDataSet(dataset[, version, …]) CachedDataSet is a dataset wrapper which caches in memory the data saved, so that the user avoids io operations with slow storage media.
kedro.io.DataCatalogWithDefault([data_sets, …]) A DataCatalog with a default DataSet implementation for any data set which is not registered in the catalog.

Errors

kedro.io.DataSetAlreadyExistsError DataSetAlreadyExistsError raised by DataCatalog class in case of trying to add a data set which already exists in the DataCatalog.
kedro.io.DataSetError DataSetError raised by AbstractDataSet implementations in case of failure of input/output methods.
kedro.io.DataSetNotFoundError DataSetNotFoundError raised by DataCatalog class in case of trying to use a non-existing data set.

Base Classes

kedro.io.AbstractDataSet AbstractDataSet is the base class for all data set implementations.
kedro.io.AbstractVersionedDataSet(filepath, …) AbstractVersionedDataSet is the base class for all versioned data set implementations.
kedro.io.AbstractTransformer AbstractTransformer is the base class for all transformer implementations.
kedro.io.Version This namedtuple is used to provide load and save versions for versioned data sets.