kedro.io.MemoryDataSet¶
-
class
kedro.io.
MemoryDataSet
(data=<object object>, copy_mode=None)[source]¶ Bases:
kedro.io.core.AbstractDataSet
MemoryDataSet
loads and saves data from/to an in-memory Python object.Example:
from kedro.io import MemoryDataSet import pandas as pd data = pd.DataFrame({'col1': [1, 2], 'col2': [4, 5], 'col3': [5, 6]}) data_set = MemoryDataSet(data=data) loaded_data = data_set.load() assert loaded_data.equals(data) new_data = pd.DataFrame({'col1': [1, 2], 'col2': [4, 5]}) data_set.save(new_data) reloaded_data = data_set.load() assert reloaded_data.equals(new_data)
Attributes
Methods
Checks whether a data set’s output already exists by calling the provided _exists() method.
MemoryDataSet.from_config
(name, config[, …])Create a data set instance using the configuration provided.
Loads data by delegation to the provided load method.
Release any cached data.
MemoryDataSet.save
(data)Saves data by delegation to the provided save method.
-
__init__
(data=<object object>, copy_mode=None)[source]¶ Creates a new instance of
MemoryDataSet
pointing to the provided Python object.- Parameters
data (
Any
) – Python object containing the data.copy_mode (
Optional
[str
]) – The copy mode used to copy the data. Possible values are: “deepcopy”, “copy” and “assign”. If not provided, it is inferred based on the data type.
-
exists
()¶ Checks whether a data set’s output already exists by calling the provided _exists() method.
- Return type
bool
- Returns
Flag indicating whether the output already exists.
- Raises
DataSetError – when underlying exists method raises error.
-
classmethod
from_config
(name, config, load_version=None, save_version=None)¶ Create a data set instance using the configuration provided.
- Parameters
name (
str
) – Data set name.config (
Dict
[str
,Any
]) – Data set config dictionary.load_version (
Optional
[str
]) – Version string to be used forload
operation if the data set is versioned. Has no effect on the data set if versioning was not enabled.save_version (
Optional
[str
]) – Version string to be used forsave
operation if the data set is versioned. Has no effect on the data set if versioning was not enabled.
- Return type
AbstractDataSet
- Returns
An instance of an
AbstractDataSet
subclass.- Raises
DataSetError – When the function fails to create the data set from its config.
-
load
()¶ Loads data by delegation to the provided load method.
- Return type
Any
- Returns
Data returned by the provided load method.
- Raises
DataSetError – When underlying load method raises error.
-
release
()¶ Release any cached data.
- Raises
DataSetError – when underlying release method raises error.
- Return type
None
-
save
(data)¶ Saves data by delegation to the provided save method.
- Parameters
data (
Any
) – the value to be saved by provided save method.- Raises
DataSetError – when underlying save method raises error.
- Return type
None
-