kedro.io.MemoryDataSet¶

class kedro.io.MemoryDataSet(data=<object object>)[source]¶

Bases: kedro.io.core.AbstractDataSet

MemoryDataSet loads and saves data from/to an in-memory Python object.

Example:

from kedro.io import MemoryDataSet
import pandas as pd

data = pd.DataFrame({'col1': [1, 2], 'col2': [4, 5],
                     'col3': [5, 6]})
data_set = MemoryDataSet(data=data)

loaded_data = data_set.load()
assert loaded_data.equals(data)

new_data = pd.DataFrame({'col1': [1, 2], 'col2': [4, 5]})
data_set.save(new_data)
reloaded_data = data_set.load()
assert reloaded_data.equals(new_data)

Methods

`MemoryDataSet.__init__`([data])	Creates a new instance of `MemoryDataSet` pointing to the provided Python object.
`MemoryDataSet.exists`()	Checks whether a data set’s output already exists by calling the provided _exists() method.
`MemoryDataSet.from_config`(name, config[, …])	Create a data set instance using the configuration provided.
`MemoryDataSet.get_last_load_version`()	Versioned datasets should override this property to return last loaded version
`MemoryDataSet.get_last_save_version`()	Versioned datasets should override this property to return last saved version.
`MemoryDataSet.load`()	Loads data by delegation to the provided load method.
`MemoryDataSet.release`()	Release any cached data.
`MemoryDataSet.save`(data)	Saves data by delegation to the provided save method.

__init__(data=<object object>)[source]¶

Creates a new instance of MemoryDataSet pointing to the provided Python object.

Parameters:	data (`Any`) – Python object containing the data.

exists()¶

Checks whether a data set’s output already exists by calling the provided _exists() method.

Return type:	`bool`
Returns:	Flag indicating whether the output already exists.
Raises:	`DataSetError` – when underlying exists method raises error.

classmethod from_config(name, config, load_version=None, save_version=None)¶

Create a data set instance using the configuration provided.

Parameters:	name (`str`) – Data set name. config (`Dict`[`str`, `Any`]) – Data set config dictionary. load_version (`Optional`[`str`]) – Version string to be used for `load` operation if the data set is versioned. Has no effect on the data set if versioning was not enabled. save_version (`Optional`[`str`]) – Version string to be used for `save` operation if the data set is versioned. Has no effect on the data set if versioning was not enabled.
Return type:	`AbstractDataSet`
Returns:	An instance of an `AbstractDataSet` subclass.
Raises:	`DataSetError` – When the function fails to create the data set from its config.

get_last_load_version()¶

Versioned datasets should override this property to return last loaded version

Return type:	`Optional`[`str`]

get_last_save_version()¶

Versioned datasets should override this property to return last saved version.

Return type:	`Optional`[`str`]

load()¶

Loads data by delegation to the provided load method.

Return type:	`Any`
Returns:	Data returned by the provided load method.
Raises:	`DataSetError` – When underlying load method raises error.

release()¶

Release any cached data.

Raises:	`DataSetError` – when underlying exists method raises error.
Return type:	`None`

save(data)¶

Saves data by delegation to the provided save method.

Parameters:	data (`Any`) – the value to be saved by provided save method.
Raises:	`DataSetError` – when underlying save method raises error.
Return type:	`None`