Create a Kedro starter¶
Kedro starters are used to create projects that contain code to run as-is, or to adapt and extend. A good example is the Iris dataset example of basic Kedro project layout, configuration and initialisation code. A team may find it useful to build Kedro starters to create reusable projects that bootstrap a common base and can be extended.
A Kedro starter is a Cookiecutter template that contains the boilerplate code for a Kedro project.
How to create a Kedro starter¶
To create a Kedro starter, you need a base project to convert to a cookiecutter
template, which forms the boilerplate for all projects that use the Kedro starter.
Install cookiecutter
as follows:
pip install cookiecutter
You then need to decide which are:
the common, boilerplate parts of the project
the configurable elements, which need to be replaced by
cookiecutter
strings
Configuration variables¶
By default, when you create a new project using a Kedro starter, kedro new
launches in interactive mode. The user is then prompted for the variables that have been set in prompts.yml
.
The most basic and empty starter triggered by kedro new
is set up with the following three variables:
project_name
- A human readable name for the new projectrepo_name
- A name for the directory that holds the project repositorypython_package
- A Python package name for the project package (see Python package naming conventions)
See the configuration for this basic configuration in the default starter setup.
As the creator of the Kedro starter you can customise the prompts triggered by kedro new
by adding your own prompts in prompts.yml
. This is an example of a custom prompt:
custom_prompt:
title: "Prompt title:"
text: |
Prompt description that explains to the user what
information they should provide.
At the very least, the prompt title
must be defined for the prompt to be valid. After Kedro gets the user’s input for each prompt, we pass the value to cookiecutter
, so every key in your prompts.yml
must have a corresponding key in cookiecutter.json
.
If the input to the prompts needs to be validated, for example to make sure it only has alphanumeric characters, you can add regex validation rules via the regex_validator
key. For more complex validation, have a look at cookiecutter pre/post-generate hooks.
If you want cookiecutter
to provide sensible defaults in case a user doesn’t provide any input, you can add those to cookiecutter.json
. See the default starter cookiecutter.json
as example.
Example Kedro starter¶
To review an example Kedro starter, clone pandas-iris
from Github.
When you create an Iris dataset example project by calling kedro new
, you supply three configuration variables as the documentation in Create a new project describes. These variables are project_name
, repo_name
and python_package
and they are supplied interactively or by means of a configuration file. You can see how these variables are used by inspecting the template:
project_name
The human-readable project-name
variable is used in the README.md for the new project.
repo_name
The project structure contains a folder labelled {{ cookiecutter.repo_name }}
, which forms the top-level folder to contain the Iris dataset example when it is created. The folder storing the example project is represented by cookiecutter.repo_name
, which is a customisable variable, as you would expect.
python_package
Within the parent folder, inside the src
subfolder, is another configurable variable {{ cookiecutter.python_package }} which contains the source code for the example pipelines. The variable is also used within __main__.py
.
Here is the layout of the project as a Cookiecutter template:
{{ cookiecutter.repo_name }} # Parent directory of the template
├── conf # Project configuration files
├── data # Local project data (not committed to version control)
├── docs # Project documentation
├── logs # Project output logs (not committed to version control)
├── notebooks # Project related Jupyter notebooks (can be used for experimental code before moving the code to src)
├── README.md # Project README
├── setup.cfg # Configuration options for tools e.g. `pytest` or `flake8`
└── src # Project source code
└── {{ cookiecutter.python_package }}
├── __init.py__
├── cli.py # A collection of Kedro command line interface (CLI) commands
├── hooks.py
├── pipelines
├── pipeline_registry.py
├── __main__.py
└── settings.py
├── requirements.txt
├── setup.py
└── tests