This tutorial provides a step by step walk-through of Prefect core concepts and instructions on how to use them.
By the end of this tutorial you will have:
- Created a Flow
- Added Tasks to It
- Created a Work Pool
- Deployed a Worker
- Deployed the Flow
- Run the Flow on the Worker
Before you start, make sure you have Python installed, then install Prefect:
pip install -U prefect
- See the install guide for more detailed instructions.
Create a GitHub repository for your tutorial, let's call it
What is Prefect?¶
Prefect orchestrates workflows — it simplifies the creation, scheduling, and monitoring of complex data pipelines. With Prefect, you define workflows as Python code and let it handle the rest.
Prefect also provides error handling, retry mechanisms, and a user-friendly dashboard for monitoring. It's the easiest way to transform any Python function into a unit of work that can be observed and orchestrated.
Just bring your Python code, sprinkle in a few decorators, and go!
If you've never used Prefect before, check out the core concepts docs. Many of these concepts will be introduced in the tutorial, however, the concept docs provide a more comprehensive overview of each concept.