Learn how to upgrade from agents to workers to significantly enhance the experience of deploying flows.
prefect>2.0,<3.0
) to workers.
If you are new to Prefect, we recommend starting with the
Prefect Quickstart..deploy()
or the alternative deployment experience with
prefect.yaml
are more flexible and easier to use than block and agent-based deployments.prefect deployment build <entrypoint>
/prefect deployment apply
—> prefect deploy
Prefect now automatically detects flows in your repo and provides a wizardDeployment.build_from_flow
—> flow.deploy
prefect.yaml
file to specify how to retrieve flow code for your deployments. You can use configuration from your
existing storage blocks to define your pull action through templating.
When using the Python deployment API, you can pass any storage block to the flow.deploy
method to
specify how to retrieve flow code for your deployment.
prefect.yaml
file or use the deploy
function.
prefect.yaml
file.
infra_override
-> job_variable
prefect agent start --pool <work pool name>
—> prefect worker start --pool <work pool name>
.publish_as_work_pool
method on any infrastructure block to create a work pool with the same configuration.
For example, if you have a KubernetesJob
infrastructure block named ‘my-k8s-job’, you can
create a work pool with the same configuration with this script:
Process
infrastructure block and a LocalFilesystem
storage block
(or aren’t using an infrastructure and storage block at all), you can use flow.serve
to create a deployment without specifying a work pool name or start a worker.This is a quick way to create a deployment for a flow and manage your
deployments if you don’t need the dynamic infrastructure creation or configuration offered
by workers.flow.deploy
for a Pythonic deployment
experience or prefect deploy
for a YAML-based deployment experience.
If you currently use Deployment.build_from_flow
, we recommend using flow.deploy
.
If you currently use prefect deployment build
and prefect deployment apply
, we recommend
using prefect deploy
.
flow.deploy
Deployment.build_from_flow
to create a deployment, you
can replace it with flow.deploy
.
You can translate most arguments to Deployment.build_from_flow
directly to flow.deploy
,
but here are some possible changes you may need:
infrastructure
with work_pool_name
.
.publish_as_work_pool
method on your infrastructure block, use the
name of the created work pool.infra_overrides
with job_variables
.storage
with a call to flow.from_source
.
flow.from_source
loads your flow from a remote storage location and makes it deployable.
You can pass your existing storage block to the source
argument of flow.from_source
.Deployment.build_from_flow
into flow.deploy
.
Deployment.build_from_flow
to deploy a flow from a local file looked like:
flow.deploy
:
example.py
:
flow.serve
.flow.from_source
to load your flow from the same location and flow.deploy
to
create a deployment:
flow.deploy
as the work_pool_name
argument. You also need to pass your storage block to
flow.from_source
as the source
argument.
flow.deploy
should look like this:
flow.from_source().deploy()
with a remote source
such as a GitHub
block or str
URL like https://github.com/me/myrepo.git), the flow you’re deploying doesn’t need to be available locally before running your script.
See the SDK reference for more info on from_source
.image
argument of flow.deploy
to build a Docker image as part of your deployment process:
flow.from_source
call when building an image with flow.deploy
. Prefect
keeps track of the flow’s source code location in the image and loads it from that location when the
flow is executed.
prefect deploy
prefect deploy
commands from the root
level of your repo!With agents, you may have multiple deployment.yaml
files. But under worker deployment
patterns, each repo has a single prefect.yaml
file located at the root of the repo
that contains deployment configuration
for all flows in that repo.prefect.yaml
file for your deployments, run the following command from the root
level of your repo:
y
on the last prompt to save the configuration for the deployment.Saving the configuration for your deployment results in a prefect.yaml
file populated
with your first deployment. You can use this YAML file to edit and define multiple deployments
for this repo.deployments
list in your prefect.yaml
file and/or by continuing to use the deployment
creation wizard.
For more information on deployments, check out our in-depth guide for deploying flows to work pools.