Workers poll work pools for new runs to execute.
Worker Type | Description | Required Package |
---|---|---|
process | Executes flow runs in subprocesses | |
kubernetes | Executes flow runs as Kubernetes jobs | prefect-kubernetes |
docker | Executes flow runs within Docker containers | prefect-docker |
ecs | Executes flow runs as ECS tasks | prefect-aws |
cloud-run-v2 | Executes flow runs as Google Cloud Run jobs | prefect-gcp |
vertex-ai | Executes flow runs as Google Cloud Vertex AI jobs | prefect-gcp |
azure-container-instance | Execute flow runs in ACI containers | prefect-azure |
coiled | Execute flow runs in your cloud with Coiled | prefect-coiled |
Option | Description |
---|---|
--name , -n | The name to give to the started worker. If not provided, a unique name will be generated. |
--pool , -p | The work pool the started worker should poll. |
--work-queue , -q | One or more work queue names for the worker to pull from. If not provided, the worker pulls from all work queues in the work pool. |
--type , -t | The type of worker to start. If not provided, the worker type is inferred from the work pool. |
--prefetch-seconds | The amount of time before a flow run’s scheduled start time to begin submission. Default is the value of PREFECT_WORKER_PREFETCH_SECONDS . |
--run-once | Only run worker polling once. By default, the worker runs forever. |
--limit , -l | The maximum number of flow runs to start simultaneously. |
--with-healthcheck | Start a healthcheck server for the worker. |
--install-policy | Install policy to use workers from Prefect integration packages. |
kubernetes
, the worker deploys flow runs to a Kubernetes cluster.
Prefect must be installed in any environment (for example, virtual environment, Docker container) where you intend to run the worker or
execute a flow run.
PREFECT_API_URL
and PREFECT_API_KEY
settings for workersPREFECT_API_URL
must be set for the environment where your worker is running. When using Prefect Cloud, you must also have a user or service account
with the Worker
role, which you can configure by setting the PREFECT_API_KEY
.ONLINE
and OFFLINE
. A worker is online if it sends regular heartbeat messages to the Prefect API.
If a worker misses three heartbeats, it is considered offline. By default, a worker is considered offline a maximum of 90 seconds
after it stopped sending heartbeats, but you can configure the threshold with the PREFECT_WORKER_HEARTBEAT_SECONDS
setting.
prefect-aws
, prefect-gcp
)prefect worker start
CLI command to start a worker. You must pass at least the work pool name.
If the work pool does not exist, it will be created if the --type
flag is used.
--type
flag:
--limit
flag.
For example, to limit a worker to five concurrent flow runs:
--prefetch-seconds
option or the PREFECT_WORKER_PREFETCH_SECONDS
setting.
If this value is more than the amount of time it takes for the infrastructure to start, the flow run will wait until its
scheduled start time.
PREFECT_WORKER_QUERY_SECONDS
setting.
--install-policy
option. The following are valid install policies:
Install Policy | Description |
---|---|
always | Always install the required package. Updates the required package to the most recent version if already installed. |
if-not-present | Install the required package if it is not already installed. |
never | Never install the required package. |
prompt | Prompt the user to choose whether to install the required package. This is the default install policy. |
If prefect worker start is run non-interactively, the prompt install policy behaves the same as never . |