CLI

In conjunction with both the GraphQL API and the UI, Prefect offers a fully integrated CLI for working with Prefect Cloud. With the GraphQL API being the backbone for communicating with Prefect Cloud, we have designed the CLI to operate as a wrapper for GraphQL queries and mutations.

In this current state the Prefect CLI functions mainly as a way to read Prefect Cloud metadata as well as a lightweight way to perform some more functional operations such as running a flow that was deployed to Cloud.

Commands subject to change!

The CLI is being built to comply with Cloud user demands on what is deemed useful. Therefore the names and outputs of some of these are subject to change during the Cloud rollout process.

Interacting with the CLI

Once you have Prefect installed (either through pip or conda) you may begin working with the CLI directly in your terminal. To see the CLI simply run prefect from the command line and you should see an output similar to the one below:

$ prefect
Usage: prefect [OPTIONS] COMMAND [ARGS]...

  The Prefect CLI for creating, managing, and inspecting your flows.

  Note: a Prefect Cloud API token is required for all Cloud related commands. If a token
  is not set then run `prefect auth login` to set it.

  Query Commands:
      get         List high-level object information
      describe    Retrieve detailed object descriptions

  Execution Commands:
      execute     Execute a flow's environment
      run         Run a flow

  Setup Commands:
      auth        Handle Prefect Cloud authorization

  Miscellaneous Commands:
      version     Get your current Prefect version
      config      Output your Prefect config

Options:
  -h, --help  Show this message and exit.

From this help output you can see that the commands are broken into the categories query, execution, setup, and miscellaneous. Each category has groups (e.g. get, describe) and each group has commands for interacting with particular Prefect objects (e.g. prefect get flows). Every group and command is also provided with help documentation that can be accessed through --help or -h.

Query Commands

get

get is used to retrieve high level information about Prefect Cloud objects. Currently CLI users can get flows, flow-runs, projects, tasks, and logs.

get flows

Running prefect get flows without any extra arguments will output a table of your current flows.

$ prefect get flows
NAME        VERSION   PROJECT NAME   AGE
Test-Flow   2         New-Proj       22 hours ago
my-flow     1         Demo           2 weeks ago

To get more specific flow information you can optionally provide values for --name, --version, --project that will filter the table that is shown.

$ prefect get flows --name my-flow
NAME        VERSION   PROJECT NAME   AGE
my-flow     1         Demo           2 weeks ago
$ prefect get flows --project New-Proj
NAME        VERSION   PROJECT NAME   AGE
Test-Flow   2         New-Proj       22 hours ago
$ prefect get flows --project New-Proj --version 1
NAME        VERSION   PROJECT NAME   AGE
Test-Flow   1         New-Proj       1 month ago

By default this command shows the most recent version of a flow. To see all versions use the --all-versions flag.

$ prefect get flows --all-versions
NAME        VERSION   PROJECT NAME   AGE
Test-Flow   2         New-Proj       22 hours ago
Test-Flow   1         New-Proj       1 month ago
my-flow     1         Demo           2 weeks ago

get flow-runs

Running prefect get flow-runs without any extra arguments will output a table of your most recent flow runs (default is the 10 most recent).

$ prefect get flow-runs
NAME                FLOW NAME    STATE      AGE             START TIME           DURATION
archetypal-terrier  my-flow      Scheduled  2 minutes ago
turquoise-gazelle   my-flow      Success    2 weeks ago     2019-07-25 18:30:04  00:00:02
agate-parakeet      my-flow      Success    2 weeks ago     2019-07-25 17:58:29  00:00:02
wonderful-boa       my-flow      Success    2 weeks ago     2019-07-25 16:37:34  00:00:02
pastoral-auk        my-flow      Success    2 weeks ago     2019-07-25 16:04:43  00:00:02
nebulous-lyrebird   my-flow      Success    2 weeks ago     2019-07-25 14:41:35  00:00:02
mighty-jellyfish    my-flow      Success    2 weeks ago     2019-07-25 14:41:30  00:00:02
coral-lemming       my-flow      Success    2 weeks ago     2019-07-25 14:40:41  00:00:02
magenta-oxpecker    my-flow      Success    2 weeks ago     2019-07-25 14:37:55  00:00:02
rapid-gibbon        my-flow      Success    2 weeks ago     2019-07-25 14:36:25  00:00:02

Prettier Output

In each output the DURATION column timestamp was trimmed in order to make the tables fit better on this page.

To increase the amount retrieved you can specify a --limit integer. Similar to other get commands get flow-runs supports filtering of --flow and --project where a certain flow name or project name can be provided to filter the table.

There is an extra argument to filter only for flow runs that have started (either are currently running or have entered a finished state) by using the --started flag. In the output below notice that the flow run with the name archetypal-terrier is missing because it has not started.

$ prefect get flow-runs --started
NAME                FLOW NAME    STATE      AGE             START TIME           DURATION
turquoise-gazelle   my-flow      Success    2 weeks ago     2019-07-25 18:30:04  00:00:02
agate-parakeet      my-flow      Success    2 weeks ago     2019-07-25 17:58:29  00:00:02
wonderful-boa       my-flow      Success    2 weeks ago     2019-07-25 16:37:34  00:00:02
pastoral-auk        my-flow      Success    2 weeks ago     2019-07-25 16:04:43  00:00:02
nebulous-lyrebird   my-flow      Success    2 weeks ago     2019-07-25 14:41:35  00:00:02
mighty-jellyfish    my-flow      Success    2 weeks ago     2019-07-25 14:41:30  00:00:02
coral-lemming       my-flow      Success    2 weeks ago     2019-07-25 14:40:41  00:00:02
magenta-oxpecker    my-flow      Success    2 weeks ago     2019-07-25 14:37:55  00:00:02
rapid-gibbon        my-flow      Success    2 weeks ago     2019-07-25 14:36:25  00:00:02

get projects

Running prefect get projects without any extra arguments will output a table of your projects. Optionally you can specify --name to retrieve a specific project.

$ prefect get projects
NAME                   FLOW COUNT    AGE          DESCRIPTION
Demo                   1             2 weeks ago

get tasks

Running prefect get tasks without any extra arguments will output a table of your current tasks (default is the 10 most recent).

$ prefect get tasks
NAME          FLOW NAME   FLOW VERSION   AGE          MAPPED   TYPE
first_task    Test-Flow   1              5 days ago   False    prefect.tasks.core.function.FunctionTask
second_task   Test-Flow   1              5 days ago   True     prefect.tasks.core.function.FunctionTask
my_task       my-flow     1              2 weeks ago  False    prefect.tasks.core.function.FunctionTask

To increase the amount retrieved you can specify a --limit integer. Similar to other get commands get tasks supports filtering with --name for the name of a task, --flow-name for the name of a flow, --flow-version for the version of a flow, and --project for a project name.

get logs

Running prefect get logs requires that a flow run name is provided through --name. It outputs a table of logs from that flow run.

$ prefect get logs --name fearless-hyrax
TIMESTAMP                         LEVEL    MESSAGE
2019-07-17T23:37:22.816988+00:00  INFO     Beginning Flow run for 'my-flow'
2019-07-17T23:37:23.214365+00:00  INFO     Starting flow run.
2019-07-17T23:37:23.365119+00:00  INFO     Flow 'my-flow': Handling state change from Scheduled to Running
2019-07-17T23:37:23.915298+00:00  INFO     Task 'my_task': Starting task run...
2019-07-17T23:37:24.113271+00:00  INFO     Task 'my_task': Handling state change from Pending to Running
2019-07-17T23:37:24.666+00:00     INFO     Task 'my_task': Calling task.run() method...
2019-07-17T23:37:24.813664+00:00  INFO     Task 'my_task': Handling state change from Running to Success
2019-07-17T23:37:25.15329+00:00   INFO     Task 'my_task': finished task run for task with final state: 'Success'
2019-07-17T23:37:25.351535+00:00  INFO     Flow run SUCCESS: all reference tasks succeeded
2019-07-17T23:37:25.514397+00:00  INFO     Flow 'my-flow': Handling state change from Running to Success

To retrieve more information about each of the logs in JSON pass the --info flag to the command.

$ prefect get logs --name fearless-hyrax --info
{
    "msg": "Beginning Flow run for 'my-flow'",
    "args": [],
    "name": "prefect.CloudFlowRunner",
    "msecs": 816.9882297515869,
    "lineno": 224,
    "module": "flow_runner",
    "thread": 140362462062400,
    "asctime": "2019-07-17 23:37:22,816",
    "created": 1563406642.8169882,
    "levelno": 20,
    "message": "Beginning Flow run for 'my-flow'",
    "process": 6,
    "exc_info": null,
    "exc_text": null,
    "filename": "flow_runner.py",
    "funcName": "run",
    "pathname": "/usr/local/lib/python3.6/site-packages/prefect/engine/flow_runner.py",
    "levelname": "INFO",
    "stack_info": null,
    "threadName": "MainThread",
    "processName": "MainProcess",
    "relativeCreated": 7002.268075942993
}
etc...

describe

describe is used to retrieve descriptive metadata information about Prefect Cloud objects. Currently CLI users can describe flows, flow-runs, and tasks.

describe flows

Running prefect describe flows requires that a flow name be provided through the --name option. This outputs flow descriptive metadata.

$ prefect describe flows --name my-flow
{
    "name": "my-flow",
    "version": 1,
    "project": {
        "name": "Demo"
    },
    "created": "2019-07-25T14:23:21.704585+00:00",
    "description": null,
    "parameters": [],
    "archived": false,
    "storage": {
        "type": "Docker",
        "flows": {
            "my-flow": "/root/.prefect/my-flow.prefect"
        },
        "image_tag": "9b444406-355b-4d32-832e-77ddc21d2073",
        "image_name": "my-flow",
        "__version__": "0.6.0",
        "registry_url": "gcr.io/my-registry/",
        "prefect_version": "master"
    },
    "environment": {
        "type": "RemoteEnvironment",
        "executor": "prefect.engine.executors.SynchronousExecutor",
        "__version__": "0.6.1",
        "executor_kwargs": {}
    }
}

This defaults to the most recent version of a flow and to describe past versions use the --version option. You can also specify the project name that a flow belongs to with --project (often used if you have multiple flows with the same name in various projects).

describe flow-runs

Running prefect describe flow-runs requires that a flow run name be provided through the --name option. This outputs flow run descriptive metadata.

$ prefect describe flow-runs --name turquoise-gazelle
{
    "name": "turquoise-gazelle",
    "flow": {
        "name": "my-flow"
    },
    "created": "2019-07-25T18:29:49.926177+00:00",
    "parameters": {},
    "auto_scheduled": false,
    "scheduled_start_time": "2019-07-25T18:29:49.908098+00:00",
    "start_time": "2019-07-25T18:30:04.201221+00:00",
    "end_time": "2019-07-25T18:30:06.603138+00:00",
    "duration": "00:00:02.401917",
    "heartbeat": "2019-07-25T18:30:06.603138+00:00",
    "serialized_state": {
        "type": "Success",
        "_result": {
            "type": "NoResultType",
            "__version__": "0.6.0"
        },
        "message": "All reference tasks succeeded.",
        "__version__": "0.6.1"
    }
}

describe tasks

Running prefect describe tasks requires that a flow name be provided through the --name option. This outputs task descriptive metadata that correspond to a flow.

$ prefect describe tasks --name my-flow
{
    "name": "my_task",
    "created": "2019-07-25T14:23:21.704585+00:00",
    "slug": "2fe4be08-5ac6-4aea-953b-cb9394596145",
    "description": null,
    "type": "prefect.tasks.core.function.FunctionTask",
    "max_retries": 0,
    "retry_delay": null,
    "mapped": false
}

This defaults to the most recent version of a flow and to describe tasks for past versions use the --version option. You can also specify the project name that a flow belongs to with --project (often used if you have multiple flows with the same name in various projects).

Execution Commands

execute

execute cloud-flow

Not Useful for Local Execution

This command executes a flow's environment in the context of Prefect Cloud and runs during Cloud execution so it is not meant for local use. Other execute commands may be added in the future.

run

run cloud

Running this command requires that a flow name (--name) and project (--project) is specified in order to create a flow run for that particular flow in Prefect Cloud.

$ prefect run cloud --name my-flow --project Demo
Flow Run ID: 2ba3ddfd-411c-4d99-bb2a-f64a6dea87f9

There is an optional --version argument that can be passed in with the command to run any version of a flow that exists in Prefect Cloud.

This command also supports two live output options, --watch and --logs. Using --watch will live update the command line with the flow run's current state until it reaches a finished state and --logs will update the command line with live logs from the flow run. These two flags currently cannot be used at the same time in a single call.

Live updating output with --watch:

$ prefect run cloud --name my-flow --project Demo --watch
Flow Run ID: 2ba3ddfd-411c-4d99-bb2a-f64a6dea87f9
Scheduled -> Submitted -> Running -> Success

Live updating output with --logs:

$ prefect run cloud --name my-flow --project Demo --logs
Flow Run ID: 2ba3ddfd-411c-4d99-bb2a-f64a6dea87f9
TIMESTAMP                         LEVEL    MESSAGE
2019-07-17T23:37:22.816988+00:00  INFO     Beginning Flow run for 'my-flow'
2019-07-17T23:37:23.214365+00:00  INFO     Starting flow run.
2019-07-17T23:37:23.365119+00:00  INFO     Flow 'my-flow': Handling state change from Scheduled to Running
2019-07-17T23:37:23.915298+00:00  INFO     Task 'my_task': Starting task run...
2019-07-17T23:37:24.113271+00:00  INFO     Task 'my_task': Handling state change from Pending to Running
2019-07-17T23:37:24.666+00:00     INFO     Task 'my_task': Calling task.run() method...
2019-07-17T23:37:24.813664+00:00  INFO     Task 'my_task': Handling state change from Running to Success
2019-07-17T23:37:25.15329+00:00   INFO     Task 'my_task': finished task run for task with final state: 'Success'
2019-07-17T23:37:25.351535+00:00  INFO     Flow run SUCCESS: all reference tasks succeeded
2019-07-17T23:37:25.514397+00:00  INFO     Flow 'my-flow': Handling state change from Running to Success

Setup Commands

auth

auth is a group of commands that handle authentication related configuration with Prefect Cloud.

auth login

Running prefect auth login requires that a Prefect Cloud API token be provided and when executed the API token is used to login to Prefect Cloud.

$ prefect auth login --token $MY_TOKEN
Login successful

If the API token is not valid then you should see:

$ prefect auth login --token BAD_TOKEN
Error attempting to use Prefect API token BAD_TOKEN

Miscellaneous Commands

prefect version outputs the current version of Prefect you are using:

$ prefect version
0.6.1

prefect config outputs your current Prefect config that will be loaded:

$ prefect config
...config output...