Settings and profiles

​

Why use settings?

Settings in Prefect help you control how your workflows behave. They let you easily customize Prefect to work the way you need it to, whether you’re testing locally or running in production.

Specifically, settings enable:

• Environment-Specific Configuration: Use different settings for development (like detailed logging), testing (like test databases), and production (like your production server) without changing your workflow code.

• Runtime Flexibility: Quickly adjust things like retry attempts or logging levels without having to modify and redeploy your workflows.

​

Get started with settings

The simplest way declare settings is by creating a prefect.toml file in your project directory. For example:

# Set more detailed logging while developing
[logging]
level = "DEBUG"

Most editors have plugins for TOML that provide syntax highlighting, linting, and autocomplete for prefect.toml files. If you use VSCode, we recommend the Even Better TOML extension.

​

Settings sources

You can configure settings via the following sources (highest to lowest precedence):

• Environment variables: Environment variables are useful for temporarily overriding settings or configuring the runtime environment of a single workflow run.

• .env file: .env files are useful for declaring local settings that you want to apply across multiple runs.

• prefect.toml file: A prefect.toml file is useful when you want to declare settings for an entire project. You can keep this file in your project directory and it will be automatically applied regardless of where you run your project.

• pyproject.toml file: If you already have a pyproject.toml file in your project or like to consolidate settings for all your tools in one place, you can declare settings in the [tool.prefect] table of your pyproject.toml file.

• Profiles: Prefect profiles are useful for switching between different environments. For example, you might use one profile for a local Prefect server and another for your production environment.

When multiple settings sources define the same setting, Prefect follows this precedence order (highest to lowest):

For example, if you set PREFECT_API_URL in both your environment and your active profile, the environment variable value will take precedence.

​

Environment variables

Environment variables are useful for temporarily overriding settings or configuring the runtime environment of a workflow.

All Prefect settings can be set using environment variables prefixed with PREFECT_. They take precedence over all other sources, making them ideal for adjustments that should only apply to a single session or process.

For example, you can run the following command to temporarily set the logging level for a single flow run:

PREFECT_LOGGING_LEVEL="DEBUG" python my_flow.py

You can also export an environment variable in your shell to apply it to all flow runs in that shell session:

export PREFECT_LOGGING_LEVEL="DEBUG"
prefect run my_flow.py

You can see supported environment variables for each setting in the settings reference documentation.

​

.env file

.env files are useful for declaring local settings that you want to apply across multiple runs.

When running prefect in a directory that contains a .env file, Prefect will automatically apply the settings in the file. We recommend keeping your .env files local and not committing them to your code repositories.

For example, the following .env file declares a local setting for the logging level:

PREFECT_LOGGING_LEVEL="DEBUG"

Any flows run in the same directory as this .env file will use the DEBUG logging level, even if they are run in different shell sessions.

View supported environment variables for each setting in the settings reference documentation.

​

prefect.toml file

A prefect.toml file is useful when you want to declare settings for an entire project.

You can keep a prefect.toml file in your project directory and the declared settings will be automatically applied when running prefect in that directory. We recommend committing this file to your code repositories to ensure consistency across environments.

For example, the following prefect.toml file declares a setting for the logging level:

[logging]
level = "DEBUG"

If you commit your prefect.toml file to a code repository, creating deployments from flows in that repository will use the settings declared in the prefect.toml file.

You can see the prefect.toml path for each setting in the settings reference documentation.

​

pyproject.toml file

Declaring settings in a pyproject.toml file is very similar to declaring settings in a prefect.toml file. The main difference is that settings are declared in the [tool.prefect] table instead of at the root of the file.

For example, the following pyproject.toml file declares a setting for the logging level:

[tool.prefect]
logging.level = "DEBUG"

The advantage of declaring settings in a pyproject.toml file is that it allows you to keep all your dependencies and settings for all your tools in one place. You can learn more about pyproject.toml files in the Python Packaging User Guide.

​

Profiles

Prefect profiles are useful for switching between different environments. By creating different profiles with different API URLs, you can easily switch between a local Prefect server and your production environment.

Profiles are stored in a TOML file located at ~/.prefect/profiles.toml by default. This location can be configured by setting PREFECT_PROFILES_PATH.

One and only one profile can be active at any time.

Immediately after installation, the ephemeral profile will be used, which only has PREFECT_SERVER_ALLOW_EPHEMERAL_MODE configured:

The prefect profile CLI commands enable you to create, review, and manage profiles:

… or you may edit your profiles.toml file directly:

vim ~/.prefect/profiles.toml

​

Configure settings for the active profile

The prefect config CLI commands enable you to manage the settings within the currently active profile.

For example, the following CLI commands set configuration in the ephemeral profile and then create a new profile with new settings:

prefect profile use ephemeral
prefect config set PREFECT_API_URL=http://127.0.0.1:4200/api

prefect profile create new-profile --from ephemeral
prefect profile use new-profile
prefect config set PREFECT_RESULTS_PERSIST_BY_DEFAULT=true PREFECT_LOGGING_LEVEL="ERROR"

prefect profile inspect
prefect config unset PREFECT_LOGGING_LEVEL -y

​

Common client settings

• api.url: this setting specifies the API endpoint of your Prefect Cloud workspace or a self-hosted Prefect server instance.
• api.key: this setting specifies the API key used to authenticate with Prefect Cloud.
• home: the home value specifies the local Prefect directory for configuration files, profiles, and the location of the default Prefect SQLite database.

​

Common server settings

• server.database.connection_url: the database connection URL for a self-hosted Prefect server instance. Must be provided in a SQLAlchemy-compatible format. Prefect currently supports SQLite and Postgres.