Run flows against a temporary database
Useprefect_test_harness as a context manager to run flows and tasks against a
temporary local SQLite database:
prefect_test_harness as a fixture in your unit testing framework. For example, when using pytest:
Session-scoped fixtureIn this example, the fixture is scoped to run once for the entire test session. In most cases, you do not need a clean database for each test. Just isolate your test runs to a test database. Creating a new test database per test creates significant overhead, so scope the fixture to the session. If you need to isolate some tests fully, place them in a separate module with a function-scoped harness fixture (see Stale state between tests).
Understand what the test harness does
When you enter theprefect_test_harness context manager, the following happens:
A temporary directory is created and
PREFECT_API_DATABASE_CONNECTION_URL is overridden
to point at a new SQLite file inside it. The database interface is reset through
temporary_database_interface() so the server uses this temporary database instead of
any previously configured one.A
SubprocessASGIServer starts in a child process on a random available port
(in the 8000-9000 range). This is a real Prefect API server backed by the temporary
database. PREFECT_API_URL is then overridden to point at this server so all client
calls route to it.Your flows and tasks execute the same code paths they would in production, including
state transitions, result persistence, and API interactions.
Test the underlying function without the engine
To test the function decorated with@task or @flow without running it through the
Prefect engine, use .fn() to call the wrapped function directly:
get_run_logger(), calling .fn() outside of a run context
raises a MissingContextError. Disable the run logger to avoid this:
Capture log output in tests
To test log output from flows and tasks, use pytest’scaplog fixture to capture log messages:
caplog requires the run logger to be enabledThe
caplog fixture only captures logs when the run logger is active. If you disable the run logger with disable_run_logger(), caplog does not capture any log output from flows or tasks.Test async flows and tasks
If your flows or tasks are async, usepytest-asyncio together with the test harness.
The harness itself is a synchronous context manager, so set it up in a synchronous
session-scoped fixture and write your test functions as async:
.fn() for async functions:
Troubleshoot common issues
Server startup timeout
The test harness starts a subprocess server and waits for it to become healthy. By default it waits 30 seconds. If your environment is slow (CI runners, constrained resources), increase the timeout:- Port conflicts: The harness selects a random port in the 8000-9000 range. If many ports are in use, it retries up to 10 times. Close other services using those ports or ensure parallel test processes each get their own harness instance.
- Missing dependencies: The server requires SQLite support. Verify your Python
installation includes the
sqlite3module.
Stale state between tests
The session-scoped fixture reuses a single database across all tests. If a test creates flows or deployments that interfere with other tests, you have two options:- Use a function-scoped fixture for tests that need a clean database. Do not nest
prefect_test_harnessinside a test that already runs under a session-scoped harness, because the inner harness exit stops the shared subprocess server and leaves later tests pointing at a dead API URL. Instead, put those tests in a separate file or module with its own function-scoped fixture:
- Clean up explicitly by deleting test data through the Prefect client after each test.
MissingContextError from get_run_logger
Callingget_run_logger() outside of a flow or task run raises a MissingContextError.
This commonly occurs when calling .fn() on a decorated function that uses the run
logger. Wrap the call with disable_run_logger():
Parallel test processes
If you usepytest-xdist or another parallel test runner, each worker process should
create its own prefect_test_harness instance. Because each harness starts its own
server on a random port with its own SQLite database, parallel workers do not conflict
with each other. Make sure the fixture is session-scoped per worker (the default
behavior with pytest-xdist).
Next steps
- Flows for an overview of flow execution and state management
- Settings and profiles for details on
PREFECT_API_URLand other configuration - Add logging to workflows for more on logging in flows and tasks