~/prefect/storage
.
To store cache records in a remote object store such as S3, pass a storage block instead:
isolation_level
parameter on the cache policy. Prefect supports two isolation levels: READ_COMMITTED
and SERIALIZABLE
.
By default, cache records operate with a READ_COMMITTED
isolation level. This guarantees that reading a cache record will see the latest committed cache value,
but allows multiple executions of the same task to occur simultaneously.
Consider the following example:
SERIALIZABLE
isolation level. This ensures that only one execution of a task occurs at a time for a given cache
record via a locking mechanism.
When setting isolation_level
to SERIALIZABLE
, you must also provide a lock_manager
that implements locking logic for your system.
Here’s an updated version of the previous example that uses SERIALIZABLE
isolation:
RedisLockManager
provided by prefect-redis
in conjunction with a shared Redis instance:process_data
task after the load_data
task has succeeded.
However, because caches are only written to when a transaction is committed, the load_data
task will not write a result to its cache key location until
the process_data
task succeeds as well.
On a subsequent run with fail=False
, both tasks will be re-executed and the results will be cached.