prefect.tasks
@task
decorator is preferred.
task_input_hash
context
: the active TaskRunContext
arguments
: a dictionary of arguments to be passed to the underlying taskNone
exponential_backoff
backoff_factor
: the base delay for the first retry, subsequent retries will
increase the delay time by powers of 2.task
name
: An optional name for the task; if not provided, the name will be inferred
from the given function.description
: An optional string description for the task.tags
: An optional set of tags to be associated with runs of this task. These
tags are combined with any tags defined by a prefect.tags
context at
task runtime.version
: An optional string specifying the version of this task definitioncache_key_fn
: An optional callable that, given the task run context and call
parameters, generates a string key; if the key matches a previous completed
state, that state result will be restored instead of running the task again.cache_expiration
: An optional amount of time indicating how long cached states
for this task should be restorable; if not provided, cached states will
never expire.task_run_name
: An optional name to distinguish runs of this task; this name can be provided
as a string template with the task’s keyword arguments as variables,
or a function that returns a string.retries
: An optional number of times to retry on task run failureretry_delay_seconds
: Optionally configures how long to wait before retrying the
task after failure. This is only applicable if retries
is nonzero. This
setting can either be a number of seconds, a list of retry delays, or a
callable that, given the total number of retries, generates a list of retry
delays. If a number of seconds, that delay will be applied to all retries.
If a list, each retry will wait for the corresponding delay before retrying.
When passing a callable or a list, the number of
configured retry delays cannot exceed 50.retry_jitter_factor
: An optional factor that defines the factor to which a
retry can be jittered in order to avoid a “thundering herd”.persist_result
: A toggle indicating whether the result of this task
should be persisted to result storage. Defaults to None
, which
indicates that the global default should be used (which is True
by
default).result_storage
: An optional block to use to persist the result of this task.
Defaults to the value set in the flow the task is called in.result_storage_key
: An optional key to store the result in storage at when persisted.
Defaults to a unique identifier.result_serializer
: An optional serializer to use to serialize the result of this
task for persistence. Defaults to the value set in the flow the task is
called in.timeout_seconds
: An optional number of seconds indicating a maximum runtime for
the task. If the task exceeds this runtime, it will be marked as failed.log_prints
: If set, print
statements in the task will be redirected to the
Prefect logger for the task run. Defaults to None
, which indicates
that the value from the flow should be used.refresh_cache
: If set, cached results for the cache key are not used.
Defaults to None
, which indicates that a cached result from a previous
execution with matching cache key is used.on_failure
: An optional list of callables to run when the task enters a failed state.on_completion
: An optional list of callables to run when the task enters a completed state.retry_condition_fn
: An optional callable run when a task run returns a Failed state. Should
return True
if the task should continue to its retry policy (e.g. retries=3
), and False
if the task
should end as failed. Defaults to None
, indicating the task should always continue
to its retry policy.viz_return_value
: An optional value to return when the task dependency tree is visualized.asset_deps
: An optional list of upstream assets that this task depends on.Task
object which, when called, will submit the task for execution.TaskRunNameCallbackWithParameters
is_callback_with_parameters
TaskOptions
Unpack
to provide type hints for **kwargs.
Task
fn
: The function defining the task.name
: An optional name for the task; if not provided, the name will be inferred
from the given function.description
: An optional string description for the task.tags
: An optional set of tags to be associated with runs of this task. These
tags are combined with any tags defined by a prefect.tags
context at
task runtime.version
: An optional string specifying the version of this task definitioncache_policy
: A cache policy that determines the level of caching for this taskcache_key_fn
: An optional callable that, given the task run context and call
parameters, generates a string key; if the key matches a previous completed
state, that state result will be restored instead of running the task again.cache_expiration
: An optional amount of time indicating how long cached states
for this task should be restorable; if not provided, cached states will
never expire.task_run_name
: An optional name to distinguish runs of this task; this name can be provided
as a string template with the task’s keyword arguments as variables,
or a function that returns a string.retries
: An optional number of times to retry on task run failure.retry_delay_seconds
: Optionally configures how long to wait before retrying the
task after failure. This is only applicable if retries
is nonzero. This
setting can either be a number of seconds, a list of retry delays, or a
callable that, given the total number of retries, generates a list of retry
delays. If a number of seconds, that delay will be applied to all retries.
If a list, each retry will wait for the corresponding delay before retrying.
When passing a callable or a list, the number of configured retry delays
cannot exceed 50.retry_jitter_factor
: An optional factor that defines the factor to which a retry
can be jittered in order to avoid a “thundering herd”.persist_result
: A toggle indicating whether the result of this task
should be persisted to result storage. Defaults to None
, which
indicates that the global default should be used (which is True
by
default).result_storage
: An optional block to use to persist the result of this task.
Defaults to the value set in the flow the task is called in.result_storage_key
: An optional key to store the result in storage at when persisted.
Defaults to a unique identifier.result_serializer
: An optional serializer to use to serialize the result of this
task for persistence. Defaults to the value set in the flow the task is
called in.timeout_seconds
: An optional number of seconds indicating a maximum runtime for
the task. If the task exceeds this runtime, it will be marked as failed.log_prints
: If set, print
statements in the task will be redirected to the
Prefect logger for the task run. Defaults to None
, which indicates
that the value from the flow should be used.refresh_cache
: If set, cached results for the cache key are not used.
Defaults to None
, which indicates that a cached result from a previous
execution with matching cache key is used.on_failure
: An optional list of callables to run when the task enters a failed state.on_completion
: An optional list of callables to run when the task enters a completed state.on_commit
: An optional list of callables to run when the task’s idempotency record is committed.on_rollback
: An optional list of callables to run when the task rolls back.retry_condition_fn
: An optional callable run when a task run returns a Failed state. Should
return True
if the task should continue to its retry policy (e.g. retries=3
), and False
if the task
should end as failed. Defaults to None
, indicating the task should always continue
to its retry policy.viz_return_value
: An optional value to return when the task dependency tree is visualized.asset_deps
: An optional list of upstream assets that this task depends on.ismethod
isclassmethod
isstaticmethod
with_options
name
: A new name for the task.description
: A new description for the task.tags
: A new set of tags for the task. If given, existing tags are ignored,
not merged.cache_key_fn
: A new cache key function for the task.cache_expiration
: A new cache expiration time for the task.task_run_name
: An optional name to distinguish runs of this task; this name can be provided
as a string template with the task’s keyword arguments as variables,
or a function that returns a string.retries
: A new number of times to retry on task run failure.retry_delay_seconds
: Optionally configures how long to wait before retrying
the task after failure. This is only applicable if retries
is nonzero.
This setting can either be a number of seconds, a list of retry delays,
or a callable that, given the total number of retries, generates a list
of retry delays. If a number of seconds, that delay will be applied to
all retries. If a list, each retry will wait for the corresponding delay
before retrying. When passing a callable or a list, the number of
configured retry delays cannot exceed 50.retry_jitter_factor
: An optional factor that defines the factor to which a
retry can be jittered in order to avoid a “thundering herd”.persist_result
: A new option for enabling or disabling result persistence.result_storage
: A new storage type to use for results.result_serializer
: A new serializer to use for results.result_storage_key
: A new key for the persisted result to be stored at.timeout_seconds
: A new maximum time for the task to complete in seconds.log_prints
: A new option for enabling or disabling redirection of print
statements.refresh_cache
: A new option for enabling or disabling cache refresh.on_completion
: A new list of callables to run when the task enters a completed state.on_failure
: A new list of callables to run when the task enters a failed state.retry_condition_fn
: An optional callable run when a task run returns a Failed state.
Should return True
if the task should continue to its retry policy, and False
if the task should end as failed. Defaults to None
, indicating the task should
always continue to its retry policy.viz_return_value
: An optional value to return when the task dependency tree is visualized.Task
instance.on_completion
on_failure
on_commit
on_rollback
submit
submit
submit
submit
submit
submit
*args
: Arguments to run the task withreturn_state
: Return the result of the flow run wrapped in a
Prefect State.wait_for
: Upstream task futures to wait for before starting the task**kwargs
: Keyword arguments to run the task withreturn_state
is False a future allowing asynchronous access to
the state of the taskreturn_state
is True a future wrapped in a Prefect State allowing asynchronous access to
the state of the taskmap
map
map
map
map
map
map
*args
: Iterable and static arguments to run the tasks withreturn_state
: Return a list of Prefect States that wrap the results
of each task run.wait_for
: Upstream task futures to wait for before starting the
task**kwargs
: Keyword iterable arguments to run the task withunmapped
to treat an iterable argument as a constant
apply_async
args
: Arguments to run the task withkwargs
: Keyword arguments to run the task withdelay
apply_async
with simpler calling semantics.
Avoids having to use explicit “args” and “kwargs” arguments. Arguments
will pass through as-is to the task.
Examples:
Define a task
MaterializingTask
assets
: List of Assets that this task materializes (can be str or Asset)materialized_by
: An optional tool that materialized the asset e.g. “dbt” or “spark”**task_kwargs
: All other Task argumentswith_options