# Cloud


# CloudFlowRunner

class

prefect.engine.cloud.flow_runner.CloudFlowRunner

(flow, state_handlers=None)[source]

FlowRunners handle the execution of Flows and determine the State of a Flow before, during and after the Flow is run.

In particular, through the FlowRunner you can specify which tasks should be the first tasks to run, which tasks should be returned after the Flow is finished, and what states each task should be initialized with.

Args:

  • flow (Flow): the Flow to be run
  • state_handlers (Iterable[Callable], optional): A list of state change handlers that will be called whenever the flow changes state, providing an opportunity to inspect or modify the new state. The handler will be passed the flow runner instance, the old (prior) state, and the new (current) state, with the following signature:

            state_handler(
                flow_runner: FlowRunner,
                old_state: State,
                new_state: State) -> State

If multiple functions are passed, then the new_state argument will be the result of the previous handler.

Note: new FlowRunners are initialized within the call to Flow.run() and in general, this is the endpoint through which FlowRunners will be interacted with most frequently.

Example:

@task
def say_hello():
    print('hello')

with Flow("My Flow") as f:
    say_hello()

fr = FlowRunner(flow=f)
flow_state = fr.run()

methods:                                                                                                                                                       

prefect.engine.cloud.flow_runner.CloudFlowRunner.call_runner_target_handlers

(old_state, new_state)[source]

A special state handler that the FlowRunner uses to call its flow's state handlers. This method is called as part of the base Runner's handle_state_change() method.

Args:

  • old_state (State): the old (previous) state
  • new_state (State): the new (current) state
Returns:
  • State: the new state

prefect.engine.cloud.flow_runner.CloudFlowRunner.check_for_cancellation

()[source]

Contextmanager used to wrap a cancellable section of a flow run.

prefect.engine.cloud.flow_runner.CloudFlowRunner.initialize_run

(state, task_states, context, task_contexts, parameters)[source]

Initializes the Task run by initializing state and context appropriately.

If the provided state is a Submitted state, the state it wraps is extracted.

Args:

  • state (Optional[State]): the initial state of the run
  • task_states (Dict[Task, State]): a dictionary of any initial task states
  • context (Dict[str, Any], optional): prefect.Context to use for execution to use for each Task run
  • task_contexts (Dict[Task, Dict[str, Any]], optional): contexts that will be provided to each task
  • parameters(dict): the parameter values for the run
Returns:
  • NamedTuple: a tuple of initialized objects: (state, task_states, context, task_contexts)

prefect.engine.cloud.flow_runner.CloudFlowRunner.run

(state=None, task_states=None, return_tasks=None, parameters=None, task_runner_state_handlers=None, executor=None, context=None, task_contexts=None)[source]

The main endpoint for FlowRunners. Calling this method will perform all computations contained within the Flow and return the final state of the Flow.

Args:

  • state (State, optional): starting state for the Flow. Defaults to Pending
  • task_states (dict, optional): dictionary of task states to begin computation with, with keys being Tasks and values their corresponding state
  • return_tasks ([Task], optional): list of Tasks to include in the final returned Flow state. Defaults to None
  • parameters (dict, optional): dictionary of any needed Parameter values, with keys being strings representing Parameter names and values being their corresponding values
  • task_runner_state_handlers (Iterable[Callable], optional): A list of state change handlers that will be provided to the task_runner, and called whenever a task changes state.
  • executor (Executor, optional): executor to use when performing computation; defaults to the executor specified in your prefect configuration
  • context (Dict[str, Any], optional): prefect.Context to use for execution to use for each Task run
  • task_contexts (Dict[Task, Dict[str, Any]], optional): contexts that will be provided to each task
Returns:
  • State: State representing the final post-run state of the Flow.



# CloudTaskRunner

class

prefect.engine.cloud.task_runner.CloudTaskRunner

(task, state_handlers=None, flow_result=None)[source]

TaskRunners handle the execution of Tasks and determine the State of a Task before, during and after the Task is run.

In particular, through the TaskRunner you can specify the states of any upstream dependencies, and what state the Task should be initialized with.

Args:

  • task (Task): the Task to be run / executed
  • state_handlers (Iterable[Callable], optional): A list of state change handlers that will be called whenever the task changes state, providing an opportunity to inspect or modify the new state. The handler will be passed the task runner instance, the old (prior) state, and the new (current) state, with the following signature: state_handler(TaskRunner, old_state, new_state) -> State; If multiple functions are passed, then the new_state argument will be the result of the previous handler.
  • flow_result: the result instance configured for the flow (if any)

methods:                                                                                                                                                       

prefect.engine.cloud.task_runner.CloudTaskRunner.call_runner_target_handlers

(old_state, new_state)[source]

A special state handler that the TaskRunner uses to call its task's state handlers. This method is called as part of the base Runner's handle_state_change() method.

Args:

  • old_state (State): the old (previous) state
  • new_state (State): the new (current) state
Returns:
  • State: the new state

prefect.engine.cloud.task_runner.CloudTaskRunner.check_task_is_cached

(state, inputs)[source]

Checks if task is cached in the DB and whether any of the caches are still valid.

Args:

  • state (State): the current state of this task
  • inputs (Dict[str, Result]): a dictionary of inputs whose keys correspond to the task's run() arguments.
Returns:
  • State: the state of the task after running the check
Raises:
  • ENDRUN: if the task is not ready to run

prefect.engine.cloud.task_runner.CloudTaskRunner.initialize_run

(state, context)[source]

Initializes the Task run by initializing state and context appropriately.

Args:

  • state (Optional[State]): the initial state of the run
  • context (Dict[str, Any]): the context to be updated with relevant information
Returns:
  • tuple: a tuple of the updated state, context, and upstream_states objects

prefect.engine.cloud.task_runner.CloudTaskRunner.load_results

(state, upstream_states)[source]

Given the task's current state and upstream states, populates all relevant result objects for this task run.

Args:

  • state (State): the task's current state.
  • upstream_states (Dict[Edge, State]): the upstream state_handlers
Returns:
  • Tuple[State, dict]: a tuple of (state, upstream_states)

prefect.engine.cloud.task_runner.CloudTaskRunner.run

(state=None, upstream_states=None, context=None, is_mapped_parent=False)[source]

The main endpoint for TaskRunners. Calling this method will conditionally execute self.task.run with any provided inputs, assuming the upstream dependencies are in a state which allow this Task to run. Additionally, this method will wait and perform Task retries which are scheduled for <= 10 minutes in the future.

Args:

  • state (State, optional): initial State to begin task run from; defaults to Pending()
  • upstream_states (Dict[Edge, State]): a dictionary representing the states of any tasks upstream of this one. The keys of the dictionary should correspond to the edges leading to the task.
  • context (dict, optional): prefect Context to use for execution
  • is_mapped_parent (bool): a boolean indicating whether this task run is the run of a parent mapped task
Returns:
  • State object representing the final post-run state of the Task

prefect.engine.cloud.task_runner.CloudTaskRunner.set_task_run_name

(task_inputs)[source]

Sets the name for this task run by calling the set_task_run_name mutation.

Args:

  • task_inputs (Dict[str, Result]): a dictionary of inputs whose keys correspond to the task's run() arguments.



This documentation was auto-generated from commit 01705d3
on January 14, 2022 at 15:39 UTC