Prefect Docs

# Fargate Agent

# FargateAgent

class

prefect.agent.fargate.agent.FargateAgent

(agent_config_id=None, name=None, labels=None, env_vars=None, max_polls=None, agent_address=None, no_cloud_logs=None, launch_type="FARGATE", aws_access_key_id=None, aws_secret_access_key=None, aws_session_token=None, region_name=None, botocore_config=None, enable_task_revisions=False, use_external_kwargs=False, external_kwargs_s3_bucket=None, external_kwargs_s3_key=None, **kwargs)[source]

Agent which deploys flow runs as tasks using Fargate.

DEPRECATED: The Fargate agent is deprecated, please transition to using the ECS agent instead.

This agent can run anywhere as long as the proper access configuration variables are set.

All kwargs are accepted that one would normally pass to boto3 for register_task_definition and run_task. For information on the kwargs supported visit the following links:

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ecs.html#ECS.Client.register_task_definition

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ecs.html#ECS.Client.run_task

Note: if AWS authentication kwargs such as aws_access_key_id and aws_session_token are not provided they will be read from the environment.

Environment variables may be set on the agent to be provided to each flow run's Fargate task:

prefect agent fargate start --env MY_SECRET_KEY=secret --env OTHER_VAR=$OTHER_VAR

boto3 kwargs being provided to the Fargate Agent:

prefect agent fargate start \
    networkConfiguration="{\
        'awsvpcConfiguration': {\
            'assignPublicIp': 'ENABLED',\
            'subnets': ['my_subnet_id'],\
            'securityGroups': []\
        }\
    }"

botocore configuration options can be provided to the Fargate Agent:

FargateAgent(botocore_config={"retries": {"max_attempts": 10}})

Args:

  • agent_config_id (str, optional): An optional agent configuration ID that can be used to set configuration based on an agent from a backend API. If set all configuration values will be pulled from backend agent configuration.
  • name (str, optional): An optional name to give this agent. Can also be set through the environment variable PREFECT__CLOUD__AGENT__NAME. Defaults to "agent"
  • labels (List[str], optional): a list of labels, which are arbitrary string identifiers used by Prefect Agents when polling for work
  • env_vars (dict, optional): a dictionary of environment variables and values that will be set on each flow run that this agent submits for execution
  • max_polls (int, optional): maximum number of times the agent will poll Prefect Cloud for flow runs; defaults to infinite
  • agent_address (str, optional): Address to serve internal api at. Currently this is just health checks for use by an orchestration layer. Leave blank for no api server (default).
  • no_cloud_logs (bool, optional): Disable logging to a Prefect backend for this agent and all deployed flow runs
  • launch_type (str, optional): either FARGATE or EC2, defaults to FARGATE
  • aws_access_key_id (str, optional): AWS access key id for connecting the boto3 client. Defaults to the value set in the environment variable AWS_ACCESS_KEY_ID or None
  • aws_secret_access_key (str, optional): AWS secret access key for connecting the boto3 client. Defaults to the value set in the environment variable AWS_SECRET_ACCESS_KEY or None
  • aws_session_token (str, optional): AWS session key for connecting the boto3 client. Defaults to the value set in the environment variable AWS_SESSION_TOKEN or None
  • region_name (str, optional): AWS region name for connecting the boto3 client. Defaults to the value set in the environment variable REGION_NAME or None
  • botocore_config (dict, optional): botocore configuration options to be passed to the boto3 client. https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html
  • enable_task_revisions (bool, optional): Enable registration of task definitions using revisions. When enabled, task definitions will use flow name as opposed to flow id and each new version will be a task definition revision. Each revision will be registered with a tag called 'PrefectFlowId' and 'PrefectFlowVersion' to enable proper lookup for existing revisions. Flow name is reformatted to support task definition naming rules by converting all non-alphanumeric characters to '_'. Defaults to False.
  • use_external_kwargs (bool, optional): When enabled, the agent will check for the existence of an external json file containing kwargs to pass into the run_flow process. Defaults to False.
  • external_kwargs_s3_bucket (str, optional): S3 bucket containing external kwargs.
  • external_kwargs_s3_key (str, optional): S3 key prefix for the location of <slugified_flow_name>/<flow_id[:8]>.json.
  • **kwargs (dict, optional): additional keyword arguments to pass to boto3 for register_task_definition and run_task

methods:                                                                                                                                                       

prefect.agent.agent.Agent.start

()[source]

The main entrypoint to the agent process. Sets up the agent then continuously polls for work to submit.

This is the only method that should need to be called externally.


This documentation was auto-generated from commit n/a
on July 1, 2021 at 18:35 UTC

← Kubernetes Agent →