Installing a Collection¶
To use a Prefect Collection, first install the collection via
As an example, to use
pip install prefect-aws
Registering Blocks from a Collection¶
Once the Prefect Collection is installed, register the blocks within the collection to view them in the Prefect Cloud UI:
As an example, to register the blocks available in
prefect block register -m prefect_aws
Updating blocks from Prefect Collections
If you install an updated Prefect collection that adds fields to a block type, you will need to re-register that block type.
Loading a block in code
To use the
load method on a Block, you must already have a block document saved either through code or through the Prefect UI.
Learn more about Blocks here!
Using Tasks and Flows from a Collection¶
Collections also contain pre-built tasks and flows that can be imported and called within your code.
As an example, to read a secret from AWS Secrets Manager with the
from prefect import flow from prefect_aws import AwsCredentials from prefect_aws.secrets_manager import read_secret @flow def connect_to_database(): aws_credentials = AwsCredentials.load("MY_BLOCK_NAME") secret_value = read_secret( secret_name="db_password", aws_credentials=aws_credentials ) # Use secret_value to connect to a database
Customizing Tasks and Flows from a Collection¶
To customize the settings of a task or flow pre-configured in a collection, use
from prefect import flow from prefect_dbt.cloud import DbtCloudCredentials from prefect_dbt.cloud.jobs import trigger_dbt_cloud_job_run_and_wait_for_completion custom_run_dbt_cloud_job = trigger_dbt_cloud_job_run_and_wait_for_completion.with_options( name="Run My DBT Cloud Job", retries=2, retry_delay_seconds=10 ) @flow def run_dbt_job_flow(): run_result = custom_run_dbt_cloud_job( dbt_cloud_credentials=DbtCloudCredentials.load("my-dbt-cloud-credentials"), job_id=1 ) run_dbt_job_flow()
Recipes and Tutorials¶
To learn more about how to use Collections, check out Prefect recipes on GitHub. These recipes provide examples of how Collections can be used in various scenarios.