This module contains a task for starting and monitoring Census sync jobs
Task for running Census connector sync jobs.
This task assumes the user has a Census sync already configured and is attempting to orchestrate the sync using Prefect task to send a post to the API within a prefect flow. Copy and paste from the api trigger section on the configuration page in the
api_trigger param to set a default sync.
api_trigger (str, optional): default sync to trigger, if none is specified in
**kwargs (dict, optional): additional kwargs to pass to the base Task constructor
Makes sure the url for the api trigger matches the Census format specified below. If it does not, it will raise a ValueError before returning.
Format of api_trigger: - "https://bearer:secret-token:email@example.com/api/v1/syncs/123/trigger"
api_trigger (str): if specified in the constructor, will call this validation there
confirmed_pattern (Match Object: https://docs.python.org/3/library/re.html#match-objects)
Task run method for Census syncs.
An invocation of
run will attempt to start a sync job for the specified
run will poll Census for the sync status, and will only complete when the sync has completed or when it receives an error status code from the trigger API call.
api_trigger (str): if not specified in run, it will pull from the default for the CensusSyncTask constructor. Keyword argument.
poll_status_every_n_seconds (int, optional): this task polls the Census API for the sync's status. If provided, this value will override the default polling time of 60 seconds and it has a minimum wait time of 5 seconds. Keyword argument.
dict: dictionary of statistics returned by Census on the specified sync, structure below.
'error_message': None / str,
'records_failed': int / None,
'records_invalid': int / None,
'records_processed': int / None,
'records_updated': int / None,
This documentation was auto-generated from commit 01705d3
on January 14, 2022 at 15:39 UTC