Prefect integrations for interacting with Databricks.
prefect-databricks
The following command will install a version of prefect-databricks
compatible with your installed version of prefect
.
If you don’t already have prefect
installed, it will install the newest version of prefect
as well.
Upgrade to the latest versions of prefect
and prefect-databricks
:
with_options
to customize options on any existing task or flowNotebook named example.ipynb
on Databricks which accepts a name parameter:
Prefect flow that launches a new cluster to run example.ipynb
:
Note, instead of using the built-in models, you may also input valid JSON. For example, AutoScale(min_workers=1, max_workers=2)
is equivalent to {"min_workers": 1, "max_workers": 2}
.
For assistance using Databricks, consult the Databricks documentation.
Refer to the prefect-databricks
SDK documentation to explore all the capabilities of the prefect-databricks
library.
Prefect integrations for interacting with Databricks.
prefect-databricks
The following command will install a version of prefect-databricks
compatible with your installed version of prefect
.
If you don’t already have prefect
installed, it will install the newest version of prefect
as well.
Upgrade to the latest versions of prefect
and prefect-databricks
:
with_options
to customize options on any existing task or flowNotebook named example.ipynb
on Databricks which accepts a name parameter:
Prefect flow that launches a new cluster to run example.ipynb
:
Note, instead of using the built-in models, you may also input valid JSON. For example, AutoScale(min_workers=1, max_workers=2)
is equivalent to {"min_workers": 1, "max_workers": 2}
.
For assistance using Databricks, consult the Databricks documentation.
Refer to the prefect-databricks
SDK documentation to explore all the capabilities of the prefect-databricks
library.