3.0.11 - Thread Carefully
Released on October 25, 2024 Enhancements ➕➕- Add setting to configure default max workers for
ThreadPoolTaskRunner
by @soamicharan in #15719 - Improve handling of interrupts to
serve
by @zzstoatzz in #15727 - Checks for active workers when deploying by @jeanluciano in #15632
- Highlight
Cached
state in console logs by @zzstoatzz in #15801
- Fix origin repo check that bypasses Docker Login on PRs originating from forks by @kzvezdarov in #15724
- Fix capture of pod logs for
run_namespaced_job
by @zzstoatzz in #15732 - Add
/api
toPREFECT_UI_API_URL
in default case by @zzstoatzz in #15736 - Pass lowercase log level to uvicorn server by @gigaverse-oz in #15680
- Fix handling of Dask DataFrame and other Awaitables passed to
run_coro_as_sync
by @kzvezdarov in #15687 - Wraps
run_dbt_
functions insync_compatible
by @EmilRex in #15740 - Fix custom task run name rendering by @zzstoatzz in #15773
- Fix incorrect import in agents to workers guide by @zzstoatzz in #15789
- Soft wrap deployment link by @zzstoatzz in #15787
- Fix worker filter param by @zangell44 in #15804
- Include env defaults from
variables.properties.env
in deep merge by @kevingrismore in #15692 - Fix wrong exception in lazy import getattr by @protagonyist in #15741
- Set minimum version of
pydantic-settings
by @zzstoatzz in #15797
prefect-gcp
prefect-dbt
- Fix dbt CLI argument parsing for multipart commands by @e1337us3r in #15714
prefect-kubernetes
- Avoid crashing flows when job watch exits but container is still running by @soamicharan in #15728
- Add
_request_timeout
to Kubernetes watches by @kevingrismore in #15744
- Create settings models for CLI and client settings by @desertaxle in #15640
- Create settings models for cloud, deployments, and logging by @desertaxle in #15749
- Create settings models for flows, results, and runners by @desertaxle in #15754
- Create server settings models by @desertaxle in #15758
- Create settings submodules by @desertaxle in #15772
- Create settings models for services, events, tasks, and the flow run graph by @desertaxle in #15776
- Create worker, tasks, testing, and UI settings models by @desertaxle in #15784
- Fix failing work queue test by @desertaxle in #15709
- Split out runner and worker tests to avoid hanging test suite by @desertaxle in #15715
- Only authenticate with Docker when running on a branch by @desertaxle in #15708
- Give more time for integration serve flow by @zzstoatzz in #15720
- De-flake serve integration test by @zzstoatzz in #15726
- Add code coverage generation and upload for Python tests by @desertaxle in #15690
- Remove
setuptools
and updatepip
pin in Docker build by @desertaxle in #15759 - Improve map typing by @zzstoatzz in #15757
- Ignore errors when cleaning up after deploy integration test by @desertaxle in #15768
- Disable automatic markdown tests by @zzstoatzz in #15782
- Fix typing for triggers schemas by @zzstoatzz in #15792
- Update instructions on bug report issue template by @desertaxle in #15795
- Format example code in
README.md
by @zzstoatzz in #15807 - Fix typo in prefect-databricks by @aaazzam in #15800
- Fix links to 3.0 docs that are missing a leading slash by @sam-phinizy in #15704
- Add a few notes to Docker docs page by @discdiver in #15706
- Improve table formatting in docs by @discdiver in #15700
- Fix docs links for integrations by @discdiver in #15702
- More accessible wording for job variable overrides docs by @zzstoatzz in #15669
- Fix
automation-assement.py
by @jakekaplan in #15710 - Give more time for state to integration flow to finish by @jakekaplan in #15723
- Fix a few spelling issues in docs by @discdiver in #15737
- Updated Docker push step in docs by @zzstoatzz in #15748
- Clear up namespace selection in Kubernetes guide by @zzstoatzz in #15765
- Update Docker image reference in aci_worker docs page by @discdiver in #15745
- Fix minor typo on “Deploy overview” page. by @Noah-Haf in #15771
- Add note about schedule kwargs change from 2.x to 3.x by @zzstoatzz in #15803
- Update @prefecthq/prefect-ui-library to version 3.11.3 by @marvin-robot in #15717
- Update @prefecthq/prefect-ui-library to version 3.11.4 by @marvin-robot in #15738
- Update @prefecthq/prefect-ui-library to version 3.11.7 by @marvin-robot in #15756
- Update @prefecthq/prefect-design to version 2.13.3 by @marvin-robot in #15721
- Update @prefecthq/prefect-design to version 2.13.4 by @marvin-robot in #15742
- Update @prefecthq/prefect-design to version 2.13.5 by @marvin-robot in #15785
- Update @prefecthq/prefect-ui-library to version 3.11.5 by @marvin-robot in #15743
- Update @prefecthq/prefect-ui-library to version 3.11.6 by @marvin-robot in #15746
- Bump tailwindcss from 3.4.13 to 3.4.14 in /ui by @dependabot in #15767
- Bump vite from 5.4.8 to 5.4.9 in /ui by @dependabot in #15766
3.0.10 - Serve returned
Released on October 15, 2024 This release fixes an issue where served flows would exit early and continues our reorganization of settings withpydantic-settings
.
Bug Fixes 🐞
- Fix bug where
.serve
exits early and add integration test by @zzstoatzz in #15691
- Add
APISettings
setting group by @desertaxle in #15701
3.0.9 - There’s no place like home
Released on October 15, 2024 Enhancements ✨- dont ask to confirm non-existent deployment by @zzstoatzz in #15653
- create
PREFECT_HOME
inroot_settings_context
by @zzstoatzz in #15696 - correctly await
config.load_kube_config
by @zzstoatzz in #15655 - Do not delete flow runs awaiting retry on deployment changes by @kevingrismore in #15688
- Ensure
TaskWorker
creates aanyio.TaskGroup
in anasync
context by @desertaxle in #15684
- Add tests to verify all settings via environment variables by @desertaxle in #15657
- Update
versioneer.py
to ignore uncommitted changes by @desertaxle in #15672 - Bump vue from 3.5.10 to 3.5.12 in /ui by @dependabot in #15675
- Add tests for specifying each setting via a profile by @desertaxle in #15670
- Add tests to verify all supported settings can be set via a
.env
file by @desertaxle in #15682 - Update @prefecthq/prefect-ui-library to version 3.11.2 by @marvin-robot in #15695
- Bump typescript from 5.6.2 to 5.6.3 in /ui by @dependabot in #15676
- Update @prefecthq/vue-compositions to version 1.11.5 by @marvin-robot in #15699
- remove note from 3.x docs about using
pydantic<2
by @zzstoatzz in #15642 - Update reference to name of profile in example by @discdiver in #15644
- follow on suggestion: simplify phrasing by @zzstoatzz in #15651
- Edits to dbt integration page by @discdiver in #15540
- Minor updates to deploy overview by @discdiver in #15606
- chore: update secret ref by @jamiezieziula in #15641
- Add link to release notes on What’s new docs page by @discdiver in #15662
- Docs note about accessing flow run logs by @EmilRex in #15683
3.0.8 - Revert a settings bug
Released on October 10, 2024 This release reverts a change that caused a loss of “set” environment variables in runtime environments.- Revert “Create
APISettings
to group to group API-specific client settings” by @zzstoatzz in #15648
3.0.7 - Version of a down bug
Released on October 10, 2024 This release includes a couple of bug fixes, including a fix for generating the Docker image name for the installed version ofprefect
.
Enhancements ➕➕
- Link to docs on starting temporary server by @zzstoatzz in #15636
- Fix PagerDuty notification formatting by @kevingrismore in #15584
- Fix version parsing error when generating Docker image name by @zzstoatzz in #15638
- Use the recommend approach for shallow dumping
pydantic
models by @zzstoatzz in #15634 - Add note about actively developed version to enhancement request form by @zzstoatzz in #15600
- Create
APISettings
to group to group API-specific client settings by @desertaxle in #15580 - Update image in Quickstart by @discdiver in #15635
3.0.6 - Client, know thyself
Released on October 09, 2024 This update focuses on fixingprefect-client
compatibility checks. Key changes include adding version information to prefect-client
and adjusting task logging levels.
Enhancements ➕➕
- Updated task creation and submission log level to
DEBUG
by @zzstoatzz in #15631
- Eliminated date range filter from Flow page by @dylanbhughes in #15630
- Accurately populate
__version__
forprefect_client
by @desertaxle in #15624
- Maintain typing information in
PrefectFuture
by @zzstoatzz in #15625
3.0.5 - Take care of the .environment
Released on October 08, 2024 This release introduces support for loading settings from.env
files, offering greater flexibility and ease of use in managing your Prefect configurations between projects.
New Features 🎉
- Allow specifying settings overrides in a
.env
by @zzstoatzz in #15565
- Update settings to use
pydantic-settings
by @zzstoatzz in #15412 - Add a helpful message on 404 with deployment by name by @zzstoatzz in #15578
- Changed
TaskSource
key computation to handleOSError("source not available")
by @kzvezdarov in #15583 - Fail correctly if
lookup_type
throws aKeyError
inBaseResult.__new__
by @zzstoatzz in #15596 - Fix issue where transaction parent look up fails in rollback hooks by @desertaxle in #15599
- Fix flow run parameters literally named
keys
by @zzstoatzz in #15611 - Ensure dynamic default values for settings are considered unset by
pydantic
by @zzstoatzz in #15613 - Fix errors when using
run_deployment
by @zzstoatzz in #15615 - Fix errors loading settings when profiles file can’t be read by @desertaxle in #15602
- Make database parameter optional in
ConnectionComponents
#15511 by @Ben-Taarit in #15553 - Update
AwsClientParameters
validation forverify
by @zzstoatzz in #15574
- Fix check to prerelease tag to allow nightly builds to publish Docker images by @desertaxle in #15563
- Update format for nightly dev release names by @desertaxle in #15568
- Add a note that deployments delegate ACLs to work pools by @abrookins in #15383
- Remove
await
for task submissions in global concurrency limits example by @biancaines in #15581 - Flow code storage docs improvements by @discdiver in #15519
- Update the store flow code docs by @discdiver in #15604
- Update first docs example by @discdiver in #15605
- Add documentation for
.env
file support by @zzstoatzz in #15617 - Update @prefecthq/prefect-design to version 2.13.2 by @marvin-robot in #15579
- Update @prefecthq/prefect-design to version 2.13.1 by @marvin-robot in #15576
3.0.4 - It’s been.. one week since you updated me
Released on October 01, 2024 Enhancements ➕➕- update
prefect deployment ls
to show work pool by @zzstoatzz in #15454 - Add —collision-strategy to
prefect deploy
CLI. by @collincchoy in #15468 - Don’t prompt custom image build for managed pools in
prefect deploy
by @zzstoatzz in #15484 - Deploy docs overview improvements by @discdiver in #15494
- Enforces deployment concurrency with orchestration polices. by @jeanluciano in #15504
- add
get_current_workspace
cloud client method by @zzstoatzz in #15542 - Enhancement: Add
end_time
to flow run filters by @znicholasbrown in #15502 - Write raw results from flows by @desertaxle in #15434
- Update
transaction
context manager to defaultwrite_on_commit
toTrue
by @desertaxle in #15515
- Use correct JSON schema dialect for param validation by @GfxKai in #15483
- Fix block reference load in async flow by @GalLadislav in #15487
- Add regression test for arbitrary state data by @desertaxle in #15506
- Update tasks to inherit the lock manager from a parent transaction if present by @desertaxle in #15505
- override
_coerce_env
for k8s worker by @zzstoatzz in #15522
- Bumps
pytest
andpytest-asyncio
to the latest versions by @chrisguidry in #15523 - Avoid shutting down Ray driver when running on a Ray worker by @desertaxle in #15552
- Fix dbt source freshness fail state by @e1337us3r in #15510
- Allow oracle connection urls by @zzstoatzz in #15496
- Use
ConnectionComponents
components fromprefect-sqlalchemy
inprefect-dbt
by @westford14 in #15499
- Improve docstring
state.result
for when used withrun_deployment
by @benjamincerigo in #15509 - Fix use of deprecated field on
websockets.exceptions.ConnectionClosedError
by @desertaxle in #15467 - Set up pytest-markdown-docs for documentation testing by @bunchesofdonald in #15457
- fix test by @zzstoatzz in #15480
- one small type fix by @zzstoatzz in #15479
- use internal logger for
get_lost_followers
retries by @zzstoatzz in #15518 - Update weekly RC workflow to a nightly development workflow by @desertaxle in #15545
- add note on terminal unawaited coro by @zzstoatzz in #15439
- Add docs on securing webhooks with service accounts by @bunchesofdonald in #15481
- Fix README code example by @OverShifted in #15492
- Fixes ugly table formatting from bash output in docs by @discdiver in #15495
- Update global concurrency limits docs for removed create_if_missing behavior by @collincchoy in #15517
- Warn if websocket connection can’t be made by @jakekaplan in #15261
- Fix flow typing when passing return_state to a coroutine by @williamjamir in #15528
- Docs for deployment concurrency limits by @collincchoy in #15527
- Fix automations-triggers doc tests by @bunchesofdonald in #15514
- Remove worker and runner handling for deployment concurrency by @jeanluciano in #15497
3.0.3 - Cache Rules Everything Around Me
Released on September 20, 2024 This release introduces improvements to cache policy configuration, offering developers more control and flexibility in managing task result caching:- Customizable Storage Location: Specify where cache entries are stored using the
key_storage
parameter. - Configurable Isolation Level: Set the isolation level for cache access with the
isolation_level
parameter. - Locking Mechanism: Implement concurrent access control using the
lock_manager
parameter.
- Add
key_storage
,isolation_level
andlock_manager
configuration to cache policies by @desertaxle in #15382
- Adds settings to control server CORS configuration by @desertaxle in #15396
- Add ability to specify
ConcurrencyOptions
by @jeanluciano in #15291 - Add
concurrency_limit
field toflow.serve
by @jeanluciano in #15376 - Allow a rich object to be passed to
concurrency_limit
in client methods by @jeanluciano in #15425 - Add support for insecure webhooks by @westford14 in #15402
- Improve deployment concurrency GCL management by @collincchoy in #15426
- Preserve
flow.name
withRunnerDeployment.from_storage
by @zzstoatzz in #15393 - Lazy import
ray
to fix server startup issues by @desertaxle in #15392 - Handle unnamed lambdas for consistency by @zzstoatzz in #15400
- Fix task linking by @desertaxle in #15418
- Fix ephemeral server start command for Windows by @desertaxle in #15421
- Fix task run count when a task is configured with
retry_delay_seconds
by @desertaxle in #15424 - Fix bug where task cache storage is misconfigured by @desertaxle in #15433
- Update task engine to increment
run_count
when enteringRUNNING
state by @desertaxle in #15436
- Add environment variables from GCP secrets to cloud run job v2 workers by @Ultramann in #15423
- Prepare
prefect-redis
for0.2.0
release by @desertaxle in #15409 - Add
AzureBlobStorageContainer.list_blobs
method by @westford14 in #15390 - Add
oracledb
driver support inprefect-sqlalchemy
by @Ben-Taarit in #15385
- Add docs for configuring storage and isolation for cache policies by @desertaxle in #15397
- Improve type hinting docs for sync_compatible by @benjamincerigo in #15327
- Refactors
__add__
and__sub__
on cache policies by @desertaxle in #15379 - Add
mypy
type checking to/models
by @bunchesofdonald in #15062 - Only load collections once by @abrookins in #15344
- Bump vue from 3.5.4 to 3.5.5 in /ui by @dependabot in #15386
- Bump typescript from 5.5.4 to 5.6.2 in /ui by @dependabot in #15387
- Bump tailwindcss from 3.4.10 to 3.4.11 in /ui by @dependabot in #15388
- Bump vite from 5.4.2 to 5.4.5 in /ui by @dependabot in #15372
- Bump vue-router from 4.4.3 to 4.4.5 in /ui by @dependabot in #15389
- Update @prefecthq/prefect-ui-library to version 3.9.5 by @marvin-robot in #15411
- Bump vite from 5.4.5 to 5.4.6 in /ui by @dependabot in #15410
- Update @prefecthq/prefect-ui-library to version 3.10.0 by @marvin-robot in #15419 Full Changelog: https://github.com/PrefectHQ/prefect/compare/3.0.2…3.0.3
3.0.2 - Groundlog day, tomorrow finally came
Released on September 13, 2024 New Features 🎉- Add a ResumeFlowRun automations action by @abrookins in #15269
- Notification security controls by @cicdw in #15272
- Create
FilesystemLockManager
by @desertaxle in #15307 - Concurrency limit UX enhancements: add
strict
mode by @cicdw in #15297 - Make
prefect_test_harness
startup timeout configurable by @desertaxle in #15354 - Fix deployment-concurrency-limiting-GCL lifecycle management by @collincchoy in #15365
Runner
enforcesDeployment.concurrency_limit
by @jeanluciano in #15085- Disable HTTP2 by default by @zangell44 in #15311
- Move expensive filter to AND by @cicdw in #15286
- Fix deep merge for existing
env
on work-pools by @zzstoatzz in #15325 - Ensure all tags are used in concurrency limiter by @cicdw in #15349
- Set propagate to false for extra loggers by @cicdw in #15348
- Allow child transactions to access parent values set after child transaction opens by @desertaxle in #15342
- Fix resolution of implicit relative import when using
safe_load_namespace
by @desertaxle in #15358 - Correctly handles recurring automations and future events by @chrisguidry in #15355
- reintroduce job model for
prefect-databricks
by @zzstoatzz in #15313 - Update
DbtCoreOperation
docstring by @zzstoatzz in #15317 - Move import from
google
into import guard by @desertaxle in #15334 - fix
S3Bucket.download_object_to_path
by @zzstoatzz in #15293 - Fix issue where
DaskTaskRunner
was attempting to create a duplicate cluster by @desertaxle in #15341
- Add dedicated enhancements section by @cicdw in #15265
- Fix labeler job by @cicdw in #15114
- Update @prefecthq/prefect-ui-library to version 3.8.2 by @marvin-robot in #15283
- Refactor some concurrency utilities to be sync_compatible by @cicdw in #15273
- Add some more nuance to labeler by @cicdw in #15288
- Use
ResultStore
by default for task transactions by @desertaxle in #15264 - Update @prefecthq/prefect-ui-library to version 3.8.3 by @marvin-robot in #15305
- Remove feature flag for nested task runs on the run graph by @pleek91 in #15304
- Add deprecation warnings to all
RecordStore
implementations by @desertaxle in #15314 - Remove some ancient deprecations by @cicdw in #15310
- Update @prefecthq/prefect-ui-library to version 3.9.1 by @marvin-robot in #15335
- Replace
RedisRecordStore
withRedisLockManager
by @desertaxle in #15320 - Add deprecation warnings to results classes and methods by @desertaxle in #15323
- Update @prefecthq/prefect-ui-library to version 3.9.2 by @marvin-robot in #15346
- Update @prefecthq/prefect-ui-library to version 3.9.4 by @marvin-robot in #15356
- Bump vue from 3.4.38 to 3.5.4 in /ui by @dependabot in #15306
- Bump @vitejs/plugin-vue from 5.1.2 to 5.1.3 in /ui by @dependabot in #15173
- Bump vue-tsc from 2.0.29 to 2.1.6 in /ui by @dependabot in #15281
- Update transactions docs by @discdiver in #15299
- Add unit test for accessing
self
as normal arg when setting task run name by @jlowin in #15324 - add
for_each
trigger example by @zzstoatzz in #15326 - Update logging.mdx to fix title display by @discdiver in #15360
- quote type checked value by @zzstoatzz in #15362
- fix: move the github conditional to the appropriate level by @jamiezieziula in #15366
- Fix broken image link and update formatting in prefect-dbt by @discdiver in #15367
- Update integrations docs by @zzstoatzz in #15303
- Rewrite block page in docs by @cicdw in #15254
- chore: only trigger helm release workflows on 3.x.x version releases by @jamiezieziula in #15268
- better common error section by @zzstoatzz in #15271
- fix inconsistent word in upgrade guide by @zzstoatzz in #15277
- Minor edits to the upgrade and what’s new docs by @discdiver in #15285
3.0.1 - The Director’s Cut
Released on September 06, 2024 This release contains a large collection of relatively small improvements and bug fixes on the back of our major 3.0.0 release. One particular set of improvements that I’d like to focus on are enhancements to server side logging and event handling. We received a few reports of task run states not showing up in the UI/API. It’s important to note that events - and now task run states! - require a websocket connection to the API (the good news is that this will not disrupt workflow execution). Websockets are sometimes blocked by default in firewalls and network configuration, so we are working to raise failures to establish such a connection more prominently and potentially allow for alternative configurations. If you notice any issues with this, please let us know! Enhancements ➕➕- update
prompt_select_from_table
by @zzstoatzz in #15202 - Adds ability to write result metadata separate from data when using
ResultStore
by @desertaxle in #15214 - Remove
CausalOrdering
fromtask-run-recorder
by @jakekaplan in #15244 - Disable griffe logging entirely when used by @cicdw in #15193
- Allow using a
ResultStore
with aTransaction
by @desertaxle in #15247 - Introduce
LockManager
protocol and add locking abilities toResultStore
by @desertaxle in #15225 - check api compatibility when entering a client context by @jakekaplan in #15252
- Add debug logging for server side event handling by @jakekaplan in #15245
- Fix task key comp by @cicdw in #15190
- Fix for Github clone on Windows by @WWM-jschuba in #15143
- remove duplicate
pull_code
instruction by @zzstoatzz in #15215 - add function name to
prefect-docker
cache key by @zzstoatzz in #15262 - Deployment.concurrency_limit cannot be zero. by @collincchoy in #15228
- Add
KeepAliveClientRequest
class for k8s async client by @kevingrismore in #15220 - Kubernetes worker uses
enable_socket_keep_alive
by @jeanluciano in #15142 prefect-kubernetes
: add missing config loaders by @srsapient in #15209- fix
prefect-azure
tests by @zzstoatzz in #15211
- Fix test flake and add
uv
dev dependency by @cicdw in #15251 - Fix release pipeline for integration packages by @desertaxle in #15189
- Touch up variable class inheritance by @cicdw in #15188
- Update @prefecthq/prefect-ui-library to version 3.8.1 by @marvin-robot in #15243
- Fix test failures for
pydantic==2.9.0
by @desertaxle in #15249 - use
is not
for type comparison instead of!=
by @zzstoatzz in #15253 - Updates the worker integration tests for post-GA by @chrisguidry in #15199