-
Notifications
You must be signed in to change notification settings - Fork 160
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Initial implementation of simplifying config interfaces #389
Conversation
👷 Deploy Preview for amazing-pothos-a3bca0 processing.
|
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## main #389 +/- ##
==========================================
+ Coverage 91.24% 91.40% +0.16%
==========================================
Files 46 48 +2
Lines 1610 1664 +54
==========================================
+ Hits 1469 1521 +52
- Misses 141 143 +2
☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this interface looks good to me, pushed some minor fixes
@tatiana wdyt? |
Co-authored-by: Tatiana Al-Chueyr <[email protected]>
Co-authored-by: Tatiana Al-Chueyr <[email protected]>
Description
Over time, we've continued adding parameters everywhere without giving much thought to the interface. This PR aims to simplify the interface for working with Cosmos DAGs/Task Groups and operators by collapsing the arguments into four config objects:
ProjectConfig
: parameters that Cosmos needs to understand where your dbt project isRenderConfig
: parameters to tell Cosmos how to render your dbt project as a DAGExecutionConfig
: parameters to tell Cosmos how to execute your dbt projectProfileConfig
: parameters to tell Cosmos what to run your dbt againstYou can see the exact parameters each config takes here: https://github.com/astronomer/astronomer-cosmos/blob/simplify-interface/cosmos/config.py
It also:
dbt_project_name
anddbt_root_path
arguments into a single argdbt_args
, which was a duplicate ofoperator_args
Related Issue(s)
closes #372
Breaking Change?
Yes, the parameters to the DAG/Task Group and operators change. This is an intentional breaking change before the 1.0 release.
Checklist