-
Notifications
You must be signed in to change notification settings - Fork 265
Make all format options available when using --dbt
#409
Comments
I'm not sure I agree as we are targeting two very diff erent uses cases in these two features, so I do think it makes sense for the output to not align by default Migrations (primary data-diff use case) vs. dbt model development. But 100% we need parity across features around the output options (not to mention the other diff options flags)
lol true @williebsweet curious on your thoughts here |
😂 The output format of regular At minimum, I'd like to be able to configure the output format via the But interested to hear everyone's thoughts on output formats that would be most useful during dbt model development! In particular, how are the needs of Mary the Migrator and Dylan the dbt Developer similar and how are their needs different? Is there particular diff output one persona needs that the other doesn't? Is there any overlap? |
This issue has been marked as stale because it has been open for 60 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days. |
@dlawin could we mark this as un-stale? My feature proposal is to achieve parity across the output options (when using the data-diff --dbt --dbt-profiles-dir . data-diff --dbt --dbt-profiles-dir . --json data-diff --dbt --dbt-profiles-dir . --stats |
Hey @dbeatty10! |
No worries @kylemcnair -- stalebot is definitely handy to keep the backlog more manageable. At the moment, the following is more of a link to interesting information rather than a feature request... Another diff format to take a peek at is the one used by I believe it originally came from the coopy toolbox, and you can see it in action here: A relevant git repo is here: |
@dbeatty10 I do really like the daff format, and believe it's a very good way to display row-level data diff info in CLI output, much better imo than the current "regular" (non-dbt) |
This issue has been marked as stale because it has been open for 60 days with no activity. If you would like the issue to remain open, please comment on the issue and it will be added to the triage queue. Otherwise, it will be closed in 7 days. |
@kylemcnair would you be open to either removing |
Hi @dbeatty10! I want to add that the traditional I know this from experience when trying to use 'main' I believe the output of 'main'
I support the general goal of feature parity across outputs for these different use cases (Mary and Dylan), but it doesn't seem trivial or obvious to me to identify what the ideal common output structure would be. (As you mentioned above, there are distinctly different use cases.) I just want to make sure this is taken into account before something is designed and shipped. Perhaps the best answer is high-level parity, as in: the two use cases should have both machine-readable and human-readable summarized output; and machine-readable value-level output; but, the exact structure and implementation might differ for the two use cases. Finally, I'm curious if these recently shipped features influence your opinion on the direction and what should be prioritized:
|
@dbeatty10 I'm planning to take on this feature for the next couple weeks. Let me know if any of your opinions changed on the above! Just a heads up this will likely turn into a cloud limited preview feature similar to the |
@kylemcnair would love your specific input since you last commented on this thread! |
Nice, @sungchun12 😎 The above comments still reflect my primary thoughts:
|
Alright, since this will be a pretty big change. I have a couple proposals in mind. Scope:
Design Notes:
Research: |
I agree that the outputs ( |
I think sampling should be opt-in because I suspect it will be pretty hard to render sometimes, and I don't want people to have to run a diff, see that samples are rendered poorly, run it again, etc. OTOH, if sampling renders nicely every time, even when there are 250 columns, then I would support opt-out. |
Is your feature request related to a problem? Please describe.
Currently, the
--dbt
flag uses a different output format than non---dbt
executions ofdata-diff
.Describe the solution you'd like
Ideally, regular
data-diff
anddata-diff --dbt
would have the same output format. Additionally,--json
and--stats
should be possible when using--dbt
.Describe alternatives you've considered
It's an option for
--dbt
to have completely different formatting and for the--json
and--stats
flags to not have any effect. But that alternative makes dolphins cry 🐬 😢Additional context
I'll submit a PR!
The text was updated successfully, but these errors were encountered: