-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ISSUE-19454: Fixes broken looker lineage #19456
Conversation
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
I have verified that this solves the issue when running it in our environment. |
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
source_table_name = self._clean_table_name(sql_table_name, dialect) | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I f i understand correctly this is so the query is compiled to the correct dialect, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The cleaning of the table name already occurs, this is just adding the functionality of making `project_id.dataset_id.table_id` -> project_id.dataset_id.table_id
. This is common to enclose table references like this in BigQuery. Our looker views are defined like:
sql_table_name: `project_id.dataset_id.table_id` ;;
. The Looker ingestion then tries to find lineage references by searching for:
db: `project_id
schema: dataset_id
table: table_id`
Thus we find no lineage between looker <-> BQ
This addition in the _clean_table_name
method will address that
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I.e. it is not compiling the query. That is happening only if the table ref is a derived table in Looker. That scenario is already handled in the elif
statement
ingestion/src/metadata/ingestion/source/dashboard/looker/metadata.py
Outdated
Show resolved
Hide resolved
…ata.py Co-authored-by: Mayur Singal <[email protected]>
Quality Gate passed for 'open-metadata-ingestion'Issues Measures |
Describe your changes:
Fixes 19454
Looker lineage is broken when a table reference is enclosed by backticks (common when using BigQuery).
This addresses that issue.
Type of change:
Checklist:
Fixes <issue-number>: <short explanation>