Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lateral view explode missing relationship #636

Open
Light-Towers opened this issue Jul 12, 2024 · 0 comments
Open

lateral view explode missing relationship #636

Light-Towers opened this issue Jul 12, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@Light-Towers
Copy link

Light-Towers commented Jul 12, 2024

Describe the bug

  • lateral view explode missing the relationship between table dw.scr and dw.dest.

To Reproduce
test.py

# -*- coding: utf-8 -*-
from sqllineage.runner import LineageRunner
def test_create_as():
    sql = """

INSERT OVERWRITE table dw.dest partition (dt)
select
    a.nameid,
    get_json_object(b.alteration_list,'$.alterDate') as alter_date,
    get_json_object(b.alteration_list,'$.alterItem') as alter_item,
    get_json_object(b.alteration_list,'$.preAlter') as alter_before,
    get_json_object(b.alteration_list,'$.postAlter') as alter_after,
    current_date() as dt
from (
     select t1.nameid,
            split(regexp_replace(regexp_replace(t1.alterationList, '[\\[\\]]', ''), '(},\\{)', '}#\\{'), '#') as dd
     from (select nameid, alterationlist
           from dw.src
           where alterationlist is not null
             and alterationlist not rlike '暂无数据'
             and alterationlist != '[]'
             and nameid is not null
          ) t1
 ) a lateral view explode(a.dd) b as alteration_list;

    """
    result = LineageRunner(sql, 'hive')

    result.print_column_lineage()
    print(result.source_tables)
    print(result.target_tables)

if __name__ == "__main__":
    test_create_as()
============================= test session starts =============================
collecting ... collected 1 item

testGetSqllineage.py::test_create_as PASSED                              [100%]dw.dest.alter_after <- <default>.b.alteration_list
dw.dest.alter_before <- <default>.b.alteration_list
dw.dest.alter_date <- <default>.b.alteration_list
dw.dest.alter_item <- <default>.b.alteration_list
dw.dest.nameid <- a.nameid <- t1.nameid <- dw.src.nameid
[Table: dw.src]
[Table: dw.dest]


============================== 1 passed in 1.75s ==============================

Expected behavior
maybe output should like dw.dest.alter_date <- b.alteration_list <- a.dd <- dw.src.alterationList

Python version (available via python --version)

  • 3.10.11

SQLLineage version (available via sqllineage --version):

  • 1.5.3
@Light-Towers Light-Towers added the bug Something isn't working label Jul 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant