Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CH] Eliminate common subexpression by ExpressionParser #8283

Open
lgbo-ustc opened this issue Dec 19, 2024 · 1 comment · May be fixed by #8284
Open

[CH] Eliminate common subexpression by ExpressionParser #8283

lgbo-ustc opened this issue Dec 19, 2024 · 1 comment · May be fixed by #8284
Labels
enhancement New feature or request performance

Comments

@lgbo-ustc
Copy link
Contributor

Description

We have had CommonSubexpressionEliminateRule to eliminate common subexpression before, but it cannot cover all cases.

  1. If there are nested common subexpressions, it may cause deep nested projections
  2. Some expressions are added inside native, spark rule cannot handle these cases.

Since we have moved all conversions from substrait expression to CH ActionsDAG into ExpressionParser, we can add the CSE elimination in ExpressionParser.
@lgbo-ustc lgbo-ustc added enhancement New feature or request performance labels Dec 19, 2024
@lgbo-ustc
Copy link
Contributor Author

lgbo-ustc commented Dec 19, 2024 •
edited
Loading

An example as following.

                select
                        case when bus_type = 34  then split(order_id,'_')[2]
                                when  bus_type   in (116,118) then split(order_id,'_')[4]
                                when  bus_type   in (117) then split(order_id,'_')[5]  end     as uid

                        ,case when bus_type = 34  then split(order_id,'_')[4]
                                when  bus_type   in (116,118) then split(order_id,'_')[6]
                                when  bus_type   in (117) then split(order_id,'_')[7]  end
                        as  id
                from t
                where day  = '2024-12-15'
                and  appid=60
                and bus_type in (34,116,117,118) ;

Without elimination in ExpressionParser, we get following actions dag, it calculates split(order_id,'_') repeatly. CommonSubexpressionEliminateRule has missed something.

  Actions: INPUT : 0 -> order_id Nullable(String) : 0
           INPUT : 1 -> bus_type Nullable(Int32) : 1
           COLUMN Const(Int32) -> 34_9 Int32 : 2
           COLUMN Const(String) -> __10 String : 3
           COLUMN Const(Int32) -> -1_13 Int32 : 4
           COLUMN Const(String) -> Nullable(Array(Nullable(String))) String : 5
           COLUMN Const(Int32) -> plus(1_14,2_17) Int32 : 6
           COLUMN Const(UInt8) -> lessOrEquals(plus(1_14,2_17),0_18) UInt8 : 7
           COLUMN Const(Nullable(String)) -> NULL_20 Nullable(String) : 8
           COLUMN Set -> __set_22 Set : 9
           COLUMN Const(String) -> __24 String : 10
           COLUMN Const(Int32) -> -1_25 Int32 : 11
           COLUMN Const(String) -> Nullable(Array(Nullable(String))) String : 12
           COLUMN Const(Int32) -> plus(1_28,4_29) Int32 : 13
           COLUMN Const(UInt8) -> lessOrEquals(plus(1_28,4_29),0_32) UInt8 : 14
           COLUMN Const(Nullable(String)) -> NULL_34 Nullable(String) : 15
           COLUMN Const(Int32) -> 117_36 Int32 : 16
           COLUMN Const(String) -> __38 String : 17
           COLUMN Const(Int32) -> -1_39 Int32 : 18
           COLUMN Const(String) -> Nullable(Array(Nullable(String))) String : 19
           COLUMN Const(Int32) -> plus(1_42,5_43) Int32 : 20
           COLUMN Const(UInt8) -> lessOrEquals(plus(1_42,5_43),0_47) UInt8 : 21
           COLUMN Const(Nullable(String)) -> NULL_49 Nullable(String) : 22
           COLUMN Const(Nullable(String)) -> NULL_50 Nullable(String) : 23
           COLUMN Const(Int32) -> 34_52 Int32 : 24
           COLUMN Const(String) -> __54 String : 25
           COLUMN Const(Int32) -> -1_55 Int32 : 26
           COLUMN Const(String) -> Nullable(Array(Nullable(String))) String : 27
           COLUMN Const(Int32) -> plus(1_58,4_59) Int32 : 28
           COLUMN Const(UInt8) -> lessOrEquals(plus(1_58,4_59),0_62) UInt8 : 29
           COLUMN Const(Nullable(String)) -> NULL_64 Nullable(String) : 30
           COLUMN Set -> __set_66 Set : 31
           COLUMN Const(String) -> __68 String : 32
           COLUMN Const(Int32) -> -1_69 Int32 : 33
           COLUMN Const(String) -> Nullable(Array(Nullable(String))) String : 34
           COLUMN Const(Int32) -> plus(1_72,6_74) Int32 : 35
           COLUMN Const(UInt8) -> lessOrEquals(plus(1_72,6_74),0_77) UInt8 : 36
           COLUMN Const(Nullable(String)) -> NULL_78 Nullable(String) : 37
           COLUMN Const(Int32) -> 117_80 Int32 : 38
           COLUMN Const(String) -> __82 String : 39
           COLUMN Const(Int32) -> -1_83 Int32 : 40
           COLUMN Const(String) -> Nullable(Array(Nullable(String))) String : 41
           COLUMN Const(Int32) -> plus(1_86,7_87) Int32 : 42
           COLUMN Const(UInt8) -> lessOrEquals(plus(1_86,7_87),0_90) UInt8 : 43
           COLUMN Const(Nullable(String)) -> NULL_92 Nullable(String) : 44
           COLUMN Const(Nullable(String)) -> NULL_94 Nullable(String) : 45
           INPUT :: 2 -> hour Nullable(String) : 46
           INPUT :: 3 -> day Nullable(String) : 47
           INPUT :: 4 -> appid Int32 : 48
           FUNCTION equals(bus_type : 1, 34_9 :: 2) -> equals(bus_type,34_9) Nullable(Bool) : 49
           FUNCTION splitByChar(__10 :: 3, order_id : 0, -1_13 :: 4) -> splitByRegexpSpark(__10,order_id,-1_13) Nullable(Array(String)) : 2
           FUNCTION in(bus_type : 1, __set_22 :: 9) -> in(bus_type,__set_22) Nullable(UInt8) : 4
           FUNCTION splitByChar(__24 :: 10, order_id : 0, -1_25 :: 11) -> splitByRegexpSpark(__24,order_id,-1_25) Nullable(Array(String)) : 9
           FUNCTION equals(bus_type : 1, 117_36 :: 16) -> equals(bus_type,117_36) Nullable(Bool) : 11
           FUNCTION splitByChar(__38 :: 17, order_id : 0, -1_39 :: 18) -> splitByRegexpSpark(__38,order_id,-1_39) Nullable(Array(String)) : 16
           FUNCTION equals(bus_type : 1, 34_52 :: 24) -> equals(bus_type,34_52) Nullable(Bool) : 18
           FUNCTION splitByChar(__54 :: 25, order_id : 0, -1_55 :: 26) -> splitByRegexpSpark(__54,order_id,-1_55) Nullable(Array(String)) : 24
           FUNCTION in(bus_type : 1, __set_66 :: 31) -> in(bus_type,__set_66) Nullable(UInt8) : 26
           FUNCTION splitByChar(__68 :: 32, order_id : 0, -1_69 :: 33) -> splitByRegexpSpark(__68,order_id,-1_69) Nullable(Array(String)) : 31
           FUNCTION equals(bus_type :: 1, 117_80 :: 38) -> equals(bus_type,117_80) Nullable(Bool) : 33
           FUNCTION splitByChar(__82 :: 39, order_id :: 0, -1_83 :: 40) -> splitByRegexpSpark(__82,order_id,-1_83) Nullable(Array(String)) : 38
           FUNCTION _CAST(splitByRegexpSpark(__10,order_id,-1_13) :: 2, Nullable(Array(Nullable(String))) :: 5) -> splitByRegexpSpark(__10,order_id,-1_13) Nullable(Array(Nullable(String))) : 40
           FUNCTION _CAST(splitByRegexpSpark(__24,order_id,-1_25) :: 9, Nullable(Array(Nullable(String))) :: 12) -> splitByRegexpSpark(__24,order_id,-1_25) Nullable(Array(Nullable(String))) : 5
           FUNCTION _CAST(splitByRegexpSpark(__38,order_id,-1_39) :: 16, Nullable(Array(Nullable(String))) :: 19) -> splitByRegexpSpark(__38,order_id,-1_39) Nullable(Array(Nullable(String))) : 12
           FUNCTION _CAST(splitByRegexpSpark(__54,order_id,-1_55) :: 24, Nullable(Array(Nullable(String))) :: 27) -> splitByRegexpSpark(__54,order_id,-1_55) Nullable(Array(Nullable(String))) : 19
           FUNCTION _CAST(splitByRegexpSpark(__68,order_id,-1_69) :: 31, Nullable(Array(Nullable(String))) :: 34) -> splitByRegexpSpark(__68,order_id,-1_69) Nullable(Array(Nullable(String))) : 27
           FUNCTION _CAST(splitByRegexpSpark(__82,order_id,-1_83) :: 38, Nullable(Array(Nullable(String))) :: 41) -> splitByRegexpSpark(__82,order_id,-1_83) Nullable(Array(Nullable(String))) : 34
           FUNCTION arrayElementOrNull(splitByRegexpSpark(__10,order_id,-1_13) :: 40, plus(1_14,2_17) :: 6) -> arrayElementOrNull(splitByRegexpSpark(__10,order_id,-1_13),plus(1_14,2_17)) Nullable(String) : 41
           FUNCTION arrayElementOrNull(splitByRegexpSpark(__24,order_id,-1_25) :: 5, plus(1_28,4_29) :: 13) -> arrayElementOrNull(splitByRegexpSpark(__24,order_id,-1_25),plus(1_28,4_29)) Nullable(String) : 6
           FUNCTION arrayElementOrNull(splitByRegexpSpark(__38,order_id,-1_39) :: 12, plus(1_42,5_43) :: 20) -> arrayElementOrNull(splitByRegexpSpark(__38,order_id,-1_39),plus(1_42,5_43)) Nullable(String) : 13
           FUNCTION arrayElementOrNull(splitByRegexpSpark(__54,order_id,-1_55) :: 19, plus(1_58,4_59) :: 28) -> arrayElementOrNull(splitByRegexpSpark(__54,order_id,-1_55),plus(1_58,4_59)) Nullable(String) : 20
           FUNCTION arrayElementOrNull(splitByRegexpSpark(__68,order_id,-1_69) :: 27, plus(1_72,6_74) :: 35) -> arrayElementOrNull(splitByRegexpSpark(__68,order_id,-1_69),plus(1_72,6_74)) Nullable(String) : 28
           FUNCTION arrayElementOrNull(splitByRegexpSpark(__82,order_id,-1_83) :: 34, plus(1_86,7_87) :: 42) -> arrayElementOrNull(splitByRegexpSpark(__82,order_id,-1_83),plus(1_86,7_87)) Nullable(String) : 35
           FUNCTION if(lessOrEquals(plus(1_14,2_17),0_18) :: 7, NULL_20 :: 8, arrayElementOrNull(splitByRegexpSpark(__10,order_id,-1_13),plus(1_14,2_17)) :: 41) -> if(lessOrEquals(plus(1_14,2_17),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__10,order_id,-1_13),plus(1_14,2_17))) Nullable(String) : 42
           FUNCTION if(lessOrEquals(plus(1_28,4_29),0_32) :: 14, NULL_34 :: 15, arrayElementOrNull(splitByRegexpSpark(__24,order_id,-1_25),plus(1_28,4_29)) :: 6) -> if(lessOrEquals(plus(1_28,4_29),0_32),NULL_34,arrayElementOrNull(splitByRegexpSpark(__24,order_id,-1_25),plus(1_28,4_29))) Nullable(String) : 41
           FUNCTION if(lessOrEquals(plus(1_42,5_43),0_47) :: 21, NULL_49 :: 22, arrayElementOrNull(splitByRegexpSpark(__38,order_id,-1_39),plus(1_42,5_43)) :: 13) -> if(lessOrEquals(plus(1_42,5_43),0_47),NULL_49,arrayElementOrNull(splitByRegexpSpark(__38,order_id,-1_39),plus(1_42,5_43))) Nullable(String) : 6
           FUNCTION if(lessOrEquals(plus(1_58,4_59),0_62) :: 29, NULL_64 :: 30, arrayElementOrNull(splitByRegexpSpark(__54,order_id,-1_55),plus(1_58,4_59)) :: 20) -> if(lessOrEquals(plus(1_58,4_59),0_62),NULL_64,arrayElementOrNull(splitByRegexpSpark(__54,order_id,-1_55),plus(1_58,4_59))) Nullable(String) : 13
           FUNCTION if(lessOrEquals(plus(1_72,6_74),0_77) :: 36, NULL_78 :: 37, arrayElementOrNull(splitByRegexpSpark(__68,order_id,-1_69),plus(1_72,6_74)) :: 28) -> if(lessOrEquals(plus(1_72,6_74),0_77),NULL_78,arrayElementOrNull(splitByRegexpSpark(__68,order_id,-1_69),plus(1_72,6_74))) Nullable(String) : 20
           FUNCTION if(lessOrEquals(plus(1_86,7_87),0_90) :: 43, NULL_92 :: 44, arrayElementOrNull(splitByRegexpSpark(__82,order_id,-1_83),plus(1_86,7_87)) :: 35) -> if(lessOrEquals(plus(1_86,7_87),0_90),NULL_92,arrayElementOrNull(splitByRegexpSpark(__82,order_id,-1_83),plus(1_86,7_87))) Nullable(String) : 28
           FUNCTION multiIf(equals(bus_type,34_9) :: 49, if(lessOrEquals(plus(1_14,2_17),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__10,order_id,-1_13),plus(1_14,2_17))) :: 42, in(bus_type,__set_22) :: 4, if(lessOrEquals(plus(1_28,4_29),0_32),NULL_34,arrayElementOrNull(splitByRegexpSpark(__24,order_id,-1_25),plus(1_28,4_29))) :: 41, equals(bus_type,117_36) :: 11, if(lessOrEquals(plus(1_42,5_43),0_47),NULL_49,arrayElementOrNull(splitByRegexpSpark(__38,order_id,-1_39),plus(1_42,5_43))) :: 6, NULL_50 :: 23) -> multiIf(equals(bus_type,34_9),if(lessOrEquals(plus(1_14,2_17),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__10,order_id,-1_13),plus(1_14,2_17))),in(bus_type,__set_22),if(lessOrEquals(plus(1_28,4_29),0_32),NULL_34,arrayElementOrNull(splitByRegexpSpark(__24,order_id,-1_25),plus(1_28,4_29))),equals(bus_type,117_36),if(lessOrEquals(plus(1_42,5_43),0_47),NULL_49,arrayElementOrNull(splitByRegexpSpark(__38,order_id,-1_39),plus(1_42,5_43))),NULL_50) Nullable(String) : 35
           FUNCTION multiIf(equals(bus_type,34_52) :: 18, if(lessOrEquals(plus(1_58,4_59),0_62),NULL_64,arrayElementOrNull(splitByRegexpSpark(__54,order_id,-1_55),plus(1_58,4_59))) :: 13, in(bus_type,__set_66) :: 26, if(lessOrEquals(plus(1_72,6_74),0_77),NULL_78,arrayElementOrNull(splitByRegexpSpark(__68,order_id,-1_69),plus(1_72,6_74))) :: 20, equals(bus_type,117_80) :: 33, if(lessOrEquals(plus(1_86,7_87),0_90),NULL_92,arrayElementOrNull(splitByRegexpSpark(__82,order_id,-1_83),plus(1_86,7_87))) :: 28, NULL_94 :: 45) -> multiIf(equals(bus_type,34_52),if(lessOrEquals(plus(1_58,4_59),0_62),NULL_64,arrayElementOrNull(splitByRegexpSpark(__54,order_id,-1_55),plus(1_58,4_59))),in(bus_type,__set_66),if(lessOrEquals(plus(1_72,6_74),0_77),NULL_78,arrayElementOrNull(splitByRegexpSpark(__68,order_id,-1_69),plus(1_72,6_74))),equals(bus_type,117_80),if(lessOrEquals(plus(1_86,7_87),0_90),NULL_92,arrayElementOrNull(splitByRegexpSpark(__82,order_id,-1_83),plus(1_86,7_87))),NULL_94) Nullable(String) : 23
           ALIAS multiIf(equals(bus_type,34_9),if(lessOrEquals(plus(1_14,2_17),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__10,order_id,-1_13),plus(1_14,2_17))),in(bus_type,__set_22),if(lessOrEquals(plus(1_28,4_29),0_32),NULL_34,arrayElementOrNull(splitByRegexpSpark(__24,order_id,-1_25),plus(1_28,4_29))),equals(bus_type,117_36),if(lessOrEquals(plus(1_42,5_43),0_47),NULL_49,arrayElementOrNull(splitByRegexpSpark(__38,order_id,-1_39),plus(1_42,5_43))),NULL_50) :: 35 -> multiIf(equals(bus_type,34_9),if(lessOrEquals(plus(1_14,2_17),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__10,order_id,-1_13),plus(1_14,2_17))),in(bus_type,__set_22),if(lessOrEquals(plus(1_28,4_29),0_32),NULL_34,arrayElementOrNull(splitByRegexpSpark(__24,order_id,-1_25),plus(1_28,4_29))),equals(bus_type,117_36),if(lessOrEquals(plus(1_42,5_43),0_47),NULL_49,arrayElementOrNull(splitByRegexpSpark(__38,order_id,-1_39),plus(1_42,5_43))),NULL_50) Nullable(String) : 45
           ALIAS multiIf(equals(bus_type,34_52),if(lessOrEquals(plus(1_58,4_59),0_62),NULL_64,arrayElementOrNull(splitByRegexpSpark(__54,order_id,-1_55),plus(1_58,4_59))),in(bus_type,__set_66),if(lessOrEquals(plus(1_72,6_74),0_77),NULL_78,arrayElementOrNull(splitByRegexpSpark(__68,order_id,-1_69),plus(1_72,6_74))),equals(bus_type,117_80),if(lessOrEquals(plus(1_86,7_87),0_90),NULL_92,arrayElementOrNull(splitByRegexpSpark(__82,order_id,-1_83),plus(1_86,7_87))),NULL_94) :: 23 -> multiIf(equals(bus_type,34_52),if(lessOrEquals(plus(1_58,4_59),0_62),NULL_64,arrayElementOrNull(splitByRegexpSpark(__54,order_id,-1_55),plus(1_58,4_59))),in(bus_type,__set_66),if(lessOrEquals(plus(1_72,6_74),0_77),NULL_78,arrayElementOrNull(splitByRegexpSpark(__68,order_id,-1_69),plus(1_72,6_74))),equals(bus_type,117_80),if(lessOrEquals(plus(1_86,7_87),0_90),NULL_92,arrayElementOrNull(splitByRegexpSpark(__82,order_id,-1_83),plus(1_86,7_87))),NULL_94) Nullable(String) : 35

With elimination in ExpressionParser, we get following actions dag. it becomes simpler

  Actions: INPUT : 0 -> order_id Nullable(String) : 0
           INPUT : 1 -> bus_type Nullable(Int32) : 1
           COLUMN Const(Int32) -> 34_9 Int32 : 2
           COLUMN Const(String) -> __11 String : 3
           COLUMN Const(Int32) -> -1_12 Int32 : 4
           COLUMN Const(String) -> Nullable(Array(Nullable(String))) String : 5
           COLUMN Const(Int32) -> plus(1_15,2_16) Int32 : 6
           COLUMN Const(UInt8) -> lessOrEquals(plus(1_15,2_16),0_18) UInt8 : 7
           COLUMN Const(Nullable(String)) -> NULL_20 Nullable(String) : 8
           COLUMN Set -> __set_22 Set : 9
           COLUMN Const(String) -> Nullable(Array(Nullable(String))) String : 10
           COLUMN Const(Int32) -> plus(1_15,4_31) Int32 : 11
           COLUMN Const(UInt8) -> lessOrEquals(plus(1_15,4_31),0_18) UInt8 : 12
           COLUMN Const(Int32) -> 117_37 Int32 : 13
           COLUMN Const(String) -> Nullable(Array(Nullable(String))) String : 14
           COLUMN Const(Int32) -> plus(1_15,5_44) Int32 : 15
           COLUMN Const(UInt8) -> lessOrEquals(plus(1_15,5_44),0_18) UInt8 : 16
           COLUMN Const(String) -> Nullable(Array(Nullable(String))) String : 17
           COLUMN Const(Int32) -> plus(1_15,6_73) Int32 : 18
           COLUMN Const(UInt8) -> lessOrEquals(plus(1_15,6_73),0_18) UInt8 : 19
           COLUMN Const(String) -> Nullable(Array(Nullable(String))) String : 20
           COLUMN Const(Int32) -> plus(1_15,7_87) Int32 : 21
           COLUMN Const(UInt8) -> lessOrEquals(plus(1_15,7_87),0_18) UInt8 : 22
           INPUT :: 2 -> hour Nullable(String) : 23
           INPUT :: 3 -> day Nullable(String) : 24
           INPUT :: 4 -> appid Int32 : 25
           FUNCTION equals(bus_type : 1, 34_9 :: 2) -> equals(bus_type,34_9) Nullable(Bool) : 26
           FUNCTION splitByChar(__11 :: 3, order_id :: 0, -1_12 :: 4) -> splitByRegexpSpark(__11,order_id,-1_12) Nullable(Array(String)) : 2
           FUNCTION in(bus_type : 1, __set_22 :: 9) -> in(bus_type,__set_22) Nullable(UInt8) : 4
           FUNCTION equals(bus_type :: 1, 117_37 :: 13) -> equals(bus_type,117_37) Nullable(Bool) : 9
           FUNCTION _CAST(splitByRegexpSpark(__11,order_id,-1_12) : 2, Nullable(Array(Nullable(String))) :: 5) -> splitByRegexpSpark(__11,order_id,-1_12) Nullable(Array(Nullable(String))) : 13
           FUNCTION _CAST(splitByRegexpSpark(__11,order_id,-1_12) : 2, Nullable(Array(Nullable(String))) :: 10) -> splitByRegexpSpark(__11,order_id,-1_12) Nullable(Array(Nullable(String))) : 5
           FUNCTION _CAST(splitByRegexpSpark(__11,order_id,-1_12) : 2, Nullable(Array(Nullable(String))) :: 14) -> splitByRegexpSpark(__11,order_id,-1_12) Nullable(Array(Nullable(String))) : 10
           FUNCTION _CAST(splitByRegexpSpark(__11,order_id,-1_12) : 2, Nullable(Array(Nullable(String))) :: 17) -> splitByRegexpSpark(__11,order_id,-1_12) Nullable(Array(Nullable(String))) : 14
           FUNCTION _CAST(splitByRegexpSpark(__11,order_id,-1_12) :: 2, Nullable(Array(Nullable(String))) :: 20) -> splitByRegexpSpark(__11,order_id,-1_12) Nullable(Array(Nullable(String))) : 17
           FUNCTION arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12) :: 13, plus(1_15,2_16) :: 6) -> arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,2_16)) Nullable(String) : 20
           FUNCTION arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12) :: 5, plus(1_15,4_31) :: 11) -> arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,4_31)) Nullable(String) : 6
           FUNCTION arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12) :: 10, plus(1_15,5_44) :: 15) -> arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,5_44)) Nullable(String) : 11
           FUNCTION arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12) :: 14, plus(1_15,6_73) :: 18) -> arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,6_73)) Nullable(String) : 15
           FUNCTION arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12) :: 17, plus(1_15,7_87) :: 21) -> arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,7_87)) Nullable(String) : 18
           FUNCTION if(lessOrEquals(plus(1_15,2_16),0_18) :: 7, NULL_20 : 8, arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,2_16)) :: 20) -> if(lessOrEquals(plus(1_15,2_16),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,2_16))) Nullable(String) : 21
           FUNCTION if(lessOrEquals(plus(1_15,4_31),0_18) :: 12, NULL_20 : 8, arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,4_31)) :: 6) -> if(lessOrEquals(plus(1_15,4_31),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,4_31))) Nullable(String) : 20
           FUNCTION if(lessOrEquals(plus(1_15,5_44),0_18) :: 16, NULL_20 : 8, arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,5_44)) :: 11) -> if(lessOrEquals(plus(1_15,5_44),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,5_44))) Nullable(String) : 6
           FUNCTION if(lessOrEquals(plus(1_15,6_73),0_18) :: 19, NULL_20 : 8, arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,6_73)) :: 15) -> if(lessOrEquals(plus(1_15,6_73),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,6_73))) Nullable(String) : 11
           FUNCTION if(lessOrEquals(plus(1_15,7_87),0_18) :: 22, NULL_20 : 8, arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,7_87)) :: 18) -> if(lessOrEquals(plus(1_15,7_87),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,7_87))) Nullable(String) : 15
           FUNCTION multiIf(equals(bus_type,34_9) : 26, if(lessOrEquals(plus(1_15,2_16),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,2_16))) :: 21, in(bus_type,__set_22) : 4, if(lessOrEquals(plus(1_15,4_31),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,4_31))) : 20, equals(bus_type,117_37) : 9, if(lessOrEquals(plus(1_15,5_44),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,5_44))) :: 6, NULL_20 : 8) -> multiIf(equals(bus_type,34_9),if(lessOrEquals(plus(1_15,2_16),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,2_16))),in(bus_type,__set_22),if(lessOrEquals(plus(1_15,4_31),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,4_31))),equals(bus_type,117_37),if(lessOrEquals(plus(1_15,5_44),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,5_44))),NULL_20) Nullable(String) : 18
           FUNCTION multiIf(equals(bus_type,34_9) :: 26, if(lessOrEquals(plus(1_15,4_31),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,4_31))) :: 20, in(bus_type,__set_22) :: 4, if(lessOrEquals(plus(1_15,6_73),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,6_73))) :: 11, equals(bus_type,117_37) :: 9, if(lessOrEquals(plus(1_15,7_87),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,7_87))) :: 15, NULL_20 :: 8) -> multiIf(equals(bus_type,34_9),if(lessOrEquals(plus(1_15,4_31),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,4_31))),in(bus_type,__set_22),if(lessOrEquals(plus(1_15,6_73),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,6_73))),equals(bus_type,117_37),if(lessOrEquals(plus(1_15,7_87),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,7_87))),NULL_20) Nullable(String) : 6
           ALIAS multiIf(equals(bus_type,34_9),if(lessOrEquals(plus(1_15,2_16),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,2_16))),in(bus_type,__set_22),if(lessOrEquals(plus(1_15,4_31),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,4_31))),equals(bus_type,117_37),if(lessOrEquals(plus(1_15,5_44),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,5_44))),NULL_20) :: 18 -> multiIf(equals(bus_type,34_9),if(lessOrEquals(plus(1_15,2_16),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,2_16))),in(bus_type,__set_22),if(lessOrEquals(plus(1_15,4_31),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,4_31))),equals(bus_type,117_37),if(lessOrEquals(plus(1_15,5_44),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,5_44))),NULL_20) Nullable(String) : 8
           ALIAS multiIf(equals(bus_type,34_9),if(lessOrEquals(plus(1_15,4_31),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,4_31))),in(bus_type,__set_22),if(lessOrEquals(plus(1_15,6_73),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,6_73))),equals(bus_type,117_37),if(lessOrEquals(plus(1_15,7_87),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,7_87))),NULL_20) :: 6 -> multiIf(equals(bus_type,34_9),if(lessOrEquals(plus(1_15,4_31),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,4_31))),in(bus_type,__set_22),if(lessOrEquals(plus(1_15,6_73),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,6_73))),equals(bus_type,117_37),if(lessOrEquals(plus(1_15,7_87),0_18),NULL_20,arrayElementOrNull(splitByRegexpSpark(__11,order_id,-1_12),plus(1_15,7_87))),NULL_20) Nullable(String) : 18

@lgbo-ustc lgbo-ustc linked a pull request Dec 19, 2024 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request performance
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[GLUTEN-8283][CH] Eliminate CSE during converting substrait expression to actions dag bigo-sg/gluten
1 participant
@lgbo-ustc