Update prometheus

[harry671003]

Changes

• This PR updates the prometheus to prometheus/prometheus@65f610353919

• This is the initial PR that addresses the following breaking changes:

  • promql: make lookback and matrix selections left-open and right-closed prometheus/prometheus#13904 - make lookback and matrix selections left-open and right-closed
  • promql(native histograms): Introduce exponential interpolation prometheus/prometheus#14677 - Exponential interpolation
  • promql: more Kahan summation (avg) and less incremental mean calculation (avg, avg_over_time) prometheus/prometheus#14413 - Kahan summation changes
  • promql: Fall back to full label sets when sorting by label prometheus/prometheus#14655 - Fall back to full label sets when sorting by label
  • promql: correctly handle unary negation of native histograms and add tests for multiplication and division of native histograms by negative scalars prometheus/prometheus#14821 - Correctly handle unary negation

• Tests are expected to fail because the following breaking (and possibly other) are not addressed yet.

  • PromQL engine: Delay deletion of __name__ label to the end of the query evaluation prometheus/prometheus#14477 - Delayed name dropping
  • Moves TestNativeHistogram_MulDivOperator to promql testing framework prometheus/prometheus#14688 - Moved the native histogram binary operation tests into acceptance tests.
  • promql: fix incorrect results and panics in sum and avg over mixed custom and exponential buckets, or incompatible custom buckets prometheus/prometheus#14611 and more - Added annotations to native histogram

Note

• The PR is made against a separate branch prometheus-update-65f610353919 to co-ordinate the work.

[GiedriusS]

I think it won't be so easy due to #486.

[fpetkovski]

I do not understand the new behavior, raised an issue here: prometheus/prometheus#14961.

[yeya24]

Quite a lot of tests failed due to round. Should we add my fix to round function to see how much it would help?

--- FAIL: TestInstantQuery/round/disableOptimizers=false/disableFallback=true#04 (0.00s)
                testutil.go:91: engine_test.go:4215: "Query: round(http_requests_total)
                    Explanation:
                    [duplicateLabelCheck]:
                    └──[function] round([http_requests_total]):
                       └──[coalesce]:
                          ├──[concurrent(buff=2)]:
                          │  └──[vectorSelector] {[__name__="http_requests_total"]} 0 mod 2
                          └──[concurrent(buff=2)]:
                             └──[vectorSelector] {[__name__="http_requests_total"]} 1 mod 2
                    
                    "
                    
                    	exp: &promql.Result{Err:error(nil), Value:promql.Vector{promql.Sample{T:0, F:1, H:(*histogram.FloatHistogram)(nil), Metric:labels.Labels{labels.Label{Name:"__name__", Value:"http_requests_total"}, labels.Label{Name:"pod", Value:"nginx-1"}, labels.Label{Name:"series", Value:"1"}}, DropName:true}, promql.Sample{T:0, F:2, H:(*histogram.FloatHistogram)(nil), Metric:labels.Labels{labels.Label{Name:"__name__", Value:"http_requests_total"}, labels.Label{Name:"pod", Value:"nginx-2"}, labels.Label{Name:"series", Value:"2"}}, DropName:true}, promql.Sample{T:0, F:5, H:(*histogram.FloatHistogram)(nil), Metric:labels.Labels{labels.Label{Name:"__name__", Value:"http_requests_total"}, labels.Label{Name:"pod", Value:"nginx-4"}, labels.Label{Name:"series", Value:"3"}}, DropName:true}, promql.Sample{T:0, F:8, H:(*histogram.FloatHistogram)(nil), Metric:labels.Labels{labels.Label{Name:"__name__", Value:"http_requests_total"}, labels.Label{Name:"pod", Value:"nginx-5"}, labels.Label{Name:"series", Value:"1"}}, D...(output trimmed)
                    
                    	got: &promql.Result{Err:error(nil), Value:promql.Vector{promql.Sample{T:0, F:1, H:(*histogram.FloatHistogram)(nil), Metric:labels.Labels{labels.Label{Name:"pod", Value:"nginx-1"}, labels.Label{Name:"series", Value:"1"}}, DropName:false}, promql.Sample{T:0, F:2, H:(*histogram.FloatHistogram)(nil), Metric:labels.Labels{labels.Label{Name:"pod", Value:"nginx-2"}, labels.Label{Name:"series", Value:"2"}}, DropName:false}, promql.Sample{T:0, F:5, H:(*histogram.FloatHistogram)(nil), Metric:labels.Labels{labels.Label{Name:"pod", Value:"nginx-4"}, labels.Label{Name:"series", Value:"3"}}, DropName:false}, promql.Sample{T:0, F:8, H:(*histogram.FloatHistogram)(nil), Metric:labels.Labels{labels.Label{Name:"pod", Value:"nginx-5"}, labels.Label{Name:"series", Value:"1"}}, DropName:false}, promql.Sample{T:0, F:2, H:(*histogram.FloatHistogram)(nil), Metric:labels.Labels{labels.Label{Name:"pod", Value:"nginx-6"}, labels.Label{Name:"series", Value:"2"}}, DropName:false}}, Warnings:annotations.Annotations{}}
                    
                    Diff:
                    --- Expected
                    +++ Actual
                    @@ -1,6 +1,6 @@
                    -(*promql.Result)({__name__="http_requests_total", pod="nginx-1", series="1"} => 1 @[0]
                    -{__name__="http_requests_total", pod="nginx-2", series="2"} => 2 @[0]
                    -{__name__="http_requests_total", pod="nginx-4", series="3"} => 5 @[0]
                    -{__name__="http_requests_total", pod="nginx-5", series="1"} => 8 @[0]
                    -{__name__="http_requests_total", pod="nginx-6", series="2"} => 2 @[0])
                    +(*promql.Result)({pod="nginx-1", series="1"} => 1 @[0]
                    +{pod="nginx-2", series="2"} => 2 @[0]
                    +{pod="nginx-4", series="3"} => 5 @[0]
                    +{pod="nginx-5", series="1"} => 8 @[0]
                    +{pod="nginx-6", series="2"} => 2 @[0])

[MichaHoffmann]

Quite a lot of tests failed due to round. Should we add my fix to round function to see how much it would help?

--- FAIL: TestInstantQuery/round/disableOptimizers=false/disableFallback=true#04 (0.00s)
                testutil.go:91: engine_test.go:4215: "Query: round(http_requests_total)
                    Explanation:
                    [duplicateLabelCheck]:
                    └──[function] round([http_requests_total]):
                       └──[coalesce]:
                          ├──[concurrent(buff=2)]:
                          │  └──[vectorSelector] {[__name__="http_requests_total"]} 0 mod 2
                          └──[concurrent(buff=2)]:
                             └──[vectorSelector] {[__name__="http_requests_total"]} 1 mod 2
                    
                    "
                    
                    	exp: &promql.Result{Err:error(nil), Value:promql.Vector{promql.Sample{T:0, F:1, H:(*histogram.FloatHistogram)(nil), Metric:labels.Labels{labels.Label{Name:"__name__", Value:"http_requests_total"}, labels.Label{Name:"pod", Value:"nginx-1"}, labels.Label{Name:"series", Value:"1"}}, DropName:true}, promql.Sample{T:0, F:2, H:(*histogram.FloatHistogram)(nil), Metric:labels.Labels{labels.Label{Name:"__name__", Value:"http_requests_total"}, labels.Label{Name:"pod", Value:"nginx-2"}, labels.Label{Name:"series", Value:"2"}}, DropName:true}, promql.Sample{T:0, F:5, H:(*histogram.FloatHistogram)(nil), Metric:labels.Labels{labels.Label{Name:"__name__", Value:"http_requests_total"}, labels.Label{Name:"pod", Value:"nginx-4"}, labels.Label{Name:"series", Value:"3"}}, DropName:true}, promql.Sample{T:0, F:8, H:(*histogram.FloatHistogram)(nil), Metric:labels.Labels{labels.Label{Name:"__name__", Value:"http_requests_total"}, labels.Label{Name:"pod", Value:"nginx-5"}, labels.Label{Name:"series", Value:"1"}}, D...(output trimmed)
                    
                    	got: &promql.Result{Err:error(nil), Value:promql.Vector{promql.Sample{T:0, F:1, H:(*histogram.FloatHistogram)(nil), Metric:labels.Labels{labels.Label{Name:"pod", Value:"nginx-1"}, labels.Label{Name:"series", Value:"1"}}, DropName:false}, promql.Sample{T:0, F:2, H:(*histogram.FloatHistogram)(nil), Metric:labels.Labels{labels.Label{Name:"pod", Value:"nginx-2"}, labels.Label{Name:"series", Value:"2"}}, DropName:false}, promql.Sample{T:0, F:5, H:(*histogram.FloatHistogram)(nil), Metric:labels.Labels{labels.Label{Name:"pod", Value:"nginx-4"}, labels.Label{Name:"series", Value:"3"}}, DropName:false}, promql.Sample{T:0, F:8, H:(*histogram.FloatHistogram)(nil), Metric:labels.Labels{labels.Label{Name:"pod", Value:"nginx-5"}, labels.Label{Name:"series", Value:"1"}}, DropName:false}, promql.Sample{T:0, F:2, H:(*histogram.FloatHistogram)(nil), Metric:labels.Labels{labels.Label{Name:"pod", Value:"nginx-6"}, labels.Label{Name:"series", Value:"2"}}, DropName:false}}, Warnings:annotations.Annotations{}}
                    
                    Diff:
                    --- Expected
                    +++ Actual
                    @@ -1,6 +1,6 @@
                    -(*promql.Result)({__name__="http_requests_total", pod="nginx-1", series="1"} => 1 @[0]
                    -{__name__="http_requests_total", pod="nginx-2", series="2"} => 2 @[0]
                    -{__name__="http_requests_total", pod="nginx-4", series="3"} => 5 @[0]
                    -{__name__="http_requests_total", pod="nginx-5", series="1"} => 8 @[0]
                    -{__name__="http_requests_total", pod="nginx-6", series="2"} => 2 @[0])
                    +(*promql.Result)({pod="nginx-1", series="1"} => 1 @[0]
                    +{pod="nginx-2", series="2"} => 2 @[0]
                    +{pod="nginx-4", series="3"} => 5 @[0]
                    +{pod="nginx-5", series="1"} => 8 @[0]
                    +{pod="nginx-6", series="2"} => 2 @[0])

It feels like the issue is on prometheus side, we should bump prometheus to include your fix i think.

[harry671003]

Quite a lot of tests failed due to round. Should we add my fix to round function to see how much it would help?

@yeya24, I tested your fix locally. The tests are passing.

It feels like the issue is on prometheus side, we should bump prometheus to include your fix i think.

Yes. This PR is intentionally only updating till prometheus/prometheus@65f610353919. The next commit in Prometheus is the switching to log/slog which is a huge change.

My plan is to work on the Prometheus upgrade incrementally on this branch so that each PR will be small and easier to review.

In retro, the branch could have been named prometheus-v3-update.

[yeya24]

@fpetkovski @MichaHoffmann Can you please take a look at the PR?
I reviewed most of the code changes other than code relevant to prometheus/prometheus#14413.

[yeya24]

Let's get this PR merged as it targets a non main branch. We can iterate on top of it

[fpetkovski]

Sorry missed this notification. Merging against a non-main branch makes perfect sense 👍