TopSQL's retention time does not work

[lilyjazz]

Bug Report

Please answer these questions before submitting your issue. Thanks!

What did you do?

Open topsql feature from March 21.

What did you expect to see?

topsql's retention time is 1 month and it does work, it should delete data that exceed retention time.

What did you see instead?

data for march 21 is still available.

What version of TiDB Dashboard are you using (./tidb-dashboard --version)?

6.0

This issue is from pingcap/tidb-dashboard#1218

[zhongzc]

Victoria-metrics does not clean up stale data in a partition in which still some valid data, so those stale data are visible before all data in that partition become stale.

Yesterday the part /data2/production/prometheus-9090/data/tsdb/data/small/2022_03/300830982_1104130_20220320163212.000_20220324122734.000_16DC81B605210EBA was removed by VM, data are as expected.