Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Apollo可能会丢消息 #4972

Closed
3 tasks
chunlaiqingke opened this issue Sep 14, 2023 · 8 comments
Closed
3 tasks

Apollo可能会丢消息 #4972

chunlaiqingke opened this issue Sep 14, 2023 · 8 comments
Labels

Comments

@chunlaiqingke
Copy link

  • I have checked the discussions
  • I have searched the issues of this repository and believe that this is not a duplicate.
  • I have checked the FAQ of this repository and believe that this is not a duplicate.

Describe the bug
当进行大事务的时候就会出现丢消息的情况,大事务比如删除数据区,就会出现被删除的数据区下面的namespace还能读到

To Reproduce
Steps to reproduce the behavior:

  1. 删除数据区,可以让这个事务等待一会,留足够的时候进行其他namespace的发布操作
  2. 当删除数据区事务没有完成的时候,发布了一个release,由于Id的增长不跟着事务提交,这样这个releaseMessage之前的消息都不会被扫描出来通知
  3. 因为Scanner里面有一个scanMissingMessages()方法里面有一个growAndCleanMissingMessages(),这里会记录missId的年龄,超过10,就会被remove掉,如果删除是事务时间超过10秒,就一定会把miss的Id干掉,导致丢失。
  4. 虽然说missReleaseMessageMaxAge这个常量后面有注释说,可以自己配置大小,但是这个不管配多大都会有问题。这里应该是防止内存泄露而写的一个补偿机制。

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional Details & Logs

  • Version
  • Error logs
  • Configuration
  • Platform and Operating System
@chunlaiqingke
Copy link
Author

消息没有通知的话,缓存就不会被删除

@nobodyiam
Copy link
Member

有实际的场景吗?比如在 apollo 中什么样的操作会产生大事务?

@chunlaiqingke
Copy link
Author

有实际的场景吗?比如在 apollo 中什么样的操作会产生大事务?

删除数据区,这个数据区下有2万个namespace,这个事务整个时间持续1分半钟。

@chunlaiqingke
Copy link
Author

有实际的场景吗?比如在 apollo 中什么样的操作会产生大事务?

这是我们生产环境遇到的问题

@chunlaiqingke
Copy link
Author

有实际的场景吗?比如在 apollo 中什么样的操作会产生大事务?

是扫描missid的时候,使用了年龄,超过10就会remove调,这边判断是死消息还是正常消息是根据时间10s来判断的

@hzdmm123
Copy link

有实际的场景吗?比如在 apollo 中什么样的操作会产生大事务?

删除数据区,这个数据区下有2万个namespace,这个事务整个时间持续1分半钟。

这应该是自己的业务问题...

@stale
Copy link

stale bot commented Oct 28, 2023

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in 7 days unless it is tagged "help wanted" or other activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Oct 28, 2023
Copy link

stale bot commented Nov 5, 2023

This issue has been automatically closed because it has not had activity in the last 7 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted". Thank you for your contributions.

@stale stale bot closed this as completed Nov 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants