fix Forever growing registry file with kubernetes autodiscovery when setting clean_removed

[bigpigeon]

• Bug

What does this PR do?

fix below issue

#13140

Why is it important?

filebeat use autodiscover with kubernetes will leaking the registry entity

it has not been resolved in 7.17

[cla-checker-service]

❌ Author of the following commits did not sign a Contributor Agreement:
388c00c, 64402af

Please, read and sign the above mentioned agreement if you want to contribute to this project

[botelastic]

This pull request doesn't have a Team:<team> label.

[elasticmachine]

❕ Build Aborted

The PR is not allowed to run in the CI yet

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2023-03-23T09:56:51.465+0000

• Duration: 8 min 28 sec

Steps errors

Expand to view the steps failures

Load a resource file from a library

• Took 0 min 0 sec . View more details here
• Description: approval-list/elastic/beats.yml

Error signal

• Took 0 min 0 sec . View more details here
• Description: githubApiCall: The REST API call https://api.github.com/orgs/elastic/members/bigpigeon return the message : java.lang.Exception: httpRequest: Failure connecting to the service https://api.github.com/orgs/elastic/members/bigpigeon : httpRequest: Failure connecting to the service https://api.github.com/orgs/elastic/members/bigpigeon : Code: 404Error: {"message":"User does not exist or is not a member of the organization","documentation_url":"https://docs.github.com/rest/reference/orgs#check-organization-membership-for-a-user"}

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[rdner]

Thank you for your contribution. A few things before we can accept the PR:

1. We need a test that confirms the bug without the change and then passes with your change
2. Why does this PR target 7.17 instead of the main branch?

[bigpigeon]

thanks
2. because My company use filebeat 7.17 version to collect log, and I have so many changes，just like named ratelimit, output traffic monitoring, conflict log extra field, and some custom output module, making it difficult to update to major 8 version

[bigpigeon]

Thank you for your contribution. A few things before we can accept the PR:

1. We need a test that confirms the bug without the change and then passes with your change

1. I'll write a unittest later, It might take some time