Fix hetzner and openstack tests by adding AWS_EC2_METADATA_DISABLED=true in ec2 #37907

kaiyan-sheng · 2024-02-07T19:00:17Z

Proposed commit message

This PR is to add AWS_EC2_METADATA_DISABLED=true to disable IMDS when the real AWS SDK IMDS client is used. With this parameter, the non-EC2 provider tests should pass when running within an EC2 VM.

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

Relates Failed tests on Arm64 platform #37903

mergify · 2024-02-07T19:01:22Z

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @kaiyan-sheng? 🙏.
For such, you'll need to label your PR with:

The upcoming major version of the Elastic Stack
The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

elasticmachine · 2024-02-07T21:14:55Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Duration: 132 min 58 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

axw · 2024-02-08T03:05:46Z

Perhaps we should also disable the IMDS client by default in the tests, by adding an init function to the top of provider_aws_ec2_test.go, like:

func init() {
        // Disable IMDS when the real AWS SDK IMDS client is used,
        // so tests are isolated from the environment. Otherwise,
        // tests for non-EC2 providers may fail when the tests are
        // run within an EC2 VM.
        os.Setenv("AWS_EC2_METADATA_DISABLED", "true")
}

WDYT?

kaiyan-sheng · 2024-02-12T04:44:34Z

libbeat/processors/add_cloud_metadata/provider_hetzner_cloud_test.go

@@ -60,6 +69,7 @@ func TestRetrieveHetznerMetadata(t *testing.T) {
 	defer server.Close()

 	config, err := conf.NewConfigFrom(map[string]interface{}{
+		// "providers": []string{"hetzner"},


@axw Thanks for pointing out the config os.Setenv("AWS_EC2_METADATA_DISABLED", "true")! Without specifying the providers to be hetzner, the test passes too now,. Should I still give a providers just to be sure? 😂

I think it may be better to leave it out, to try and avoid issues like #13816 as much as possible.

libbeat/processors/add_cloud_metadata/provider_hetzner_cloud_test.go

axw · 2024-02-12T04:57:11Z

libbeat/processors/add_cloud_metadata/provider_hetzner_cloud_test.go

@@ -60,6 +69,7 @@ func TestRetrieveHetznerMetadata(t *testing.T) {
 	defer server.Close()

 	config, err := conf.NewConfigFrom(map[string]interface{}{
+		// "providers": []string{"hetzner"},


I think it may be better to leave it out, to try and avoid issues like #13816 as much as possible.

libbeat/processors/add_cloud_metadata/provider_aws_ec2.go

elasticmachine · 2024-02-14T00:44:16Z

💚 Build Succeeded

Buildkite Build
Commit: 2bd5754

History

💔 Build #1348 failed db264f8
💚 Build #1344 succeeded c000d00
💚 Build #1299 succeeded 025f2fc
💚 Build #1296 succeeded 7491e47
💚 Build #1284 succeeded bc51a26
💚 Build #1224 succeeded 7a57b70

cc @kaiyan-sheng

elasticmachine · 2024-02-14T00:54:21Z

💚 Build Succeeded

Buildkite Build
Commit: 2bd5754

History

💚 Build #1341 succeeded db264f8
💚 Build #1337 succeeded c000d00
💚 Build #1293 succeeded 025f2fc
💚 Build #1290 succeeded 7491e47
💚 Build #1278 succeeded bc51a26
💚 Build #1219 succeeded 7a57b70

cc @kaiyan-sheng

elasticmachine · 2024-02-14T01:04:35Z

💚 Build Succeeded

Buildkite Build
Commit: 2bd5754

History

💔 Build #512 failed db264f8
💚 Build #508 succeeded c000d00
💚 Build #464 succeeded 025f2fc
💚 Build #461 succeeded 7491e47
💚 Build #449 succeeded bc51a26

cc @kaiyan-sheng

elasticmachine · 2024-02-14T01:11:15Z

💚 Build Succeeded

Buildkite Build
Commit: 2bd5754

History

💚 Build #1667 succeeded db264f8
💚 Build #1663 succeeded c000d00
💚 Build #1624 succeeded 025f2fc
💚 Build #1621 succeeded 7491e47
💚 Build #1609 succeeded bc51a26
💚 Build #1549 succeeded 7a57b70

cc @kaiyan-sheng

elasticmachine · 2024-02-14T01:11:16Z

💚 Build Succeeded

Buildkite Build
Commit: 2bd5754

History

💚 Build #2561 succeeded db264f8
💔 Build #2557 failed c000d00
💔 Build #2554 failed 689b16c
💚 Build #2513 succeeded 025f2fc
💚 Build #2510 succeeded 7491e47
💚 Build #2498 succeeded bc51a26

cc @kaiyan-sheng

elasticmachine · 2024-02-14T01:24:24Z

💚 Build Succeeded

Buildkite Build
Commit: 2bd5754

History

💚 Build #3006 succeeded db264f8
💔 Build #3002 failed c000d00
💚 Build #2963 succeeded 025f2fc
💚 Build #2960 succeeded 7491e47
💚 Build #2948 succeeded bc51a26
💔 Build #2888 failed 7a57b70

cc @kaiyan-sheng

…rue in ec2 (#37907) (cherry picked from commit 38e7d6f) # Conflicts: # libbeat/processors/add_cloud_metadata/provider_aws_ec2_test.go # libbeat/processors/add_cloud_metadata/provider_hetzner_cloud_test.go

…rue in ec2 (#37907) (cherry picked from commit 38e7d6f)

…rue in ec2 (#37907) (#38016) (cherry picked from commit 38e7d6f) Co-authored-by: kaiyan-sheng <[email protected]>

Specify provider name in hetzner and openstack tests

5a8e9b4

kaiyan-sheng self-assigned this Feb 7, 2024

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Feb 7, 2024

kaiyan-sheng mentioned this pull request Feb 7, 2024

Failed tests on Arm64 platform #37903

Closed

fix golangci-lint

ecbc669

fix lint

db30e1f

kaiyan-sheng added 4 commits February 8, 2024 16:53

add correct provider name

3aebfac

Merge branch 'main' into add_cloud_metadata_tests

fc491a2

test

b074148

Merge branch 'main' into add_cloud_metadata_tests

7a57b70

kaiyan-sheng commented Feb 12, 2024

View reviewed changes

axw reviewed Feb 12, 2024

View reviewed changes

kaiyan-sheng added 3 commits February 12, 2024 10:19

set env variable

bc51a26

Merge branch 'main' into add_cloud_metadata_tests

7491e47

Merge branch 'main' into add_cloud_metadata_tests

025f2fc

axw approved these changes Feb 12, 2024

View reviewed changes

kaiyan-sheng marked this pull request as ready for review February 13, 2024 01:54

kaiyan-sheng requested a review from a team as a code owner February 13, 2024 01:54

kaiyan-sheng changed the title ~~Specify provider name in hetzner and openstack tests~~ Fix hetzner and openstack tests by adding AWS_EC2_METADATA_DISABLED=true in ec2 Feb 13, 2024

axw requested changes Feb 13, 2024

View reviewed changes

libbeat/processors/add_cloud_metadata/provider_aws_ec2.go Outdated Show resolved Hide resolved

kaiyan-sheng added 4 commits February 13, 2024 10:19

Move init func into provider_aws_ec2_test.go

689b16c

Merge branch 'main' into add_cloud_metadata_tests

c000d00

Merge branch 'main' into add_cloud_metadata_tests

db264f8

Merge branch 'main' into add_cloud_metadata_tests

2bd5754

axw approved these changes Feb 14, 2024

View reviewed changes

zmoog added the Team:Cloud-Monitoring Label for the Cloud Monitoring team label Feb 14, 2024

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Feb 14, 2024

zmoog added the bug label Feb 14, 2024

zmoog approved these changes Feb 14, 2024

View reviewed changes

kaiyan-sheng added backport-7.17 Automated backport to the 7.17 branch with mergify backport-v8.12.0 Automated backport with mergify backport-v8.13.0 Automated backport with mergify labels Feb 14, 2024

kaiyan-sheng merged commit 38e7d6f into elastic:main Feb 14, 2024
107 checks passed

kaiyan-sheng deleted the add_cloud_metadata_tests branch February 14, 2024 16:06

mergify bot pushed a commit that referenced this pull request Feb 14, 2024

Fix hetzner and openstack tests by adding AWS_EC2_METADATA_DISABLED=t…

72bbb00

…rue in ec2 (#37907) (cherry picked from commit 38e7d6f)

This was referenced Feb 14, 2024

[7.17](backport #37907) Fix hetzner and openstack tests by adding AWS_EC2_METADATA_DISABLED=true in ec2 #38015

Closed

[8.12](backport #37907) Fix hetzner and openstack tests by adding AWS_EC2_METADATA_DISABLED=true in ec2 #38016

Merged

kaiyan-sheng added a commit that referenced this pull request Feb 14, 2024

Fix hetzner and openstack tests by adding AWS_EC2_METADATA_DISABLED=t…

71e15c6

…rue in ec2 (#37907) (#38016) (cherry picked from commit 38e7d6f) Co-authored-by: kaiyan-sheng <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix hetzner and openstack tests by adding AWS_EC2_METADATA_DISABLED=true in ec2 #37907

Fix hetzner and openstack tests by adding AWS_EC2_METADATA_DISABLED=true in ec2 #37907

kaiyan-sheng commented Feb 7, 2024 •

edited by zmoog

Loading

mergify bot commented Feb 7, 2024

elasticmachine commented Feb 7, 2024 •

edited

Loading

Build stats

axw commented Feb 8, 2024

kaiyan-sheng Feb 12, 2024

axw Feb 12, 2024

axw Feb 12, 2024

elasticmachine commented Feb 14, 2024

elasticmachine commented Feb 14, 2024

elasticmachine commented Feb 14, 2024

elasticmachine commented Feb 14, 2024

elasticmachine commented Feb 14, 2024

elasticmachine commented Feb 14, 2024

Fix hetzner and openstack tests by adding AWS_EC2_METADATA_DISABLED=true in ec2 #37907

Fix hetzner and openstack tests by adding AWS_EC2_METADATA_DISABLED=true in ec2 #37907

Conversation

kaiyan-sheng commented Feb 7, 2024 • edited by zmoog Loading

Proposed commit message

Checklist

Author's Checklist

mergify bot commented Feb 7, 2024

elasticmachine commented Feb 7, 2024 • edited Loading

💚 Build Succeeded

Build stats

❕ Flaky test report

🤖 GitHub comments

axw commented Feb 8, 2024

kaiyan-sheng Feb 12, 2024

Choose a reason for hiding this comment

axw Feb 12, 2024

Choose a reason for hiding this comment

axw Feb 12, 2024

Choose a reason for hiding this comment

elasticmachine commented Feb 14, 2024

💚 Build Succeeded

History

elasticmachine commented Feb 14, 2024

💚 Build Succeeded

History

elasticmachine commented Feb 14, 2024

💚 Build Succeeded

History

elasticmachine commented Feb 14, 2024

💚 Build Succeeded

History

elasticmachine commented Feb 14, 2024

💚 Build Succeeded

History

elasticmachine commented Feb 14, 2024

💚 Build Succeeded

History

kaiyan-sheng commented Feb 7, 2024 •

edited by zmoog

Loading

elasticmachine commented Feb 7, 2024 •

edited

Loading