Add E2E test for L7 NetworkPolicy Logging

[qiyueyao]

This PR adds an E2E test for L7 NetworkPolicy logging. It checks both allowed and dropped HTTP event logs under l7engine directory.

Resolving this issue from discussion.

[antoninbas]

let's do this after the decoding if necessary

[qiyueyao]

Removed this part, and added checking the decoded object against matchers.

[antoninbas]

we should probably retry if the entries are not there (yet). You used to handle this case (you had some code before the decoding). We should do it here instead.

I would personally recommend replacing wait.PollUntilContextTimeout here with https://pkg.go.dev/github.com/stretchr/testify/assert#EventuallyWithT. This way you can keep using assert.ElementsMatch.

assert.EventuallyWithT(t, func(c *assert.CollectT) {
        stdout, stderr, err := data.RunCommandFromPod(antreaNamespace, antreaPodName, "antrea-agent", cmd)
        assert.NoError(t, err, ...)
        // ...
        assert.ElementsMatch(t, gotLogs, matchers)
}

[qiyueyao]

Sounds good, updated to this.

[antoninbas]

I think I gave you a bad idea here. This is not great, because we want the polling to fail immediately if either RunCommandFromPod or dec.Decode fails, and there is currently no way to do that with assert.EventuallyWithT. So I think we need to go back to wait.PollUntilContextTimeout here, as it lets us do that :/

That also means that instead of assert.ElementsMatch, you may to use something like slices.Equal

[qiyueyao]

Actually I was quite convinced, if RunCommandFromPod fail, it means the file did not exist yet and we should wait? dec.Decode probably shouldn't fail unless, after waiting, unrecognized characters are overwritten by logs..

[antoninbas]

That makes sense. Ok, since we cannot use require.Error in this context (see stretchr/testify#1396), and assuming you want to keep using assert.EventuallyWithT, we should at least interrupt the iteration if there is a decoding error:

			if !assert.NoError(t, dec.Decode(&log)) {
				return
			}
			if slices.Contains(matchers, log) {
				gotLogs = append(gotLogs, log)
			}

(And same thing if RunCommandFromPod fails.)

If decoding errors are never supposed to happen, then I guess in practice it doesn't really matter that we have to wait for the full polling duration (10s) before failing the test.

But note that wait.PollUntilContextTimeout offers all the granularity we need:

stdout, stderr, err := data.RunCommandFromPod(antreaNamespace, antreaPodName, "antrea-agent", cmd)
if err != nil {
        return false, nil // not ready yet
}
// ...
if err := dec.Decode(&log); err != nil {
        return false, err // fail immediately
}
// ...
return !slices.Equal(), nil

Which is why I kind of regret asking you to change it :P

[qiyueyao]

haha It's good for me to learn this, and I've changed to wait.PollUntilContextTimeout which makes sense.

[antoninbas]

/test-e2e

[antoninbas]

/test-conformance
/test-networkpolicy