Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add some more ExpectNoError checks to the e2e tests #4623

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

danwinship
Copy link
Contributor

What this PR does and why is it needed

More fixes pulled out of @martinkennelly 's branch to start adding support for running ovn-k e2e downstream in OCP. This just takes a bunch of places where the test cases were doing something and ignoring possible errors, and changes them to assert that an error didn't occur instead.

(In theory, this should not result in us discovering any new bugs in the existing code, since if something actually went wrong, the test would presumably fail later on anyway. It just means that in case of future bugs, the tests should fail closer to the point where something actually went wrong, rather than failing more mysteriously later.)

Does this PR introduce a user-facing change?

NONE

@danwinship danwinship requested a review from a team as a code owner August 14, 2024 16:52
@github-actions github-actions bot added feature/egress-ip Issues related to EgressIP feature area/e2e-testing feature/egress-gateway All issues related to ICNI/APBR labels Aug 14, 2024
martinkennelly
martinkennelly previously approved these changes Aug 19, 2024
tssurya
tssurya previously approved these changes Aug 19, 2024
Copy link
Member

@tssurya tssurya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ping me when CI runs, I can merge this LGTM

@tssurya
Copy link
Member

tssurya commented Aug 19, 2024

control-plane's are failing..

@@ -789,7 +789,8 @@ var _ = ginkgo.Describe("e2e control plane", func() {
pod.Name != "etcd-ovn-control-plane" &&
!strings.HasPrefix(pod.Name, "ovs-node") {
framework.Logf("%q", pod.Namespace)
e2epod.DeletePodWithWaitByName(context.TODO(), f.ClientSet, pod.Name, ovnNs)
err = e2epod.DeletePodWithWaitByName(context.TODO(), f.ClientSet, pod.Name, ovnNs)
framework.ExpectNoError(err, fmt.Sprintf("failed to delete pod %s", pod.Name))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LOL, this is causing e2e failures because of a bug in e2epod.DeletePodWithWaitByName; it deletes the pod, and then loops until no pod with the given name exists. But if something else recreates the pod immediately, it doesn't notice that the pod that exists isn't the same one as the one it deleted.

(So, without the ExpectNoError check, the failure mode would be "it deletes the pod successfully, then gets confused and loops for 5 minutes and then the function returns an error, but we ignore the error and keep going (which is fine, because the pod we meant to delete was deleted)", whereas with this PR, the failure mode is "it deletes the pod successfully, then gets confused and loops for 5 minutes and then the function returns an error, which causes us to fail the test".)

@danwinship danwinship marked this pull request as draft August 19, 2024 16:28
@danwinship
Copy link
Contributor Author

Filed kubernetes/kubernetes#126785. Let's see what happens there.

This bug may be affecting other places where we kill control-plane pods...

@danwinship
Copy link
Contributor Author

Filed kubernetes/kubernetes#126785. Let's see what happens there.

No consensus on a short-term fix, so I'll just fix this locally.

@danwinship danwinship marked this pull request as ready for review September 5, 2024 13:00
@danwinship danwinship force-pushed the e2e-no-error branch 2 times, most recently from 9f75c2d to 514f3a6 Compare September 9, 2024 13:55
danwinship and others added 2 commits September 17, 2024 10:24
…t restarted.

Signed-off-by: Dan Winship <danwinship@redhat.com>
Signed-off-by: Dan Winship <danwinship@redhat.com>
Co-authored-by: Martin Kennelly <mkennell@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/e2e-testing feature/egress-gateway All issues related to ICNI/APBR feature/egress-ip Issues related to EgressIP feature
Projects
Status: Todo
Development

Successfully merging this pull request may close these issues.

3 participants