Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(network): add behave tests to ensure network errors are handled #3272

Merged
merged 1 commit into from
Sep 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 91 additions & 0 deletions features/network_failures.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
@uses.config.contract_token
Feature: Ensure network errors are handled gracefully across various services

Scenario Outline: Various HTTP errors are handled gracefully on attaching contract token
# This test simulates various HTTP errors by mocking the response from the serviceclient
# when trying to attach contract token
Given a `<release>` `<machine_type>` machine with ubuntu-advantage-tools installed
# 400 Bad Request
When I create a response overlay for `/v1/context/machines/token` with response code `400` and error message `Bad Request`
And I append the following on uaclient config:
"""
features:
serviceclient_url_responses: "/tmp/response-overlay.json"
"""
When I attempt to attach `contract_token` with sudo
Then stderr contains substring:
"""
Error connecting to /v1/context/machines/token: 400 {"error": "Bad Request"}
"""
Then the machine is unattached
# 404 Not Found
When I create a response overlay for `/v1/context/machines/token` with response code `404` and error message `Not Found`
And I append the following on uaclient config:
"""
features:
serviceclient_url_responses: "/tmp/response-overlay.json"
"""
When I attempt to attach `contract_token` with sudo
Then stderr contains substring:
"""
Error connecting to /v1/context/machines/token: 404 {"error": "Not Found"}
"""
Then the machine is unattached
# 503 Bad Gateway
When I create a response overlay for `/v1/context/machines/token` with response code `503` and error message `Bad Gateway`
And I append the following on uaclient config:
"""
features:
serviceclient_url_responses: "/tmp/response-overlay.json"
"""
When I attempt to attach `contract_token` with sudo
Then stderr contains substring:
"""
Error connecting to /v1/context/machines/token: 503 {"error": "Bad Gateway"}
"""
Then the machine is unattached

Examples: ubuntu release
| release | machine_type |
| xenial | lxd-container |
| noble | lxd-container |

Scenario Outline: Network errors for attaching contract token are handled gracefully
# This test simulates network failure by disabling internet connection
# and then trying to attach contract token
Given a `<release>` `<machine_type>` machine with ubuntu-advantage-tools installed
When I disable any internet connection on the machine
And I attempt to attach `contract_token` with sudo
Then stderr contains substring:
"""
Failed to attach machine. See https://ubuntu.com/pro/dashboard
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this actually needs to be fixed for us to consider the respective LP bug closed. The problem here is that you don't have internet connection:

  • there is no indication that lack of connection was the problem
  • how are people supposed to access their dashboard if the network is down?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@a-dubs if you want, we can land the tests and then you change this message on a next step?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh shoot yeah you're right. That's not exactly a very useful or specific message. So you're right we definitely need to update the message to consider those bugs closed. As such, I think it would be best if I put this back in draft state and make the changes to the source code such that the source code changes and tests are bundled together into one logical commit, so that when I say "fixes lp #XXXX", I mean fixes

"""
Then the machine is unattached

Examples: ubuntu release
| release | machine_type |
| xenial | lxd-container |
| noble | lxd-container |

Scenario Outline: Network errors for enabling Realtime kernel and Livepatch are handled gracefully
# This test simulates network failure by disabling internet connection
# and then trying to enable realtime-kernel or livepatch
Given a `<release>` `<machine_type>` machine with ubuntu-advantage-tools installed
When I attach `contract_token` with sudo and options `--no-auto-enable`
Then the machine is attached
Then I verify that `<service>` is disabled
When I disable any internet connection on the machine
And I verify that running `pro enable <service> --assume-yes` `with sudo` exits `1`
Then stderr contains substring:
"""
Failed to connect to https://contracts.canonical.com/v1/contracts/
"""
Then I verify that `<service>` is disabled

# Realtime kernel is not supported on LXD containers so we must use a VM
Examples: ubuntu release
| release | machine_type | service |
| xenial | lxd-vm | realtime-kernel |
| noble | lxd-vm | realtime-kernel |
| xenial | lxd-container | livepatch |
| noble | lxd-container | livepatch |
25 changes: 25 additions & 0 deletions features/steps/network.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
from behave import when

from features.steps.files import when_i_create_file_with_content
from features.steps.shell import when_i_run_command
from features.util import SUT

Expand Down Expand Up @@ -77,3 +78,27 @@ def disable_internet_connection(context, machine_name=SUT):
"with sudo",
machine_name=machine_name,
)


@when(
"I create a response overlay for `{endpoint}` with response code "
"`{response_code}` and error message `{error_message}`"
)
def create_response_overlay(context, endpoint, response_code, error_message):
text = f"""
{{
"https://contracts.canonical.com{endpoint}": [
{{
"code": "{response_code}",
"response": {{
"error": "{error_message}"
}}
}}
]
}}
"""
when_i_create_file_with_content(
context=context,
file_path="/tmp/response-overlay.json",
text=text,
)
Loading