Implement cluster size check and loggin addon test case on test_upgrade.py

[albinsun]

Which issue(s) this PR fixes & What it does:

1. [BUG] Upgrade is NOT blocked even there are Degraded volume harvester#6425
   • Skip test_upgrade.py::TestInvalidUpgrade::test_degraded_volume when cluster size < 3.
   • If not skip, 1 and 2 nodes cluster will just start upgrade in this negtive case Class.
2. [e2e] [BUG] Addon, rancher-logging only shows "processing" after it had been DeploySuccessful prior to Upgrade #901
   • Implement test_preq_setup_logging TODO
   • Post-upgrade check rancher-logging addon status in test_verify_logging_pods

Special notes for your reviewer:

n/a

Additional documentation or context

Verification

[lanfon72]

We should check whether it is a feature or not.
If it's a bug, the failure of test case will be expected, which means we would not need to skip it. (or skip it when the bug is not fixed in previous version.)

[albinsun]

We should check whether it is a feature or not. If it's a bug, the failure of test case will be expected, which means we would not need to skip it. (or skip it when the bug is not fixed in previous version.)

Yes, currently it's by design (feature).
Ref. harvester/harvester#6425 (comment)

[lanfon72]

We should check whether it is a feature or not. If it's a bug, the failure of test case will be expected, which means we would not need to skip it. (or skip it when the bug is not fixed in previous version.)

Yes, currently it's by design (feature). Ref. harvester/harvester#6425 (comment)

Let's add it as reference with comment in the code.
cluster_state is used to store the information before and after upgrade, so we didn't use it in this test case, we shouldn't too.

I think I would need to add some docstring to explain the ClusterState 🤔

[khushboo-rancher]

Typo: Degraded volumes only checked when nodes >= 3 :)

[albinsun]

Fixed in 702026f.

[albinsun]

Guys, as we on the way of test_upgrade.py enhancement, so I also implement a TODO and #901 together.
Please help to review again for the new committed about logging addon.

Sorry for inconvenience, no more new item will be add to this PR. :P

Verification (harvester-upgrade-test/281)

[lanfon72]

I need to explain how the TestUpgrade works, then you can know why the implementation need to change.

As the manual test, steps would like:

1. Set up environment before upgrade
2. Perform upgrade
3. Verify the environment is expected

What we do in upgrade automation is,

1. test case starts with test_pre_setup is going to be executed before upgrade
2. test case test_perform_upgrade will start the upgrade
3. test case starts with test_verify is going to verify the test case (after upgraded) hit criteria

cluster state changes during upgrade will be recorded in cluster_state. (that's why we will not use it in TestInvalidUpgrade)

Let's look into the case of setup_logging, we move logging to addon after v1.2.0, so we need to consider following cases:

1. A upgrade to v1.2.0 (implemented)
2. >v1.2.0 upgrade to B

When you implement case 2, you would still need to consider the case 1 in both pre_setup and verify.

[albinsun]

Hi @lanfon72 ,
Have tried to refactor according comments in 7516c7e, please help to review, thanks.

• Single node
  
• Three nodes
  

[albinsun]

Verification for 6a19ea8

• Logging addon enabled already
  
• Logging addon enabled yet
  

[lanfon72]

LGTM