Skip to content

Commit

Permalink
Updated the results with latest 0.0.24 image
Browse files Browse the repository at this point in the history 
Signed-off-by: Chandrakala Subramanyam <[email protected]>
  • Loading branch information
@chandrams
chandrams committed Sep 5, 2024
1 parent 966e2f8 commit a8a5408
Showing 1 changed file with 19 additions and 17 deletions.
36 changes: 19 additions & 17 deletions tests/test_plans/test_plan_rel_0.0.24.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ This document describes the test plan for Kruize remote monitoring release 0.0.2

* Metric profile creation, listing and deletion through APIs
* Support for ‘kind’ in Kruize local monitoring demo
* Support for kubernetes job type

------

Expand All @@ -45,11 +46,12 @@ This document describes the test plan for Kruize remote monitoring release 0.0.2

### New Test Cases Developed

| # | ISSUE (NEW FEATURE) | TEST DESCRIPTION | TEST DELIVERABLES | RESULTS | COMMENTS |
|---|------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------|---------| --- |
| 1 | Tests for create, list operations in Metric profile API | [Included new tests](https://github.com/kruize/autotune/blob/master/tests/scripts/local_monitoring_tests/Local_monitoring_tests.md) | Tests added - [1231](https://github.com/kruize/autotune/pull/1231) | PASSED | |
| 2 | [Support for ‘kind’ in kruize local demo](https://github.com/kruize/kruize-demos/pull/81) | local monitoring demo has been updated to use kind | | PASSED | |
| 3 | [Kruize notifications fix for production issue](https://github.com/kruize/autotune/pull/1276) | Tested manually | | PASSED | |
| # | ISSUE (NEW FEATURE) | TEST DESCRIPTION | TEST DELIVERABLES | RESULTS | COMMENTS |
|---|-----------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------|---------| --- |
| 1 | Tests for create, list operations in Metric profile API | [Included new tests](https://github.com/kruize/autotune/blob/master/tests/scripts/local_monitoring_tests/Local_monitoring_tests.md) | Tests added - [1231](https://github.com/kruize/autotune/pull/1231) | PASSED | |
| 2 | [Support for ‘kind’ in kruize local demo](https://github.com/kruize/kruize-demos/pull/81) | local monitoring demo has been updated to use kind | | PASSED | |
| 3 | [Kruize notifications fix for production issue](https://github.com/kruize/autotune/pull/1276) | Tested manually | | PASSED | |
| 4 | [Support for job type workload](https://github.com/kruize/autotune/pull/1223) | [New Tests added /updated existing tests](https://github.com/kruize/autotune/pull/1223) | | PASSED | |



Expand All @@ -75,7 +77,7 @@ Kruize Release | Exps / Results / Recos | Execution time | Latency (Max/ Avg) in
-- |------------------------|----------------|-------------------------------|---------------|----------------------|----------------------|----------------| --
| | | | UpdateRecommendations | UpdateResults | LoadResultsByExpName | | |
0.0.23_mvp | 5K / 72L / 3L | 3h 30 mins | 0.66 / 0.41 | 0.14 / 0.12 | 0.22 / 0.18 | 21767 | 6.96 | 36.35
0.0.24_mvp | 5K / 72L / 3L | 4h 03 mins | 0.8 / 0.47 | 0.12 / 0.11 | 0.38 / 0.28 | 21756 | 6.61 | 43.21
0.0.24_mvp | 5K / 72L / 3L | 4h 04 mins | 0.8 / 0.47 | 0.13 / 0.12 | 0.53 / 0.36 | 21752 | 4.63 | 34.72

----
## RELEASE TESTING
Expand All @@ -92,17 +94,17 @@ As part of the release testing, following tests will be executed:
- [Kruize local monitoring Functional tests](/tests/scripts/local_monitoring_tests/Local_monitoring_tests.md)


| # | TEST SUITE | EXPECTED RESULTS | ACTUAL RESULTS | COMMENTS |
| --- | ---------- |-----------------------------------------|-----------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------|
| 1 | Kruize Remote monitoring Functional testsuite | TOTAL - 357, PASSED - 314 / FAILED - 43 | TOTAL - 357, PASSED - 314 / FAILED - 43 | No new regressions seen, existing issues - [559](https://github.com/kruize/autotune/issues/559), [610](https://github.com/kruize/autotune/issues/610) |
| 2 | Fault tolerant test | PASSED | PASSED | |
| 3 | Stress test | PASSED | PASSED | |
| 4 | Scalability test (short run)| | | Exps - 5000, Results - 72000, execution time - 3 hours 51 mins |
| 5 | DB Migration test | PASSED | PASSED | Tested on Openshift |
| 6 | Recommendation and box plot values validations | PASSED | PASSED | |
| 7 | Kruize remote monitoring demo | PASSED | PASSED | Tested manually |
| 8 | Kruize Local monitoring demo | PASSED | PASSED | |
| 9 | Kruize Local Functional tests | TOTAL - 64, PASSED - 60 / FAILED - 4 | TOTAL - 64, PASSED - 60 / FAILED - 4 | [Issue 1217](https://github.com/kruize/autotune/issues/1217), [Issue 1273](https://github.com/kruize/autotune/issues/1273) |
| # | TEST SUITE | EXPECTED RESULTS | ACTUAL RESULTS | COMMENTS |
| --- | ---------- |-----------------------------------------|-----------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 1 | Kruize Remote monitoring Functional testsuite | TOTAL - 359, PASSED - 308 / FAILED - 51 | TOTAL - 359, PASSED - 308 / FAILED - 51 | Intermittent issue seen [1281](https://github.com/kruize/autotune/issues/1281), existing issues - [559](https://github.com/kruize/autotune/issues/559), [610](https://github.com/kruize/autotune/issues/610) |
| 2 | Fault tolerant test | PASSED | PASSED | |
| 3 | Stress test | PASSED | PASSED | |
| 4 | Scalability test (short run)| | | Exps - 5000, Results - 72000, execution time - 3 hours 51 mins |
| 5 | DB Migration test | PASSED | PASSED | Tested on Openshift |
| 6 | Recommendation and box plot values validations | PASSED | PASSED | |
| 7 | Kruize remote monitoring demo | PASSED | PASSED | Tested manually |
| 8 | Kruize Local monitoring demo | PASSED | PASSED | |
| 9 | Kruize Local Functional tests | TOTAL - 64, PASSED - 60 / FAILED - 4 | TOTAL - 64, PASSED - 60 / FAILED - 4 | [Issue 1217](https://github.com/kruize/autotune/issues/1217), [Issue 1273](https://github.com/kruize/autotune/issues/1273) |

---

Expand Down

0 comments on commit a8a5408

Please sign in to comment.