Adding rt test plan (New)

[fernando79513]

Description

To test the real-time kernel, we have to include RT tests into the generic providers and create a SRU specific to RT devices.

The rt-performance tests are already in canonical/checkbox under the base provider and the TSN and TGPIO tests will be included in the future.

At the moment, the sru-rt will only include the rt-performance tests, but the TSN and TGPIO will be added in the future.

Since the rt-performance-tests-stress takes 24 hours to complete, the sru-rt only includes rt-performance-tests-automated with a shorter version of the cyclic test for latency. Once we have tested the sru-rt on some devices and we ensure it is stable, we should probably add this test back, but taking into consideration that test runs using this test are going to take more than one day.

Resolved issues

Documentation

There are no changes to the documentation

Tests

To list the RT jobs currently present on checkbox:

checkbox-cli expand com.canonical.certification::rt-performance-tests-full

To list the jobs currently in sru-rt:

checkbox-cli expand com.canonical.certification::sru-rt
...
Job 'com.canonical.certification::recovery_info_attachment'
Job 'com.canonical.certification::rt-performance-tests/cyclictest_rt'
Job 'com.canonical.certification::stress/cpu_stress_ng_test'
...

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 43.04%. Comparing base (ab47668) to head (311c3df).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1170   +/-   ##
=======================================
  Coverage   43.04%   43.04%           
=======================================
  Files         355      355           
  Lines       38610    38610           
  Branches     6556     6556           
=======================================
  Hits        16619    16619           
  Misses      21327    21327           
  Partials      664      664           

Flag	Coverage Δ
provider-base	16.09% <ø> (ø)
provider-sru	100.00% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[bladernr]

Is the plan here to require the 24h test as part of a certification baseline?

[kissiel]

Is the plan here to require the 24h test as part of a certification baseline?

In the RT testing that is being rolled out we intend to run this only for 60s.
Testing devices how they run the RT kernel is not [yet?] part of the certification AFAIK.

[kissiel]

Fantastic stuff, TYVM!

[fernando79513]

Is the plan here to require the 24h test as part of a certification baseline?

Most surely we are not going to require runnug the test for 24 hours. We still have to set a definite value for this specific test. The 24 hour limit was set for the Intel IoTG devices on the iiotg provider. For the Real-time SRU, we will set a smaller duration in the future.