Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update definitions in base/audio jobs using an AI tool (BugFix) #1083

Merged
merged 1 commit into from
Mar 19, 2024
Merged

Conversation

kissiel
Copy link
Contributor

@kissiel kissiel commented Mar 19, 2024

This is a set of changes that my new AI tool did on the pxu files located in the providers/base/audio/ directory.

The system prompt I used was:

The task is to help improve test job definition records.
Each record is composed of fields. The name of the field starts in column 0,
and ends with a colon. Everything after the colon is the value. The value can span multiple lines. The continuation has to be indented,
otherwise the line would be treated as the next field.

First task is to reformat a given record by extracting
specific information from the '_description' field and placing it into new fields. 
Skip this task if there are not all three fields - "PURPOSE", "STEPS", or "VERIFICATION" - written in the "_description" field.
After extracting information for 'PURPOSE', 'STEPS', and 'VERIFICATION' headers, place the extracted text into new
fields named '_purpose', '_steps', and '_verification' respectively. The rest of the record should remain unchanged. Finally, rewrite the record with the new fields, maintaining the original syntax.
If the `_description` field looks like it only describes the purpose of the test, change it to `_purpose`.
Very important: preserve all the other fields!
Preserve the indentation of those three fields.
Do not add anything new to the record.

The second task is to find and fix typos and other language problems in fields that start with an underscore character `_`.

The third task is to add a `_summary` field, if it's missing. Use other fields to generate its contents.

Output the corrected record using the same syntax.
If nothing was changed, just output "===NO CHANGE===".

Example 2:

Input:

id: audio_test
_summary:
 Test the aydio subsystem
_description:
 PURPOSE:
  foo
 STEPS: 
  1. First step
  2. Second step
 VERIFICATION:
  bar

Output:

id: audio_test
_summary:
 Test the audio subsystem
_description:
_purpose
 foo
_steps: 
 1. First step
 2. Second step
_verification
 bar

Example 2:

Input:

id: bluetooth_test
_summary:
 Test bluetooth headphones
_purpose:
 Verify that we can connect bluetooth headphones to the host
_steps:
 1. Open bluetooth settings menu
 2. Turn on scan
 3. Follow the wizard
_verification:
 Can you hear the test sound in your headphones?

Output:

===NO CHANGE===

Example 3:

Input:

plugin: shell
category_id: com.canonical.plainbox::audio
id: audio/list_devices
estimated_duration: 1.0
requires:
 device.category == 'AUDIO'
 package.name == 'alsa-base'
command: cat /proc/asound/cards
_description: Test to detect audio devices

Output:

plugin: shell
category_id: com.canonical.plainbox::audio
id: audio/list_devices
estimated_duration: 1.0
requires:
 device.category == 'AUDIO'
 package.name == 'alsa-base'
command: cat /proc/asound/cards
_purpose: Test to detect audio devices
_summary: Check if audio devices can be detected.

The record to be improved starts at the next line:

Copy link

codecov bot commented Mar 19, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 41.31%. Comparing base (3991a48) to head (a73d64b).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1083   +/-   ##
=======================================
  Coverage   41.31%   41.31%           
=======================================
  Files         339      339           
  Lines       37648    37648           
  Branches     6400     6400           
=======================================
  Hits        15555    15555           
  Misses      21451    21451           
  Partials      642      642           
Flag Coverage Δ
provider-base 14.67% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Collaborator

@tang-mm tang-mm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice job! LGTM

Copy link
Collaborator

@pieqq pieqq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before your patch:

$ rg STEPS | wc -l
212

After your patch:

$ rg STEPS | wc -l
199

It looks promising, but your AI tool seems to be quite lazy as it seems it only changed 13 jobs out of 212 :D

@kissiel
Copy link
Contributor Author

kissiel commented Mar 19, 2024

Before your patch:

$ rg STEPS | wc -l
212

After your patch:

$ rg STEPS | wc -l
199

It looks promising, but your AI tool seems to be quite lazy as it seems it only changed 13 jobs out of 212 :D

rg STEPS | wc -l 
0

You may have missed the first line of my description :) I've run this only on the base/audio directory to get feedback.

Copy link
Collaborator

@pieqq pieqq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had missed the "audio category only" explanation. My bad!

All good then :)

@pieqq pieqq merged commit ade5369 into main Mar 19, 2024
16 checks passed
@pieqq pieqq deleted the ai-fixes branch March 19, 2024 08:21
binli pushed a commit to binli/checkbox that referenced this pull request Mar 22, 2024
…nical#1083)

- Update old descriptions into the purpose/steps/verification
- Fix grammar and typo
- Add summary for jobs without a summary
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants