Lighthouse plugin ignores skipAudits

[matejchalk]

What happened?

I integrated the Lighthouse plugin for a customer demo and there were a couple of errors related to interaction-to-next-paint, uses-responsive-images-snapshot and work-during-interaction. Although I'm not sure why the audits failed, for the demo purposes I just wanted to skip them while including all the other audits.

I noticed there was a skipAudits option in the lighthousePlugin function, so I tried to exclude the audits with:

await lighthousePlugin('<web-url>', {
  skipAudits: [
    'interaction-to-next-paint',
    'uses-responsive-images-snapshot',
    'work-during-interaction',
  ],
})

But the Lighthouse plugins still failed with the exact same errors as before.

What would you expect to happen?

I expected the Lighthouse plugin to exclude the mentioned audits and therefore to run without error.

What steps did you take?

Since I was building the plugin from source and saw that the filterAuditsAndGroupsByOnlyOptions function takes onlyCategories and onlyAudits into account, but not skipAudits. In order to avoid having to configure a long list of Lighthouse slugs for onlyAudits (I wanted to run all but three), I hacked in some simple changes to the source code to make it work:

diff --git a/packages/plugin-lighthouse/src/lib/utils.ts b/packages/plugin-lighthouse/src/lib/utils.ts
index ab22ebdf2..c34b203aa 100644
--- a/packages/plugin-lighthouse/src/lib/utils.ts
+++ b/packages/plugin-lighthouse/src/lib/utils.ts
@@ -137,7 +137,7 @@ export function validateOnlyCategories(
 export function filterAuditsAndGroupsByOnlyOptions(
   audits: Audit[],
   groups: Group[],
-  options?: Pick<CliFlags, 'onlyAudits' | 'onlyCategories'>,
+  options?: Pick<CliFlags, 'onlyAudits' | 'onlyCategories' | 'skipAudits'>,
 ): {
   audits: Audit[];
   groups: Group[];
@@ -171,8 +171,11 @@ export function filterAuditsAndGroupsByOnlyOptions(
   }
   // return unchanged
   return {
-    audits,
-    groups,
+    audits: audits.filter(audit => !options?.skipAudits?.includes(audit.slug)),
+    groups: filterItemRefsBy(
+      groups,
+      ({ slug }) => !options?.skipAudits?.includes(slug),
+    ),
   };
 }

Code PushUp package version

not released, built PR #549 branch from source

What operation system are you on?

Linux

Node version

20.11.0

Relevant log output

No response

[BioPhoton]

Related #612

[BioPhoton]

@matejchalk the problem should be fixed and the mentioned issue #635 covers a proper implementation / the current state.

Can we close this one?

[matejchalk]

Yes, tried adding skipAudits: ['largest-contentful-paint'] and it's no longer in the report, so seems to work as it should 👍