Test performance benchmark

[mukeshpanchal27]

No description provided.

[mukeshpanchal27]

Hi @swissspidy Sorry for ping. Do you know why the Performance workflow failed. What is the best way to debug the npm run --silent test:performance $ADDITIONAL_ARGS?

[swissspidy]

Hmm you might be running into swissspidy/wp-performance-action#224, which I thought I had been fixing in swissspidy/wp-performance-action#227. Let me merge that one and then you can try with swissspidy/wp-performance-action@main

[mukeshpanchal27]

Yes please

[swissspidy]

OK I just merged it. Let me know if the @main version still fails, then I can do some debugging.

[mukeshpanchal27]

Same error:

Performance Test Results
Status: failed

Tests

┌─────────┬─────────────┬───────────────┬──────────────────┬────────────┬─────────┬────────────────────────┬─────────────────┬──────────────┐
│ (index) │ wpDbQueries │ wpMemoryUsage │ wpBeforeTemplate │ wpTemplate │ wpTotal │ largestContentfulPaint │ timeToFirstByte │ lcpMinusTtfb │
├─────────┼─────────────┼───────────────┼──────────────────┼────────────┼─────────┼────────────────────────┼─────────────────┼──────────────┤
│ 0       │ 101         │ 55559440      │ 655              │ 148        │ 803     │ 8852.05                │ 7903.15         │ 948.9        │
│ 1       │             │               │                  │            │         │                        │                 │              │
└─────────┴─────────────┴───────────────┴──────────────────┴────────────┴─────────┴────────────────────────┴─────────────────┴──────────────┘
Error: Process completed with exit code 1.

[swissspidy]

Where'd you run it? This PR still uses @v2

[swissspidy]

Also, if you re-run a failed workflow with debug enabled, the performance action should show some more info

[mukeshpanchal27]

Working now. 🚀

[swissspidy]

Great, I'll tag a new release so that @v2 will work again.

[mukeshpanchal27]

Performance Test Results

Performance test results for 227d990 are in 🛎️!

Tests

Run	wpDbQueries	wpMemoryUsage	wpBeforeTemplate	wpTemplate	wpTotal	largestContentfulPaint	timeToFirstByte	lcpMinusTtfb
0	86	58.55 MB	478.00 ms	343.00 ms	613.00 ms	1437.40 ms	1202.15 ms	233.60 ms

@swissspidy As i set iterations: 5 but the test is run for one time only Does it average value?

[swissspidy]

That‘s not what the iterations config is for

 # Number of times the tests should be repeated.
    #
    # Default: 2
    repetitions: ''

    # Number of iterations (loops) within a single run.
    #
    # Default: 20
    iterations: ''

so repetitions = how often the whole test suite rund = more table rows
iterations = how often each individual test runs

[mukeshpanchal27]

Performance Test Results

Performance test results for 4c708e8 are in 🛎️!

Tests

Run	wpDbQueries	wpMemoryUsage	wpBeforeTemplate	wpTemplate	wpTotal	largestContentfulPaint	timeToFirstByte	lcpMinusTtfb
0	86	58.55 MB	478.00 ms	345.00 ms	669.00 ms	1581.25 ms	1237.00 ms	275.00 ms

Per the job Artifacts JSON data wpBeforeTemplate and wpTemplate get 5 iteration value and other are 10 times 🤔

[
  {
    "file": "/home/runner/work/_actions/swissspidy/wp-performance-action/v2/env/tests/performance/specs/main.spec.ts",
    "title": "Tests",
    "results": [
      {
        "wpDbQueries": [
          101,
          130,
          95,
          95,
          95,
          77,
          76,
          76,
          76,
          76
        ],
        "wpMemoryUsage": [
          55634136,
          57976944,
          57964040,
          57960440,
          57960440,
          59122336,
          59148856,
          59149000,
          59149000,
          59149000
        ],
        "wpBeforeTemplate": [
          660,
          625,
          478,
          471,
          475
        ],
        "wpTemplate": [
          151,
          398,
          348,
          345,
          339
        ],
        "wpTotal": [
          811,
          1023,
          827,
          816,
          814,
          517,
          520,
          516,
          527,
          518
        ],
        "largestContentfulPaint": [
          2215,
          17596,
          1578.9000000000233,
          1583,
          1579.5,
          9920,
          886.7999999999884,
          1897.5,
          1355.5999999999767,
          1333.5
        ],
        "timeToFirstByte": [
          2003,
          16470.29999999999,
          1232.5,
          1218.9000000000233,
          1241.5,
          2443.899999999965,
          731.1999999999534,
          1739.399999999965,
          1201.899999999965,
          1180.7000000000116
        ],
        "lcpMinusTtfb": [
          212,
          1125.7000000000116,
          346.4000000000233,
          364.0999999999767,
          338,
          7476.100000000035,
          155.60000000003492,
          158.10000000003492,
          153.70000000001164,
          152.79999999998836
        ]
      }
    ]
  }
]

[swissspidy]

Oooh.... I think I know what's going on. The JSON file structure just merges together all values from all URLs. I somehow didn't think of that when I added the multi-URL support 🤦

The reason you see different number of entries is because the first 5 are from /, the next 5 from /wp-admin. In /wp-admin you don't have the full Server-Timing information though, hence wpTemplate for example only has 5.

Anyhow, it's utterly wrong to put all the results for a URL into the same array 🤦

I'll need to fix that.

[swissspidy]

I just re-tagged v2 with the fix now

[mukeshpanchal27]

🎉

Performance Test Results

Performance test results for 4c708e8 are in 🛎️!

URL: /

Run	wpDbQueries	wpMemoryUsage	wpBeforeTemplate	wpTemplate	wpTotal	largestContentfulPaint	timeToFirstByte	lcpMinusTtfb
0	95	57.96 MB	498.00 ms	353.00 ms	851.00 ms	1644.20 ms	1310.30 ms	333.90 ms

URL: /wp-admin/

Run	wpDbQueries	wpMemoryUsage	wpTotal	largestContentfulPaint	timeToFirstByte	lcpMinusTtfb
0	76	59.14 MB	528.00 ms	1356.90 ms	1205.70 ms	157.90 ms

[swissspidy]

I should probably make those labels more readable 🤔 The table is quite wide this way 😅

Edit: swissspidy/wp-performance-action#230

[mukeshpanchal27]

@swissspidy Sorry for ping.

There tests shows lots of delay 10 minutes and show Test timeout of 600000ms exceeded. Anything we can increase timeout https://playwright.dev/docs/test-timeouts?

[swissspidy]

The issue is not a timeout ( 10 minutes is extremely high! ), but the Error: page.goto: Target page, context or browser has been closed and ECONNREFUSED 127.0.0.1:9400 errors you see there in the log

In other words: the Playground server probably crashed and then Playwright tried for 10 minutes to still connect to it.

[mukeshpanchal27]

In other words: the Playground server probably crashed and then Playwright tried for 10 minutes to still connect to it.

Does it Performance action issue? or Playground? or Playwright?

[swissspidy]

The Playground server crashing due to heavy load would be a Playground issue.