Performance Benchmarks for WordPress 6.7

[mukeshpanchal27]

As we enter the beta phase for WordPress 6.7, I am opening this issue to gather and track all performance benchmarks. This will help us collaborate more effectively, identify potential regressions, and ensure all performance-related data is centralized in one place.

Feel free to contribute with your benchmarks, results, or any findings related to performance in WordPress 6.7.

[mukeshpanchal27]

WP 6.7 Beta 1 Performance Benchmark Report:

I ran benchmarks comparing WordPress 6.7 Beta 1 and the latest released 6.6 branch (6.6.2). I took measurements with the twentytwentyone (TT1) theme using en_US locale, both with object caching disabled and enabled, then again with twentytwentyfour (TT4), This version of WordPress shows a regression in server rendering time from the previous release for TT4.

With object caching disabled, Classic themes (TT1) show a ~3% regression (0.35ms slower) in wp-before-template measurements; LCP remains relatively stable. Block themes (TT4) show a ~6% regression in TTFB (6ms slower) and a ~8% regression (8 ms slower) for wp-total measurements. LCP shows a ~4.5% regression (7ms). Running the tests with object caching enabled shows similar regressions.

Here are the results of the test runs:

Caching Disabled:
Classic theme - https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11122906286
Block theme - https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11123131063

Caching Enabled:
Classic theme - https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11123295178
Block theme - https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11123530293

cc. @felixarntz @joemcgill @adamsilverstein @westonruter @swissspidy

[joemcgill]

Thanks @mukeshpanchal27. Repeating what I said in our weekly performance team chat earlier today:

the performance metrics for classic themes looks pretty steady, which is great! For TT4, it seems like we need to investigate the cause of the additional performance regression during template rendering, because adding 10% is not great.

I started trying to investigate the likely cause by looking at the WordPress Code Vitals dashboard and realized that the last commit recorded to that dashboard seems to be this one from five months ago 😱. It's likely that the following commit, r58165, likely broke these.

Fortunately, the Gutenberg tests are still being recorded, and it looks like this commit is worth investigating further. It was synced to WordPress in r58936. Here's the performance run from that commit in WP-dev.

[joemcgill]

I've opened https://core.trac.wordpress.org/ticket/62153 to fix the dashboard issue.

[mukeshpanchal27]

I did the quick compare through xhprof. Here is some finding.

• Use of strpos is significantly more then the previous version.

The compression between 6.6.2 and 6.7 beta1

6.7 beta1 strpos function details:

6.6.2 strpos function details:

[mukeshpanchal27]

Fortunately, the Gutenberg tests are still being recorded, and it looks like this commit is worth investigating further. It was synced to WordPress in r58936. Here's the performance run from that commit in WP-dev.

@joemcgill The code https://github.com/WordPress/wordpress-develop/blob/trunk/src/wp-includes/class-wp-theme-json.php#L3239-L3251 is too expensive see the profiling result. I removed the code and did the comparison.

Server-timing

Mertic	Beta 1	Beta 1 with remove code	Diff %	Diff abs.
Response Time	252.84	247.12	-2.26%	-5.72
wp-before-template	76.26	76.28	0.03%	0.02
wp-before-template-db-queries	3.3	3.35	1.52%	0.05
wp-template	173.42	167.66	-3.32%	-5.76
wp-total	249.87	244.23	-2.26%	-5.64
wp-template-db-queries	2.6	2.59	-0.38%	-0.01

🔴 It clear shows that the code introduce ~3% regression

[mukeshpanchal27]

JFYI, I re-open https://core.trac.wordpress.org/ticket/61858 so editor team also can check it.

[mukeshpanchal27]

Opened PR WordPress/wordpress-develop#7486 that solve the regression. Waiting for the feedback.

[felixarntz]

Since we now reinstated the Code Vitals dashboard, I just took a look at it and noticed https://www.codevitals.run/project/wordpress/home-block-theme-lcpMinusTtfb shows a notable regression that must have happened at some point while it was not working.

It's interesting that our Web Vitals benchmarks for "LCP - TTFB" didn't show a similar regression. Since the Code Vitals dashboard is still using the TT3 theme (which we should probably update at some point), it would be helpful to run another Web Vitals benchmark comparison using TT3, to check whether we see the regression there too - in which case the problem might be something that mostly affects TT3 for some reason.

[mukeshpanchal27]

I ran performance benchmarks for Beta 1 with TT3 theme: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11228728635

Web Vitals benchmarks

Mertic	WP 6.6.2	Beta 1	Diff %	Diff abs.
FCP	99.7	99.15	-0.55%	-0.55
LCP	99.7	99.15	-0.55%	-0.55
TTFB	54.8	56	2.19%	1.20
LCP-TTFB	44.6	42.8	-4.04%	-1.80

The benchmarks didn't show the regression that we see in dashboard.

[mukeshpanchal27]

WP 6.7 Beta 2 Performance Benchmark Report:

With object caching disabled,

• Classic themes (TT1) web vitals and server-timing remains relatively stable.
• Block themes (TT3) show slight regression but remains relatively stable.
• Block themes (TT4) show a ~6% regression in LCP shows a ~4.5% regression (7ms), TTFB shows a ~8% regression (6ms), wp-template shows a ~10% regression (6ms) and wp-total shows a ~10% regression (6ms)

Here are the results of the test runs:

TT1: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11228940062
TT3: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11229064497
TT4: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11229259874

TT1 (twentytwentyone) benchmarks

Web Vitals

Metric	WP 6.6.2	WP 6.7 Beta 2	Diff %	Diff abs.
FCP	102	102.65	0.64%	0.65
LCP	102	102.65	0.64%	0.65
TTFB	47.4	47.25	-0.32%	-0.15
LCP-TTFB	54.8	55.2	0.73%	0.40

Server-Timing

Metric	WP 6.6.2	WP 6.7 Beta 2	Diff %	Diff abs.
Response Time	39.8	39.71	-0.23%	-0.09
wp-before-template	11.47	11.61	1.22%	0.14
wp-template	27.33	27.21	-0.44%	-0.12
wp-total	38.93	38.86	-0.18%	-0.07

TT3 (twentytwentythree) benchmarks

Web Vitals

Metric	WP 6.6.2	WP 6.7 Beta 2	Diff %	Diff abs.
FCP	99.35	99.3	-0.05%	-0.05
LCP	99.35	99.3	-0.05%	-0.05
TTFB	55	55.95	1.73%	0.95
LCP-TTFB	43.35	42.55	-1.85%	-0.80

Server-Timing

Metric	WP 6.6.2	WP 6.7 Beta 2	Diff %	Diff abs.
Response Time	47.64	47.9	0.55%	0.26
wp-before-template	15.89	15.77	-0.76%	-0.12
wp-template	30.76	31.15	1.27%	0.39
wp-total	46.63	46.89	0.56%	0.26

TT4 (twentytwentyfour) benchmarks

Web Vitals

Metric	WP 6.6.2	WP 6.7 Beta 2	Diff %	Diff abs.
FCP	155.55	162.75	4.63%	7.20
LCP	155.55	162.75	4.63%	7.20
TTFB	81.55	88.35	8.34%	6.80
LCP-TTFB	73.8	73.7	-0.14%	-0.10

Server-Timing

Metric	WP 6.6.2	WP 6.7 Beta 2	Diff %	Diff abs.
Response Time	74.11	80.16	8.16%	6.05
wp-before-template	16.31	16.38	0.43%	0.07
wp-template	56.7	62.62	10.44%	5.92
wp-total	72.95	79.01	8.31%	6.06

🔴 alert for TT4, In WordPress/wordpress-develop#7486 we are checking one approach to solve the background image regression that account for 3-4% regression in block theme.

[joemcgill]

Following up on #1572 (comment), I noticed while testing a fix to the Performance Workflow that the LCP-TTFB metric has returned to a more "normal" range but doesn't seem related to any specific code change, so I'm thinking there was a temporary problem with the workflow that has cleared itself up.

[mukeshpanchal27]

It means we should debug as the difference it ~6ms for normal code commit.

[mukeshpanchal27]

WP 6.7 Beta 3 Performance Benchmark Report:

With object caching disabled,

• Classic themes (TT1) web vitals and server-timing remains relatively stable.
• Block themes (TT3) show slight regression but remains relatively stable.
• Block themes (TT4)
  • LCP shows a ~3% regression (~5ms)
  • TTFB shows a ~4% regression (~4ms)
  • wp-template shows a ~9% regression (5ms)
  • wp-total shows a ~7% regression (5ms)

Here are the results of the test runs:

TT1: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11339147727
TT3: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11339238140
TT4: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11339330952

TT1 (twentytwentyone) benchmarks

Web Vitals

Metric	WP 6.6.2	WP 6.7 Beta 3	Diff %	Diff abs.
FCP	100.5	99.7	-0.80%	-0.80
LCP	100.5	99.7	-0.80%	-0.80
TTFB	45.7	45.7	0.00%	0.00
LCP-TTFB	54.4	53.65	-1.38%	-0.75

Server-Timing

Metric	WP 6.6.2	WP 6.7 Beta 3	Diff %	Diff abs.
Response Time	37.87	38.07	0.53%	0.20
wp-before-template	10.43	10.74	2.97%	0.31
wp-template	26.66	26.52	-0.53%	-0.14
wp-total	37.14	37.31	0.46%	0.17

TT3 (twentytwentythree) benchmarks

Web Vitals

Metric	WP 6.6.2	WP 6.7 Beta 3	Diff %	Diff abs.
FCP	97.4	97.45	0.05%	0.05
LCP	97.4	97.45	0.05%	0.05
TTFB	53.95	54.5	1.02%	0.55
LCP-TTFB	42.9	42.75	-0.35%	-0.15

Server-Timing

Metric	WP 6.6.2	WP 6.7 Beta 3	Diff %	Diff abs.
Response Time	47.29	47.74	0.95%	0.45
wp-before-template	15.88	15.95	0.44%	0.07
wp-template	30.3	30.56	0.86%	0.26
wp-total	46.19	46.59	0.87%	0.40

TT4 (twentytwentyfour) benchmarks

Web Vitals

Metric	WP 6.6.2	WP 6.7 Beta 3	Diff %	Diff abs.
FCP	157.2	162.05	3.09%	4.85
LCP	157.2	162.05	3.09%	4.85
TTFB	81.3	85.6	5.29%	4.30
LCP-TTFB	75.45	75.3	-0.20%	-0.15

Server-Timing

Metric	WP 6.6.2	WP 6.7 Beta 3	Diff %	Diff abs.
Response Time	71.41	76.4	6.99%	4.99
wp-before-template	14.99	15.02	0.20%	0.03
wp-template	55.5	60.38	8.79%	4.88
wp-total	70.5	75.44	7.01%	4.94

Beta 3 shows improvement compared to Beta 2, but there is still a regression in Web Vitals and Server-Timing.

[mukeshpanchal27]

Just for visibility, I found another background image related PR WordPress/wordpress-develop#6836 (comment) that is responsible for regression.

[ramonjd]

Just for visibility, I found another background image related PR WordPress/wordpress-develop#6836 (comment) that is responsible for regression.

Following the set up in the docs, I'm having trouble replicating the figures you quoted here locally.

Comparing with and then without those changes, my benchmark stats are unmoved or are neglible in TT5, TT4 and TT3.

I know I'm probably doing something wrong though.

E.g., TT3,

Metric	Percentile	WP trunk	WP trunk without the changes in #6836	Diff %
wp-before-template	p10	16.58	17.21	3.80%
wp-before-template	p25	16.78	17.38	3.58%
wp-before-template	p50 (median)	17.28	17.79	2.95%
wp-before-template	p75	18.04	18.21	0.94%
wp-before-template	p90	18.47	18.78	1.68%
wp-template	p10	12.71	12.79	0.63%
wp-template	p25	12.8	12.84	0.31%
wp-template	p50 (median)	12.88	12.95	0.54%
wp-template	p75	13.01	13.14	1.00%
wp-template	p90	13.41	13.7	2.16%
wp-total	p10	29.32	30.03	2.42%
wp-total	p25	29.66	30.32	2.23%
wp-total	p50 (median)	30.2	30.87	2.22%
wp-total	p75	31	31.22	0.71%
wp-total	p90	31.81	32.6	2.48%

That the "without change" measurement is higher for me I cannot explain, but I ran it multiple times and it swings +- by 0.5 on each side.

[mukeshpanchal27]

Again ran the benchmark for TT4

Server-Timing benchmarks

Mertic	Beta 3	WP trunk without the changes in #6836	Diff %	Diff abs.
Response Time (p10)	65.43	66.71	1.96%	1.28
Response Time (p25)	65.72	67.1	2.10%	1.38
Response Time (p50)	66.6	68.14	2.31%	1.54
Response Time (p75)	67.65	69.45	2.66%	1.80
Response Time (p90)	69.65	72.08	3.49%	2.43
wp-before-template (p10)	29.91	30.51	2.01%	0.60
wp-before-template (p25)	30.19	30.77	1.92%	0.58
wp-before-template (p50)	30.48	31.18	2.30%	0.70
wp-before-template (p75)	30.82	31.89	3.47%	1.07
wp-before-template (p90)	31.75	32.51	2.39%	0.76
wp-template (p10)	33.38	34.07	2.07%	0.69
wp-template (p25)	33.56	34.29	2.18%	0.73
wp-template (p50)	33.93	34.69	2.24%	0.76
wp-template (p75)	34.84	35.68	2.41%	0.84
wp-template (p90)	35.42	37.51	5.90%	2.09
wp-total (p10)	63.5	64.74	1.95%	1.24
wp-total (p25)	63.81	65.14	2.08%	1.33
wp-total (p50)	64.72	66.08	2.10%	1.36
wp-total (p75)	65.52	67.27	2.67%	1.75
wp-total (p90)	67.34	69.87	3.76%	2.53

[ramonjd]

Again ran the benchmark for TT4

Thanks for running it again. I think I still need some clarification on how to interpret these results.

Based on the tables, the server load time increases when the changeset in question is reverted.

Unless a higher number is better 😅 ? My assumption is that a lower server load time number is preferable.

Anyway, I'll keep testing and report back if I find something significant.

It's probably worth looking into caching the output of the method as well.

Thank you for your help!

[ramonjd]

I've run tests both locally and on CI and cannot find any conclusive evidence that WordPress/wordpress-develop#6836 causes a statistically significant regression in performance.

In fact, I'm more confounded than anything. Some reports suggest that trunk is faster than the reverted branch, other times the needle falls in the other direction by between 1-4ms.

Example from a complete revert on CI ("After" is the revert):

Locally the benchmark tests reveal nothing significant for me as mentioned above. I've been running all variations for a couple of hours and am yet to find any reliable results.

Furthermore I added some relative paths to TT4 and tried caching the results of WP_Theme_JSON_Resolver::resolve_theme_file_uris() and WP_Theme_JSON_Resolver::get_resolved_theme_uris(), both of which yielded inconclusive results.

All that said, I do believe we need a caching strategy for these resolved URIs however.

I think perhaps caching them in WP_Theme_JSON_Resolver as is done with theme_json might work, using get_stylesheet() or get_stylesheet_directory() as a key. I'll add it to my list of things to look into.

Edit: This will improve things in general perhaps:

• Update transient cache for wp_add_global_styles_for_blocks wordpress-develop#6879

[aaronrobertshaw]

I've run tests both locally and on CI and cannot find any conclusive evidence that WordPress/wordpress-develop#6836 causes a statistically significant regression in performance.

I've also followed the steps outlined in the docs, using a completely fresh site locally for testing. Every test run appears to yield inconsistent results for me.

This was the latest set of results for me, running a batch of 10000 requests, with PR being a branch reverting the changes in question:

In fact, I'm more confounded than anything.

💯 in the same boat.

I've been running all variations for a couple of hours and am yet to find any reliable results.

+1

[mukeshpanchal27]

Have you both checked for TT4?

[ramonjd]

Have you both checked for TT4?

Yes.

[joemcgill]

Just committed r59256, which shows pretty good improvements for TT4

[ramonjd]

@joemcgill does r59256 require backporting to Gutenberg? Happy to take care of it if you don't have time.

[aaronrobertshaw]

does r59256 require backporting to Gutenberg?

I believe it does. I was also planning to help out there if needed 👍

Core's action calling wp_enqueue_global_styles is replaced in Gutenberg with its version. Which ultimately means gutenberg_add_global_styles_for_blocks is used instead of wp_add_global_styles_for_blocks.

[ramonjd]

I think perhaps caching them in WP_Theme_JSON_Resolver as is done with theme_json might work, using get_stylesheet() or get_stylesheet_directory() as a key.

Took a naive stab at this: WordPress/gutenberg#66155

[mukeshpanchal27]

WP 6.7 RC 1 Performance Benchmark Report:

With object caching disabled,

• Classic themes (TT1) web vitals and server-timing (~2% improvement) remains relatively stable.
• Block themes (TT3)
  • LCP shows a ~3% improvement (~3ms)
  • TTFB shows a ~2% improvement (~1ms)
  • LCP-TTFB shows a ~5% improvement (~2ms)
  • wp-template shows a ~4% improvement (1ms)
  • wp-total shows a ~2% improvement (1ms)
• Block themes (TT4)
  • LCP shows a ~5% regression (~8ms)
  • TTFB shows a ~8% regression (~6ms)
  • wp-template shows a ~10% regression (6ms)
  • wp-total shows a ~8% regression (6ms)

Here are the results of the test runs:

TT1: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11472667306
TT3: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11472757880
TT4: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11477476622

TT1 (twentytwentyone) benchmarks

Web Vitals

Metric	WP 6.6.2	WP 6.7 RC 1	Diff %	Diff abs.
FCP	100.1	99.6	-0.50%	-0.50
LCP	100.1	99.6	-0.50%	-0.50
TTFB	46.1	45.55	-1.19%	-0.55
LCP-TTFB	53.8	53.95	0.28%	0.15

Server-Timing

Metric	WP 6.6.2	WP 6.7 RC 1	Diff %	Diff abs.
Response Time	39.54	38.7	-2.12%	-0.84
wp-before-template	11.35	11.26	-0.79%	-0.09
wp-template	27.19	26.48	-2.61%	-0.71
wp-total	38.61	37.77	-2.18%	-0.84

TT3 (twentytwentythree) benchmarks

Web Vitals

Metric	WP 6.6.2	WP 6.7 RC 1	Diff %	Diff abs.
FCP	97	93.4	-3.71%	-3.60
LCP	97	93.4	-3.71%	-3.60
TTFB	53.85	52.65	-2.23%	-1.20
LCP-TTFB	42.3	40.2	-4.96%	-2.10

Server-Timing

Metric	WP 6.6.2	WP 6.7 RC 1	Diff %	Diff abs.
Response Time	46.53	45.26	-2.73%	-1.27
wp-before-template	15.4	15.4	0.00%	0.00
wp-template	30.14	28.82	-4.38%	-1.32
wp-total	45.59	44.32	-2.79%	-1.27

TT4 (twentytwentyfour) benchmarks

Web Vitals

Metric	WP 6.6.2	WP 6.7 RC 1	Diff %	Diff abs.
FCP	155.9	164.15	5.29%	8.25
LCP	155.9	164.25	5.36%	8.35
TTFB	81.25	87.8	8.06%	6.55
LCP-TTFB	73.65	76.25	3.53%	2.60

Server-Timing

Metric	WP 6.6.2	WP 6.7 RC 1	Diff %	Diff abs.
Response Time	73.54	79.48	8.08%	5.94
wp-before-template	16.2	16.28	0.49%	0.08
wp-template	55.95	61.95	10.72%	6.00
wp-total	72.28	78.24	8.25%	5.96

[mukeshpanchal27]

WP 6.7 RC 2 Performance Benchmark Report:

With object caching disabled,

• Classic themes (TT1) web vitals and server-timing (~2% improvement) remains relatively stable.
• Block themes (TT3)
  • LCP shows a ~1% improvement (~1ms)
  • TTFB shows a ~1% improvement (~1ms)
  • LCP-TTFB shows a ~1% improvement (~0.5ms)
  • wp-template shows a ~4% improvement (~1ms)
  • wp-total shows a ~2% improvement (~1ms)
• Block themes (TT4)
  • LCP shows a ~1% regression (~1.5ms)
  • TTFB shows a ~1.5% regression (~1ms)
  • wp-template shows a ~1% regression (0.5ms)
  • wp-total shows a ~0.5% regression (~0.5ms)

Here are the results of the test runs:

TT1: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11579114106
TT3: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11579298956
TT4: https://github.com/mukeshpanchal27/compare-wp-performance/actions/runs/11579469822

TT1 (twentytwentyone) benchmarks

Web Vitals

Metric	Before	After	Diff %	Diff abs.
FCP	100.2	99.65	-0.55%	-0.55
LCP	100.2	99.65	-0.55%	-0.55
TTFB	46.35	45.8	-1.19%	-0.55
LCP-TTFB	53.75	54	0.47%	0.25

Server-Timing

Metric	Before	After	Diff %	Diff abs.
Response Time	40.55	40.16	-0.96%	-0.39
wp-before-template	11.86	12.04	1.52%	0.18
wp-template	27.63	27.07	-2.03%	-0.56
wp-total	39.53	39.08	-1.14%	-0.45

TT3 (twentytwentythree) benchmarks

Web Vitals

Metric	Before	After	Diff %	Diff abs.
FCP	98.6	96.9	-1.72%	-1.70
LCP	98.6	96.9	-1.72%	-1.70
TTFB	55	54.2	-1.45%	-0.80
LCP-TTFB	42.85	42.4	-1.05%	-0.45

Server-Timing

Metric	Before	After	Diff %	Diff abs.
Response Time	48.23	46.9	-2.76%	-1.33
wp-before-template	16.25	16.22	-0.18%	-0.03
wp-template	30.82	29.6	-3.96%	-1.22
wp-total	47.1	45.82	-2.72%	-1.28

TT4 (twentytwentyfour) benchmarks

Web Vitals

Metric	Before	After	Diff %	Diff abs.
FCP	155.95	157.5	0.99%	1.55
LCP	155.95	157.5	0.99%	1.55
TTFB	80.85	82.1	1.55%	1.25
LCP-TTFB	74.5	74.3	-0.27%	-0.20

Server-Timing

Metric	Before	After	Diff %	Diff abs.
Response Time	72.95	73.39	0.60%	0.44
wp-before-template	15.79	15.78	-0.06%	-0.01
wp-template	55.94	56.51	1.02%	0.57
wp-total	71.82	72.28	0.64%	0.46

RC 2 shows improvement compared to RC 1 in block theme.

[mukeshpanchal27]

I ran local benchmarks for TT4 on versions 6.6.2 and 6.7 RC2 and saw similar results. Core didn’t introduce any performance-related changes between RC1 and RC2, though the Editor packages were updated in the meantime.

Interestingly, the Code Vitals analysis didn’t show any improvement 🤔