Use Curl for the FPM health check requests if it's available

[techanvil]

Feature Description

At the moment we're unconditionally using file_get_contents() to request the FPM health check URLs.

As discussed on Slack, there can be scenarios where file_get_contents() doesn't work, but using Curl does.

It should also be noted that the measurement.php script itself uses Curl in preference to file_get_contents() if it's installed:

site-kit-wp/fpm/measurement.php

Lines 193 to 201 in 1ea478e

	public function sendRequest($url): array
	{
	if ($this->isCurlInstalled()) {
	$response = $this->sendCurlRequest($url);
	} else {
	$response = $this->sendFileGetContents($url);
	}
	return $response;
	}

We should therefore apply the same logic to the health check requests, using Curl to make the request if it's installed on the server.

---

Do not alter or remove anything below. The following sections will be managed by moderators only.

Acceptance criteria

• If the Curl PHP extension is installed on the Site Kit host, the FPM health check requests should be made using Curl.

Implementation Brief

Note that the implementation should use code from measurement.php to make the requests, in order to keep the health check logic as close to the proxy behaviour as possible. This will involve reusing the RequestHelper class, as well as possibly routing a request through measurement.php itself.

• Update includes/Core/Tags/First_Party_Mode/First_Party_Mode.php:

  • Use Google\FirstPartyLibrary\RequestHelper;.
  • Update the is_endpoint_healthy method:
    • Create a new instance of RequestHelper.
    • Get $response by calling the sendRequest method on RequestHelper.

• Update composer.json adding a files key to the autoload key with the value [ 'fpm/measurement.php' ] to autoload the classes in this file.

• Update fpm/measurement.php:

  • Update the following conditional block:
    

    site-kit-wp/fpm/measurement.php

    Line 285 in d1c9ec6

    if (!defined('IS_FIRST_PARTY_MODE_TEST')) {

  • Add the condition !defined( 'WPINC' ), this prevents the code from being executed when the classes are included via autoload within the plugin rather than being called directly.
    • Add code comments here to make the changes clear and to ensure we retain these changes in future updates.

Test Coverage

• No additional test coverage required.

QA Brief

• Set up Site Kit with the firstPartyMode feature flag enabled and Analytics connected (and/or test with the Ads module).
• On a site where the FPM health checks are known to be passing, navigate to the Analytics settings edit page and verify the "Your server’s current settings prevent first-party mode from working" warning notice does not appear.
• Using a site where the FPM health checks are known to be previously failing _for a reason other than the allow_url_fopen PHP setting being disabled on the server, verify the warning notice still does appear.
• For sites where the health check has failed because allow_url_fopen is disabled on the server, the warning should no longer appear because Curl doesn't rely on the allow_url_fopen setting.

If Curl is installed, it will be listed in the Server section of Site Health. The expectation is that it will be installed because Site Kit relies on it in order to use WordPress' HTTP request utilities:

To see if the health checks are failing due to the allow_url_fopen setting, either examine your site's php.ini settings directly to see if the setting is enabled, or install the build for PR #9913 and examine the response of a request to fpm-server-requirement-status:

await googlesitekit.api.get('core','site','fpm-server-requirement-status', undefined, { useCache: false });

See this Asana task for more details, and ask @jamesozzie for access to his sites with health check failures which meet the criteria listed above if necessary.

Changelog entry

• Update FPM health check to use the approach from the measurement script.

[benbowler]

If we have more control over measurement.php we could extract the RequestHandler class and import it later within the plugin and measurements.php.

With the above approach we could contribute back the additional conditional logic so that this change is included in future updates.

[techanvil]

Thanks @benbowler, SGTM.

It would be good to feed this back to the FPM library developers... Tagging @aaemnnosttv, who's had a hand in the development.

IB ✅

[techanvil]

Actually, further on the above I've noticed this comment in measurement.php, which suggests the RequestHelper class is available as an independent file in the source library that we could consider copying over as well. This would allow us to remove one of our modifications, although it would add ~1.5KB to the build size.

site-kit-wp/fpm/measurement.php

Lines 146 to 156 in bb6bf17

	// REQUEST_HELPER_START
	/**
	* NOTE: DO NOT edit RequestHelper directly nor remove the start and end tags.
	*
	* This class is copied over from src/RequestHelper.php. If any changes are
	* needed, change that file and run the command `npm run copy:RequestHelper`.
	*/
	/**
	* Isolates network requests and other methods like exit to inject into classes.
	*/
	class RequestHelper

Of course even if it wasn't a separate file we could extract it ourselves, but the fact that it is would make the exercise more within the spirit of just copying things over without modifying them.

[kelvinballoo]

QA Update ⚠

Hi @techanvil ,

I wanted to drill down a bit more on the following QAB steps:

examine your site's php.ini settings directly to see if the setting is enabled, or install the build for PR #9914

• Could you elaborate a bit more on those please? Unless when we say 'install the build for PR Return error messages from FPM healthchecks and use Curl if it's available. #9914, it means connecting to the branch for Return error messages from FPM healthchecks and use Curl if it's available. #9914 instead of Develop?

[techanvil]

Hi @kelvinballoo - apologies, that was actually a typo and should have read "install the build for PR #9913", by which I do mean changing to the branch for that PR instead of develop.

Examining your site's php.ini file will be a site-specific process. If in doubt I'd say give the branch for #9913 a go.

With that branch installed, examine the result of a request to the fpm-server-requirement-status endpoint by running the following in the browser's devtools. If the health checks fail you should see error messages in the response.

await googlesitekit.api.get('core','site','fpm-server-requirement-status', undefined, { useCache: false });

[kelvinballoo]

QA Update ⚠

Hi @techanvil , thanks for the clarification.
I do have some further queries on this ticket.

From my understanding, we have 3 main scenarios here:

• Scenario 1: Healthcheck passing

  • No issue here

• Scenario 2: Healthcheck failed + cURL enabled

  • When we have this, the warning is not supposed to show based on the QAB. However, I do see the warning.
  • Video is attached below for reference and there are 2 main issues from the video:
    • Why does the warning show up? From my understanding, since cURL is there, it should not show up. Or is it because my error is different than yours?
  9916.02.-.failed.msg.+.cURL.on.mov

• Scenario 3: Healthcheck failed + cURL disabled
  Currently, I've been working with James to have a site that doesn't have cURL set up but in vain. All sites are having cURL even after a few attempts from James to disable it.
  Is there any other way to test this? Or would you be able to set up James' with that scenario?

In case I'm missing any scenario or detail, leading to my wrong result, please let me know.

[techanvil]

Hi @kelvinballoo, thanks for your further testing. To answer your questions:

• Scenario 2: Healthcheck failed + cURL enabled
  • When we have this, the warning is not supposed to show based on the QAB. However, I do see the warning.
  • Video is attached below for reference and there are 2 main issues from the video:
    • Why does the warning show up? From my understanding, since cURL is there, it should not show up. Or is it because my error is different than yours?

As per the QAB, the only error that we're expecting to fix as a result of using Curl is the one where the error message indicates that the healthcheck has failed due to the allow_url_fopen setting being set to 0, e.g. with the error message file_get_contents(): https:// wrapper is disabled in the server configuration by allow_url_fopen=0 (I've updated the devtools screen grab in the QAB to illustrate this message).

As you've surmised the healthcheck for the generatorsireland.net site is failing for a different reason and the expectation is for this to be unaffected by the switch to Curl.

Btw, reviewing the QAB I've realised it would be useful to test this healthcheck still fails and the warning does appear for a site with Curl both enabled and disabled. I've updated the QAB as follows to clarify this:

* Using a site where the FPM health checks are known to be failing for a reason other than the allow_url_fopen PHP setting being disabled on the server, and where Curl is not installed, verify the warning notice still does appear.

• Using a site where the FPM health checks are known to be previously failing for a reason other than the allow_url_fopen PHP setting being disabled on the server, and where the PHP Curl extension is installed, verify the warning notice still does appear.
• Disable the Curl extension on the above site and verify the warning notice still does appear.

As for a site where the allow_url_fopen=0 error is occurring, as discussed on Asana, the site we've tested which has this result is plasticskip.com. So, the expectation is that with the PR build installed on plasticskip.com the healthcheck should now pass.

• Scenario 3: Healthcheck failed + cURL disabled
  Currently, I've been working with James to have a site that doesn't have cURL set up but in vain. All sites are having cURL even after a few attempts from James to disable it.
  Is there any other way to test this? Or would you be able to set up James' with that scenario?

Taking a look at generatorsireland.net via SSH, it looks like it should be possible to disable Curl as follows:

1. Edit /etc/php/8.2/mods-available/curl.ini and comment out the line extension=curl.so.
2. Restart nginx: sudo service nginx restart.

@jamesozzie might want to try this, or I'd be happy to do so myself or with James on a call, or to look into this further as needs be.

Please take another look and let me know how you get on, I'd be happy to sync to discuss this further if that would help.

[kelvinballoo]

QA Update ⚠

Thanks @techanvil .
As per our slack conversations and testings, we have narrowed down the testing and we are not disabling cURL anymore as SK needs cURL to function properly.
Also, we are not able to properly test the scenario with health check failing because allow_url_fopen is disabled on the server.

I was able to verify:

• That when cURL is installed and health check is failing due to reason other than allow_url_fopen is disabled. The warning will show up accordingly. This is good.
  However, I did notice that the 'enhance measurement' keeps on loading. Video is attached below. ⚠
  Is it due to me not having enough access to analytics? (image attached as well)

  9916.01.-.other.than.f.url.open.+.curl.installed.mov

  

• On a site where healthcheck is passing, the warning doesn't show up.

  

[techanvil]

Hi @kelvinballoo, thanks for the update.

I was able to verify:

• That when cURL is installed and health check is failing due to reason other than allow_url_fopen is disabled. The warning will show up accordingly. This is good.
  However, I did notice that the 'enhance measurement' keeps on loading. Video is attached below. ⚠
  Is it due to me not having enough access to analytics? (image attached as well)

Yup, as the error message in your screenshot indicates, you don't have view access to the connected property and this is what's will be causing the enhanced measurement control/value to be stuck in the loading state.

It's out of scope for this issue, but please do go ahead and create a new issue where we can consider how to improve the UX for this scenario.

[kelvinballoo]

QA Update ✅

Thanks @techanvil , new ticket is attached here: #9988

As for this ticket, it's good to be moved to approval.
I was able to verify:

• That when cURL is installed and health check is failing due to reason other than allow_url_fopen is disabled. The warning will show up accordingly. This is good.

  9916.01.-.other.than.f.url.open.+.curl.installed.mov

• On a site where health check is passing, the warning doesn't show up.