engine: "unknown_failure: tls: first record does not look like a TLS handshake"

[gurshabad]

Describe the bug

While analysing measurements classified as 'failed', I found many instances of this error (specfically in Pakistan). In all such cases, this error was symptomatic of censorship. A middlebox is likely interrupting HTTPS connections in an improper way, which is unrecognised by OONI clients. For such measurements, the control was usually successful in making a connection with the web endpoint.

Expected behavior

This error should be intercepted and handled (and probably classified as 'anomaly').

System information

All the forty (40) instances I found of this error were on ooniprobe-android (3.6.0). Have classified it as such for the purposes of this bug, I am currently unsure if this exists on other platforms or not.

Additional context

3 examples: 1, 2, 3. 37 other examples in this Google Sheet (Please Ctrl-F for the error description).

[bassosimone]

I could reproduce this issue:

% ./miniooni -n web_connectivity -i https://www.example.com:80/
[...]

[      0.000876] <info> ooniprobe-engine/v3.18.0-alpha b69759ade1c88c263105e55503d9c687c11d709e dirty=false go1.19.6

[...]

[      1.146934] <info> DNS analysis result: consistent
[      1.370952] <info> TCP/TLS endpoints: 1/2 reachable
[      1.371045] <info> GET https://www.example.com:80/...
[      1.595625] <info> GET https://www.example.com:80/... unknown_failure: tls: first record does not look like a TLS handshake

[...]

% tail -n1 report.jsonl | jq '.test_keys|.tls_handshakes'

[
  {
    "network": "",
    "address": "93.184.216.34:80",
    "cipher_suite": "",
    "failure": "unknown_failure: tls: first record does not look like a TLS handshake",
    "negotiated_protocol": "",
    "no_tls_verify": false,
    "peer_certificates": null,
    "server_name": "www.example.com",
    "t": 0.987417,
    "tags": [
      "tcptls_experiment"
    ],
    "tls_version": ""
  }
]

What happened in the real measurement could be different. However, the above straw man test case indicates the direction in which to move to shed light on what was happening at the network level: we need to record the bytes of the first message we receive during the TLS handshake. We should also map this error to an OONI error.

To summarize, here's what we need to do:

• map the specific error string to an OONI error;
• record the bytes of the first message after the TLS handshake.

By mapping the error, we stop treating this kind of failure as an unknown failure. By exposing the bytes in the first message, we shed light on what actually happened.