Remove boringssl interop testing

[baentsch]

Given (oqs-)boringssl apparently does not receive the same level of "care and attention" as oqs-openssl111 would it be reasonable to remove the interop-test with it? It always breaks when algorithms get changed in liboqs without prior proper downstream preparation.

[christianpaquin]

Where does it run, in the openssl's porject CI?

[baentsch]

Yes:

openssl/.circleci/config.yml

Lines 97 to 124 in 5118e6e

	ubuntu_boringssl_interop:
	description: A job that tests interoperability between OQS-OpenSSL and OQS-BoringSSL on a x64 Ubuntu Bionic Docker VM
	docker:
	- image: openquantumsafe/ci-ubuntu-focal-x86_64:latest
	# Re-enable iff docker enforces rate limitations without auth:
	# auth:
	# username: $DOCKER_LOGIN
	# password: $DOCKER_PASSWORD
	steps:
	- checkout # change this from "checkout" to "*localCheckout" when running CircleCI locally
	- run:
	name: Clone liboqs and BoringSSL
	command: ./oqs-scripts/clone_liboqs.sh && ./oqs-scripts/clone_boringssl.sh
	- run:
	name: Build liboqs
	command: env LIBOQS_USE_OPENSSL=OFF ./oqs-scripts/build_liboqs.sh
	- run:
	name: Build BoringSSL
	command: ./oqs-scripts/build_boringssl.sh
	- run:
	name: Build OpenSSL
	command: ./Configure linux-x86_64 no-shared no-tests && make -j15
	- run:
	name: Test BoringSSL client against OpenSSL server
	command: python3 -m pytest --numprocesses=auto oqs-interop-test/test_full.py --client-type=bssl
	- run:
	name: Test OpenSSL client against BoringSSL server
	command: python3 -m pytest --numprocesses=auto oqs-interop-test/test_full.py --client-type=ossl

[dstebila]

Do we do the reverse direction of testing? I.e., BoringSSL's CI runs an OpenSSL interop? If so then I think that would suffice.

[baentsch]

Do we do the reverse direction of testing? I.e., BoringSSL's CI runs an OpenSSL interop?

No. This highlights the issue: BoringSSL's CI and interop testing is not at the same level as that of OQS-OpenSSL/oqsprovider. We (also therefore) once decided to "demote" it from "OQS-supported" status, but now seem to have reverted that decision in the interest of retaining Chromium demo-only support. This means a "second class/demo-only" BoringSSL can hold back (a whole chain of supported) OpenSSL use cases.

So I'd suggest we either decide to "promote" BoringSSL again to "fully supported" (but then need to first do PRs for that before merging "breaking" liboqs changes) OR move the "onus of interop testing" to the "second class citizen" BoringSSL: If interop issues happen then, it only breaks CI for BoringSSL and someone may (or may not :) care.

My personal opinion is clear: Do (BoringSSL/OpenSSL) interop testing only in BoringSSL.

[christianpaquin]

My personal opinion is clear: Do (BoringSSL/OpenSSL) interop testing only in BoringSSL.

I think that makes sense

[dstebila]

I agree, we are treating BoringSSL as a second-tier project for now, and we don't want its limitations to negatively impact first-tier projects like OpenSSL. So it also makes sense to put the onus of interop testing in BoringSSL.