Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix] Remove gcompat from the Alpine based image since real glibc is installed #23374

Closed
wants to merge 1 commit into from

Conversation

lhotari
Copy link
Member

@lhotari lhotari commented Sep 30, 2024

Motivation

In 3.3.1 and 4.0.0-preview.1 images, Conscrypt cannot be loaded. Jetty uses it for native OpenSSL based TLS.
This caused a problem described in #23364 where the warning log message "2024-09-27T19:25:20,336+0000 [main] WARN org.apache.pulsar.common.util.SecurityUtility - Conscrypt isn't available for Linux amd64. Using JDK default security provider." broke pulsar-admin output parsing in other scripts.
The problem was addressed by setting log level to debug.

While investigating the issue, it appeared that musl + gcompat and glibc in Alpine shouldn't be mixed at runtime in dynamic linking. It's not recommended to mix musl and glibc either.

However, for the Conscrypt case, the loading of the library works when gcompat is removed. There shouldn't be a reason to have gcompat exist with the current real glibc solution in Alpine.

Modifications

  • remove gcompat package from Alpine.

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

@lhotari lhotari added this to the 4.0.0 milestone Sep 30, 2024
@lhotari lhotari requested a review from merlimat September 30, 2024 05:46
@lhotari lhotari self-assigned this Sep 30, 2024
@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Sep 30, 2024
@lhotari
Copy link
Member Author

lhotari commented Sep 30, 2024

IIRC, the reason to add real glibc support was due to the fact that the Pulsar IO Kinesis connector didn't work with the Alpine image. The Kinesis connector uses Amazon Kinesis Producer which contains an embedded native application that has been compiled for glibc.
https://github.com/awslabs/amazon-kinesis-producer/blob/3be1d3d6e77ffa80eb173fd7e06ac962db6ec6b1/java/amazon-kinesis-producer/src/main/java/com/amazonaws/services/kinesis/producer/Daemon.java#L429-L461

There's a script in the repository to compile it for other runtime environments: https://github.com/awslabs/amazon-kinesis-producer/blob/master/bootstrap.sh

For using amazon-kinesis-producer, the correct approach would be to have build a musl compatible binary or have a solution to run the process in an isolated glibc environment. It would be possible to build a minimal environment with the minimal set of libraries and then pass LD_LIBRARY_PATH environment variable to use it for running the kinesis_producer binary. The current library already supports overriding the binary and passing environment variables.

@lhotari lhotari marked this pull request as draft September 30, 2024 13:10
@lhotari lhotari closed this Sep 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
doc-not-needed Your PR changes do not impact docs ready-to-test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant