Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixing integration test starting #477

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

asalamon74
Copy link
Contributor

Description

When I want to run the integration tests with ./gradlew check I've got an error message:

> Could not resolve all files for configuration ':qa:kerberos:downloadHadoop#3.3.2'.
   > Could not resolve hadoop.common:hadoop-3.3.2:hadoop-3.3.2.
     Required by:
         project :qa:kerberos
      > Could not resolve hadoop.common:hadoop-3.3.2:hadoop-3.3.2.
         > Could not get resource 'https://repo.spring.io/plugins-release-local/hadoop/common/hadoop-3.3.2/hadoop-3.3.2/hadoop-3.3.2-hadoop-3.3.2.pom'.
            > Could not GET 'https://repo.spring.io/plugins-release-local/hadoop/common/hadoop-3.3.2/hadoop-3.3.2/hadoop-3.3.2-hadoop-3.3.2.pom'. Received status code 401 from server:
      > Could not resolve hadoop.common:hadoop-3.3.2:hadoop-3.3.2.
         > Could not get resource 'https://artifacts.opensearch.org/snapshots/hadoop/common/hadoop-3.3.2/hadoop-3.3.2/hadoop-3.3.2-hadoop-3.3.2.pom'.
            > Could not GET 'https://artifacts.opensearch.org/snapshots/hadoop/common/hadoop-3.3.2/hadoop-3.3.2/hadoop-3.3.2-hadoop-3.3.2.pom'. Received status code 403 from server: Forbidden
      > Could not resolve hadoop.common:hadoop-3.3.2:hadoop-3.3.2.
         > Could not get resource 'https://snapshots.opensearch.org/releases/core/opensearch/hadoop-3.3.2/hadoop-3.3.2-min-hadoop-3.3.2.xml'.
            > Could not GET 'https://snapshots.opensearch.org/releases/core/opensearch/hadoop-3.3.2/hadoop-3.3.2-min-hadoop-3.3.2.xml'.
               > snapshots.opensearch.org: nodename nor servname provided, or not known
      > Could not resolve hadoop.common:hadoop-3.3.2:hadoop-3.3.2.
         > Could not get resource 'https://artifacts.opensearch.org/releases/core/opensearch/hadoop-3.3.2/hadoop-3.3.2-min-hadoop-3.3.2.xml'.
            > Could not GET 'https://artifacts.opensearch.org/releases/core/opensearch/hadoop-3.3.2/hadoop-3.3.2-min-hadoop-3.3.2.xml'. Received status code 403 from server: Forbidden

and the test are not started.

Issues Resolved

The error message is a bit misleading, we are trying to download artifacts from https://apache.osuosl.org/ but the required versions are not available anymore. After updating the version numbers I've got new errors:

> Could not resolve opensearch-distribution-snapshot:opensearch:2.13.0-SNAPSHOT.
         > Could not get resource 'https://artifacts.opensearch.org/snapshots/core/opensearch/2.13.0-SNAPSHOT/opensearch-min-2.13.0-SNAPSHOT-linux-x64-latest.tar.gz'.
            > Could not HEAD 'https://artifacts.opensearch.org/snapshots/core/opensearch/2.13.0-SNAPSHOT/opensearch-min-2.13.0-SNAPSHOT-linux-x64-latest.tar.gz'.
               > The server may not support the client's requested TLS protocol versions: (TLSv1.2, TLSv1.3). You may need to configure the client to allow other protocols to be used. For more on this, please refer to https://docs.gradle.org/8.5/userguide/build_environment.html#sec:gradle_system_properties in the Gradle documentation.
                  > Received fatal alert: handshake_failure

I can fix this with a gradle.properties change.

The next error message is keytab releated

 What went wrong:
A problem was found with the configuration of task ':qa:kerberos:hadoopFixture#namenode.configure' (type 'DefaultTestClustersTask').
  - In plugin 'org.opensearch.gradle.JdkDownloadPlugin' type 'org.opensearch.gradle.testclusters.DefaultTestClustersTask' property 'clusters.integTest$0.nodes.$0.extraConfigFiles.opensearch.keytab$0.file' specifies file '/home/systest/opensearch-hadoop/qa/kerberos/build/fixtures/kdcFixture/opensearch.keytab' which doesn't exist.

We need to create keytabs at the beginning. I've also fixed this.

Finally I've got the following error message:

Caused by: org.opensearch.hadoop.rest.OpenSearchHadoopRemoteException: no handler found for uri [/_security/user/client] and method [POST]
        at org.opensearch.hadoop.rest.ErrorExtractor.extractError(ErrorExtractor.java:108)
        at org.opensearch.hadoop.rest.RestClient.checkResponse(RestClient.java:473)
        ... 7 more

I'm not too familiar with this API but I have the feeling that this is some ancient code from elasticsearch and opensearch have a different API.

Can someone with opensearch experience help here?

-Was it possible to run these tests ever? Some of my fixes indicates that it's not working for a long time.
-Is it worth committing this half-solution? At least we can start the tests and some of them are working with this.
-I think it would be better to fix this no handler for uri error. Can someone help me what is the correct API here?

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Andras Salamon <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant