chore: add ci analysis for sonar #337

bmunguli · 2024-02-21T15:02:54Z

Description

Run CI analysis with sonar instead of automatic analysis which gives the opportunity for publishing the test coverage results in sonar cloud
Use maven plugin to integrate with sonar as this is the recommended way
Add jacoco to produce test coverage reports and show results in sonarcloud

Pre-review checks

Please ensure to do as many of the following checks as possible, before asking for committer review:

DEPENDENCIES are up-to-date. Dash license tool. Committers can open IP issues for restricted libs.
Copyright and license header are present on all affected files

bmunguli · 2024-02-22T14:24:34Z

Can someone with sonar cloud access rights disable the automatic analysis on sonar so the cli integration can be fully tested if this PR looks good and we would like to merge it? Currently the pipeline is failing because of conflicts between automatic analysis and ci analysis.
Refs: https://docs.sonarsource.com/sonarcloud/advanced-setup/automatic-analysis/#conflict-with-ci-based-analysis

tuncaytunc-zf · 2024-03-04T13:20:16Z

pom.xml


 		<!-- build -->
 		<maven.compiler.version>3.8.1</maven.compiler.version>
+
+


Please remove the second empty line, one empty line should be enough.

sonarcloud · 2024-03-04T15:49:24Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

tuncaytunc-zf

LGTM

tunacicek · 2024-03-04T20:58:14Z

Hi @bmunguli @tuncaytunc-zf ,
the build failed. Do you have the possibility to see the logs?

bmunguli · 2024-03-05T07:10:50Z

Can someone with sonar cloud access rights disable the automatic analysis on sonar so the cli integration can be fully tested if this PR looks good and we would like to merge it? Currently the pipeline is failing because of conflicts between automatic analysis and ci analysis.
Refs: https://docs.sonarsource.com/sonarcloud/advanced-setup/automatic-analysis/#conflict-with-ci-based-analysis

@tunacicek thanks for reviewing this, as mentioned in my previous comment the pipeline is currently failing because of conflicts between automatic analysis and ci analysis analysis enabled in sonarcloud. I need someone with access rights in tractus-x sonarcloud project https://sonarcloud.io/project/overview?id=eclipse-tractusx_sldt-digital-twin-registry to disable the configured automatic analysis so the ci analysis could work.

From the logs:

Error: Failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.10.0.2594:sonar (default-cli) on project digital-twin-registry: You are running CI analysis while Automatic Analysis is enabled. Please consider disabling one or the other. -> [Help 1]

Refs: https://docs.sonarsource.com/sonarcloud/advanced-setup/automatic-analysis/#conflict-with-ci-based-analysis

tunacicek · 2024-03-05T08:40:33Z

Can someone with sonar cloud access rights disable the automatic analysis on sonar so the cli integration can be fully tested if this PR looks good and we would like to merge it? Currently the pipeline is failing because of conflicts between automatic analysis and ci analysis.
Refs: https://docs.sonarsource.com/sonarcloud/advanced-setup/automatic-analysis/#conflict-with-ci-based-analysis

@tunacicek thanks for reviewing this, as mentioned in my previous comment the pipeline is currently failing because of conflicts between automatic analysis and ci analysis analysis enabled in sonarcloud. I need someone with access rights in tractus-x sonarcloud project https://sonarcloud.io/project/overview?id=eclipse-tractusx_sldt-digital-twin-registry to disable the configured automatic analysis so the ci analysis could work.

From the logs:

Error: Failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.10.0.2594:sonar (default-cli) on project digital-twin-registry: You are running CI analysis while Automatic Analysis is enabled. Please consider disabling one or the other. -> [Help 1]

Refs: https://docs.sonarsource.com/sonarcloud/advanced-setup/automatic-analysis/#conflict-with-ci-based-analysis

@bmunguli : Thanks for your feedback.
@Siegfriedk, @RoKrish14 : Could one of you please disable the configured automatic analysis?

tunacicek · 2024-03-05T08:45:03Z

@bmunguli : Could you please update the dependencies file? See also trg-7-04
https://eclipse-tractusx.github.io/docs/release/trg-7/trg-7-04/#checking-libraries-using-the-eclipse-dash-license-tool

You can run the Eclipse Dash License Tool:
https://github.com/eclipse/dash-licenses/blob/master/README.md

bmunguli · 2024-03-05T10:26:14Z

@tunacicek running the dash tool via maven plugin locally updates the DEPENDENCIES file with some libraries i have not introduced as part of this PR. The two new plugins that are introduced: jacoco and sonar are not reflected because they are not dependencies.

Should I skip including those changes as part of this PR?

tunacicek · 2024-03-05T10:41:25Z

@tunacicek running the dash tool via maven plugin locally updates the DEPENDENCIES file with some libraries i have not introduced as part of this PR. The two new plugins that are introduced: jacoco and sonar are not reflected because they are not dependencies.

Should I skip including those changes as part of this PR?

@bmunguli : yes, you can skip those changes. thank you

Siegfriedk · 2024-03-05T14:14:09Z

@tunacicek i disabled the automated analysis

tunacicek · 2024-03-05T16:39:11Z

@tunacicek i disabled the automated analysis

@Siegfriedk :Thank you :)

- passing project key in pom.xml seems to cause

bmunguli · 2024-03-06T09:21:18Z

@tunacicek can you please check if the SONAR_TOKEN is set as a secret on this repository or organisation level?

tunacicek · 2024-03-06T09:45:44Z

@tunacicek can you please check if the SONAR_TOKEN is set as a secret on this repository or organisation level?

@bmunguli i have no permission to check it.
@Siegfriedk : Could you please check if the token is set?

@bmunguli : A reference implementation can be found here:
https://github.com/eclipse-tractusx/traceability-foss

Siegfriedk · 2024-03-06T09:47:47Z

@tunacicek only EF can check it. For adhoc, you can ask frederric gurr per matrix and hope he doesn't bite you :)

tunacicek · 2024-03-06T09:50:52Z

@Siegfriedk thanks, I will try it :)

fredg02 · 2024-03-06T11:09:43Z

Neither a repo-level (for sldt-digital-twin-registry) nor an org-level SONAR_TOKEN secret exists.

fredg02 · 2024-03-06T11:10:32Z

@tunacicek only EF can check it. For adhoc, you can ask frederric gurr per matrix and hope he doesn't bite you :)

1:1 communication does not scale. So please rather mention me in a comment here or create a HelpDesk issue.

tunacicek · 2024-03-06T11:30:48Z

Neither a repo-level (for sldt-digital-twin-registry) nor an org-level SONAR_TOKEN secret exists.

@fredg02 : thanks for your feedback. I would create a ticket to have a token.

tunacicek · 2024-03-06T15:46:49Z

@bmunguli : I created the ticket
https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/4399

tunacicek · 2024-03-07T12:55:30Z

@bmunguli : I created the ticket https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/4399

@bmunguli ticket is closed. SONAR_TOKEN is available now. I rerun the build

tuncaytunc-zf · 2024-03-07T15:24:10Z

.github/workflows/build-snapshot.yml

+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}  # Needed to get PR information, if any
+          SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
+        run: mvn -B --batch-mode sonar:sonar -Dsonar.projectKey=eclipse-tractusx_sldt-digital-twin-registry


I think the -Dsonar.organization=eclipse-tractusx parameter is missing here.

The organization its defined in pom.xml , will add in workflow too and see

bmunguli · 2024-03-08T12:24:37Z

@tunacicek from the logs its complaining that:
failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.10.0.2594:sonar (default-cli) on project digital-twin-registry: Project not found. Please check the 'sonar.projectKey' and 'sonar.organization' properties, the 'SONAR_TOKEN' environment variable, or contact the project administrator

Double Checking the projectKey and organization look correct to me and matching with the project in sonar.
The SONAR_TOKEN is usually bound to a user, could it be that the user who generated the token does not have permissions to perform analysis on this project?

tunacicek · 2024-03-08T14:21:21Z

@tunacicek from the logs its complaining that: failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.10.0.2594:sonar (default-cli) on project digital-twin-registry: Project not found. Please check the 'sonar.projectKey' and 'sonar.organization' properties, the 'SONAR_TOKEN' environment variable, or contact the project administrator

Double Checking the projectKey and organization look correct to me and matching with the project in sonar. The SONAR_TOKEN is usually bound to a user, could it be that the user who generated the token does not have permissions to perform analysis on this project?

Hi @pstankie, Could you check it please?

pstankie · 2024-03-08T14:38:05Z

I added SONAR_ORGANIZATION and SONAR_PROJECT_KEY github variables to repository settings. You can use them:

orgs.newRepoVariable('SONAR_ORGANIZATION') {
      value: "eclipse-tractusx",
    },
    orgs.newRepoVariable('SONAR_PROJECT_KEY') {
      value: "eclipse-tractusx_sldt-digital-twin-registry",
    },

tunacicek · 2024-03-08T14:41:53Z

I added SONAR_ORGANIZATION and SONAR_PROJECT_KEY github variables to repository settings. You can use them:
orgs.newRepoVariable('SONAR_ORGANIZATION') {
      value: "eclipse-tractusx",
    },
    orgs.newRepoVariable('SONAR_PROJECT_KEY') {
      value: "eclipse-tractusx_sldt-digital-twin-registry",
    },

@pstankie :Thank you very much for your support.
@bmunguli : Could you please use the variables instead of hard coded values.

- Align with traceablitiy foss

bmunguli · 2024-03-12T10:45:57Z

@tunacicek i tried the same configuration with a private sonar cloud instance and the integration works fine.
My only guess here why its not working on this PR is the sonar token which is out of my control. Can we please review that token or generate a new one and make sure it has Execute Analysis permissions?

pstankie · 2024-03-12T12:01:00Z

I updated SONAR_TOKEN. Can you try again ?

- it seems accessing the values from variables does not work

pstankie · 2024-03-13T08:05:51Z

So the SONAR_TOKEN is not visible in PR from forked repository. You may try pull_request_target trigger in this specific case.

bmunguli · 2024-03-13T12:45:46Z

@pstankie thank you for your feedback here, will check the triggers

bmunguli · 2024-03-18T09:25:52Z

@tunacicek as @pstankie commented previously github does not allow access to secrets on forked PRs so the sonar analysis can not happen there. I asked the question on the community. Seems there are few options recommended:

changing to pull_request_target
using a github app for the integration
switching on again automatic sonar analysis as the support from sonar is not that great
https://github.com/eclipse-tractusx/traceability-foss/actions/runs/8319310761/workflow or running the scans only in the main branch as they do in this reference implementation
Also for commiters this will work if they create PRs directly into the main branch

I am not super confident about the setup of the first two, shall we go ahead and use same approach as traceability-foss for now (the drawback with this is that we can not get feedback if the integration works until changes got to main)?

tunacicek · 2024-03-27T10:50:39Z

@tunacicek as @pstankie commented previously github does not allow access to secrets on forked PRs so the sonar analysis can not happen there. I asked the question on the community. Seems there are few options recommended:

changing to pull_request_target

using a github app for the integration

switching on again automatic sonar analysis as the support from sonar is not that great

https://github.com/eclipse-tractusx/traceability-foss/actions/runs/8319310761/workflow or running the scans only in the main branch as they do in this reference implementation

Also for commiters this will work if they create PRs directly into the main branch

I am not super confident about the setup of the first two, shall we go ahead and use same approach as traceability-foss for now (the drawback with this is that we can not get feedback if the integration works until changes got to main)?

Hi @bmunguli ,
yes, we can do it like in traceability and then check whether it works or not. Could you fix the merge conflicts?

- sonar can not run on prs from forks

bmunguli · 2024-03-28T11:17:53Z

@tunacicek merged main into the branch and added the soanar changes on a dedicate workflow same as tarceability repo

tunacicek

LGTM

bmunguli added 2 commits February 21, 2024 15:58

ci: add ci analysis for sonar cloud

d68fc21

refactor: add spaces between steps in workflow

b692ef4

tuncaytunc-zf reviewed Mar 4, 2024

View reviewed changes

chore: remove extra empty lines

bd79bf0

tuncaytunc-zf approved these changes Mar 4, 2024

View reviewed changes

tuncaytunc-zf requested a review from tunacicek March 4, 2024 16:10

chore: add sonar project name

28e3a45

fix: pass project key as part of the task instead in pom.xml

1b45bda

- passing project key in pom.xml seems to cause

tuncaytunc-zf reviewed Mar 7, 2024

View reviewed changes

bmunguli added 2 commits March 8, 2024 09:20

fix: add sonar properties as part of the cli task

a08c700

fix: remove redudant argument

0afbbf0

chore: add debugging option to help identify the sonar task

01a8ca8

bmunguli added 3 commits March 11, 2024 10:50

chore: access sonar organization and key from repository varuables

9e3ba02

chore: remove module key configuration

60aa458

- Align with traceablitiy foss

chore: bring back moduleKey configuration needed by multimodule projects

7c7e7fe

ci: revert back to hardcoded values

c3ae786

- it seems accessing the values from variables does not work

bmunguli added 5 commits March 28, 2024 11:20

Merge branch 'main' into ci/Add-sonarcloud-ci-analysis

a000a90

fix: remove sonar cloud steps from build workflow

aaabb26

- sonar can not run on prs from forks

ci: add dedicated workflow for running sonar on schedule and main branch

aa13265

chore: clean up unused variable.

1490f77

chore: remove empty line

090c1ff

tunacicek approved these changes Mar 28, 2024

View reviewed changes

tunacicek merged commit bee47ef into eclipse-tractusx:main Mar 28, 2024
7 checks passed

tunacicek mentioned this pull request Oct 22, 2024

Check Sonarcloud eclipse-tractusx/sldt-discovery-finder#169

Closed


		<!-- build -->
		<maven.compiler.version>3.8.1</maven.compiler.version>

chore: add ci analysis for sonar #337

chore: add ci analysis for sonar #337

Conversation

bmunguli commented Feb 21, 2024 • edited Loading

Description

Pre-review checks

bmunguli commented Feb 22, 2024 • edited Loading

tuncaytunc-zf Mar 4, 2024

Choose a reason for hiding this comment

sonarcloud bot commented Mar 4, 2024

Quality Gate passed

tuncaytunc-zf left a comment

Choose a reason for hiding this comment

tunacicek commented Mar 4, 2024

bmunguli commented Mar 5, 2024

tunacicek commented Mar 5, 2024

tunacicek commented Mar 5, 2024

bmunguli commented Mar 5, 2024

tunacicek commented Mar 5, 2024

Siegfriedk commented Mar 5, 2024

tunacicek commented Mar 5, 2024

bmunguli commented Mar 6, 2024

tunacicek commented Mar 6, 2024

Siegfriedk commented Mar 6, 2024

tunacicek commented Mar 6, 2024

fredg02 commented Mar 6, 2024

fredg02 commented Mar 6, 2024

tunacicek commented Mar 6, 2024

tunacicek commented Mar 6, 2024 • edited Loading

tunacicek commented Mar 7, 2024

tuncaytunc-zf Mar 7, 2024

Choose a reason for hiding this comment

bmunguli Mar 8, 2024 • edited Loading

Choose a reason for hiding this comment

bmunguli commented Mar 8, 2024 • edited Loading

tunacicek commented Mar 8, 2024

pstankie commented Mar 8, 2024

tunacicek commented Mar 8, 2024

bmunguli commented Mar 12, 2024

pstankie commented Mar 12, 2024

pstankie commented Mar 13, 2024

bmunguli commented Mar 13, 2024

bmunguli commented Mar 18, 2024 • edited Loading

tunacicek commented Mar 27, 2024

bmunguli commented Mar 28, 2024

tunacicek left a comment

Choose a reason for hiding this comment

bmunguli commented Feb 21, 2024 •

edited

Loading

bmunguli commented Feb 22, 2024 •

edited

Loading

tunacicek commented Mar 6, 2024 •

edited

Loading

bmunguli Mar 8, 2024 •

edited

Loading

bmunguli commented Mar 8, 2024 •

edited

Loading

bmunguli commented Mar 18, 2024 •

edited

Loading