Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge redesign into develop #2

Merged
merged 14 commits into from
Oct 21, 2021
Merged

Merge redesign into develop #2

merged 14 commits into from
Oct 21, 2021

Conversation

johannesduesing
Copy link
Owner

@johannesduesing johannesduesing commented Oct 21, 2021

Reason for this PR
We decided to reengineer parts of the delphi system to be more flexible in terms of new feature and new data. We first decided to split the ES index into multiple parts, and reduce the application to parsing POM files for now.

Changes in this PR

  • Upgrade to Scala 2.12.15 and SBT 1.5.5
  • Upgrade to latest versions of Elastic4s, AKKA, Apache Maven and OPAL
  • Removed code that was stale or not needed for the redesign
  • Switched to a new library for parsing POM files
  • Introduced PomFileReadActor for processing POM files plus unit tests
  • Adopted storage interface to store identifiers, metadata and hermes results in separate indices
  • Adopted storage maintenance interface to create three separate indices
  • Made download more flexibel, JAR file download success is now not required anymore if packaging is not set to jar
  • Improved error handling: Errors in the different phases (downloading, metadata extraction, hermes processing) are caught and stored in a separate ES index.
  • Improved handling of OPAL logging, can now be switched on or off as desired

References
Fully incorporates all functionalities proposed in base repo PRs #49, #48 and #47

@johannesduesing johannesduesing self-assigned this Oct 21, 2021
@johannesduesing johannesduesing merged commit bb72b1c into develop Oct 21, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant