Refactor rs-e2e project to remove Hapi HL7 library (#1654)

* WIP: refactored to remove hapi hl7 library

Co-authored-by: luis-pabon-tf <[email protected]>

* Moved files to more appropiate location

* Update HL7MessageTest.groovy

Remove library usage on test class, one test not finished yet.

* Accounting for edge cases

* Made class with static methods static

* Improved delimiter handling and added test assertions

* Handling additional edge case

* Update HL7MessageTest.groovy

Corrected structure on test object

* Improved encoding character handling to use defaults and move to HL7Parser

* Update HapiHL7ExpressionEvaluatorTest.groovy

Replacing Message with HL7Message, some incompatible methods remain

* Extracted parsing logic to HL7Parser

* Added method to return string of message and fixed a couple of tests

* Moved flies from /external/hapi folder to /hl7 folder

* Renamed files to remove Hapi prefix

* Renamed files to remove Hapi prefix

* Replaced magic strings

* Update HL7Message.java

Add method for getting segment counts

* Update HL7ExpressionEvaluator.java

Adding replacement method for evaluation collection counts

* Updating tests and some cleanup

* Update HL7ExpressionEvaluator.java

Adding some deprecation flags and replacement methods

* Update HL7ExpressionEvaluatorTest.groovy

Updated test data and a mock

* Refactored for HL7ExpressionEvaluator to use the new parser and remove hapi dependency

Co-authored-by: luis-pabon-tf <[email protected]>

* Update HL7ExpressionEvaluatorTest.groovy

Most tests are passing, and HAPI Message is gone now.

* Fixed test + cleanup

* Fixed a couple of issues in HL7Message and refactored to extract segment into a HL7Segment record

Co-authored-by: luis-pabon-tf <[email protected]>

* Fixed HL7MessageTest tests

* Added HL7MessageException, improved error handling in HL7Message, refined regex pattern and fixed test

* Update HL7FileMatcherTest.groovy

Added more code coverage

* Added test cobverage for HL7Message and fixed parsing issue

* Add more HL7Parser coverage

* Added test coverage for HL7ParserTest

* Removed unused HL7Segment.getIndex

* Completed HL7Parser coverage

* Removed gradle dependency to the hapi library and removed last few mentions of Hapi

* Added test coverage and changed exception type

* Fixed a couple of issues brought up by sonar

* Replacing IllegalArgumentException with HL7ParserException custom exception

* Changed parser behaviour to return an empty string instead of null when value not found. Throwing an exception if hl7 path is not allowed

Co-authored-by: luis-pabon-tf <[email protected]>

* Added a not to the readme about returning an empty string when value not found

Co-authored-by: luis-pabon-tf <[email protected]>

* Added javadocs

* Refactored for better separation of concerns. Created HL7Path and HL7Encoding for this purpose

* Fixed tests, added test coverage and some more refactoring

* Fixed a few more tests. A couple more to go

* Added javadocs

* Updated edge case handling and fixed test

* Fixed remaining test

* Removed unnecessary logic

* Override equals method in HL7Path to address issue raised by SonarCloud

* Need to override hashCode and toString as well

* Added test coverage for new methods

* Moved logic in HL7Parser.parseMessageFieldValue to HL7Message.getValue to avoid circular dependency

* Moved logic in HL7Parser.parsePath to HL7Path.parse to avoid circular dependency

* Removed HL7Parser and moved message parsing logic to HL7Message to avoid circular dependencies

* Fixed equals overload logic

* Update HL7MessageException.java

Removed unused constructor

* Update HL7ExpressionEvaluatorTest.groovy

Added a little more coverage

---------

Co-authored-by: luis-pabon-tf <[email protected]>
Co-authored-by: Luis Pabon <[email protected]>

rs-e2e/README.md

@@ -10,6 +10,7 @@ Information on how to set up the sample files evaluated by the tests can be foun
 
 - The output files generated by the framework are stored in an Azure blob storage container. Every time the tests are run, a cleanup task moves the files to a folder with the year/month/day format for better organization. The files are retained in the container for 90 days before being deleted
 - The code that organizes the files is using EST time zone. This means that if you are in PST, you may run into an issue if you submit a run before 9 PM PST and then run the tests after 9pm. You'd need to make sure to run both tasks before or after 9pm so the files are where they are expected to be
+- The HL7 parser and expression evaluator returns an empty string when the value is not found. It will only throw an exception if the path is not a valid HL7 notation
 
 ## Running the tests
 

rs-e2e/build.gradle

@@ -20,10 +20,6 @@ dependencies {
     // azure
     implementation 'com.azure:azure-storage-blob:12.29.0'
 
-    // hapi hl7
-    implementation 'ca.uhn.hapi:hapi-base:2.5.1'
-    implementation 'ca.uhn.hapi:hapi-structures-v251:2.5.1'
-
     testImplementation 'org.apache.groovy:groovy:4.0.24'
     testImplementation 'org.spockframework:spock-core:2.3-groovy-4.0'
 }

rs-e2e/src/main/java/gov/hhs/cdc/trustedintermediary/rse2e/FileFetcher.java

@@ -1,5 +1,6 @@
 package gov.hhs.cdc.trustedintermediary.rse2e;
 
+import gov.hhs.cdc.trustedintermediary.rse2e.hl7.HL7FileStream;
 import java.util.List;
 
 /**

rs-e2e/src/main/java/gov/hhs/cdc/trustedintermediary/rse2e/AzureBlobFileFetcher.java → rs-e2e/src/main/java/gov/hhs/cdc/trustedintermediary/rse2e/external/azure/AzureBlobFileFetcher.java

@@ -1,10 +1,12 @@
-package gov.hhs.cdc.trustedintermediary.rse2e;
+package gov.hhs.cdc.trustedintermediary.rse2e.external.azure;
 
 import com.azure.storage.blob.BlobClient;
 import com.azure.storage.blob.BlobContainerClient;
 import com.azure.storage.blob.BlobContainerClientBuilder;
 import com.azure.storage.blob.models.BlobItem;
 import com.azure.storage.blob.models.ListBlobsOptions;
+import gov.hhs.cdc.trustedintermediary.rse2e.FileFetcher;
+import gov.hhs.cdc.trustedintermediary.rse2e.hl7.HL7FileStream;
 import java.time.LocalDate;
 import java.time.ZoneId;
 import java.time.ZoneOffset;

rs-e2e/src/main/java/gov/hhs/cdc/trustedintermediary/rse2e/AzureBlobHelper.java → rs-e2e/src/main/java/gov/hhs/cdc/trustedintermediary/rse2e/external/azure/AzureBlobHelper.java

@@ -1,4 +1,4 @@
-package gov.hhs.cdc.trustedintermediary.rse2e;
+package gov.hhs.cdc.trustedintermediary.rse2e.external.azure;
 
 import java.time.LocalDate;
 

rs-e2e/src/main/java/gov/hhs/cdc/trustedintermediary/rse2e/AzureBlobOrganizer.java → rs-e2e/src/main/java/gov/hhs/cdc/trustedintermediary/rse2e/external/azure/AzureBlobOrganizer.java

@@ -1,4 +1,4 @@
-package gov.hhs.cdc.trustedintermediary.rse2e;
+package gov.hhs.cdc.trustedintermediary.rse2e.external.azure;
 
 import com.azure.storage.blob.BlobClient;
 import com.azure.storage.blob.BlobContainerClient;