Skip to content

Commit

Permalink
SMILE refactoring
Browse files Browse the repository at this point in the history
- refactored metadb references to smile
- packages and class names updated
- updated some prop names
- updated mocked test data

Signed-off-by: Angelica Ochoa <[email protected]>
  • Loading branch information
ao508 committed Mar 25, 2022
1 parent a18c2bc commit ea94969
Show file tree
Hide file tree
Showing 27 changed files with 241 additions and 240 deletions.
18 changes: 9 additions & 9 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,19 +10,19 @@ orbs:
jobs:
run_checkstyle:
<<: *defaults
working_directory: /tmp/repos/cmo-metadb-label-generator
working_directory: /tmp/repos/smile-label-generator
steps:
- checkout
- run:
name: Pull cmo metadb common code
name: Pull smile commons code
command: |
export METADB_COMMON_VERSION=$(grep 'cmo_metadb_common\.version' pom.xml | head -1 | sed 's/<cmo_metadb_common\.version>//g' | sed 's|</cmo_metadb_common\.version>||' | tr -d '[:blank:]' | cut -f1) && \
export METADB_COMMON_ORG=$(grep 'cmo_metadb_common\.group' pom.xml | head -1 | sed 's/<cmo_metadb_common\.group>//g' | sed 's|</cmo_metadb_common\.group>||' | tr -d '[:blank:]' | cut -d. -f3) && \
export SMILE_COMMONS_VERSION=$(grep 'smile_commons\.version' pom.xml | head -1 | sed 's/<smile_commons\.version>//g' | sed 's|</smile_commons\.version>||' | tr -d '[:blank:]' | cut -f1) && \
export SMILE_COMMONS_ORG=$(grep 'smile_commons\.group' pom.xml | head -1 | sed 's/<smile_commons\.group>//g' | sed 's|</smile_commons\.group>||' | tr -d '[:blank:]' | cut -d. -f3) && \
cd .. && \
git clone https://github.com/$METADB_COMMON_ORG/cmo-metadb-common.git && \
cd cmo-metadb-common && \
git clone https://github.com/$SMILE_COMMONS_ORG/smile-commons.git && \
cd smile-commons && \
git fetch --tags && \
git checkout ${METADB_COMMON_VERSION}
git checkout ${SMILE_COMMONS_VERSION}
- run:
name: "Build project..."
command: mvn clean install
Expand All @@ -31,9 +31,9 @@ jobs:
command: mvn checkstyle:checkstyle
- run:
name: "Validating checkstyle plugin report..."
command: bash /tmp/repos/cmo-metadb-common/test/scripts/generate_checkstyle_report.sh /tmp/repos/cmo-metadb-label-generator
command: bash /tmp/repos/smile-commons/test/scripts/generate_checkstyle_report.sh /tmp/repos/smile-label-generator
- store_artifacts:
path: /tmp/repos/cmo-metadb-label-generator/checkstyle_report_compiled.txt
path: /tmp/repos/smile-label-generator/checkstyle_report_compiled.txt
destination: /checkstyle_report.txt

workflows:
Expand Down
11 changes: 6 additions & 5 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,10 @@ Please follow these checks if any changes were made to any classes in the web, s

**Data checks:**
Updates were made to the mocked incoming request data and/or mocked published request data:
- [ ] [cmo-metadb test data](https://github.com/mskcc/cmo-metadb/tree/master/service/src/test/resources/data)
- [ ] [cmo-metadb-common test data](https://github.com/mskcc/cmo-metadb-common/tree/master/src/test/resources/data)
- [ ] [cmo-metadb-label-generator test data](https://github.com/mskcc/cmo-metadb-label-generator/tree/master/src/test/resources/data)

- [ ] [smile-server test data](https://github.com/mskcc/smile-server/tree/master/service/src/test/resources/data)
- [ ] [smile-commons test data](https://github.com/mskcc/smile-commons/tree/master/src/test/resources/data)
- [ ] [smile-label-generator test data](https://github.com/mskcc/smile-label-generator/tree/master/src/test/resources/data)

**Code checks:**
- [ ] Unit tests were updated in relation to updates to the mocked test data.
Expand All @@ -56,8 +57,8 @@ Please describe how the workflow and messaging was tested/simulated:

- NATS [local, local docker, dev server, production]
- Neo4j [local, local docker, dev server, production]
- MetaDB [local, local docker, dev server, production]
- Message publishing simulation [nats cli, docker nats cli, metadb publisher tool, other (describe below)]
- SMILE Server [local, local docker, dev server, production]
- Message publishing simulation [nats cli, docker nats cli, smile publisher tool, other (describe below)]

Other: [insert details on how messages were published or simulated for testing]

Expand Down
4 changes: 2 additions & 2 deletions .github/release-drafter.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name-template: '$NEXT_PATCH_VERSION.RELEASE'
tag-template: '- $TITLE @$AUTHOR (#$NUMBER)'
tag-template: '$NEXT_PATCH_VERSION.RELEASE'
categories:
- title: 🚀 Features
label: feature
Expand All @@ -14,4 +14,4 @@ template: |
## 🕵️ Full commit logs
https://github.com/mskcc/cmo-metadb-label-generator/compare/$PREVIOUS_TAG...$NEXT_PATCH_VERSION
https://github.com/mskcc/smile-label-generator/compare/$PREVIOUS_TAG...$NEXT_PATCH_VERSION.RELEASE
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -7,5 +7,5 @@ RUN mvn clean install

# copy jar and set entrypoint
FROM openjdk:8-slim
COPY --from=0 /label-generator/target/cmo_label_generator.jar /label-generator/cmo_label_generator.jar
COPY --from=0 /label-generator/target/smile_label_generator.jar /label-generator/smile_label_generator.jar
ENTRYPOINT ["java"]
18 changes: 9 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# CMO Label Generator 🔍

CMO Label Generator is a distributed microservices system. It receives messages from LIMS when a request is marked delivered. This message is processed and a unique CMO label Id is generated based on specifications that can be found in CMOPatientIDandSampleIDgeneration.pdf. Any new message is then published to downstream subscribers. The message published is a JSON reqpresentation of MetaDbRequest (more details can be found in cmo-metadb repository).
CMO Label Generator is a distributed microservices system. It receives messages from LIMS when a request is marked delivered. This message is processed and a unique CMO label Id is generated based on specifications that can be found in CMOPatientIDandSampleIDgeneration.pdf. Any new message is then published to downstream subscribers. The message published is a JSON reqpresentation of SmileRequest (more details can be found in smile-server repository).

Label Generator is dependent on MetaDb, Nats Jetstream and Nats RequestReply.
Label Generator is dependent on SMILE, Nats Jetstream and Nats RequestReply.

## Run

Expand All @@ -12,7 +12,7 @@ Make an `application.properties` based on [application.properties.EXAMPLE](src/m

Nats Pub/Sub topics:
- `igo.cmo_label_generator_topic` : receives requests to process
- `request_reply.patient_samples_topic` : used to patient samples set from MetaDB
- `request_reply.patient_samples_topic` : used to patient samples set from SMILE Server
- `igo.new_request_topic` : publishes the processed request to this topic

All properties are required with the exception of some NATS connection-specific properties. The following are only required if `nats.tls_channel` is set to `true`:
Expand Down Expand Up @@ -40,7 +40,7 @@ mvn clean install
Run with

```
java -jar server/target/cmo_label_generator.jar
java -jar server/target/smile_label_generator.jar
```

### With Docker
Expand All @@ -65,14 +65,14 @@ If the Docker image is built with the properties baked in then simply run with:

```
docker run --name cmo-label-generator <repo>/<tag>:<version> \
-jar /cmo-label-generator/cmo_label_generator.jar
-jar /label-generator/smile_label_generator.jar
```

Otherwise use a bind mount to make the local files available to the Docker image and add `--spring.config.location` to the java arg

```
docker run --mount type=bind,source=<local path to properties files>,target=/cmo-label-generator/src/main/resources \
--name cmo-label-generator <repo>/<tag>:<version> \
-jar /cmo-label-generator/cmo_label_generator.jar \
--spring.config.location=/cmo-label-generator/src/main/resources/application.properties
docker run --mount type=bind,source=<local path to properties files>,target=/label-generator/src/main/resources \
--name label-generator <repo>/<tag>:<version> \
-jar /label-generator/smile_label_generator.jar \
--spring.config.location=/label-generator/src/main/resources/application.properties
```
56 changes: 28 additions & 28 deletions pom.xml
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.mskcc.cmo.metadb</groupId>
<groupId>org.mskcc.smile</groupId>
<artifactId>master</artifactId>
<packaging>jar</packaging>
<name>MSKCC CMO MetaDB Label Generator</name>
<name>SMILE CMO Label Generator</name>
<version>0.1.0</version>
<description>master maven module</description>
<url>https://github.com/mskcc/cmo-metadb-label-generator</url>
<url>https://github.com/mskcc/smile-label-generator</url>

<parent>
<groupId>org.springframework.boot</groupId>
Expand Down Expand Up @@ -35,35 +35,35 @@
<spring.boot.version>2.3.3.RELEASE</spring.boot.version>
<slf4j.version>1.7.30</slf4j.version>
<jackson.version>2.11.2</jackson.version>
<!-- metadb messaging -->
<cmo_metadb_messaging_java.group>com.github.mskcc</cmo_metadb_messaging_java.group>
<cmo_metadb_messaging_java.version>1.2.2.RELEASE</cmo_metadb_messaging_java.version>
<!-- metadb common centralized config properties -->
<cmo_metadb_common.group>com.github.mskcc</cmo_metadb_common.group>
<cmo_metadb_common.version>1.2.2.RELEASE</cmo_metadb_common.version>
<!-- metadb server model module -->
<cmo_metadb_server.group>com.github.mskcc.cmo-metadb</cmo_metadb_server.group>
<cmo_metadb_server.version>1.2.3.RELEASE</cmo_metadb_server.version>
<!-- smile messaging and shared entities dependency versions -->
<smile_messaging_java.group>com.github.mskcc</smile_messaging_java.group>
<smile_messaging_java.version>1.3.1.RELEASE</smile_messaging_java.version>
<!-- smile commons centralized config properties -->
<smile_commons.group>com.github.mskcc</smile_commons.group>
<smile_commons.version>1.3.1.RELEASE</smile_commons.version>
<!-- smile server model module -->
<smile_server.group>com.github.mskcc.smile-server</smile_server.group>
<smile_server.version>1.3.0.RELEASE</smile_server.version>
</properties>

<dependencies>
<!-- cmo messaging library -->
<!-- smile messaging library -->
<dependency>
<groupId>${cmo_metadb_messaging_java.group}</groupId>
<artifactId>cmo-messaging-java</artifactId>
<version>${cmo_metadb_messaging_java.version}</version>
<groupId>${smile_messaging_java.group}</groupId>
<artifactId>smile-messaging-java</artifactId>
<version>${smile_messaging_java.version}</version>
</dependency>
<!-- cmo metadb common -->
<!-- smile commons -->
<dependency>
<groupId>${cmo_metadb_common.group}</groupId>
<artifactId>cmo-metadb-common</artifactId>
<version>${cmo_metadb_common.version}</version>
<groupId>${smile_commons.group}</groupId>
<artifactId>smile-commons</artifactId>
<version>${smile_commons.version}</version>
</dependency>
<!-- cmo metadb server model -->
<!-- smile server model -->
<dependency>
<groupId>${cmo_metadb_server.group}</groupId>
<groupId>${smile_server.group}</groupId>
<artifactId>model</artifactId>
<version>${cmo_metadb_server.version}</version>
<version>${smile_server.version}</version>
</dependency>
<!-- spring -->
<dependency>
Expand Down Expand Up @@ -116,7 +116,7 @@
</dependencies>

<build>
<finalName>cmo_label_generator</finalName>
<finalName>smile_label_generator</finalName>
<resources>
<resource>
<directory>src/main/resources</directory>
Expand Down Expand Up @@ -158,9 +158,9 @@
<version>3.1.1</version>
<dependencies>
<dependency>
<groupId>${cmo_metadb_common.group}</groupId>
<artifactId>cmo-metadb-common</artifactId>
<version>${cmo_metadb_common.version}</version>
<groupId>${smile_commons.group}</groupId>
<artifactId>smile-commons</artifactId>
<version>${smile_commons.version}</version>
</dependency>
</dependencies>
<configuration>
Expand All @@ -187,7 +187,7 @@
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<mainClass>org.mskcc.cmo.metadb.CmoLabelGeneratorApp</mainClass>
<mainClass>org.mskcc.smile.CmoLabelGeneratorApp</mainClass>
</configuration>
</plugin>
</plugins>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
package org.mskcc.cmo.metadb;
package org.mskcc.smile;

import java.util.concurrent.CountDownLatch;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.mskcc.cmo.messaging.Gateway;
import org.mskcc.cmo.metadb.service.MessageHandlingService;
import org.mskcc.cmo.metadb.service.RequestReplyHandlingService;
import org.mskcc.smile.service.MessageHandlingService;
import org.mskcc.smile.service.RequestReplyHandlingService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication(scanBasePackages = {"org.mskcc.cmo.messaging",
"org.mskcc.cmo.common.*", "org.mskcc.cmo.metadb.*"})
"org.mskcc.smile.commons.*", "org.mskcc.smile.*"})
public class CmoLabelGeneratorApp implements CommandLineRunner {
private static final Log LOG = LogFactory.getLog(CmoLabelGeneratorApp.class);

Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
package org.mskcc.cmo.metadb.service;
package org.mskcc.smile.service;

import java.util.List;
import org.mskcc.cmo.metadb.model.SampleMetadata;
import org.mskcc.cmo.metadb.model.igo.IgoSampleManifest;
import org.mskcc.smile.model.SampleMetadata;
import org.mskcc.smile.model.igo.IgoSampleManifest;

/**
*
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
package org.mskcc.cmo.metadb.service;
package org.mskcc.smile.service;

import org.mskcc.cmo.messaging.Gateway;
import org.mskcc.cmo.metadb.model.SampleMetadata;
import org.mskcc.smile.model.SampleMetadata;

public interface MessageHandlingService {
void initialize(Gateway gateway) throws Exception;
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package org.mskcc.cmo.metadb.service;
package org.mskcc.smile.service;

import org.mskcc.cmo.messaging.Gateway;

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package org.mskcc.cmo.metadb.service.impl;
package org.mskcc.smile.service.impl;

import java.util.Arrays;
import java.util.HashMap;
Expand All @@ -9,14 +9,14 @@
import org.apache.commons.lang3.StringUtils;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.mskcc.cmo.common.enums.CmoSampleClass;
import org.mskcc.cmo.common.enums.NucleicAcid;
import org.mskcc.cmo.common.enums.SampleOrigin;
import org.mskcc.cmo.common.enums.SampleType;
import org.mskcc.cmo.common.enums.SpecimenType;
import org.mskcc.cmo.metadb.model.SampleMetadata;
import org.mskcc.cmo.metadb.model.igo.IgoSampleManifest;
import org.mskcc.cmo.metadb.service.CmoLabelGeneratorService;
import org.mskcc.smile.commons.enums.CmoSampleClass;
import org.mskcc.smile.commons.enums.NucleicAcid;
import org.mskcc.smile.commons.enums.SampleOrigin;
import org.mskcc.smile.commons.enums.SampleType;
import org.mskcc.smile.commons.enums.SpecimenType;
import org.mskcc.smile.model.SampleMetadata;
import org.mskcc.smile.model.igo.IgoSampleManifest;
import org.mskcc.smile.service.CmoLabelGeneratorService;
import org.springframework.stereotype.Service;

/**
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package org.mskcc.cmo.metadb.service.impl;
package org.mskcc.smile.service.impl;

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
Expand All @@ -20,11 +20,11 @@
import org.apache.commons.logging.LogFactory;
import org.mskcc.cmo.messaging.Gateway;
import org.mskcc.cmo.messaging.MessageConsumer;
import org.mskcc.cmo.metadb.model.SampleMetadata;
import org.mskcc.cmo.metadb.model.igo.IgoSampleManifest;
import org.mskcc.cmo.metadb.service.CmoLabelGeneratorService;
import org.mskcc.cmo.metadb.service.MessageHandlingService;
import org.mskcc.cmo.metadb.service.util.RequestStatusLogger;
import org.mskcc.smile.model.SampleMetadata;
import org.mskcc.smile.model.igo.IgoSampleManifest;
import org.mskcc.smile.service.CmoLabelGeneratorService;
import org.mskcc.smile.service.MessageHandlingService;
import org.mskcc.smile.service.util.RequestStatusLogger;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
Expand All @@ -45,7 +45,7 @@ public class LabelGenMessageHandlingServiceImpl implements MessageHandlingServic
@Value("${igo.cmo_sample_label_update_topic}")
private String CMO_LABEL_UPDATE_TOPIC;

@Value("${metadb.sample_update_topic}")
@Value("${smile.sample_update_topic}")
private String IGO_SAMPLE_UPDATE_TOPIC;

@Value("${num.new_request_handler_threads}")
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package org.mskcc.cmo.metadb.service.impl;
package org.mskcc.smile.service.impl;

import com.fasterxml.jackson.databind.ObjectMapper;
import io.nats.client.Message;
Expand All @@ -17,9 +17,9 @@
import org.apache.commons.logging.LogFactory;
import org.mskcc.cmo.messaging.Gateway;
import org.mskcc.cmo.messaging.MessageConsumer;
import org.mskcc.cmo.metadb.model.SampleMetadata;
import org.mskcc.cmo.metadb.service.CmoLabelGeneratorService;
import org.mskcc.cmo.metadb.service.RequestReplyHandlingService;
import org.mskcc.smile.model.SampleMetadata;
import org.mskcc.smile.service.CmoLabelGeneratorService;
import org.mskcc.smile.service.RequestReplyHandlingService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
package org.mskcc.cmo.metadb.service.util;
package org.mskcc.smile.service.util;

import java.io.File;
import java.io.IOException;
import java.time.LocalDate;
import java.time.format.DateTimeFormatter;
import org.apache.commons.lang3.StringUtils;
import org.mskcc.cmo.common.FileUtil;
import org.mskcc.smile.commons.FileUtil;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;
Expand Down Expand Up @@ -34,7 +34,7 @@ public class RequestStatusLogger {
* - CMO_REQUEST_MISSING_REQ_FIELDS: a CMO request with sample metadata
* that is missing required fields (cmoPatientId, baitSet)
* - REQUEST_PARSING_ERROR: json parsing exception thrown
* - CMO_REQUEST_FILTER_SKIPPED_REQUEST: applies if metadb server is running
* - CMO_REQUEST_FILTER_SKIPPED_REQUEST: applies if smile server is running
* with the cmoRequestFilter enabled and a non-cmo request is encountered
* - REQUEST_WITH_FAILED_CMO_LABEL_GENERATION: request contains samples for which a
* CMO label was not successfully generated
Expand Down
Loading

0 comments on commit ea94969

Please sign in to comment.