PACS Event-Driven Niffler Integration for Real-Time DICOM Retrieval

[mahmoudnasser1561]

I want to propose PACS event-driven approach for integrating Niffler with Orthanc (PACS) to enable real-time DICOM retrieval using a C-MOVE workflow.

current :

I have designed the following architecture to implement this:

I also think of adding an Anonymization Service
to automatically remove or encrypt patient identifiers based on configurable rules and access policies
any thoughts on that ?

I'm Seeking Feedback On:

Are there any potential bottlenecks or failure points in the current design?
Would it be better to use an event queue (e.g., Kafka, RabbitMQ) ?
is there anything missing or not here ?

please @pradeeban @mdxabu
I’d love to hear your thoughts on this approach and whether any improvements could be made ?

Thanks!

[pradeeban]

Event queue is good, configuring with a DICOM receiver such as Orthanc.
There is an anonymization module in Niffler, you could use, if you need.

[mahmoudnasser1561]

@pradeeban
I have a question sir Pradeeban regarding the concept of "dynamic routing" in our project. Could you please clarify its exact nature? Specifically, does it mean that the PACS endpoints are predefined, and we are distributing DICOM data among them based on load/performance? Or does it involve discovering PACS endpoints dynamically, tracking them, and then managing the distribution accordingly?

Understanding this will help me choose the right architectural components for implementation.

Thanks!

[pradeeban]

It implies the PACS endpoints are dynamically detected. However, I am going to say your first interpretation of existing known/static DICOM endpoints is a more realistic target to begin with. Because, especially as clinical systems, detecting DICOM endpoints on the go maybe difficult (against/violates the DICOM standard) and lacks real use case. We could research on it. Maybe list that as a research activity and "Implementation: optional, if time permits"

[mahmoudnasser1561]

yes exactly that what I was thinking.
That approach makes a lot of sense. Having a list of known or predefined PACS endpoints in the gateway gives us more control and flexibility, especially from a security perspective.
with that said I'm thinking of configuring a firewall between the gateway the MRI source this could add another level of security
is that practical in our context sir @pradeeban ?

[pradeeban]

You should assume that a firewall already exists (since that is the realistic deployment scenario).

[mahmoudnasser1561]

okay we are moving forward!
regarding the routing in the gateway

as I have for now some metrics like

• Network Latency
• Storage Availability
• queue size
  is it the queue of tasks being processed on PACS (CPU utilization) ?
  and also could we consider Urgency of the Image this would require some processing to decide ?

[pradeeban]

The queue is mostly just a queue of DICOM series waiting to be delivered to the destination PACS server.

That could be possible to consider urgency. But I am not sure whether we have a real use case for that beyond research curiosity.

[mahmoudnasser1561]

I'm also thinking of replacing PACS event listener with Kafka I think this simplified approach would give us more freedom to introduce more optional features and instead of reinventing the wheel

I think Kafka will handle that very well
any thoughts sir @pradeeban ?

[pradeeban]

That sounds good.

[mahmoudnasser1561]

Thank you sir @pradeeban
your guidance is appreciated.

[mahmoudnasser1561]

sir @pradeeban
Please have a look sir Pradeeban

I have built demo1
also tried to do demo2

are those valid for the code challenge ?

Thanks!

[pradeeban]

Looks neat. @anbhimi, do you have any comments?

[anbhimi]

Great Work! The documentation is thorough. Reviewing the code..

[anbhimi]

Both the Idea and implementation are wonderful. Please attach all this work while preparing the proposal.

[mahmoudnasser1561]

Thank you sir!
I'm so grateful for your positive feedback.

[mahmoudnasser1561]

Hi sir @pradeeban,

I'm currently researching the Gateway component for our project.
with factors like queue depth, server load, and network conditions.

Are we planning to use an existing open-source component (such as HAProxy, Orthanc with plugins, or NGINX) for this, or
Are we considering building a new one ?

[anbhimi]

Hi, Using a stable open-source gateway component should be good for solving the problem at hand. If you are interested in working further, you can take it as an optional task.

[pradeeban]

Yes, Kong API gateway is a good choice. It is built on Nginx with additional features.

[mahmoudnasser1561]

Thanks for response! That sounds good, I will consider that.

[mahmoudnasser1561]

sir @pradeeban,
sir @anbhimi

I am currently in the phase of doing research and writing Proposal

I have documented a full working workflow architecture
like an approach to solve the problem

I’d appreciate your feedback on the current status to ensure I’m on the right track and can move forward with confidence

Would it be okay to send my current proposal as an email attachment for your review?

[pradeeban]

ok, sure.

[mahmoudnasser1561]

Hello
@pradeeban
@anbhimi
when creating my proposal
do I need to include my recent contributions to Beehive
or just Ignore them as we are going to work on a different project ?

[pradeeban]

Definitely include, with links. You could also include your other open source contributions.