How to filter out duplicate message in mirth

[Poodeshwaran]

How to filter out duplicate message in mirth

[pacmano1]

Typically by creating a hash of each message and storing it, then comparing new messages to the stored value. That storage may be a database or some map in mirth, it depends on your particular problem.

[Poodeshwaran]

Since i am new to Mirth. Kindly share any examples or steps to be followed.
Thanks for your support

[mjtrac]

Here's minimal code that will filter out duplicates by creating a hash map in your Deployment script and checking it in a source filter. You'd have to preserve the hash map across deployments, or release it upon undeploy, and you'd probably want a way to age out old message control ids.

In your deployment script (Edit channel, Scripts tab):

globalChannelMap.put('msh10s', new java.util.HashMap());
return;

In your source filter (Edit channel, Source tab, Create a filter and set Type to Javascript in dropdown):

var msh10s = globalChannelMap.get('msh10s');
var incoming_msh10 = msg['MSH']['MSH.10']['MSH.10.1'].toString();
// if incoming not in hash map, add it, pass message
if ( ! msh10s.containsKey(incoming_msh10) ){
	msh10s.put(incoming_msh10, 1);
	return true;
}
// otherwise, filter duplicate message
return false;

This assumes that your intention is to reject a message arriving with an MSH.10.1 (Message Control ID) that the channel has already seen during its current deployment. It checks each incoming message's MSH.10.1 value to see if it is a key value for the hash map, and filters the message if its MSH.10.1 value is found, or adds that value to the hash map if it's not found, and allows the message to continue processing.

[pacmano1]

It would be helpful for @Poodeshwaran to share their definiton of a duplicate message.

@mjtrac 's code assumes MSH-10.1 can't restart due to bad design on the other side sending the message - which is bad practice of course. Also a sender might send a message with a new MSH-10.1, but the relevant contents of the message are a duplicate from the consumer's perspective.

Also, as @mjtrac noted at the top - you need to purge values based on time and you also might time window the check, if you don't do the former you are putting every MSH-10.1 in there forever which eventually will exhaust memory.

[mjtrac]

Coming from some experience at an HIE, I can only say that you can do what you can do. If someone sent a new MSH.10 with duplicate information, I'd have considered it my responsibility to forward that duplicate; a new MSH.10 says its new. If someone sent an MSH.10 that I'd already seen, and it was a message that had new information, I'd have felt fine telling them "don't do that." I really wish the standards were enforced. Of course, I wish a lot, but imagine trying to work with a database table where you couldn't rely on the primary key being a unique identifier. At some point, no matter how nice you want to be, you have to require technical constraints if you want things to work. Just my two cents.