Support json payloads in MessageFlow

[commi235]

With message flow typically involving a JSON payload, now that we have json process variables it seems like we should support CLOB payloads.

Problem is that we have a legacy from 23.1 of CLOB payloads - although customer adoption of 23.1 message flow is probably limited.

Desired Solution:

• Outbound messages would still send payload as a CLOB. The payload could be built from a process variable which is a CLOB, JSON, or VC2 before sending.

• The Model specifies the process variable containing the payload, which is built into an outbound message in flow_message_util.prepare_message. Currently the variable must be a CLOB. Simplest idea would be to allow the process variable that's sent as the payload to be of types CLOB, JSON, and even VC2. prepare_message could look at the var type, and then use CLOB, JSON or VC2 values to build the outbound message.

• This is simple and the user doesn't need to specify the type - we can intelligently convert at runtime.

• Inbound messages would need to specify the datatype of the variable used to store the payload. Currently this is required (assumed) to be CLOB, but we could allow the message to specify the required type is CLOB, JSON or VC2 (??)

• Extend flow_message_subscriptions with column msub_payload_type. vc2(50) = 'CLOB' | 'JSON' | 'VC2' (??)

• Extend t_subscription_details with payload_type as flow_message_subscriptions.msub_payload_type%type

• when a receive / catch event subscribes to a message, it includes the payload variable type in flow_message_subscription table.

• When a message is received, it is saved by flow_message_util.save_payload. flow_message_util.save_payload would look at the payload_type.

  • if clob, stores as CLOB proc var
  • if json, validates is correct json & stores as JSON proc var
  • if vc2, checks length < 4000 & then stores as vc2 proc var

Modeler would need to add an input item for Payload Variable Type for all message catch events - so:

• Receive Task
• Intermediate Message Catch Event
• Message Boundary Event (Interrupting & Non Interrupting)
• Message Start Event

All current models have to be assumed to be CLOB, but ideally default for new models should be JSON.

Parser would need to parse out Payload Type and store in the key:
objt.objt_attributes."apex"."payloadVariableType"

If we don't want to handle the model migration & UI issues we could implement using extension elements with syntax of
objt.objt_attributes."apex"."extensionElements"."payloadVariableType"

requiring user to add following syntax into the extension elements item for any receiving task / event:
{"payloadVariableType":"JSON"}

Migration Script:

• add column to flow_message_subscriptions
• update all rows in flow_message_subscriptions to have payload variable type CLOB to handle in-flight messages

Model Migration. If modeler default is to be json, we would need to migrate XML on parsing so that any model containing key "apex:payloadVariable" adds a new key "apex:payloadVariableType"="CLOB" immediately below it.

Regression Testing - Add new tests for the new payload types

Open Issues:

• Add VC2 as possible type for send message
• Add VC2 as possible destination type
• default type in modeler = JSON
• Support XML migration or defer to later release
• Allow use of extension elements ??