Mediator doesn't enable mover successfully with high mover counts

[netbymatt]

Using a system in simulation, XTS 4.2.0, and tasks divided among 8 cores of a CX2072 the following happens during enable. After the detection process starts all the movers go into error because of the axes lost enable while a command was executing. No commands should be executing at this time because the mover state machine waits for several flags from the XPU/Mediator to confirm everything is ready and there's a hard-coded 500ms delay as well.

MSG | 1/2/2025 9:07:55 AM 953 ms | 'MAIN.XTS.Mover[116]': XTS_Base.Main.MAIN.XTS.Mover[116] Enable command issued. Source: Direct access
MSG | 1/2/2025 9:07:55 AM 953 ms | 'MAIN.XTS.Mover[117]': XTS_Base.Main.MAIN.XTS.Mover[117] Enable command issued. Source: Direct access
MSG | 1/2/2025 9:07:55 AM 953 ms | 'MAIN.XTS.Mover[118]': XTS_Base.Main.MAIN.XTS.Mover[118] Enable command issued. Source: Direct access
MSG | 1/2/2025 9:07:55 AM 953 ms | 'MAIN.XTS.Mover[119]': XTS_Base.Main.MAIN.XTS.Mover[119] Enable command issued. Source: Direct access
MSG | 1/2/2025 9:07:55 AM 953 ms | 'MAIN.XTS.Mover[120]': XTS_Base.Main.MAIN.XTS.Mover[120] Enable command issued. Source: Direct access
MSG | 1/2/2025 9:07:55 AM 963 ms | XtsProcessingUnit 1 (0x01010010): Redetection is triggered.
MSG | 1/2/2025 9:07:55 AM 964 ms | XtsProcessingUnit 1 (0x01010010): All XTS mover(s) are detected.
MSG | 1/2/2025 9:07:55 AM 966 ms | XtsProcessingUnit 1 (0x01010010): The Mover ID Detection has been triggered.
MSG | 1/2/2025 9:07:55 AM 966 ms | XtsProcessingUnit 1 (0x01010010): Mover ID is detected.
ERR | 1/2/2025 9:07:56 AM 709 ms | 'TCNC' (500): 'Mover Axis 1' (Axis-ID: 1, Grp-ID: 1): The axis or a coupled slave axis has lost its controller enable signal while executing a command => error 0x4260 (StateDWord: 0x2104105, CoupleState: 0, ActPos: 16250.192906, ActVelo: 1.364294)
ERR | 1/2/2025 9:07:56 AM 709 ms | 'TCNC' (500): 'Mover Axis 2' (Axis-ID: 2, Grp-ID: 2): The axis or a coupled slave axis has lost its controller enable signal while executing a command => error 0x4260 (StateDWord: 0x2104105, CoupleState: 0, ActPos: 16339.995314, ActVelo: 2.377404)
ERR | 1/2/2025 9:07:56 AM 709 ms | 'TCNC' (500): 'Mover Axis 3' (Axis-ID: 3, Grp-ID: 3): The axis or a coupled slave axis has lost its controller enable signal while executing a command => error 0x4260 (StateDWord: 0x2104105, CoupleState: 0, ActPos: 16430.012570, ActVelo: 6.233132)
ERR | 1/2/2025 9:07:56 AM 709 ms | 'TCNC' (500): 'Mover Axis 4' (Axis-ID: 4, Grp-ID: 4): The axis or a coupled slave axis has lost its controller enable signal while executing a command => error 0x4260 (StateDWord: 0x2104105, CoupleState: 0, ActPos: 16520.015053, ActVelo: 0.481074)

It is currently unclear if this is an issue related to the 4.x update to XTS or the new redetection steps in the mediator that were added.

[netbymatt]

This has been tracked down to a PLC cycle time overrun during the group reset and enable calls. With ~120 movers 5-6 back-to-back cycle overruns occur at the default 2ms PLC tick from the base code. This causes the NC to go into watchdog alarm and remove the enable from the axes.

The workaround is to increase the PLC cycle time to 3 or 4 ms.