Analyzing desynchronizations between players
This article is supposed to help with analyzing desynchronization bugs. These bugs may result in:
- game crashes due to "Random check mismatch"
- replay inconsistencies, which break such replays
It's in work-in-progress state right now, so I will keep updating it, as I understand more.
The code responsible for game's flow of events is included in
- src\game\gip\KM_GameInputProcess_Multi.pas
- src\game\KM_Game.pas
- src\net\KM_NetClientOverbyte.pas
There is a tool called ParallelRunner, used for automatically running multiple games between AI players, in order to catch most obvious desynchronization bugs.
TODO write how to compile, run it
It can be very useful, to check which commands were ran, and in which order, for a specific player. You need to get a replay/report from the player whose game crashed, but also from a player who was the one that had a mismatch.
TODO - how to see it in game? in logs?
To export player command from replay, you can use:
Export data ->
- Random checks
- GIC from rpl
Those events can also be logged (see: F11 -> Logs)
After exporting data, you can see what kind of commands were issued:
legend:
[CommandName]
tick_number random_number command_type hand_id data data data ...
[gicTempDoNothing]
17403 1244675611 40h 03h 0 0 0 0 0 0 0 0 0 0 0 0
[gicGameAutoSave]
17403 0 2Fh 03h 236 81 145 120 251 195 229 64 0 0 0 0
[gicGamePlayerChange]
17406 0 36h 00h 3 0 0 0 0 0 0 0 0 0 0 0
[gicGamePlayerChange]
17406 0 36h 00h 75 105 110 103 104 116 115 32 68 122 97 112
Now we need to check, what happened right before the tick mentioned on screenshot:
In this example, gicGameAutoSave was the command issued on tick 17403. So, it is likely that this command was not issued by player who had the crash report.
Replay inconsistency will not crash (or does it only do that with a debug build?), so to find out the tick number (through bisecting), use F11 -> "Pause before tick".