Image stops being sent to the headset when joining a server with a custom resource pack

[Ryhon0]

When joining a server with server resource packs enabled, the game stops being rendered to the headset and is still being rendered in the window with all the logic working perfectly.
Disabling resource packs for a server fixes this but some servers make heavy use of resource packs and won't let you in without it.
Tested for 1.20.2 and 1.20.4 with Vivecraft 1.1.6 fabric on Linux with SteamVR streamed using ALVR on Linux

[fayer3]

should be fixed, by togling vr once on the server, can be done with right control + F7

[fayer3]

closing as duplicate of #280, because the other one has a bit more info