Linux support

[janoc]

Hello,

Here is an initial attempt at a native (not cross-compiled) Linux support for UE4. It does work to the point that the plugin actually loads and lets me move a cube on the screen using my Razer Hydra in the editor. I have not tested the HMD/RenderManager stuff - it compiles and loads, but I don't have a working HMD set up in Linux at the moment and I have never used it before, so I would prefer someone more knowledgeable about it to test that.

There are a few things that need modifying:

• In the build files, the names of the Linux libraries to link against and to preload may have to be adapted to whatever UE4 ships with. I am referring mainly to SDL and GLEW versions, however OSVR depends also on Boost and a few other things. The plugin may have to include all of these in some 3rdparty folder, because different Linux distros have their own (potentially incompatible) versions. Maybe rely on the SteamOS SDK? This probably needs to be addressed together with Unreal developers, I don't know what their policy for individual platforms is regarding this. I have built the code on my Mageia Linux 5, but I presume the official build is more likely to be something Ubuntu LTS-like.
• The same story with the path to the OSVR header files - this must be changed in OSVR.Build.cs and OSVRInput.Build.cs, right now there is a hardwired path to OSVR on my system - i.e. it won't work for anyone else! Unfortunately, I don't know what to change this to, no idea where UE ships OSVR's headers.
• There is a temporary kludge in OSVR/Private/OSVREntryPoint.cpp - OSVR doesn't implement void osvrClientAttemptServerAutoStart() for Linux for some reason, so I had to add a dummy implementation to that file to make it link. Once this function is available for Linux this kludge needs to be removed.

[janoc]

I have attempted to make the rendering support work as well. It compiles and tries to start in the VR preview mode, but then there is this crash on the display initialization:

Fatal error: [File:/media/u/build/unreal/UnrealEngine/Engine/Source/Runtime/OpenGLDrv/Private/OpenGLDevice.cpp] [Line: 315]
[API][Error][High][1280] GL_INVALID_ENUM error generated. Operation is not valid from the core profile.

It seems it doesn't like something that the RenderManager does. However, the standalone (not Unreal) RenderManager examples work on my machine without issues.

---- Stack trace ----

#0  0x00007ffff7bd1039 in raise () from /lib64/libpthread.so.0
#1  0x00007ffff68e8cc5 in DebugBreak () at /media/u/build/unreal/UnrealEngine/Engine/Source/Runtime/Core/Public/Linux/LinuxPlatformMisc.h:38
#2  FOutputDeviceLinuxError::Serialize (this=0x7ffff7a39ae8 <FLinuxOutputDevices::GetError()::Singleton>, 
    Msg=0x7ffffffd6640 L"Assertion failed: Assertion failed:  [File:/media/u/build/unreal/UnrealEngine/Engine/Source/Runtime/OpenGLDrv/Private/OpenGLDevice.cpp] [Line: 315] \n[API][Error][High][1280] GL_INVALID_ENUM error gene"..., Verbosity=ELogVerbosity::Log, Category=...) at /media/u/build/unreal/UnrealEngine/Engine/Source/Runtime/Core/Private/Linux/LinuxPlatformOutputDevices.cpp:84
#3  0x00007ffff6b7109c in FOutputDevice::Logf (this=0x7ffff7a39ae8 <FLinuxOutputDevices::GetError()::Singleton>, Fmt=0x7ffff758d1f8 L"Assertion failed: %s [File:%s] [Line: %i] \n%s\n")
    at /media/u/build/unreal/UnrealEngine/Engine/Source/Runtime/Core/Private/Misc/OutputDevice.cpp:94
#4  0x00007ffff6ae619f in FDebug::AssertFailed (Expr=0x7ffff756d2e4 "", File=0x7fffccd96535 "/media/u/build/unreal/UnrealEngine/Engine/Source/Runtime/OpenGLDrv/Private/OpenGLDevice.cpp", Line=315, 
    Format=0x7ffffffdc6f0 L"[API][Error][High][1280] GL_INVALID_ENUM error generated. Operation is not valid from the core profile.")
    at /media/u/build/unreal/UnrealEngine/Engine/Source/Runtime/Core/Private/Misc/OutputDevice.cpp:421
#5  0x00007ffff6b738be in FMsg::Logf (File=0x7fffccd96535 "/media/u/build/unreal/UnrealEngine/Engine/Source/Runtime/OpenGLDrv/Private/OpenGLDevice.cpp", Line=315, Category=..., Verbosity=ELogVerbosity::Fatal, 
    Fmt=0x7fffccd939c8 L"[%s][%s][%s][%u] %s") at /media/u/build/unreal/UnrealEngine/Engine/Source/Runtime/Core/Private/Misc/OutputDevice.cpp:588
#6  0x00007fffccb62130 in OpenGLDebugMessageCallbackARB (Source=33350, Type=33356, Id=1280, Severity=37190, Length=78, 
    Message=0x7fffb6520cc0 "GL_INVALID_ENUM error generated. Operation is not valid from the core profile.", UserParam=0x0)
    at /media/u/build/unreal/UnrealEngine/Engine/Source/Runtime/OpenGLDrv/Private/OpenGLDevice.cpp:315
#7  0x00007fffb4df4c38 in ?? () from /usr/lib64/nvidia-current/libnvidia-glcore.so.352.79
#8  0x00007fffb4df4d80 in ?? () from /usr/lib64/nvidia-current/libnvidia-glcore.so.352.79
#9  0x00007fffb4df5042 in ?? () from /usr/lib64/nvidia-current/libnvidia-glcore.so.352.79
#10 0x00007fffb4e1ba83 in ?? () from /usr/lib64/nvidia-current/libnvidia-glcore.so.352.79
#11 0x00007fff5d53e9a7 in glewInit () from /lib64/libGLEW.so.1.11
#12 0x00007fff5e9228d2 in osvr::renderkit::RenderManagerOpenGL::OpenDisplay() () from /media/u/build/OSVR/install/lib64/libosvrRenderManager.so
#13 0x00007fff5e9337d6 in char (anonymous namespace)::osvrRenderManagerOpenDisplayImpl<void*, OSVR_OpenResultsOpenGL>(void*, OSVR_OpenResultsOpenGL*) ()
   from /media/u/build/OSVR/install/lib64/libosvrRenderManager.so
#14 0x00007fff5e9332ba in osvrRenderManagerOpenDisplayOpenGL () from /media/u/build/OSVR/install/lib64/libosvrRenderManager.so
#15 0x00007fff66cba593 in FOpenGLCustomPresent::LazyOpenDisplayImpl (this=0x7fff6bc9cd00) at /home/janoc/src/cpp/unreal/Tracking/Plugins/OSVR/Source/OSVR/Private/OSVRCustomPresentOpenGL.h:360
#16 0x00007fff66cb66ea in FOSVRCustomPresent::LazyOpenDisplay (this=0x7fff6bc9cd00) at /home/janoc/src/cpp/unreal/Tracking/Plugins/OSVR/Source/OSVR/Private/OSVRCustomPresent.h:129
#17 0x00007fff66c8355f in FOSVRHMD::CalculateRenderTargetSize (this=0x7fff7aea3800, Viewport=..., InOutSizeX=@0x7ffffffe252c: 1920, InOutSizeY=@0x7ffffffe2528: 1200)
    at /home/janoc/src/cpp/unreal/Tracking/Plugins/OSVR/Source/OSVR/Private/OSVRRender.cpp:157
#18 0x00007fff66c837da in non-virtual thunk to FOSVRHMD::CalculateRenderTargetSize(FViewport const&, unsigned int&, unsigned int&) ()
    at /home/janoc/src/cpp/unreal/Tracking/Plugins/OSVR/Source/OSVR/Private/OSVRRender.cpp:184
#19 0x00007ffff1fd9a1a in FSceneViewport::InitDynamicRHI (this=0x7fffce742900) at /media/u/build/unreal/UnrealEngine/Engine/Source/Runtime/Engine/Private/Slate/SceneViewport.cpp:1547
#20 0x00007ffff1fde85c in non-virtual thunk to FSceneViewport::InitDynamicRHI() () at /media/u/build/unreal/UnrealEngine/Engine/Source/Runtime/Engine/Private/Slate/SceneViewport.cpp:1647
#21 0x00007fffecd8c5ad in FRenderResource::InitResource (this=0x7fffce742918) at /media/u/build/unreal/UnrealEngine/Engine/Source/Runtime/RenderCore/Private/RenderResource.cpp:28
#22 0x00007ffff2208433 in FSceneViewport::InitResource (this=0x7fffce742900) at Runtime/Engine/Public/Slate/SceneViewport.h:252
#23 0x00007ffff2208bec in non-virtual thunk to FSceneViewport::InitResource() () at Runtime/Engine/Public/Slate/SceneViewport.h:252
#24 0x00007fffecd8f61c in BeginInitResource(FRenderResource*)::EURCMacro_InitCommand::DoTask(ENamedThreads::Type, TRefCountPtr<FGraphEvent> const&) (this=0x7ffffffe31e0, CurrentThread=ENamedThreads::GameThread, 
    MyCompletionGraphEvent=...) at /media/u/build/unreal/UnrealEngine/Engine/Source/Runtime/RenderCore/Private/RenderResource.cpp:124
#25 0x00007fffecd8e445 in BeginInitResource (Resource=0x7fffce742918) at /media/u/build/unreal/UnrealEngine/Engine/Source/Runtime/RenderCore/Private/RenderResource.cpp:124
#26 0x00007ffff1fd151b in FSceneViewport::UpdateViewportRHI (this=0x7fffce742900, bDestroyed=false, NewSizeX=1920, NewSizeY=1200, NewWindowMode=EWindowMode::Windowed)
    at /media/u/build/unreal/UnrealEngine/Engine/Source/Runtime/Engine/Private/Slate/SceneViewport.cpp:1305
#27 0x00007ffff1fd5277 in FSceneViewport::EnqueueBeginRenderFrame (this=0x7fffce742900) at /media/u/build/unreal/UnrealEngine/Engine/Source/Runtime/Engine/Private/Slate/SceneViewport.cpp:1361
#28 0x00007ffff1fd864c in non-virtual thunk to FSceneViewport::EnqueueBeginRenderFrame() () at /media/u/build/unreal/UnrealEngine/Engine/Source/Runtime/Engine/Private/Slate/SceneViewport.cpp:1409
#29 0x00007ffff26e3756 in FViewport::Draw (this=0x7fffce742908, bShouldPresent=true) at /media/u/build/unreal/UnrealEngine/Engine/Source/Runtime/Engine/Private/UnrealClient.cpp:1132
#30 0x00007ffff1fd09a7 in FSceneViewport::ResizeViewport (this=0x7fffce742900, NewSizeX=1920, NewSizeY=1200, NewWindowMode=EWindowMode::Windowed)
    at /media/u/build/unreal/UnrealEngine/Engine/Source/Runtime/Engine/Private/Slate/SceneViewport.cpp:1221
#31 0x00007ffff1fd017e in FSceneViewport::SetViewportSize (this=0x7fffce742900, NewViewportSizeX=1920, NewViewportSizeY=1200)
    at /media/u/build/unreal/UnrealEngine/Engine/Source/Runtime/Engine/Private/Slate/SceneViewport.cpp:1180
#32 0x00007fff66c7c0d2 in FOSVRHMD::EnableStereo (this=0x7fff7aea3800, bStereo=true) at /home/janoc/src/cpp/unreal/Tracking/Plugins/OSVR/Source/OSVR/Private/OSVRHMD.cpp:575
#33 0x00007fff66c7c15c in non-virtual thunk to FOSVRHMD::EnableStereo(bool) () at /home/janoc/src/cpp/unreal/Tracking/Plugins/OSVR/Source/OSVR/Private/OSVRHMD.cpp:599
#34 0x00007fffe913355d in UEditorEngine::CreatePIEGameInstance (this=0x7fff6bcac000, InPIEInstance=0, bInSimulateInEditor=false, bAnyBlueprintErrors=false, bStartInSpectatorMode=false, bRunAsDedicated=false, 
    PIEStartTime=82804.2188) at /media/u/build/unreal/UnrealEngine/Engine/Source/Editor/UnrealEd/Private/PlayLevel.cpp:3282
#35 0x00007fffe9123e29 in UEditorEngine::PlayInEditor (this=0x7fff6bcac000, InWorld=0x7fff4d78cc00, bInSimulateInEditor=false)
    at /media/u/build/unreal/UnrealEngine/Engine/Source/Editor/UnrealEd/Private/PlayLevel.cpp:2424
#36 0x00007fffe90fbbee in UEditorEngine::StartQueuedPlayMapRequest (this=0x7fff6bcac000) at /media/u/build/unreal/UnrealEngine/Engine/Source/Editor/UnrealEd/Private/PlayLevel.cpp:1135
#37 0x00007fffe8249164 in UEditorEngine::Tick (this=0x7fff6bcac000, DeltaSeconds=0.0193545111, bIdleMode=false) at /media/u/build/unreal/UnrealEngine/Engine/Source/Editor/UnrealEd/Private/EditorEngine.cpp:1306
#38 0x00007fffe998aaf1 in UUnrealEdEngine::Tick (this=0x7fff6bcac000, DeltaSeconds=0.0193545111, bIdleMode=false) at /media/u/build/unreal/UnrealEngine/Engine/Source/Editor/UnrealEd/Private/UnrealEdEngine.cpp:368
#39 0x0000000000461d04 in FEngineLoop::Tick (this=0x6f9c30 <GEngineLoop>) at /media/u/build/unreal/UnrealEngine/Engine/Source/Runtime/Launch/Private/LaunchEngineLoop.cpp:2828
#40 0x000000000044a1a3 in EngineTick () at /media/u/build/unreal/UnrealEngine/Engine/Source/Runtime/Launch/Private/Launch.cpp:52
#41 0x000000000047ceb7 in GuardedMain (CmdLine=0x7fffd1f94f20 L" -nocore") at /media/u/build/unreal/UnrealEngine/Engine/Source/Runtime/Launch/Private/Launch.cpp:158
#42 0x00007fffe6100482 in CommonLinuxMain (argc=2, argv=0x7fffffffcea8, RealMain=0x479ff0 <GuardedMain(wchar_t const*)>)
    at /media/u/build/unreal/UnrealEngine/Engine/Source/Runtime/Linux/LinuxCommonStartup/Private/LinuxCommonStartup.cpp:230
#43 0x00000000004a23bc in main (argc=2, argv=0x7fffffffcea8) at /media/u/build/unreal/UnrealEngine/Engine/Source/Runtime/Launch/Private/Linux/LaunchLinux.cpp:24

[janoc]

OK, it seems that the crash above is apparently a known problem with GLEW and Core profile (which Unreal seems to be using):

nigels-com/glew#3
Time to move away from GLEW?

This thread has some alternatives:
https://www.reddit.com/r/opengl/comments/3m28x1/glew_vs_glad/

[russell-taylor]

We work around an issue with Glew issuing spurious errors here: https://github.com/sensics/OSVR-RenderManager/blob/master/osvr/RenderKit/RenderManagerOpenGL.cpp#L650

Does this solve the problem?

[janoc]

Hello Russell,

No that doesn't actually solve the issue. That thread I have linked to actually speaks about that too. What the glewExperimental = true does is that it only let's GLEW try to fill function pointers for all extensions that are present in the platform header files. It doesn't do anything about GLEW attempting to use a function not allowed in Core profile.

The RenderManager examples are likely not affected because they are using Compatibility profile by default, where the function is only deprecated.

I am not quite sure why the error isn't cleared by calling glGetError(), though. Could be driver behaviour difference/bug or it is one of the later commands in that function triggering it (there doesn't seem to be anything suspicious, though), no clue there.

Edit: Just noticed that the crash actually happens before the glGetError() even gets called, directly in glewInit(). It seems that UE installs an error handler (using this: https://www.opengl.org/registry/specs/ARB/debug_output.txt , see stack frame 6) which gets triggered right away, before the error has a chance to get cleared by glGetError(). So the workaround has no chance to work.