Enable Oculus Fixed Foveated Rendering #654

kearwood · 2018-10-22T23:09:00Z

A value of 2 was selected through experimentation on the Oculus Go. (0 = none, 3 = max).

Ideally, this would be a user-select-able value; however, a default of 2 seems reasonable as I could not notice any artifacts in the peripheral vision.

MortimerGoro · 2018-10-23T09:29:12Z

app/src/oculusvr/cpp/DeviceDelegateOculusVR.cpp

+    // Enable fixed-foveated rendering.
+    // TODO - This should be user configurable. VRAPI_SYS_PROP_FOVEATION_AVAILABLE
+    //        can be checked to determine if the option should be presented.
+    vrapi_SetPropertyInt(&java, VRAPI_FOVEATION_LEVEL, 2);


I think it's better to enable this only for the chrome scene. You can do that viaDeviceDelegateOculusVR::SetRenderMode()

According to the spec foveated rendering for WebVR should be up to the JS code right? via WebGL extensions or WebXR when available.

Agree, we should probably have this preference affect the browser's 2d modes separately.

There is no defined API for setting foveated level in the WebXR spec yet. Unlike MRS (Mixed resolution shading) and LMS (Lens Matched Shading), this kind of fixed-foveated-rendering does not need changes to the content to function. Turning it on manually for immersive content could make the difference between being able to see some immersive sites and it not working. Perhaps we should give the user choice in this case.

MortimerGoro · 2018-10-23T09:29:55Z

app/src/oculusvr/cpp/DeviceDelegateOculusVR.cpp

@@ -172,6 +172,10 @@ struct DeviceDelegateOculusVR::State {
    vrapi_SetPropertyInt(&java, VRAPI_BLOCK_REMOTE_BUTTONS_WHEN_NOT_EMULATING_HMT, 0);
    // Reorient the headset after controller recenter.
    vrapi_SetPropertyInt(&java, VRAPI_REORIENT_HMD_ON_CONTROLLER_RECENTER, 1);
+    // Enable fixed-foveated rendering.
+    // TODO - This should be user configurable. VRAPI_SYS_PROP_FOVEATION_AVAILABLE


Check VRAPI_SYS_PROP_FOVEATION_AVAILABLE before calling vrapi_SetPropertyInt(&java, VRAPI_FOVEATION_LEVEL, 2);

Initially (my earlier patch), I was attempting to check VRAPI_SYS_PROP_FOVEATION_AVAILABLE before enabling the UI, but it would require many more changes (eg, getting the value asynchronously from the render thread while building the UI). Setting this value when it is not supported will simply be a NULL operation. I've hidden the UI when not using the Oculus API, but perhaps we could enhance this at a later time, together with better adaptive UI selections for other rendering features?

I imagine the only case right now where VRAPI_SYS_PROP_FOVEATION_AVAILABLE would return false would be for GearVR / clip-in headsets using the Oculus API. In those cases with the current patch, the option would appear but not have any effect.

cvan

nice work on adding this 👍

it's quite possible my eyes aren't trained well enough, but I can't discern any visual difference with two Oculus Go headsets side by side. what should I notice? what are good examples of objects to look at or any test cases to assess the difference?

cvan · 2018-10-25T20:39:34Z

app/src/main/res/values/strings.xml

+    <!-- This string is used to label the Foveated radio button that disables fixed-foveated-rendering in immersive mode. -->
+    <string name="developer_options_foveated_disabled">Disabled</string>
+    <!-- This string is used to label the Foveated radio button that enables low-level fixed-foveated-rendering. -->
+    <string name="developer_options_foveated_1">1</string>


could we call these Low, Medium, and High?

I can only really notice the difference between #2 and #3. With #3, you can see the edges having a slightly lower resolution when you move the headset so something with contrast is near the top or sides.

I originally wanted to use "low", "medium", and "high" but the longer strings broke the UI causing overlaps. Perhaps we could improve this later with some changes to the developer options panel layout?

cvan · 2018-10-25T20:39:48Z

app/src/main/res/values/strings.xml

@@ -102,6 +102,16 @@
    <string name="developer_options_msaa_2">2x</string>
    <!-- This string is used to label the MSAA radio button that enables four times MSAA in immersive mode. -->
    <string name="developer_options_msaa_4">4x</string>
+    <!-- This string is used to label radio buttons for setting fixed-foveated-rendering -->
+    <string name="developer_options_foveated">Foveated Level</string>


can we add a note that explains in a little bit more detail what this means. I assume this string will be difficult to translate for most.

I've added some more comments. I hope it helps!

I suspected "foveated" would be a non-localized term.

cvan · 2018-10-25T20:44:17Z

app/src/noapi/cpp/DeviceDelegateNoAPI.cpp

+
+int32_t
+DeviceDelegateNoAPI::GetMaxFoveatedLevel() {
+  // Return 0 to indicate that it is not supported


can add puncutation to these so it's similar to the punctation in the comments elsewhere.

I've removed the GetMaxFoveatedLevel() related code and replaced it with a simple check for the Oculus build in DeveloperOptionsWidget.java. My original implementation was not thread safe, and would take a lot longer to fix. I'd like to see a better mechanism for selectively enabling platform specific UI in the options widget.

cvan · 2018-10-25T20:45:09Z

app/src/oculusvr/cpp/DeviceDelegateOculusVR.cpp

+  if (available == VRAPI_TRUE) {
+    return 3;
+  }
+  return 0;


curious: why do we return 0 instead of available? what if it's 1 or 2?

This code as been removed. It wasn't clear how we plan to make the options dialogue dynamic to support other headsets that also support foveated rendering, so I opted for a simpler solution for now in DeveloperOptionsWidget.java

bluemarvin · 2018-10-25T21:45:14Z

app/src/common/shared/org/mozilla/vrbrowser/ui/widgets/DeveloperOptionsWidget.java

@@ -127,6 +128,15 @@ public void onClick(View view) {
        mMSAARadio.setOnCheckedChangeListener(mMSSAChangeListener);
        setMSAAMode(mMSAARadio.getIdForValue(msaaLevel), false);

+        int foveatedLevel = SettingsStore.getInstance(getContext()).getFoveatedLevel();
+        mFoveatedRadio = findViewById(R.id.foveated_radio);
+        if (!true) { // FINDME!! KIP!! HACK!!


This should be:
if (BuildConfig.FLAVOR_platform != "oculusvr") {

- Option to change value added to the developer settings UI. - Default with a value of 2 (0 = none, 3 = max).

kearwood · 2018-10-26T22:09:45Z

Sorry for the multiple pushes.. This final one seems to be working well. I've tested on Oculus Go and HTC Vive Focus. (On the Vive Focus, the foveated option disappears as intended)

MortimerGoro · 2019-02-14T17:26:01Z

@kearwood could you rebase this PR with master? The other change required is disable/enable foveated when the user starts/exits a WebVR session. You could do that in SetRenderMode() based on the device::RenderMode::StandAlone or device::RenderMode::Immersive parameter

philip-lamb · 2019-03-13T04:24:03Z

@kearwood It'd be great to get this into 1.1.3.

MortimerGoro · 2019-03-29T11:00:21Z

@kearwood Do you have time to work on this? I can work in the remaining bits if you want

philip-lamb · 2019-04-08T03:00:36Z

The other change required is disable/enable foveated when the user starts/exits a WebVR session. You could do that in SetRenderMode() based on the device::RenderMode::StandAlone or device::RenderMode::Immersive parameter

I'm still not clear on why we'd not just apply fixed foveation mode all the time?

MortimerGoro · 2019-04-08T09:38:13Z

I'm still not clear on why we'd not just apply fixed foveation mode all the time?

The WebXR spec says: implementations should not enable foveated rendering or performance optimizations that may negatively affect the correctness of an application unless the application explicitly opts-in to that behavior.

philip-lamb · 2019-04-08T10:18:37Z

The WebXR spec says: implementations should not enable foveated rendering or performance optimizations that may negatively affect the correctness of an application unless the application explicitly opts-in to that behavior.

True, except that we don't have WebXR yet, and according to @kearwood above, there is no defined API for setting foveated level in the WebXR spec yet. I think we should make the choice in favour of performance now and offer the user an option to enable/disable when WebXR lands.

MortimerGoro · 2019-04-08T16:58:41Z

PR moved to #1064

larsbergstrom added the in progress label Oct 22, 2018

kearwood requested a review from MortimerGoro October 22, 2018 23:49

MortimerGoro suggested changes Oct 23, 2018

View reviewed changes

kearwood force-pushed the oculus-fixed-foveated-rendering branch 5 times, most recently from b05e3b1 to d6a20ba Compare October 25, 2018 00:49

cvan reviewed Oct 25, 2018

View reviewed changes

bluemarvin reviewed Oct 25, 2018

View reviewed changes

kearwood force-pushed the oculus-fixed-foveated-rendering branch 3 times, most recently from ade3aa2 to 9bcc906 Compare October 26, 2018 21:49

Enable Oculus Fixed Foveated Rendering

19418e0

- Option to change value added to the developer settings UI. - Default with a value of 2 (0 = none, 3 = max).

kearwood force-pushed the oculus-fixed-foveated-rendering branch from 9bcc906 to 19418e0 Compare October 26, 2018 22:03

philip-lamb assigned kearwood Feb 19, 2019

philip-lamb added P2 Fix in the next development iteration P1 Fix in the current development iteration and removed P2 Fix in the next development iteration labels Feb 19, 2019

philip-lamb added this to the v1.1.3 milestone Mar 13, 2019

philip-lamb added platform:Oculus Issues specific to the Oculus VR platform perf Performance and removed P1 Fix in the current development iteration labels Mar 13, 2019

philip-lamb modified the milestones: v1.1.3, v1.1.4 Mar 18, 2019

MortimerGoro self-assigned this Apr 8, 2019

MortimerGoro closed this Apr 8, 2019

philip-lamb removed the in progress label Apr 8, 2019

cvan modified the milestones: v1.1.4, v1.1.3 Apr 15, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable Oculus Fixed Foveated Rendering #654

Enable Oculus Fixed Foveated Rendering #654

kearwood commented Oct 22, 2018

MortimerGoro Oct 23, 2018

kearwood Oct 31, 2018

MortimerGoro Oct 23, 2018

kearwood Oct 31, 2018

kearwood Oct 31, 2018

cvan left a comment

cvan Oct 25, 2018

kearwood Oct 26, 2018

cvan Oct 25, 2018

kearwood Oct 26, 2018

cvan Oct 25, 2018

kearwood Oct 26, 2018

cvan Oct 25, 2018

kearwood Oct 26, 2018

bluemarvin Oct 25, 2018

kearwood commented Oct 26, 2018

MortimerGoro commented Feb 14, 2019

philip-lamb commented Mar 13, 2019

MortimerGoro commented Mar 29, 2019

philip-lamb commented Apr 8, 2019

MortimerGoro commented Apr 8, 2019

philip-lamb commented Apr 8, 2019

MortimerGoro commented Apr 8, 2019

Enable Oculus Fixed Foveated Rendering #654

Enable Oculus Fixed Foveated Rendering #654

Conversation

kearwood commented Oct 22, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cvan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kearwood commented Oct 26, 2018

MortimerGoro commented Feb 14, 2019

philip-lamb commented Mar 13, 2019

MortimerGoro commented Mar 29, 2019

philip-lamb commented Apr 8, 2019

MortimerGoro commented Apr 8, 2019

philip-lamb commented Apr 8, 2019

MortimerGoro commented Apr 8, 2019