Skip to content

Commit

Permalink
Fixes to the ET section.
Browse files Browse the repository at this point in the history
  • Loading branch information
mbucchia committed Nov 6, 2023
1 parent 4709cca commit 24a0080
Show file tree
Hide file tree
Showing 14 changed files with 87 additions and 40 deletions.
6 changes: 3 additions & 3 deletions FAQ.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,23 +22,23 @@ The goal of the OpenXR Toolkit is to bring innovative features to the community

## Q: What headset does the OpenXR Toolkit work with?

It works with any VR headset thanks to OpenXR. See [index#supported-headsets](supported headsets).
It works with any VR headset thanks to OpenXR. See [supported headsets](index#supported-headsets).

## Q: What GPUs does the OpenXR Toolkit work with?

Any GPU that is compatible DirectX 11.

Yes, even the NVIDIA Image Scaling (NIS) will work on non-NVIDIA cards, and AMD's FSR will work on non-AMD cards.

Foveated rendering requires additional support, see See [index#supported-graphics-cards](supported graphics cards).
Foveated rendering requires additional support, see See [supported graphics cards](index#supported-graphics-cards).

## Q: Does the OpenXR Toolkit work with OpenGL or Vulkan.

No.

## Q: Will the OpenXR Toolkit work with other games like the ones from Steam?

This software only works with OpenXR applications, not OpenVR not OVR applications. See [index#supported-applications](supported applications).
This software only works with OpenXR applications, not OpenVR not OVR applications. See [supported applications](index#supported-applications)).

## Q: Do I need to uninstall the OpenXR Toolkit if a game is not compatible?

Expand Down
119 changes: 83 additions & 36 deletions et.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,96 +25,143 @@ The use of Eye Tracking (ET) in conjunction with [Foveated Rendering](fr) (FR) c

**In addition to the requirements for your graphics card and headset, it must be possible for OpenXR Toolkit to distinguish when the application is rendering the left and right views. This is not possible for all applications, and therefore eye tracking might not be offered for all applications.** When not available, Fixed Foveated Rendering will still be offered as a fallback.

Below is a table of games tested with eye-tracked foveated rendering:
Below is a (non-exhaustive) table of games tested with eye-tracked foveated rendering:

| Game | Note |
| --- | --- |
| Assetto Corsa Competizione | Requires [OpenComposite](opencomposite) |
| Contractors | Requires to add `-hmd=openxr` to the command-line |
| Assetto Corsa Competizione | Requires OpenComposite |
| Contractors | Must start the game in OpenXR mode (`-hmd=openxr` parameter) |
| Fallout 4 VR | Requires OpenComposite |
| Ghosts of Tabor | |
| Hubris | |
| iRacing | Requires OpenXR mode |
| iRacing | Must start the game in OpenXR mode |
| Microsoft Flight Simulator 2020 | |
| Pavlov VR (Beta) | |
| Praydog's UE4VR Injector | Mod is not yet released |
| The Elder Scrolls V: Skyrim VR | Requires OpenComposite |

Most Unreal Engine games using OpenXR directly (not OpenVR) should work, since Unreal Engine uses a rendering technique that allows left and right eye to be identified at rendering time.

### Considerations for Varjo
For Digital Combat Simulator, check out [Quad-Views-Foveated](https://github.com/mbucchia/Quad-Views-Foveated/wiki) as an alternative to OpenXR Toolkit.

**Varjo supports eye tracking for PC applications when using Varjo as the OpenXR runtime. If you are using SteamVR as the OpenXR runtime, this runtime cannot support eye tracking for PC applications.**
### Considerations for Quest Pro

You must toggle the _Allow eye tracking_ option from the _System_ tab of the Varjo Base software:
#### Using Oculus OpenXR with Oculus Link

<details>
<summary>Click for details...</summary>

<img alt="Allow ET" src="site/varjo-et.png"><br>
<b>This option must be enabled for OpenXR Toolkit to access the eye tracking data for its own implementation of Foveated Rendering.</b>
<p>- Select Oculus as your OpenXR runtime.<br>
<img src="site/oculus-openxr-1.jpg"><br>
<img src="site/oculus-openxr-2.png"></p>

</details>
<p><b>- YOU MUST CREATE A <a href="https://developer.oculus.com/sign-up/">DEVELOPER ACCOUNT</a>.</b></p>

### Considerations for HP G2 Omnicept
<p>- Open the <i>Oculus</i> application on your PC, then under 'Settings' -> 'Beta' make sure both 'Developer Runtime Features' and 'Eye tracking over Oculus Link' are toggled on.<br>
<img src="site/et-oculus-1.png"></p>

In order to use the eye tracker on your Omnicept device, you must first install the [HP Omnicept Runtime](https://developers.hp.com/omnicept/downloads/hp-omnicept-runtime).
<p>- Open the <i>Meta Quest</i> app on your phone, then under 'Headset Settings' -> 'Developer Mode' make sure to 'Developer Mode' is toggled on.<br>
<img src="site/et-oculus-2.png"></p>

Once installed, please run the HP Omnicept eye calibration application.
<p>- Ensure that eye tracking is enabled on your Quest Pro, and you have performed calibration. These settings can be found under 'Settings' -> 'Movement tracking'.<br>
<img src="site/et-oculus-3.png"></p>
</details>

You must then allow applications to use the eye tracker. You can enable all applications by disabling the _Require user approval for Omnicept clients using sensor data_ option under the _Omnicept Client Approval Settings_ in the HP Omnicept software:
#### Using VirtualDesktopXR (VDXR) with Virtual Desktop

<details>
<summary>Click for details...</summary>
<p><b>- YOU MUST USE VIRTUAL DESKTOP 1.29.3 OR HIGHER</b></p>

<p>- Select VDXR as your OpenXR runtime.<br>
<img src="site/vdxr-openxr.png"></p>

<img alt="Allow all" src="site/omnicept-allow.png"><br>
<b>Toggling Off the Require user approval will allow all applications to use the eye tracker.</b>
<p>- Ensure that eye tracking is enabled on your Quest Pro, and you have performed calibration. These settings can be found under 'Settings' -> 'Movement tracking'.<br>
<img src="site/et-vd-1.png"></p>

<p>- Open the <i>Virtual Desktop</i> app and navigate to the 'Streaming' settings. Enable 'Forward face/eye tracking to PC'.<br>
<img src="site/et-vd-2.png"></p>
</details>

You may otherwise selectively allow applications, one by one. You must first run the application once, enable eye tracking in the OpenXR Toolkit menu, then open the HP Omnicept software and allow the client, under the _Clients_ tab, in the _Incoming subscriptions request_ table:

#### Using SteamVR OpenXR with Virtual Desktop

Follow the instructions to install [OpenXR-Eye-Trackers](https://github.com/mbucchia/OpenXR-Eye-Trackers/wiki/Meta-Quest-Pro#with-virtual-desktop).

### Considerations for Varjo

#### Using Varjo OpenXR

<details>
<summary>Click for details...</summary>
<p>- Open the <i>Varjo Base</i> application on your PC then select Varjo as your OpenXR runtime.<br>
<img src="site/varjo-openxr.png"></p>

<img alt="Allow client" src="site/omnicept-perms.png"><br>
<b>You can Accept Incoming subscriptions requests to selecting allow an application, while forbidding others.</b>
<p>- In the 'System tab', under 'Privacy', toggle 'Allow eye tracking' to on.<br>
<img src="site/et-varjo-1.png"></p>

<p>- Ensure that your eye tracker is calibrated. In the 'Headset' tab, under 'Foveated Rendering', make sure to select a method for using calibration..<br>
<img src="site/et-varjo-2.png"></p>
</details>

### Considerations for Quest Pro

**Oculus Quest Pro only supports eye tracking for PC applications when using Oculus as the OpenXR runtime. If you are using SteamVR as the OpenXR runtime (for example, to use Virtual Desktop), this runtime cannot support eye tracking for PC applications.**
#### Using SteamVR OpenXR

1. IMPORTANT: Create and log in with a [developer account](https://developer.oculus.com/sign-up/). You must be a registered developer for the Oculus runtime to expose the hand tracking features.
Follow the instructions to install [OpenXR-Eye-Trackers](https://github.com/mbucchia/OpenXR-Eye-Trackers/wiki/Varjo-Aero-VR%E2%80%903-XR%E2%80%903).

2. Ensure that your Oculus software is up-to-date (v49 or later).
### Considerations for HP G2 Omnicept

3. Turn on Developer mode on the headset.
#### Using OpenXR for Windows Mixed Reality

<details>
<summary>Click for details...</summary>

<img alt="Enabling developer mode" src="site/oculus-dev-1.jpeg">
<img alt="Enabling developer mode" src="site/oculus-dev-2.jpeg">
<p>- Select Windows Mixed Reality as your OpenXR runtime.<br>
<img src="site/wmr-openxr-1.png"><br>
<img src="site/wmr-openxr-2.png"></p>

<p>- Install the <a href="https://developers.hp.com/omnicept/downloads/hp-omnicept-runtime">HP Omnicept Runtime</a>.</p>

<p>- From the <i>HP Omnicept Tray</i>, open the 'Settings' page, and under 'General' -> 'Omnicept Client Approval Settings', toggle the 'Require user approval for Omnicept clients using sensor data' to Off.<br>
<img src="site/et-omnicept-1.png"></p>
</details>

4. On your PC, open the _Oculus App_, and navigate to _Settings_ and the _Beta_ menu.
#### Using SteamVR OpenXR

Follow the instructions to install [OpenXR-Eye-Trackers](https://github.com/mbucchia/OpenXR-Eye-Trackers/wiki/HP-Reverb-G2-Omnicept).

5. Enable _Developer Runtime Features_.
### Considerations for Pimax Crystal

#### Using PimaxXR

<details>
<summary>Click for details...</summary>
<p>- Open the <i>PimaxXR Control Center</i> application on your PC then select PimaxXR runtime.<br>
<img src="site/pimax-openxr.png"></p>

<p>- Enable 'Allow use of the eye tracker'.<br>
<img src="site/et-pimax-1.png"></p>

<p>- Open the <i>Pimax Play</i> application on your PC then navigate to 'Device Settings'. In the 'Device' tab, make sure that 'Eye Tracking' is enabled and use the 'Calibration' button to perform the calibration if needed.<br>
<img src="site/et-crystal-1.png"></p>
</details>

<img alt="Enabling developer mode" src="site/oculus-dev-3.jpeg">
#### Using SteamVR OpenXR

</details>
Follow the instructions to install [OpenXR-Eye-Trackers](https://github.com/mbucchia/OpenXR-Eye-Trackers/wiki/Pimax-Crystal).

### Considerations for Pimax+Droolon

In order to use the Droolon eye tracking module on your Pimax device, you must first install the [aSeeVR SDK](https://drive.google.com/file/d/1ELDtOnMa-MkgchmWFf7w5an-iPOFtQL8/view?usp=sharing&_ga=2.110383681.599346747.1650530138-1983392096.1642581798).
#### Using PimaxXR

<details>
<summary>Click for details...</summary>
<p>- Open the <i>PimaxXR Control Center</i> application on your PC then select PimaxXR runtime.<br>
<img src="site/pimax-openxr.png"></p>

<p>- Enable 'Allow use of the eye tracker'.<br>
<img src="site/et-pimax-1.png"></p>

<p>Download and install the <a href="https://github.com/guppyexpress/Pimax-Eye-Tracking-Fix">7invensun runtime</a>, specifically the version modified by GuppyExpress. Follow the instructions to install and calibrate the eye tracker. _PimaxXR_ will automatically detect the Droolon Pi1 module once the 7invensun runtime is running.</p>
</details>

Once installed, please run the aSeeVR runtime. In the system tray bar, right-click on the runtime, and make sure to enable _deviced_ -> _Pimax_.
### Considerations for Vive Pro Eye

When using Foveated Rendering in the OpenXR Toolkit, adjust the _Eye projection distance_ from the _Foveated rendering_ menu until you reach comfortable sensitivity.
Coming soon.
2 changes: 1 addition & 1 deletion index.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ Certain features, like Foveated Rendering, have additional requirements, per the
| Praydog's UEVR injector | Yes | Must use the injector in OpenXR mode |
| The 7th Guest VR | Yes | - |
| War Thunder | Yes | - |
| X-Plane 12 | No | Must start the game in OpenXR mode (`--open_xr` parameter) |
| X-Plane 12 | No | - |

#### OpenComposite

Expand Down
Binary file added site/et-crystal-1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added site/et-oculus-1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added site/et-oculus-2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added site/et-oculus-3.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added site/et-omnicept-1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added site/et-pimax-1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added site/et-varjo-1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added site/et-varjo-2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added site/et-vd-1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added site/et-vd-2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added site/vdxr-openxr.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 24a0080

Please sign in to comment.