Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for windows mixed reality controllers #3013

Merged
merged 11 commits into from
Sep 8, 2017
1 change: 1 addition & 0 deletions docs/components/gltf-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@ file.
| Event Name | Description |
|--------------|--------------------------------------------|
| model-loaded | glTF model has been loaded into the scene. |
| model-error | glTF model could not be loaded. |

## Loading Inline

Expand Down
2 changes: 2 additions & 0 deletions docs/components/laser-controls.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ across all VR platforms with a single line of HTML.
[oculus-touch-controls]: ./oculus-touch-controls.md
[tracked-controls]: ./tracked-controls.md
[vive-controls]: ./vive-controls.md
[windows-motion-controls]: ./windows-motion-controls.md

laser-controls is a **higher-order component**, meaning the component wraps and
configures other components, rather than implementing any logic itself. Under
Expand All @@ -33,6 +34,7 @@ the hood, laser-controls sets all of the tracked controller components:
- [oculus-touch-controls]
- [daydream-controls]
- [gearvr-controls]
- [windows-motion-controls]

[cursor]: ./cursor.md
[raycaster]: ./raycaster.md
Expand Down
5 changes: 3 additions & 2 deletions docs/components/tracked-controls.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,13 @@ examples: []
[oculustouchcontrols]: ./oculus-touch-controls.md
[vivecontrols]: ./vive-controls.md
[daydreamcontrols]: ./daydream-controls.md
[windowsmotioncontrols]: ./windows-motion-controls.md

The tracked-controls component interfaces with tracked controllers.
tracked-controls uses the Gamepad API to handle tracked controllers, and is
abstracted by the [hand-controls component][handcontrols] as well as the
[vive-controls][vivecontrols], [oculus-touch-controls][oculustouchcontrols], and
[daydream-controls][daydreamcontrols]
[vive-controls][vivecontrols], [oculus-touch-controls][oculustouchcontrols],
[windows-motion-controls][windowsmotioncontrols], and [daydream-controls][daydreamcontrols]
components. This component elects the appropriate controller, applies pose to
the entity, observes buttons state and emits appropriate events. For non-6DOF controllers
such as [daydream-controls][daydreamcontrols], a primitive arm model is used to emulate
Expand Down
60 changes: 60 additions & 0 deletions docs/components/windows-motion-controls.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
---
title: windows-motion-controls
type: components
layout: docs
parent_section: components
source_code: src/components/windows-motion-controls.js
examples: []
---

[trackedcontrols]: ./tracked-controls.md

The windows-motion-controls component interfaces with any spatial controllers exposed through
Windows Mixed Reality as Spatial Input Sources (such as Motion Controllers).
It wraps the [tracked-controls component][trackedcontrols] while adding button
mappings, events, and a controller model that highlights applies position/rotation transforms
to the pressed buttons (trigger, grip, menu, thumbstick, trackpad) and moved axes (thumbstick and trackpad.)

## Example

```html
<a-entity windows-motion-controls="hand: left"></a-entity>
<a-entity windows-motion-controls="hand: right"></a-entity>
```

## Value

| Property | Description | Default Value |
|----------------------|---------------------------------------------------------------------------------------------------|----------------|
| hand | The hand that will be tracked (i.e., right, left). | right |
| pair | Which pair of controllers, if > 2 are connected. | 0 |
| model | Whether the controller model is loaded. | true |
| hideDisconnected | Disable rendering of controller model when no matching gamepad (based on ID & hand) is connected. | true |


## Events

| Event Name | Description |
| ---------- | ----------- |
| thumbstickdown | Thumbstick button pressed. |
| thumbstickup | Thumbstick button released. |
| thumbstickchanged | Thumbstick button changed. |
| thumbstickmoved | Thumbstick axis moved. |
| triggerdown | Trigger pressed. |
| triggerup | Trigger released. |
| triggerchanged | Trigger changed. |
| gripdown | Grip button pressed. |
| gripup | Grip button released. |
| gripchanged | Grip button changed. |
| menudown | Menu button pressed. |
| menuup | Menu button released. |
| menuchanged | Menu button changed. |
| trackpaddown | Trackpad pressed. |
| trackpadup | Trackpad released. |
| trackpadchanged | Trackpad button changed. |
| trackpadmoved | Trackpad axis moved. |
| controllerdisplayready | The model file is loaded and completed parsing. |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe controllermodelready is a better name?


## Assets

TBC.
2 changes: 2 additions & 0 deletions docs/introduction/interactions-and-controllers.md
Original file line number Diff line number Diff line change
Expand Up @@ -464,6 +464,7 @@ AFRAME.registerComponent('custom-controls', {
el.setAttribute('oculus-touch-controls', controlConfiguration);
el.setAttribute('daydream-controls', controlConfiguration);
el.setAttribute('gearvr-controls', controlConfiguration);
el.setAttribute('windows-motion-controls', controlConfiguration);

// Set a model.
el.setAttribute('gltf-model', this.data.model);
Expand Down Expand Up @@ -492,6 +493,7 @@ event handlers how we want:
- [hand-controls events](../components/hand-controls.md#events)
- [oculus-touch-controls events](../components/oculus-touch-controls.md#events)
- [vive-controls events](../components/vive-controls.md#events)
- [windows-motion-controls events](../components/windows-motion-controls.md#events)

For example, we can listen to the Oculus Touch X button press, and toggle
visibility of an entity. In component form:
Expand Down
5 changes: 5 additions & 0 deletions src/components/gltf-model.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
var registerComponent = require('../core/component').registerComponent;
var THREE = require('../lib/three');
var utils = require('../utils/');
var warn = utils.debug('components:windows-motion-controls:warn');
Copy link
Member

@donmccurdy donmccurdy Sep 2, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be:

var warn = utils.debug('components:gltf-model:warn');

...since the warning will appear for any glTF model.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, copy-paste error. Will fix!


/**
* glTF model loader.
Expand All @@ -26,6 +28,9 @@ module.exports.Component = registerComponent('gltf-model', {
self.model.animations = gltfModel.animations;
el.setObject3D('mesh', self.model);
el.emit('model-loaded', {format: 'gltf', model: self.model});
}, undefined /* onProgress */, function gltfFailed () {
warn('Failed to glTF-model: ' + src);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The gltfFailed callback will receive an argument, error — IMO the warning and event should use that message, as it's probably more informative than the URL. Suggested:

var message = error ? error.message : 'Failed to load glTF model';
warn(message);
el.emit('model-error', {format: 'gltf', message: message});

el.emit('model-error', {format: 'gltf', src: src});
Copy link
Member

@dmarcos dmarcos Aug 31, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we throw an error instead of an event? A similar scenario can be found in the text component: https://github.com/aframevr/aframe/blob/master/src/components/text.js#L391

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could do both? I listen to the error message to try loading a different (fallback) asset file, I think it could be useful in other contexts for the caller to know that the file load failed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, we could have both

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 for both. The errors should be helpful for developers debugging, who may not know to listen for the event.

But my inclination would be to log an error or warning, through the debug util, rather than throwing something that can't be caught. See obj-model.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool! Let's add both then 👌

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do both, thanks!

});
},

Expand Down
33 changes: 18 additions & 15 deletions src/components/hand-controls.js
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,9 @@ EVENTS[ANIMATIONS.point] = 'pointing';
EVENTS[ANIMATIONS.thumb] = 'thumb';

/**
* Hand controls component that abstracts 6DoF controls: oculus-touch-controls, vive-controls.
* Hand controls component that abstracts 6DoF controls:
* oculus-touch-controls, vive-controls, windows-motion-controls.
*
* Originally meant to be a sample implementation of applications-specific controls that
* abstracts multiple types of controllers.
*
Expand All @@ -49,7 +51,7 @@ module.exports.Component = registerComponent('hand-controls', {
var self = this;
// Current pose.
this.gesture = ANIMATIONS.open;
// Active buttons populated by events provided by oculus-touch-controls and vive-controls.
// Active buttons populated by events provided by the attached controls.
this.pressedButtons = {};
this.touchedButtons = {};
this.loader = new THREE.ObjectLoader();
Expand Down Expand Up @@ -158,14 +160,15 @@ module.exports.Component = registerComponent('hand-controls', {
var el = this.el;
var hand = this.data;

// Get common configuration to abstract Vive and Oculus.
// Get common configuration to abstract different vendor controls.
controlConfiguration = {
hand: hand,
model: false,
rotationOffset: hand === 'left' ? 90 : -90
};
el.setAttribute('vive-controls', controlConfiguration);
el.setAttribute('oculus-touch-controls', controlConfiguration);
el.setAttribute('windows-motion-controls', controlConfiguration);

// Set model.
if (hand !== previousHand) {
Expand Down Expand Up @@ -233,22 +236,22 @@ module.exports.Component = registerComponent('hand-controls', {
var isTrackpadActive = this.pressedButtons['trackpad'] || this.touchedButtons['trackpad'];
var isTriggerActive = this.pressedButtons['trigger'] || this.touchedButtons['trigger'];
var isABXYActive = this.touchedButtons['AorX'] || this.touchedButtons['BorY'];
var isOculusTouch = isOculusTouchController(this.el.components['tracked-controls']);
var isVive = isViveController(this.el.components['tracked-controls']);

// Works well with Oculus Touch but Vive needs tweaks.
// Works well with Oculus Touch and Windows Motion Controls, but Vive needs tweaks.
if (isGripActive) {
if (!isOculusTouch) {
if (isVive) {
gesture = ANIMATIONS.fist;
} else
if (isSurfaceActive || isABXYActive || isTrackpadActive) {
gesture = isTriggerActive ? ANIMATIONS.fist : ANIMATIONS.point;
} else {
gesture = isTriggerActive ? ANIMATIONS.thumbUp : ANIMATIONS.pointThumb;
}
if (isSurfaceActive || isABXYActive || isTrackpadActive) {
gesture = isTriggerActive ? ANIMATIONS.fist : ANIMATIONS.point;
} else {
gesture = isTriggerActive ? ANIMATIONS.thumbUp : ANIMATIONS.pointThumb;
}
} else {
if (isTriggerActive) {
gesture = isOculusTouch ? ANIMATIONS.hold : ANIMATIONS.fist;
} else if (!isOculusTouch && isTrackpadActive) {
gesture = !isVive ? ANIMATIONS.hold : ANIMATIONS.fist;
} else if (isVive && isTrackpadActive) {
gesture = ANIMATIONS.point;
}
}
Expand Down Expand Up @@ -355,8 +358,8 @@ function getGestureEventName (gesture, active) {
return;
}

function isOculusTouchController (trackedControls) {
function isViveController (trackedControls) {
var controllerId = trackedControls && trackedControls.controller &&
trackedControls.controller.id;
return controllerId && controllerId.indexOf('Oculus Touch') === 0;
return controllerId && controllerId.indexOf('OpenVR ') === 0;
}
1 change: 1 addition & 0 deletions src/components/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ require('./tracked-controls');
require('./visible');
require('./vive-controls');
require('./wasd-controls');
require('./windows-motion-controls');

require('./scene/debug');
require('./scene/embedded');
Expand Down
27 changes: 22 additions & 5 deletions src/components/laser-controls.js
Original file line number Diff line number Diff line change
Expand Up @@ -16,21 +16,33 @@ registerComponent('laser-controls', {
el.setAttribute('gearvr-controls', {hand: data.hand});
el.setAttribute('oculus-touch-controls', {hand: data.hand});
el.setAttribute('vive-controls', {hand: data.hand});
el.setAttribute('windows-motion-controls', {hand: data.hand});

// Wait for controller to connect before
el.addEventListener('controllerconnected', function (evt) {
// Wait for controller to connect, or have a valid pointing pose, before creating ray
el.addEventListener('controllerconnected', function (evt) { createRay(evt); });
el.addEventListener('controllerdisplayready', createRay);

function createRay (evt) {
var controllerConfig = config[evt.detail.name];

if (!controllerConfig) { return; }

el.setAttribute('raycaster', utils.extend({
var raycasterConfig = utils.extend({
showLine: true
}, controllerConfig.raycaster || {}));
}, controllerConfig.raycaster || {});

if (evt.detail.name === 'windows-motion-controls') {
var motionControls = el.components['windows-motion-controls'];
raycasterConfig = utils.extend(raycasterConfig, motionControls.rayOrigin);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we pass the rayOrigin in the evt.detail object? This way we don't have to check for windows-motion-controls We should maybe move all the controllers to use a controllermodelready event for consistency. Maybe better in a separate PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the pattern of passing the ray details in the event, will modify to follow that pattern. Happy to also modify the others, just let me know if you would prefer that in a separate PR once this is completed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! We can do it in a separate PR so we move things along.

raycasterConfig.showLine = motionControls.rayOriginInitialized;
}

el.setAttribute('raycaster', raycasterConfig);

el.setAttribute('cursor', utils.extend({
fuse: false
}, controllerConfig.cursor));
});
}
},

config: {
Expand All @@ -50,6 +62,11 @@ registerComponent('laser-controls', {

'vive-controls': {
cursor: {downEvents: ['triggerdown'], upEvents: ['triggerup']}
},

'windows-motion-controls': {
cursor: {downEvents: ['triggerdown'], upEvents: ['triggerup']},
raycaster: {showLine: false}
}
}
});
2 changes: 1 addition & 1 deletion src/components/raycaster.js
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ module.exports.Component = registerComponent('raycaster', {
// Draw line.
if (data.showLine &&
(data.far !== oldData.far || data.origin !== oldData.origin ||
data.direction !== oldData.direction)) {
data.direction !== oldData.direction || data.showLine !== oldData.showLine)) {
this.unitLineEndVec3.copy(data.origin).add(data.direction).normalize();
this.drawLine();
}
Expand Down
33 changes: 16 additions & 17 deletions src/components/tracked-controls.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
var registerComponent = require('../core/component').registerComponent;
var controllerUtils = require('../utils/tracked-controls');
var THREE = require('../lib/three');
var DEFAULT_CAMERA_HEIGHT = require('../constants').DEFAULT_CAMERA_HEIGHT;

Expand All @@ -14,13 +15,15 @@ var FOREARM = {x: 0, y: 0, z: -0.175};
* Select the appropriate controller and apply pose to the entity.
* Observe button states and emit appropriate events.
*
* @property {number} controller - Index of controller in array returned by Gamepad API.
* @property {number} controller - Index of controller in array returned by Gamepad API. Only used if hand property is not set.
* @property {string} id - Selected controller among those returned by Gamepad API.
* @property {number} hand - If multiple controllers found with id, choose the one with the given value for hand. If set, we ignore 'controller' property
*/
module.exports.Component = registerComponent('tracked-controls', {
schema: {
controller: {default: 0},
id: {type: 'string', default: ''},
hand: {type: 'string', default: ''},
idPrefix: {type: 'string', default: ''},
rotationOffset: {default: 0},
// Arm model parameters when not 6DoF.
Expand All @@ -31,6 +34,7 @@ module.exports.Component = registerComponent('tracked-controls', {
init: function () {
this.axis = [0, 0, 0];
this.buttonStates = {};
this.targetControllerNumber = this.data.controller;

this.dolly = new THREE.Object3D();
this.controllerEuler = new THREE.Euler();
Expand Down Expand Up @@ -70,25 +74,20 @@ module.exports.Component = registerComponent('tracked-controls', {
},

/**
* Handle update to `id` or `idPrefix.
* Handle update controller match criteria (such as `id`, `idPrefix`, `hand`, `controller`)
*/
updateGamepad: function () {
var controllers = this.system.controllers;
var data = this.data;
var i;
var matchCount = 0;

// Hand IDs: 0 is right, 1 is left.
for (i = 0; i < controllers.length; i++) {
if ((data.idPrefix && controllers[i].id.indexOf(data.idPrefix) === 0) ||
(!data.idPrefix && controllers[i].id === data.id)) {
matchCount++;
if (matchCount - 1 === data.controller) {
this.controller = controllers[i];
return;
}
}
}
var controller = controllerUtils.findMatchingController(
this.system.controllers,
data.id,
data.idPrefix,
data.hand,
data.controller
);

// Only replace the stored controller if we find a new one.
this.controller = controller || this.controller;
},

applyArmModel: function (controllerPosition) {
Expand Down
Loading