WebVR Experience Helper
WebVR vs WebXR
While the WebVR experience helper will continue to work, it is strongly recommended that projects instead use the WebXR experience helper. For more information, check out our introduction to WebXR.
Introduction
The WebVR Experience Helper provides a quick way to add WebVR support to a Babylon scene.
Features include:
- WebVR camera and non-WebVR camera initialization
- Enter WebVR button
- Teleportation and rotation in the world
- Gaze tracking with mesh selection from HMD and controllers
Setup
A VRExperienceHelper can be created directly from the scene.
var scene = new BABYLON.Scene(engine);var vrHelper = scene.createDefaultVRExperience();
This will initialize a WebVR camera and a non-WebVR camera in the scene. It will also create an enterVR button at the bottom right of the screen which will start rendering to the HMD on click.
Options
- createDeviceOrientationCamera(default: true): If the non-WebVR camera should be created. To use an existing camera, create it and then initialize the helper with this set to false in the constructor.
- createFallbackVRDeviceOrientationFreeCamera(default: true): When no HMD is connected, this flag specifies if the VR camera should fallback to a VRDeviceOrientationFreeCamera which will render each eye on the screen. This can be set to false to only enable entering VR if an HMD is connected.
Detect if fallback orientation camera is supported
If a webVR capable device is not detected Babylon will fallback to using a vrDeviceOrientationCamera however device orientation will only be available if the device has an orientation sensor available. In the latest version of Safari, the orientation sensor is disabled by default and it does not prompt users to enable it in settings so currently this must be done by the app. See https://www.applemust.com/how-and-why-to-use-motion-orientation-settings-in-ios/
vrHelper.onAfterEnteringVRObservable.add(() => {if (scene.activeCamera === vrHelper.vrDeviceOrientationCamera) {BABYLON.FreeCameraDeviceOrientationInput.WaitForOrientationChangeAsync(1000).then(() => {// Successfully received sensor input}).catch(() => {alert("Device orientation camera is being used but no sensor is found, prompt user to enable in safari settings");});}});
See it in action here: Fallback Orientation Camera Example
Teleportation and Rotation
To enable teleportation in the scene, create a mesh that the user should be able to teleport to and then enable teleportation with that mesh's name.
var ground = BABYLON.Mesh.CreateGround("ground", 6, 6, 2, scene);vrHelper.enableTeleportation({ floorMeshName: "ground" });
To teleport, hold up on the joystick to display where the user will be teleported to and then release to teleport. To rotate, move the joystick to the left or to the right.
When WebVR controllers are connected, the teleportation will be based on where the controller is pointing.
When WebVR controllers are not connected, the user will teleport to where the user is looking and teleportation can be triggered with an Xbox controller.
Teleportation events
Teleportation has two observables you can subscribe to:
onBeforeCameraTeleport: Observable raised when teleportation is requested, receiving target Vector3 position as parameter:
vrHelper.onBeforeCameraTeleport.add(targetPosition => {//Raised before camera is teleported});
onAfterCameraTeleport: Observable raised when teleportation animation finishes, receiving target Vector3 position as parameter:
vrHelper.onAfterCameraTeleport.add(targetPosition => {//Raised after teleportation animation finishes});
To enable teleportation in the scene, create a mesh that the user should be able to teleport to and then enable teleportation with that mesh's name.
var ground = BABYLON.Mesh.CreateGround("ground", 6, 6, 2, scene);vrHelper.enableTeleportation({ floorMeshName: "ground" });
Enabling / disabling teleportation
Teleportation can be enabled or disabled on demand by using the property teleportationEnabled:
// Enable teleportationvrHelper.teleportationEnabled = true;//Disable teleportation (teleportation mesh will not be displayed)vrHelper.teleportationEnabled = false;
To customize the teleportation target mesh the following property can be set to the mesh you'd like to use:
vrHelper.teleportationTarget = BABYLON.Mesh.CreateSphere("sphere1",4,0.1,scene);
Accessing cameras
The VR and non-VR camera can be accessed from the helper to handle any application specific logic.
// Initial camera before the user enters VRvrHelper.deviceOrientationCamera;// WebVR camera used after the user enters VRvrHelper.webVRCamera;// One of the 2 cameras above depending on which one is in usevrHelper.currentVRCamera;
Accessing controllers
The controllers can be accessed from the helper to handle any application specific logic.
vrHelper.onControllerMeshLoaded.add(webVRController => {var controllerMesh = webVRController.mesh;webVRController.onTriggerStateChangedObservable.add(() => {// Trigger pressed event});});
Please note that the microsoft controllers are using the GLB file format and require the GLTF Loader.
Accessing vr device position and rotation
Position and rotation in Babylon space can be accessed through the webVRCamera's devicePosition and deviceRotationQuaternion
// Left and right hand position/rotationif (vrHelper.webVRCamera.leftController) {leftHand.position = vrHelper.webVRCamera.leftController.devicePosition.clone();leftHand.rotationQuaternion = vrHelper.webVRCamera.leftController.deviceRotationQuaternion.clone();}if (vrHelper.webVRCamera.rightController) {rightHand.position = vrHelper.webVRCamera.rightController.devicePosition.clone();rightHand.rotationQuaternion = vrHelper.webVRCamera.rightController.deviceRotationQuaternion.clone();}// Head position/rotationhead.position = vrHelper.webVRCamera.devicePosition.clone();head.rotationQuaternion = vrHelper.webVRCamera.deviceRotationQuaternion.clone();
See an Example here: Accessing VR Device position and rotation Example
Gaze and interaction
Gaze and interactions can be enabled through the enableInteractions method. See Example: Gaze and Interactions Example
vrHelper.enableInteractions();
This will start casting a ray from either the user's camera or controllers. Where this ray intersects a mesh in the scene, a small gaze mesh will be placed to indicate to the user what is currently selected.
Please note the gaze controllers will simulate pointer events so scene.onPointerObservable
will be raised when gaze is enabled.
To filter which meshes the gaze can intersect with, the raySelectionPredicate can be used:
vrHelper.raySelectionPredicate = mesh => {if (mesh.name.indexOf("Flags") !== -1) {return true;}return false;};
This will cause the user's gaze to pass through any mesh which results in the raySelectionPredicate returning false.
As the user moves between meshes with their gaze, the onNewMeshSelected event will occur. Note: This only works after interactions have been enabled.
vrHelper.onNewMeshSelected.add(mesh => {// Mesh has been selected});
This will return the single closest mesh that was selected.
Prior to onNewMeshSelected an event called onNewMeshPicked is raised when a mesh is selected based on meshSelectionPredicate successful evaluation. This observable notifies a PickingInfo object to subscribers.
vrHelper.onNewMeshPicked.add(pickingInfo => {//Callback receiving ray cast picking info});
As the user unselects a mesh with their gaze or controller, the onSelectedMeshUnselected event will occur.
vrHelper.onSelectedMeshUnselected.add(mesh => {// Mesh has been unselected});
You can add your own filtering logic with meshSelectionPredicate. Note: This will be applied after the raySelectionPredicate.
vrHelper.meshSelectionPredicate = mesh => {if (mesh.name.indexOf("Flags01") !== -1) {return true;}return false;};
The logic order for raySelectionPredicate, meshSelectionPredicate, onNewMeshPicked, onNewMeshSelected are as followed:
- Ray is casted from the controller
- When the ray hits an object the raySelectionPredicate will be called and if true the ray will collide there and be stopped otherwise the ray will pass through the object
- Teleportation target location is updated to where the ray collided if the collision is also a floor mesh
- If the collision object was not collided with on the last frame meshSelectionPredicate is checked, if it returns true the onNewMeshPicked event is fired and then onNewMeshSelected is fired
The gaze tracker can be customized by setting the gazeTrackerMesh. GazeTrackerMesh Example
vrHelper.gazeTrackerMesh = BABYLON.Mesh.CreateSphere("sphere1", 4, 0.1, scene);
On specific devices like iOS (where fullscreen is not supported), you may want to set vrHelper.enableGazeEvenWhenNoPointerLock = true
to let the gaze controller run even when not under fullscreen and pointer lock.
Grab
By combining By combining WebVR controller method and add/removeChild method, you can grab objects by pressing trigger button.
webVRController.onTriggerStateChangedObservable.add(stateObject => {if (webVRController.hand == "left") {if (selectedMesh != null) {//grabif (stateObject.value > 0.01) {webVRController.mesh.addChild(selectedMesh);//ungrab} else {webVRController.mesh.removeChild(selectedMesh);}}}});
Selected mesh is detected by onNewMeshSelected method.
VRHelper.onNewMeshSelected.add(function(mesh) {selectedMesh = mesh;});VRHelper.onSelectedMeshUnselected.add(function() {selectedMesh = null;});
See the example.
Multiview
To improve rendering performance by up to 2x, try using Multiview which will render both eyes in a single render pass
Examples
Scenes:
Games: