Getting Started with WebXR using Babylon.js

2.1K Views

January 16, 23

スライド概要

profile-image

可視化技術や人間計測/空間計測技術を活用した問題解決に関する研究開発。 ARコンテンツ作成勉強会(tryAR)を主催。

シェア

またはPlayer版

埋め込む »CMSなどでJSが使えない場合

(ダウンロード不可)

関連スライド

各ページのテキスト
1.

Getting Started with WebXR using Babylon.js for HoloLens 2 / Meta Quest

2.

Download sample files https://github.com/TakashiYoshinaga/AR-Fukuoka/ raw/main/Babylonjs_20220727/Samples.zip

3.

Todayʼs Goal https://youtu.be/MyY8gUxv0vA Introduction to the basics of using Babylon.js and the use of hand interaction by MRTK

4.

Duplicate Template Source Code https://glitch.com/~babylon-template GET STARTED

5.

Duplicate Template Source Code Remix Your Own

6.

Duplicate Template Source Code Click on index.html and confirm that the code is displayed index.html

7.

Duplicate Template Source Code Click on index.html and confirm that the code is displayed Editor Preview

8.

Tip: If Preview is Not Displayed PREVIEW

9.

Tip: If Preview is Not Displayed Open preview pane

10.

Exercise Procedure Web browser on PC Creation of 3D space HoloLens/Quest XR Mode Interaction by MRTK Visualize Hands

11.

Check the Template Lesson01

12.

Check the Template Loading Library Description of drawing and interaction (today's main focus) Layout of drawing area

13.

Check the Template Layout of drawing area

14.
[beta]
Explanation of HTML Description
<!-- Area to be drawn -->
<canvas id="renderCanvas" style="width: 100%; height: 100%;"></canvas>

ID is used when accessing from javascript

Paste the canvas in the window at the full size.

15.

Check the Template Loading Library

16.

テンプレートの確認 Description of drawing and interaction (today's main focus)

17.

Exercise Procedure Web browser on PC Creation of 3D space HoloLens/Quest XR Mode Interaction by MRTK Visualize Hands

18.

Declaration of Variables Add variable here //Initialization of variables after page loading is completed window.onload = function() { } //Initialization of 3D scene function createScene() { } //Initialization of WebXR settings async function initializeXR(){ } //Initialization of 3D objects to be drawn function addObjects(){ }

19.

Declaration of Variables let canvas; //canvas element (drawing area) ← "where" to draw let engine; //Drawing functionality with Babylon.js ← "What" to draw with let scene; //Virtual 3D space ← "What" to draw //Initialization of variables after page loading is completed window.onload = function() { } //Initialization of 3D scene function createScene() { } //Initialization of WebXR settings async function initializeXR(){ } Lesson02

20.
[beta]
Initialization of Drawing Functions of Babylon.js
let canvas; //canvas element (drawing area) ← "where" to draw
let engine; //Drawing functionality with Babylon.js ← "What" to draw with
let scene; //Virtual 3D space ← "What" to draw
//Initialization of variables after page loading is completed
window.onload = function() {
//Obtain the drawing area (canvas element) from HTML
canvas = document.getElementById("renderCanvas");
//Preparation for drawing with Babylon.js (Arguments: destination, anti-aliasing)
engine = new BABYLON.Engine(canvas, true);
//カメラやライトの設定を⾏ったscene(3D空間)を取得
scene= createScene();
}
//Initialization of 3D scene
function createScene() {
}

Lesson03

21.
[beta]
Initialization of Scene (3D Space)
let canvas; //canvas element (drawing area) ← "where" to draw
let engine; //Drawing functionality with Babylon.js ← "What" to draw with
let scene; //Virtual 3D space ← "What" to draw
//Initialization of variables after page loading is completed
window.onload = function() {
//Obtain the drawing area (canvas element) from HTML
canvas = document.getElementById("renderCanvas");
//Preparation for drawing with Babylon.js (Arguments: destination, anti-aliasing)
engine = new BABYLON.Engine(canvas, true);
//Create a scene (3D space) with camera and light settings
createScene();
}
// Initialization of 3D scene
function createScene() {
}

Description of the initialization process of a scene (next page)

Lesson04

22.
[beta]
Initialization of Scene (3D Space)
function createScene() {
//Create 3D space
scene = new BABYLON.Scene(engine);
//Set background color
scene.clearColor = new BABYLON.Color3.Black();
//Set Light (arguments: name, direction of reflection, destination)
let light = new BABYLON.HemisphericLight("light",
new BABYLON.Vector3(1, 1, -0.5) ,scene);
//Create a camera[Details]
let camera = new BABYLON.ArcRotateCamera(
"camera",
//name of camea
-Math.PI/2, Math.PI/2, 3, //alpha,beta,distance form origin
new BABYLON.Vector3(0, 0, 0)); //look at
// Enables camera operation with mouse or keyboard
camera.attachControl(canvas, true);
}

Lesson05

23.
[beta]
Start Drawing
//Initialization of variables after page loading is completed
window.onload = function() {
//Obtain the drawing area (canvas element) from HTML
canvas = document.getElementById("renderCanvas");
//Preparation for drawing with Babylon.js (Arguments: destination, anti-aliasing)
engine = new BABYLON.Engine(canvas, true);
//Create a scene (3D space) with camera and light settings
createScene();
//Update drawing every frame
engine.runRenderLoop(function () {
if (scene && scene.activeCamera) {
scene.render(); //Drawing!
}
});
}

Lesson06

24.

Operation Check Preview background becomes the specified color (=black)

25.
[beta]
Adding a Display Object
// Initialization of variables after page loading is completedう
window.onload = function() {
//Obtain the drawing area (canvas element) from HTML
canvas = document.getElementById("renderCanvas");
//Preparation for drawing with Babylon.js (Arguments: destination, anti-aliasing)
engine = new BABYLON.Engine(canvas, true);
//Create a scene (3D space) with camera and light settings
createScene();
//Creating Drawing Objects
addObjects();
//Update drawing every frame
engine.runRenderLoop(function () {
if (scene && scene.activeCamera) {
scene.render();
}
});
Lesson07
}

26.

Adding a Display Object Edit here next

27.
[beta]
Adding a Display Object
function addObjects(){

}

//Create ground (for visual checking the location of the origin.)
let ground = BABYLON.MeshBuilder.CreateGround(
"ground", {width: 1, height: 1});
//Creation of Box object (30cm)
let box = BABYLON.MeshBuilder.CreateBox(
"box", {width: 0.3, height: 0.3, depth: 0.3});
box.position.x = 0.0;
box.position.y = 1; // 1m above the ground
//Create Material
let boxMaterial = new BABYLON.StandardMaterial("material");
//Set the color of the box (in this case randomly changing) and apply it to the box
boxMaterial.diffuseColor = BABYLON.Color3.Random();
box.material = boxMaterial;
//Create an octahedron
let octa = BABYLON.MeshBuilder.CreatePolyhedron("octa", {type:1,size: 0.15});
octa.position.x=0.5;
octa.position.y=1;
Lesson08

28.

Operation Check Box and octahedron is shown

29.

Adding a Display Object Support for mouse and keyboard operation with one line of camera.attachControl(canvas, true);. Initial Viewpoint Left-click and drag to rotate Scroll back and forward Right-click and drag to move vertical/horizontal

30.

Exercise Procedure Web browser on PC Creation of 3D space HoloLens/Quest XR Mode Interaction by MRTK Visualize Hands

31.

Experience the Content on Quest or HoloLens2 window.onload = function() { /* Omitted */ //Creating Drawing Objects addObjects(); //Initialization of WebXR settings initializeXR(); } //Update drawing every frame engine.runRenderLoop(function () { if (scene && scene.activeCamera) { scene.render(); } }); //Initialization of WebXR settings async function initializeXR(){ let xr = await scene.createDefaultXRExperienceAsync({ /*options*/ }); } Lesson09

32.

Change the URL Before Testing on a Device Click Settings

33.

Change the URL Before Testing on a Device Edit project details Change the name to something easy to remember Save

34.

Check URL of Your XR Content Click Copy Link

35.

Check URL of Your XR Content Check the URL by pasting it into a text editor, etc. https://your-content.glitch.me/

36.

Operation Check HoloLens2 Input the URL Goggle Button Quest Input the URL Goggle Button

37.

Operation Check

38.

How to Exit HoloLens2 Home Button Quest Close Button Exit content via Home window shown by wrist tap (HoloLens2) or Oculus button (Quest)

39.

Exercise Procedure Web browser on PC Creation of 3D space HoloLens/Quest XR Mode Interaction by MRTK Visualize Hands

40.
[beta]
Manipulate Objects (1)
function addObjects(){
/*Omitted*/
// Set box color randomly
let boxMaterial = new BABYLON.StandardMaterial("material");
boxMaterial.diffuseColor = BABYLON.Color3.Random();
box.material = boxMaterial;
//Create an octahedron
let octa = BABYLON.MeshBuilder.CreatePolyhedron("octa", {type:1,size: 0.15});
octa.position.x=0.5;
octa.position.y=1;
//Add manipulatable attribute to the box object
let sixDofDragBehavior = new BABYLON.SixDofDragBehavior();
sixDofDragBehavior.allowMultiPointer = false;
sixDofDragBehavior.rotateWithMotionController = false;
box.addBehavior(sixDofDragBehavior);
}

Lesson10

41.

Manipulate Objects (1)

42.

Operation Check Reload Reload Always click the Reload button before testing

43.

Operation Check For Quest users, this content is currently only available for controllers

44.
[beta]
Manipulate Objects (2)
function addObjects(){
/*Omitted*/
let boxMaterial = new BABYLON.StandardMaterial("material");
boxMaterial.diffuseColor = BABYLON.Color3.Random();
box.material = boxMaterial;
//Create an octahedron
let octa = BABYLON.MeshBuilder.CreatePolyhedron("octa", {type:1,size: 0.15});
octa.position.x=0.5;
octa.position.y=1;
//Add manipulatable attribute to the box object
let sixDofDragBehavior = new BABYLON.SixDofDragBehavior();
sixDofDragBehavior.allowMultiPointer=false;
sixDofDragBehavior.rotateWithMotionController=false;
box.addBehavior(sixDofDragBehavior);
}

Code for manipulation with bounding boxes (next page)

45.

Manipulate Objects (2) //Add manipulatable attribute to the box object let sixDofDragBehavior = new BABYLON.SixDofDragBehavior(); sixDofDragBehavior.allowMultiPointer=false; sixDofDragBehavior.rotateWithMotionController=false; box.addBehavior(sixDofDragBehavior); //Manipulation with bounding box var gizmoManager = new BABYLON.GizmoManager(scene); gizmoManager.boundingBoxGizmoEnabled = true; gizmoManager.boundingBoxDragBehavior.allowMultiPointer=false; //Scaling settings (enable scaling, synchronize x-y-z scaling) gizmoManager.gizmos.boundingBoxGizmo.setEnabledScaling(true,true); //Setting the appearance of the Bounding box gizmoManager.gizmos.boundingBoxGizmo.scaleBoxSize=0.04; gizmoManager.gizmos.boundingBoxGizmo.rotationSphereSize=0.05; //⼋⾯体オブジェクトを登録 gizmoManager.attachableMeshes = [octa]; Lesson11

46.

Operation Check on PC

47.

Operation Check on XR Device Reload Reload Always click the Reload button before testing

48.

Operation Check on XR Device For Quest users, this content is currently only available for controllers

49.

Exercise Procedure Web browser on PC Creation of 3D space HoloLens/Quest XR Mode Interaction by MRTK Visualize Hands

50.
[beta]
Turn On Hand Tracking (For Quest)
//Initialization of WebXR settings
async function initializeXR() {
let xr = await scene.createDefaultXRExperienceAsync({});
// Check if XR mode available or not
if (!xr.baseExperience) { }
else {
//Turn on hand tracking if XR mode available
xr.baseExperience.featuresManager.enableFeature(
BABYLON.WebXRFeatureName.HAND_TRACKING,
"latest",
{xrInput: xr.input}
);
}
}

Lesson12

51.

Operation Check on XR Device

52.

Completed!

53.

References Camera https://doc.babylonjs.com/divingDeeper/cameras/camera_introduction WebXR Experience Helper https://doc.babylonjs.com/divingDeeper/webXR/webXRExperienceHelpers#webxr-defaultexperience-helper MRTK https://doc.babylonjs.com/divingDeeper/gui/mrtk SixDofDragBehaviour https://doc.babylonjs.com/typedoc/classes/BABYLON.SixDofDragBehavior GizmoManager https://doc.babylonjs.com/typedoc/classes/BABYLON.GizmoManager Gizmos https://doc.babylonjs.com/divingDeeper/mesh/gizmo Hand Tracking https://doc.babylonjs.com/divingDeeper/webXR/WebXRSelectedFeatures#hand-tracking