Introduction to three.js & Leap Motion
0 likes1,141 views
This document provides an overview of three.js and the Leap Motion controller for creating 3D graphics and interactions in a web browser. It explains some basic 3D graphics concepts like scenes, objects, materials and lighting used in three.js. It then demonstrates how to load three.js, add basic 3D shapes, import 3D models, and add lighting and shadows. It also introduces the Leap Motion controller for hand tracking input and shows an example of using it with three.js. Finally, it discusses a project using these tools to create an interactive 3D experience of the US Capitol dome.
Introduction to three.js & Leap Motion
- 1. three.js & leap motion An introduction & overview of 3D graphics in the browser
- 2. • 3D primer • What is three.js • First steps with three.js • What is a Leap Motion Controller • Leap demo • Washington Post special project review Overview
- 3. This is not a tutorial.3D graphics in and of itself is a very complicated subject. This presentation is not a tutorial but does attempt to cover (very briefly) the basics and help you understand where to get started with three.js & how things fit together. With that said- the documentation around three.js is very good and it’s fairly easy to find tutorials online.
- 4. A simple scene consisting of a light, cube, plane and camera. An Example 3D Scene 3D Primer • The lights, cameras & objects are collectively called a scene. In three.js it is a hierarchy & the scene is the root of the tree. • Objects or geometry such as cubes and planes (also called primitives) are created by points in space called vertices. • A group of vertices and their edges create polygons. • A group of polygons (objects) along with a material create a mesh. • The camera captures the picture and in the example to the left we are looking through the camera. • Lights allow us to cast shadows and render highlights.
- 5. First Steps with three.js • Follow the three.js “Getting Started” section in the docs. https://threejs.org/docs/index.html#Manual/Introduction/Creating_a_scene • Load a custom model instead of using geometry. • Add a light & cast shadows.
- 7. . ├── index.html ├── js │ ├── ext │ │ ├── three.js │ │ └── three.modules.js │ └── main.js └── run.py directory structure
- 8. 1 console.info("main.js loaded"); 2 3 var scene = new THREE.Scene(); 4 var camera = new THREE.PerspectiveCamera(75, window.innerWidth / window.innerHeight, 0.1, 1000); 5 console.info("scene & camera created"); 6 7 var renderer = new THREE.WebGLRenderer(); 8 renderer.setSize(window.innerWidth, window.innerHeight); 9 console.info("renderer created"); 10 11 document.body.appendChild(renderer.domElement); 12 console.info("DOM modified"); 13 14 var geometry = new THREE.BoxGeometry(1, 1, 1); 15 var material = new THREE.MeshBasicMaterial({color: 0x00ff00}); 16 var cube = new THREE.Mesh(geometry, material); 17 scene.add(cube); 18 console.info("Cube created"); 19 20 camera.position.z = 5; 21 console.info("Camera repositioned"); 22 23 function render() { 24 // Animate the cube 25 cube.rotation.x += 0.01; 26 cube.rotation.y += 0.01; 27 requestAnimationFrame(render); 28 renderer.render(scene, camera); 29 } 30 console.info("Starting render") 31 render();
- 9. demo
- 10. Free “digger” model from TurboSquid. loading custom models
- 11. Model geometry must be triangulated. Note how there are now diagonal lines and all of the surfaces have triangles.
- 12. Sidebar: Use a 3D graphics program to adjust your model’s geometry. Any 3D program can adjust geometry. Blender is free and can easily do this for us.
- 13. Loading JSON • Use the free utility from the three.js project to convert from OBJ to JSON. https://github.com/alteredq/three.js/blob/master/utils/exporters/convert_obj_three.py • Use the built-in JSONLoader to load the object. • Watch file sizes! If you are using a “real” web server enable gzip compression!
- 14. 1 var greenMat = new THREE.MeshBasicMaterial({color: 0x00ff00}); 2 3 // Create a JSON loader 4 var loader = new THREE.JSONLoader(); 5 6 // Load triangulated model as JSON 7 loader.load( 8 'models/digger_scaled_tri.json', 9 // onLoad callback 10 function (geometry) { 11 // Add geometry to a new mesh combining a material 12 var object = new THREE.Mesh(geometry, greenMat); 13 // Add our object to the scene 14 scene.add(object); 15 } 16 );
- 15. demo
- 17. three.js has many types of lights but we’ll focus on the directional light. • Directional lights shine from a specific direction not a specific location. • The documentation describes this as a light that acts like the sun. • Shadows are only calculated for objects that exist within the light’s shadow camera’s field of view. • The shadow camera for a directional light is orthographic (think 2D, e.g. top down). • In the picture to the left the orange lines denote the shadow camera’s field of view.
- 18. three.js uses shadow mapping with the WebGL renderer. • Shadow mapping was introduced by Lance Williams in 1978. • Computation takes place on the GPU with WebGL. • Think of a shadow map like a texture map. They are applied (mapped) to the surface of the geometry during rendering. • The size of the map directly affects the shadow detail because the size of the map is the size of the buffer and a larger map/buffer can hold more information about the shadow. • Shadows aren’t necessarily hard but they require some tweaking and experimenting to get right.
- 19. demo
- 20. The Leap Motion Controller lets you use your computer in a whole new way. Reach out and swipe, grab, pinch, or punch your way through the digital world. • Uses two wide angle IR cameras to detect hands in 3D space above the device. • Tracks hands, fingers and wrists / arms and translates motion from real world space to 3D environment coordinates. • For developers it is plug & play and very accessible. • It is a bit finicky and detecting / supporting custom gestures can be laborious. Leap Motion Controller
- 21. demo
- 22. In 2015 I was contracted to help build a 3D interactive experience exploring the restoration of the U.S. Capitol Dome. The interactive experience took place at the White House Correspondents Dinner. The scope was narrowed down to build a kiosk where visitors could approach one of two large displays and interact with a 3D model of the dome using a Leap Motion controller. Washington Post Special Project
- 23. 3D Model three.js enabled the rapid development & delivery of the experience within a browser. Many issues could be ignored because we were operating as a kiosk in a known environment: • Device hardware specs were known (fast enough to ignore optimizations) • No external bandwidth requirements (all files are hosted & served locally) • Human explanations & help to explain how to interact with the demo. • No need to open up “heavier” tooling such as Unity. Leap Motion Web Browser three.js
- 24. early prototype
- 25. Things didn’t go as planned
- 26. 3 Weeks to Deliver Interaction Spent the majority of the time figuring out gestures and hand controls. Tried “gimbal” controls. Settled on “joystick” controls. “Hot Spots” & Polishing Used ray casting for “fast enough” collision detection between hands and “hot spots”. Added dynamic “demo” mode to pique interest. Rendering & Lighting Lost a good chunk of time to loading a model of the entire capital. Reverted to just the dome. Had trouble with shadows & model size.
- 27. demo