MEAP Edition Manning Early Access Program Oculus Rift in Action Version 12 Copyright 2015 Manning Publications ©Manning Publications Co. We welcome reader comments about anything in the manuscript - other than typos and other simple mistakes. These will be cleaned up during production of the book by copyeditors and proofreaders. https://forums.manning.com/forums/oculus-rift-in-action For more information on this and other Manning titles go to www.manning.com ©Manning Publications Co.
welcome Hello, and welcome to our MEAP! Thanks for joining us in this project to document and make easy-to-use that brand new, not-even-released yet piece of the future of Virtual Reality, the Oculus Rift. We’re ridiculously excited about the prospects of the hardware, and if our book improves the quality of the software that folks write for it, then our work here is done.
brief contents PART 1: GETTING STARTED 1 Meet the Oculus Rift PART 2: USING THE OCULUS C API 2 Creating Your First Rift Interactions 3 Pulling Data Out of the Rift: Working with the Head Tracker 4 Sending Output to the Rift: Working with the display 5 Putting it all Together: Integrating Head Tracking and 3D Rendering 6 Performance and quality PART 3: USING UNITY 7 Unity: Creating Applications That Run on the Rift 8 Unity: Tailoring Your Application for the Rift PART 4: THE VR USER EXPERIENCE 9 User Interfa
1 Part 1 Getting Started Part 1 of the book introduces you to the Oculus Rift hardware and to virtual reality. We begin with an exploration of what virtual reality is and why you would want to develop for the Rift. From there, we move on to an overview of the Rift hardware and how it works. After getting to know the Rift hardware, we’ll take a look at the different development paths you can take for creating your Rift application.
2 1 Meet the Oculus Rift This chapter covers • Why you should support the Rift • How the Rift is being used today • The Rift hardware and how it works • Setting up your hardware • Dealing with motion sickness • Development paths: C, Java, Python, or Unity If you've picked up this book, you probably already know that the Rift is a virtual reality head-mounted display (VR HMD).
3 1.1 Why support the Rift? There are really two questions here. The first is whether you should support VR in general, and the second is whether you should support the Rift specifically. 1.1.1 The call of virtual reality If you’ve ever seen an episode of Star Trek and imagined what you could do with your own personal holodeck, or wished you were the greatest swordfighter in the Metaverse, then this book is for you.
4 But all this cheerleading might not be assuaging your doubts. Maybe you feel the call of VR, but you (or your manager) don’t know if your project has the budget to include such frivolous features as virtual reality. Well here’s the great part. Supporting the Rift is cheap and easy, and we’re going to show you how to do it. Need a little more inspiration? Let’s take a look at what people are already doing with the Rift. 1.
5 While visiting the Eiffel Tower is possible in real life, visiting outer space is a bit out of reach for most people. That brings us to another one of our favorite demos, “Titans of Space” by Drash LLC (Figure 1.2). 4 Figure 1.2: Titans of Space by Drash LLC In Titans of Space, you can get a real feel for the beauty and vastness of space. VR can do more than just make you feel what it is like to be someplace else.
6 When you combine a virtual world, with thrills and goals, you’ve got what some consider the ultimate experience: immersive gaming. Team Fortress 2 6 was one of the first existing games to be updated with Oculus Rift support and is well worth a look. Figure 1.4: Team Fortress 2: One of the first games to be updated with Oculus Rift support Of course, not all of Rift experiments we’ve seen are fun and games. The Rift has also facilitated some serious work.
7 Figure 1.5: Two subjects in an experiment by Be Another Lab look down and see themselves as the other person thanks to a set of cameras and the Rift The Be Another Lab experiment allows researcher to get a view into human empathy that previously was simply not affordable to a lab on a modest budget. In even more practical terms, we think the Norwegian army 8 is taking an intriguing approach to using the Rift (Figure 1.6) to increase the safety of soldiers during combat. 8 http://www.tu.
8 Figure 1.6: An experiment using the Rift to allow tank drivers to drive with the hatch closed as seen in a report on Norwegian TV station TuTV In this experimental use of the Rift, there are cameras mounted on all sides of the tank. The images are then fed to a driver wearing the Rift inside the tank. The intent is to allow the driver to drive the tank with the hatch closed during combat situations. Ready to meet the Rift? Let’s go! 1.
9 • A 5v DC power adapter for US style power with international adapters for various other countries. In addition to the hardware, the kits also include the Oculus Rift Development Kit Instruction Manual. This manual covers basic usage of the headset along with some important health and safety notes. Please read and observe all precautions before using the Rift. For the most up-to-date health and safety information, please check the Oculus VR website 9.
10 Figure 1.7 The DK 2 headset: front, side and back views It has small adjustment wheels on the left and right side that allow you to move the display closer to or further from your face. There is foam padding on the surfaces intended to rest against the skin and straps that secure the Rift to your head. In addition to the normal ‘around the sides’ strap that you might find on any pair of ski goggles, an additional strap goes directly over the top of your head.
11 you are not getting a signal, see the troubleshooting section in the appendix on hardware setup.) The headset incorporates • a single 1920x1080 display • An inertial measurement unit (or IMU) that reports linear and angular acceleration as well as magnetic field strength and direction • several infrared lights. These lights are tracked by the included tracking-camera to provide user position data. • a built-in latency tester.
12 THE LENSES The DK2 model includes two pairs of lenses, termed the “A” and “B” lenses. The “A” lenses are for those with 20/20 vision and are installed in the headset by default. The “B” lenses (shown in Figure 1.9) are for those who are very nearsighted. Figure 1.9: The “B” Lenses The pairs are identical in terms of how they transmit light. How they differ is that they place the lens at slightly different distances from the actual display.
13 Figure 1.10: DK 2 positional camera The camera is connected to your computer via USB and to the headset using the included camera sync cable. The placement of the camera is very important to how well positional tracking will work. The camera should be placed about 5 feet from the headset and you should make sure that the camera has an unobstructed view of the headset at all times.
14 read and observe all precautions before using the Rift. For the most up-to-date health and safety information, please check the Oculus VR website 11. Let's now take a look at the parts of the DK 1. THE HEADSET The DK 1 headset, Figure 1.
15 The DK1 headset incorporates a single 1280x800 display at the heart of the device, as well as motion tracking hardware which reports acceleration, rotation rate, and magnetic field strength and direction. The display is split between both eyes (each eye can only see one half of the display), yielding 640 × 800 per eye as seen in Figure 1.12.
16 THE CONTROL BOX Figure 1.13 The Control Box: front and back views In addition to the cable extending to the headset, the control box, Figure 1.13, also has a DC power connector, a USB mini-B female port, and DVI and HDMI female ports. It has 5 buttons, one for power, and two each for controlling brightness and contrast on the display. It also has a small blue LED in the center of the Oculus VR logo that glows blue when the Rift display is active. THE LENSES Figure 1.
17 The DK1 model includes three pairs of lenses, pictured in Figure 1.14. The pairs are all identical in terms of how they transmit light. How they differ is that they place the lens at slightly different distances from the actual LCD display. (You can see this in Figure 1.14 by comparing their heights; the “C” lenses are visibly shallower than the “A” lenses.
18 • Providing a much wider field of view than conventional displays • Providing a different image to each eye • Blocking out the real environment around you, which would otherwise serve as a contradiction to the rendered environment. On the Rift display, we can display frames that have been generated to conform to this wide field of view and offer a distinct image to each eye.
19 Figure 1.15: The typical loop for conventional applications The details can be abstracted in many ways, but for just about any program you can eventually look at it as an implementation of this loop. For as long as the application is running, it responds to any user input, renders a frame and outputs that frame to the display. RIFT APPLICATIONS Rift-specific applications embellish this loop, as seen in Figure 1.16. Figure 1.
20 In addition to the rendering step, we also need to distort it to account for the effects of the lenses on the Rift. Practically speaking, the head pose is really a specialized kind of user input, and the Riftrequired distortion is part of the overall process of rendering a frame, but we’ve called them out here as separate boxes to emphasize the distinction between Rift and non-Rift applications.
21 with an array of infrared lights built into the headset itself, this allows the position of the Rift to be tracked, as long as it is within view of the camera. Beyond the hardware, the Oculus SDK includes support for taking the raw messages from the hardware and coalescing them into a single head pose.
22 Figure 1.18: Comparison of the field of view of a typical monitor with that of the Rift The Rift achieves this high field of view through the placement of the display and the use of special lenses. Inside the Rift is a small high-resolution LCD display, and wearing the device places the display directly in front of you at a distance of about 4 centimeters, as seen in Figure 1.19.
23 Figure 1.19 Panel and lens position. The resolution listed is for the DK2. Between your eyes and the display panel are lenses designed to distort light in such a way as to both make the apparent focal depth infinitely far away (resolving the focus issue) and to make the panel appear much wider than it is, further increasing the field of view (Figure 1.20). Figure 1.
24 MAXIMIZING THE RIFT’S FIELD OF VIEW To understand how to maximize the Rift’s field of view, you need to look at how the brain interprets the images perceived by the eyes. Each eye covers its own field of view, with the left eye showing more of the world to your left, and the right showing more to your right, with a large amount of crossover at the center, as shown in Figure 1.21. Figure 1.21: Left and right field of view Your brain takes the two images and fuses them into a single panoramic view.
25 Figure 1.22: Left and right images are fused by your brain to create a single image To maximize the field of view in the Rift, the images presented to each eye need to mimic real vision in that more data is presented on the left of the image for the left eye and more data is presented on the right for the right eye. RENDERING FOR THE RIFT To render images properly for the Rift means you need to take into account the LCD display, how vision works with separate images for each eye, and the lenses used.
26 Figure 1.23: A grid of lines as they would appear on the screen and through the Rift lenses To counter the pincushion effect, software needs to apply the inverse (“barrel”) distortion to the source image before sending it to the Rift, as seen in Figure 1.24. Now the image appears as intended in the Rift. Figure 1.24: The source image, the same image after an inverse distortion has been applied, and then the distorted image as seen through the Rift.
27 Figure 1.25: A screenshot of the Oculus Tuscany demo as seen on a conventional monitor Now that you understand how the Rift works, time to get started developing for it. 1.5 Setting up the Rift for development Getting the Rift setup for use is pretty well documented in the Oculus Rift Development Kit Instruction Manual that comes with the kit and if you follow those instructions, you should be able to get the Rift up and running.
28 type of person who never gets motion sickness in real life, you still might get motion sickness from using a Rift. Even when following all precautions, you may still feel motion sickness. Here are some strategies to use when you first start working with the Rift that can help with motion sickness: • Do not try to power through it. If you experience nausea or other symptoms of motion sickness, stop and take a break right away. We cannot stress enough how important this is.
29 • Work in a quiet environment. Ambient noise can interfere with how your brain perceives the images on screen and can trigger motion sickness in some people. Spatialized sounds that disagree with your virtual environment send conflicting messages that your mind will struggle to contextualize. Try working in a quiet environment or wearing noise-canceling headphones to remove ambient noise. • Watch what you eat before strapping on the Rift. This is common sense.
30 1.7 Development Paths This book covers several different approaches to Rift development. • Using the C APIs: If you plan to work directly with the Rift C APIs, head to part 2 and start with chapter 2. It will introduce you to the C API and get you started writing your first Rift integrations.
31 customize the output to conform to the Rift’s display will provide a good experience. • Using the Rift may cause motion sickness. There are some simple tips that can be used to manage any symptoms of motions sickness you might feel when using the Rift. • When using the Rift, be sure to use the right lenses for your vision, set the distance between the lenses and your face, and adjust the straps to ensure a perfect headset fit.
