VRguy Podcast Episode: Jason Jerald, Principal Consultant at NextGen Interactions

Jason and I talk about VR sickness, precision input, and design tradeoffs in VR interactions. We have a particularly interesting discussion about pen input in VR

Listen to the Podcast or read the transcript at this link 

Interview on Redirected Walking with Professor Eric Hodgson

Prof. Eric Hodgson

My previous post regarding redirected walking generated a good bit of interest, so I decided to dive deeper into the subject by interviewing Prof. Eric Hodgson of Miami University in Ohio, a true expert on the subject.

Eric, thank you for speaking with me. For those that do not know you, please tell us who are you and what do you do?

I'm a psychologist and professor at Miami University (the original one in Ohio, not that *other* Miami). I split my time between the Psychology department -- where I use virtual environments to study spatial perception, memory, and navigation -- and the interdisciplinary Interactive Media Studies program, where I teach courses in 3D modeling, data visualization, and virtual reality development to students from all across the university. I help oversee two sister facilities at Miami, the HIVE (Huge Immersive Virtual Environment) and the SIVC (Smale Interactive Visualization Center). The HIVE is a HMD-based facility with an 1,100 square meter tracking area. The SIVC houses a 4-walled CAVE along with several other 3D projection systems, immersive desktops, development labs, and several motion-capture systems. The HIVE has been funded mostly by the National Science Foundation, the Army Research Office, and the Ohio Board of Regents. The Smale Center was started with a $1.75m gift from the late John Smale, a former CEO of Proctor & Gamble, which uses CAVEs and other visualization systems in their R&D cycle.

You are the director of Smale Interactive Visualization Center. What kind of work is being performed at the Center?

It's a multi-disciplinary center, and a large part of my job is to enable students and faculty from across to university to leverage VR for their work, especially if they don't have the skillset to do it themselves. We also work with regional, national, and international industry partners on specific projects. The work can vary widely, which I find interesting and encouraging -- VR is becoming a very general-purpose tool rather than a niche for a few narrow fields. One of our first projects was building an immersive, 3D Mandala for the Dali Lama for his visit to campus. We've also done motion capture of proper violin-playing arm motion for the music department, developed medical training simulations for the nursing program, developed experiments to study postural sway with psychology, done interactive virtual walk-thoughs of student-designed architectural projects, supported immersive game development, and done work on developing next-generation motion sensing devices and navigation interfaces. Not to mention a 3D visualization of 18th century poetry, which was a collaboration between the Center, the English department, Computer Science, and Graphic Design. I love my job. I also do a lot of tours, field trips, and workshops. When you have a CAVE, a zSpace, a pile of HMDs, and lots of other fun toys (um... I mean tools), you end up being a must-see stop on the campus tour.

A good portion of your research seems to be in the area of redirected walking. Can you explain, in a lay person’s terms, what is redirected walking?

In layman's terms, Redirected Walking is a way of getting people into walking in circles without them realizing it, while it looks like they are walking in a straight line visually. Virtual environments and game levels can be very big; tracking spaces in a lab are typically pretty small. Redirected walking lets immersed users double-back into the same physical space while traveling through a much larger virtual space. There are other techniques that can come into play, such as magnifying or compressing turns, or stretching distances somewhat, but the basic techniques are all aimed at getting people to double back into open physical space so they can keep walking in the virtual environment. It's a bit like the original holodeck on Star Trek... users go into a small room, it turns into an alternate reality, and suddenly they can walk for miles without hitting the walls.

What made you become interested in this area?

Necessity, mostly. I'm a psychologist, studying human spatial cognition and navigation. My colleagues and I use a lot of virtual environments and motion tracking to do our research. VEs allow us to have complete control over the spaces people are navigating, and we can do cool things like moving landmarks, de-coupling visual and physical sensory information, and creating geometrically impossible spaces for people to navigate through. Our old lab was a 10m X 10m room, with a slightly smaller tracking area. As a result, we were stuck studying, essentially, room-sized navigation. There are a lot of interesting questions we could address, though, if we could let people navigate through larger spaces. So, we outfitted a gymnasium (and later a bigger gymnasium) with motion tracking that we called the HIVE, for Huge Immersive Virtual Environment. We built a completely wearable rendering system with battery-powered HMDs, and viola... we could study, um, big-room sized navigation. Since that still wasn't enough, we started exploring Redirected Walking as a way to study truly large-scale navigation in VEs with natural walking.

It seems that one of the keys to successful redirection is providing visual cues that are imperceptible. Can you give us an example of the type of cues and their magnitude?

Some of the recent work we've done uses a virtual grocery store, so I'll use that as an example. Let's say you're immersed, and trying to walk down an isle and get to the milk cooler. I can rotate the entire store around an axis that's centered on your head, slowly, so that you'll end up veering in the direction I rotate the store (really we manipulate the virtual camera, but the end result is the same as rotating the store). The magnitude of the rotation scales up with movement speed in our algorithm, so if you walk faster -- and thus create more optic flow -- I can inject a little bit more course correction. The rotations tend to be on the order of 8 - 10 degrees per second. By comparison, when you turn and look around in an HMD, you can easily move a few hundred degrees per second. You could detect these kind of rotations easily if you were standing still, but while walking or turning there's enough optic flow, head bob, and jarring from foot impact that the adjustments get lost in all the movment. Our non-visual spatial senses (e.g., inertial sensing by the inner ear, kinesthetic senses from your legs, etc.) have just enough noise in them that the visuals still appear to match.

Are all the cues visual or are the auditory or other cues that can be used?

Right now the cues we use are primarily visual, and to a lesser extent auditory, but it's easy to add in other senses if you have the ability. Since 3D audio emanates from a particular location in the virtual world, not the physical world, rotating the visuals brings all of the audio sources along with it. An active haptics display could work the same way, or a scent generator. Redirected walking essentially diverts the virtual camera, so any multisensory display that makes calculations based on your position in the virtual world will still work as expected. Adding more sensory feedback just reinforces what your eyes are seeing and should strengthen the effect.

What are practical applications of redirected walking? Is there a case study of someone using redirected walking outside an academic environment?

A gymnasium is about the smallest space you can work with and still curve people around without them noticing, so this is never going to run in your living room. We do have a portable, wearable version of our system with accurate foot-based position tracking that can be taken out of lab and used in, say, a park or a soccer field. It's a bit tricky, though, since the user is essentially running around the great outdoors blindfolded. If you're a liability lawyer for a VR goggle manufacturer, that's the kind of use case that gives you nightmares, but redirected walking could actually work in the gaming market with the right safety protocols. For example, we have safety mechanisms built in to our own systems, which usually includes a sighted escort and an automated warning system when users approach a pre-defined boundary. This could work in a controlled theme park or arcade-type setting, or with home-users that use some common sense. I can also see this technique being useful in industry and military applications. For example, the portable backpack system could easily be used for mission rehearsal in some remote corner of the globe. A squad of soldiers could each wear their own simulation rig and have their own ad-hoc tracking area to move around in. Likewise, some industry simulations incorporate large spaces and can benefit from physical movement. One scenario that came up a few years ago related to training repair technicians for large oil refineries, which can cover a square mile or more. Standing in a CAVE and pushing the forward button on a joystick just doesn't give you the same experience of having to actually walk a thousand meters across the facility while carrying heavy equipment, and then making a mission-critical repair under time pressure. Redirected walking would increase the realism of the training simulation without requiring a mile-square tracking area. Finally, I can see this benefiting the K-12 education system. Doing a virtual field trip in the gym would be pretty cool, and a responsible teacher or two could be present to watch out for the kids' safety.

Can redirected walking be applicable to augmented reality scenarios or just to immersive virtual reality?

It really doesn't make sense with augmented reality, in which you want the real and virtual worlds to be as closely aligned as possible. With redirected walking, the relationship between the real and virtual diverges pretty quickly. If you're doing large-scale navigation in AR, such as overlaying underground geological formations through a drill site, you'll want to actually navigate across the corresponding real-world space. It could make sense in some AR game situations, but it would be hard to make any continual, subtle adjustments to the virtual graphics without making them move perceptably relative to the real-world surroundings.

Is this technique applicable also to multi-person scenarios?

Definitely, and that's something we're actively exploring now. As long as you're redirecting people, and effectively steering where they go in the real world, there's no reason not to take several immersed people in the same tracking space and weave them in and around each other. Or, as I mentioned above with our portable system, if you can reliably contain people to a certain physical space with redirection, you can spread people out across a field and let everyone have their own little region while traveling through extremely large VEs. Adding multiple users does add some unexpected complexities, however. Under normal conditions, for example, when two immersed users come face to face in the VE, they would also be face to face in the physical world, and they could talk to each other normally, or reach out and touch each other, or share tools, etc. With redirected walking, those same users could be tens or hundreds of meters apart in the real world, requiring some sort of VOIP solution. By the same token, someone who is a mile away virtually might actually be very close to you, and you could hear them talking but not be able to see them, leading to an Uncanny Valley scenario.

How large or how small can a physical space be to implement successful redirected walking? Can this be used in a typical living room?

The HIVE is about 25m across in its narrowest dimension, which is about as small as you'd want to go. This is definitely not living-room material, which is where devices like the Omni will thrive instead. A lot of the literature recommends a space with a minimum radius of 30m+, which I think is about right. We have to stop people occasionally who are on a collision course with one of the lab's walls. A slightly larger space would let us catch and correct those trajectories automatically instead of manually stopping and redirecting the user. One thing to note is that the required tracking space interacts a lot with how much you turn up the redirection -- higher levels of steering constrain people to a smaller space, but they also become more noticeable. The type of VE and the user's task can also play a role. It seems like close-in environments like our virtual store make redirection more perceptible than open, visually ambiguous VEs like a virtual forest.

How immersive does an experience need to be for redirected walking to be successful?

High levels of immersion definitely help, but I'm not sure there's a certain threshold for success or failure here. Redirection relies on getting people to focus on their location in the virtual world while ignoring where they are in the room, and to innately accept their virtual movement as being accurate, even though it's not. Anytime you're in a decent HMD with 6-DOF tracking, the immersion level is going to be fairly high anyways, so this turns out to be a fairly easy task. As long as redirection is kept at reasonably low levels, it has been shown to work without being noticed, without increasing simulator sickness, and without warping people's spatial perception or mental map of the space.

Can you elaborate a bit on plans for future research in this area?

Right now the focus in our lab is on implementing mutli-user redirection and on improving the steering algorithms we use. We're also looking at behavioral prediction and virtual environment structure to try and predict where people might go next, or where they can't go next. For example, if I know you're in a hallway and can't turn for the next 10m, I can let you walk parallel to a physical wall without fear that you'll turn suddenly and hit it. There's a lot of other research going on right now in other labs that explore the perceptual limits of the effect and alternative methods of redirecting people. For example, it's possible to use an effect called "change blindness" to essentially restructure any part of the environment that's out of a person's view. So, if I'm looking at something on a virtual desk, the door behind me might move from one wall to another, causing me to leave alter my course by 90 degrees when I move to a different area. There's also a lot of work that's been done on catching potential wall collisions and gracefully resetting the user without breaking immersion too much.

For those that want to learn more about redirected walking, what other material would you recommend?

I'd really recommend reading Sharif Razzaque's early work on the topic, much of which he published with Mary Whitton out of UNC Chapel HIll. (http://scholar.google.com/scholar?q=razzaque+redirected+walking&btnG=&hl=en&as_sdt=0%2C36)

I'd also recommend reading some of Frank Steinicke's recent work on the techniques and perceptable limits of redirection (http://img.uni-wuerzburg.de/personen/prof_dr_frank_steinicke/), or some of our lab's work comparing higher-level redirection strategies such as steering people towards a central point versus steering people onto an ideal orbit around the room (http://www.users.miamioh.edu/hodgsoep/publications.php).

Finally, there's a good book that just came out on Human Walking in Virtual Environments that contains several good chapters on redirection as well as a broader look at the challenges of navigating in VEs and the perceptual factors involved. (http://www.amazon.com/Human-Walking-Virtual-Environments-Applications/dp/1441984313).

Eric, thank you very much for speaking with me. I look forward to learning more about your research in the future.

An interview with Ben Lang of RoadToVR

I see greater and greater opportunities for intersection between professional VR equipment and consumer VR gear. As such, I had the chance to interview Ben Lang who has been actively writing on consumer VR at his blog roadtovr.com for quite some time.

Ben, thank you for speaking with me. For those that do not know you, please tell us who are you and what do you do?

My name is Ben Lang, I've been entrenched in the tech blogging world for around six years now. Much of my prior work was with mobile tech (smartphones and the like). I started Road to VR in 2011 to chart the world's progress toward true virtual reality. I've been a life long gamer and gaming industry observer. Much of my desire to see VR happen has to do with experiencing games in new and exciting ways, though I realize that VR has much more to offer.

What attracted you most to virtual reality?

The infinite possibilities are what draws me to learning everything I can about virtual reality. With a true VR system (a la Matrix), you could essentially do anything. You could rewrite the laws of physics if you felt like it. VR is a platform to make your imagination come to life -- a blank canvas upon which to invent your own paradise -- who wouldn't be attracted to that!

Are you a gamer? What are your favorite games? What are your favorite goggle-enabled games? Which games would you most like to see VR-enabled?

I've been gaming for pretty much my entire life. While I don't have as much time to game these days as I'd like -- and I'm not one of those people who owns every console and has played every game -- I've always try to keep my finger on the pulse of the game industry -- in fact, I might spend more time reading and learning about the game industry than playing games!

Mega Man X is one of my favorite early games. In the modern generation I've been a longtime Halo player (or maybe I should say 'was', now that Halo 4 derailed the series after changing developers). At this point I'm looking forward most to Bungie's latest title, Destiny. And like everyone else, I'm still waiting on Half-Life 3.
Since it's still the early days of consumer VR, there aren't many full-fledged games out on the market supporting HMDs with headtracking. I've played Valve's Half-Life 2 with Oculus Rift support. It's a pretty interesting experience -- much more immersive than on a 2D monitor, but the game was never intended to be used with VR so there are some less than desirable facets of gameplay which can cause problems in VR. Things like sprinting, falling, and quickly turning can lead to nausea.

That said, some of the cooler Oculus Rift demos out there are Sixense's Tuscany (with Razer Hydra support), and Titans of Space. I'm also very much looking forward to two forthcoming Rift-ready titles, The Gallery: Six Elements and Among the Sleep.

Naturally, I'd love to see Halo in VR. In the game you control a supersoldier in futuristic combat armor. The lore of the game is such that the HUD elements are supposed to be part of the character's helmet and suit interface. He also has an AI partner which inhabits the suit and helps him on missions. It would be neat to feel like you were really in the suit, looking through the helmet, and hear the AI character's voice as though it was coming from within the suit.

Today, most VR gaming is with a PC. Do you envision a big market for goggles that are paired with a smartphone or tablet?

It does seem that in the near future we'll have smartphones and tablets capable of creating decent VR experiences, and I fully expect a market to emerge. For one, it would allow for portable VR experiences that would inherently untether you from your desk, allowing you to play a game and actually walk around in a large space without getting tangled.

The portability would also be great for productivity. Today many of us use laptops to work from one place to the next, but the size of the screen often makes the productivity experience feel like a compromise With VR you could have a full virtual reality desktop, that's bigger than any laptop screen, and it would be accessible with some goggles and a smartphone, affording you a huge virtual productivity space in almost any location.

What has been your experience, or the experience of those that you have spoken to, regarding using goggles for a long time. Do you expect people to use goggles for hours and hours just like they play in front of a PC?

I've given a number of Oculus demos and what I see is that some people are affected more than others when it comes to simulator sickness. In my experience, and what I've heard from others, is the more you use VR, the more you get used to it and avoid any nausea for the most part -- though there are still some things you can do which will make anybody dizzy no matter how experienced they are.
As long as the ergonomics are right, I could certainty see people spending extended periods of time inside VR games.

Looking 18-24 months into the future, do you think the hottest VR game is an existing game that has been well-adapted to VR, or is it an entirely new game or experience?

Definitely a new game. There are lots of great titles out there now, but the best games will almost certainly be designed specifically for VR. For one they need to be designed with gameplay that doesn't make people sick. Also, taking away the mouse and keyboard in favor of a more natural way of interacting, like the Razer Hydra, adds a lot to the immersive experience even on top of using an HMD. New games will be able to use such natural interactions to do stuff that wouldn't have been possible in traditional games.

The opportunity for new genres will be present too. While action-paced FPS games are a staple of the current generation of games, VR offers a new perspective. Something like a murder mystery game might work really well in VR. Imagine being able to kneel down, reach out with your virtual hand, and dust for fingerprints at a crime scene. That kind of interaction is something you'd normally see in a cutscene, but rarely something that the gamer would actually be involved in.

There is work on wearable accessories to enhance the gaming experience, such as a haptic vest that might allow you to feel when you are hit. How open would you be to wearing such accessories when playing games?

I'd be very willing to wear that kind of accessory, so long as it's reasonably comfortable, and I think that such a vest will be an important part of the VR equipment puzzle. I see anything that enhances the level of immersion as a positive. With a proper vest, explosions and gun shots are obvious and exciting. But imagine feeling a hug from a virtual character... or even from another avatar who is controller by a real person; suddenly you've opened up a new channel of communication between real and virtual; game designers and storytellers will use those new channels to tell ever more compelling stories.

What do you feel gamers want beyond affordable high-quality goggles?

A way to walk in VR. One of the questions I get with almost everyone I put into the Sixense Tuscany Oculus Rift demo is "can I actually walk around in here?" The sense of presence is so great that it's natural to want to stroll around. Sadly I have to tell them that their walking is controlled by a thumbstick!
Fortunately there are some interesting consumer VR locomotion solutions in the works, like the Virtuix Omni and the WizDish.

What's your stance on AR vs. VR? Is one more useful than the other? Is one bound to come before the other?

I find VR inherently more useful because it offers unlimited possibilities in a fully virtual space, whereas AR is constrained by reality. Still, AR has the potential to be highly useful in our daily lives, advising us on everything from traffic to cooking instructions -- though I think the world has yet to see a much needed 'killer app' for AR.
In the near term, VR and AR are aiming for very different use-cases. In the longer term, when AR and VR are perfected, the terms will become largely irrelevant as we'll all be wearing devices that are capable of both.

What have you seen in the professional virtual reality market that you would like to see ported to the consumer world?

Professional motion capture systems are an ideal VR interface. Unfortunately they are complex and are cost-prohibitive. A system that is inexpensive and easy to set up would let you transport your every action into your favorite game.

Let’s say you were advising Samsung on how to best create a consumer gamer goggle, building on their consumer-electronics expertise and high-quality smartphone displays. What would you recommend that Samsung do?

Focus on the experience first. It doesn't matter what your HMD looks like, what optical solution you use, or what specs you put on the box -- if it doesn't live up to the promise, and isn't affordable, it won't go anywhere. Gamers want a wide field of view, extremely responsive head tracking, and high resolution.
Content is also extremely important. Getting developers to make content compatible with your HMD is a chicken and egg problem; developer funds and contests are great ways to seed early development to ensure that your platform has content worth playing.

Ben, thank you very much. Looking forward to reading more of your thoughts on roadtovr.com

UPDATE: Ben has now interview me on his site