Expect 2012 to be a transformational year for HMDs

2011 was a 'business as usual' year as HMDs go. Sure, there were some new products and product improvements, but nothing earth shattering. At the same time, improvements in displays, motion sensing, augmented reality and processing power are coming together to enable 2012 to be truly transformational.

Sure, some nice progress was made in 2011 in HMDs:

• Higher resolution OLEDs became part of a new generation of lighter, brighter HMDs such as the zSight.
• Low-latency wireless video is moving into mainstream. I believe it will become a common accessory to HMDs, much like head trackers.
• In general, companies like Sensics continued to improve their optics designs, packaging, level of integration and features of professional HMDs.

Having said that, I believe that what's truly going to impact HMDs - the professional models - and VR goggles - the consumer models in 2012 were recent improvements made outside the direct HMD realm:

• Lower-cost micro-displays of 'good enough' resolution are becoming available. Once 720P displays are available for ~50$, the economics of goggles change, because displays have historically been a key cost driver for the HMD/goggle cost. What's the difference between a $10K HMD and a $15K HMD? $5K, of course, but otherwise there is no big difference in market penetration. It's hard to see why a $10K price would suddenly cause cause skyrocketing quantities whereas a $15K HMD did not. However, the difference between a $1K goggle and a $6K HMD, or better yet a $500 goggle to a $6K HMD would be huge as the $500 product would suddenly allow an entirely new group of users and entirely new class of goggle applications.
• Motion tracking is improving - both inside and outside the HMD. Driven by new generations of cell phones, tablets and game controllers, embedded motion trackers with 6 degrees of freedom (X/Y/Z, yaw/pitch/roll) are becoming good enough and inexpensive enough to make it into practically every product. Not only does this this make them an automatic add-on to products, but their increased quantity in the market opens up new ideas and new development techniques to use them. Outside the HMD, the success of the Wii and the Kinect made it obvious how powerful gesture-based and natural interaction are, and these developments will soon make it into goggles, such as the new SmartGoggles offering.
• Just like micro-displays are experiencing their version of Moore's law, embedded CPUs are continuing their march. New phones and tablets these days could have 1.5 GHz dual-core processors with embedded graphics and 3D accelerators. This is a very powerful computing platform. Because they are embedded in phones and tablets, these processors have to be lower-cost, power-efficient and small - all a perfect fit to embedding them into or alongside goggles. At the same time, having an open operating system like Android allows support of new hardware platforms as well as a nearly-endless supply of apps and content to consume.
• The improvements in processing power as well as cost and resolution of embedded cameras have given a huge boost to augmented reality applications. Driven by phones and tablets, both the applications and the knowledge gathered while building them will become very useful for goggles.

All taken together, a new generation of goggles will appear. If the typical goggle in 2010 was a lens, a micro-display and a video cable, it will evolve to be much more in 2012/2013. Combine on-board processing, embedded motion tracking for head, hands and others, augmented reality power and lower-cost micro-displays, and the mindless HMD will now morph into something between a fully-fledged independent computing and user interface platform to at least a highly-sophisticated display and virtual reality co-processor for phones, tablets, consoles or PCs.

Sure, there will still be a market for traditional HMDs just like there is still a market for flip phones. Sometimes, users don't need the sophistication of the Smart Phone, or don't want to pay for the data plan or have some ruggedization requirement that can only be satisfied by a flip-phone. Similarly, a special defense packaging, an ultra-low cost or some other requirement will continue to keep traditional HMDs in the market for years. However, I believe true growth will come from the smarter goggles that will take advantage of the above technologies to create a more complete and compelling user experience.

Happy Near Year!

The SmartGoggle

Today, my company introduced the SmartGoggle, the solution for the common virtual reality goggle.
The issue with today's goggles is obvious: they are mindless; just a monitor on your head. Just like a monitor has a video input and will display almost anything you pipe into that video input, a traditional goggle will display that signal in front of your eyes. If you provide two signals, or side-by-side video, it can do so in stereo as well.

But a traditional goggle, like a monitor, requires an external video source. It can be a computer, in which case you can do something interactive. It can be your iPod, so you can use the goggle as a media viewer. Useful, but boring.

While all of this is going on, tablets and smart phones have become increasingly powerful in their computing and graphics capabilities. So, as we were thinking about how to make goggles better, it became obvious that putting some good processing power inside the goggles could do a lot of good. For one thing, you could run applications on the goggles and not have to carry around an external computer with you. The goggles, and these applications, could go with you anywhere, which is cool. These could be connected applications - streaming something from the Web, or these could be local applications such as those using an on-board camera to drive an augmented reality app. Yes, you should still be able to use an external video source to drive your goggle, but now you can also drive it 'from the inside'. This is a bit like the new crop of smart TVs that are becoming very popular. You could connect them to your cable provider or DVD player, but they can also stream a Netflix movie or surf the web using an on-board processor. So far, so good.

The problem of the user interaction still remains. Sure, you can put a head tracker so that the view can change as you rotate your head, but this is typically not enough for true interaction with the application. You can use an external device - a phone, joystick or even a Kinect - as input, but these approaches have limitations: you have to carry them with you; you have to stand in front of a sensor; you are limited in your tracking area; good, but not good enough for goggles.

We then started thinking: what if we put a camera that can track your hands and make that information - both in raw form as well as after gestural analysis - available to the application running on the goggle. That might be cool, because the hands go with you everywhere and because cameras on your head can often see your hands regardless of where you are and which direction you are looking at. Better yet, if we put an array of cameras on the head, we can get depth perception of the hand location as well as get a really wide tracking area. We call this 'first person hand tracking'.

Last, we realized that many goggles are essentially the same on the inside. Sensics, for instance, makes a commercial goggle and then repackages it into a different enclosure for training and simulation applications. Anyone wanting to build a goggle will need to cover several areas: driving displays; head tracking; video processing and more. Given this, it made sense to design a module that essentially encapsulates all these functions and allows goggle developers to focus on the design/styling aspects of the goggles rather than on building everything from scratch time and again.

We've been working on this for a while and are excited with the progress and the initial feedback we are receiving. If you combine these three innovations: on-board Android machine, real-time hand tracking from a first-person perspective, and a 'system on a module' approach for encapsulating most of what's needed to build a goggle, we think you get something. A SmartGoggle.

Remember when the iPhone came out and suddenly people started realizing that it's not just a phone, but much more than that? We think SmartGoggles can be to mindless goggles what smart phones are to flip phones. A major step forward, which we are very excited to take today.

HMDs are mindless - a monitor on your head

My company makes VR goggles (professionally called: HMDs), so when I say that HMDs are mindless (or stupid or unintelligent or dumb), it is with a somewhat heavy heart.

After all, HMDs are sophisticated devices. To build one, you need to understand optics, electronics, ergonomics, mechanical design and system engineering. You need to balance features with field of view with weight and comfort. HMDs are indeed sophisticated, but nevertheless mindless.

I call HMDs mindless because they are not much more than a fancy monitor on your head. Like a monitor, you need to connect an external video source: a computer, MP4 player, phone or tablet, to provide a signal to be viewed. If the video signal is provided by a cable, it limits your mobility. If it provided via a wireless video link, it cuts the cable but still provides limits on your distance from the video source.

The passive nature of HMDs is not just because it's a 'monitor on your head'. Unless you are content with using the HMD as a media viewer, you will want to interact somehow with the content and with the HMD. In today's products, you might have some push buttons on the HMD and you usually have a head orientation tracker than can let the application know where you are looking. This is a decent start, but most of the user interface experience - selecting menus, interacting with content, moving 3D objects around, still relies on external devices: a joystick, a mouse, a data glove and others. By the way, many wireless video links don't have the ability to send head tracking information back to the video source.

Sure, this is fine for some applications. Military training and simulation applications are sometimes OK with having the soldier being trained carry a computer on his back. Stationary applications (a tank trainer) or those relying on lots of peripheral equipment (such as an academic research project measuring brain activity) can greatly benefit from today's HMDs, but is this enough for widespread use?

The promise of HMDs was to be able to take 3D content anywhere and interact with it in a useful way. Though more complex, it is just like how the iPod allowed us to take our music library with us anywhere and sufficiently interact with it to be useful. I don't think today's HMDs are fulfilling this promise for a broad-enough market.

Something needs to be done about it.

.