Expect 2012 to be a transformational year for HMDs

2011 was a 'business as usual' year as HMDs go. Sure, there were some new products and product improvements, but nothing earth shattering. At the same time, improvements in displays, motion sensing, augmented reality and processing power are coming together to enable 2012 to be truly transformational.

Sure, some nice progress was made in 2011 in HMDs:

• Higher resolution OLEDs became part of a new generation of lighter, brighter HMDs such as the zSight.
• Low-latency wireless video is moving into mainstream. I believe it will become a common accessory to HMDs, much like head trackers.
• In general, companies like Sensics continued to improve their optics designs, packaging, level of integration and features of professional HMDs.

Having said that, I believe that what's truly going to impact HMDs - the professional models - and VR goggles - the consumer models in 2012 were recent improvements made outside the direct HMD realm:

• Lower-cost micro-displays of 'good enough' resolution are becoming available. Once 720P displays are available for ~50$, the economics of goggles change, because displays have historically been a key cost driver for the HMD/goggle cost. What's the difference between a $10K HMD and a $15K HMD? $5K, of course, but otherwise there is no big difference in market penetration. It's hard to see why a $10K price would suddenly cause cause skyrocketing quantities whereas a $15K HMD did not. However, the difference between a $1K goggle and a $6K HMD, or better yet a $500 goggle to a $6K HMD would be huge as the $500 product would suddenly allow an entirely new group of users and entirely new class of goggle applications.
• Motion tracking is improving - both inside and outside the HMD. Driven by new generations of cell phones, tablets and game controllers, embedded motion trackers with 6 degrees of freedom (X/Y/Z, yaw/pitch/roll) are becoming good enough and inexpensive enough to make it into practically every product. Not only does this this make them an automatic add-on to products, but their increased quantity in the market opens up new ideas and new development techniques to use them. Outside the HMD, the success of the Wii and the Kinect made it obvious how powerful gesture-based and natural interaction are, and these developments will soon make it into goggles, such as the new SmartGoggles offering.
• Just like micro-displays are experiencing their version of Moore's law, embedded CPUs are continuing their march. New phones and tablets these days could have 1.5 GHz dual-core processors with embedded graphics and 3D accelerators. This is a very powerful computing platform. Because they are embedded in phones and tablets, these processors have to be lower-cost, power-efficient and small - all a perfect fit to embedding them into or alongside goggles. At the same time, having an open operating system like Android allows support of new hardware platforms as well as a nearly-endless supply of apps and content to consume.
• The improvements in processing power as well as cost and resolution of embedded cameras have given a huge boost to augmented reality applications. Driven by phones and tablets, both the applications and the knowledge gathered while building them will become very useful for goggles.

All taken together, a new generation of goggles will appear. If the typical goggle in 2010 was a lens, a micro-display and a video cable, it will evolve to be much more in 2012/2013. Combine on-board processing, embedded motion tracking for head, hands and others, augmented reality power and lower-cost micro-displays, and the mindless HMD will now morph into something between a fully-fledged independent computing and user interface platform to at least a highly-sophisticated display and virtual reality co-processor for phones, tablets, consoles or PCs.

Sure, there will still be a market for traditional HMDs just like there is still a market for flip phones. Sometimes, users don't need the sophistication of the Smart Phone, or don't want to pay for the data plan or have some ruggedization requirement that can only be satisfied by a flip-phone. Similarly, a special defense packaging, an ultra-low cost or some other requirement will continue to keep traditional HMDs in the market for years. However, I believe true growth will come from the smarter goggles that will take advantage of the above technologies to create a more complete and compelling user experience.

Happy Near Year!

The SmartGoggle

Today, my company introduced the SmartGoggle, the solution for the common virtual reality goggle.
The issue with today's goggles is obvious: they are mindless; just a monitor on your head. Just like a monitor has a video input and will display almost anything you pipe into that video input, a traditional goggle will display that signal in front of your eyes. If you provide two signals, or side-by-side video, it can do so in stereo as well.

But a traditional goggle, like a monitor, requires an external video source. It can be a computer, in which case you can do something interactive. It can be your iPod, so you can use the goggle as a media viewer. Useful, but boring.

While all of this is going on, tablets and smart phones have become increasingly powerful in their computing and graphics capabilities. So, as we were thinking about how to make goggles better, it became obvious that putting some good processing power inside the goggles could do a lot of good. For one thing, you could run applications on the goggles and not have to carry around an external computer with you. The goggles, and these applications, could go with you anywhere, which is cool. These could be connected applications - streaming something from the Web, or these could be local applications such as those using an on-board camera to drive an augmented reality app. Yes, you should still be able to use an external video source to drive your goggle, but now you can also drive it 'from the inside'. This is a bit like the new crop of smart TVs that are becoming very popular. You could connect them to your cable provider or DVD player, but they can also stream a Netflix movie or surf the web using an on-board processor. So far, so good.

The problem of the user interaction still remains. Sure, you can put a head tracker so that the view can change as you rotate your head, but this is typically not enough for true interaction with the application. You can use an external device - a phone, joystick or even a Kinect - as input, but these approaches have limitations: you have to carry them with you; you have to stand in front of a sensor; you are limited in your tracking area; good, but not good enough for goggles.

We then started thinking: what if we put a camera that can track your hands and make that information - both in raw form as well as after gestural analysis - available to the application running on the goggle. That might be cool, because the hands go with you everywhere and because cameras on your head can often see your hands regardless of where you are and which direction you are looking at. Better yet, if we put an array of cameras on the head, we can get depth perception of the hand location as well as get a really wide tracking area. We call this 'first person hand tracking'.

Last, we realized that many goggles are essentially the same on the inside. Sensics, for instance, makes a commercial goggle and then repackages it into a different enclosure for training and simulation applications. Anyone wanting to build a goggle will need to cover several areas: driving displays; head tracking; video processing and more. Given this, it made sense to design a module that essentially encapsulates all these functions and allows goggle developers to focus on the design/styling aspects of the goggles rather than on building everything from scratch time and again.

We've been working on this for a while and are excited with the progress and the initial feedback we are receiving. If you combine these three innovations: on-board Android machine, real-time hand tracking from a first-person perspective, and a 'system on a module' approach for encapsulating most of what's needed to build a goggle, we think you get something. A SmartGoggle.

Remember when the iPhone came out and suddenly people started realizing that it's not just a phone, but much more than that? We think SmartGoggles can be to mindless goggles what smart phones are to flip phones. A major step forward, which we are very excited to take today.

HMDs are mindless - a monitor on your head

My company makes VR goggles (professionally called: HMDs), so when I say that HMDs are mindless (or stupid or unintelligent or dumb), it is with a somewhat heavy heart.

After all, HMDs are sophisticated devices. To build one, you need to understand optics, electronics, ergonomics, mechanical design and system engineering. You need to balance features with field of view with weight and comfort. HMDs are indeed sophisticated, but nevertheless mindless.

I call HMDs mindless because they are not much more than a fancy monitor on your head. Like a monitor, you need to connect an external video source: a computer, MP4 player, phone or tablet, to provide a signal to be viewed. If the video signal is provided by a cable, it limits your mobility. If it provided via a wireless video link, it cuts the cable but still provides limits on your distance from the video source.

The passive nature of HMDs is not just because it's a 'monitor on your head'. Unless you are content with using the HMD as a media viewer, you will want to interact somehow with the content and with the HMD. In today's products, you might have some push buttons on the HMD and you usually have a head orientation tracker than can let the application know where you are looking. This is a decent start, but most of the user interface experience - selecting menus, interacting with content, moving 3D objects around, still relies on external devices: a joystick, a mouse, a data glove and others. By the way, many wireless video links don't have the ability to send head tracking information back to the video source.

Sure, this is fine for some applications. Military training and simulation applications are sometimes OK with having the soldier being trained carry a computer on his back. Stationary applications (a tank trainer) or those relying on lots of peripheral equipment (such as an academic research project measuring brain activity) can greatly benefit from today's HMDs, but is this enough for widespread use?

The promise of HMDs was to be able to take 3D content anywhere and interact with it in a useful way. Though more complex, it is just like how the iPod allowed us to take our music library with us anywhere and sufficiently interact with it to be useful. I don't think today's HMDs are fulfilling this promise for a broad-enough market.

Something needs to be done about it.

.

Gentex acquires Intersense

A couple of days ago, Gentex Corporate announced that it had acquired Intersense. Gentex makes various military products including helmets, eye protection devices, aluminized fabric and helmet-mounted displays. Intersense makes various inertial trackers.

I've had the pleasure of working with Intersense over several years, where we partnered on delivering virtual reality solutions for academic and defense markets. To many of our customers, Intersense has  always been considered the 'gold standard' of tracking, which is why I find this acquisition disappointing.

The financial details of the acquisition were not published, so perhaps this is a big win for the shareholders of Intersense, but I feel the company will now take an even stronger defense focus and will thus miss the bigger consumer product opportunity.

It is ironic that the Intersense acquisition happened just a week after the Invensense IPO (INVN), which values Invensense at some $800M. 

Founded in 1996, Intersense makes professional motion sensing products. Motion sensing is becoming more and more commonplace - just look at the Wii or the iPhone. Surely, Intersense had plenty of expertise in sensors and signal processing to offer a compelling low-cost product that builds off  Intersense's professional reputation but offers a compelling price point. 

As motion tracking became more common, lower-cost solutions started to appear. For instance, Sensics offers an integrated three degree of freedom (yaw/pitch/roll) head tracker inside the zSight professional HMD. Though the performance of the embedded tracker is not as good an Intersense IC-3, for instance, we found that for many customers it was good enough and saved them the need to pay a couple of thousand dollars for an external tracker. Intersense could have probably easily offered this lower-cost tracker, but they did not. Missed opportunity, I think, both for Intersense and for the market.

Good luck to the excellent team at Intersense, in whatever market you choose to serve!

Towards socially-acceptable goggles

Holiday season is fast approaching in the US and retailers are starting to offer attractive shopping discounts. A few years ago, my wife asked me to go to a home goods store during holiday season and buy some wine glasses. It was a cold day, and my cell phone was tucked inside my jacket while I was using an earphone in the store to speaking with my wife and decide together on which wine glass set to buy. At that time, mobile earphones were such an uncommon sight that several older ladies approached me - as I had appeared to be speaking to myself - to inquire if everything was OK.

How times have changed. Using a cell phone earpiece or Bluetooth device in a shopping mall - a complete oddity just a few years back - is now completely socially acceptable.

When would it and what would it take to have wearing virtual reality goggles become socially acceptable? Today, when I meet someone at a coffee shop and take out the goggles, people start coming over from adjacent tables to take a look. That's great if you want to sell goggles, but what would need to happen so that you can wear goggles in public unnoticed?

Clearly, the definition of what is 'socially acceptable' changes over time. Purple hair color? Diamond studs for man? Body piercing? Bluetooth headsets? All of these used to be outside the norm and now hardly worth a second look.

What were the motivations of the first people that stepped out of the 'socially acceptably' domain? Using a headset is probably about utility: it is more convenient, perhaps healthier. I don't think too many people used a headset because they felt it was overly aesthetic. On the flip side, there was little actual utility in dying your hair purple. A diamond stud doesn't make you hear better, but some think it makes you look better or draw more attention.

In the case of virtual reality goggles, we'll probably have to go the utility route. If you just want a pair of cool-looking sunglasses, just go to the store and buy cool-looking sunglasses!

What will goggles need to do so that they would be widely worn? If a Bluetooth headset is an audio extension of your phone, what will the visual extension of the phone be? Is it just about reading text messages or looking at driving directions without starting at the phone? What is the killer app? Write back to let me know what you think.

The Stark HUD 2020 Augmented Reality goggles

A few weeks ago, I wrote about the Stark HUD augmented reality goggles. These are not real AR goggles, but rather a clever promotion for Iron Man 2. Nevertheless, the video is compelling and worth reflecting on what it is so.

1. The goggles are visually attractive and appear to be 'socially acceptable' to wear. Though the definition of socially acceptable is fluid - just a few years ago it seemed odd to walk around with a Bluetooth headset - the closer goggles look like glasses, the better.

2. Information is context sensitive. The idea of context-aware computing has been around for quite some time, and there are patent filings on it dating at least 10 years ago. However, the availability of location-based services, and development in applications that know more and more about the user (Apple's Siri is a great new example), make the goggles useful much more than just being used as a media viewer.

3. The field of view appears wide, and resolution is good enough to read.

4. There is plenty of natural interaction with the device. The video demonstrates interaction using hands and voice commands.

5. No battery or control box. Science fiction, maybe, but still cool.

Will we really need to wait until 2020 to get them?

Steve Jobs and the HMD

The Web is buzzing about iTV, Apple's HDTV and what it could look like and do. Though Apple has filed some patents related to HMDs, it does not appear that Steve Jobs left us with an HMD design.

What would an HMD designed by Steve Jobs look like? What would it do?

As a business person, I want to build successful companies, and while success can be measured in many financial metrics, one aspect of a successful company would be a company that creates an emotional attachment with its customers.

My little personal story with Apple began many years ago, when I was about 12 years old and living in Boston. I was one of these kids that stood at the corner Radio Shack stores trying to do fun stuff with their TRS-80 computers. The personal computer revolution just started. IBM had not introduced the PC yet, and I wanted a computer of my own. My family lived in an apartment building which, for a 12-year old, was fertile ground do earn money babysitting. Lots of babysitting. Once I decided I wanted an Apple (other candidates were the TRS-80, the CompuColor and perhaps some others that I don't remember), I saved every babysitting penny towards the $1200 goal of getting an Apple II. My parents opened a bank account for me and I would stand in line to deposit $12 here, $30 there, until I reached the magic number and get my Apple II. Its serial number had 4 digits and it was that older model that had a reset key on the keyboard that, while spring loaded, was too easy to press and wipe out everything. I spent hours and hours programming, playing and hacking on that computer. Many years later, my mother donated it to a local school - it was the right thing to do but I miss my old Apple II. I can't say I miss any of the numerous PCs I've went through since.

What was so good about that Apple II? I loved its design. It was open so that you could plug cards inside. It was simple to use and "it just worked" year after year after year. That allowed so many people to write cool software and so on in a vitreous cycle.

What can we learn from Apple about HMDs?

• Design matters. Make people want to use or - in the case of HMDs - wear your product
• Keep it simple. Plug and play. No drivers to install. No configuration to go through. "It just works".
• Open it up so that value could be added outside your company.
• Price matters, but you don't have to be the lowest-cost product to succeed
• Make it simple to use content that you already own
• Humanize the product, if possible.