HMDs and the Out-of-Body Experience

It seems that one the favorite things for HMD users is to look at themselves - virtually. There are several ways to do this.

One way is to install video cameras that capture the user's image. This could be from the front to reproduce the feeling of using a mirror, but it is often more interesting to capture the user from the top or back, this providing an 'out of body' experience. Experiments along these lines have been reported several years ago by the Karolinska Institute.

Another way is to perform 3D laser scanning of self, such as using the services of our friends at ShapeShot. This allows creating a virtual 3D representation of a person and then incorporating that representation in a virtual scene. The effect of browsing a virtual museum and then seeing your head rotate on a pedestal like a Greek head sculpture is quite amazing.

A third way is to use motion capture technologies and then create a moving avatar that mimics your movements. The avatar does not have to be a true representation. It could have a over sized head, or a severed leg, or have limited range of motion. This can be used to illustrate the impact of several disabilities, or sometimes to help with a treatment or physical therapy plan.

In a recent trade show, we installed an iMove spherical video system on top of a post and allowed users to explore the trade show from high above by wearing an HMD with an integrated head tracker and having that head tracker control the direction of the camera. Interestingly, a favorite target was to look down, which caused the camera on the post to look down, thus providing a view of self.

Why is this fascination with viewing self? How can this be used, in 3D, to provide new experiences such as a virtual shopper that lets you see how you would look in a new piece of clothing, or help explore what it would feel like to be taller, or shorter, or otherwise different?

Reflections on a reflective LCOS press release

A few days ago, a press release from one of the liquid crystal on silicon (LCOS) chip vendors landed in my inbox. Two things caught my interest:
1. That sometimes in 2012 (e.g. 13 to 24 months from today) they would release a display with 2048x1536 pixels.
2. That this chip would eventually power head-mounted displays that could serve as an alternative to achieving wide field of view and high resolution through multiple low-resolution displays.

I think Sensics is the only company that has tiled head-mounted displays as part of the product line, I took that as presenting a 2012 alternative to Sensics. If imitation is the most sincere form of flattery, than I and the rest of the Sensics team should take this press release with pride.

Other than pride, it gives us an opportunity to review the criteria we use for selecting the display chips that power our products. We are not married to a specific display technology, but strive the provide an excellent combination of panoramic field of view and high definition and a lightweight design. Panoramic field of view provide greater sense of immersion and heightened situational awareness. High definition provides more realistic images. Lightweight designs can be used for extended periods without neck strain or long-term health concerns. Customers have come to understand why all these attributes are important and want to go forward, not backward, with future HMDs.

Last time we made that choice, OLEDs were best for us and our customers as they offer unique advantages:
- Simpler optical design which translates into smaller, lighter products. OLEDs are self-emitting whereas LCOS is a reflective technology requiring an external light source and more complex optical path.
- Higher contrast. Because the external light source for LCOS shines even when the displays need to show black, the overall contrast in LCOS is typically much lower than OLED
- Faster switching and no motion blur. LCOS relies on an external light source an LCOS chip is typically lit by three alternating light sources. In contrast, OLED is faster and the colors are continuously lit.
- Lower power consumption which translates into less heat near and head and longer operation when used with a battery.
- Simpler drive electronics which make for smaller products.

LCOS chips do offer two advantages:
- Greater fill ratio, which is the percent of useful area in the chip.
- Ability to offer very high brightness if lit by a powerful lamp. This could be important for actual flight helmets that need to be used in direct sunlight but not as much for HMDs.

As part of the xSight and piSight product line, Sensics achieves panoramic field of view, high definition 1080P and light weight by optically tiling several 800x600 OLED displays. Tiling is getting better and better every year, and tiling HMDs could be introduced for 1280x1024 or even higher resolution OLEDs, effectively decreasing the number of tiles while maintaining all the other advantages of the tiled design.

BTW, the Sensics zSight is not tiled and offers a single-screen solution for those that prefer it.

Come late 2011, 2012 or whenever a higher-resolution LCOS chip is ready, Sensics will be more than happy to take a serious look at it and see if it could improve the customer experience relative to then-available alternatives. Until then, we're just glad to be on everyone's mind.

Vega Prime, VBS2, Adacel, iMove, WorldViz and Harry Potter integration

We're back from I/ITSEC, the world's largest training and simulation show where we demonstrated several really cool integrations between popular software packages and the our head-mounted displays.

Here are some of the demos we showed:

• xSight HMD with Vega Prime from Presagis. Vega Prime is a popular environment for creating 3D simulations, and powers many commercial flight simulators, often in dome or multi-screen configurations. Integrating Vega Prime with a panoramic HMD allows a very portable solution which is useful if the simulation needs to come to the trainee, or if there are very confined spaces. With a good graphics card and a professional motion tracker from Intersense, we allowed visitors to immerse themselves in a virtual construction site. Update: Presagis also thinks it was cool
• zSight HMD with VBS2 from Bohemia Interactive coupled with voice recognition from Adacel. VBS2 is often used for ground training and provides a rich environment for creating, executing and debriefing training missions. In our demo, we coupled VBS2 with the zSight SXGA HMD including integrating the zSight's built-in three degree of freedom tracker, so that the user head movements are reflected on the screen. The voice recognition engine from Adacel, coupled with the zSight's built-in microphone and stereo sound, allowed simulated artillery training. Speaking commands while wearing the HMD results the computer recognizing, processing, executing and then responding to the commands by voice for a very compelling experience.
• iMove provide multi-camera solution that allow complete spherical viewing and recording of space. We placed an iMove camera pod on top of our booth and it provided true 360 degree views of the conference floor. By integrating the iMove viewing application with the built-in tracker inside the zSight HMD, conference attendees were able to don the zSight and explore the conference from up above. Since the camera was right above our booth and provides full spherical imaging, users could sometimes see themselves wearing the HMD for somewhat of an 'out of body' experience.
• The Vizard software by WorldViz is an interactive development environment for all things 3D. We have been working with our friends from WorldViz for several years now, and their demos do a great job in showcasing the power of VR. At the show, we have Vizard running with an xSight panoramic, high-definition HMD with full 6-degree (yaw/pitch/roll, x/y/z) tracking.
• Last but not least, we demonstrated the low-latency wireless link for the zSight HMD as well as the zSight's ability to display 3D side-by-side content being popularized by YouTube and 3D broadcasts. The wireless link worked great at the show providing high-definition images, long range (we were able to walk more than 100 feet without losing signal in an otherwise busy show environment), stereo audio and long battery life. For a full show day, we probably used 3 to 4 rechargeable batteries, and with the quick charge feature it was easy to charge one while another was being used. The highlight the 3D aspect of the zSight, we found several side-by-side movie trailers. The new Harry Potter movie (3D trailer link here) beat Cloudy with a chance of meatballs as well as Toy Story 3 as the show favorite.

Overall, lots of fun in an otherwise serious exhibition.

What users want from a wireless video link for HMDs

Sensics surveyed 179 users to determine their needs and wants on a wireless video link for head-mounted displays. The results of the survey, already shared with its participants, will be published in a few weeks, but I wanted to share some of the insights from it.

As part of the survey, we asked respondents to indicate the importance of several wireless link attributes such as distance, latency or battery life. Each feature was given an importance score from 1 (completely unimportant) to 6 (very important). We then sorted the aggregate results by the average importance assigned to them by the users. Results were:

Importance of various attributes of wireless video links for HMDs

It seems that the highest priorities were assigned to these features that make the wireless link nearly transparent to the application: high refresh rate, low latency, closed-loop operation to include down link tracking and stereo operation. The combination of these features differentiate wireless video products designed for HMDs with those targeted towards in-home use primarily to extend the distance between the set top box and the HDTV. A 1/2 second video link latency, for instance, might be quite insignificant when watching a basketball game on TV, but is devastating to closed-loop virtual reality applications.

Another key difference between in-home HDMI extenders and professional HMD-driven wireless links is the transmission technology. In-home HDMI extenders typically use ultra-wide band or the new 60 GHz extremely high-frequency radio band. Both of these need line of sight between the wireless transmitter and the wireless receiver. However, imagine an HMD user wearing the receiver in a backpack and then turning around. The user's body can easily cut off the line of sight between transmitter and receiver. Thus, technologies such as Wireless N which are not sensitive to line of sight limitations are clearly preferable for wireless HMDs.

I believe that wireless video links will help unleash the true potential of the personal, portable HMD. The freedom of movement, multi-user interaction opportunities offered by wireless links are substantial and I'd expect many exciting applications to materialize in 2011.

Fourth generation optics are here

Someone once said 'Science is like a horse, not a cow; feed it, don't worship it'. I read this to say that even if you have very good technology, don't rest on your laurels but rather keep investing to make it great.

For several years now, My company has been building virtual reality goggles based on a unique optical tiling technology which allows creating goggles that are lighter, have wider field of view and higher definition.

At the core, the problem to be solved and the tiling solution are both simple to explain. Users want panoramic field of view because once the peripheral vision is engaged when wearing goggles, the sense of realism, immersion, "being there" is greatly enhanced. At the same time, users want high resolution and high pixel density so that they can get lifelike images. With today's micro displays, it is difficult to do both. If you magnify a micro-display just a little bit, you get good pixel density but narrow field of view, or "tunnel vision". If you magnify a display too much, you can get good field of view but low pixel density. High magnification also brings concerns about total weight, image distortion, clarity at the edge of the image, and all kinds of things that goggle manufacturers worry about.

If you could somehow make micro-displays that physically overlap, you would be in good shape because you can create a really high-definition micro display from many lower-resolution ones. However, making displays physically overlap is practically impossible. Our technology makes the displays optically overlap, essentially by carefully putting small magnifying glasses in front of each display. If they are properly positioned and aligned, the result is a nearly seamless image of both high resolution and panoramic field of view.

After investing quite a lot, we have now made 'nearly seamless' even better. Our fourth-generation optics are even better now. We've improved image clarity at the edges. We've improved what's called the 'eye box' (larger eye box means that you can still see a good image even if your eyes are not at the optimal position relative to the goggles). We've even changed the material from which we make the lenses.

So, we are excited about these fourth-generation optics and people that have tried it on are also excited. If you are in the market for a professional set of goggles, make sure you stop by to see our progress.

Overweight HMDs are still heavy, regardless of how they are supported

When I was a kid, I was discussing with a friend how overweight people sometimes wear black because it makes them look thinner. "Yeah", he replied, "but such clothing still uses a lot of fabric, even if it's black fabric".

I feel the same way about some of the professional HMDs out there. 2 lbs (1 Kg) is still 2 lbs no matter how hard you work to distribute the weight on the head. It's still heavy.

Most of the weight in head-mounted displays is concentrated in the front. This is where the optics are, as well as the micro-displays, and some electronics. Using such front-heavy HMDs can cause neck strain and general discomfort. Some companies actually put a counter-weight on the back of the head to offset the front-loaded weight. Others experiment with sophisticated head mounts that are designed to more evenly-distribute the weight on the head. But even if not heavy immediately, wear a 2 lb HMD for 10 minutes and you won't forget that it's 2 lbs.

HMDs should simply go on a diet.

Users get this. We just returned from Europe where we'd have a chance to see potential customers try on different HMDs from different vendors. Often times, heavy products were discarded very quickly. "Just too heavy", we heard.

Would you put up with sunglasses that hurt your nose? How about a hat that became truly heavy after 30 minutes?

Big may be beautiful, and black fabric does not hurt either, but heavy on the head is just too much to take.

Cut the Cord

My company has been busy demonstrating a high-performance wireless video solution that works with our virtual reality goggles. As part of product launch preparations, we've surveyed many professionals worldwide to understand what they are looking for in a wireless HMD and why would they find such a product attractive.

A simple question that was asking the survey participants what the key benefits they are interested in obtaining from a wireless HMD. About 170 people answered this survey and the benefits they identified were as follows:

Survey participants could choose more than one benefit, so it's ok that the total number of benefits selected exceeds the number of respondents.

These responses also came with an area for entering free text. Many of the free text responses associated with this question dealt with the complexity and inconvenience of having an HMD being tethered. For instance, some respondent wrote “Detangle the user, prevent them from pulling on the cord and knocking the HMD off”, and “The main reason I have for obtaining a wireless HMD is that users get entangled with the cables VERY often, causing discomfort and frustration”.

This is perfectly understandable. A key attraction of using HMDs is the potential for a portable, personal experience with 360° freedom of movement. However, as long as HMDs are connected via an 'umbilical cord' to a PC, movement is restricted and cables become a major distraction.

I do think early implementations of wireless goggles will have the wireless receiver and perhaps a battery pack as some sort of 'belt pack' from which a short cable will run to the HMD. This still provides very substantial mobility benefits. Just try rotating 360 degrees with wired HMD! As far as we have seen, the level of integration and power consumption is not there yet to combine power and wireless in package that is so small and so light that it can be worn on the head next to the goggles.