HMDs and the Out-of-Body Experience

It seems that one the favorite things for HMD users is to look at themselves - virtually. There are several ways to do this.

One way is to install video cameras that capture the user's image. This could be from the front to reproduce the feeling of using a mirror, but it is often more interesting to capture the user from the top or back, this providing an 'out of body' experience. Experiments along these lines have been reported several years ago by the Karolinska Institute.

Another way is to perform 3D laser scanning of self, such as using the services of our friends at ShapeShot. This allows creating a virtual 3D representation of a person and then incorporating that representation in a virtual scene. The effect of browsing a virtual museum and then seeing your head rotate on a pedestal like a Greek head sculpture is quite amazing.

A third way is to use motion capture technologies and then create a moving avatar that mimics your movements. The avatar does not have to be a true representation. It could have a over sized head, or a severed leg, or have limited range of motion. This can be used to illustrate the impact of several disabilities, or sometimes to help with a treatment or physical therapy plan.

In a recent trade show, we installed an iMove spherical video system on top of a post and allowed users to explore the trade show from high above by wearing an HMD with an integrated head tracker and having that head tracker control the direction of the camera. Interestingly, a favorite target was to look down, which caused the camera on the post to look down, thus providing a view of self.

Why is this fascination with viewing self? How can this be used, in 3D, to provide new experiences such as a virtual shopper that lets you see how you would look in a new piece of clothing, or help explore what it would feel like to be taller, or shorter, or otherwise different?

Reflections on a reflective LCOS press release

A few days ago, a press release from one of the liquid crystal on silicon (LCOS) chip vendors landed in my inbox. Two things caught my interest:
1. That sometimes in 2012 (e.g. 13 to 24 months from today) they would release a display with 2048x1536 pixels.
2. That this chip would eventually power head-mounted displays that could serve as an alternative to achieving wide field of view and high resolution through multiple low-resolution displays.

I think Sensics is the only company that has tiled head-mounted displays as part of the product line, I took that as presenting a 2012 alternative to Sensics. If imitation is the most sincere form of flattery, than I and the rest of the Sensics team should take this press release with pride.

Other than pride, it gives us an opportunity to review the criteria we use for selecting the display chips that power our products. We are not married to a specific display technology, but strive the provide an excellent combination of panoramic field of view and high definition and a lightweight design. Panoramic field of view provide greater sense of immersion and heightened situational awareness. High definition provides more realistic images. Lightweight designs can be used for extended periods without neck strain or long-term health concerns. Customers have come to understand why all these attributes are important and want to go forward, not backward, with future HMDs.

Last time we made that choice, OLEDs were best for us and our customers as they offer unique advantages:
- Simpler optical design which translates into smaller, lighter products. OLEDs are self-emitting whereas LCOS is a reflective technology requiring an external light source and more complex optical path.
- Higher contrast. Because the external light source for LCOS shines even when the displays need to show black, the overall contrast in LCOS is typically much lower than OLED
- Faster switching and no motion blur. LCOS relies on an external light source an LCOS chip is typically lit by three alternating light sources. In contrast, OLED is faster and the colors are continuously lit.
- Lower power consumption which translates into less heat near and head and longer operation when used with a battery.
- Simpler drive electronics which make for smaller products.

LCOS chips do offer two advantages:
- Greater fill ratio, which is the percent of useful area in the chip.
- Ability to offer very high brightness if lit by a powerful lamp. This could be important for actual flight helmets that need to be used in direct sunlight but not as much for HMDs.

As part of the xSight and piSight product line, Sensics achieves panoramic field of view, high definition 1080P and light weight by optically tiling several 800x600 OLED displays. Tiling is getting better and better every year, and tiling HMDs could be introduced for 1280x1024 or even higher resolution OLEDs, effectively decreasing the number of tiles while maintaining all the other advantages of the tiled design.

BTW, the Sensics zSight is not tiled and offers a single-screen solution for those that prefer it.

Come late 2011, 2012 or whenever a higher-resolution LCOS chip is ready, Sensics will be more than happy to take a serious look at it and see if it could improve the customer experience relative to then-available alternatives. Until then, we're just glad to be on everyone's mind.

Vega Prime, VBS2, Adacel, iMove, WorldViz and Harry Potter integration

We're back from I/ITSEC, the world's largest training and simulation show where we demonstrated several really cool integrations between popular software packages and the our head-mounted displays.

Here are some of the demos we showed:

• xSight HMD with Vega Prime from Presagis. Vega Prime is a popular environment for creating 3D simulations, and powers many commercial flight simulators, often in dome or multi-screen configurations. Integrating Vega Prime with a panoramic HMD allows a very portable solution which is useful if the simulation needs to come to the trainee, or if there are very confined spaces. With a good graphics card and a professional motion tracker from Intersense, we allowed visitors to immerse themselves in a virtual construction site. Update: Presagis also thinks it was cool
• zSight HMD with VBS2 from Bohemia Interactive coupled with voice recognition from Adacel. VBS2 is often used for ground training and provides a rich environment for creating, executing and debriefing training missions. In our demo, we coupled VBS2 with the zSight SXGA HMD including integrating the zSight's built-in three degree of freedom tracker, so that the user head movements are reflected on the screen. The voice recognition engine from Adacel, coupled with the zSight's built-in microphone and stereo sound, allowed simulated artillery training. Speaking commands while wearing the HMD results the computer recognizing, processing, executing and then responding to the commands by voice for a very compelling experience.
• iMove provide multi-camera solution that allow complete spherical viewing and recording of space. We placed an iMove camera pod on top of our booth and it provided true 360 degree views of the conference floor. By integrating the iMove viewing application with the built-in tracker inside the zSight HMD, conference attendees were able to don the zSight and explore the conference from up above. Since the camera was right above our booth and provides full spherical imaging, users could sometimes see themselves wearing the HMD for somewhat of an 'out of body' experience.
• The Vizard software by WorldViz is an interactive development environment for all things 3D. We have been working with our friends from WorldViz for several years now, and their demos do a great job in showcasing the power of VR. At the show, we have Vizard running with an xSight panoramic, high-definition HMD with full 6-degree (yaw/pitch/roll, x/y/z) tracking.
• Last but not least, we demonstrated the low-latency wireless link for the zSight HMD as well as the zSight's ability to display 3D side-by-side content being popularized by YouTube and 3D broadcasts. The wireless link worked great at the show providing high-definition images, long range (we were able to walk more than 100 feet without losing signal in an otherwise busy show environment), stereo audio and long battery life. For a full show day, we probably used 3 to 4 rechargeable batteries, and with the quick charge feature it was easy to charge one while another was being used. The highlight the 3D aspect of the zSight, we found several side-by-side movie trailers. The new Harry Potter movie (3D trailer link here) beat Cloudy with a chance of meatballs as well as Toy Story 3 as the show favorite.

Overall, lots of fun in an otherwise serious exhibition.