The Alienware curved monitor and a curved HMD

Earlier this year, Alienware (now part of Dell), demonstrated a high-resolution, panoramic monitor. The 'Curved monitor' blends together four DLP projectors to create a wide display that is over 3 ft wide and has 2880x900 pixel resolution. You can read the Gizmodo review of this monitor, and see some videos at here

The process of tiling together multiple displays to achieve a curved, panoramic, high-resolution product is exactly what my company does to generate a curved, panoramic, high-resolution head-mounted display. The multiple displays are positioned so that they approximate a sphere around the eye, for a 'surround video' experience. Resolution is similar or higher than the curved monitor.

The decision whether to use a curved panoramic HMD or a curved panoramic monitor
is a bit like comparing a car to a plane: there are uses where one is better than the other, with some area of overlap. If you need many people to see the image at once, a monitor is better. If you are not inclined to wear anything on the head, the monitor is better. However, if you want stereo images, if you need portability, or if you want 360 degree immersion (image changes when you move your head), the HMD is your choice.

Some of the things I'm curious regarding the curved monitor:
- Does it require special video output modes, or can it accept stadnard display resolutions?
- How many video inputs does it require?
- How good is the tiling/stitching between the displays? In the Gizmodo video you can clearly see the seams between the projectors. Is there a way to achieve a seamless image?
- From a manufacturing standpoint, how does Alienware perfectly align the DLP projectors to achieve pixel-by-pixel matching? Does this alignment change over time?
- How good is the projector-to-projector color matching?

It's great to see tiled displays hit high-end consumer applications. I'm looking forward to trying a curved monitor one day soon and comparing it - with the same application - to a curved HMD

The Display Inside The Display

When users talk about their recent experience with head-mounted displays, they often refer to image quality. "The colors were great", "the image was very bright" and so forth. Indeed, image quality (lack of smear, brightness, contrast, color gamut) scores very high on HMD requirement surveys, right next to high resolution and wide field of view.

It turns out that the micro-display technology used inside the HMD can critically impact the user experience, often more than other design decisions for a particular HMD.

Many HMDs today using LCOS (liquid crystal on silicon), which has similarities to LCD technology, with the exception the LCD is transmissive - that is, the light source is behind the display, where LCOS is reflective - which means the light source needs to project the front of the display. LCOS displays are a common choice because they come in higher resolution than comparable single-chip OLED (Organic LED) display, and are available from multiple vendors. Unfortunately, LCOS-based designs have some disadvantages:
- They require a light source which reduces the display contrast (the light is on even when a black picture is displayed) as well as substantially increases power consumption.
- They often have limited temperature range, or require somewhat exotic solutions like local LCOS heaters to warm up the display before initial use.
- They typically exhibit motion blur and smear, just like many of us experienced with LCD monitors. This is particularly relevant in HMDs since images inside an HMD constantly change to track user head movements.
- Depending on the specific design, users may experience color flashes or other artifacts during head movement.

OLEDs are also used in several HMDs. OLEDs are self-emitting, meaning that they do not require independent lighting. OLEDs, such as those available from eMagin, offer a larger color gamut, higher brightness, higher contrast and lower power consumption. OLEDs are also faster than LCOS and thus are free of LCOS artifacts. However, OLEDs typically come in lower resolutions that LCOS displays and are available from fewer vendors. To overcome the lower resolution of individual displays, vendors such as my company, use a tiled approach to combine multiple micro-displays into a larger, higher-resolution image.

So... look inside! Next time you look at an HMD, inquire what display technology is inside the display.

The 2008 HMD Market Study Survey

Sensics released today the 2nd annual HMD market survey. In this survey, conducted in February of 2008, we re-examine the same questions asked last year in the 2007 HMD survey, and then go on to compare and contrast the market needs.

The 2007 survey was titled "Are existing HMDs good enough?". The answer was a resounding "no!", and it helped quantify what we all intuitively know - that HMDs with narrow field of view ("tunnel vision") and low resolution may be sufficient for very early adopters, but are certainly not good enough for widespread use.

The 2008 survey is titled "Are we there yet?". The short answer is also "no", though perhaps without the exclamation point. Requirements have not changed much, and vendors are certainly making progress in working towards what users are asking for - wider field of view, higher resolution, in a light-weight, easy-to-use product. Panoramic field of view products such as those that we make are maturing, and other companies have innovative products (though sometimes curious-looking, such as the "wedding cake HMD") that try to solve these problems. However, the "holy grail of HMDs": panoramic field of view, high def, low cost, easy to use - has not been conquered yet. Look for additional fascinating developments in 2008.

I'll be posting additional insight from these survey from time to time. Stay tuned.

Optical see-through vs. Video see-through

If you're interested in augmented reality, you've probably thought about optical see-through and video see-through.

Is one better than the other? That's like asking whether a car is better than an plane. It depends on what you need to do: get groceries? the car is unmatched. Get from New York to San Francisco? a plane is very often your choice.

Video-see through systems present video feeds from cameras inside head-mounted devices. You can see an example here. This can be useful when you need to experience something remotely: a robot which you send to fix a leak inside a chemical plant; a vacation destination that you're thinking about. This is also useful when using an image enhancement system: a thermal imagery, night-vision devices, etc.

Optical see-through systems combine computer-generated imagery with "through the glasses" image of the real world, usually through a slanted semi-transparent mirror. If you are in a mission-critical application and you're concerned what happens should your power fail, an optical see-through solution will allow you to see something in that extreme situation. If you are concerned about the utmost image quality, portable cameras and fully-immersive head-mounted display can't match the "direct view" experience.

One aspect of video see-through systems is that it's much easier to match the video latency with the computer graphics latency. Latency (delay) is inherent to immersive imaging systems: motion trackers are not instantaneous; computer graphic generation is not immediate and even when refreshing images at 60, 70, even 120 Hz, there is a lag from sensing to imaging. When computer graphics need to be overlaid on the image from the actual world, there is a difference between video see-through and optical see-through. Optical see through offers no latency, which sounds great, except that there is an inherent mismatch or lack of synchronization between what you see through the glasses and the graphics. If you're showing a virtual sofa inside a real living room, this mismatch can be distracting. In contrast, using video see-through allows you to synchronize the delay so that your video and graphics are always in sync.

With optical and video see-through, no solution is perfect for all problems.

Are you blind when using a head-mounted display?

I received a letter from the Maryland Motor Vehicle Administration that my driver's license needs to be renewed. This happens once every five years, and also is a nice reminder that my birthday is coming up.

Checking the MVA site to determine the closest location and list of required documents, I also saw the vision test requirements. Take a look at these requirements from the Web site:

To qualify for an unrestricted driver’s license, the State of Maryland requires drivers to have:

• Binocular vision
• Visual acuity (Snellen) of at least 20/40 in each eye
• A continuous field of vision of at least 140 degrees

Restricted licenses may be issued to persons having:

• Visual acuity of at least 20/70 in one or both eyes
• A continuous field of vision of at least 110 degrees, with at least 35 degrees lateral to the midline of each side

A related definition of visual performance is the 1934 American Medical Association of blindness: "Central visual acuity of 20/200 or less in the better eye with corrective glasses or central visual acuity of more than 20/200 if there is a visual field defect in which the peripheral field is contracted to such an extent that the widest diameter of the visual field subtends an angular distance no greater than 20 degrees in the better eye."

Those definitions made me wonder: how close you are to being blind when you wear an HMD?

With most head-mounted displays, field of view is limited: typically 30 to 50 degrees horizontal (not far from the definition of legal blindness). Only very few products, such as those from my company offer wide field of view that would be enough to meet the driving requirements.

Similarly, many HMDs have limited resolution that would constitute legal blindness. While resolution is sometimes reported as total number of pixels (e.g. 800x600 pixels/eye), an important measure is the number of pixels per degree, or pixel density. I saw a product recently that claimed wide field of view but offered less than 10 pixels/degree. This is about 20/125 visual acuity according to Snellen notation (6/38 metric). For more technical information, see this excellent Visual Acuity web page at the University of Utah.

I'm not suggesting in any way that anyone should attempt to drive with an HMD. It does seem, however, that most HMDs still have a long way to go in enhancing field of view and resolution/pixel density before they become truly usable.

How important is VR in education?

I had the pleasure of participating in a panel discussion in the "Simulation and the learners of tomorrow" conference organized by the Center for Technology (CTE) in Education at Johns Hopkins University. The conference was attended by teachers, educational administrators, technologists, government and state officials and more.

CTE is one of our first customer, focusing their research in science and math education on the middle-school level. It was fun spending the day with educators to discuss the future of education and how simulations, virtual reality and other advanced technologies can take a part in educating our kids.

I picked up on several interesting questions :

• How effective will VR be in education? There is no question "VR" is super cool - just ask the dozens of kids that tried Sensics head-mounted displays, but does this coolness translate into more effective learning and retention?
• What is the preferred display device for VR learning? Is it a large projection display which engages all the class at the same time, or is it head-mounted displays that allow different students to learn immersively at their own pace with little or no distraction from other classmates?
• Who will pay for VR? Will the school system do it using technology budgets? will this become an after-school enrichment program?
• How will VR get into the classroom? Will it be a top-down decision from the administrators, or a grass-roots action where rebel teachers that are excited about the VR experience they had elsewhere (home, arcade, science museum) experiment with it in class?
• If VR is effective, is this effectiveness the result of the newness factor (students are glued to the device because it is new and they have not experienced 3D immersion before), or is it really more effective than standard interactive software on a desktop?
• What material will be most effective for VR? Is it math and science (e.g. "fantastic voyage inside the body"), history (where VR can take you to other civilizations)? something else?
• How quickly will VR in education become mainstream?

CTE plans to have a follow-up conference next year, and I hope to be able to report progress from there as well.


Yuval

Fulfilling your childhood dreams

Make sure you view Randy Pausch's lecture "fulfilling your childhood dreams" about life, VR and everything in-between.