Can the GPU compensate for all Optical Aberrations?

Photo Credit: davidyuweb via Compfight cc

As faster, newer Graphics Processing Units (GPUs) become available, graphics cards can perform real-time image transformations that were previously relegated to custom-designed hardware. Can these GPUs overcome all the important optical aberrations, thus allowing HMD vendors to use simple, low-cost optics?

The short answer is: GPUs can overcome some, but not all aberrations. Let's look deeper into this question.

Optical aberrations are the result of imperfect optical systems. Every optical system is imperfect, though of course some imperfections are more noticeable than others. There are several key types of aberrations in HMD optics which take an image from a screen and pass it through viewing optics:

• Geometrical distortion, which we covered in a previous post would cause a square image to appear curved. The most common variants are pincushion distortion and barrel distortion.
• Color aberration. Optical systems impact different colors in different ways, as can be seen in a rainbow or when light passes through a prism. This results in color breakup where a white dot in the original screen breaks up into its primary colors when passing through the optical system.
• Spot size (also referred to as astigmatism), which shows how a tiny dot on the original screen appears through the optical system. Beyond the theoretical limits (diffraction limit), imperfect optical systems cause this tiny dot to appear as a blurred circle or ellipse. In essence, the optical system is unable to perfectly focus each point from the source screen. When the spot size becomes large enough, it blurs the distinction between adjacent pixels and can make viewing the image increasingly difficult.

The diagram below shows an example of the spot size and color separation on various points in the field of view of a certain HMD optical system. This is shown for the three primary colors, with their wavelengths specified in the upper right corner. As you can see, the spot size is much larger for some areas than others, and colors start to appear separated.

Which of these issues can be corrected by a GPU, assuming no practical limits on processing power?

Geometrical distortion can be corrected in most cases. One approach is for the GPU to remap the image generated by the software so that it compensates for known optical distortion. For instance, if the image through the optical system appears as if the corners of a square are pulled inwards, the GPU would morph that part of the image by pushing these corners outwards. Another approach is to render the image up-front with the understanding of the distortion, such as the algorithm covered in this article about an Intel researcher.

Color aberration may also be addressed, though it is more complex. Theoretically, the GPU can understand not only the generic distortion function for a given optical system, but the color-specific one as well, and remap the color components in the pixels accordingly. This requires understanding not only the optical system but also the primary colors that are being used in a particular display. Not all "greens", for instance, are identical

Where the GPU fails is in correcting astigmatism. If the optical system causes some parts of the image to be defocused, the GPU cannot generate an image that will 're-focus' the system. In simpler optics, this phenomena is particularly noticeable away from the center of the image. 

One might say that some defocus in the edge of an image is not an issue since the central vision of a person is much better then the peripheral vision, but this argument does not take into account the rotation of the eye and the desire to see details away from the center.

Another discussion is the cost-effectiveness of improving optics, or the "how good is good-enough" debate. Better optics often cost more, perhaps weigh more, and not everyone needs this improved performance or is willing to pay for it. Obviously, less distortion is better to more distortion, but at what price?

Higher-performance GPUs might cost more, or might require more power. This might prove to be important in portable systems such as smartphones or goggles with on-board processors (such as the SmartGoggles), so fixing imperfections on the GPU is not as 'free' as it might appear at first glance.

HMD design is a study in tradeoffs. Modern GPUs are able to help overcome some imperfections in low-cost optical systems, but they are not the solution to all the important issues.

For additional VR tutorials on this blog, click here
Expert interviews and tutorials can also be found on the Sensics Insight page here

To Occlude or not to Occlude?

A question came up on the Natalia Gameplay Youtube video a couple of days ago:

I have a question? since it doesn't cover the whole eye area can you be distracted by light and stuff coming through the sides. i love the ideal of a headset and cameras and stuff in the front of it to track hand movement but i just don't like the idea of there an opening on the sides ?

This brings up a nice opportunity to speak about occlusion (the blocking of light) in goggles. At Sensics, we have done it both ways: some products block pretty much all external light from coming into the goggle, making the user entirely focused on the image displayed inside, and some products allow some peripheral vision. For instance, two products that can be configured for identical resolution and field of view are:

piSight - not occluded

xSight - fully occluded

the xSight is based on a ski-goggle design with a mask that touches the face all around the edge of the goggle. The piSight, on the other hand, hangs the optics in front of the eyes using an over-the-head rail structure which is very comfortable (in spite of looking like a torture device).

What are the advantages of an occluded design (such as the xSight)?

• Allows the user to completely focus on the displayed image
• Increases display contrast by blocking outside light
• Enhances the sense of immersion by blocking outside distractions

What are the advantages of a non-occluded design (such as the piSight)?

• Better orientation in the physical space. Goggle allows peeking sideways or looking down to the floor or to find a keyboard underneath the goggle. If the user of the goggles is expected to substantially move around in a room, a non-occluded design will feel safer to the user.
• If coordination with additional people is needed, easier to see where these people are and view their behavior and gestures. For instance, in an infantry training application, most goggles used are not occluded.
• Easier access to vicinity of the eyes if there is a need to adjust devices such as built-in eye tracker
• Easier to wear glasses. Most often, the difficulty in wearing glasses with goggles is not so much the eye relief (distance from the optics to the eyes) but rather the frame of the eyeglasses interfering with the enclosure of the goggles. A non-occluded design goes a long way to alleviate this problem.

In some instances, we tried to have the best of both worlds: a non-occluded design but with detachable blinders that allow to increase then occlusion when required.

In short - there is no right answer. Goggle design is about tradeoffs and the right choice depends on the requirements and applications.