New Year "Resolutions", part 2

A couple of years ago I wrote a blog post 'Resolutions for the New Year' which described upcoming 2560x1440 displays and how they impact virtual reality. I think it's time to revisit that discussion.

Back in 2014, HMD makers were happy to use displays made for smartphones because that was the best way to get inexpensive, high-resolution displays that were small enough to be used in HMDs. Some of these displays made it possible to implement low-persistence schemes (OLED displays, strobing) and some could be over-driven beyond the standard 60 FPS refresh rate.

Now that virtual reality is coming into the mainstream, we are seeing that it is starting to make sense for screen manufacturers to build screens where VR is the main goal, not a secondary afterthought. As a result, we start seeing a divergence between the specifications of smartphone screens and of VR-specific screens. Let's consider this along three dimensions: higher resolutions, physically smaller screens and flexible screens.

Higher resolutions

The 2560x1440 screens that I wrote about 2 years ago became commonplace in phones like the Samsung Galaxy S6 and LG G3. These have become popular in phone-based VR products such as the Samsung Gear VR.

The next step-up in resolution could be 3840x2160, but many have argued that aside from using the phone for VR, there is very little end-user benefit to having a 5-6" phone with such high resolution. Thus, first point of divergence where what's good for VR is not very useful for a regular phone.

For VR, if we use a standard phone-based VR configuration where the screen is divided into two and a lens provides approx 90 degree horizontal field of view from each half, a screen with 3840 pixels across would provide 1920 pixels per eye or approximately 21 pixels/degree. This is about the pixel density that professional-grade HMDs delivered 5-10 years ago. For comparison purposes, "eye limiting resolution" beyond which increased resolution isn't noticeable is considered to be about 60 pixels/degree, so there is still a long way to go before photo-realistic VR with wide field of view.

Smaller screens

The downside of basing an HMD on a single 5.5" phone screen is that it physically looks like you are wearing a phone screen, so the designs are a bit more bulky than desired. If, instead, displays with a similar pixel density appeared in a smaller size, say 3.5", these displays would open up several opportunities if two such displays - one per eye - is being used:

• Greater flexibility for industrial designers
• The ability to deliver an convenient IPD mechanism to accommodate the distance between the eyes. One very convenient feature of a 5.5" screen is that it works well for nominal IPD. For instance, the AUO H546DLB01 1920x1080 OLED screen that is used in the OSVR HDK has an active area of approx 68 x 121 mm. If we assume one half of the screen is used for each eye, and if we assume that the optics are designed to be centered on precisely the middle of each half, than the distance between the optical centers is about 60.5 mm, which is not far from an average IPD of about 63mm. However, a single screen make it difficult to increase or decrease the IPD because you "run out of screen" if you move the optics farther from each other. However, if a separate 3.5" screen was dedicated to each eye, you could move the screen and the optics together and easily accommodate larger IPD. 
• Ability of both screens to update together. if a single 5.5" screen is updated left to right at 80 FPS, each frame lasts 12.5 mSec and thus the middle of the screen lags behind the beginning of the screen by about 6.25 mSec, so the software has to consider this when deciding what to render for each eye. However, if the two screens are separate, they can start their update at the same time.
• Opportunity to tilt each screen outwards, thus generating wider binocular field of view at the expense of generating partial overlap.
• Opportunity to tile multiple screens to achieve even wider field of view and higher resolution, similar to what Sensics has done many years ago with micro-displays

For completeness, it is worth mentioning that direct-to-eye DLP technologies are also beginning to provide wider field of view for high resolutions, so phone screens or their derivatives are not the only options.

Flexible screens

If a screen was very flexible, it could generate some value for phones: it could allow a curved edge of a phone; it could allow slightly curved phones that fit the face better (like the LG G Flex phones) and it might be better protected from shock (like the Motorola Droid Turbo 2). However, a flexible screen could generate a lot of value for HMDs as it would allow curving the screen around the head. This could generate two benefits:

• Better aesthetics
• Wider field of view. If one desires to get closer and closer to truly panoramic field of view - say 180 degrees - a screen that wraps around the head will make that easier, though special optics will need to be designed. This approach is reported to be a key part of the HMD being developed by The Void

The new displays bring new challenges together with the new opportunities. One such challenge is optimizing the optical design to these newer configurations (though companies like Sensics can help in this effort). But in all, as HMDs move from the background to the forefront in terms of an interesting market, we will this continue to drive display innovation.