Showing posts with label HDK. Show all posts
Showing posts with label HDK. Show all posts

Monday, October 26, 2015

The Video Processing FPGA inside the OSVR HDK and the Sensics dSight

Now that OSVR hacker developer kits are in the wild, @skyworxx Mark Schramm has posted some teardown photos of the 1.2 version of the HDK. He notes the FPGA on the board, and I thought I'd take the opportunity to explain what the FPGA can do in the HDK, and in it's high-end brother the dSight

The FPGA and the DRAM chips next to it can perform real-time video processing, giving hackers the ability to experiment with hardware-level transformations that do not require cooperation from the video-generating device or application.

Some of the options that are currently implemented:
1. Pass-through. This is the simplest mode and it does not involve the DRAMs. 1080x1920 video from the HDMI receiver is essentially passed through to the MIPI interface that drives the OLED screen.

2. Image rotation. This mode allows real-time 90 degree rotation, so that standard landscape mode 1920x1080 video can be presented in the 1080x1920 display. To do this, a full video frame is stored in the DRAM chips while the previous frame is sent to the display. This 90 degree rotation does cost you 1 frame of latency, but can be very useful in some of the following scenarios:

  • Video coming from a 1920x1080 source such as as DVD player or in "replicated desktop" mode.
  • Video coming over a low-latency wireless link. These links primarily support 1920x1080 today and not the native 1080x1920 mode.
3. Conversion of full screen to side-by-side mode. Ever saw a desktop in the HMD and found yourself squinting to see each half at a time. When this conversion mode is enabled, the video signal is converted into two identical copies of the signal, that can then be shown on both eyes at the same time. Control over this mode is via a command via the HID feature interface or with a simple utility.

Additional modes that are not yet implemented but can be implemented by the community:
  • Real-time distortion correction. If you have a non-cooperative video source or just prefer to use your GPU for something else, real-time distortion correction in the FPGA can be useful.
  • Resolution up-scaling: converting from lower resolution into the full resolution of the HDK
  • Color enhancements (e.g. gamma, HSI improvements)
  • Rearrange the video signal. One cool application that we saw from one of our partners is rearranging a 1080x1920 on the output of a GPU to reformat it into a non-legible 1920x1080 image, send that over the low-latency wireless video link and then use the FPGA to unscramble the image. This allows wireless video transmission without "paying" the 1-frame latency penalty.
If a manufacturer was very cost-conscious, they probably would not include the FPGA in the design, but as a hacker developer kit, we think it's an excellent exploratory option.

What could you do with it?



Saturday, January 17, 2015

How Things Work: the Dual-Element Optics of the OSVR HDK

When the OSVR HDK was unveiled last week, one aspect that received a lot of positive reviews was the quality of the optics - clear to the edges, no pre-distortion required. Since the Open-Source Virtual Reality project is indeed open-source, and its Hacker Development Kit will have the full production file available to download in a couple of weeks, I can talk openly about how Sensics designed it for those that are interested.

What's all the fuss about? Take a look at this comparison photo:
Test pattern photographed using an iPhone 4 camera through a dual-element design (left) and single-element design (right)

I took these photos a few months ago when we received the first batch of HDK optics. I found a test pattern on the Internet and displayed it on a 5.5" display, much like the one inside the HDK. I then used an iPhone 4 camera to take both photos. The left photo shows the test image through the HDK dual-element optics. The right photo shows the same test image through a popular single-element eyepiece. I was trying to get the best image in both cases, Is this a scientifically precise comparison? Probably not, but it's quite telling as is.

As you can see, the test pattern is in nice focus at the center of both the left and right photos. However, as you look towards the edges of the photo, the image remains in focus on the dual-element photo but is blurry in the single-element photo. Compare the "gmail" header on the left, quite readable, with the "gmail" header on the right, quite blurry.

You can also see that colors also break up on the right image. You start seeing the separation between red, green and blue. The colors stay intact on the left image. Last, if you look at the three bands of test pattern (right half of left image), you can see that they remain pretty much straight, whereas the same bands on the right image appear curvy.

In short, we are seeing a blurry image, chromatic aberration and geometrical distortion in the single-element design and not so much on the dual-element one.

Is the dual-element design in the HDK perfect? The best Sensics has ever done? The design to beat all designs from here on? Of course not! It's just a good design and solid engineering. Let's look at what makes it better in some aspects than the single-element design.

When we start an optical design project, we look at the requirements. How much field of view are we looking for? What is the screen size we are trying to image? What are the materials we are allowed to use (e.g. plastic? glass?) How much is it allowed to weigh? What are our cost constraints? How much eye relief (distance from cornea to first optical element) do we want? What is the desired eye box (how much the eye can move from the optimal location without significantly losing image quality)? Are we allowing aspheric optics? How wide or deep do we allow the optics to be? and so on...

All of these are constraints and every design has them. But the constraints are different from design to design, much like some cars are built for gas mileage and others are built for super-quick acceleration. That's why there is no single design that is good for every optical problem.

Usually, optics for HMDs have lots of constraints. You typically don't want to make them from glass because glass is heavier than plastic. You have limits on the lens diameter and size because you want to use them in a binocular setting. You don't want them to cost too much because you are aiming at some price target. When you give these constraints to the designer - after he or she finishes pulling their hair out - the design starts.

An optical lens has two sides. The exact curvature of each side can be different. If you have two lenses, you have four surfaces to work with. If you have 5 lenses, you have 10 surfaces to work with. More surfaces mean more degrees of freedom and more degrees of freedom mean that you can meet more of the constraints. So, having more optical elements usually means you can product a better image. In our case better meant less distortion, more focus, less color aberration.

It's almost like curve fitting: trying to fit a polynomial (e.g. y= a + b*x + c*x^2 + d*x^3 + ...) to set of (x,y) points. The more parameters you are allowed to use, the better your fit will be. If your curve is only "y = a", you'd often be in trouble. If you can use "y= a + b*x", you'll have a better fit and if you can use "y= a + b*x + c*x^2" you'll do even better.

So, we decided that a single-element design did not give us a good-enough answer to our constraints and decided to add a second element and give our designers additional degrees of freedom.

Here is what the design looks like


The screen is on the right side. The eye pupil is on the left side. The two lenses are in between: a 32-mm diameter lens closer to the eye and a 43mm diameter lens close to the screen. Note the interesting left-side surface of the bigger lens. It is made this way to optimize the image quality.

Here is what this looks in 3D:


Both lenses are made from optical-grade plastic. The smaller lens is from a material called Zeonex F52R and the larger one is from Polystyrene . There are hundreds of types of optical-grade glass but just a handful of plastics, but plastics are lighter and sometimes cheaper to use. One more constraint to worry about.

Why doesn't everyone use dual-element design? Because others might be focused on different constraints. For instance, dual elements roughly cost twice as much to make as a single element. Dual elements are heavier than a single element. Dual element eyepieces do a better job of controlling color and geometric distortion, but one could correct for color and geometric distortion with the GPU (though I don't know how one can correct for blurriness with the GPU). So, when we designed the OSVR HDK we were focused on some constraints and that's why we chose two elements.

Why not three or four or even five elements? Same story. We don't want it to cost too much. We don't want the optical path to be longer. We want to keep the weight down. Is someone going to soon write a blog post on their 7-element design? Maybe so. I can't wait to read it!


The post "How Things Work: the Dual-Element Optics of the OSVR HDK" first appeared on the VRguy's blog: www.vrguy.net

For additional VR tutorials on this blog, click here
Expert interviews and tutorials can also be found on the Sensics Insight page here