Diagramming the VR Stack

While preparing for last week's Consumer Electronics Show, I created a simple diagram to explain and VR application stack and how it related to the work that we are doing with OSVR. I thought it might be interesting to share this diagram and explain its contents:

The VR Stack diagram, Copyright (c) Sensics, Inc.

At the risk of contradicting Copernicus, we see the HMD at the center. Without the display, whether VR or AR, there is not much of an experience.

The display by itself does not make the experience, just like a game console without game controllers are not very interesting. Thus, the HMD is surrounded by a variety of sensors and output devices. These are often orientation and position trackers, input devices, but could also be eye trackers, motion platforms, gesture and skeleton sensors, depth sensors and more.

Moving for a moment to the outer circle, there are games and other applications. These needs to be distributed (and often monetized), but let's leave the distribution discussion for another time.

Very often, these applications are written using graphics or game engines. They provide a wide range of services: from assets to lighting to physics and more. These engines - such as Unity, Unreal, Cryengine or other are shown in the next to last circle.

To me, what is often overlooked in VR is the need for a middleware layer - a layer that connects the various engines to the various sensors and peripherals. I submit that no one likes to write applications that work only on one device and because of that, a layer that allows a game engine to connect to many different HMDs, to many different eye trackers, to many different sensors is critical. Think about your computer at home. Does everyone use the same monitor? Keyboard? Mouse? Printer? Not everyone would want to use the same HMD. That's why middleware is required and an excellent example of middleware is the OSVR software platform supporting hundreds of devices, peripherals and game engines. All layers are important, so don't forget the middleware!

The post "Diagramming the VR Stack" first appeared on "Virtual Reality and Head Mounted Displays"