In this post, I will cover useful information about motion trackers in the context of head-mounted displays: how they work, what features are important, and what you want to think about when integrating them.
It used to be that you'd have to pay $2000 or more for a accurate head tracking, but these days good trackers come included inside many professional HMDs such as the
Sensics zSight and even in some of the recent consumer goggle entries.
Head tracker basics
All head trackers measure rotational orientation - yaw, pitch and roll.
 |
| Head Yaw, Pitch and Roll (from www.resourceonbalance.com) |
Yaw is the side to side movement as in looking left and right. Pitch would be the motion when you look up or down. And roll is tilting the head side to side.
This post deals with inertial head-trackers as opposed to head-tracking systems that use markers, cameras, ultrasound or other methods. Inertial head trackers are the most popular type of trackers in HMDs
Most inertial head trackers include three sensors: accelerometer, gyroscope and magnetometer. The accelerometer measures the linear acceleration along three axes, including the earth's gravitational force, thus providing information regarding both the magnitude and direction of acceleration. The gyroscope measures angular velocity. The magnetometer measures the strength and direction of a magnetic field, most notably the earth's magnetic field. When all three sensors are used, and each provides measurement in three directions, the resulting head tracker is typically referred to as a '9-axis' or '9-sensor' module. Vendors sometimes refer to this as a '9 degree of freedom' module but this is often confusing and inaccurate.
What role do the vendors of head tracker modules play?
 |
| xSight HMD with IntertialLabs head tracker |
Companies that make head trackers -
Intersense,
IntertialLabs,
Hillcrest Labs just to name a few - don't make the sensors themselves. The accelerometers, gyroscopes and magnetometers are from other vendors such as Panasonic, Robert Bosch, InvenSense, Kionix, Seiko Epson, STMicroelectronics, Honeywell and Analog Devices. What the head tracker companies do with these sensors are several things:
- Sensor Fusion. This is arguable the most important function, involving combining the data from the various sensors into output that is more accurate than each of the sensors. For example, the accelerometer senses gravity and thus is very helpful in determining what the vertical axis (up/down) is. However, if the module is in motion, other non-gravity accelerations can cause error in the axis determination. The module can query the gyroscope to see if there is angular velocity and determine the direction of gravity only when the module is not rotating. Similarly, the magnetometer can help calibrate the reading of the gyroscope and align yaw readings with the magnetic north. Module vendors often have their own "sensor fusion" algorithms and that is a key part of their intellectual property.
- Calibration. The individual sensors might not be linear or might output a 'non-zero' reading when they are in complete rest. Calibration parameters can be different from one sensor vendor to another and between two sensors from the same vendor. Calibration parameters can also be temperature-dependent so calibration ends up being an important part of combining sensors into a head tracking module.
- Data smoothing and filtering. Sensor output is noisy. If the heading from a motion tracker module is used to change the viewpoint in a virtual world, sensor noise gets translated into jitter or other undesired image artifacts. Data smoothing could be as simple as averaging samples over a sliding window or as sophisticated as Kalman Filter algorithms. Data smoothing can also be contextual - when the sensor is approaching rest, one kind of data smoothing may be used whereas when the sensor is moving quickly, different data smoothing might be applicable. More on data smoothing below as we discuss performance.
- Predictive tracking. In an effort to reduce the apparent response time of a tracker (more on that below), some trackers including algorithms that try to predict the orientation of the head a short time into the future. For instance, the algorithm could simplistically say "I see that the head rotated left at a constant rate of 10 degrees/sec for the last couple of seconds, so I could estimate where the head will be 20-30 mSec into the future".
- Format translation. Orientation data can be consumed in various formats by the application: Euler angles, Quaternions, Rotational Matrices and more. An on-board processor on the module often provides the host computer with this data in the most convenient format. Aside from the actual content data, modules can offer the data in various physical layer and low-level protocols such as USB, RS-232/RS-485, SPI and more.
- Additional processing. Some modules go as far as including an on-board gesture engine that can analyze movements and report gestures when recognized. Those modules that perform this kind of additional processing on-board help reduce computational load on the host computers.
Key performance parameters
There are several key performance parameters that differentiate between the various tracker modules.
- Refresh rate. The rate - number of times of second - that a tracker can report new orientation data has come into the spotlight in the last year with the increased focus on virtual reality gaming. The quicker the game is able to sense the motion of the user, the lower the delay will be between motion and screen updates. However, one must pay attention to what data is received in each cycle - is it raw or smoothed/filtered. The sensors on a tracker module can report raw data very quickly, but if the tracker module does not provide smoothing and filtering, the host computer will have to do that. For instance, a tracker module might provide 1000 updates per second of raw data, but 250 updates per second of processed, smoothed data. The choice of the physical interface between the tracker module and the host can impact the refresh rate. For instance, using an RS-232-based protocol would likely limit the number of messages that can be received per second as a result of the baud rate and message length.
- Resolution. A low resolution tracker module would create a noticeable distraction in the visual experience, especially when moving the head in a low speed. As an example, if a tracker only reported yaw readings in 1 degree resolution, rotating the head slowly in a head-tracked environment would result in a 'ratchet' effect where the image stays static in spite of head movement and then makes a large, noticeable jump once the tracker sensed a different position. This is particularly noticeable in images that simulate narrow field of view such as when looking through a rifle sight. A good tracker would provide resolutions that are better than 0.1 degrees in all directions.
- Accuracy. Resolution aside, how much is the tracker reading correlated with actual yaw/pitch/roll? Accuracy matters when trying to match tracker orientation with a real-world object. For instance, an completely inaccurate yaw heading would make navigation practically impossible. Another example might be in augmented reality applications where the application needs to place a graphical overlay on a particular point in real space. Accuracy also matters when using two or more orientation orientation trackers in the same application. For instance, if a soldier is wearing a tracked HMD and is aiming a simulated weapon that is also tracked, matching the orientation of the HMD with the weapon is very important for training effectiveness.
- Repeatability. This shows whether returning to the same orientation heading in the physical world also equates to obtaining the same readings from the tracker module.
 |
| Compass Interference |
- Magnetic field compensation. Have you ever seen the compass interference sign on an iPhone navigation app? This happens because other metal objects in the area cause confusion with the built-in compass/magnetometer, resulting in unstable heading reading. In an HMD context, this could be a problem in certain usage scenarios. I remember doing a demo at USC a few years ago and we set up the demo right underneath a large metal sculpture hanging from the ceiling. It took us awhile to understand why tracking was completely off! While in our case we could move away from the sculpture, a soldier that uses an HMD to train inside a tank cannot move away from the tank. In this case, some vendors can ignore the magnetometer, which precludes the tracker module from understanding where 'true north' is but provides a reasonably stable but slowly drifting reading. Other more sophisticated solutions include manual or automatic algorithms to sense and compensate for the surrounding metal objects.
Trackers modules for head tracking vs. tracker modules for other uses
Orientation trackers can be used for many applications that are not HMDs: remote control devices, smart phones and more. Even though the underlying technology is the same, HMD applications often need a tweaking of tracker parameters such as smoothing. Several years ago,
my company started using a tracker that was originally made for a 3D pointer. When such a pointer is placed on a desk, it is not being used for pointing and the original firmware used this state to reset various tracker parameters and turn the tracker off to save energy. However, when an HMD moving very slowly, it may be that the user is trying to focus on a particular feature in the virtual world. Resetting tracker parameters - thus causing yaw/pitch/roll correction - is the wrong thing to do. We worked with the tracker vendor to correct this in a special firmware build, but the lesson is that one needs to investigate the tracker performance settings and make sure that they are optimized for head-mounted use.
Comment? Question? Correction? Write a comment and I will try to address it.
Have a VR-related topic you'd like me to discuss? Write a comment and I'll consider it for a future post.
For additional VR tutorials on this blog,
click here
Expert interviews and tutorials can also be found on the
Sensics Insight page here
