Thermal Stereo Solutions for Seeing Through Dust in Rugged Environments

Stereoscopic Long-Wave Infrared Imaging Shows Promise for Industrial Autonomous Vehicles Navigating Low-Visibility Conditions

Autonomous vehicles (AVs) have shown to work well in ideal weather conditions with clear skies, well-marked roads, and predicable routes. However, creating a system that can handle all types of weather and environmental conditions is significantly more challenging. Those issues become more pronounced in the development of autonomous and unmanned vehicles and other ground-based systems designed for off-road use, especially for vital industrial functions that power the global economy including resource extraction, agriculture, and construction. One of the key hurdles to overcoming harsh environments is accounting for particulate matter in the air, especially dust.

Existing sensor technologies found on vehicles equipped with advanced driver assistance systems (ADAS) and AVs today are degraded in dusty environments by reduced visibility and range of visible (RGB) cameras. Radar can effectively penetrate adverse conditions, including dust, but can only capture a partial picture of scene—it can’t classify objects nor distinguish between a living thing (the thing drivers least want to hit) and an inanimate object, a critical aspect to improving workplace safety as it relates to transportation. Radar also suffers from high false positive rate which limits its ability to operate independently without fusion with other sensors, typically mono cameras.

Perception Through Dust with Thermal Imaging

Through testing conducted by Teledyne FLIR and other government-funded programs, long-wave infrared imaging (LWIR), which detects electromagnetic light in the 8 μm to 14 μm spectrum band, has proven to be effective at detecting objects where particulate matter suspended in the atmosphere reduces visibility. Thermal imaging is especially effective for pedestrian detection in comparison with RGB, near-infrared and short-wave infrared camera systems at night.

Improving Perception with Stereoscopic Thermal Imaging

To further improve the perception accuracy of thermal imaging cameras, pairing two together stereoscopically adds a third dimension to the data captured. Stereo paired thermal cameras can provide crucial depth perception through dust, including the ability to measure distances by triangulating the cameras with objects in the field of view.

That comprehension, combined with a convolutional neural network, enables the system to both identify objects and their respective distances to make the most appropriate decisions for the situation, in conjunction with the other sensor modalities mentioned above.

Furthermore, stereoscopic thermal imaging has recently become significantly more effective thanks to automotive vision systems innovator Foresight Automotive, part of the Thermal by FLIR® program. Its industry-first, patent-pending automatic calibration system, is designed to ensure that stereo cameras remain calibrated regardless of their configuration or position, in order to create accurate and continuous depth perception.

The image below shows data captured by thermal stereo on the left versus visible-light stereo on the right. The thermal cameras easily detect the vehicle, including its distance 55.6 meters ahead, whereas the visible cameras fail to detect and classify the vehicle.

Stereo thermal cameras can dramatically improve obstacle detection and distance measurements within dusty environments day or night. Photo credit: Foresight

A Promising Future for Industrial Autonomous Vehicles

For industrial environments, heavy machinery manufacturers have already made significant progress in bringing AVs to market. Caterpillar noted recently that its autonomous trucks have afforded its customers significant gains in productivity and truck utilization rates with lower costs per ton, generating up to 30 percent higher productivity. However, to achieve wide adoption, there remain significant technological gaps in industrial environments, just as there are on public roadways. Stereoscopic thermal imaging can play a crucial role in helping close those gaps, especially in dust, blowing sand, or other low-visibility conditions.

For more background on thermal imaging for ADAS and AVs, visit https://www.flir.com/oem/adas/. For more about stereoscopic imaging from Foresight, visit: https://www.foresightauto.com/