Development of a 3D thermal imaging camera system to perform analyses from the air

Oblique aerial camera systems—which record high-resolution three-dimensional images—are often used to map urban areas and obtain geographic data from the air. Until 2017, none of these systems were able to record 3D thermal images. In response to this need, a team at Anhalt University of Applied Sciences in Dessau, Germany, developed a thermal imaging/RGB system that produces 3D imagery by overlapping pictures from four digital and four FLIR A65sc cameras with 25° fields of view.

The Institute for Geoinformation and Surveying

The Anhalt University of Applied Sciences in Dessau has been providing courses in the field of surveying since 1992, and in geoinformatics since 2002. The former surveying department was transformed into the Institute for Geoinformation and Surveying. As an institute within the Faculty of Architecture, Facility Management and Geoinformation, it bundles the academic and research competencies of the surveying and geoinformatics disciplines. In addition to teaching, the institute focuses on applied research.

The Idea and Its Areas of Application

One of the institute's projects consisted of developing a new thermal imaging and RGB camera system, which produces 3D images by overlapping the pictures taken from a gyrocopter using eight cameras. Professor Lutz Bannehr, who heads the Institute's departments of Geodata Acquisition and Sensor Technology, came up with the idea in April 2016. While 3D camera systems (so-called RGB oblique camera systems) with very high resolutions were available, none of these systems could provide the benefits of thermal data. Professor Bannehr had experience with thermal imaging, having purchased a FLIR SC3000 cooled infrared camera in 2001 and participated in thermography training. He was certain that a solution using uncooled thermal cameras would be feasible. There could be many potential uses, including: collecting inventory data, surveillance, volume monitoring in opencast mining operations, forest fire monitoring, insulation analysis, yield estimates for photo-voltaic and solar heating systems, environmental monitoring, geological and topographical imaging, and even producing digital city models.

The Project

Professor Bannehr submitted the research proposal and put together a small development team including his Ph.D. student Christoph Ulrich, Hermann Kaubitzsch from bgk infrarotservice GmbH (Riesa)—a FLIR integration company—and Henrik Pohl from gyrocopter producer, Airborne Technical Systems (Berlin). Even with overlaps of 85% in both length and width, conventional high-resolution cameras could not depict every detail of the sides of buildings. Therefore, the team designed a system consisting of four RGB cameras and four thermal imaging cameras arranged so that their images would overlap to produce 3D thermal images and 3D geodata.This data could then be analyzed and evaluated using standard software.

The Cameras

To create the system, the team selected four A65sc FLIR thermal imaging cameras as well as four compact RGB cameras from another manufacturer that produce images with approximately 5 megapixels. Kaubitzsch recommended the team use the FLIR A65sc thermal imaging camera "due to its good thermal image resolution of 640 × 512 pixels, its frame rate of 30 Hz, its Ethernet port and its extremely compact dimensions of 106 × 40 × 43 mm." Hermann Kaubitzsch was also responsible for synchronization and evaluation of the cameras, which turned out to be no small task.

The AOS-Tx8 includes a helical arrangement of four A65sc FLIR thermal imaging cameras and four compact RGB cameras, which all produce overlapping images.

AOS-TX8 Controls and Specifications

A team of students developed a 3D arrangement for the eight cameras, which had to take up as little space as possible in the ultralight aircraft. An open custom-made baseplate was even made to mount the system in the gyrocopter. A name for the "Aerial Oblique System" was also quickly found: AOS-Tx8. The system is controlled via Ethernet and image data is displayed on a 10-inch screen. "A few years ago, we experimented with a different thermal imaging camera, but controlling it via Ethernet did not work as promised," explained Professor Bannehr. "With the FLIR A65sc models, however, this was no problem." The entire AOS-Tx8 system only weighs 11.6 kg and measures 330 × 400 × 320 mm. It provides for connections for manual camera operation and the flight management system as well as a mouse, screen, keyboard (all via USB) and the power supply.

During calibration, the IR cameras showed a deviation of up to 5°C compared to a reference spotlight. Camera 1 was selected as a reference camera and the other cameras were adjusted to match its reference values.

Synchronization of the Thermal Imaging Cameras

The overlap between the FLIR cameras is 12% or 3°. The four FLIR thermal imaging cameras had to be synchronized to obtain useful data and avoid temperature variations in the measured values where the images overlap. Due to technological reasons, uncooled thermal imaging cameras have a variance of up to ±5% in temperature measurement. A test of all four cameras using a reference spotlight showed that the predicted deviations were also present, but that they were distributed linearly across the spectrum. It was therefore possible to use one of the cameras as a reference  camera (ideally the one with the mean value) and then adjust the other cameras to match the reference camera.

Maiden Flight

On August 15th, 2017 the time had come to try it out. The AOS-Tx8 had been mounted in the gyrocopter and was ready to make its initial measurements from the air. The sequence of events had already been clearly defined. The team used flight planning programs to plot the maiden voyage and used Google Earth for the map data. The flight planning data including the points where pictures are to be taken are copied into the flight management system. 

The AOS Tx8 mounted in the gyrocopter before its maiden flight in August 2017.

During flight, this data was used to trigger the AOS-Tx8 and other sensors. Following test flights over the Anhalt University's Strenzfeld campus, flights were also made in August over Magdeburg. Images were not only taken from a vertical angle (so-called IR orthography images), but also used to produce large-scale 3D thermal images that also showed effectiveness of insulation on buildings.

Results Achieved by the Aerial Obliqu System (AOS-TX8)

Using the AOS-Tx8, it is possible for the first time to obtain a digital surface model with precise determination of building heights as well as a digital terrain model in RGB and infrared. The AOS-Tx8 is easy to operate and the data can be evaluated using standard software products such as Photoscan or Pix4D.

Outlook

The institute now also has a FLIR A655sc for aerial data collection that does not require 3D imaging. We can therefore look forward to the next infrared research project conducted by Professor Bannehr and his innovative team at the Institute for Geoinformation and Surveying at the Anhalt University of Applied Science in Dessau.