Thermal Imaging to Boost Pedestrian Safety
Vehicle vision solutions built on IR sensors are ramping up, with the aim to help reduce nighttime pedestrian fatalities. For developers of IR sensors and cameras, the race is on to position SWIR and LWIR technologies for mass adoption by the automotive industry.
Last year, the NHTSA finalized FMVSS 127, which requires automatic emergency braking (AEB), including pedestrian AEB (PAEB), to be standard equipment on all passenger cars by September 2029. New vehicles will have to have PAEB systems to help nighttime drivers avoid collisions with other vehicles when traveling at 62 mph (~100 km/h) and pedestrians when traveling at 37 mph (~60 km/h).
A car equipped with PAEB traveling at 37 mph needs about 148 ft (45 m) to stop. It is possible to achieve high-confidence detection of a pedestrian at this distance with a low-resolution thermal camera (<0.1 megapixels). According to Sebastien Tinnes, global market leader for IR detectors designer and manufacturer Lynred, a 12-µm pixel pitch quarter video graphics array resolution thermal camera (320 × 240 pixels) can fulfil all FMVSS 127 requirements.
Currently, thermal imaging and lidar are the leading technologies to meet the PAEB requirements, but other technologies exist as well; the ADAS in most vehicles primarily use radar, and some also include visible cameras.
Strengthening synergies among detection technologies will be crucial for complying with FMVSS 127. While ADAS work well during clear weather conditions, their effectiveness diminishes in poor weather and darkness. Teledyne FLIR has seen increased demand from automakers to expand sensing capabilities, often adding a thermal camera to existing solutions.
Sensor fusion combines multiple modalities into a common system, but it’s still evolving in the context of ADAS. Last fall, Teledyne FLIR partnered with Ansys and Valeo to integrate thermal cameras for night-vision ADAS, enhancing functions like nighttime AEB for various vehicles.
Teledyne FLIR’s next-generation automotive thermal camera will be smaller and lighter, with a resolution of 640 × 512 pixels, surpassing previous systems. Comparisons between thermal and visible sensors highlight advantages such as size, cost, and performance. Lynred plans to introduce a product with an 8.5 µm pixel pitch within the next two years.
Many of the technologies involved in autonomous mobility and automotive safety systems are well established. Sensing and imaging in the SWIR band, for example, is often used in harsh environments and nighttime settings. This approach provides high-resolution images in such challenging conditions. Solutions for industrial manufacturing, surveillance, counterfeiting detection, and food inspection use SWIR sensing and imaging.
Israeli startupTriEye is developing a CMOS-based SWIR sensing solution for ADAS. TriEye’s detection and ranging platform uses a CMOS-based high-definition SWIR sensor that combines the functions of a camera and lidar into one modality, offering 2D and 3D mapping capabilities. Unlike LWIR sensors, TriEye’s SWIR technology can be placed behind standard glass windshields and headlamps. Windshield glass is opaque to LWIR light, preventing system integrators from positioning many thermal cameras behind standard windshields due to the limitations of incoming and perceived light.
“The compatibility of SWIR camera outputs with existing computer vision algorithms simplifies the integration process, avoiding the need for extensive training of new deep learning algorithms required by thermal camera systems due to their unique heat-based imagery,” said Nitzan Yosef Presburger, head of marketing at TriEye. Presburger stated that SWIR illumination, compared to MIR and LWIR illumination, is less affected by external environmental conditions. SWIR cameras rely on a photodiode effect that produces detailed views distinct from those based on thermal differences measured by bolometric sensors commonly found in LWIR cameras. Additionally, thermal imaging’s extended wavelength (~10 µm) may reduce resolution by enlarging pixel sizes and limiting pixel density. Larger pixels also necessitate larger lenses for thermal cameras compared to visible light counterparts.
TriEye’s technology seeks to address the challenges posed by traditional windshields while taking advantage of the benefits of SWIR imaging.
However, SWIR solutions may be costly, and according to Merrill, these cost concerns are pushing automotive industry customers away from SWIR and towards LWIR. SWIR sensing and imaging’s reliance on active illumination also makes it susceptible to interference from living light sources. The additional light sources needed to support SWIR sensors add costs to current solutions as well as creating power concerns in some instances.
“If all cars are equipped with SWIR illuminators, it creates a similar effect to headlamps for visible cameras: the camera can be overwhelmed by other illuminators from other vehicles,” Tinnes said.
System integrators have historically positioned LWIR thermal cameras on the grills of vehicles due to their inability to function effectively behind a standard windshield. These exterior-situated cameras do not benefit from windshield wipers that clean water and debris from the lens. However, methods such as hydrophobic coatings, air blasts, and spray nozzles are often employed to clear the window, as noted by Merrill.
In response, several automotive industry suppliers have developed windshields conducive to LWIR. For instance, last September, Lynred, in collaboration with glass experts Saint-Gobain Sekurit, introduced a design that integrates visible and thermal cameras. This windshield features a section that is transparent to longwave radiation, allowing a sensor fusion that enhances daytime detection rates through redundancy and extends the operational design domain of PAEB at night. This third-generation windshield with improved integration was showcased by the collaborators at AutoSens USA 2024 in Detroit, according to Tinnes.
Furthermore, Tokyo-based glass technology company AGC has developed a far-IR (FIR) sensor-enabled windshield. Windshields are typically composed of a single material and must adhere to stringent standards, including those for reliability and safety. Introducing a FIR-conducive material posed several challenges, such as ensuring resistance to scratching from sand-mixed water drops and ultraviolet light damage. To address these issues while minimizing glare, AGC created an optical coating film and anticipates integrating FIR windshields into new vehicles by 2027.
Many LWIR cameras benefit from designs that optimize these imagers for military use. However, these cameras may not offer levels of adaptability that make them suitable for the wide temperature ranges demanded by automotive standards, said Raz Peleg, vice president of business development at smart thermal sensing technologies developer Adasky.
This quality makes athermalization, or the consistent performance in varying environmental conditions, critical, he said.
Adasky has developed a shutterless LWIR camera that is designed to withstand the harsh automotive requirements of −40 to 85 °C (−40 to 185 °F). Adasky’s camera weighs <50 g and is sized at 26 × 44 mm. Similarly, Teledyne FLIR’s forthcoming automotive thermal camera is specified to operate athermally from −40 to 85 °C ambient temperature, Merrill said. However, Adasky has designed their solution to be located outside the vehicle. It is envisioned to be paired with a heated lens to prevent frost. “LWIR is a camera-based modality. The camera needs to stay clean, and the OEM is responsible for cleaning it,” Peleg said. He said that the small size of the camera supports high mounting, such as on the roof of the vehicle. This positioning would help to minimize the effects of debris and reduce damage, according to Peleg.
Though the Adasky and Teledyne solutions differ in the specifics of how they are to be deployed, both reflect the advantages of harnessing the LWIR band for automotive. According to Merrill, tests conducted at Sandia National Laboratories’ fog facility in 2021 showed that LWIR sensors provided the best overall visibility through various fog types and densities. The Teledyne FLIR-Sandia tests compared LWIR results to those obtained via visible, MIR, SWIR, and lidar sensors.
Whether the automotive thermal cameras are integrated inside or outside the windshield, they still face cost challenges. As designers pursue cost-effectiveness, autonomous intelligence, a subset of AI, is already making inroads in that direction. Chuck Gershman, CEO and cofounder of Owl Autonomous Imaging in Fairport, New York, says the three biggest roadblocks facing cost reductions to thermal cameras are converters that turn sensor analog outputs into digital data; the processor that subjects the sensor data to nonuniformity corrections; and the need to provide flat image data via a mechanical shutter for defining the correction factors.
Owl has developed a thermal HD camera system that performs the necessary digitization and corrections in a readout circuit in a semiconductor layer below the microbolometer array. As a result, the mechanism bypasses the costs that are typically associated with a mechanical shutter. This design also supports improvements in resolution that are most straightforward, reducing the need for high amounts of circuitry around the sensing area. While most LWIR cameras perform digitizing, correction, and interfacing in circuitry on several PC boards, Owl’s design leverages the sensor device to perform many of those functions. This supports a single-board camera with designs similar to its visible light counterparts.
“Properly executed internal corrections for nonuniformity and temperature remove the need for additional preparation of the image for the classification and range-finding tasks,” Gershman said. “AI, therefore, is needed only for tasks directly related to extracting the data used by the ADAS, not for fixing bad images.”
Owl AI’s software identifies hundreds of objects at frame rates of 30 per second. Currently, the AI can identify several types of objects, including pedestrians, cyclists, some animals, and various types of vehicles. It can accurately process thermal signatures to ~40 m, and Owl aims to extend that range to 180 m with their next-generation system.
At the same time, Adasky’s shutterless LWIR offering has also incorporated generative AI in its synthetic thermal data set. This incorporation has enabled the company to advance detection (≥100 m) as well as classification.
DVN comments
The tests conducted in 2021 at Sandia National Laboratories showed that LWIR sensors provided the best visibility through different types and densities of fog. These tests compared the LWIR results to those obtained via visible, MIR, SWIR and lidar sensors. Automotive thermal cameras, whether inside or outside the windshield, face cost challenges that delay their integration in new vehicles. Nevertheless, the cost of IR cameras has decreased significantly, targeting $100 by 2029.
Omnivision’s New 1.5-mpx DMS Sensor
Omnivision has a new OX01N1B image sensor for in-cabin automotive driver monitoring systems (DMS). Part of the supplier’s Nyxel near-infrared (NIR) technology family, it is a 1.5-megapixel RGB-IR or monochrome BSI global shutter sensor with a pixel size of 2.2 µm and an optical format of 1/4.51″. It was showcased at Auto Shanghai from April 23-May 2, 2025. Key features include 36 per cent NIR quantum efficiency for low-light performance, high MTF for improved image quality, low power consumption, and compact camera module design. Using OmniPixel 4-GS technology, it accurately reproduces rapid motion without distortions.
“With DMS becoming mandatory in Europe by 2026, and increasing adoption globally, OMNIVISION introduces the OX01N1B as a mainstream solution balancing performance, size, and cost,” said Dr. Paul Wu, head of automotive product marketing. The OX01N1B’s chip size is smaller than its predecessor but retains the same optical path, with added image signal processing. It offers flexibility for OEMs to place the DMS camera in various vehicle designs.”
Smart Eye CEO Martin Krantz praised the sensor’s performance and compact format for their integrated DMS innovation, the AI ONE camera.
The OX01N1B has integrated ASIL-B and cybersecurity features, meeting industry standards. It comes in an OMNIVISION a-CSP package, and is available in a reconstructed wafer option for bare die assembly. Samples are available now, with mass production slated to start in Q3 2026.
DVN comments
The automotive CMOS image sensor market is projected to grow at a 5.4-per-cent CAGR through 2029, mainly driven by in-cabin applications like driver monitoring systems (DMS), occupant monitoring systems (OMS), facial recognition, and gesture recognition. Driver eye-tracking utilizes global shutter pixels to capture fast-moving pupils without distortion. Smaller pixel size of 2.2µm offered by this new retina provides higher resolution but face challenges such as slower shutter speeds and noise from miniaturized circuits.
Indie’s New Vision Processor Family
Indie Semiconductor has launched the iND880xx vision processor product range for ADAS and driver vision applications, such as surround-view systems and electronic mirrors.
These new processors augment Indie’s portfolio of visual, lidar, radar and ultrasound sensing solutions. The new sensors deliver low-light and high dynamic range sensing to improve perception performance. They support real-time image signal processing for four simultaneous sensor inputs, including emerging sensor types like infrared. This capability enables simultaneous human viewing and ADAS sensing within a single system-on-chip (SoC) platform.
Key applications of vision sensing include automated emergency braking (AEB), lanekeeping assist, blind spot detection, driver and occupant monitoring, smart reverse, and vulnerable road user protection—that lattermost one the subject of FMVSS 127, which includes stringent testing requirements for pedestrian AEB under low-light and nighttime conditions without streetlights.
An automotive camera, to deliver usable images, requires an effective lens, a high-fidelity digital image sensor, and an advanced video processing engine to retain, extract, and highlight relevant information about a vehicle’s environment.
Central to the video processing engine is the image signal processor (ISP). Several attributes contribute to a reliable ISP. It should use the image sensor’s capability to see farther while ensuring near-real-time responses to changing road and environmental conditions. The ISP should not be affected by bright lights, yet it should preserve features in dark regions of a scene. It should support various image sensor technologies and spectral combinations while maintaining color fidelity. Additionally, it should support both human vision applications such as surround view, backup assist, and e-mirrors, and machine vision sensing applications used in ADAS. These applications can have different requirements and may require support for multiple video streams concurrently at high pixel throughput rates.
The new Indie iND880xx video processor SoC family excels in image signal processing for both viewing and sensing. It addresses low-light performance and color differentiation, crucial for perceiving traffic signals.
The ISP pipeline supports simultaneous processing of four sensor inputs with a throughput of 1,400 MP per second. Key features include a 24-bit processing pipeline for HDR up to 144 dB, support for various CFAs (RGGB, RCCB, RYYCy, RGB-IR), and proprietary eWarp technology for correcting image distortion from wide-angle lenses.
Indie’s chief product officer Abhay Rai calls the new processors “a major advancement in automotive vision processing, meeting growing ADAS demands and consumer expectations in challenging environments.”
DVN comments
Indie is involved in developing semiconductors, photonics, and software platforms for the automotive industry. The company focuses on creating technology for ADAS, in-cabin user experience, and electrification applications. Their mixed-signal SoCs support edge sensors including radar, lidar, ultrasound, and computer vision. Their embedded system control, power management, and interfacing solutions enhance the in-cabin experience and support automated and electrified vehicles. Indie is a recognized vendor to tier-1 partners, and their solutions are used by major automakers worldwide.
