Jeffrey McWilliams is a seasoned professional in thermal sensing and automotive electronics, currently serving as Technical Sales, Thermal Sensing at Magna Electronics. He holds a Bachelor of Science in Electrical Engineering with a focus on analog microelectronics from the University of Michigan.
1. DVN: Can current RCCB cameras meet the specs for the NHTSA FMVSS 127 AEB regulation? Is SWIR an option? Will quantum dot allow IR cameras to reach a reasonable cost? Can a single (Forward) camera be used for level 2 and meet nighttime AEB specs or do we need a separate solution for this?
- Regular cameras, such as RCCB, are generally sufficient for the AEB component of FMVSS 127, but their adequacy for the PAEB part is uncertain. Compliance with FMVSS 127 is not the sole criterion; the system must also not pose an unreasonable risk to motor vehicle safety.
- For robust performance, these camera systems should ideally be supplemented by radar. Even with this supplementation, there may still be challenges at night, particularly with detecting stopped vehicles in traffic lanes that lack taillight illumination.
- SWIR technology’s effectiveness for PAEB would necessitate active illumination. The primary issue is not SWIR itself but the active illumination systems required. To avoid interference from other light sources in the same wavelength, gating is necessary. Alternatives to SWIR exist, such as regular CMOS FPAs, which are sensitive in the NIR spectrum (<1000nm). Although the quantum efficiency diminishes with longer wavelengths, CMOS FPAs are a viable alternative. However, careful configuration of the illumination is required to ensure it meets eye-safe class 1 standards, a domain in which we hold patents
- A single forward-facing camera might suffice for level 2 automation and meet nighttime AEB specifications, but a more comprehensive solution, potentially involving supplementary radar and advanced illumination technologies, will likely be necessary to ensure robust performance and compliance with safety regulations.
2. DVN: Are LWIR Thermal cameras a better solution and if so what are the pros and cons? Can they achieve the costs required? How does power compare to other solutions?
- LWIR thermal cameras offer significant benefits in darkness for detecting objects with distinct heat signatures. However, they are not without drawbacks.
- The cost of LWIR thermal cameras in the automotive industry is a critical consideration. While the target is generally to reduce costs, the real issue is the cost of alternative solutions to comply with FMVSS 127, including the risk of recalls. Regular FPAs are inexpensive but may not perform adequately in darkness, potentially leading to inadvertent braking. Lidar solutions, although viable, are currently more expensive. Other active systems, such as gated imaging, present interesting options but may not yet match the cost-effectiveness of thermal cameras. As thermal camera volumes scale up, their costs are expected to decrease, especially with solutions optimized for augmenting AEB/PAEB systems to meet nighttime PAEB mandates. Our thermal product is specifically targeted for this purpose.
- One of the primary applications of thermal sensors is to meet pedestrian AEB requirements as standalone sensors. However, in practice, they serve as additional sensors that support the existing technology in the vehicle. This integration provides a more comprehensive and real-world aspect to the functional safety of the vehicle. By enhancing the capabilities of current ADAS systems, thermal sensors contribute to a more reliable and effective safety mechanism, ensuring better protection for pedestrians and other road users.
- In terms of power consumption, thermal cameras are passive sensors and thus have low power consumption. In contrast, active solutions, including imaging radar, consume more power than thermal cameras.
3. DVN: Is HD Radar an alternative option for this application, and what are the pros and cons of radar versus the other solutions proposed today?
- All sensors have their strengths and weaknesses. Magna believes that a complementary sensor suite, such as a combination of HD radars and thermal sensors, provides the best system-wide solution. HD radars show significant improvements over traditional radars and can meet almost all AEB cases. They excel at distinguishing small objects in all weather conditions, providing range and velocity information. Although identifying pedestrians near guardrails can be challenging for some HD radars, Magna has resolved this issue. Additionally, classifying nearly motionless pedestrians is difficult, but multi-modal systems, where thermal sensors augment radar data, can fully classify pedestrians.
4. DVN: LiDAR is still expensive – US/EU vendors might approach $500 in this timeframe and China vendors might be half of that – will that be a better approach for AEB systems?
- The question of whether LiDAR is a better approach for AEB systems should focus on the cost to comply with the regulation rather than a subjective definition of “better.” LiDAR remains relatively expensive, with US/EU vendors pricing it around $500 and Chinese vendors potentially offering it at half the cost. In contrast, Thermal sensors can already achieve a price point significantly below $500 even in low volumes. Additionally, LiDAR, as an active system, consumes more power compared to thermal sensors. Therefore, considering both cost and power consumption, thermal sensors are likely a more cost-effective solution for AEB systems.
5. DVN: The AEB solution requires more than just the sensor of course – is the AI Component of the solution best in the camera/radar/lidar module or in central compute? What sort of processing is needed to achieve this (TOPS)?
- The AEB solution requires more than just the sensor; it also necessitates a robust perception stack, which includes machine learning (ML) components. The placement of this perception stack—whether in the camera/radar/lidar module or in central compute—depends on various factors and the OEM’s approach.
- The processing requirement, measured in TOPS (Tera Operations Per Second), is linked to sensor resolution but is not the only factor to consider. Thermal imaging can achieve PAEB with lower resolution compared to other imaging alternatives, while high-resolution radar currently has lower resolution. The specific compute requirements for high-resolution radar should be addressed by the radar team.
6. DVN: Will the AEB system be a separate domain, or will it always be part of the L2 or L3 driving system?
- The AEB system, mandated by FMVSS 127 for all light vehicles starting in September 2029, is not dependent on any SAE classification for L2, L3, or L4 driving systems. While some may include AEB within Advanced Driver Assistance Systems (ADAS), AEB/PAEB functions are typically hidden from the driver until triggered by an event. Therefore, it is not appropriate to bundle them with ADAS or any level of Automated Driving Systems (ADS). However, all ADS systems will also need to comply with FMVSS 127.
- Functional partitioning and system design depend on the OEM’s preference and implementation. AEB and ADAS (including L2+/L3) are related features derived from inputs originating from a common set of sensors and are likely implemented on common hardware. It is unlikely that AEB will occupy a distinct “domain” separate from other driving assistance functions. However, safety analysis will be required to demonstrate the robustness and integrity of AEB independently from these other features.
