"Millimeter-wave radar + vision": an essential perception fusion solution for autonomous driving and high-speed cruising

Autonomous driving technology is gradually moving from science fiction to real life, and one of the key technologies is sensor fusion, especially the combination of millimeter-wave radar and vision systems. This fusion solution provides autonomous vehicles with more powerful and reliable sensing capabilities, especially in high-speed cruising.

Characteristics of millimeter wave radar

Millimeter wave radar measures the distance, speed and angle of objects by emitting electromagnetic waves in the millimeter wave band and receiving the reflected signals. It has the following salient features:

Working around the clock:

It is not limited by light conditions and can work stably in rain, fog, smoke and dust environments.

High penetration:

Able to penetrate some non-metallic objects and obstacles.

high resolution:

The new 4D millimeter-wave radar can provide richer point cloud data and enhance target recognition and classification capabilities.

Advantages of visual systems
Visual systems, mainly vehicle-mounted cameras, can provide rich color and texture information, suitable for object classification and recognition. The advantages of visual systems are:

Capturing details:

It can capture finer visual details and help identify the shape and category of objects.

Color recognition:

The ability to recognize and differentiate between different colors helps with the identification of traffic signals and signs.

The need for integration

Single sensor systems have limitations, such as the performance of cameras degrades in poor lighting conditions and millimeter wave radars are not as accurate as vision systems in object classification and recognition .

By integrating millimeter-wave radar and vision systems, the advantages of both can be complemented to achieve better autonomous driving perception functions during high-speed cruising :

Improved accuracy:

The distance and speed information provided by the radar assists the vision system in more accurate object detection.

Enhanced robustness:

In poor weather or lighting conditions, radar can be a powerful supplement to vision systems.

Improved safety:

fused system can identify potential dangers earlier, providing more reaction time for the autonomous vehicle.

Front Fusion:

Fusion is performed at the raw data level, combining the radar’s point cloud data and the camera’s image data and inputting them together into the processing algorithm.

Feature-level fusion:

Fusion is performed in the feature extraction stage. After the radar and vision systems independently extract features, the features are merged for subsequent processing.

Decision-level fusion:

Fusion is performed in the decision-making stage, that is, after the two sensor systems independently complete object detection, the final judgment is made by combining the results of the two.

Application Cases

In practical applications, the integration of millimeter-wave radar and vision systems has been widely used in autonomous driving functions such as adaptive cruise control (ACC), forward collision warning (FCW), and automatic emergency braking (AEB). For example, the autonomous driving system of some Tesla models on sale uses a fusion solution of cameras and radars to achieve its high-speed cruise autonomous driving function.

As technology advances, the fusion solution of millimeter-wave radar and vision systems will become more mature and popular. Future autonomous vehicles will be able to achieve higher levels of autonomous driving functions, such as fully autonomous driving on highways. At the same time, as costs decrease and performance improves, this fusion solution is expected to be applied to more models around the world .

In short, the "millimeter wave radar + vision" fusion solution is one of the key technologies to realize the high-speed cruise function of autonomous driving. It integrates the advantages of the two sensors to provide autonomous driving vehicles with more powerful and reliable perception capabilities. With the continuous development of technology, we have reason to believe that this fusion solution will play a greater role in the field of autonomous driving in the future.