How to make autonomous driving more reliable?

We are under actual conditions,

It cannot be improved infinitely

Autonomous driving intelligent technology itself,

Instead, you should consider car, people and society,

Continuously expand our security boundary.

Li Huiyun · Deputy Director of the Institute of Integrated by Shenzhen Advanced Technology Research Institute of China Academy of Sciences

Gezhi Dao Dao · Bay Area No. 16 | March 30, 2022 Guangzhou

Hello everyone, I am Li Huiyun from the Shenzhen Advanced Technology Research Institute of the Chinese Academy of Sciences. Today I shared the safety boundary of exploring autonomous driving.

Autonomous driving is the future trend, getting closer to us. Presumably, everyone often heard that in some closed parks, such as docks and mines, unmanned trucks are already working, and sometimes some low -speed unmanned delivery vehicles can be seen on the road.

The level of autonomous driving is divided into L0-L5, which is a transition from the completion of the driving task of humans to the machine to complete the driving task.

If there is no active auxiliary driving function, it is what we call L0. At this time, people's hands, eyes, brains, and feet are comprehensively cooperated to complete the task of driving.

At the L1 level, the vehicle can complete some autonomous vertical control. Our feet can temporarily leave the throttle and brakes to enter a fixed speed cruise.

By the L2 level, we can temporarily liberate the hand and eyes, and we can drink a water or something. The vehicle will automatically keep the lane and speed cruise, and even perform simple overtaking behaviors.

By the L3 level, intelligence gradually appeared on the stage. We can basically liberate our hands and eyes without care about driving tasks. We can even watch the mobile phone, but we still cannot liberate the brain. When we encounter an emergency, we may require us to complete the takeover in tens of seconds or even 10 seconds.

By the time of L4, our hands and eyes are basically liberated, and we can "lie down".

At the L5 level, you can even cancel the steering wheel, brakes and throttle. At this time, it is called unconditional autonomous driving.

How does a autonomous driving accident happen?

Completed: 10%///////////

The technology of autonomous driving is changing with each passing day, but people still have some concerns.

Police investigations in 2021 found that consumers in China and the United States have many concerns about driverless driving. Especially for security correlations, such as technical failures, hackers, how to ensure the safety of passengers and pedestrians when accidents.

These hidden concerns of safety are reasonable, because we will hear such accidents from time to time: for example, a factory's autonomous driving technology mistakes white trucks to believe that the sky has been mistaken for the sky, and similar accidents have occurred, causing 2 deaths and 2 injuries. This is beyond the cognition of "more reliable machine computing and mechanical performance" we usually hold.

In fact, the safety of autonomous driving is intertwined in many ways. The most important thing is that the development of vehicle intelligent technology is not enough, and the cost limit has also caused us not to adopt a sufficient expensive sensor. At the same time, industry regulations are not perfect, and the test verification is quite insufficient.

For example, the accident just mentioned in the L2 -level autonomous driving scheme with a camera and millimeter wave radar -based sensor. The camera and the millimeter wave radar can detect the target a certain degree. Any detection of the target can be avoided, over -vehicle, or emergency braking. But they usually have headaches for static vehicles. Why?

Because our camera, that is, the visual sensor, will continue to perform various types of target detection on the front and around during driving. Generally speaking, pedestrians, roads, lane lines, front cars, and even buildings and trees can be accurately detected.

However, it also has limitations, especially for some static, such as the pedestrian photos or signs on the bus and advertising screen that is considered real. We can imagine that if you frequently trigger emergency braking, this driving experience may not be so good. Therefore, the manufacturer will not use the target of the visual sensor as a single judgment of emergency braking. At this time, the millimeter wave radar needs to play a role. The millimeter wave radar uses the principle of wave reflection to detect the target. According to the reason, this is much better than the resolution of the visual target just now, but the millimeter wave radar also has its own limitations.

The millimeter wave radar mainly depends on the goal of moving the Model effect. When the goal is in the move, there is usually no problem: the target is detected, and the corresponding safety operation is performed. But if the goal ahead is still, although there are certain reflex waves, this reflection wave and grounding waves are intertwined, so they are usually filtered out.

The reflection wave of the target information lacks radial Doppler component, and is filtered

The accident we just mentioned is usually the goal in the direction of the waves of the waves, but the amount of reflexes in the radial direction is not much or even. So at this time the millimeter wave radar also filtered it as a non -existent target, or as a target mixed with ground mixed waves. This is why some sad accidents mentioned earlier.

So if we add more sensors, will it be better? It will be better. For example, we can add laser radar. If you are not assured of vision, you can increase the infrared sensor at night at night, or an event camera sensor.

However, each additional sensor brings technical problems such as rising costs, as well as time synchronization, space synchronization, and ruling after a single sensor failure. In other words, to solve security problems, first of all our intelligent technology and sensor technology still have a long way to go.

What kind of industry standards do we need?

Completed: 30%///////////

Secondly, we also have the incompleteness of industry regulations and the lack of test verification.

The Standard for Updated by the International Automobile Engineer Society (SAE) 2021

"Classification and Definition of Road Motor Vehicle Driving Automation Systems"

I will explain industry regulations. For example, this is the definition of some terms of autonomous driving in 2021 of the International Automobile Engineering Association (SAE). Compared with the 2018 version, this version has made great progress, especially for what we have mentioned from what we have mentioned. The classification of L0-L5 is more accurate.

But we still found some places with not clear definitions, such as in the operation design domain, the definition of roads is not clear enough.

Waterproof watch operation design domain

Here is a proper term called "operation design domain". For inappropriate analogy, it is the boundary of the safety operation. Take waterproof watches as an example. If water droplets are splashed, we may need a 30 -meter waterproof level; if we want to cope with daily grooming, it may require a waterproof level of 50 meters; if we wear during swimming, it may require a 100 -meter waterproof level ; If you go to the deep diving, you may need a 200 -meter waterproof level. It should be reliable in the original scheduled operation design domain. And once this security boundary is exceeded, the consequences are difficult to say.

Let's go back to autonomous driving. The definition of autonomous driving on the operation design domain is definitely a bit broader than the waterproof watch. For example, for low -speed unmanned driving, in the international standard ISO22737, there are the following requirements for the operation design domain: in terms of speed, low speed generally refers to less than 35 km/h, or less than 40 km/h.

For another example, is the scope of road applications proprietary roads or open roads? There are also conditions such as weather, lighting, etc., whether there are static obstacles in the driving area, and so on.

What is the definition of security boundaries related to the road just mentioned? I made some labels with the red line, and everyone can see: "A small paragraph" distance, "a small paragraph" time, "longer", these are obviously not clear enough.

There are also "short -term loss" or "wear", which is not clear enough. Why? Because various lane line detection methods can get different identification results.

As shown in the road above, we can see some lanes wear, and we use image processing.

Of course, at this time, the wear lane lines that are worn are invisible, but because the lane lines have some geometric information: for example, it is a straight line or curve, it can be enhanced through the geometric algorithm. Completion, or semantic segmentation, etc.

Therefore, different detection methods will get different identification results, how can they correspond to the "wear" or "loss" of the lane lines mentioned one by one?

Secondly, the lane lines are not clear. For example, in the congested road section, whether it is autonomous driving or human driving, the lane line may not be seen. But for human driving, this does not seem to be a particularly big problem. We just need to follow the car. why? Because the driving of the front car is in line with road geometric information.

For example, in some extreme weather, we can't see the lane line, but human driving can maintain a safe driving by maintaining a certain distance with the front car.

Or in the case of weak lighting conditions, we only need to keep a certain distance and position with the front car.

So, if we want to do a further clarification of the regulations mentioned just now, what work do we do?

For human driving, we may be vaguely felt that maintaining a certain distance and orientation with the front car. But for autonomous driving, it needs a relatively accurate target.

The license plate may be a good target. Because the traffic regulations of various countries are clear about the size, material, and size of the license plates. If the autonomous vehicle is recognized by the license plate as one of the intermediaries, it can probably estimate the distance and orientation of the car.

In this picture, we can see an angle, as long as the angle between the car and the front car is adjusted to make them relatively parallel, and keep a safe distance, the vehicle should be able to run within the security boundary.

We think we should further clarify the regulations from several aspects.

First of all, clarify the concept. Do not "use the lane line" to refer to road information, but should be called "road geometric information". Road geometric information is intertwined in many ways.

Second, it is to quantify the indicators, such as "a short period of distance" and "a short period of time" should not appear, but should replace the actual distance or time based on international units as the benchmark.

Third, the boundary threshold should also be set up to exceed such a security boundary threshold. The vehicle should immediately enable the spare plan.

In addition to lane lines, other perception, decision-making, control, and execution related to autonomous driving should conform to the process of "clarifying-quantitative indicators-establishment of boundary thresholds", so as to enable the vehicle to run under confidence.

How should I test autonomous driving?

Completed: 60%///////////

Let's talk about the test: What improvements should we do? Then, first of all, we have to answer such a core three question: Why test? How to test it? What to test?

Why do you test first? Because our testing needs may be only a few words at first, and then we have developed a series of development, such as software development, hardware development, module development to vehicle integration, and so on.

At each stage, the designer has made a lot of product form changes to the previous stage. We should carefully test each change to see if it is consistent with expectations. This is why.

Virtual simulation test: cannot achieve road load response, decision -making and execution cannot be verified

How do you test? There are several more common methods. For example, in the research and development iteration, virtual simulation testing is essential. Simply speaking, it is a bit like a game engine. We use simplified vehicle models, road models, and sensor models in the middle to verify functional projects.

However, because there is no real road response and vehicle power response, many parts related to vehicle power performance, such as decision -making and execution, cannot test it.

Test site test: The site setting is time -consuming, high -cost, high cost, and simple scenes and functions

Of course, after the vehicle is manufactured, it is definitely a better way to test on the road. This is a test venue test. You can see the traffic logo, fake car cars, etc. in an empty place. We put real vehicles on the road for testing.

But such venues are very expensive. When various countries, including various provinces and cities in my country, when they are building autonomous driving test venues such as construction, their investment is hundreds of millions of dollars or even billions of dollars. The entire test process is also particularly time -consuming and labor -intensive, and the scene and function are relatively simple.

Public road test: high cost, unable to reproduce extreme scenes and build special test needs

If we test on the open road, of course, we can encounter more open and complex scenes. In this way, the diversity is enough, but the cost is very high. And we cannot reproduce extreme scenes and build special test needs. For example, we dare not test an emergency brake on the road. Once an accident occurs, the consequences are unimaginable. Another idea is that the vehicle is real and puts it in the laboratory; but the road is fake, the car is not moved, so it is safer.

In fact, the traditional vehicle will also do such table trials, but for the traditional vehicle, the intelligent part does not need to be measured because people are driving. You only need to measure the related mechanical properties, electronic performance, etc. of the car itself; do not need accurate road simulation, including steering simulation, inertia simulation, and so on; do not need to perform accurate calculations; there is no need for key scenarios -usually it is In a section of cyclic conditions, noise performance, fuel economic performance, etc. are found through statistical data.

In response to these issues, our team did the vehicle for autonomous driving in the ring test table.

First of all, we carried out accurate road steering resistance simulation and rotating inertia simulation. In this way, we can load various types of working conditions, the friction resistance, wind resistance, and slope resistance of roads can be loaded.

Secondly, although the vehicle is standing on the table, but because it has a real power input and the road load just mentioned, we can get a relatively accurate pose, and then after calibration, we can calculate the accurate attitude. This can be tested during the entire operation that the vehicle can be perfectly avoided, perfect overtaking, and so on.

The third is the key scene. Even if we measure the 10 billion kilometers on the long straight pavement, it does not make much sense, because we cannot expose those real problems. These real problems need to use relatively complex accident distribution, etc., and the parameters must be consistent.

In addition, we have to accelerate testing. Through importance sampling, proxy models and other methods, we can accelerate the scene about 200 times.

The next thing is to measure. We just mentioned why the question and how to measure, and the question of "testing" is also important.

The three ministries and commissions are promoted to the "Intelligent Connected Automobile Road Test and Demonstration Application Management Specification (Trial)"

Ministry of Industry and Information Technology Unicom [2021] No. 97

This is a China Intelligent Connected Automobile Test Project jointly released by the three ministries and commissions, with about 14 large categories and more than 30 small items.

I take a few examples, for example, in the recognition and response of the traffic identification line, we may have to measure the speed identification, lane line identification, stop line identification, and zebra crossing identification.

In the traffic light, we need to measure dozens of types, requiring vehicles to make accurate identification and response.

Then during the driving process, it is necessary to test the sudden static static in the middle of the vehicle operation, and the emergency braking of pedestrians when crossing the road. If this project can be tested well, the accident we start to mention may not happen. These functional tests are very useful.

In fact, the autonomous cars developed by our team have also undergone a dozen major items and more than thirty small items before obtaining the open road sign of Shenzhen.

From the "Birth Certificate" to the "ID card"

Completed: 80%///////////

But we still think that such testing is not enough. Because this is just a "birth certificate", we may still need it to gradually grow and give it a "ID card". What does this "ID card" represent? It means that it can deal with problems in the security boundary.

Test the risk relief ability of the security boundary

When all the functions are perfect, that is, when the nominal operation is running, it is certainly no problem to complete these functions. But what should I do if a single sensor fails under power resistance? At this time we should do the warning of the border and the relief of risks, so we need to test it. This is an example, a test of adaptive cruise on the table we developed.

ACC status enters follow → front car still → visual failure,

Target test fails → collision accident occurs

If the single sensor fails during the test process, and there is no reserve plan, a serious car accident may occur, which is the scene shown in the figure above.

The front car static → visual failure → the radar target detection is successful → trigger emergency braking

If our vision is invalid, the radar can still work, and the vehicle is safe to trigger emergency braking.

Except for binary test results, whether it is passed. All intermediate data and statistics are also very effective for our overall risk assessment.

We all know that slippery roads on rainy days have a great impact on the safe driving of the vehicle. Then we can first get the probability of a certain area of ​​a certain area through meteorological data, and then continue to obtain the probability of wetting on the road. Of course, there may also be sprinkler vehicles on sunny days, causing slippery roads.

In this case, we need to do some tests, such as different manufacturers have different control modules, and even the same manufacturer has different control algorithms in responding to different situations. After we test the car in detail, we can get risk transmission and overall risk calculation, as well as visualization and traceability of risk relationships.

This example is simple, but the functional safety and expected functional safety test of the actual vehicle are complicated. We can also derive the aging experiments, reliability experiments, and the effects of different temperatures and dimensions.

As mentioned earlier, products, regulations, testing, and the operation of the full life cycle. Finally, we also need to mention the risk transfer and pricing during the operation. In other words, under practical conditions, we cannot infinitely improve the intelligent technology of autonomous driving. Instead, we should comprehensively consider the factors of cars, people, and society to continuously expand our safety boundary. In the security boundary, let our autonomous driving run safely and the commercialization of safely.

thank you all!

- END -