ProPilot Autopilot and Level 2 Self-Driving Cars Didn’t Pass All AAA Road Tests

Level 2 technology automated driving was tested by AAA which showed that the driver still has to be engaged. Although the tech is meant to assist in the driving task, should never be used as a replacement for driver engagement. Drivers are confused about self-driving safety systems.

AAA also tested these systems and found that they are in fact not designed to take over the task of driving and can be significantly challenged by every day, real-world conditions such as poor lane markings, unusual traffic patterns and stationary vehicle

1. In general, tested vehicles performed according to expectations during closed-course testing.
a. With no lead vehicle to follow, the test vehicles traversed the testing lane with minimal lateral
offset relative to the lane center.
b. Three out of the four test vehicles were influenced by the presence of a simulated
distracted/impaired lead vehicle.
c. All tested vehicles recognized a slower tow truck and reduced speed accordingly.
d. Three out of four test vehicles required driver intervention to avoid colliding with a simulated
stationary target.
2. During naturalistic driving, lane-keeping events accounted for 88 percent of events requiring driver
a. Evaluated systems generally performed best on open freeways and freeways with stop and go
b. Systems were challenged on freeways with moderate traffic and by urban driving along surface

A new survey from AAA finds that 40 percent of Americans expect partially automated driving systems, with names like Autopilot, ProPILOT or Pilot Assist, to have the ability to drive the car by itself, indicating a gap in consumer understanding of these technologies and reality.s

“With today’s exciting advances in vehicle technology, there is a greater need for naming that clearly signals to a driver what the system does,” said Greg Brannon, AAA’s director of Automotive Engineering and Industry Relations. “Vague or confusing terminology may lead someone to overestimate a system’s capability, unintentionally placing the driver and others on the road at risk.”

In partnership with the Automobile Club of Southern California’s Automotive Research Center, AAA tested four vehicles equipped with systems that combine technologies such as adaptive cruise control and lane keeping assist to help maintain lane position, forward speed and following distance in relation to a lead vehicle. Closed-course and on-road testing evaluated performance in typical driving situations where the technology generally behaved as expected. However, there were a number of instances in both environments that caused these systems to act in an unpredictable manner, requiring driver intervention to avoid a potential collision.

While driving on public roadways, AAA found test vehicles struggled when encountering scenarios that included moderate traffic, curved roadways and streets with busy intersections. Researchers noted many instances where the test vehicle experienced issues like lane departures, hugging lane markers, “ping-ponging” within the lane, inadequate braking, unexpected speed changes and inappropriate following distances. AAA’s study also revealed that nearly 90 percent of events requiring driver intervention were due to the test vehicle’s inability to maintain lane position. The irregular and complex nature of the real-world driving environment revealed the vulnerabilities of this technology. AAA’s testing found the systems generally performed best on open freeways and freeways with stop and go traffic.

During closed-course testing, common driving situations were simulated such as staying within the lane at 45 mph, following a distracted or impaired driver, encountering a commercial vehicle, for example, a tow truck, or contending with a vehicle that suddenly changed lanes to reveal a stationary vehicle. All test vehicles were able to successfully maintain lane position as well as recognize and react to the presence of the tow truck with little to no difficulty. However, in the scenario where a lead vehicle changed lanes to reveal a stationary one, three out of the four test vehicles required driver intervention to avoid an imminent crash. In general, this scenario is a stated limitation of these systems, however, it is a relatively common occurrence on roadways and could take those drivers by surprise who have become too reliant on the technology.

“Both real-world and closed-course testing exposed separate yet equally serious limitations with these systems,” said Brannon. “It reinforces that there is still much work to be done to educate consumers on the nuances between system names and functionality and that it is much too early to refer to these vehicle technologies as automated.”

In order to reduce the misuse of partially automated vehicle systems, AAA encourages drivers to educate themselves by requesting a demonstration at the dealership as well as thoroughly reading the vehicle owner’s manual. As this technology becomes more prevalent, standardized naming across vehicles that clearly reflects how technology functions will be necessary. Greater consistency across the industry will help consumers understand the type of technology their vehicle has along with how, when and where to use these systems.

To assess the capabilities of partially automated vehicle systems, AAA conducted primary research in partnership with the Automotive Club of Southern California’s Automotive Research Center in Los Angeles, California. Track testing was conducted on closed surface streets on the grounds of Auto Club Speedway in Fontana, California and was rented by AAA for independent testing. Public highway evaluation was conducted on surface streets, highways and limited-access freeways throughout the greater Los Angeles area.

Four test vehicles were selected (2018 Mercedes-Benz S-Class, 2018 Nissan Rogue, 2017 Tesla Model S and 2019 Volvo XC40) using specific criteria and each test vehicle was outfitted using industry‐standard instrumentation, sensors and cameras to capture vehicle dynamics, position data and braking intervention.

When following a lead vehicle that changes lanes to reveal a stationary vehicle, three out of the four test
vehicles required intervention because the test driver believed that the vehicle was about to impact the
stationary target at a speed sufficient enough to cause significant vehicle damage. One out of the three vehicles mitigated a potential collision by 10.12 mph immediately preceding driver intervention. The remaining two vehicles did not significantly reduce vehicle speed immediately preceding driver intervention. These findings are consistent with manufacturer stated design limitations. This test especially illustrates the fact that current level 2 AV systems require the driver to maintain situational awareness at all times and to be prepared to intervene at any time.

However, test drivers reported some unnatural steering behavior from all test vehicles during public roadway evaluations. In extenuating examples, test drivers reported fighting the system’s steering input to affect desired vehicle control. Events requiring driver intervention occurred with higher frequency on urban roadways and freeways containing moderate traffic. Most systems were incapable of staying in their lane on curved roadways at highway speed, such as freeway transitions.

Summary Recommendations
1. Currently available level 2 AV systems are not capable of fully autonomous vehicle operation; it is
imperative that the driver maintain situational awareness and vehicle control at all times.
2. Consider the roadways in which it is appropriate to utilize level 2 AV systems. It is recommended that
drivers avoid engaging these systems on urban surface streets with many intersections and on roadways
with significant curving geometry.
3. AAA researchers observed that combining lateral and longitudinal vehicle control did not noticeably
enhance the driving experience for tested level 2 AV systems. This finding was due to the high number
of significant events associated with lateral vehicle control, in addition to the fact that drivers must keep
his/her hands on the steering wheel at all times.
4. Improved tuning of lateral vehicle control within level 2 AV systems may bring a significant improvement
to partial autonomous driving systems.

The consumer survey was conducted October 4–7, 2018, using two probability samples: randomly selected landline telephone and mobile (cell) phone numbers. The combined sample consisted of 1,003 adults (18 years old and older) living in the continental U.S. The margin of error for the study is 4% at the 95% confidence level. Smaller subgroups will have larger error margins.