Tests of vehicles with autonomous technology show flaws that could lead to crashes without attentive drivers, and imply that Waymo's plan to put fully driverless vehicles on the road in metro Phoenix may be premature.
Recent real-world road tests by the Insurance Institute for Highway Safety show that the semi-autonomous technology built into the newest high-tech cars now available for purchase is not reliable.
Although the institute didn't test vehicles by Waymo, Uber, or other self-driving car companies, the findings released on Tuesday reveal that fully autonomous vehicles are not yet ready for public roads.
Waymo, owned by Google parent Alphabet, has been testing its self-driving cars in the Phoenix area for more than a year. The company claims it has vehicles with no backup drivers regularly driving the streets, and that it will launch a fully driverless ride hailing service in metro Phoenix no later than the end of this December.
"Driverless cars aren't here yet," the institute report says. "More vehicles are incorporating a degree of automation with such technologies as adaptive cruise control and lane-keeping support, but the driver will continue to share driving responsibilities for the foreseeable future."
If Waymo puts cars without backup drivers on public streets, it would be only in an extremely limited way, institute spokesman Russ Rader said.
"Turning a fully autonomous vehicle loose into the wild is not something we see happening anytime soon," Rader continued. "We have a lot of concern about the testing of these vehicles on public roads."
The five-part study released this week, "Evaluating Autonomy," included a chapter on the March 18 Uber self-driving vehicle collision that killed a pedestrian, predicting that the lack of regulation will lead to more problems.
Uber relocated to Arizona in late 2016 after Arizona Governor Doug Ducey invited the company to enjoy the state's lack of regulation.
The Insurance Institute for Highway Safety doesn't appreciate such laissez-faire policies.
"Absent regulatory oversight, the race to deploy autonomous vehicles risks jeopardizing public trust and safety and the lifesaving promise of the technology," the report says. "The early results underscore the fact that today's systems aren't robust substitutes for human drivers."
The institute tested several vehicles with various autonomous features, including a Mercedes, BMW, two Teslas, and a Volvo. Problems ranged from "irksome," like too much braking for shadows, to failures deemed "dangerous."
Positive results were seen in systems like adaptive cruise control, which allows cars to be programmed to follow other cars at a certain distance, and brake to maintain that distance. Most of the time, the systems worked well, and were likely to reduce accidents over the long term, the study found. On the other hand, the systems occasionally don't work as planned, showing the absolute need for an attentive driver who isn't relying fully on such systems.
On a closed track, all the vehicles detected a vehicle stopped in their lanes and stopped accordingly. Out on the street, all the cars except the Tesla Model 3 had instances in which they failed to detect stopped vehicles and would have hit them without intervention.
"ACC systems require drivers to pay attention to what the vehicle is doing at all times and be ready to brake manually," said Jessica Jermakian, the institute's senior research engineer, in the study.
All of the vehicles tested had lane-keeping systems, which are supposed to center a vehicle within marked lanes if a vehicle begins to stray from a lane. But only the Tesla Model 3 stayed within its lanes throughout all 18 of the trials.
Once the cars' abilities were tested on roads in "hilly Central Virginia," however, none of them worked flawlessly. The Tesla Model 3 again did the best, staying in the lane in all but one trial, "when it hugged the line."
The other cars, though, all had trouble staying in their lanes through all 18 trials, with the BMW and Tesla Model S doing more poorly than others.
While it's impressive that a Mercedes E-Class with Drive Pilot technology stayed in its lane for 15 of 18 trials, straying from a lane even once could lead to a serious crash.
Rader said the institute doesn't have enough information to contradict Waymo's assertions that it will put a fleet of fully driverless cars on the streets of Phoenix within a few months. But such a program "would need to be very limited at the moment."
One of the most significant concerns, Rader said, is the lack of safety information coming from companies like Waymo.
"There is no structure in place ... to independently assess the safety of these vehicles in real-world driving because we don't have what we need," he said.
The public should have access to a database containing information about the vehicles being tested on public streets, Rader said. That way, researchers "can look at the crash experience of these vehicles on the road."
Phoenix New Times has tried repeatedly over the past year to get Waymo to break its silence on safety information and show the public how its vehicles have been performing on metro Phoenix roads. The company refuses to do so.
Waymo's spokespeople maintain that the company is transparent, but, in fact it's hiding many details about its operations, including those that could help the public determine if Waymo vehicles are truly ready to be go without backup drivers.
As New Times has reported, Waymo refuses to divulge how often the vehicles it's testing in metro Phoenix have required human intervention, and why.
It's possible that Waymo's fully driverless system will, indeed, be very limited — perhaps far more so than Waymo has led the public and media to believe.
Numerous headlines mentioning Waymo and touting "robo-taxis" coming to Phoenix have added to the hype over the company and helped boost Alphabet's stock price.
"You can catch a self-driving taxi in 2018, if you're traveling to Phoenix," says a May 9 headline in TechRepublic.com that's typical of the optimistic coverage. "The Alphabet subsidiary will begin operating a self-driving car service akin to Uber or Lyft by the end of the year, with no driver present."
Yet as of early August, it was still incredibly rare to see a Waymo vehicle driving on a public street "with no driver present."
The company won't give details about the routes used by its fully driverless vehicles, so that they can be observed along with the vehicles with backup drivers.
No members of the general public or news media have been allowed to ride in a fully driverless vehicle on public streets yet. Although the company has about 400 Arizonans signed up for its Early Rider program, participants must sign nondisclosure agreements that limits how they can talk about their observations.
Waymo vehicles, usually Chrysler Pacificas with their telltale black radar bulb on the roof, are regularly rolling on east Valley streets, mostly in Chandler and Tempe. The company recently announced it has put 8 million miles on its self-driving vehicles, and many of those have been in Arizona. It also tests vehicles in California, Georgia, Michigan, Texas, and Washington.
Yet how many of those miles are truly self-driving? Only the company knows for sure. Waymo's backup drivers can often be seen with their hands on the steering wheel, and actively turning the wheel.
Waymo won't say how many of the 8 million miles driven by its vehicles utilized backup drivers. Samantha Jackson, a Chandler resident whose family has been part of Waymo's Early Rider program, and who has been allowed by Waymo to talk to the news media, told New Times at a Waymo event last week that the cars she's been in have always had a backup driver, but that her husband once took a ride in a car with no backup driver.
None of this means Waymo won't start allowing the public to buy rides in its fully driverless transportation service vehicles by the end of the year, like the company says it will.
But the IIHS study implies that if Waymo does turn these vehicles loose on the streets of Phoenix, residents can expect more crashes like the March 18 one involving Uber.
Maybe the potential for fatalities is seen as part of the cost of doing business. Experts estimate that self-driving cars someday could be an industry worth more than $1 trillion. An analyst with Morgan Stanley this week predicted Waymo is on track to become a $175 billion company over a decade.
It's also possible that Waymo's ride-hailing service won't be fully driverless for some time yet.
Shaun Stewart, Waymo's chief business development officer, said last week that while the end goal of Waymo is to go fully autonomous, "until we feel confident, we'll always keep the driver in the vehicle."
In spite of the company's own lack of confidence in deploying fully driverless vehicles just yet, the vehicles have many supporters, including people eager to see autonomous vehicles save lives by replacing error-prone human drivers. As the Insurance Institute for Highway Safety report showed, autonomous technology — at least when paired with a human driver — does have the potential to reduce crashes.
Mesa Mayor John Giles, who says he recently saw a Waymo car with no backup driver cruising the streets of Mesa, said he would have no problem riding in a similarly driverless vehicle. Yet at the same time, he agrees the technology won't be "foolproof."
"We have to realize that this is the future and we have to be committed to the future," he said. "I have confidence in Waymo."
Philip Yavar, general manager of Element Chandler Fashion Center Hotel, said he would "absolutely" jump at the chance to ride in a fully driverless Waymo vehicle. Waymo recently partnered with the hotel to provide rides for select hotel guests. Yavar wasn't sure if the rides would utilize backup drivers, but added that he felt comfortable for the safety of hotel guests even if the vehicles were driverless.
"This is where the future is," he said. "How wonderful it will be to reduce automobile deaths."