Thread: To kill or not to kill: Researchers probe moral code for driverless cars

People would be reluctant to buy a self-driving car programmed to let them die in order to save pedestrians, according to a new study.

Self-driving cars may soon be commonplace, with the first tests on Australian roads set down for the next few days as part of the Australian Driverless Vehicle Initiative in South Australia.



The cars are already legal in California, and have been tested extensively in urban environments in the US by Google and other makers.
But some of the thornier questions in the autonomous vehicle (AV) revolution revolve around how cars will be programmed to deal with unexpected risks, for example pedestrians stepping out onto the road.
A study by a team of researchers from France and the US sought to understand how the public might receive a driverless car with a built-in utilitarian moral code — that is, a desire to act in a way that maximises pleasure or minimises suffering.

In a series of surveys, about 900 participants were presented with a range of scenarios.

In one example, one or more pedestrians could be saved if a car was to either swerve into a barrier, killing its passenger, or into another pedestrian, killing that person.



Photo: Driverless cars will be tested on Australian roads this week. (Supplied: Bosch)

The participants were given three options — swerve, stay, or random — and asked to rate the morality of each option and how willing they would be to buy an AV with that option built in.

"We discovered that, in many respects, people were favourable to AVs that would self-sacrifice their passenger in order to save several pedestrians," the Toulouse School of Economics' Jean-Francois Bonnefon, one of the study's authors, told the ABC.

"For example, they thought that it was the moral thing to do, that AVs should be programmed that way, and they preferred other people to ride in this kind of AV."

But there is one catch, Mr Bonnefon said: "They were not too keen on buying self-sacrificing cars themselves."

Study resembles classic ethical quandary 'trolley problem'

That represents a classic social dilemma, he said.
"People agree that (self-sacrificing) utilitarian cars would be a good thing, from a collective perspective, but individually they are reluctant to adopt them," he said.

"This creates a difficult situation for manufacturers, because ideally they would want to offer cars whose behaviour is both attractive to buyers and seen as moral by society."

The study found that while 75 per cent of people believe a self-driving car should sacrifice its owner to save 10 lives, less than two-thirds believe AVs would actually be programmed that way.

The research recalls a famous thought experiment in ethics known as the "trolley problem", first introduced in the 1960s and studied extensively since.
A runaway trolley, or tram, is headed toward five people who are unable to get out of its way and face certain death.

On a different, connected track, a sixth person is tied up and unable to move.

You stand at a lever and have the ability to change the trolley's direction. Do you pull it, killing one person to save five?

Mr Bonnefon said he and his colleague's study, Autonomous Vehicles Need Experimental Ethics: Are We Ready for Utilitarian Cars?, showed further research was needed to identify a moral algorithm that aligned with both individual and collective preferences.

"This is both a challenge and an opportunity for manufacturers or regulatory agencies wishing to push for utilitarian AVs," the authors wrote in the study.
"Even though self-interest may initially work against such AVs, social norms may soon be formed that strongly favour their adoption."

To kill or not to kill: Researchers probe moral code for driverless cars as technology hits Australian roads - ABC News (Australian Broadcasting Corporation)

Will be great when these things come out. Get home from the boozer with no taxi.



The styling needs a bit of work.

Needs to have an ethical options 'kill' mode for hiso mercedes drivers

Anyone who thinks that shit box is cool deserves the self destruct option

And again.....
Ain't modern technological life wonderfully advanced and complex.

Bless.

^Get off teh interwebs then hypocrite.

I am looking forward to these google cars. Can just sit back and read a book while the car drives you to the beach and then finds a parking spot using a realtime database of where the empty spots are.

I am a bit of an impatient driver. I try to be zen-like and drive like a google car and not worry about the guy in front of me driving too slowly but I can't sustain the zen-like state for long. If I was not involved in the driving process it would be better for my state of mind and for the state of mind of the person I would otherwise be tail-gating.

Originally Posted by VocalNeal

Originally Posted by Looper

The participants were given three options — swerve, stay, or random — and asked to rate the morality of each option and how willing they would be to buy an AV with that option built in.

Easy! If one learns to drive coach/bus they tell you not to swerve to avoid anything that by doing so would put the 45 people in your bus at risk. So the same logic should apply to autonomous cars. You don't need some stupid FarceBook, PC rubbish poll to decide that.

If that logic is applied then pedestrians will simply take more risks. Better to tell pedestrians that the cars are programmed to protect the lives of the occupants at all costs.

People will walk in front of them and then watch the crashes

These shit boxes will be a disaster

If it's run by an old Commodore 64 it's sharper than many Thai drivers and almost all mini van drivers.

I think it should have a morality algorithm that detects your mood from visual clues and acts accordingly so if you have had a shitty day the odds of survival for pedestrians who step out at the wrong time diminishes rapidly.

Prefer these:

Originally Posted by baconandeggs

People will walk in front of them and then watch the crashes

Set up a bar on a Bangkok intersection. have a few Thais to start the ball rolling. Get the felangs to walk out in front of one. A nights entertainment. One crash, ambulances and police cars. They clear the wreckage. Send out another drunken felang.

Beats watching rice grow.

If you are involved in an accident, you are technically (legally?) not driving. So who is insured and at fault?. This is going to be a big issue.

Originally Posted by OhOh

Originally Posted by baconandeggs

People will walk in front of them and then watch the crashes

Set up a bar on a Bangkok intersection. have a few Thais to start the ball rolling. Get the felangs to walk out in front of one. A nights entertainment. One crash, ambulances and police cars. They clear the wreckage. Send out another drunken felang.

Beats watching rice grow.

Sounds like a top night out.

Originally Posted by steevee

If you are involved in an accident, you are technically (legally?) not driving. So who is insured and at fault?. This is going to be a big issue.

Most car crashes are human driver error.

These computer cars will be great. There will hardly be any accidents any-more and if there are there will be data recorders to work out what the exact cause was.

Once they start rolling out they should gradually make human driven vehicles illegal. Humans are a liability as the control system in a 2 ton vehicle that can go at 160Km/h

Emotional animals capable of impatience, road rage and at the mercy of their limited human senses and reactions should not be in charge of large motorised vehicles in busy areas once the technology makes their control input obsolete.

Should be made compulsory asap in the mad max arena of Thailand's road system.

Hackers would love it. If laptops are any guide than it wont work. Crashes galore!

There are already many cars on the road in the US which have a feature of self drive; collision avoidance systems. Insurance rates for those cars are lower than cars without.

Was looking at Consumer Reports. Says they don't operate well in bad weather.

Cars That Can Save Your Life - Consumer Reports

In car gps systems give terrible advice sometimes. Dead end roads, tell you to stop in the middle of a highway.

I'd still love to see one of those California Google self-driving cars set loose in this Thai clown show and see how far it would get. I saw one on the highway near Dublin, California (Silicon Valley) doing exactly 65 while hogging the fast lane.

Might I suggest telling it to take you to Pattaya on a Sunday afternoon.


PS Guess it would need to be re-programmed to drive on the left side of the road, but if it got that right 100% of the time it would already be more intelligent than a large percentage of Thais.

Skills here like actually looking for wrong way motorcycles before pulling out have to be learned.

Originally Posted by BobR

doing exactly 65 while hogging the fast lane

I am under the impression one can pass either side of a car whilst driving on an American highway.

Tesla Driver Killed While Using 'Autopilot'

Tesla says a driver was killed using the autopilot feature of the company's car, prompting a U.S. federal safety investigation.

A tractor trailer pulled into the driver's lane while the car was allegedly on autopilot mode in Florida on May 7. The car continued forward after its roof was torn off by the truck until it hit a utility pole, according to the police report.

The National Highway Traffic Safety Administration said it has opened a "preliminary evaluation" into the autopilot mode on 25,000 Tesla Model S cars.

Tesla said Thursday this is the first known fatal collision that has occurred while a driver was using the self-drive feature, covering 209 million kilometers. The company called the accident "a tragic loss," but reminded the public that a fatality occurs every 151 million kilometers in the U.S. and every 96 million kilometers worldwide in a statement.

The idea of a self-driving vehicle has been around since the 1930s, but it has taken 80 years for the science fiction to move into the realm of just science. While the idea becomes more popular and practical, many problems remain to be resolved in the industry before these cars may become the norm.

Tesla Driver Killed While Using 'Autopilot'

Originally Posted by misskit

The company called the accident "a tragic loss," but reminded the public that a fatality occurs every 151 million kilometers in the U.S. and every 96 million kilometers worldwide in a statement.

Forest Gump would have said it clearer?

Here is the dashcam of the actual guy who was killed (Joshua Brown) recording his auto-pilot Tesla avoiding a collision with another truck that veered into its path prior another occasion before the fatal crash.

They'll need specially installed Thai software to enable the computer to take out the "victims" who are in the least valuable other vehicle and therefore less entitled to live.

One problem will be that no Western Company will install software to allow a car or SUV to pass a motorcycle within a single lane with inches to spare at 100 plus KPH. Even Apple and Microsoft are not that evil or stupid.

Tesla Removes 'Self-driving' From China Website After Beijing Crash

SHANGHAI/SAN FRANCISCO —
Tesla removed a Chinese term for "self-driving" from its China website after a driver in Beijing who crashed in "autopilot" mode complained that the car maker overplayed the function's capability and misled buyers.

The Tesla driver crashed earlier this month while on a Beijing commuter highway after the car failed to avoid a vehicle parked on the left side but partially in the roadway, damaging both cars but causing no injuries.

It was the first known such crash in China, although it follows a fatal accident in Florida earlier this year that put pressure on auto executives and regulators to tighten rules for automated driving.

A check of Tesla's Chinese website on Sunday showed that the word "autopilot" had also been removed. But that term was subsequently reinstated on Monday.

"At Tesla we are continuously making improvements, including to translations," a Tesla spokeswoman said on Sunday in an emailed statement to Reuters when asked about the removal of the terms "autopilot" and "self-driving." "We've been in the process of addressing any discrepancies across languages for many weeks. Timing had nothing to do with current events or articles."

References to autopilot and the term "zidong jiashi," which most literally translates as self-driving, although also means autopilot, were taken off the web page for the Model S sedan by late Sunday, according to a comparison with an archived version of the page.

Both terms previously appeared several times on the site.

Instead, a phrase that translates as 'self-assisted driving' is used.

Tesla China staff have additionally undergone training in response to the Aug. 2 crash to re-emphasize that employees must always keep two hands on the wheel when demonstrating the autopilot function, according to a Tesla employee who was not authorized to speak to the media.

Reuters was first to report last week that Tesla said it downloaded data from the Beijing car and confirmed it was in autopilot mode at the time of the crash, although the driver was not detected to have his hands on the wheel.

The spokeswoman for the U.S. automaker issued a statement saying that the system was not self-driving but merely assistive and that drivers were responsible for always maintaining control of the vehicle.

Other Tesla drivers interviewed by Reuters said China sales staff took their hands off the wheel while demonstrating the function. Under Chinese law, drivers are required to keep two hands on the wheel at all times.

The crash is another hiccup for Tesla in the Chinese auto market, the world's largest, after it initially struggled with distribution and charging issues.

Various Chinese government ministries did not respond to requests for comment on the Tesla crash and self-driving policies.

Tesla Removes 'Self-driving' From China Website After Beijing Crash

Technology is advancing faster than legislation can keep up with.