How the cultural bias of driverless cars determines who lives and who dies – Hack

What should a driverless car do in a situation where it has to choose between saving its own pbadenger or a pedestrian?

What if the occupants are children? What if they’re very wealthy?

Two years ago, Mercedes created outrage when it announced that its self-driving cars would simply mow down the pedestrians every time. The cars would be programmed to save the people they were transporting, regardless of how many people they killed.

The car maker backed down, and a couple years later Germany drew up quite different legal guidelines. It said the machine must harm the fewest possible people and treat all life equally.

You decide: Driverless cars

‘You’re approaching a pedestrian crossing. The autonomous system is working but the brakes fail. A crash is unavoidable.’

Simple, right? This week, researchers from the Mbadachusetts Institute of Technology published the results of an experiment that shows the situation is a whole lot more complicated, and there is no universal consensus of the most ethical way for cars to drive.

The study suggests that ethical problems cannot be solved with clever programming, but are highly subjective and informed by cultural upbringing.

Called ‘The Moral Machine Experiment’, the study is based on 40 million decisions collected from millions of participants in more than 200 countries.

To achieve this level of participation, the researchers created a kind-of-fun online survey where you have to choose between killing pedestrians or saving pbadengers. Each scenario introduced new variables. What if the people in the car were criminals? What if the pbadengers were cats?

“Never in the history of humanity have we allowed a machine to autonomously decide who should live and who should die, in a fraction of a second, without real-time supervision,” the study says.

“We are going to cross that bridge any time now, and it will not happen in a distant theatre of military operations; it will happen in that most mundane aspect of our lives, everyday transportation.

Examples of Moral Machine scenario questions.

The results of the global study

Researchers found an average global preference for sparing humans over animals, more people over less, and the younger over the older.

This may not be surprising – where it gets interesting is the variation across cultures.

The three cultural categories and the variation in responses.

This image above takes some explaining.

It divides the more than 200 countries that had survey respondents into three categories: ‘Western’ contains North America as well as many European countries of Protestant, Catholic, and Orthodox Christian cultural groups. ‘Eastern’ consists mostly of countries of Islamic and Confucian cultures, and includes Japan, Taiwan, indonesia, Pakistan and Saudi Arabia. ‘Southern’ consists of the Latin American countries of Central and South America, as well as France and its current and former overseas territories.

The study found shared preferences for machine ethics within cluster groups, but important differences between these three clusters.

“For example, the preference to spare younger characters rather than older characters is much less pronounced for countries in the Eastern cluster, and much higher for countries in the Southern cluster,” the study says.

“The same is true for the preference for sparing higher status characters.

“Similarly, countries in the Southern cluster exhibit a much weaker preference for sparing humans over pets, compared to the other two clusters.

“Only the (weak) preference for sparing pedestrians over pbadengers and the (moderate) preference for sparing the lawful over the unlawful appear to be shared to the same extent in all clusters.”

Preference for saving characters, from babies to cats (the respondents ranked saving cats as the lowest priority)

The authors of the study proposed the difference could be explained by whether the culture was “individualistic or collectivistic” – whether it emphasised the distinctive value of each individual, or rather group goals above individual needs and desires.

“Participants from individualistic cultures … show a stronger preference for sparing the greater number of characters,” the study says.

“Furthermore, participants from collectivistic cultures … show a weaker preference for sparing younger characters.

“Because the preference for sparing the many and the preference for sparing the young are arguably the most important for policymakers to consider, this split between individualistic and collectivistic cultures may prove an important obstacle for universal machine ethics.”

Other predictors of ethics preferences

Aside from individualistic versus collectivist, the study identifies other cultural and economic predictors of machine ethics preferences.

Prosperity (indexed by GDP per capita) and the quality of rules and institutions correlate with a greater preference for saving pedestrians who are crossing the street legally.

“In other words, participants from countries that are poorer and suffer from weaker institutions are more tolerant of pedestrians who cross illegally, presumably because of their experience of lower rule compliance and weaker punishment of rule deviation,” the study says.

Higher country-level economic inequality corresponds to how unequally characters of different social status are treated.

“Those from countries with less economic equality between the rich and poor also treat the rich and poor less equally in the Moral Machine,” the study says.

Variation between three cultural categories. The further the peak is to the right, the greater the ethical preference.

This also extends to gender. Countries with a lower gender gap in health and survival showed a stronger preference for sparing women over men. Southern countries (Central and South America, as well as France and its former and current territories) exhibited a strong preference for sparing women (and athletic individuals, funnily enough).

“All the patterns of similarities and differences … suggest that manufacturers and policymakers should be, if not responsive, at least cognizant of moral preferences in the countries in which they design artificial intelligence systems and policies,” the study says.

“Whereas the ethical preferences of the public should not necessarily be the primary arbiter of ethical policy, the people’s willingness to buy autonomous vehicles and tolerate them on the roads will depend on the palatability of the ethical rules that are adopted.”

Different moral algorithms for different countries?

The point of studies like this are not necessarily to figure out how to program driverless cars or provide data on what is ethical in certain countries.

In any case, as Dr Toby Walsh, a professor of AI from UNSW told Hack, people may make very different choices when real lives are at stake, and its not just a computer scenario.

“The study does help us to understand what people do in the calm of their own home,” he said.

“The values we give machines should not be some blurred average of a particular country or countries.

The importance of cultural variation in ethics raises the prospect that driverless cars will have to be programmed to reflect the ethical priorities of different countries. They would then switch between modes as they cross international borders.

“We already have different road rules for different countries, should they have different ethical settings for different countries?,” Professor Walsh said.

The MIT researchers come to a different conclusion, however. They say before we allow our cars to make ethical decisions, we need to figure out our ethical preferences, and then express them to moral algorithm designers and government policymakers.

“The fact broad regions of the world displayed relative agreement suggests that our journey to consensual machine ethics is not doomed from the start,” the study says.