Modern fears: Can intuitive artificial intelligence be ethical?

Steven Hawking has recently warned that “the development of full artificial intelligence could spell the end of the human race”, because “it would take off on its own, and re-design itself at an ever increasing rate”.    It is interesting to compare this very 21^st century fear with earlier ones.

Early Artificial Intelligence (AI) was based on a rather Victorian train of thought:  intelligence is what distinguishes humans from other animals; humans are different mainly because of their ability to use language and think logically; AI should be concerned primarily with language and logic.  Literary paranoia, in the tradition of Frankenstein, therefore gave us artificial intelligences which were dangerous because they were ultra-rational.  We feared AI would be too logical, not “human” enough.

In I Robot, Asimov invented the now-famous “Three laws of robotics”:

One, a robot may not injure a human being, or, through inaction, allow a human being to come to harm.  Two, a robot must obey the orders given it by human beings except where such orders would conflict with the First Law.  And three, a robot must protect its own existence as long as such protection does not conflict with the First or Second Laws.

He then used a beautifully constructed series of short stories to show how in certain scenarios interpreting these laws literally could lead to all sorts of problems.

In Arthur C Clark’s 2001: A Space Odyssey, HAL has a sort of mental breakdown caused by the logical inconsistency of having to lie, and kills crew members because (as HAL sees it) they are jeopardizing the mission.  This was magnificently parodied in John Carpenter's movie Dark Star where Lt. Doolittle teaches Bomb 20 existentialist philosophy in an unsuccessful attempt to stop it exploding.

But the pursuit of robotics has shown that many of the really hard problems of AI are more to do with physical embodiment:  combining multiple streams of data into a constantly updated world model, planning with uncertain information, and acting / responding in real-time.  These abilities, of course, are shown by all mammals, not just humans.  Psychological and neurophysiological research has shown that such systems do not use symbolic logic.  They use massively parallel computation based on extensively connected networks of neurons.

So intelligence involves two fundamentally different types of inference.  Daniel Kahneman characterises them as Thinking, Fast and Slow:  and shows that people tend to use logic (slow thinking) only when intuition (fast thinking) is not immediately available.

Deep learning, text mining, big data analytics:  all these recent advances in applied AI are based on building models using massive data sets.  Look at AI in everyday life now:  Amazon accurately predicts what books I will enjoy, Twitter presents me with news items I am likely to find interesting, banks automatically refuse me credit cards because they recognize me as a bad risk.  These systems are not logical, they are deeply intuitive.

In my line of work – automating organisational decision-making – analytic models are often central to the solution, with rules being used mainly to constrain the bounds of the decision-making to comply with product design, company policy and legislation.  The decisions made using such systems cannot be explained with a logical argument (even to the rejected customer);  they are essentially automated intuitions.

And so new fears are stirring, as voiced by Steven Hawking.  IBM and Google have formed a partnership aiming to create an artificial intelligence with access to all the information (words and pictures) on the internet.  “IBM” is (coincidentally, Kubrick assures us) only a one-letter step forward from “HAL”.  What might such an intelligence do?  In a thought experiment, Steve Omohundro imagines a rational chess robot given the “innocuous” goal of winning lots of chess games against good opponents:

Its one and only criteria [sic] for taking an action is whether it increases its chances of winning chess games in the future. If the robot were unplugged, it would play no more chess. This goes directly against its goals, so it will generate subgoals to keep itself from being unplugged. You did not explicitly build any kind of self-protection into it, but it nevertheless blocks your attempts to unplug it. And if you persist in trying to stop it, it will eventually develop the subgoal of trying to stop you permanently.

The modern-day equivalent of Asimov’s Three Laws is found in the theory of rational agents, used in economics and AI.  A rational agent is any decision-making entity which maximises the expected utility of its actions.  Note the subtle shift in the meaning of “rational”, from “logical” to “pragmatic”.  We can make such agents safe (suggests Omohundro) by (a) limiting their possible actions so that they cannot do bad things, and (b) by giving them utility functions aligned with human values, so that they want to do good things.

My concern with this analysis is that it still assumes that the agent is a single entity with a single utility function.  When a decision-making system comprises many model-based sub-systems it acts more as a confederacy.  A “person” is actually a loose collection of decision-making systems, and may only become conscious of a decision after s/he has made it.  We should expect the same to be true of any sufficiently complex artificial intelligence.  To produce artificial intelligences which are ethical agents, we therefore need to build ethics into the very architecture of the agent, rather than expecting to apply it just as an overlying “conscience”.  This has been Sam Franklin’s approach with LIDA: Learning Intelligent Distribution Agent (as described in Wallach and Allen’s excellent book Moral Machines: Teaching Robots Right from Wrong).

Data scientists working in commercial analytics are careful to constrain the factors used to build predictive models in line with legislation.  For example, in many countries it is illegal to use ethnic background or gender (or related characteristics) for decision-making.  If an artificial intelligence builds its own models with whatever data are objectively most predictive, what is to prevent it becoming racist or sexist?  And how would you know if this had happened?  Somehow we need to ensure that the entity develops within a framework of ethical principles and learns to monitor its own decision-making.  This strikes me as hard, but not impossible.

Most current research into the AI of ethics still addresses the HAL / Frankenstein fear:  that an artificial intelligence may become evil and consciously do bad things which are hard to prevent.  Although this work is necessary, it is not enough.  My concern is about something more insidious:  that a benign artificial intelligence could unconsciously do wrong, and this will be hard to detect.  The solution will not lie in rules which directly constrain an agent’s behaviour, but in a system of ethics providing a framework for action, development and learning.