AS ARTIFICIAL INTELLIGENCE (AI) worms its way into many areas of life, society will need to become comfortable with algorithms, not people, making decisions. The systems have already shown promise in areas ranging from banking to e-commerce, healthcare and policing. Yet worries grow that the algorithms may take on too much control—especially if people forfeit decision-making to machines, such as with self-driving cars or courtroom sentencing. If this prevents AI’s use, then there is a risk that society and the economy will fail to receive its potential benefits.
Hannah Fry has studied these systems for years as a mathematician focusing on urban issues at the Centre for Advanced Spatial Analysis at University College London. However she is better known as a great populariser of maths and sciences through her public lectures and documentaries on the BBC.
In her latest book, “Hello World,” Ms Fry demystifies how the technologies work, looks back into history to explain how we came to adopt data-driven decisions and offers a clear-eyed analysis of the pros and cons. The benefit is that AI can often perform tasks more quickly and accurately than people; the drawback is that if the data are biased, then the output may be discriminatory.
The Economist’s Open Future initiative asked Ms Fry how society should harness the technology. The interview is followed by an excerpt from the book on the criminal-justice system and an algorithmic approach called “random forest”.
* * *
The Economist: All data have biases; should we delay the introduction of algorithmic systems until we are confident that we have uncovered and remedied the key ones, or should we accept a lower standard: make a “best effort” to identify and fix biases, but release the code and remedy on the fly, as flaws get uncovered?
Hannah Fry: There’s an easy way to fall into a trap here. Once you see the problems that algorithms can introduce, people can be quick to want to throw them away altogether and think the situation would be resolved by sticking to human decisions until the algorithms are better. But in truth, human systems are littered with biases and riddled with their own kinds of problems.
It depends on the setting as to how carefully you need to tread. (You can’t responsibly introduce a system with teething problems in healthcare, for instance, in the same way you can with, say, “video assistant referees” in football. Generally, the overall objective has to be to build the fairest, most consistent system possible. That means recognising that perfection is impossible and that trade-offs are inevitable. But it also means, in the meantime, that we should focus on making it easier to appeal the decisions of algorithms when they inevitably do go wrong.
The Economist: The criminal-justice system sometimes vaunts the value that “ better a criminal go free than an innocent person go to jail ” . Should we refuse to adopt algorithms in courtrooms for serious decisions (ie, sentencing) on that basis, since we may never be sure it is truly blind justice?
Ms Fry: Every criminal-justice system has to find some kind of balance between protecting the rights of innocent people falsely accused of crimes and protecting the victims of crimes. Getting that balance is difficult, and the judicial system is not perfect—and it doesn’t try to be. That’s why phrases such as “reasonable doubt” and “substantial grounds” are so fundamental to the legal vocabulary: the system accepts that absolute certainty is unachievable.
But even within those bounds, there’s a hell of a lot of inconsistency and luck involved in judges’ decisions. People are terrible at making fair, consistent decisions. And judges—like the rest of us—are not very good at putting their subconscious biases to one side.
If you’re careful, I do think there is potential to minimise some of those problems by using algorithms to support judges’ decisions. You just have to make sure you use them in a way that makes the system fairer, and doesn’t end up accidentally exacerbating the biases that are already there.
The Economist: Do you worry that humans will eventually give up their authority and power on important areas of life to machines, in the way that we already give up our sense of direction (and common sense!) to online maps?
Ms Fry: I certainly think there are some skills we’ll lose as we hand things over to automation. I can barely remember my own phone number now, let alone the long list of numbers I used to know, and my handwriting has completely gone to pot. But I wouldn’t say I particularly worry about it.
There are times where de-skilling is a real concern though. Pilots have already been through this: the better autopilot got, the less comfortable junior pilots became at controlling their planes by hand. In the operating theatre, where junior surgeons would have trained by assisting a consultant in open surgery—with their hands physically inside a patient, getting the touch and feel of a body—now they train by watching a keyhole procedure being played by a consultant sitting at a console, and an internal camera relaying to a screen.
And if we get to the stage where driverless cars become prevalent, the population’s competence in driving unassisted will drop without serious consideration around how we keep up our skills—which is something we need to do if we’re still expected to step in and take control of the car in an emergency.
There are things you can do to avoid this problem, like deliberately switching the machine off every now and then. But it starts, I think, with acknowledging that automation is still sometimes going to fail, and making sure that the human—and their needs and failings—stay at the very centre of your consideration at all times.
The Economist: When algorithms move into medicine, law and elsewhere, their decisions may be called “ recommendations ” that humans-in-the-loop can override. But most of what we know from behavioural psychology says that this is a fiction: people will be inordinately influenced by it. How can we realistically overcome this problem?
Ms Fry: People are often quite lazy. We like taking the easy way out—we like handing over responsibility, we like being offered shortcuts that mean we don’t have to think.
If you design an algorithm to tell you the answer but expect the human to double check it, question it, and know when to override it, you’re essentially creating a recipe for disaster. It’s just not something we’re going to be very good at.
But if you design your algorithms to wear their uncertainty proudly front and centre—to be open and honest with their users about how they came to their decision and all of the messiness and ambiguity it had to cut through to get there—then it’s much easier to know when we should trust our own instincts instead.
I think this was one of the best features of IBM’s Watson, which played the American quiz show Jeopardy! and won. While the format of the quiz show meant it had to commit to a single answer, the algorithm also presented a series of alternatives that it had considered in the process, along with a score indicating how confident it was in each being correct.
It’s also what is good about the more recent sat-navs: they don’t just decide on a route for you, they give you three to choose from and the pros and cons of each. Just enough information for you to make your own, informed decision, rather than blindly handing over control.
The Economist: What do humans do that machines can’t? What changes do we need to make in society to help humans to flourish in the algorithmic age?
Ms Fry: Humans are still much better than machines at understanding context and nuance. We’re still far more adaptable. You can pick us up and drop us in a totally new environment and we’ll know how to behave, something that even the best AI is a very long way away from achieving.
But aside from anything else, this is a human world, not an algorithmic one. And so the humans should always be front and centre of the thinking for any new technology.
That seems obvious, but it’s something that hasn’t always happened of late. There’s been a trend to push new algorithms out into the world quickly, run live experiments with them on real people, without stopping to think if they’re doing more harm than good, and worry about adapting them later if they’re shown to be problematic. (Social media: I’m looking at you).
I think that society needs to insist that new technology, like new pharmaceuticals, is careful and upfront about the worst-case scenarios. I think that the algorithms we build should be designed to be honest about their weaknesses and candid about how perfection is often impossible. But most of all, I think that the algorithms we build should be designed to accept our human failings, rather than turn a blind eye to them.
* * *
The justice equation Excerpted from “Hello World: How to be Human in the Age of the Machine Book” by Hannah Fry (Doubleday, 2018):
Algorithms can’t decide guilt. They can’t weigh up arguments from the defence and prosecution, or analyse evidence, or decide whether a defendant is truly remorseful. So don’t expect them to replace judges any time soon. What an algorithm can do, however, incredible as it might seem, is use data on an individual to calculate their risk of re-offending. And, since many judges’ decisions are based on the likelihood that an offender will return to crime, that turns out to be a rather useful capacity to have.
Data and algorithms have been used in the judicial system for almost a century, the first examples dating back to 1920s America. At the time, under the US system, convicted criminals would be sentenced to a standard maximum term and then become eligible for parole after a period of time had elapsed. Tens of thousands of prisoners were granted early release on this basis. Some were successfully rehabilitated, others were not. But collectively they presented the perfect setting for a natural experiment: could you predict whether an inmate would violate their parole?
Enter Ernest W. Burgess, a Canadian sociologist at the University of Chicago with a thirst for prediction. Burgess was a big proponent of quantifying social phenomena. Over the course of his career he tried to forecast everything from the effects of retirement to marital success, and in 1928 he became the first person to successfully build a tool to predict the risk of criminal behaviour based on measurement rather than intuition.
Using all kinds of data from three thousand inmates in three Illinois prisons, Burgess identified 21 factors he deemed to be ‘possibly significant’ in determining the chances of whether someone would violate the terms of their parole. These included the type of offence, the months served in prison and the inmate’s social type, which—with the delicacy one would expect from an early-twentieth-century social scientist—he split into categories including ‘hobo’, ‘drunkard’, ‘ne’er do-well’, ‘farm boy’ and ‘immigrant’.
Burgess gave each inmate a score between zero and one on each of the 21 factors. The men who got high scores (between 16 and 21) he deemed least likely to re-offend; those who scored low (four or less) he judged likely to violate their terms of release.
When all the inmates were eventually granted their release, and so were free to violate the terms of their parole if they chose to, Burgess had a chance to check how good his predictions were. From such a basic analysis, he managed to be remarkably accurate. Ninety-eight per cent of his low-risk group made a clean pass through their parole, while two-thirds of his high-risk group did not. Even crude statistical models, it turned out, could make better forecasts than the experts.
But his work had its critics. Sceptical onlookers questioned how much the factors which reliably predicted parole success in one place at one time could apply elsewhere. (They had a point: I’m not sure the category ‘farm boy’ would be much help in predicting recidivism among modern inner-city criminals.) Other scholars criticized Burgess for just making use of whatever information was on hand, without investigating if it was relevant. There were also questions about the way he scored the inmates: after all, his method was little more than opinion written in equations. None the less, its forecasting power was impressive enough that by 1935 the Burgess method had made its way into Illinois prisons, to support parole boards in making their decisions. And by the turn of the century mathematical descendants of Burgess’s method were being used all around the world.
Fast-forward to the modern day, and the state-of-the-art risk-assessment algorithms used by courtrooms are far more sophisticated than the rudimentary tools designed by Burgess. They are not only found assisting parole decisions, but are used to help match intervention programmes to prisoners, to decide who should be awarded bail, and, more recently, to support judges in their sentencing decisions. The fundamental principle is the same as it always was: in go the facts about the defendant—age, criminal history, seriousness of the crime and so on—and out comes a prediction of how risky it would be to let them loose.
So, how do they work? Well, broadly speaking, the best-performing contemporary algorithms use a technique known as random forests, which—at its heart—has a fantastically simple idea. The humble decision tree.
Ask the audience
You might well be familiar with decision trees from your schooldays. They’re popular with maths teachers as a way to structure observations, like coin flips or dice rolls. Once built, a decision tree can be used as a flowchart: taking a set of circumstances and assessing step by step what to do, or, in this case, what will happen.
Imagine you’re trying to decide whether to award bail to a particular individual. As with parole, this decision is based on a straightforward calculation. Guilt is irrelevant. You only need to make a prediction: will the defendant violate the terms of their bail agreement, if granted leave from jail?
To help with your prediction, you have data from a handful of previous offenders, some who fled or went on to re-offend while on bail, some who didn’t. Using the data, you could imagine constructing a simple decision tree by hand, like the one below, using the characteristics of each offender to build a flowchart. Once built, the decision tree can forecast how the new offender might behave. Simply follow the relevant branches according to the characteristics of the offender until you get to a prediction. Just as long as they fit the pattern of everyone who has gone before, the prediction will be right.
But this is where decision trees of the kind we made in school start to fall down. Because, of course, not everyone does follow the pattern of those who went before. On its own, this tree is going to get a lot of forecasts wrong. And not just because we’re starting with a simple example. Even with an enormous dataset of previous cases and an enormously complicated flowchart to match, using a single tree may only ever be slightly better than random guessing.
And yet, if you build more than one tree—everything can change. Rather than using all the data at once, there is a way to divide and conquer. In what is known as an ensemble, you first build thousands of smaller trees from random subsections of the data. Then, when presented with a new defendant, you simply ask every tree to vote on whether it thinks awarding bail is a good idea or not. The trees may not all agree, and on their own they might still make weak predictions, but just by taking the average of all their answers, you can dramatically improve the precision of your predictions.
It’s a bit like asking the audience in Who Wants To Be A Millionaire? A room full of strangers will be right more often than the cleverest person you know. (The ‘ask the audience’ lifeline had a 91 per cent success rate compared to just 65 per cent for ‘phone a friend’.?) The errors made by many can cancel each other out and result in a crowd that’s wiser than the individual.
The same applies to the big group of decision trees which, taken together, make up a random forest (pun intended). Because the algorithm’s predictions are based on the patterns it learns from the data, a random forest is described as a machine-learning algorithm, which comes under the broader umbrella of artificial intelligence. ([…] It’s worth noting how grand that description makes it sound, when the algorithm is essentially the flowcharts you used to draw at school, wrapped up in a bit of mathematical manipulation.) Random forests have proved themselves to be incredibly useful in a whole host of real-world applications. They’re used by Netflix to help predict what you’d like to watch based on past preferences; by Airbnb to detect fraudulent accounts; and in healthcare for disease diagnosis.
When used to assess offenders, they can claim two huge advantages over their human counterparts. First, the algorithm will always give exactly the same answer when presented with the same set of circumstances. Consistency comes guaranteed, but not at the price of individualized justice. And there is another key advantage: the algorithm also makes much better predictions.
Excerpted from “Hello World: How to be Human in the Age of the Machine Book” by Hannah Fry. Published by Doubleday. Copyright © 2018 by Hannah Fry. All rights reserved.
- The AFL’s administration has failed and it must be changed
- NHTSA Fines Ferrari $3.5 Million for Failing to Submit Reports
- Beyond Big Data: Why Human Interpretation Still Counts
- Who’s Really to Blame for Robot-Human Crashes? Are We Really Such Awful Drivers?
- Monster Roster’s algorithm predicts what’s next on The Bachelor
- 3 Ways to Survive Instagram's Algorithm Update
- Can the Roboracer improve actual human racing?
- Watch DURUS Robot Walking Like A Human
- Beyond Human Resources: 4 Ways to Improve Human Capital Management
- Maintenance ignorance costing drivers £326 each at MOT time