Daniel Kahneman on Making Intelligent Decisions in a Chaotic World | Intelligence Squared

[Applause] welcome everyone to the Union Chapel and to intelligence squar it is so fabulous to be back with all of you in the hall not on a computer there are however people watching online so hello to you thank you for joining in the very fact that uh we're all here tonight is proof enough isn't it that we've already made at least one intelligent decision in a chaotic World added to which I haven't arrived with a suitcase full of wine or indeed a karaoke machine uh and I know we'll all be polite to the security staff

and the cleaners um our decision making so far has been impeccable but uh this evening's conversation May reveal that perhaps not all of our decisions are as sound as we'd like to believe our confidence in our own expertise may be misplaced and our judgments may be flawed so without further Ado let me introduce our speakers Daniel kman probably doesn't need much of an introduction he's a Nobel Prize winner and the author of The hugely influential International bestseller thinking fast and slow and his day job is as professor of psychology and public affairs at Princeton University

um Professor caraman is of course here to talk about his new book noise a flaw in human judgment and I'm delighted to say that we're also joined by one of his co-authors Olivier bourney who is Professor of strategy and business policy at H Paris previously he was a senior partner at McKenzie and he's also the author of you're about to make a terrible mistake well I hope we're not about to do that right now so um before we really get going I just want to remind you that I want all of you to get involved

both in the hall and at home online uh there's a chance for lots of q&as at towards the end after about an hour or so there will be microphones down here at the front and up there please come forward I would love to hear from you all and if you're online there is a sort of a a bit at the bottom of the screen as it were uh you can ask questions in the in the Box just beneath the screen type them in we want to hear from you get thinking and uh I'll make sure

there's plenty of time for questions from everyone let's get going then noise fresan basic definitions what is noise well noise has many meanings but the noise that we're talking about is Judgment noise and judgment noise is variability that shouldn't exist it's when people make judgments that should be identical because they're about the same subject but they turn out to be variable and and there is a lot of noise and our statement about this is that wherever there is Judgment there is noise and there is a lot more than you think and you know that's really

what the book is about and there is more noise Than People expect to see and how is that different from what we think of as bias well to draw the distinction sharply between noise and bias we we have to see what we have to talk about what judgment is and the way we think about judgment is we think of judgment as a species of measurement where the measuring instrument is the human mind and like other measurements you deal with a subject an object topic and you assign that topic of value on a scale and this

is what Jud the kind of judgment that we talk about is we talk about really judgments about specific topics now in measurement there is a theory of measurement that is a basic Theory you know that serves uh all the sciences and there is a of course a theory of Errors of measurement so when you measure you make take a physical measurement uh say of a line with a very very fine ruler and you take repeated measurements of the length of the same lines uh then what's going to happen is this you're not going to get

the same result every time if the the ruler is fine enough uh and that variability that is noise but there is another kind of error and this is that every measurement is likely to be an error an overestimate or an underestimate and the average error is called bias and this is really those are the meanings of bias and noise that we use in discussing judgment so bias is an average systematic error and noise is just a variability of judgments and it's a variability that shouldn't exist so Olivier why are we all so so much more

familiar with the idea of bias is it the sort of sexy cousin of noisy well it's sexier it's more visible so a way to think about the definition Danny gave which is perhaps more visual is to imagine that you've got a team of Shooters shooting at a Target they're all using the same rifle and they're all shooting at the same Target and you look at the Target after they all took their shot and if you see that they all hit the bullseye you say great they're accurate now if you see that they've all hit the

same place but that place is not the bullseye you immediately say this calls for an explanation there must be a reason there's something wrong with the rifle or there is something wrong with the Target or somebody moved the Target or maybe the wind is blowing really hard you immediately look for a c that's why bias is sexier as you say right because bias is something you can explain you can point the finger at the reason why these people all made the same mistake if by contrast you see your five or six shooters shooting all over

the place you just shrug and say Well they're not great marksmen but that doesn't give you an urge to explain so noise is more random by definition and is less easy to spot when you see it I'm still curious about why we haven't thought about this more systematically in the way that you do in the book is it that in a sense people in general just have too much confidence in their judgment in their decision making well you know it would be an exaggeration to say that people have not thought about it U people have

looked at reliability of judgment for a long time what I think Justified writing the noise writing the book was with the pre well the prevalence just a sheer amount of noise and for that possibly an example would use absolutely why should we worry about it I'd like to tell the story of how the study began and and the study began number of years ago when I was doing a consulting job at uh at an insurance company and I had the idea of running what we now would call a noise audit but it's really a fairly

standard exercise of constructing problems that were typical underwriting problems quite realistic and having multiple Underwriters put a dollar value on the same risk and of course you wouldn't expect these people to agree but I asked the executives if you take two under writers at random by how much do you expect them to vary in percentages of and and that question it turns out has an answer and without asking asking you I think we know what number is in your mind because we've we've asked that question of a large number of people and the number is

roughly 10% people think that 10% is a reasonable difference between people we know that they won't agree perfectly because it's a matter of judgment and judgment is defined by the possibility of reasonable disagreement but we expect a limited amount of noise it turned out that the difference the the average difference between two Underwriters was 50% wow five times larger than the executives expected and that is really the reason for writing the book is that there was so much more noise Than People expected and this came as a complete surprise to their organization and Olivier soon

joined and we worked together on on that topic and it turns out that it's not only in the insurance company but wherever you look at judgment you find noise and a lot of it and there's also a misunderstanding which explains why people have paid less attention to it the misunderstanding is the the misconception that it doesn't matter because on average it cancels out right noise is an error of average zero essentially and people tend to say well we care about bias because bias is the shared error but we don't care about noise because on average

it cancels out of course that's wrong right if you think of the underwriters if on average they price correctly but sometimes they price way too low and sometimes it's way too high well when it's way too low the company is going to lose money by paying too much in claims and when it's way too high is going to lose money by losing business to a competitor so both mistakes are costly but when we think of Noise We tend to think that well average errors you on average errors cancel out they don't and that's what we

want wanted to draw attention to so so it is about in a business context and we can come back to some more examples but in a business context there is money at stake essentially is what you're saying it's a financial cost oh absolutely it's a Mis it's an error what I actually the a notion I I should have mentioned earlier from the theory of measurement is that in the theory of measurement noise and bias are equally important in fact there is a Formula the ma the mathematics don't matter but the formula is that a measure

of global inaccuracy is the square of the bias plus the square of the noise and that's just to give you an idea that bias and noise are independent sources of inaccuracy and in principle equally important in practice we think there is more noise than bias in judgment it has a cost as you point out but it's not just about the business cost another example which we looked at is Justice Criminal Justice and when you give the same cases this is an old study that was done in the US we're not aware of it being done

anywhere else including here but I'm sure the results would be very similar if you give the same cases simplified cases vignettes to multiple experienced judges you will find a lot of variability in their judgments for some of those cases you will find some judges saying one you're in prison and others saying like in prison for some of these cases you will find some judges saying 15 years in prison and others saying no prison at all on average for a sentence of seven years you have almost four years of difference between two judges again looking at

the same case we're not talking about individualized sentencing you taking care of the specifics of each situation that is already in the case we're talking about different judges looking at the same case this is not a business cost but this is a huge problem of fairness and one of the big consequences of noise is unfairness the thing that I found fascinating about that particular example was that it had been recognized in a proper study mandatory guidelines were put in place and then they were taken they there was so much the judges were so unhappy that

they became advisory it's it's slightly more complicated in fact it was a long battle led by a judge called Franco in the US and then by Ted Kennedy when he was Senator which led to establishing guidelines that Define rather precisely what sentencing should be with some leeway to adjust to the situation and the possibility of going out of the guidelines if you Justified it Etc and then for technical reasons that have nothing to do with what we're discussing here the Supreme Court turned those guidelines into advisory guidelines and noise crept back in it's not because

the judges were unhappy about them but the judges were unhappy but but they were unhappy and as soon as they could actually regain their discretion they used it to the full extent that they could discussing noise with judges is quite an interesting exercise because the you know the whole idea is there is this is a justice system Justice is being determined as being me of to defendants and the idea that there is noise in that system is acceptable to to the extent that I was actually invited to talk to a large group of Judges but

what they what the organizers asked me to talk about was noise in medicine so uh noise drilling noise in Justice is quite quite complex but actually noise in medicine sorry carry on but what's the noise in medicine there is a lot of noise in medicine uh about as much noise as as wherever there is Judgment there is noise and so this is true for diagnosis this is true for treatment uh in in some cases it really is quite shocking we had uh a set of data sent to us from Harvard Medical School on the diagnosis

of epilepsy from EG recordings and the average correlation between two physicians in judging cases is not exactly zero but not much higher than zero and in negative I think I if I recall correctly I think it was a small negative number no wasn't it I I don't think I remember it as slightly positive but Cas in either case it was very small adding adding to that view that medicine is an art rather than a science but so far I mean these are decisions that are made repeatedly so if you go back to the judge or

the the doctor indeed that they're making similar decisions over and over again does it apply to oneoff decisions can you talk talk about noise in in singular decisions well we we ask ourselves that question because obviously when we measure a noise we me we when we Define no we Define it as variability between multiple decisions that should be identical that definition does not apply to a decision that you make only once but when we look at the things that cause noise which we haven't talked about yet the the psychological mechanisms behind noise it's very clear

that they're also present when you make one of those one of life or death momentus change the world decisions you know brexit or no brexit is there noise in that decision well if you think that it's a judgment that is being made by a human being there must be because the same causes that create noise out there we we came to think of those singular decisions those unique decisions as repeated decisions that happen only once right if they were repeated you would actually see that there is noise in them you can't see it but that

doesn't mean there isn't noise so so let's talk about the causes of noise I mean why do three judges facing very similar cases make three different decisions or three doctors looking at the same x-ray come to three different conclusions well there are several sources of noise actually there are three main sources of noise and one source of noise is differences consistent differences between noise in between judges in the of the judgments they make so some judges will be more severe than others so on average their sentences the sentences of one judge will be higher than

the other this is well known this is quite recognized there is another source of noise which is also we is not surprising to people and this is noised within a judge that is the same judge uh in a good mood or in a bad mood it turns out on a sunny day or an rainy day and a hot day or a cooler day will pass make somewhat different judgments we call that occasion noise but then there is a third source of noise which turns out to be the most important and and I would call that

it's a difference in the almost in personality if you will that and to see it in the decisions in the Justice context if you take sever if you take two judges and you show them 20 cases the new type of noise is that they will not order the cases in terms of severity in the same way so you might have one judge who is particularly shocked by uh when you know the defendants are uh young or when the victims are old and and there are differences in taste you can have a judge who is strongly

affected by someone was similar to to a member of their family and that is that would be consistent and we call that pattern noise and it turns out that pattern noise is the more important the more interesting and the most difficult kind type of noise so pattern noise is not quite like group think is it where a group works but does that affect noise as well then group well let's clarify patter noise first okay go clarify P noise yes so p and that touches on something that you mentioned earlier that when people look at the

world or at any problem in the world uh you have the sense every one of us had the sense that we see the world the way we do because that's the way it is so we have the sense that we're in touch with reality and if the world is as we see it we expect other reasonable people to see it in the same way and it turns out that they don't to an extent that is radically surprising so what what we have to take on board here is is the idea that people are really more

different from each other in how they see any problem in the world than any of them would expect and that's the basic phenomenon of no and we also have to come to grips with the idea that that's not good news when when we say oh we're all different we we're all unique we're all diverse we have different views on everything we usually celebrate that we say that's beautiful that's diversity that's where creativity comes from that's where Innovation comes from but the thing is when you go to one doctor and he says you have this and

then you go to another doctor and he says you have that you don't say oh beautiful creativity Innovation diversity no you say one of you guys must be wrong perhaps both by the way so when we think there is a correct answer which is how we Define judgment that kind of diversity is what we call pattern noise the fact that people project their own history and their own sensibilities and their own biases on a situation is what creates this pattern noise and it's not good news isn't that what people call human agency it is what

people call projecting their personality into their judgment and of course if you expect them to do that that's beautiful if you are choosing whom to marry in that way congratulations that's how I would strongly recommend you do it if you're making a hiring decision on behalf of the BBC we might have a problem so to go back to the the earlier point then if you have a meeting and the first person stands up and says I definitely want to do it this way uh you know I definitely think we should only employ people who wear

red t-shirts and the second person gets up and says or are they more likely to to make the same decision how do people affect one another how does that create noise groups tend to create noise in just the way that you said in that when you want to minimize noise or to reduce noise then you want the judgments of people to be independent of each other so you would want witnesses to the same crime to discuss their testimony before they give it and this is precisely what happens in a meeting in a meeting people influence

each other and the first person to speak has a disproportionate influence on the others and that is an element of noise because the first person to speak is not necessarily the best is not necessarily the most accurate uh so so are you saying in a sense that we're disinclined to disagree well certainly there is a massive amount of conformity that goes on and this is where you know otherwise meetings might go on much longer but uh and and as it is they G too long but there is there is Convergence and there is in in

that sense more convergence than we would want that is ideally the ideal form for a meeting would be one in which people to which people come prepared each with their own opinion and and you establish the amount of Divergence in their opinions and now you discuss it and now you try to reach convergence but the automatic kind of convergence that happens in a debate is actually not a noise reduction mechanism when we were working on this topic we interviewed an organizational psychologist who worked with the admissions officers of universities and he told us the following

story he had come to an admissions office in large University where admissions officers would read the application essays of applicants and they would grade them and then they would give the essay with the grade to another admissions officer who would make a separate judgment so our friend recommended of course that they should make those judgments Independent by hiding the grade given by the first officer from the Second Officer so that these two judgments could be different and the answer he got was oh that's how we used to do it but we disagreed so much that

we adopted the current method and this is a perfect summary of the trade-off that organizations make between consensus which they must achieve at some point and disagreement which they do have they tend to suppress dis agreement as best they can in order to achieve consensus and to be able to make decisions and what we're saying here is if you want consensus to be achieved on something closer to the best possible answer you might want to delay the moment when you you achieve that consensus and to make sure that you get independent opinions at the beginning

of the process it's interesting that we're talking about convergence and people wanting to agree when certainly in politics what we talk about all the time is polarization well uh we talk about judgments and and we try to avoid judgments of values and differences in values so the assumption is when when you have underr or or judges they speak for an organization the organization speaks through individual functionaries and you want an organization to speak in one voice uh and and noise is a failure is a cacophony but when it comes to values or when it comes

to politics in a democracy you certainly want to allow differences and Divergence of values and and and you certainly don't want to impose noise reduction oh that's interesting no noise reduction in politics H um so let's just think about some of the things that this has thrown up that convergence in a sense hide well it's a way of avoiding the fact that there is noise why do you think institutions like educational establishments or businesses are reluctant to confront the idea of noise Beyond just wanting to get on and make any decision whether it's good or

bad well I think first they are not aware of the amount of noise that there is I think you know every time we show the results of a noise audit to an organization or we just share the results of noise audits performed in other organizations people are very surprised we could hear the surprise in the room when Denny mentioned the 50% number from the insurance company this is very surprising so I think the obvious answer I mean the the first and most important answer is they just don't know and that's partly why we wrote the

book really but in lots of fields you're getting the use of algorithms to I think about Actuarial work which is obviously in the insurance field you know humans used to do all the risk calculations and now a proportion will be done by a computer but they still as far as I can tell ask humans to make the final judgment is that because we don't want to hand over that that power uh it clearly depends on the topic I mean there is a long history about 70 years of comparing the judgments of people to formulas and

and more recently to algorithms that are generated by Ai and and humans do not fair very well in those comparisons and the main reason for human inferiority is actually noise because algorithms have and rules simple or complex have the advantage of being Noise free and that gives them a real leg up when it comes to accuracy but but isn't the obvious Counterpoint to that that algorithms can make matters worse because they will have inherent in them the biases of of whoever wrote the code they will although there is no reason why those biases should be

worse than the biases of the humans again it depends what the comparison point is you see the bias of algorithms is a big topic right it's being discussed everywhere and it should be because it's real but the B of algorithms are there because algorithms are trained on data that is the past decisions and the past judgments of humans so what algorithms are doing is they are the mirror of our own biases they are not worse what makes the bias more visible when you actually have an algorithm is two things first you can actually run a

million decisions into the algorithms and see how many you know biased decisions there are which you cannot do with a human being second the algorith is Noise free as Denny pointed out it's very consistent in its biases so if you train your hiring algorithm and you tell the algorithm you these are all the people who have been promoted in my company figure out what it takes to be successful in my company and the algorithm comes back and says you have to hire men not women is the algorithm sexist no you are the reason you hadn't

noticed is because you are noisy in your sexism even the even the most sexist recruiter is occasionally going to hire a woman the algorithm won't do that the algorithm will say well it's pretty clear that here to be successful you need to be a man so the the absence of noise makes the bias more visible it doesn't actually make it worse it also makes it easier to solve if you want to solve it you can actually design an algorithm that would be devoid of such biases so algorithms could make very good decisions do we want

them to that's a very different issue yeah when you know algorithms are biased biased algorithms are poorly constructed algorithms and typically the bias enters somewhere in the definition of what it is that you're using as as a Criterion for Success so for example Amazon uh tried to develop an algorithm to replicate its high decisions and then that algorithm turned out the way they constructed it they looked at people who had been hired or had been rejected and they looked for the what distinguished them and then they found there was a gender bias because people had

in fact preferred men to women in their hiring and that went on to the algorithm that's a flawed Criterion that is when you want to hire hire properly you should not go by the previous hiring decisions then you are guaranteed to replicate the bias but when you do it properly and then bias is not a necessary feature of uh algorithm although as Olivier pointed out the biases of algorithms are always going to be more visible because they're going to be more detectable because they're not masked by noise whereas the biases of humans are very frequently

M away noise I'm beginning to think that you think expertise is overrated is it a concept we shouldn't even talk about expertise in the field if you want us to say oh we've had enough of experts we're not going to say that no there is an interesting question when you think about expertise how we evaluate expertise and this is that many of the judgments where we look at experts are actually unverifiable that is there are judgments that we make like long-term forecasts uh which cannot be verified and yet so you would think that people who

specialize in that kind of judgments how how do we know how to distinguish a good professional from a weaker professional and there it turns out that there are ways for people to emerge as experts we call them respect experts because they're experts because they're respected not because of the quality of their performance because their performance cannot be evaluated so there were expert astrologers and that is very useful to remember that some astrologers ERS had the respect of their peers and they were taken more seriously than others and and they operated as experts and what creates

a respect expert is self-confidence eloquence intelligence I mean a lot of properties that we want but they do not guarantee good performance good performance requires some verifiable Criterion some way some feedback about the accuracy of the drug made now if you are a chess player if you are a weather forecaster if you are an investor your expertise can be measured can be assessed against objective benchmarks and we can decide after a while that you are an excellent or a poor weather forecaster because we see how accurate you are these are not the respect experts they

are the experts you can see the outcomes yes so a fair chunk of the book is actually a sort of a how-to manual and by that I mean you sort of say how to detect and reduce noise give us some pointers what's what if I if I decide this is a really bad thing I want my decisions to be more efficient to be more accurate what do I do so we've talked about algorithms and the basic idea of algorithms is to say as Denny pointed out wherever there is human judgment there is going to be

noise if you want there not to be noise take out the human judgment let's put that aside because there's many situations where it's either impractical or undesirable for either good or bad reasons to use algorithms and you will want to use human judgment when you are going to use human judgment the approach or the set of approaches we suggest to reduce noise in human judgment is what we call decision hygiene and the reason we use this odd phrase is because the analogy of washing your hands is apt here we when we wash our hands we

are not saying oh this was this germ that I take out and this is this disease that I've avoided we don't know what problem we've avoided we just know it's good prevention we're putting the process under control and decision hygiene does that it puts the decision process it puts the process of judgment under some sort of control how do you do that we've talked about aggregating independent opinions that's a good way to introduce decision hygiene making sure these opinions are independent if you aggregate opinions through discussion as we pointed out that actually does more harm

than good another way would be to structure your decisions yet another way would be to use relative judgments rather than absolute judgments wherever you can there's a whole series of things which taken together help introduce a little bit more discipline into your decision making and I should point out here that when we talk about noise reduction or about decision hygiene we had in writing the book we had organizations in mind that the book is not self-help for individuals um I'm quite skeptical about the ability of individuals to improve their own thinking but organizations have procedures

they have processes it is possible although it this is not typical that there's few organizations that I know about have designed processes for reaching judgments or decisions but we believe that more organizations should have designs for making decisions and design that embodied or that incorporate the principles of decision hygiene so organizations have an opportunity that individuals do not have at the beginning you you described the the underwriters at the insurance agency understand realizing how much noise there was but is it okay so there you identified it but is it an expensive process a difficult process

for for for organizations to undertake do you think there will be a reluctance to identify noise well certainly you know in part because noise is so abstract that as you can point out to an error to a single error and it looks like a bias and it's easy to explain as Olivier was mentioning earlier noise there is no single error unless it's a complete outlier but normally you cannot point to an eror and say this is noise it takes more than error it takes the variability of errors this gives noise that abstract sort of character

that makes it quite difficult to conceive of and it is true that when you try to mitigate noise in an organization you will certainly face resistance and you will face resistance which in part is fully Justified because there is the danger of bureaucracy there is a danger of imposing procedures that turn mechanical and that make people less involved in the judgments that they make so there is really a tradeoff between design decisions and spontaneous decisions and it's up to individual organizations to find their way in that tradeoff and before you even get to noise reduction

just becoming aware of noise is something that some organizations that many organizations will resist because noise is embarrassing you we've talked about one problem with noise which is that is costly we've talked about not another which is that it creates unfairness there's a third reason why noise is detrimental which is that it's embarrassing to The credibility of the organization if in an insurance company you become aware that you might get a quote for $100,000 and the next customer with exactly the same need might get a quote for2 200,000 that doesn't reflect very well on the

organization and it's very tempting to move on and talk about something else and I can tell you that this is what happened in the insurance company they they hadn't known about the problem and once they were made aware of the problem I think they forgot it very quickly but but does that perhaps reflect that the to come back to this idea of human agency that the power to exercise discretion it may be far less efficient but actually within a company within a hierarchy it's a necessary trade-off I'm the top dog I know what I'm doing

and I'm going to tell you well it's it's certain very tempting I we we had an interesting conversation yesterday with a very senior HR executive who said you know we read everything that you talk about and it's clear that it would improve the odds of hiring the best people but as you point out yourself hygiene is a bit tedious it's much more fun to hire people I like and if that's what you're going to do that's fine it's your business if I were your boss I would probably raise some questions but I'm not so you

do whatever and there is another reason that really comes up and that judgment is typically and quite often very highly fallible so for example in hiring real accuracy in hiring is impossible it's impossible because future performance is not fully predictable a lot of things are going to happen on the job that can be known in advance that are not characteristics of the individual and that will determine the individual's performance so the best that you can do in hiring is quite poor but you can do worse than that and that is and and people do but

what you get as a response quite often well if all you can promise me is that performance will be quite poor I might as well trust my God and that's a mistake