What do you think?
Rate this book
Noise: A Flaw in Human Judgment
From the bestselling author of Thinking, Fast and Slow and the co-author of Nudge, a groundbreaking exploration of why most people make bad judgments, and how to control for that noise.
Imagine that two doctors in the same city give different diagnoses to identical patients — or that two judges in the same courthouse give different sentences to people who have committed the same crime. Suppose that different food inspectors give different ratings to indistinguishable restaurants — or that when a company is handling customer complaints, the resolution depends on who happens to be handling the particular complaint. Now imagine that the same doctor, the same judge, the same inspector, or the same company official makes different decisions, depending on whether it is morning or afternoon, or Monday rather than Wednesday. These are examples of noise: variability in judgments that should be identical.
In Noise, Daniel Kahneman, Cass R. Sunstein, and Olivier Sibony show how noise contributes significantly to errors in all fields, including medicine, law, economic forecasting, police behavior, food safety, bail, security checks at airports, strategy, and personnel selection. And although noise can be found wherever people make judgments and decisions, individuals and organizations alike are commonly oblivious to the role of chance in their judgments and in their actions.
Drawing on the latest findings in psychology and behavioral economics, and the same kind of diligent, insightful research that made Thinking, Fast and Slow and Nudge groundbreaking New York Times bestsellers, Noise explains how and why humans are so susceptible to noise in judgment — and what we can do about it.
Imagine that two doctors in the same city give different diagnoses to identical patients — or that two judges in the same courthouse give different sentences to people who have committed the same crime. Suppose that different food inspectors give different ratings to indistinguishable restaurants — or that when a company is handling customer complaints, the resolution depends on who happens to be handling the particular complaint. Now imagine that the same doctor, the same judge, the same inspector, or the same company official makes different decisions, depending on whether it is morning or afternoon, or Monday rather than Wednesday. These are examples of noise: variability in judgments that should be identical.
In Noise, Daniel Kahneman, Cass R. Sunstein, and Olivier Sibony show how noise contributes significantly to errors in all fields, including medicine, law, economic forecasting, police behavior, food safety, bail, security checks at airports, strategy, and personnel selection. And although noise can be found wherever people make judgments and decisions, individuals and organizations alike are commonly oblivious to the role of chance in their judgments and in their actions.
Drawing on the latest findings in psychology and behavioral economics, and the same kind of diligent, insightful research that made Thinking, Fast and Slow and Nudge groundbreaking New York Times bestsellers, Noise explains how and why humans are so susceptible to noise in judgment — and what we can do about it.
454 pages, Hardcover
First published May 18, 2021
About the author
Daniel Kahneman
61 books8,751 followersFrom Wikipedia:
Daniel Kahneman (Hebrew: דניאל כהנמן; born 5 March 1934 - died 27 March 2024), was an Israeli-American psychologist and winner of the 2002 Nobel Memorial Prize in Economic Sciences, notable for his work on behavioral finance and hedonic psychology.
With Amos Tversky and others, Kahneman established a cognitive basis for common human errors using heuristics and biases (Kahneman & Tversky, 1973, Kahneman, Slovic & Tversky, 1982), and developed Prospect theory (Kahneman & Tversky, 1979). He was awarded the 2002 Nobel Prize in Economics for his work in Prospect theory. Currently, he is professor emeritus of psychology at Princeton University's Department of Psychology.
http://us.macmillan.com/author/daniel...
Daniel Kahneman (Hebrew: דניאל כהנמן; born 5 March 1934 - died 27 March 2024), was an Israeli-American psychologist and winner of the 2002 Nobel Memorial Prize in Economic Sciences, notable for his work on behavioral finance and hedonic psychology.
With Amos Tversky and others, Kahneman established a cognitive basis for common human errors using heuristics and biases (Kahneman & Tversky, 1973, Kahneman, Slovic & Tversky, 1982), and developed Prospect theory (Kahneman & Tversky, 1979). He was awarded the 2002 Nobel Prize in Economics for his work in Prospect theory. Currently, he is professor emeritus of psychology at Princeton University's Department of Psychology.
http://us.macmillan.com/author/daniel...
Ratings & Reviews
Friends & Following
Create a free account to discover what your friends think of this book!
Community Reviews
Displaying 1 - 30 of 1,420 reviews
May 26, 2021
You know what the real lesson here is, don’t pre-order books based on the authors reputation alone. In a world filled with noise, these authors contribute to it through their generally inadequate book.
I really wanted to like this. I liked Nudge which has Cass as an author, I generally liked Thinking Fast and Slow, and I want someone who’s not Nate Silver explain signal to noise ratios to help me curate better information in my life. But this book isn’t it. This book is literally noise. Worthless noise in an already noisy world.
Someone like Kahneman, a founder of behavioral economics, you would think would have interesting new research and considered takes on how to cut through the amount of chatter out there in the world. It’s an important problem. But it seems like behavioral economics has stalled out into finding goofy and minor errors in our cognitive biases. Hey look! two people came to different answers when asked to mentally calculate an abstract concept. Look at how I can create methodologically dubious and unreplicatible studies that confuse people into making decisions against their best interests. Am I a behavioral economist yet?
I’m so sick of people writing shitty books to promote themselves as “thought leaders” and charge more for their consulting. I expected a better book out of these authors but found myself extremely disappointed in the shallowness of the ideas and writing. It’s a bad regurgitation of ideas that has been done better in other places.
If you like feeling cocktail-party smart without actually having to put in the effort to be smart, you will probably like this. It’s full of pithy blurbs. (Judges are impacted by whether or not their favorite football team won the night before.) Memorize a few and you’ll impress your wife’s-bosses-cousin in no time. Freakonomics did it better. But the fundamental problem is that this book doesn’t say anything that hasn’t been beaten to death before.
Essentially decisions come down to judgments and judgments can be skewed through bias and noise. Noise = randomness except it’s a lot harder to charge six figure consulting fees when you say “oh jeez, there’s just a lot of randomness all up in here.” Much sexier to call it a “noise audit” and point to your crappy book as a guide. People may not be great predictors but we sure are predictably gullible.
Then this book plays a bad game of telephone where the authors summarize research they did not do, and at times seems like it might’ve been sourced from a Reddit comment section, in an effort to make their publishers and publicist happy by hitting a page count.
Read Phillip Tetlocks “Expert Political Judgement” and “Superforcasting” for better and more in depth research on the core topics covered here. Invisible Women does a good job with some of this. Honestly this book felt like a psych sophomore five solo cups of thunder punch deep trying to explain their thoughts on cognitive bias. Don’t waste your time.
I really wanted to like this. I liked Nudge which has Cass as an author, I generally liked Thinking Fast and Slow, and I want someone who’s not Nate Silver explain signal to noise ratios to help me curate better information in my life. But this book isn’t it. This book is literally noise. Worthless noise in an already noisy world.
Someone like Kahneman, a founder of behavioral economics, you would think would have interesting new research and considered takes on how to cut through the amount of chatter out there in the world. It’s an important problem. But it seems like behavioral economics has stalled out into finding goofy and minor errors in our cognitive biases. Hey look! two people came to different answers when asked to mentally calculate an abstract concept. Look at how I can create methodologically dubious and unreplicatible studies that confuse people into making decisions against their best interests. Am I a behavioral economist yet?
I’m so sick of people writing shitty books to promote themselves as “thought leaders” and charge more for their consulting. I expected a better book out of these authors but found myself extremely disappointed in the shallowness of the ideas and writing. It’s a bad regurgitation of ideas that has been done better in other places.
If you like feeling cocktail-party smart without actually having to put in the effort to be smart, you will probably like this. It’s full of pithy blurbs. (Judges are impacted by whether or not their favorite football team won the night before.) Memorize a few and you’ll impress your wife’s-bosses-cousin in no time. Freakonomics did it better. But the fundamental problem is that this book doesn’t say anything that hasn’t been beaten to death before.
Essentially decisions come down to judgments and judgments can be skewed through bias and noise. Noise = randomness except it’s a lot harder to charge six figure consulting fees when you say “oh jeez, there’s just a lot of randomness all up in here.” Much sexier to call it a “noise audit” and point to your crappy book as a guide. People may not be great predictors but we sure are predictably gullible.
Then this book plays a bad game of telephone where the authors summarize research they did not do, and at times seems like it might’ve been sourced from a Reddit comment section, in an effort to make their publishers and publicist happy by hitting a page count.
Read Phillip Tetlocks “Expert Political Judgement” and “Superforcasting” for better and more in depth research on the core topics covered here. Invisible Women does a good job with some of this. Honestly this book felt like a psych sophomore five solo cups of thunder punch deep trying to explain their thoughts on cognitive bias. Don’t waste your time.
March 3, 2023
Daniel Kahneman is a Psychologist and an economist. He is one of the prominent personalities in the field of behavioral economics. With all his experience and expertise in these two fields, Kahneman is trying to find out what noise is and how it alters our judgements through this book.
What I learned from this book
1) What is the difference between bias and noise
We are so focused on removing bias that we commonly forget about the noise that also needs equal emphasis. Noise is something that is created when we behave in a different manner in similar situations. It is created at random.
Bias is an inclination or prejudice for or against one person or group, especially in a way considered to be unfair. Bias is a subjective way of thinking that originates from an individual's own perception or points of view.
“A general property of noise is that we can recognise and measure it, when knowing nothing about the target or bias.”
“Noise is rarely recognised. Bias is always the star of the show.”
“System noise is inconsistency and inconsistency damages the credibility of the system.”
2) How can noise affect Doctors?
Noise in the Medical profession is an important topic as it can directly and drastically affect human lives. I think this is a portion of the book that all the Doctors must read as it will help every Doctor a lot in improving the quality of life of their patients.
Doctors are not patricians reincarnated from heaven who always do the right things. But I have been confounded by the overconfidence and ego of some experienced Doctors who consider avoiding noise as a peccadillo. I think it is a philistine nature to believe that noise is always negligible to consider. A similar action can sometimes hurt a human being's life. So it is the people in the Medical profession who should be most circumspective about noise.
“When physicians are under time pressure, they are apparently more inclined to choose a quick-fix solution, despite its serious downsides. ”
"Another illustration of the role of fatigue among clinicians is the lower rate of appropriate handwashing during the end of hospital shifts. (Handwashing turns out to be noisy, too.) ”
"A study of nearly seven hundred thousand primary care visits, for instance, showed that physicians are significantly more likely to prescribe opioids at the end of a long day."
“Other studies showed that, toward the end of the day, physicians are more likely to prescribe antibiotics and less likely to prescribe flu shots.”
“Doctors are more likely to order cancer screening early in the morning than late in the afternoon.”
"In the words of one observer, "the reliance on the patient's subjective symptoms, the clinician's interpretation of the symptoms, and the absence of objective measure (such as a blood test) implant the seeds of diagnostic unreliability of psychiatric disorders." In this sense, psychiatry may prove especially resistant to attempts at noise reduction."
“A standard practice to reduce noise in Medical profession is to advise patients to get the second opinion.”
3) How can noise affect Judges?
A single judgment has a propensity to positively and negatively change a person's life. So the Judges should also be cautious about the noise when giving judgements in each and every case. Judges should be neither munificent nor heedless while giving judgements to mendacious convicts. Noise has the perturbing propensity to push the judgement both ways.
“Bad weather is associated with improved memory; judicial sentences tend to be more severe when it is hot outside; and stock market performance is affected by sunshine.”
"A study of thousands of juvenile court decisions found that when the local football team loses a game on the weekend, the judges make harsher decisions on the Monday (and, to a lesser extent, for the rest of the week). Black defendants disproportionately bear the brunt of that increased harshness. A different study looked at 1.5 million judicial decisions over three decades and similarly found that judges are more severe on days that follow a loss by the local city's football team than they are on days that follow a win.”
"But noisy systems do not make multiple judgments of the same case. They make noisy judgments of different cases. If one insurance policy is overpriced and another is underpriced, pricing may on average look right, but the insurance company has made two costly errors. If two felons who both should be sentenced to five years in prison receive sentences of three years and seven years, justice has not, on average, been done. In noisy systems, errors do not cancel out. They add up."
4) When can we say that a judge and their judgement is good?
The author enumerates the qualities needed for a good judge and good judgement. Noise and bias are two of the main components that they should be careful about, damaging their judgments. Those who are able to reduce both these can give a better judgement.
"Judgements are both less noisy and less biased when those who make them are well trained, are more intelligent and have the right cognitive style. In other words good judgements depend on what you know, how well you think and how you think. Good judges tend to be experienced and smart but they tend to be actively open mended and willing to learn from the new information."
5) Does clouds makes nerds look good?
This might be an interesting question to ask. The answer to it is yes. So if your academic aspects are strong in your CV, getting an interview during bad weather will be actually beneficial for you.
"Uri Simonson showed that college admissions officers pay more attention to the academic attributes of candidates on cloudier days and are more sensitive to nonacademic attributes on sunnier days. The title of the article in which he reported these findings is memorable enough: "Clouds Make Nerds Look Good."
6) The secret behind calorie labels
The first impression is the best impression. The initial positive or negative response of a customer towards a product will play a significant role in whether they purchase it. If the calorie label is placed on the left, we will be reading it first as we read from left to right. The opposite is also true that is calorie label placed on the right will affect purchasing decisions for the Hebrew readers as they read from right to left.
"Consumers are more affected by the calorie label if they are placed on the left of the food label rather than the right. When calories are on the left, consumers read this information first and think a lot of calories or not so many calories before they see the item."
7) What is gambler’s fallacy?
The gambler's fallacy is a false belief that a random event is less or more likely to happen based on the results from a previous event.
"After a streak, or a series of decisions that go in the same direction, they are more likely to decide in the opposite direction than would be strictly justified. As a result, errors (and unfairness) are inevitable. Asylum judges in the United States, for instance, are 19% less likely to grant asylum to an applicant when the previous two cases were approved. A person might be approved for a loan if the previous two applications were denied, but the same person might have been rejected if the previous two applications had been granted. This behavior reflects a cognitive bias known as the gambler's fallacy: we tend to underestimate the likelihood that streaks will occur by chance."
My favourite three lines from this book
“There is at least one source of occasion noise that we have all noticed: mood.”
“A good mood makes us more likely to accept our first impressions as true without challenging them.”
"From the perspective of noise reduction, a singular decision is a recurrent decision that happens only once. Whether you make a decision only once or a hundred times, your goal should be to make it in a way that reduces both bias and noise. And practices that reduce error should be just as effective in your one-of-a-kind decisions as in your repeated ones."
What could have been better?
This book has 454 pages, and the author solely focuses on a single topic - Noise. I believe that someone of the stature of Daniel Kahneman should have done dedicated research on this topic to write this book. Sadly, the author has not done proper research and uses the data from the research done by other people here. This would have been a perfect five-star book for me if he could substantiate his findings much more solidly using his own research.
The main problem associated with all the non-fiction books is also seen here, which is the repetition of some of the ideas multiple times at various parts of this book. Proper editing would have easily solved the problem.
Rating
4/5 This book will help you to know more about noise and will help you to prevent it, especially if you are someone new to this concept.
June 3, 2021
I’ve only ever come across the idea of noise in the context of information theory – something I thought this book would have made more mention of, but it didn’t, really. The idea being that the transmission of any signal is likely to involve noise (entropy being the one truly inevitable law of the universe �� more than taxes, on par with death) and so figuring out ways to reduce noise ultimately depends on how important the signal is. At the start of the Life of Brian there is a perfect example. Jesus is giving his sermon on the mount, and he says, “Blessed are the peacemakers” – but some people further back hear “blessed are the cheese makers.” Unsurprisingly, this causes an argument over why cheese makers might be singled out for a special blessing. Perhaps one of the reasons information theory isn’t mentioned here is that a major means of controlling noise in information theory is by redundancy, you put some form of redundancy into the signal and it lets you know if you are getting signal, or noise. I don’t think redundancy is something the authors of this book are interested in increasing, perhaps even the opposite.
I’m not sure what to make of this book – so, I’m going to give you my view of what it is about and then some concerns I have about what it is about. One of the things I like about this book is that if you don’t have time to read the whole thing, you can flick to the last chapter and get all the major ideas of the book fired at you in quick succession. All signal, no noise, and very little redundancy, if you like...
This is, at least in part, a treatise against judgement. You only need to make a judgement when you do not know for certain – no one says, “I judge the fire to be hot” or “to the best of my judgement, the sea is salty.” Judgement implies a kind of weighing of variables, and so black and white ideas don't require judgement. All the same, we tend to be far too confident in our ‘judgement calls’, and if the methods discussed in this book have one thing in common, it is to make us pause before we pass judgement. A range of methods are discussed to achieve this, but almost all of them involve delaying our reaching a judgement, that is, ways of ensuring you do look before you lead, or kicking people off your team who you know are going to prejudge or ensuring you organise the inputs to your judgements so that you create lots of diversity.
I think the last point is the one that I will be most likely to take away from this book. This idea is designed to correct the problems with the ‘wisdom of crowds’. That is, that if you get lots of people making a judgement, and you average their judgements, you are likely to get closer to the real value than most, if not all, of the individual judgements themselves. So, if you have to decide how many jellybeans there are in a jar, and you can chose the average of all guesses as your guess, then always do that. In real life you probably don’t get to do too many ‘how many jellybeans are there?’ type quizzes. But just about any judgement call is improved by having an increased diversity of opinions added to the mix.
The only problem is that it is remarkably easy to mess this up. When my second child was born she was breech and so everything was panic in the delivery room – that is, right up until the obstetrician walked into the room. I really have never known a man to have such a presence. Everyone deferred to him. The point made here is that such a person cancels the wisdom of crowds, because people are much less likely to ‘put their own two cents worth in’ if they think they will contradict the wise one. As the authors here say, finding ways to ensure people provide their own judgements independent of everyone else is a major step towards making better judgements. Which is part of the reason why I write these reviews without reading other people's reviews.
And this also goes for choosing inputs that you will use to make your judgements. You should make sure these are diverse too. So that if you are thinking of employing someone, one of your inputs might be their intelligence. But that might mean that your next input shouldn’t really be what university did they go to, because those two inputs are probably quite strongly correlated. You want to avoid asking the same question in five different ways and then thinking you have covered all the bases.
Okay – so, what has any of this got to do with noise? Well, their argument is that the nature of judgements is that they are always noisy. And whenever this is tested, judgements prove to be much more noisy than we would guess. This means that one judge might give a drug addict a suspended sentence while another might give someone else under the exact same charge 20 years. Now, normally when we read that that has happened the next thing said is, ‘can you guess which of them was black?’ But this book isn’t about bias, it is about noise. Bias shifts all results in a predictable direction – noise is unpredictable. We can think that noise is fairer than bias – except, you probably wouldn’t think that if you were sitting next to the guy that got the suspended sentence.
The takeaway here is ‘wherever there is judgement there is noise, and more of it than you imagine.” There are some lovely metaphors too – like the idea of treating reducing noise as a hygiene task – since, you are unlikely to actually know the consequence of decisions you didn’t make, but reducing noisy decisions is likely to make better judgements anyway. And so, like washing your hands, you can never know the infection you might have ended up with if you hadn’t washed them – but that is, after all, the point of washing your hands in the first place.
The problem I have with this book is that it says that one way to reduce noise is to reduce the situations where judgements are necessary. And we can do this by building algorithms. Anyone who has read 'Weapons of Math Destruction' will be feeling a little uncomfortable right about now. And they even mention that book here. Their point is that the identified problems with algorithms are more likely to be them being impacted by bias, rather than their ability to reduce noise. And even if the bias is unconscious, well, we need to find ways to tackle that bias. Getting rid of the algorithms isn’t the solution, the solution is in getting rid of the bias.
Which all sounds well and good. But the problem seems to me to be much more fundamental than that. Signal and noise are interesting because we can assume they are obvious and absolutes. There is a ‘true’ signal and noise gets in the way of that signal. But it isn’t clear that a lot of things in life do fit the definition of a true signal. Or better (and perhaps clearer) is the example they give at the start of the book. Throwing darts at a dart board. We all want all of our darts to go into the bullseye. If they did, universal happiness will be achieved and the kingdom of heaven shall reign for a thousand years – something like that, anyway. Except, who decides where the bullseye is? Are we all really aiming towards identical bullseyes? It may well be true that algorithms can reduce noise in our decision making, but I struggled to get over one of the examples they gave – mandatory sentencing. Sure, there is no noise, but I certainly don’t feel comfortable with that particular bullseye. The authors don’t either, by the way, but I don’t know that I was convinced by their brushing of this aside.
I guess I am worried that this book is written by social psychologists, and I’m not sure all of the questions to be answered here fit all that neatly into their field. In fact, a lot of what was said here, particularly in the bits on algorithms, made me think of sociology and the need to take intersectional perspectives into account – and to question if judgements are no less unfair just because they reinforce and reproduce the accepted prejudices of our society.
I know this sounds like I’m arguing about bias, rather than noise – but it wasn’t clear to me how one can reduce the scatter of noise without some sort of an organising principle – or how that organising principle could be something other than some fundamental form of prejudice.
I might sound much more certain about all this than I feel. Clearly, noisy decisions are anything but good decisions. I guess one of my problems here is that I want to agree and disagree at the same time. They give this lovely example where the algorithm says there is a 70% chance Jane will go to the movies tonight, but you know Jane has a broken leg – and so, you assume there is actually no chance she will go to the movies. The problem is that while this is perhaps true, given she has a broken leg – we tend to exaggerate other exceptions to the rule as if they were the same as a broken leg. I do this all the time, by the way. I can’t believe anyone could vote for the Coalition Parties here in Australia – them having been for as long as I can remember a mixture of corruption, unspeakable nastiness and incompetence – but they are re-elected time and again. The thing is that what I take to be broken legs, others don’t even notice as problems.
The authors give an example of where reducing noise might prove too costly – that is, in teachers marking essays – it being obviously too expensive to mark every essay twice. Except, that isn’t how teachers go about reducing noise in their marking. They create rubrics and they cross-mark selections of essays and then they compare results. And they add comments to essays and give the students the right to appeal if they think the mark was too low and they can justify that on the basis of the rubric.
And that is where I’m going to leave this. I do want to see judgements, because I don’t want to be ruled by algorithms that I can’t see or understand or follow the internal decisions of. But I want those decisions and judgements that are made to be based on standards that are clearly documented, with reasons given for those standards, that I can challenge or vote against or shout about when I get seriously pissed off with them. Yeah, judgements are messy and noisy, they are trying to solve complex problems, and so noise really is inevitable, but it would be nice and better to make them less noisy, I can agree with that – but while a lot of the suggestions in this book go a long way to reducing noise, some of the suggestions make me feel very, very uncomfortable.
I’m not sure what to make of this book – so, I’m going to give you my view of what it is about and then some concerns I have about what it is about. One of the things I like about this book is that if you don’t have time to read the whole thing, you can flick to the last chapter and get all the major ideas of the book fired at you in quick succession. All signal, no noise, and very little redundancy, if you like...
This is, at least in part, a treatise against judgement. You only need to make a judgement when you do not know for certain – no one says, “I judge the fire to be hot” or “to the best of my judgement, the sea is salty.” Judgement implies a kind of weighing of variables, and so black and white ideas don't require judgement. All the same, we tend to be far too confident in our ‘judgement calls’, and if the methods discussed in this book have one thing in common, it is to make us pause before we pass judgement. A range of methods are discussed to achieve this, but almost all of them involve delaying our reaching a judgement, that is, ways of ensuring you do look before you lead, or kicking people off your team who you know are going to prejudge or ensuring you organise the inputs to your judgements so that you create lots of diversity.
I think the last point is the one that I will be most likely to take away from this book. This idea is designed to correct the problems with the ‘wisdom of crowds’. That is, that if you get lots of people making a judgement, and you average their judgements, you are likely to get closer to the real value than most, if not all, of the individual judgements themselves. So, if you have to decide how many jellybeans there are in a jar, and you can chose the average of all guesses as your guess, then always do that. In real life you probably don’t get to do too many ‘how many jellybeans are there?’ type quizzes. But just about any judgement call is improved by having an increased diversity of opinions added to the mix.
The only problem is that it is remarkably easy to mess this up. When my second child was born she was breech and so everything was panic in the delivery room – that is, right up until the obstetrician walked into the room. I really have never known a man to have such a presence. Everyone deferred to him. The point made here is that such a person cancels the wisdom of crowds, because people are much less likely to ‘put their own two cents worth in’ if they think they will contradict the wise one. As the authors here say, finding ways to ensure people provide their own judgements independent of everyone else is a major step towards making better judgements. Which is part of the reason why I write these reviews without reading other people's reviews.
And this also goes for choosing inputs that you will use to make your judgements. You should make sure these are diverse too. So that if you are thinking of employing someone, one of your inputs might be their intelligence. But that might mean that your next input shouldn’t really be what university did they go to, because those two inputs are probably quite strongly correlated. You want to avoid asking the same question in five different ways and then thinking you have covered all the bases.
Okay – so, what has any of this got to do with noise? Well, their argument is that the nature of judgements is that they are always noisy. And whenever this is tested, judgements prove to be much more noisy than we would guess. This means that one judge might give a drug addict a suspended sentence while another might give someone else under the exact same charge 20 years. Now, normally when we read that that has happened the next thing said is, ‘can you guess which of them was black?’ But this book isn’t about bias, it is about noise. Bias shifts all results in a predictable direction – noise is unpredictable. We can think that noise is fairer than bias – except, you probably wouldn’t think that if you were sitting next to the guy that got the suspended sentence.
The takeaway here is ‘wherever there is judgement there is noise, and more of it than you imagine.” There are some lovely metaphors too – like the idea of treating reducing noise as a hygiene task – since, you are unlikely to actually know the consequence of decisions you didn’t make, but reducing noisy decisions is likely to make better judgements anyway. And so, like washing your hands, you can never know the infection you might have ended up with if you hadn’t washed them – but that is, after all, the point of washing your hands in the first place.
The problem I have with this book is that it says that one way to reduce noise is to reduce the situations where judgements are necessary. And we can do this by building algorithms. Anyone who has read 'Weapons of Math Destruction' will be feeling a little uncomfortable right about now. And they even mention that book here. Their point is that the identified problems with algorithms are more likely to be them being impacted by bias, rather than their ability to reduce noise. And even if the bias is unconscious, well, we need to find ways to tackle that bias. Getting rid of the algorithms isn’t the solution, the solution is in getting rid of the bias.
Which all sounds well and good. But the problem seems to me to be much more fundamental than that. Signal and noise are interesting because we can assume they are obvious and absolutes. There is a ‘true’ signal and noise gets in the way of that signal. But it isn’t clear that a lot of things in life do fit the definition of a true signal. Or better (and perhaps clearer) is the example they give at the start of the book. Throwing darts at a dart board. We all want all of our darts to go into the bullseye. If they did, universal happiness will be achieved and the kingdom of heaven shall reign for a thousand years – something like that, anyway. Except, who decides where the bullseye is? Are we all really aiming towards identical bullseyes? It may well be true that algorithms can reduce noise in our decision making, but I struggled to get over one of the examples they gave – mandatory sentencing. Sure, there is no noise, but I certainly don’t feel comfortable with that particular bullseye. The authors don’t either, by the way, but I don’t know that I was convinced by their brushing of this aside.
I guess I am worried that this book is written by social psychologists, and I’m not sure all of the questions to be answered here fit all that neatly into their field. In fact, a lot of what was said here, particularly in the bits on algorithms, made me think of sociology and the need to take intersectional perspectives into account – and to question if judgements are no less unfair just because they reinforce and reproduce the accepted prejudices of our society.
I know this sounds like I’m arguing about bias, rather than noise – but it wasn’t clear to me how one can reduce the scatter of noise without some sort of an organising principle – or how that organising principle could be something other than some fundamental form of prejudice.
I might sound much more certain about all this than I feel. Clearly, noisy decisions are anything but good decisions. I guess one of my problems here is that I want to agree and disagree at the same time. They give this lovely example where the algorithm says there is a 70% chance Jane will go to the movies tonight, but you know Jane has a broken leg – and so, you assume there is actually no chance she will go to the movies. The problem is that while this is perhaps true, given she has a broken leg – we tend to exaggerate other exceptions to the rule as if they were the same as a broken leg. I do this all the time, by the way. I can’t believe anyone could vote for the Coalition Parties here in Australia – them having been for as long as I can remember a mixture of corruption, unspeakable nastiness and incompetence – but they are re-elected time and again. The thing is that what I take to be broken legs, others don’t even notice as problems.
The authors give an example of where reducing noise might prove too costly – that is, in teachers marking essays – it being obviously too expensive to mark every essay twice. Except, that isn’t how teachers go about reducing noise in their marking. They create rubrics and they cross-mark selections of essays and then they compare results. And they add comments to essays and give the students the right to appeal if they think the mark was too low and they can justify that on the basis of the rubric.
And that is where I’m going to leave this. I do want to see judgements, because I don’t want to be ruled by algorithms that I can’t see or understand or follow the internal decisions of. But I want those decisions and judgements that are made to be based on standards that are clearly documented, with reasons given for those standards, that I can challenge or vote against or shout about when I get seriously pissed off with them. Yeah, judgements are messy and noisy, they are trying to solve complex problems, and so noise really is inevitable, but it would be nice and better to make them less noisy, I can agree with that – but while a lot of the suggestions in this book go a long way to reducing noise, some of the suggestions make me feel very, very uncomfortable.
April 14, 2021
The sheer variety of ways judgment can be clouded is mind-boggling. The more closely we examine judgments, the more noise turns up as a factor. In Noise, an A-list team of celebrity psych stars, Daniel Kahneman, Olivier Sibony and Cass Sunstein pull together their confrères and evidence from the usual innumerable studies to delineate how bad it really is.
Noise, at least in psychology, is “unwanted variability”. In practical terms, that means even the most focused person might be swayed by unnoticed noise. Noise can be the home team losing the night before, lunch coming up in half an hour, miserable weather, a toothache – pretty much anything that has nothing to do with the issue at hand. This is all in addition to personal prejudices and the framework of bureaucratic rules that are always in play, restricting the range of possible decisions, and misdirecting them where they should not be going.
All kinds of studies show that trial judges are inconsistent when not totally wrong. The authors say two judges viewing the same evidence in the same case will come to two completely different decisions. So will the same judge given the same case on two different occasions. Sentencing is all over the place, which has led to enforced sentencing guidelines that often make things worse. It has also led to judge-shopping, as the decision patterns of judges builds up over the years. This is not based on evidence or argument, but in which way the judge’s decisions can be erroneous. Think political parties, religion, and stubborn pig-headedness.
The same goes for mere mortals, like supervisors. They all believe they do a creditable job, but the stats show the direct opposite. Even simple linear models do a far better job in every case. Not just sometimes – every time, according to Noise. Even randomly generated models do a far more accurate job of judging people correctly than people do. Artificial intelligence algorithms can also add a little more accuracy, though surprisingly, not significantly so. But people on their own perform miserably.
Still, no one, but no one, would trust a simple model to make a decision on their future; they feel better having personally tried with another human, regardless of the facts. It immediately reminded me of Lake Wobegon, where all the kids are above average. Doesn’t work like that. In the authors’ words, “Models of reality, of a judge or randomly generated models all perform better than nuanced, intuitive, insightful and experienced humans.” To which I would add: anyone who claims they can accurately size up a person on meeting them, can’t.
Errors occur far more frequently than people realize, because everyone trusts their own judgment foremost, and far too often, the judgment of others (their lawyers, doctors and managers, for example).
The worst example of this occurs in job interviews and performance appraisals. Everyone knows the single worst way to make a hire is through a personal, unstructured interview. Yet managers still insist on interviews, and so do candidates, thinking they can master the battle and win the job if they can simply deal with someone in person. Both are totally wrong, yet nonetheless, they both persist. Job interviews have become a nightmare for candidates, going back multiple times for essentially no good reason, as the more people interview them, the more inaccurate their ultimate decision will be.
As for quarterly, semi-annual and annual performance appraisals, those who have to work with the results know they are usually totally worthless. Managers burdened with multiple reports grind them out against a deadline, having little or nothing to do with an individual’s performance. Most everyone is “satisfactory”, especially when managers are required to rate them on a scale. No decisions can validly be taken from these exercises in frustration, but they are taken anyway. And while essentially no one in any organization likes or ever looks forward to the whole process, the noise persists, clouding futures.
Scales themselves are useless, as the authors show in examples such as for astronauts. A bell-curve distribution would show one or two excellent performers, one or two total failures, and most in the middle. But there are no total failures among astronauts. The yearslong training requires and ensures it. So grading on a scale against a bell-curve can be just more noise.
For the open-minded, Noise provides details, tips and tricks to leverage. For example, deliberation, the vaunted value of teams, actually increases the noise around a decision. The mere fact that team members discuss their reasoning before they make a decision increases the noise for everyone participating. The key to making teams work, ironically, is for everyone to do their own research in isolation, and once they have all come to a decision, they can then compare with others on the team.
They call this independent work “decision hygiene”. It cuts down noise in general, but no one can know what specifically, or by how much. The authors liken it to handwashing- no one knows what germs were there to kill. All they know is that handwashing kills germs, and that you can never get rid of all of them.
The authors show that noise occurs in almost any shape or form. The quality of the paper used for a business plan, and the font it is presented in, can tip the success or failure of a proposal in the hands of potential investors.
Another interesting noise source is called whitecoat syndrome. This is noise some people generate going to see a doctor, nurse or lab technician. Their blood pressure rises in anticipation, sometimes causing an erroneous diagnosis.
Things like prejudice are not so much noise as bias. When assessing decisions that go wrong, noise is the standard deviation of errors, while bias is the mean itself. The book is a thorough attempt to make a science of noise and errors in judgment.
Bias is a likely driver of noise. But the book is all about separating the two. It shows that biases, such as “planning fallacy, loss aversion, overconfidence, the endowment effect, status quo bias, excessive discounting of the future, and various biases against various categories of people” are all factors in erroneous decisions. But despite all this, sheer noise outweighs bias heavily.
They use Gaussian mean squared errors to demonstrate the effect of both bias and noise, with noise the clear winner, and dramatically so. Squaring the errors makes them visually arresting, But they still need to be stopped - somehow.
It transpires that errors do not cancel each other out, either. Instead, they add up, taking decisionmakers farther away from the right decision. And with the book piling on a seemingly infinite selection of noise factors and sources, it’s a wonder Man has made it even this far.
Speaking of erroneous judgments, it is difficult to decide what kind of book Noise is. It is steeped in psychology, but it is not a groundbreaking new discipline. People and firms have been actively trying to filter out noise since forever (the better ones, anyway). Nor is it a psych textbook, really, though there are exercises the reader can use right while poring over it. I think it is closer to a handbook of what to be aware of: forewarned is forearmed sort of thing. Though clearly, mere knowledge of the situation is far from enough to counteract it. The book includes how-tos like implementing an audit to identify and isolate noise, so the book definitely has practical applications. Handbook it is, then.
This noise thing is ego-deflating for all humans, who run their lives continually making decisions, not only on facts, but predictive judgments as well (Predictions provide an “illusion of validity”). That we are not equipped to pull this off successfully – at all – should cause a total rethink of where we go from here. Noise is pernicious. Trusting models looms heavily over us all.
David Wineberg
June 12, 2021
This book is criticized primarily for 2 reasons:
* first of all, because it doesn't bring such striking mental models as System 1 and System 2 (from "Thinking ...")
* because some have expected that Kahneman will ride on SJW wave and write a book on "racial/social bias", full of political correctness, etc. (hint: it didn't happen)
Unjustly, because surprisingly this is a really good book. Seriously, if you think about this - is it even possible to write a good book, without avoiding excessive repetitions, on such a specific (and narrow) topic? I know I wouldn't be able to this, but well - I'm certainly no Daniel Kahneman.
So, if you're really interested in the topic, you will really enjoy the book. What were my favorite parts?
* the one about rating content on public marketplaces
* why noise is sometimes useful (rules vs standards)
* noise in medical examinations
* what has bigger negative implications: noise or bias?
I've enjoyed reading "Noise" more than I expected. Don't be fooled - it is SPECIFIC and it is focused on a very narrow topic (which you may find annoying), but in my case - I loved it.
* first of all, because it doesn't bring such striking mental models as System 1 and System 2 (from "Thinking ...")
* because some have expected that Kahneman will ride on SJW wave and write a book on "racial/social bias", full of political correctness, etc. (hint: it didn't happen)
Unjustly, because surprisingly this is a really good book. Seriously, if you think about this - is it even possible to write a good book, without avoiding excessive repetitions, on such a specific (and narrow) topic? I know I wouldn't be able to this, but well - I'm certainly no Daniel Kahneman.
So, if you're really interested in the topic, you will really enjoy the book. What were my favorite parts?
* the one about rating content on public marketplaces
* why noise is sometimes useful (rules vs standards)
* noise in medical examinations
* what has bigger negative implications: noise or bias?
I've enjoyed reading "Noise" more than I expected. Don't be fooled - it is SPECIFIC and it is focused on a very narrow topic (which you may find annoying), but in my case - I loved it.
June 15, 2021
This reads like a required lecture. Three stars, they are all for the information I picked when it held my attention.
May 25, 2021
Although interesting, the authors clearly show their bias in “Noise”. It was a disappointing book after reading the incredibly interesting and applicable “Thinking Fast and Slow”. My main concern is that they imply causation where statisticians would not claim more than correlation. Implying causation is sloppy and a bad statistical practice.
They are greatly concerned with the randomness of individual impacts to people from judgments, insurance companies, and job interviews. Although they state that overall impacts to individuals on average are fair (i.e. unbiased), the impact to each individual may have a large variance (be wildly different from the average), which is caused by systemic noise. They assume this is inherently bad and that we should systematically reduce this noise by implementing more algorithms and rules into all sorts of private and public institutions. My concern is: who determines the rules for those algorithms? (Unbiased statisticians or policy makers?)
Essentially, this is a long discussion about statistical models that have larger variances than the authors would like (and larger variances than the general population would expect). They use the variability in human judgement to illustrate that humans are flawed. Their solution is to use more models, but they also point out that models can be flawed in similar ways. It’s a conflicting book of “don’t trust anyone’s judgement” and “don’t trust models”, but “do trust that individuals are likely having unfair things happen to them, even if there isn’t any bias in the system.” It was unfortunate that they didn’t include a discussion of what individuals can do to improve themselves (reducing their own biases and noise) rather than waiting for big institutions to reduce that noise for them.
They are greatly concerned with the randomness of individual impacts to people from judgments, insurance companies, and job interviews. Although they state that overall impacts to individuals on average are fair (i.e. unbiased), the impact to each individual may have a large variance (be wildly different from the average), which is caused by systemic noise. They assume this is inherently bad and that we should systematically reduce this noise by implementing more algorithms and rules into all sorts of private and public institutions. My concern is: who determines the rules for those algorithms? (Unbiased statisticians or policy makers?)
Essentially, this is a long discussion about statistical models that have larger variances than the authors would like (and larger variances than the general population would expect). They use the variability in human judgement to illustrate that humans are flawed. Their solution is to use more models, but they also point out that models can be flawed in similar ways. It’s a conflicting book of “don’t trust anyone’s judgement” and “don’t trust models”, but “do trust that individuals are likely having unfair things happen to them, even if there isn’t any bias in the system.” It was unfortunate that they didn’t include a discussion of what individuals can do to improve themselves (reducing their own biases and noise) rather than waiting for big institutions to reduce that noise for them.
January 29, 2022
Since reading Thinking, Fast and Slow by Daniel Kahneman a long time ago, I thought, "wow! another book by Kahneman about psychology -- cool!"
Well, Kahneman is only one of the three authors. This book is about as boring as it could be. I highly recommend this book for people who love turgid statistics and humorless, pedantic style.
I didn't actually finish this book. I fell asleep too often to finish it.
Well, Kahneman is only one of the three authors. This book is about as boring as it could be. I highly recommend this book for people who love turgid statistics and humorless, pedantic style.
I didn't actually finish this book. I fell asleep too often to finish it.
May 20, 2021
This book was a disappointment ... I thought that it's gonna be a scientific book . But it seemed written by Malcolm Gladwell ... Its a punch of stories nothing more
January 3, 2023
"Noise" is positioned as another ground-breaking dual lens to look at the world, fresh from Kahneman's desk. However, it is not as radical as TFAS since it only extends the central argument and is not half as well written. This one could have been an additional chapter in an updated edition.
The central thesis is that while we worry about bias a lot (the basis of which was explored in TFAS), noise is the silent enemy - affecting our ability to think clearly and make sound decisions. Historically we have been blind to bias of various sorts, and we need to continue our pursuit of eliminating bias, but noise can be just as iniquitous and needs to be addressed as well. Noise can be insidious, sneaking into our decision-making processes in subtle and often unconscious ways.
The book ultimately extends the central thesis of TFAS - our decisions are flawed: We make wrong decisions, yes, but even more so, we make random decisions. If asked to replay our decisions in a DBRCT of some sort, we'll most likely create a spread of decisions that'll make us question our identity. That's the reality.
Kahneman leaves us worse off than where we were at the end of TFAS. However, he does offer some techniques for improving our focus and concentration, as well as ways to reduce the number of distractions and interruptions that can cause noise in our decisions. But in the end, the two books together show us up as even more flawed beings than ever.
The central thesis is that while we worry about bias a lot (the basis of which was explored in TFAS), noise is the silent enemy - affecting our ability to think clearly and make sound decisions. Historically we have been blind to bias of various sorts, and we need to continue our pursuit of eliminating bias, but noise can be just as iniquitous and needs to be addressed as well. Noise can be insidious, sneaking into our decision-making processes in subtle and often unconscious ways.
The book ultimately extends the central thesis of TFAS - our decisions are flawed: We make wrong decisions, yes, but even more so, we make random decisions. If asked to replay our decisions in a DBRCT of some sort, we'll most likely create a spread of decisions that'll make us question our identity. That's the reality.
Kahneman leaves us worse off than where we were at the end of TFAS. However, he does offer some techniques for improving our focus and concentration, as well as ways to reduce the number of distractions and interruptions that can cause noise in our decisions. But in the end, the two books together show us up as even more flawed beings than ever.
October 22, 2021
I really have no idea who the intended audience was for this book: the authors really, really dumb it down, to the point of explaining what variance is over several pages of prose. We did not all fail high school.
At the same time, they bring into the discussion some serious tools you won’t even meet until you get to graduate school in statistics, like the “percentage concordant,” which is not some type of supersonic airplane, but a rank correlation type of measure, and even provide a mini-table to move you from percentage concordant (PC) to correlation. The table, by the way, is bogus in the absence of context, as percentage concordant is a construct that I’m willing to bet relies heavily on assumptions that go unmentioned here.
The chapters end with summaries, which was OK for Thinking Fast and Slow, but a bit of an insult when the subject matter is so plain.
The style is pompous and paternalistic.
System A and System B are parachuted in, but (i) they’re barely explained (ii) that’s a theory to explain bias rather than noise (and invite a celebrity author to the proceedings)
Most annoyingly, terribly little ground is covered in this weighty tome. Gun to my head, I could probably get it all down to one page. Let me try:
1. Noise is just as bad as bias in terms of messing up your results
2. A good way to measure how bad your results are is the mean square error
3. Composition of Mean Square Error:
• Mean square error is made up of Bias and Noise
• Noise is made up of Level Noise and Pattern Noise
• Pattern Noise is made up of Stable Pattern Noise and Occasion Noise
• Level Noise is the kind of noise that comes from the fact that some judges are harsh and some are lenient, so two guys who did the same crime could get very different punishment.
• Pattern Noise is the kind of noise that comes from the fact that a judge may have a daughter, making him less harsh on young women that remind him of his daughter. He could be a harsh judge who is less harsh on young women who remind him of his daughter; or he could be a lenient judge who is extra lenient on young women who remind him of his daughter.
• Occasion Noise is the kind of noise that comes from the fact that judges are harsher right before lunch. Same judge, same crime, same perpetrator, different outcome, because it was a different occasion
4. If you ask people to measure something independently from one another, the more the merrier; but if they talk to each other first, then they will amplify errors for a variety of reasons that lead to groupthink
5. Machines beat people when it comes to cutting noise
6. In the quest to limit noise, people can fight back by sticking to simple rules
7. We humans like to build stories after the fact to explain what happened; they’re usually bogus: statistical explanations beat causal explanations
8. Bias can be the source of noise: inconsistency in bias is noise
9. Noise can arise when you’re told to rank things on a scale; to cut noise, it’s better to go ordinal than cardinal
10. To improve judgements you need (i) better judges (ii) a decision process that aggregates in a way that maintains independence among the judges (iii) guidelines (iv) relative rather than absolute judgements
11. There is a place for intuition: it’s got to be brought in at the very end, after all the mechanical work has finished
12. There actually is a place for noise: when people are bound to game the system
Read something else!
At the same time, they bring into the discussion some serious tools you won’t even meet until you get to graduate school in statistics, like the “percentage concordant,” which is not some type of supersonic airplane, but a rank correlation type of measure, and even provide a mini-table to move you from percentage concordant (PC) to correlation. The table, by the way, is bogus in the absence of context, as percentage concordant is a construct that I’m willing to bet relies heavily on assumptions that go unmentioned here.
The chapters end with summaries, which was OK for Thinking Fast and Slow, but a bit of an insult when the subject matter is so plain.
The style is pompous and paternalistic.
System A and System B are parachuted in, but (i) they’re barely explained (ii) that’s a theory to explain bias rather than noise (and invite a celebrity author to the proceedings)
Most annoyingly, terribly little ground is covered in this weighty tome. Gun to my head, I could probably get it all down to one page. Let me try:
1. Noise is just as bad as bias in terms of messing up your results
2. A good way to measure how bad your results are is the mean square error
3. Composition of Mean Square Error:
• Mean square error is made up of Bias and Noise
• Noise is made up of Level Noise and Pattern Noise
• Pattern Noise is made up of Stable Pattern Noise and Occasion Noise
• Level Noise is the kind of noise that comes from the fact that some judges are harsh and some are lenient, so two guys who did the same crime could get very different punishment.
• Pattern Noise is the kind of noise that comes from the fact that a judge may have a daughter, making him less harsh on young women that remind him of his daughter. He could be a harsh judge who is less harsh on young women who remind him of his daughter; or he could be a lenient judge who is extra lenient on young women who remind him of his daughter.
• Occasion Noise is the kind of noise that comes from the fact that judges are harsher right before lunch. Same judge, same crime, same perpetrator, different outcome, because it was a different occasion
4. If you ask people to measure something independently from one another, the more the merrier; but if they talk to each other first, then they will amplify errors for a variety of reasons that lead to groupthink
5. Machines beat people when it comes to cutting noise
6. In the quest to limit noise, people can fight back by sticking to simple rules
7. We humans like to build stories after the fact to explain what happened; they’re usually bogus: statistical explanations beat causal explanations
8. Bias can be the source of noise: inconsistency in bias is noise
9. Noise can arise when you’re told to rank things on a scale; to cut noise, it’s better to go ordinal than cardinal
10. To improve judgements you need (i) better judges (ii) a decision process that aggregates in a way that maintains independence among the judges (iii) guidelines (iv) relative rather than absolute judgements
11. There is a place for intuition: it’s got to be brought in at the very end, after all the mechanical work has finished
12. There actually is a place for noise: when people are bound to game the system
Read something else!
May 30, 2021
Нова фундаментална книга от нобеловия лауреат Даниел Канеман, известен с предишната си творба "Мисленето", разглеждаща систематичните грешки (bias).
В "Шум" Канеман разкрива един неразпознаван проблем при вземане на решения. Определя шумът като разликите (вариации) в резултатите на различни хора в една и съща ситуация. Например няколко лекари може да поставят различни диагнози на един и същ пациент. Поне няколко от тях грешат. Различни съдии дават различни по тежест присъди на сходни престъпления. Разликата в субективните мнения води до шум.
Шумът и систематичните грешки водят до грешни решения, но по различен начин. Книгата обяснява (в някои случаи прекалено сложно) разликите между шум и систематични грешки. Дава препоръки как да направим "одит на шума" и как да намалим влиянието му.
Със сигурност новата книга на Канеман заслужава вашето внимание. Някои от концепциите са напълно нови и няма да ги срещнете на друго място. Силно препоръчвам книгата на всеки, който трябва да прави сложни и повтарящи се решения.
В "Шум" Канеман разкрива един неразпознаван проблем при вземане на решения. Определя шумът като разликите (вариации) в резултатите на различни хора в една и съща ситуация. Например няколко лекари може да поставят различни диагнози на един и същ пациент. Поне няколко от тях грешат. Различни съдии дават различни по тежест присъди на сходни престъпления. Разликата в субективните мнения води до шум.
Шумът и систематичните грешки водят до грешни решения, но по различен начин. Книгата обяснява (в някои случаи прекалено сложно) разликите между шум и систематични грешки. Дава препоръки как да направим "одит на шума" и как да намалим влиянието му.
Със сигурност новата книга на Канеман заслужава вашето внимание. Някои от концепциите са напълно нови и няма да ги срещнете на друго място. Силно препоръчвам книгата на всеки, който трябва да прави сложни и повтарящи се решения.
March 31, 2021
Noise is bad no matter where in life we find it. In their new book Daniel Kahneman, Olivier Sibony, and Cass Sunstein say there is too much of it in our judgments and explain how noise arises and what might be done about it.
“Judgment” is not “thinking”.The book defines “judgment” as “a form of measurement in which the instrument is a human mind.” Judgments may be less than optimal due to bias, which is systematic deviation from optimal, e.g.the group’s predictions are ALWAYS overly optimistic, or noise, which is a more random scatter. The main topic of the book is “system noise”, which is “unwanted variability in judgments that should ideally be identical.” (I should get the same jail sentence no matter which judge hears my case.) System noise has two main components. There is level noise ( A particular judge is lenient in granting bail.) and pattern noise. Pattern noise also has two components: stable pattern noise, (Such as a tendency to give women lighter jail terms), and occasion noise ( I just had a run-in with my boss)..
The book discusses each of these types of noise and their psychological aspects, drawing on earlier work such as Sunstein’s “nudge” and Kahneman’s “System 1 and 2” thinking. Readers who are not somewhat familiar with this work might find a quick google search helpful. There is also some discussion of the statistics involved that I suspect will be cryptic to most people who do not already know a bit about statistics. If so, you can certainly ignore the math.
So once you know sources of noise in judgment, what do you do about it? The authors describe some remedies, such as a “noise audit” or a “decision observer” to help remove bias from judgments in groups or a judicious use of rules or standards.
There is a lot of good and thought-provoking insight in Noise, principles that everyone will recognize once they are pointed out but that interfere with good judgment unless we identify and address them. The authors show how to do this with extensive descriptions of judgments in a number of fields, like selecting new hires, setting bail or sentences in criminal cases, and medical decisions. As a result, this is rather a long book, and these descriptions can be skimmed if you are very focused on task, but they are interesting.
The applications described in this book are primarily decisions made by multiple people, whether they be judges setting bail or group recommendations on whether a company should acquire another company. It does not focus much on decisions people might make in their personal lives, but the principles certainly seem applicable there as well. I am sure the authors would recommend that I not review this book just before lunch and after an argument with my spouse!
Insightful analysis of why we make bad judgments
“Judgment” is not “thinking”.The book defines “judgment” as “a form of measurement in which the instrument is a human mind.” Judgments may be less than optimal due to bias, which is systematic deviation from optimal, e.g.the group’s predictions are ALWAYS overly optimistic, or noise, which is a more random scatter. The main topic of the book is “system noise”, which is “unwanted variability in judgments that should ideally be identical.” (I should get the same jail sentence no matter which judge hears my case.) System noise has two main components. There is level noise ( A particular judge is lenient in granting bail.) and pattern noise. Pattern noise also has two components: stable pattern noise, (Such as a tendency to give women lighter jail terms), and occasion noise ( I just had a run-in with my boss)..
The book discusses each of these types of noise and their psychological aspects, drawing on earlier work such as Sunstein’s “nudge” and Kahneman’s “System 1 and 2” thinking. Readers who are not somewhat familiar with this work might find a quick google search helpful. There is also some discussion of the statistics involved that I suspect will be cryptic to most people who do not already know a bit about statistics. If so, you can certainly ignore the math.
So once you know sources of noise in judgment, what do you do about it? The authors describe some remedies, such as a “noise audit” or a “decision observer” to help remove bias from judgments in groups or a judicious use of rules or standards.
There is a lot of good and thought-provoking insight in Noise, principles that everyone will recognize once they are pointed out but that interfere with good judgment unless we identify and address them. The authors show how to do this with extensive descriptions of judgments in a number of fields, like selecting new hires, setting bail or sentences in criminal cases, and medical decisions. As a result, this is rather a long book, and these descriptions can be skimmed if you are very focused on task, but they are interesting.
The applications described in this book are primarily decisions made by multiple people, whether they be judges setting bail or group recommendations on whether a company should acquire another company. It does not focus much on decisions people might make in their personal lives, but the principles certainly seem applicable there as well. I am sure the authors would recommend that I not review this book just before lunch and after an argument with my spouse!
Insightful analysis of why we make bad judgments
May 25, 2021
I have been very interested in the work of the psychologist and economist Daniel Kahneman since around 2000 where I came across some of the ideas around over-confidence bias on an Executive MBA at Insead, and this was only cemented with his Nobel Prize win (with Amos Tversky) in 2002.
I spent a lot of time over the years researching their work including their 2000 publication “Choices, Values and Frames” and applying the ideas (both Prospect Theory and the various heuristics and biases they identified in the field of Behavioural Economics( in some of my own professional work (as well as speaking on it to fellow actuaries and to others in insurance).
Kahneman of course came to much wider prominence in 2011 with his publication “Thinking, Fast and Slow” which made it easier to talk about his ideas and their applications to professional work in general and to my own fields of insurance and actuarial work more specifically as more people had familiarity with them.
See here for example for an article I joint authored which discussed both that book and Taleb’s “Anti- Fragility” (https://web.actuaries.ie/sites/defaul...)
So of course I was immediately going to read any new book co-authored by Kahneman – and I was interested to see that one of the other three authors is the co-writer of “Nudge” (a book I have not read but whose ideas I am familiar with, not least for the way that the UK government established a “Nudge Unit” in 2010 to apply some of its ideas).
What were my first impressions of this book:
My first – and negative reaction - was that it was a lot simpler than I was used to from this author and not in a good way.
“Choices, Values and Frames” was effectively a compilation of academic papers (of course academic papers from the “Dismal Science” of Economics where it seems possible to logically argue both X and not-X from the same data and where empirical evidence is gathered from experiments which are both artificial and with ridiculously small sample sizes – normally a group of 20 graduate students earning $10 to take part constitutes major experimental evidence). And while “Thinking Fast and Slow” attempted to be for mass-consumption it was still dense with ideas. This book by contrast seems to be light on ideas (particularly early on)– explaining what to me seemed sometimes very simple ideas in rather excruciating detail. It felt like the first 80 pages in particular would have almost have been taken as a page of initial definitions in the 2000 work.
The second – and by contrast positive reaction – was that the book was much more addressed to my own field. In fact the very first example given in the book is actually from underwriting premium judgements and claims case assessment in an unnamed insurance company which given I run a global team of mathematicians whose key functions are assisting underwriters with the provision of tools to assist in setting premiums, and in carrying out calculations to complement case setting seemed rather relevant.
Whereas much of the earlier work was drawn on social science type examples and often on the aforemention artificial experiments, this book draws heavily both in its empirical data and its recommendations on areas of professional judgement. In fact in this interview – which serves as a good introduction to the book https://www.mckinsey.com/business-fun...# – one of the authors actually describes Noise as “the unwanted variability in professional judgments”. Most of the repeated examples – the insurance example is one of a number of one-offs - are drawn from judicial work (particularly sentencing), forensic science medical work, and HR areas (both recruitment and performance assessment) – the former much more mappable to my own work and the latter of course relevant to almost all workers.
So what is the book about? Well I am sure there will be copious articles over time on the book, but to use the McKinsey article again and Kahneman’s explanation – it is all about distinguishing between bias and noise. “bias is the average error in judgments. If you look at many judgments, and errors in those judgments all follow in the same direction, that is bias. By contrast, noise is the variability of error. If you look at many judgments, and the errors in those judgments follow in many different directions, that is noise”.
A key assertion of the book is that noise has been largely overlooked – particularly in professional areas as professionals are not prepared to admit quite how noisy their views actually are. They claim and aim to show from data that in terms of accuracy in post-fact verifiable judgements – noise is a much greater source of error than bias; and also make the point that with non-verifiable judgements, bias is anyway not a concept that can be easily investigated anyway. Note on the latter though they perhaps miss the point that whereas individual judgements may not be verifiable, aggregate ones perhaps are (insurance premiums – which they correctly say cannot be verified on an individual basis – being a case of an area that can be on an aggregate basis).
In terms of noise they later split noise into level noise (taking the examples of judge sentencing – the difference in average sentences between lenient and draconian judges) and pattern noise (variability of judges responses to particular cases). They later split pattern noise into stable pattern noise (this could be seen as for example an otherwise lenient judge who is systematically harsh on knife crime, or a harsh judge who is sympathetic to young offenders) and occasion noise.
And this goes some way towards explaining why they define noise as so critical as I think many people would more naturally group both level noise and even pattern noise with bias.
Some interesting areas:
- Although later showing that stable pattern noise is perhaps one of the biggest contributors to error, an earlier chapter gives lots of example of how occasion noise is perhaps the most embarrassing part for professionals to admit: caused either (hence its name) by judgements being changed by extraneous circumstance (weather, time of day, results of local sports teams have all been shown to influence judicial sentences) or by internal inconsistency (forensic scientists – including fingerprint experts – will commonly reach a different conclusion if given the same case months later, as well clinicians).
- They spend a lot of time arguing for the greater use of models – and often simple models – in professional judgement. One example is that “The Model of You beats You” – for many professional judgement a model simply built off a weighted average of past judgements by an individual outperforms the future judgements of the individual
- It is clear that in both judicial areas and medical areas there is an almost complete resistance to the use of models (despite examples like the AGPAR score which have been used successfully for years) as being de-humanising, over-simplified, arbitrary etc. From my own professional viewpoint this can seem odd – underwriting professionals are more than happy to have models to complement and ground their assessments
- There is an interesting discussion on model sophistication which effectively argues for one of two ends of a continuum. Either simple equal weights (or frugal simply weighted) models which aggregate a number of assessments known to be partly predictive OR a complex machine learning model (when large data sets are available including factors not traditionally assessed). In the absence of the latter, they find the former comes very close to (or sometimes outperforms) a regression/GLM type model (partly due to data-mining/over-fitting and partly due to often faulty professional judgement used to interpret the findings of the modelling).
Although not really in scope - while acknowledging the risk of models having inbuilt biases (either implicit and hidden ones due to proxy variables or ingrained ones due to reinforcing the past biases data sets used to build them) they also make the point that these biases are typically much more pernicious in individual judgements
- Some of their recommendations include Noise Audits, Decision Observers and then Decision Hygiene which includes such things as: using judgements rooted in statistical and external evidence (how many M&A deals of this type actually succeed) rather than causal/narrative judgement (constructing a story to fit the case); taking judgements into several, distinct steps which ideally are carried out independently - so that the halo effect of the first judgement does not outweigh the rest (for interviewing this for example could mean an evidence based competency assessment rather than an informal chat which is likely biased by early impressions); ideally use multiple professions and aggregate their judgements; try to use relative judgements – for example for performance reviews use pair-wise comparisons and scales which set out explicitly what is required for each level of achievement
I am tempted to finalise my review by adding a mark based on :
- comparing it to a number of other books
- evaluating it against an objective rating scale
- aggregating the views of other readers
- running it through my book rating algorithm
Instead as its a thought provoking book and one I am already starting to apply to my day to day work - I will settle for 5* - but I would urge persisting through the first few chapters.
I spent a lot of time over the years researching their work including their 2000 publication “Choices, Values and Frames” and applying the ideas (both Prospect Theory and the various heuristics and biases they identified in the field of Behavioural Economics( in some of my own professional work (as well as speaking on it to fellow actuaries and to others in insurance).
Kahneman of course came to much wider prominence in 2011 with his publication “Thinking, Fast and Slow” which made it easier to talk about his ideas and their applications to professional work in general and to my own fields of insurance and actuarial work more specifically as more people had familiarity with them.
See here for example for an article I joint authored which discussed both that book and Taleb’s “Anti- Fragility” (https://web.actuaries.ie/sites/defaul...)
So of course I was immediately going to read any new book co-authored by Kahneman – and I was interested to see that one of the other three authors is the co-writer of “Nudge” (a book I have not read but whose ideas I am familiar with, not least for the way that the UK government established a “Nudge Unit” in 2010 to apply some of its ideas).
What were my first impressions of this book:
My first – and negative reaction - was that it was a lot simpler than I was used to from this author and not in a good way.
“Choices, Values and Frames” was effectively a compilation of academic papers (of course academic papers from the “Dismal Science” of Economics where it seems possible to logically argue both X and not-X from the same data and where empirical evidence is gathered from experiments which are both artificial and with ridiculously small sample sizes – normally a group of 20 graduate students earning $10 to take part constitutes major experimental evidence). And while “Thinking Fast and Slow” attempted to be for mass-consumption it was still dense with ideas. This book by contrast seems to be light on ideas (particularly early on)– explaining what to me seemed sometimes very simple ideas in rather excruciating detail. It felt like the first 80 pages in particular would have almost have been taken as a page of initial definitions in the 2000 work.
The second – and by contrast positive reaction – was that the book was much more addressed to my own field. In fact the very first example given in the book is actually from underwriting premium judgements and claims case assessment in an unnamed insurance company which given I run a global team of mathematicians whose key functions are assisting underwriters with the provision of tools to assist in setting premiums, and in carrying out calculations to complement case setting seemed rather relevant.
Whereas much of the earlier work was drawn on social science type examples and often on the aforemention artificial experiments, this book draws heavily both in its empirical data and its recommendations on areas of professional judgement. In fact in this interview – which serves as a good introduction to the book https://www.mckinsey.com/business-fun...# – one of the authors actually describes Noise as “the unwanted variability in professional judgments”. Most of the repeated examples – the insurance example is one of a number of one-offs - are drawn from judicial work (particularly sentencing), forensic science medical work, and HR areas (both recruitment and performance assessment) – the former much more mappable to my own work and the latter of course relevant to almost all workers.
So what is the book about? Well I am sure there will be copious articles over time on the book, but to use the McKinsey article again and Kahneman’s explanation – it is all about distinguishing between bias and noise. “bias is the average error in judgments. If you look at many judgments, and errors in those judgments all follow in the same direction, that is bias. By contrast, noise is the variability of error. If you look at many judgments, and the errors in those judgments follow in many different directions, that is noise”.
A key assertion of the book is that noise has been largely overlooked – particularly in professional areas as professionals are not prepared to admit quite how noisy their views actually are. They claim and aim to show from data that in terms of accuracy in post-fact verifiable judgements – noise is a much greater source of error than bias; and also make the point that with non-verifiable judgements, bias is anyway not a concept that can be easily investigated anyway. Note on the latter though they perhaps miss the point that whereas individual judgements may not be verifiable, aggregate ones perhaps are (insurance premiums – which they correctly say cannot be verified on an individual basis – being a case of an area that can be on an aggregate basis).
In terms of noise they later split noise into level noise (taking the examples of judge sentencing – the difference in average sentences between lenient and draconian judges) and pattern noise (variability of judges responses to particular cases). They later split pattern noise into stable pattern noise (this could be seen as for example an otherwise lenient judge who is systematically harsh on knife crime, or a harsh judge who is sympathetic to young offenders) and occasion noise.
And this goes some way towards explaining why they define noise as so critical as I think many people would more naturally group both level noise and even pattern noise with bias.
Some interesting areas:
- Although later showing that stable pattern noise is perhaps one of the biggest contributors to error, an earlier chapter gives lots of example of how occasion noise is perhaps the most embarrassing part for professionals to admit: caused either (hence its name) by judgements being changed by extraneous circumstance (weather, time of day, results of local sports teams have all been shown to influence judicial sentences) or by internal inconsistency (forensic scientists – including fingerprint experts – will commonly reach a different conclusion if given the same case months later, as well clinicians).
- They spend a lot of time arguing for the greater use of models – and often simple models – in professional judgement. One example is that “The Model of You beats You” – for many professional judgement a model simply built off a weighted average of past judgements by an individual outperforms the future judgements of the individual
- It is clear that in both judicial areas and medical areas there is an almost complete resistance to the use of models (despite examples like the AGPAR score which have been used successfully for years) as being de-humanising, over-simplified, arbitrary etc. From my own professional viewpoint this can seem odd – underwriting professionals are more than happy to have models to complement and ground their assessments
- There is an interesting discussion on model sophistication which effectively argues for one of two ends of a continuum. Either simple equal weights (or frugal simply weighted) models which aggregate a number of assessments known to be partly predictive OR a complex machine learning model (when large data sets are available including factors not traditionally assessed). In the absence of the latter, they find the former comes very close to (or sometimes outperforms) a regression/GLM type model (partly due to data-mining/over-fitting and partly due to often faulty professional judgement used to interpret the findings of the modelling).
Although not really in scope - while acknowledging the risk of models having inbuilt biases (either implicit and hidden ones due to proxy variables or ingrained ones due to reinforcing the past biases data sets used to build them) they also make the point that these biases are typically much more pernicious in individual judgements
- Some of their recommendations include Noise Audits, Decision Observers and then Decision Hygiene which includes such things as: using judgements rooted in statistical and external evidence (how many M&A deals of this type actually succeed) rather than causal/narrative judgement (constructing a story to fit the case); taking judgements into several, distinct steps which ideally are carried out independently - so that the halo effect of the first judgement does not outweigh the rest (for interviewing this for example could mean an evidence based competency assessment rather than an informal chat which is likely biased by early impressions); ideally use multiple professions and aggregate their judgements; try to use relative judgements – for example for performance reviews use pair-wise comparisons and scales which set out explicitly what is required for each level of achievement
I am tempted to finalise my review by adding a mark based on :
- comparing it to a number of other books
- evaluating it against an objective rating scale
- aggregating the views of other readers
- running it through my book rating algorithm
Instead as its a thought provoking book and one I am already starting to apply to my day to day work - I will settle for 5* - but I would urge persisting through the first few chapters.
June 2, 2021
This is one of the worst popular press social sciences books I've ever read, and I've read many. It gets a lot wrong about what we know regarding decision-making and basic statistics. While it's true that algorithms are highly useful when applied appropriately, this book massively overstates the case in their favor while neglecting important counterpoints, among other serious problems. Kahneman's "Thinking, Fast and Slow" remains one of my favorite books on research in psychology and this is an extremely disappointing step down. I recommend skipping "Noise" entirely and looking elsewhere if you're interested in the subjects it touches on. Want a book on statistics? Try "Naked Statistics" by Charles Wheelan. Interested in decision-making? "Thinking, Fast and Slow" is still good, but skip the chapter of priming (it doesn't hold up). "Thinking in Bets" is decent as well. Want critical thinking with a healthy dose of data interpretation? "Calling Bullshit: The Art of Skepticism in a Data-Driven World" is pretty good. Just, whatever you do, skip "Noise" and spend your time elsewhere.
July 2, 2021
A boring, amateur, and often misleading take on concepts that decision scientists, machine learning engineers, and statisticians have known and systematically studied for decades with far more rigor than these authors do. The authors are out of their depth here and contribute nothing new to the conversation. (For example, their "error equation," which they call the "intellectual foundation" of their book, is a basic concept taught in high school statistics.) Their folk, popular-press series of books have grown tired and at this point seem mostly like money-making machines for them in which they restate the obvious and botch the nuances and state of the art. Remind me again why we're listening to a psychology professor, a business professor, and a law professor's amateur thoughts on statistics?
December 24, 2021
The famous psychologist, economist and cognitive scientist Daniel Kahneman, known in Russia for the bestseller "Think slowly... Decide quickly" in collaboration with behavioral economics specialist Cass Sunstein and psychologist specializing in strategic decision-making Olivier Siboni wrote a scientific treatise on noise that has nothing to do with decibels.
"Noise. The imperfection of human judgments" is interesting, although not the easiest to understand non-fiction. the main theme of which is the distortion of human judgments, which are expressed in serious discrepancies in the assessment of certain things: people, events, situations, This phenomenon is called "noise" in the language of sociology and, along with "displacement", negatively affects the performance of any system of human interactions.
All three authors are world-renowned scientists, practitioners no less than theorists, each has its own area of specialization. They gathered together to jointly give an idea of a problem that does not become less serious because it is not noticed.
Минимизировать ошибки в суждениях
Если двух преступников, которым полагается по пять лет тюрьмы, приговаривают к трем и пяти годам соответственно, справедливость в целом не торжествует. В шумных системах ошибки не компенсируют друг друга. Они накапливаются.
Знаменитый психолог, экономист и когнитивист Даниэль Канеман, известный в России по бестселлеру "Думай медленно... Решай быстро" в соавторстве со специалистом по поведенческой экономике Кассом Санстейном и психологом, специализирующимся на принятии стратегических решений Оливье Сибони написали ученый трактат о шуме, не имеющем отношения к децибелам.
"Шум. Несовершенство человеческих суждений" Интересный, хотя не самый простой для понимания нон-фикшн. основная тема которого искажения человеческих суждений, которые выражаются в серьезных расхождениях в оценке тех или иных вещей: людей, событий, ситуаций, Это явление на языке социологии называется "шумом" и, наряду со "смещение��" отрицательно влияет на работоспособность любой системы человеческих взаимодействий.
Все трое авторов ученые с мировым именем, практики не в меньшей степени, чем теоретики, у каждого своя область специализации. Вместе они собрались, чтобы объединенными усилиями дать представление о проблеме, которая не становится менее серьезной оттого, что ее не замечают.
Для того, чтобы дать понятие о шуме в социально-психологическом смысле, книга использует следующий пример: три команды стрелков, три мишени. выстрелы первой команды кучно попали в десятку. У второй попадания тоже локализованы достаточно близка, но справа от центра; что до третьей, то ее результаты можно описать поговоркой "в белый свет как в копеечку".
Первый случай иллюстрирует вариант идеального решения, к которому надо стремиться. Второй относится к понятию "смещения", когда к решению по какому бы то ни было вопросу подходят изначально предвзято более мягкое наказание за одно и то же нарушение для вежливого красивого образованного представителя титульной нации, суровое - для гастарбайтера; предпочтение при устройстве на работу по гендерному, национальному, расовому, возрастному и т.д. признакам.
Смещение признанное зло, которое дискредитирует систему и вредит ее работоспособности, со смещением в порядке вещей бороться и привлекать к нему внимание во всех возможных случаях. В отличие о�� него шум незаметен как протечка в подвале, однако вреда от него не меньше.
Когда один судья назначает в качестве наказания за аналогичное преступление условный срок, а другой десять лет; когда одному выплачивают травму по страховке двести тысяч, а второму за ту же травму тридцать пять; когда один врач говорит о необходимости операции, а другой назначает медикаментозное лечение - это вопросы жизни и смерти, и утешать себя мыслью о бренности всего сущего не очень получается. Не говоря о том, что совершенно непродуктивно.
Шум результат не злого умысла или изначальной предубежденности, как в случае со смещением, но несовершенства человеческой природы, уверенности, что собственная картина мира является таковой и для окружающих, недостаточной информированности. Книга подробно разбирает виды ловушек мышления, мешающих выносить объективное решение и предлагает комплекс мер, , названных "гигиеной принятия решений", которые помогут максимально эффективно избегать их.
Постоянная ревизия текущих обстоятельств, прецедентность, хотя не прямая совещательность - обсуждение часто ведет к тому, что менее компетентный, но более авторитетный участник навязывает мнение. И главное - четкая алгоритмизованность, поэтапное взвешивание всех аспектов вопроса, когда интуиция может вступать лишь на финальной стадии. Инструкция, которая должна стать не рекомендательным даже, но обязательным руководством
Аудиокнига, в которой Игорь Князев практически не задействует актерскую игру, и предстает в непривычной для себя ипостаси лектора, со спокойной, размеренной, настойчивой подачей материала, раскрывает его исполнительский талант с непривычной стороны.
"Noise. The imperfection of human judgments" is interesting, although not the easiest to understand non-fiction. the main theme of which is the distortion of human judgments, which are expressed in serious discrepancies in the assessment of certain things: people, events, situations, This phenomenon is called "noise" in the language of sociology and, along with "displacement", negatively affects the performance of any system of human interactions.
All three authors are world-renowned scientists, practitioners no less than theorists, each has its own area of specialization. They gathered together to jointly give an idea of a problem that does not become less serious because it is not noticed.
Минимизировать ошибки в суждениях
Если двух преступников, которым полагается по пять лет тюрьмы, приговаривают к трем и пяти годам соответственно, справедливость в целом не торжествует. В шумных системах ошибки не компенсируют друг друга. Они накапливаются.
Знаменитый психолог, экономист и когнитивист Даниэль Канеман, известный в России по бестселлеру "Думай медленно... Решай быстро" в соавторстве со специалистом по поведенческой экономике Кассом Санстейном и психологом, специализирующимся на принятии стратегических решений Оливье Сибони написали ученый трактат о шуме, не имеющем отношения к децибелам.
"Шум. Несовершенство человеческих суждений" Интересный, хотя не самый простой для понимания нон-фикшн. основная тема которого искажения человеческих суждений, которые выражаются в серьезных расхождениях в оценке тех или иных вещей: людей, событий, ситуаций, Это явление на языке социологии называется "шумом" и, наряду со "смещение��" отрицательно влияет на работоспособность любой системы человеческих взаимодействий.
Все трое авторов ученые с мировым именем, практики не в меньшей степени, чем теоретики, у каждого своя область специализации. Вместе они собрались, чтобы объединенными усилиями дать представление о проблеме, которая не становится менее серьезной оттого, что ее не замечают.
Для того, чтобы дать понятие о шуме в социально-психологическом смысле, книга использует следующий пример: три команды стрелков, три мишени. выстрелы первой команды кучно попали в десятку. У второй попадания тоже локализованы достаточно близка, но справа от центра; что до третьей, то ее результаты можно описать поговоркой "в белый свет как в копеечку".
Первый случай иллюстрирует вариант идеального решения, к которому надо стремиться. Второй относится к понятию "смещения", когда к решению по какому бы то ни было вопросу подходят изначально предвзято более мягкое наказание за одно и то же нарушение для вежливого красивого образованного представителя титульной нации, суровое - для гастарбайтера; предпочтение при устройстве на работу по гендерному, национальному, расовому, возрастному и т.д. признакам.
Смещение признанное зло, которое дискредитирует систему и вредит ее работоспособности, со смещением в порядке вещей бороться и привлекать к нему внимание во всех возможных случаях. В отличие о�� него шум незаметен как протечка в подвале, однако вреда от него не меньше.
Когда один судья назначает в качестве наказания за аналогичное преступление условный срок, а другой десять лет; когда одному выплачивают травму по страховке двести тысяч, а второму за ту же травму тридцать пять; когда один врач говорит о необходимости операции, а другой назначает медикаментозное лечение - это вопросы жизни и смерти, и утешать себя мыслью о бренности всего сущего не очень получается. Не говоря о том, что совершенно непродуктивно.
Шум результат не злого умысла или изначальной предубежденности, как в случае со смещением, но несовершенства человеческой природы, уверенности, что собственная картина мира является таковой и для окружающих, недостаточной информированности. Книга подробно разбирает виды ловушек мышления, мешающих выносить объективное решение и предлагает комплекс мер, , названных "гигиеной принятия решений", которые помогут максимально эффективно избегать их.
Постоянная ревизия текущих обстоятельств, прецедентность, хотя не прямая совещательность - обсуждение часто ведет к тому, что менее компетентный, но более авторитетный участник навязывает мнение. И главное - четкая алгоритмизованность, поэтапное взвешивание всех аспектов вопроса, когда интуиция может вступать лишь на финальной стадии. Инструкция, которая должна стать не рекомендательным даже, но обязательным руководством
Аудиокнига, в которой Игорь Князев практически не задействует актерскую игру, и предстает в непривычной для себя ипостаси лектора, со спокойной, размеренной, настойчивой подачей материала, раскрывает его исполнительский талант с непривычной стороны.
July 9, 2021
تدور أبحاث عالم النفس "دانيال كانمان" حول التفكير. يبحث في أنواعه وآلياته وزلاته.
"التفكير(1)" في عمله الشهير السابق، و"التقدير(2)" في عمله هذا الذي بين أيدينا يخضعان لخلل داخلي خفي. ويؤديان إلى نتائج خاطئة؛ متحيّزةً حينًا وفوضوية حينًا آخر.
عن "الفوضى(3)" تحديدًا يدور هذا الكتاب. والفوضى المقصودة هنا هي تباين الأحكام والتقديرات في المسألة الواحدة من أصحاب المهنة الواحدة. بل ومنه تباين أحكام الفرد الواحد نفسه في المسائل المتشابهة.
كيف يحدث هذا التباين ولماذا؟ وكيف يمكن قياسه رياضيًا؟ وما هي أنواعه؟ وما علاقته بالتفكير السريع والبطيء، والميل(4) (الهوى) الإدراكي والمعرفي؟ وكيف يمكن إصلاح هذا الخلل؟
هذه الفوضى التي يشير إليها "كانمان" في كتابه على أنواعٍ، ولها أسباب، ولها عواقب. بل ويمكن قياسها. وفوق ذلك، يمكن تلافيها، غير أنّ ذلك العلاج الوقائي لا يخلو من مشكلاته أيضًا. كتابه هذا عن كلّ ذلك.
تمتاز كتابات "كانمان" بأصالة الطرح، وعمق الفكرة التي يلتقطها، وذكاء الأمثلة التي يضربها. لكنّ القراءة له رحلة شاقّة. وأشبهها بالقراءة لـ"نسيم طالب"، مع فارق حسن ترتيب الأفكار لصالح "كانمان". كلاهما ستخسر إن مررت سريعًا أو تركت شيئًا مما يكتب. وكلاهما ستقول: ما الذي يريده تحديدًا؟ وحين تدرك ما يريد، ستقول لماذا لم يكتبه في صفحة واحدة فقط؟ وستوقن بحكمة أنه لم يفعل! كلاهما ستفكّر مرارًا بترك كتابه وأنت تقرأه، ولن تفعل لثقتك الكبيرة بأهمية ما تقرأ.
أرى أن سبب هذا "العُسر" في كتب "كانمان" هو حرصه على طرح الموضوع من خلال التفكير العلني به، لا طرحه كنتائج جاهزة وتقريرات ناجزة. يجعله هذا حريصًا على العرض الشامل المحيط لفكرته، محاولاً اختصار ما كتب عنها وحولها من قبل قبل البناء عليه أو نقضه. كلّ ذلك في فصول قصيرة، تصير إلى التشعب وتحرص على التركيز.
كتاب كانمان هذا، مثل سابقه(5)، عميق فذّ ونافع. يفتح للعقل نافذة يطل منها على نفسه، فـ"يعقل" زمام رأيه، و"يبصر" منها ما حوله بصورةٍ أجلى وأدقّ.
----
1-thinking
2-judegment
3-Noise
4-bias
5-thinking fast & slow
"التفكير(1)" في عمله الشهير السابق، و"التقدير(2)" في عمله هذا الذي بين أيدينا يخضعان لخلل داخلي خفي. ويؤديان إلى نتائج خاطئة؛ متحيّزةً حينًا وفوضوية حينًا آخر.
عن "الفوضى(3)" تحديدًا يدور هذا الكتاب. والفوضى المقصودة هنا هي تباين الأحكام والتقديرات في المسألة الواحدة من أصحاب المهنة الواحدة. بل ومنه تباين أحكام الفرد الواحد نفسه في المسائل المتشابهة.
كيف يحدث هذا التباين ولماذا؟ وكيف يمكن قياسه رياضيًا؟ وما هي أنواعه؟ وما علاقته بالتفكير السريع والبطيء، والميل(4) (الهوى) الإدراكي والمعرفي؟ وكيف يمكن إصلاح هذا الخلل؟
هذه الفوضى التي يشير إليها "كانمان" في كتابه على أنواعٍ، ولها أسباب، ولها عواقب. بل ويمكن قياسها. وفوق ذلك، يمكن تلافيها، غير أنّ ذلك العلاج الوقائي لا يخلو من مشكلاته أيضًا. كتابه هذا عن كلّ ذلك.
تمتاز كتابات "كانمان" بأصالة الطرح، وعمق الفكرة التي يلتقطها، وذكاء الأمثلة التي يضربها. لكنّ القراءة له رحلة شاقّة. وأشبهها بالقراءة لـ"نسيم طالب"، مع فارق حسن ترتيب الأفكار لصالح "كانمان". كلاهما ستخسر إن مررت سريعًا أو تركت شيئًا مما يكتب. وكلاهما ستقول: ما الذي يريده تحديدًا؟ وحين تدرك ما يريد، ستقول لماذا لم يكتبه في صفحة واحدة فقط؟ وستوقن بحكمة أنه لم يفعل! كلاهما ستفكّر مرارًا بترك كتابه وأنت تقرأه، ولن تفعل لثقتك الكبيرة بأهمية ما تقرأ.
أرى أن سبب هذا "العُسر" في كتب "كانمان" هو حرصه على طرح الموضوع من خلال التفكير العلني به، لا طرحه كنتائج جاهزة وتقريرات ناجزة. يجعله هذا حريصًا على العرض الشامل المحيط لفكرته، محاولاً اختصار ما كتب عنها وحولها من قبل قبل البناء عليه أو نقضه. كلّ ذلك في فصول قصيرة، تصير إلى التشعب وتحرص على التركيز.
كتاب كانمان هذا، مثل سابقه(5)، عميق فذّ ونافع. يفتح للعقل نافذة يطل منها على نفسه، فـ"يعقل" زمام رأيه، و"يبصر" منها ما حوله بصورةٍ أجلى وأدقّ.
----
1-thinking
2-judegment
3-Noise
4-bias
5-thinking fast & slow
August 5, 2021
Noise: A Flaw in Human Judgment is the new book by Nobel Prize winner in Economics Daniel Kahneman. His previous book, Thinking, Fast and Slow, was an eye-opener to me.
Here is my understanding of the core concepts in Noise:
A judgement is a decision made when a definite, invariant result can not be obtained at hand. Answering a school math question is not a judgement. Weather forecasting is still a judgement but it is less likely so because of our improved understanding of weather science and the better measurement of weather data. Two factors contribute to the errors in human judgement: bias and noise. The chance variability of judgments is noise. Bias and noise can be both present but they are different. Bias has been studied and policies have been make in an attempt to reduce biases in many areas, but noise gets little attention.
"Such biased-based explanations are satisfying because the human mind craves causal explanations. Whenever something goes wrong, we look for a cause and often find it. In many cases, the cause will appear to be a bias. Bias has a kind of explanatory chrasma, which noise lacks. If we try to explain in hindsight why a particular decision is wrong, we will easily find bias and never find noise. Only a statistical view of the world enables us to see noise but that statistical view does not come naturally. We prefer causal stories. The absence of statistical thinking from our intuitions is one reason that noise receives so much less attention than bias does. Another reason is that professionals seldom see the need to confront noise in their own judgments and in those of their colleagues. "
The authors set out to study different kinds of noises (pattern noise and occasion noise) and the ways to reduce them. Standard vs rules, the pros and cons of using computer algorithms in noise reduction. A well-defined algorithm will reduce noise, but it may also reveal the underneath biases. The authors believe this does not mean we should get rid of algorithms. Instead, we should improve them. Sadly when it comes to judgments, we are forgiving to ourselves but expect 100% accuracy from machines.
Different strategies for different problems. Implementation is key. There is no recipe for all. The complexity of all kinds of possible human judgement errors makes me wonder if it is possible to reduce noise at all, and at what costs.
I find Noise is dry to read compare to Thinking, Fast and Slow.
Here is my understanding of the core concepts in Noise:
A judgement is a decision made when a definite, invariant result can not be obtained at hand. Answering a school math question is not a judgement. Weather forecasting is still a judgement but it is less likely so because of our improved understanding of weather science and the better measurement of weather data. Two factors contribute to the errors in human judgement: bias and noise. The chance variability of judgments is noise. Bias and noise can be both present but they are different. Bias has been studied and policies have been make in an attempt to reduce biases in many areas, but noise gets little attention.
"Such biased-based explanations are satisfying because the human mind craves causal explanations. Whenever something goes wrong, we look for a cause and often find it. In many cases, the cause will appear to be a bias. Bias has a kind of explanatory chrasma, which noise lacks. If we try to explain in hindsight why a particular decision is wrong, we will easily find bias and never find noise. Only a statistical view of the world enables us to see noise but that statistical view does not come naturally. We prefer causal stories. The absence of statistical thinking from our intuitions is one reason that noise receives so much less attention than bias does. Another reason is that professionals seldom see the need to confront noise in their own judgments and in those of their colleagues. "
The authors set out to study different kinds of noises (pattern noise and occasion noise) and the ways to reduce them. Standard vs rules, the pros and cons of using computer algorithms in noise reduction. A well-defined algorithm will reduce noise, but it may also reveal the underneath biases. The authors believe this does not mean we should get rid of algorithms. Instead, we should improve them. Sadly when it comes to judgments, we are forgiving to ourselves but expect 100% accuracy from machines.
Different strategies for different problems. Implementation is key. There is no recipe for all. The complexity of all kinds of possible human judgement errors makes me wonder if it is possible to reduce noise at all, and at what costs.
I find Noise is dry to read compare to Thinking, Fast and Slow.
July 28, 2021
I loved Thinking Fast and Slow, so I picked this book up without thinking about it. However, this was certainly not as well formulated, deep or interesting. Soemthing about the writing style felt disjointed. The thoughts were not cohesive or conclusive. For a book about Noise this felt rather noisy. It read more like a textbook or lecture than I wanted it to.
April 14, 2024
I had no business picking up this book. Sometimes, I’ll be in a bookstore and randomly decide “I don’t want escapism, I want to learn”. As an average 25 year old girl with little-to-no knowledge in social science or psychology, I was in over my head with this one. But I powered through, kept re-reading pages to actually understand what I just read. But hey, I now understand the concept of Noise, and tbh, trust my judgement more than ever. I’m not the target audience here, but whilst reading I often would relate a scenario to my own life and work, such as how brands will push a product, how my managers opinion maybe impacts my own, how I let millions of people on the internet impact my thought process. There was a whole bit on AI and algorithm which I found most interesting, as a Tiktoker whose job relies on an algorithm, I get it, it likes patterns and formulas and can’t hold a bias. Maybe that’s why I love tiktok more than other platforms. Anyways, here were some quotes that I underlined in the book so they would stay in my brain:
“Noise is rarely recognised. Bias is always the star of the show” “.
“A good mood makes us more likely to accept our first impressions as true, without challenging them”.
“It is more useful to pay attention to people who disagree with you than to pay attention to those who agree.”
“The denial of ignorance is all the more tempting when ignorance is vast”.
“You are not always the same person, and you are less consistent over time than you think. But somewhat reassuringly, you are more similar to yourself yesterday than you are to another person today”
“If people are not making their own judgements and are relying instead on what other people think, crowds might not be so wise after all”.
“Noise is rarely recognised. Bias is always the star of the show” “.
“A good mood makes us more likely to accept our first impressions as true, without challenging them”.
“It is more useful to pay attention to people who disagree with you than to pay attention to those who agree.”
“The denial of ignorance is all the more tempting when ignorance is vast”.
“You are not always the same person, and you are less consistent over time than you think. But somewhat reassuringly, you are more similar to yourself yesterday than you are to another person today”
“If people are not making their own judgements and are relying instead on what other people think, crowds might not be so wise after all”.
June 10, 2021
کتاب در مورد نقش نویز در قضاوتهای انسانی نوشته شده است و جاهایی که به صورت آشکار در تصمیم ها و قضاوتها دیده میشود، که به طور ویژه رو قضاوت قاضی ها و تشخیصهای پزشکی تایید کرده است. در ابتدا نویسنده به شرح تفاوت بایاس و نویز میپردازه که با مثال خوبی به زیبایی به تصویر کشیده شده است. نویز به نوعی پرکندگی تصادفی ق��اوتها در مورد یک مسئله است.
کتاب به چند بخش تقسیم شده است که در مورد شناسایی نویز، علل آن، اندازه گیری، چگونگی روی دادن آن، پیدا کردن در بخش های مختلف، مقایسه نویز در تصمیمهای انسانی، الگوریتمهای هوش مصنوعی و اینکه چگونه در تصمیم گیری هایمان دقت کنیم که نویز را کاهش دهیم، صحبت میکند. به کرات در این کتاب از تحقیقات مختلف و مثال برای تفهیم مسئله استفاده شده است که ارزش علمی کتاب را زیاد میکند.
اسم کانمن و موضوعش باعث شد در اسرع وقت کتاب رو بخونم و خیلی لذت بردم. کتابهای دیگه ای که خونده بودم از ادمهای سرشناس این حوزه تا بایاس جلو رفته بودند ولی خب کانمن و دو نویسنده دیگه موضوع نویز رو خیلی خوب مطرح و باز کردند که متوجه بشیم که نویز چقدر از بایاس هم جدی
تر هست و میتونه تصمیم گیری های ما رو تحت تاثیر قرار بده و شناسایی اش هم آنقدر ساده نیست
چیزهایی که خیلی از کتاب دوست داشتم بخش مربوط به مقایسه مدلهای پیچیده، ساده و هوش مصنوعی بود. اینکه چقدر مدلهای ساده در شرایط مختلف میتونند کاراتر از مدلهای پیچیده باشند ( در شرایطی که به خصوص تو اکادمیا میبینی مدلهای به کار برده شده روز به روز دارند پیچیده تر میشن) در مورد دسیژن هایجین و روشهای کاهش نویز در تصمیم گیری ها هم که نوشته بود دوست داشتم خیلی. تجربه این روزهای خودم که درگیر اتفاقی هستم که هم بایاس داره و هم نویز، خیلی کمک کرد در ارتباط برقرار کردن با کتاب، اگرچه هر وقت اسم کانمن میاد خب علاقه ام به خوندن /دیدن/شنیدن اون موضوع بیشتر میشه (بله میدونم اینم یک کاگنتیو بایاسه که دابلی هم تو یکی از کتابهاش به خوبی بهش اشاره کرده بود :D)
کتاب به چند بخش تقسیم شده است که در مورد شناسایی نویز، علل آن، اندازه گیری، چگونگی روی دادن آن، پیدا کردن در بخش های مختلف، مقایسه نویز در تصمیمهای انسانی، الگوریتمهای هوش مصنوعی و اینکه چگونه در تصمیم گیری هایمان دقت کنیم که نویز را کاهش دهیم، صحبت میکند. به کرات در این کتاب از تحقیقات مختلف و مثال برای تفهیم مسئله استفاده شده است که ارزش علمی کتاب را زیاد میکند.
اسم کانمن و موضوعش باعث شد در اسرع وقت کتاب رو بخونم و خیلی لذت بردم. کتابهای دیگه ای که خونده بودم از ادمهای سرشناس این حوزه تا بایاس جلو رفته بودند ولی خب کانمن و دو نویسنده دیگه موضوع نویز رو خیلی خوب مطرح و باز کردند که متوجه بشیم که نویز چقدر از بایاس هم جدی
تر هست و میتونه تصمیم گیری های ما رو تحت تاثیر قرار بده و شناسایی اش هم آنقدر ساده نیست
چیزهایی که خیلی از کتاب دوست داشتم بخش مربوط به مقایسه مدلهای پیچیده، ساده و هوش مصنوعی بود. اینکه چقدر مدلهای ساده در شرایط مختلف میتونند کاراتر از مدلهای پیچیده باشند ( در شرایطی که به خصوص تو اکادمیا میبینی مدلهای به کار برده شده روز به روز دارند پیچیده تر میشن) در مورد دسیژن هایجین و روشهای کاهش نویز در تصمیم گیری ها هم که نوشته بود دوست داشتم خیلی. تجربه این روزهای خودم که درگیر اتفاقی هستم که هم بایاس داره و هم نویز، خیلی کمک کرد در ارتباط برقرار کردن با کتاب، اگرچه هر وقت اسم کانمن میاد خب علاقه ام به خوندن /دیدن/شنیدن اون موضوع بیشتر میشه (بله میدونم اینم یک کاگنتیو بایاسه که دابلی هم تو یکی از کتابهاش به خوبی بهش اشاره کرده بود :D)
May 31, 2021
Doesn't add enough to "Thinking, Fast and Slow" to warrant another book. Feels like one of those books where the author gets paid for every time they use a specific word (in this case, "noise") and have said it to themselves so much it has become a cult-like world view. In this instance, noise refers to the variations in human decision making which Kahneman attributes to a mixture of situational and systemic cognitive biases that covers old territory in the behavioural psychology world. He makes a case for a utopian rules-based slash AI system to guide decision making in spheres including law, medicine and HR, which can work to a degree to eliminate noise and bias but can also mute gestalt and out-of-the-box thinking. Aside from the odd forcefully inserted and admittedly interesting behavioural psychology study The 5 page conclusion at the end is all that's worth your time here.
June 20, 2021
I should have known. Past experience has taught me that everything written or co-written by Cass Sunstein, be it papers or books, quickly turns out to be excruciatingly boring. This plodding, repetitive, and bland door-stopper is alas no exception.
August 2, 2021
พูดถึง judgement ในระดับองค์กรและเสนอแนวทางการแก้ไขแบบรวมๆ ไม่ค่อยได้อะไรเป็นชิ้นเป็นอัน (ซึ่งก็อาจจะเป็นความตั้งใจของคนเขียน) ไม่ได้ตื่นตาตื่นใจ ทึ่งกับความเข้มข้นของเนื้อหาเหมือนตอนอ่าน Thinking Fast and Slow ถ้าเทียบกันเล่มนี้นี่แผ่วไปมาก
February 3, 2023
The basic argument of the book can be boiled down to this: 1) leave judgment tasks to the smartest and most logical people (i.e., those who have the highest intelligence and pass the cognitive reflection test), 2) force those geniuses to follow guidelines and use the mediating assessments protocol (structured ratings of relevant factors) when making decisions, and 3) subject them to periodic noise audits (tests for unwanted variability in ratings and in subsequent evaluations).
Overall, I’d say the book is just okay. Mostly, it repeats in abbreviated form the contents of Thinking, Fast and Slow. That said, there are a couple of new things in Noise: A Flaw in Human Judgment, the most useful of which is the noise audit—having different people (e.g., court judges) evaluate the same case (e.g., offender vignette) to determine how much variability there is in evaluations (e.g., sentences). I also found the distinction between level noise (main effects of individual differences), pattern noise (interactions between case characteristics and individual differences), and occasion noise (the effects of situational factors, like mood, weather/temperature, and outcomes of local sporting events) to be helpful.
The discussion of the effects of herding, social pressure, and information cascades on judgments were also informative. For example, the material on forensic confirmation bias—incorrect identification of suspects per DNA or fingerprints—due to occasion noise and social pressure was startling.
Probably the most shocking thing in the whole book was the discussion of noise in sentencing. For example, two offenders with no prior criminal record received sentences of 30 days versus 15 years for the same non-violent offense (cashing counterfeit checks)! Similarly, when local football teams lose games, court judges give harsher sentences!
Overall, I’d say the book is just okay. Mostly, it repeats in abbreviated form the contents of Thinking, Fast and Slow. That said, there are a couple of new things in Noise: A Flaw in Human Judgment, the most useful of which is the noise audit—having different people (e.g., court judges) evaluate the same case (e.g., offender vignette) to determine how much variability there is in evaluations (e.g., sentences). I also found the distinction between level noise (main effects of individual differences), pattern noise (interactions between case characteristics and individual differences), and occasion noise (the effects of situational factors, like mood, weather/temperature, and outcomes of local sporting events) to be helpful.
The discussion of the effects of herding, social pressure, and information cascades on judgments were also informative. For example, the material on forensic confirmation bias—incorrect identification of suspects per DNA or fingerprints—due to occasion noise and social pressure was startling.
Probably the most shocking thing in the whole book was the discussion of noise in sentencing. For example, two offenders with no prior criminal record received sentences of 30 days versus 15 years for the same non-violent offense (cashing counterfeit checks)! Similarly, when local football teams lose games, court judges give harsher sentences!
December 4, 2021
This is a non-fic about the way how uneven or ‘noisy’ are a lot of decisions we all do, some quite life changing. The ‘main’ author of the trio is Nobel prize winner for economics Daniel Kahneman, whose (together with Tversky) article was in the mid-2000s the most cited in economics and who is one of the founding fathers of behavioral economics. I read it as a part of monthly reading for November 2021 at Non Fiction Book Club group.
There is a lot of talk about bias and it is definitely important but another important issue is noise i.e. giving different answers to the same question. To introduce it, the book starts with a 1973 study by a judge Marvin Frankel, who showed on several almost identical criminal cases quite different rulings, e.g. Two men, neither of whom had a criminal record, were convicted for cashing counterfeit checks in the amounts of $58.40 and $35.20, respectively. The first man was sentenced to fifteen years, the second to 30 days. His study was more case by case, but soon statistical studies started and they showed a great diversity. In 1981, 208 federal judges were exposed to the same sixteen hypothetical cases (so they judge the same case!)
In only 3 of the 16 cases was there a unanimous agreement to impose a prison term. Even where most judges agreed that a prison term was appropriate, there was a substantial variation in the lengths of prison terms recommended. In one fraud case in which the mean prison term was 8.5 years, the longest term was life in prison.
This works in private business as well: in insurance companies there are qualified underwriters or claims adjusters, who evaluate expert judgments. Most executives assume that their estimates for the same case should differ like 10%. They study showed much greater differences: the median difference in underwriting was 55%, for claims adjusters, the median ratio was 43%. And this means that these professionals significantly under- or overvalue insurances/claims and that executives aren’t even aware of such magnitude.
Then they shift to the question why there is so much noise and how to lower it.
System noise can be broken down into level noise and pattern noise. Level noise is the variability of the average judgments made by different individuals. Regardless of the average level of their judgments, two judges may differ in their views of which crimes deserve the harsher sentences. Their sentencing decisions will produce a different ranking of cases. This variability is pattern noise (the technical term is statistical interaction). The main source of pattern noise is stable: it is the difference in the personal, idiosyncratic responses of judges to the same case.
The main suggestion for reducing noise in judgment is decision hygiene. Noise reduction, like health hygiene, is prevention against an unidentified enemy. Handwashing, for example, prevents unknown pathogens from entering our bodies. In the same way, decision hygiene will prevent errors without knowing what they are. Think statistically, and take the outside view of the case. Resist premature intuitions. Obtain independent judgments from multiple judges, then consider aggregating those judgments. Favor relative judgments and relative scales.
Overall an interesting study of a problem that many may think doesn’t exist.
There is a lot of talk about bias and it is definitely important but another important issue is noise i.e. giving different answers to the same question. To introduce it, the book starts with a 1973 study by a judge Marvin Frankel, who showed on several almost identical criminal cases quite different rulings, e.g. Two men, neither of whom had a criminal record, were convicted for cashing counterfeit checks in the amounts of $58.40 and $35.20, respectively. The first man was sentenced to fifteen years, the second to 30 days. His study was more case by case, but soon statistical studies started and they showed a great diversity. In 1981, 208 federal judges were exposed to the same sixteen hypothetical cases (so they judge the same case!)
In only 3 of the 16 cases was there a unanimous agreement to impose a prison term. Even where most judges agreed that a prison term was appropriate, there was a substantial variation in the lengths of prison terms recommended. In one fraud case in which the mean prison term was 8.5 years, the longest term was life in prison.
This works in private business as well: in insurance companies there are qualified underwriters or claims adjusters, who evaluate expert judgments. Most executives assume that their estimates for the same case should differ like 10%. They study showed much greater differences: the median difference in underwriting was 55%, for claims adjusters, the median ratio was 43%. And this means that these professionals significantly under- or overvalue insurances/claims and that executives aren’t even aware of such magnitude.
Then they shift to the question why there is so much noise and how to lower it.
System noise can be broken down into level noise and pattern noise. Level noise is the variability of the average judgments made by different individuals. Regardless of the average level of their judgments, two judges may differ in their views of which crimes deserve the harsher sentences. Their sentencing decisions will produce a different ranking of cases. This variability is pattern noise (the technical term is statistical interaction). The main source of pattern noise is stable: it is the difference in the personal, idiosyncratic responses of judges to the same case.
The main suggestion for reducing noise in judgment is decision hygiene. Noise reduction, like health hygiene, is prevention against an unidentified enemy. Handwashing, for example, prevents unknown pathogens from entering our bodies. In the same way, decision hygiene will prevent errors without knowing what they are. Think statistically, and take the outside view of the case. Resist premature intuitions. Obtain independent judgments from multiple judges, then consider aggregating those judgments. Favor relative judgments and relative scales.
Overall an interesting study of a problem that many may think doesn’t exist.
January 2, 2022
Daniel Kahneman tem 87 anos e o seu legado está construído, tendo-lhe sido reconhecido o mesmo com o Nobel pela criação de uma área inteira: a economia comportamental. Neste livro, Kahneman apresenta-se com outros dois autores, Olivier Sibony quase desconhecido, e Cass R. Sunstein, reconhecido pela sua hiperatividade, com mais de 30 livros publicados, só em 2021 já vai com 3, mas também conhecido pela sua crença numa sociedade governada por algoritmos. Os autores dedicam-se à apresentação de uma nova variável de viés, ou melhor, uma nova designação para uma especificidade de viés, o ruído. Para os autores o "ruído é a variabilidade indesejável de juízos". Ou seja, é a variabilidade que acontece numa mesma decisão quando tomada por pessoas diferentes. O clássico exemplo é o dos juízes que podem atribuir uma sentença de 11 meses a 11 anos, em função do juiz que decide, podendo este ser influenciada pela hora do dia, pela vitória de uma equipa de futebol, e uma miríade de outros fatores, no momento da sua decisão.
O comentário continua no blog:
https://virtual-illusion.blogspot.com...
O comentário continua no blog:
https://virtual-illusion.blogspot.com...
August 4, 2021
The authors identify and name another “hidden” social phenomenon present in decision-making: noise.
The term has already been utilized in communication to designate those misunderstandings when the talker’s speech is unclear, ambiguous or vague, and/or the listener does not pay much attention or already has preconceptions about what’s been listened to.
To put it simply, noise refers to disparity in predictions or decisions, made by different people or even the same person repeatedly, in situations in which more uniformity would be expected or desired.
Differently from bias, which designates decisions going all in the same (incorrect) direction, noise designates scattered decisions, even when there is no objective predetermined referential decision to be made.
The authors exemplify it with: variance in insurance premiums depending on the underwriter analyzing the same case, disparities in severity of criminal sentences (or even acquittals) for the same criminal fact, hiring of employees, case handlers deciding on immigration or child care, diagnostic of diseases, reviews of school essays by teachers, etc.
As in many social phenomena, noise is hardly perceived by single individuals immersed in broad decision-making processes, but from a cold overview of statistics. That is why statistic-based decisions or forecasts are often counterintuitive, and sometimes even upset people, who refuse to accept them and cling fiercely to their (myopic) beliefs.
The authors address the frequent “romantic” criticism that decisions based on statistics and averages are often perceived as unfair — a cost-benefit analysis usually shows that the benefits by far outweigh the costs collectively (yes, pretty much utilitarian, I must acknowledge).
Naturally, the authors defend that algorithmic decisions are always superior than human-made ones, as they tend to eliminate noise. Nevertheless, they are also aware of recent discussions on algorithmic biases which sometimes commit scandalous mistakes such as discrimination against people. For the authors, it is just a matter of being attentive to those mistakes and adjust the algorithms properly, so human intervention tends to remain required.
There are also propositions of interesting methods to eliminate noise and reach more accurate decisions (decision hygiene), such as (i) the use of external observers to identify biases, (ii) more judges to assess the same case and then one judge takes the average and finally decides, and (iii) division of complex decisions (where people often hastily jump into overall, holistic conclusions about the target of assessment) into minor — and rigid — thematic decision environments.
Noise is one of those concepts which appears simple to understand once it is explained. Of course, such an insight supposes the hard work and sensitivity of some geniuses to identify and systematize it in the first place! Still, 400 pages seem too much and one can feel bored on and off. Yet when the phenomenon is put in context and perspective it is clear that it carries a lot of complexity.
The term has already been utilized in communication to designate those misunderstandings when the talker’s speech is unclear, ambiguous or vague, and/or the listener does not pay much attention or already has preconceptions about what’s been listened to.
To put it simply, noise refers to disparity in predictions or decisions, made by different people or even the same person repeatedly, in situations in which more uniformity would be expected or desired.
Differently from bias, which designates decisions going all in the same (incorrect) direction, noise designates scattered decisions, even when there is no objective predetermined referential decision to be made.
The authors exemplify it with: variance in insurance premiums depending on the underwriter analyzing the same case, disparities in severity of criminal sentences (or even acquittals) for the same criminal fact, hiring of employees, case handlers deciding on immigration or child care, diagnostic of diseases, reviews of school essays by teachers, etc.
As in many social phenomena, noise is hardly perceived by single individuals immersed in broad decision-making processes, but from a cold overview of statistics. That is why statistic-based decisions or forecasts are often counterintuitive, and sometimes even upset people, who refuse to accept them and cling fiercely to their (myopic) beliefs.
The authors address the frequent “romantic” criticism that decisions based on statistics and averages are often perceived as unfair — a cost-benefit analysis usually shows that the benefits by far outweigh the costs collectively (yes, pretty much utilitarian, I must acknowledge).
Naturally, the authors defend that algorithmic decisions are always superior than human-made ones, as they tend to eliminate noise. Nevertheless, they are also aware of recent discussions on algorithmic biases which sometimes commit scandalous mistakes such as discrimination against people. For the authors, it is just a matter of being attentive to those mistakes and adjust the algorithms properly, so human intervention tends to remain required.
There are also propositions of interesting methods to eliminate noise and reach more accurate decisions (decision hygiene), such as (i) the use of external observers to identify biases, (ii) more judges to assess the same case and then one judge takes the average and finally decides, and (iii) division of complex decisions (where people often hastily jump into overall, holistic conclusions about the target of assessment) into minor — and rigid — thematic decision environments.
Noise is one of those concepts which appears simple to understand once it is explained. Of course, such an insight supposes the hard work and sensitivity of some geniuses to identify and systematize it in the first place! Still, 400 pages seem too much and one can feel bored on and off. Yet when the phenomenon is put in context and perspective it is clear that it carries a lot of complexity.
This entire review has been hidden because of spoilers.
August 25, 2021
The Behavioral Decision Theory (BDT) authors of Thinking, Fast and Slow, and Nudge are the authors of this review of theory and research on errors, biases, and noise in human decision making. These are hugely important ideas that are often poorly understood by many readers, including many who should know better.
The book is well written and entertaining, with lots of examples and clear approaches for making use of these somewhat arcane ideas in our everyday decisions. Towards the end of the book, there are chapters that focus on particular professional areas and their difficulties in handling noise. The chapter on medicine is particularly good, including its discussion of noise issues in psychiatry. The chapters on more common problems of noise encountered in work settings, such as in hiring and performance evaluation, are also excellent.
Even if one follows the research literature, this is an extremely useful book and well worth the time to read. The book also has numerous diagrams and some useful exhibits, along with numerous references, for those wishing to learn more.
The book is well written and entertaining, with lots of examples and clear approaches for making use of these somewhat arcane ideas in our everyday decisions. Towards the end of the book, there are chapters that focus on particular professional areas and their difficulties in handling noise. The chapter on medicine is particularly good, including its discussion of noise issues in psychiatry. The chapters on more common problems of noise encountered in work settings, such as in hiring and performance evaluation, are also excellent.
Even if one follows the research literature, this is an extremely useful book and well worth the time to read. The book also has numerous diagrams and some useful exhibits, along with numerous references, for those wishing to learn more.
Displaying 1 - 30 of 1,420 reviews