Some Thoughts On Superforecasting By Philip Tetlock

“For scientists, not knowing is exciting. It’s an opportunity to discover; the more that is unknown, the greater the opportunity.”

Philip E. Tetlock

“Suppose someone says, “Unfortunately, the popularity of soccer, the world’s favorite pastime, is starting to decline.” You suspect he is wrong. How do you question the claim? Don’t even think of taking a personal shot like “You’re silly.” That only adds heat, not light. “I don’t think so” only expresses disagreement without delving into why you disagree. “What do you mean?” lowers the emotional temperature with a question but it’s much too vague. Zero in. You might say, “What do you mean by ‘pastime’?” or “What evidence is there that soccer’s popularity is declining? Over what time frame?” The answers to these precise questions won’t settle the matter, but they will reveal the thinking behind the conclusion so it can be probed and tested. Since Socrates, good teachers have practiced precision questioning, but still it’s often not used when it’s needed most. Imagine how events might have gone if the Kennedy team had engaged in precision questioning when planning the Bay of Pigs invasion: “So what happens if they’re attacked and the plan falls apart?” “They retreat into the Escambray Mountains, where they can meet up with other anti-Castro forces and plan guerrilla operations.” “How far is it from the proposed landing site in the Bay of Pigs to the Escambray Mountains?” “Eighty miles.” “And what’s the terrain?” “Mostly swamp and jungle.” “So the guerrillas have been attacked. The plan has fallen apart. They don’t have helicopters or tanks. But they have to cross eighty miles of swamp and jungle before they can begin to look for shelter in the mountains? Is that correct?” I suspect that this conversation would not have concluded “sounds good!” Questioning like that didn’t happen, so Kennedy’s first major decision as president was a fiasco. The lesson was learned, resulting in the robust but respectful debates of the Cuban missile crisis—which exemplified the spirit we encouraged among our forecasters.”

Philip E. Tetlock

“It follows that the goal of forecasting is not to see what’s coming. It is to advance the interests of the forecaster and the forecaster’s tribe.”

Philip E. Tetlock

“Here’s a very simple example,” says Annie Duke, an elite professional poker player, winner of the World Series of Poker, and a former PhD-level student of psychology. “Everyone who plays poker knows you can either fold, call, or raise [a bet]. So what will happen is that when a player who isn’t an expert sees another player raise, they automatically assume that that player is strong, as if the size of the bet is somehow correlated at one with the strength of the other person’s hand.” This is a mistake.”

Philip E. Tetlock

“If you don’t get this elementary, but mildly unnatural, mathematics of elementary probability into your repertoire, then you go through a long life like a one-legged man in an ass-kicking contest.”

Charles T. Munger

Here is a video from Philip Tetlock on having an open mind:

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s