Doggerel #219: "I Did My Own Research!"

Welcome back to "Doggerel," where I ramble on about words and phrases that are misused, abused, or just plain meaningless.

The information age allows people incredible access to scientific data and exchange of information. Unfortunately, it also allows incredible access to urban legends and crazed rants that would normally be relegated to badly copied manifestos handed out on the street.

What separates real research from from what the typical woo does is discrimination: You have to know what forms of evidence are more reliable than others. A double-blind control study is superior to a case study, which is superior to an anecdote. A video recording is superior to human memory. Quality also matters more than quantity, most of the time. A thousand eyewitnesses are less objective than a handful of cameras.

Woo "research" typically involves collecting anecdotes, a form of cherry-picking. In alternative medicine, for example, positive outcomes for an alleged remedy are popularized, but, because of the culture, negative outcomes are generally ignored, usually leading to the patient to quietly change remedies. The result is a large collection of isolated positive anecdotes divorced from the larger context of the world.

Real research has to look at large numbers. With small groups, and worse, individual test subjects, there is little ability to rule out coincidences. Scientific studies require large numbers of subjects or trials because large numbers make coincidence less likely of an explanation. Most people simply have trouble thinking about the large numbers involved in the world, and how millions of people experiencing the world over many years can produce all sorts of "unlikely" events. We use the scientific method because of such shortcomings in our thinking.

Another form of false research comes in the form of blindly trusting people with fancy degrees or even Nobel Prizes. Being a scientist, earning a degree, or being awarded a prestigious prize is not a certification of papal infallibility. Scientists are people. People can make mistakes. Working in a manner that minimizes the chance of mistakes is what makes your conclusions more likely to be accurate. It doesn't matter who you are. High quality work is high quality work. Knowing how to distinguish real science from anecdote and hearsay is what separates a researcher from just another student of Google University.