Winning a medal at a national wine competition means a lot. In fact, wineries often spend thousands of dollars on entry fees every year in hopes of taking one home. And according to a 2001 study, a one-point increase on the Robert Parker wine ratings can translate to 7 percent increase in price. To put it simply, the influence judges have is quite significant.
But just how reliable are "expert" judges at rating wines?
Experiments by one oceanographer turned wine maker indicate something pretty crazy: They're not at all.
See also: - Sam Pillsbury: Treat Your Wine Tasting Room with Respect - Wine Tasting 101
Robert Hodgson, a retired professor who taught statistics at Humboldt State University, has been making wine at Fieldbrook Winery since 1976. He and his wife operate the winery, producing 10 wines each year. They've entered them -- and won -- at national and international wine competitions.
But Hodgson, in typical academic fashion, became curious about the science of wine tasting and judging when he realized how inconsistently his wines would perform in contests. To find answers, he teamed up with the organizers of the California State Fair wine competition to conduct an experiment.
Hodgson and the organizers gave each four-person panel of judges at the competition the usual flight of wines to taste, but also gave them certain wines two more times -- poured out of the same bottle for each test.
The results? Judges' scores of the same exact wine typically varied by plus or minus four points. Meaning a single judge could give a single bottle of wine scores ranging from 82 to 90 on any given tasting. In fact only 10 percent of expert judges' scores varied by plus or minus two. And even that's enough to swing the outcome of a contest.
There are plenty of reasons why it's so difficult to judge wine. For one, it's a subjective art. But beyond that, scientists have identified 400 aroma compounds that work together and independently to create a wine's bouquet and flavor. Some can be detected right away, but others can be sensed only as an aftertaste. Plus, the volatiles in wine are greatly affected by temperature, meaning serving a wine too warm will completely change its taste.
And Hodgson is not the only one who's investigated the science of wine tasting. In 2011, Professor Richard Wiseman at Hertfordshire University asked 578 people to take a blind taste test of a selection of red and white wines varying in price from £3.49 for a claret to £30 for champagne. He found people were successful at telling the difference between the cheap and expensive wines only about 50 percent of the time. They might as well have been flipping a coin.
"I think there are individual expert tasters with exceptional abilities sitting alone who have a good sense, but when you sit 100 wines in front of them, the task is beyond human ability," Hodgson told the Guardian. "We have won our fair share of gold medals, but now I have to say we were lucky."
