The Topic: Styles of scoring and judging beer
Ken Carman is a BJCP judge; homebrewer since 1979, club member at Escambia Bay, Salt City and Music City Homebrewers, who has been interviewing professional brewers all over the east coast for over 10 years.
There are various versions of judging beer, like bottom up or top down: one where the judge decides what score the beer probably should be then adjusts as he or she scores, or scores the beer and then adjusts the score to what they think is closer to what it should be. Then you always have adjusting because two, or more, judges differ and their score is too far apart.
Then you have one of the oddest I’ve ever encountered. I understand judging in silence and then discussing after we’ve each come to a score, but one judge claimed the correct way to do it is to judge all the beers in a flight and then go back and adjust.
I simply can’t support that. If the head of the table insisted, well of course I would do it.
There is another method I have used that most judges probably frown upon, but I think can work quite well. The BJCP doesn’t seem all that fond of it either. But in certain circumstances: with certain judges, I think it works amazingly well.
If I’m the head of the table: and I think it might work with these judges, I’ll ask the other judges what they prefer. Of course if they have a problem with what I do then we go back to the standard: I am no head of the table dictator.
But, if they have no problem with it: we do discuss while sampling.
Before we take a single sip I tell them be sure to contradict me if they’re either are not getting some sense I am finding, or if they’re getting something I’m not.
If one senses it, the other doesn’t or disagrees, all judges should be willing to step back and basically say, “Well, go ahead and write down what you found, and I’ll write my perceptions.” Indeed, no matter what method is used that’s the standard.
Honestly? It seems to work well, and we adjust if we find there are any problems. Usually there aren’t, though often we vary as to exactly when we start discussing. And, of course, we might agree to discuss from the start, or wait until some point mid-judging.
Think of it like any other judge. Does a judge ban evidence, or witnesses, before declaring the sentence, then adjust after discussing with other judges? I’m not declaring beer judging is exactly like a judge in a criminal case. Obviously not the same. But it’s not the exact opposite either.
The question you must ask before even considering, or agreeing to, this method: how independent are you as a judge? Are you a follower, or highly influenced by suggestions? Do you like to lead, make independent decisions, stand your ground if you firmly believe a beer has diacetyl, for example, or not? But: and this is crucial, are you willing to let other judges stick to their assessments? If you’re willing to check your ego at the door while judging, but still willing and able to stand your ground: I highly recommend.
First, I think it helps with scoring and score sheets. Nothing’s more confusing to an entrant than getting score sheets back where comments radically conflict and then the score seems to reflect neither score sheet because the judges had to adjust. This method doesn’t eliminate that, but it will more accurately reflect that adjustment. For example: let’s go back to diacetyl. It’s not my palate’s strongest sense, and there have been times I’ve had to take a second taste, another sniff, let it linger upon the palate a tad longer and I’ll get it… or not. With discussion, occasionally, I will admit: yes, maybe I’m missing that.
But then you have a contest I did BOS for recently and when we got down to 1, 2,3 the organizer told us one of the judges before BOS had felt it might have a hint of diacetyl. These were IPAs.
OK, maybe he really shouldn’t have told us that, according to how most judge, but I was actually grateful for the extra information. Millie, who is better at diacetyl than I am, said, “Well, I get a hint of slick, but nothing other than that in the nose or taste.”
I got nothing, and did sense the slick and something else someone could confuse with butter, but was actually caramelization. And here’s was the final decider: the other one two or threes were great but one was more a pale, and the other more astringent than it should be: with no aroma.
Essentially: it was a judgement call, and I left a short, polite, note for the brewer as to the decision process we went through in regard to the claim of diacetyl. Otherwise he wouldn’t have had a clue as to how previous comments were resolved during the judging process.
Here’s what we did: we helped the entrant understand the judging process better and still maybe consider diacetyl the next time he brews. But if the original judge’s decision had stood the beers that fit the style less would have come in ahead of the one that really nailed the style quite well, other than the supposed defect we both felt it didn’t really have. And the brewer might have changed the recipe to less fit the style, but get rid of that same defect we thought wasn’t there. To be honest, even if we were wrong we were told one judge thought it might have just a tiny, tiny bit. So the beers that mostly missed the style should win?
Yup, “judgement call.”
I don’t claim it’s perfect. In fact, I think as most judges know, in this subjective pursuit there is no such thing as “perfect,” just the attempt to be as objective as possible.
With each sheet standing on its own: 2, 3, 4 judges separate opinions, sometimes a final score really makes no sense. With this method you can give them a better idea how we arrived at that score. Fewer comments erased, or left behind, that make little sense when compared to the scoring. Comments, when done right, might be a little more in tune with each other, but only when one judge comes to the true realization, with a, “Oh! Now I get it.”
I don’t recommend for all tables. In fact, probably very few tables.
But I believe, when judges are comfortable with the idea, it can produce better score sheets. It just takes judges willing to stand their ground, but not step on the other judge, and judges not willing to be persuaded when they disagree.
And, as with all beer judging, it all depends on how well we work together.
Brew Biz: Werts and All, is a column dedicated to reviewing, discussing and commenting on all things beer including, but not limited to: marketing, homebrewing and homebrew/beer related events, how society perceives all things beer. Also: reviews of beer related businesses, opinions about trends in the beer business, and all the various homebrew, judging and organizations related to beer. Essentially, all things “beer.”