The Topics: Consensus Judging and Online Exams vs. Legacy
Ken Carman is a BJCP judge; homebrewer since 1979, club member at Escambia Bay, Salt City and Music City Homebrewers, who has been interviewing professional brewers all over the east coast for over 10 years.
In a previous edition of Brew Biz I discussed a different way to score as judges. This edition I would like to discuss two possible changes in judging and how to rank judges…
I have been considering a comment I made about how the brewer who enters his or her beer often doesn’t understand how a consensus score was arrived at. Maybe one judge’s score sheet has comments that totally conflict with the other score sheet. As we all know judges have different palates, different talents and perceptions. But in the end we not only have to be at least 7 points from each other, but through that process and score adjustment, come up with a consensus score.
Perhaps on a separate sheet, or on the back, it would be valuable to the entrant to provide some information on how the consensus score was arrived at. Maybe one judge felt there was a hint of DMS, another felt not, a kind comment mentioning that and why it was skewed one way or another like, “At first Judge Marvin couldn’t sense it at first…†or “Judge George decided it was so minor maybe the rest of the experience of this fine entry was more important.â€
One might assume all the brewer has to do is look at the sheets and assess this, but I have met many a brewer, many who are also judges, who scratched their heads and said, “So how did they come up with…?â€
I’m sure, if viable, others can improve on this idea: finds ways to phrase what I have so roughly outlined here.
Another discussion-based viewpoint I have arrived at was inspired by a discussion of the new online exam. Well, “new†for some of us, I suppose. The discussion was on a Facebook BJCP page.
Some of the judges felt it unfair that some could “game†the system by taking the online exam open book, or using some “system†that takes advantage of the nature of it being online.
I’m legacy, and my wife Millie, who is still an apprentice, is trying to prepare to take the online so she doesn’t lose her BJCP status. But I have spoken with many folks who have taken it online, and they say, “This is no give me.†They also say passing it due to having an open book would be difficult, at best, the questions must be answered so fast.
But I’ll leave all that to those who have designed, or taken, the online exam. But I do believe there is a greater value to have taken the sit down exam vs. an online exam. First I would think there’s a little more initiative to actual go somewhere and take a closed book, proctored, exam. And having taken other exams, not on line but simple true/false/color in the circle, like we all have, there’s a greater ease to that than having to write several essays, provide a full recipe for a style and other intense test taking activities that are inherent to the more traditional, sit down, legacy exam.
As it stands right now if you want to be above Certified, you take the sit down exam: otherwise you can get to Certified by taking the online exam. But to me that’s not quite enough “reward†for one’s dedication to the process. “Recognized,†I suppose, is apt for either if one shoots for the stars but doesn’t make it.
I know during this discussion it was mentioned that splitting ranks was discussed once and rejected, but perhaps it’s time to revisit that decision. Maybe we should consider Certified 1 and Certified 2 categories? First, I must admit, being Certified myself, having the exact same rank as someone who went online, maybe open book, is bothersome. So I agree I have a stake here, but this isn’t about ego: otherwise I wouldn’t bother writing this.
So maybe Certified L and Certified O? This way the dice can fall whatever way they fall in regard to how folks regard this distinction.
As I stated, it’s not about ego: more about encouraging judges to that extra step of preparing for the special day, then going somewhere and taking a closed book exam. To me that shows a bit more initiative and dedication to judging homebrew well. For my second exam I drove to Knoxville, up at 2am. I was so keyed up I couldn’t sleep while my wife drove.
I’m sure many folks have gone to greater effort than I have.
After all, if online is just as good as sit down and dedication, effort, means nothing, should there be any sit down at all? And if net is as good as doing things as a group, if that’s the tude why not just have brewers submit 6 bottles and three judges go home and “discuss†via the net for all judging?
I hope we never get to that standard for judging all homebrew beer. Whether submitting beer, or putting the effort forward to become a judge, and then a better judge, I think, perhaps, more consideration is due, more kudos, for those who leave their castle and meet with the others to learn, be tested and judge beer.
Obviously even the BJCP considers sit down, legacy-style, exams of greater value or it all would be online testing. My purpose here is to offer the idea that maybe we should provide just a tad more incentive to take that extra effort.
Just a suggestion.
-30-
Brew Biz: Werts and All, is a column dedicated to reviewing, discussing and commenting on all things beer including, but not limited to: marketing, homebrewing and homebrew/beer related events, how society perceives all things beer. Also: reviews of beer related businesses, opinions about trends in the beer business, and all the various homebrew, judging and organizations related to beer. Essentially, all things “beer.”