Improving Adjudication in Debating Championship

This article is adapted from the Chief Adjudicator's report which I completed after the Haifa EUDC. It is customary for CA's to file a report for the next year's CA spelling out their experience from the championship, and (more importantly) evaluating the adjudicators in the championship. This second part is obviously confidential - I suspect that if those reports were made public, the CAs would have far fewer friends. Another part of this report - offering a frank advice to the EFL CA from an ESL CA , and a summary of championship results.

 

 

• on the down side, not enough of the top clubs in the British Isles sent representatives, with Oxford, Leeds and Cambridge being the exception that proves the rule, and the presence of many weak teams from Israel (for whom price was not deterrent), Turkey (several absolutely horrendous teams, with zero experience and next to zero English, who never scored a "C").  
• Second, we've made extensive efforts to attract top judges, even with outright bribing. This is how we ended up having Andy, Tim and Mark, but also Tom Hamilton and Rob Garson.
• The last reason, which is of most interest, is the steps we took to allocate and brief judges appropriately.  Our actions must be seen in the time perspective of Europeans - While much longer than an average IV, at 4 days for 11 rounds, it is a very tight schedule, leaving little room for accreditation or Briefings.

The first step, which proved immensely beneficial, was to have a separate and (almost) compulsory briefing for novices. This has addressed a recurring problem in large tournaments - Briefings are invariably too complex for novices, and too regurgitative for the experienced.  As a result, some of the novice judges bring horrific misconceptions into the debates (A personal favorite - Prop Whip should be penalized for adding positive matter).  Conversely, experienced judges feel that their intelligence is being insulted, and therefore (at best) see the briefings as shallow or (at worst) simply do not show up. 

Also, with the basics out of the way, we were able to run the main briefing at a higher level. That very much pleased the veterans, who felt they were getting their money's worth in terms of seeing exactly what we refer to in all those tricky debating issues- extensions, barracking, challenges, etc.  The novices briefing (held by Iva) was less formal and more of a "workshop" style, bringing some "how-to" advice.  The laptop for that briefing crashed, but this has turned out to be a blessing, because it made the

workshop much more interactive and involved the audience.  A main benefit of Iva running this briefing is that she is well versed with many debating formats, and so the workshop included specific remarks referring to the differences between BD and Karl-Popper, American style, Australs etc.  This was a problem we've all seen in Toronto with American adjudicators who "didn't get it". Additional recommendations-

The second thing we did, which was also useful, but could have been done even better, was a frontal, written test.  Given that we didn't have time for a proper video accreditation, and given that we had plenty of  novice adjudicators which we knew nothing about, we had to find a fast way to tell them from one another. 

The method we used was a frontal test, composed of multiple-choice questions (to make checking fast), asking for specific debating-theory issues. Another part presented them with hypothetical situations and asked them to make a decision, hopefully one such test is attached to this document, if not, it should be made available soon on our website.  It proved very good in telling those novices who've got a brain from those who haven't got a clue.  It also enabled us to see which points need to be emphasized in the briefing. One problem with the test is that we didn't stress to the judges that the test will determine their ranking, and so some of them genuinely thought it was a joke.  Clearly, some people would always make a joke of things, especially the veterans, but it still proved very useful. 

For worlds, we would recommend running it before the briefing, regardless of whether you'll run a video accreditation - it simply tests different knowledge areas.

The third thing we did was to add a numeric evaluation for adjudicators.  We ran a "peer evaluation form" which every judge had to fill out about his panel-mates. A copy should be attached as well.  In the form we had them write their comments, but more importantly, we had them give a numeric figure.  The numbers ran from 0 for "shouldn't be allowed to judge at Europeans" to 10 for "should be judging the grand final". 

The fomrs enabled us to see quickly and easily where troubles were brewing or to notice potential stars.  There was also room to write comments, and people actually did. One of the ways we got people to treat this form properly (and it also gave us very useful information), was by treating the form as a measure for the EVALUATOR, as well as the evaluated.  We took special notice of those who filled insightful comments, or who were quick to spot potential stars.  Note that we had chairs evaluating panel members, but also the panel members evaluating chairs and one-another.

Finally, we made the role of the chair significant.  Given that we had many excellent chairs and many novices, but very little in between, we felt it would be a shame if Andy would be out-voted by a couple of junior debaters from Haifa who think they are very smart.  We obviously couldn't tell people to agree with whatever the chair said.  For one it is against the rules, but more importantly it is not the value we wanted to project.