Improving Adjudication in Debating Championship
This article is adapted from the Chief-Adjudicator’s report which I complied after the Haifa EUDC. It is customary for CA’s to file a report for the next year’s CA spelling out their experience from the championship, and (more importantly) evlauating the adjudicators in the championship. This second part is obviousely confidential – i suspect that if those reports were made public, the CAs would have far fewer friends. Another part of this report – offering a frank advice to the EFL CA from an ESL CA , and a summary of championship results.
Overall, the standard of adjudication in Europeans was very good. The general feedback we got, from debaters and adjudicators alike was that “the level of adjudication was two notches higher than the level of the debates”. There are three main contributions for that-
- on the down side, not enough of the top clubs in the British Isles sent representatives, with Oxford, Leeds and Cambridge being the exception that proves the rule, and the presence of many weak teams from Israel (for whom price was not deterrent), Turkey (several absolutely horrendous teams, with zero experience and next to zero English, who never scored a “C”).
- Second, we’ve made extensive efforts to attract top judges, even with outright bribing. This is how we ended up having Andy, Tim and Mark, but also Tom Hamilton and Rob Garson.
- The last reason, which is of most interest, is the steps we took to allocate and brief judges appropriately. Our actions must bee seen in the time perspective of Europeans- While much longer than an average IV, at 4 days for 11 rounds, it is a very tight schedule, leaving little room for accreditation or Briefings.
‘Novice Briefing’ –
The first step, which proved immensely beneficial, was to have a separate and (almost) compulsory briefing for novices. This has addressed a recurring problems in large tournaments- Briefings are invariably to complex for novices, and too regurgitative for the experienced. As a result, some of the novice judges bring horrific misconceptions into the debates (A personal favorite- Prop Whip should be penalized for adding positive matter). Conversely, experienced judges feel that their intelligence is being insulted, and therefore (at best) see the briefings as shallow or (at worst) simply do not show up.
Our solution which (allow me to say that again, I enjoy this) worked wonders, was to add another briefing for novices PRIOR to the main briefing.
Also, with the basics out of the way, we were able to run the main briefing at a higher level. That very much pleased the veterans, who felt they are getting their money’s worth in terms of seeing exactly what we refer to in all those tricky debating issues- extensions, barracking, challenges, etc. The novices briefing (held by Iva) was less formal and more of a “workshop” style, bringing some “how-to” advices. The laptop for that briefing crashed, but this has turned out to be a blessing, because it made the
workshop much more interactive and involved the audience. A main benefit of Iva running this briefing is that she is well versed with many debating formats, and so the workshop included specific remarks referring to the differences between BD and Karl-Popper, American style, Australs etc. This was a problem we’ve all seen in Toronto with American adjudicators who “didn’t get it”. Additional recommendations-
- Allow debaters to attend as well- some of the newer teams would love dearly love to be told by an experienced debater how to debate.
- Make sure to have a decent break (preferably with a meal) between the novice briefing and the main briefing. People need to digest what they’ve heard, and a 5 hours briefing is an oxymoron.
- Attendance should be compulsory for everyone who “hasn’t participated in a major BP tournament, in ENGLISH”.
Debating Aptitude Test
The second thing we did, which was also useful, but could have been made even better, was a frontal, writen test. Given that we didn’t have time for a proper video accreditation, and given that we had plenty of novice adjudicators which we knew nothing about, we had to find a fast way to tell them from one another.
The method we used was a frontal test, composed of multiple-choice questions (to make checking fast), asking for specific debating-theory issues. Another part presented them with hypothetical situations and asked them to make a decision, hopefully one such test is attached to this document, if not, it should be made available soon on our website. It proved very good in telling those novices who’ve got a brain from those who haven’t got a clue. It also enabled us to see which points need to be emphasized in the briefing. One problem with the test is that we didn’t stress to the judges that the test will determine their ranking, and so some of them genuinely thought it was a joke. Clearly, some people would always make a joke of things, especially the veterans, but it still proved very useful.
For worlds, we would recommend running it before the briefing, irrespective of whether you’ll run a video accreditation- it simply tests different knowledge areas.
Peer Evaluation by Judges and Chairs
The third thing we did, was to add a numeric evaluation for adjudicators. We ran a “peer evaluation form” which every judge had to fill about his panel-mates. A copy should be attached as well. In the form we had them write their comments, but more importantly, we had them give a numeric figure. The numbers ran from 0 for “shouldn’t be allowed to judge at Europeans” to 10 for “should be judging the grand final”.
The fomrs enabled us to see quickly and easily where troubles were brewing or to notice potential stars. There was also room to write comments, and people actually did. One of the ways we got people to treat this form properly (and it also gave us very useful information), was by treating the form as a measure for the EVALUATOR, as well as the evaluated. We took special notice of those who filled insightful comments, or who were quick to spot potential stars. Note that we had chairs evaluating panel members, but also the panel members evaluating chairs and one-another.
Lastly, we made the role of the chair significant. Given that we had many excellent chairs and many novices, but very little in between, we felt it would be a shame if Andy would be out-voted by a couple of junior debaters from Haifa who think they are very smart. We obviously couldn’t tell people to agree with whatever the chair say. For one it is against the rules, but more importantly it is not the value we wanted to project.