rethinking the mixer quiz

It's now apparent that my English conversation students don't quite know how to handle my mixer quizzes. The quiz's basic concept is simple: forbid students to write on their own papers, forcing them to tell their partners-- in English only-- what to write. The product-- i.e., what gets turned in to the teacher--would be written, but the process would involve speaking because, as I've said, it makes little sense to give students a written exam after they've spent all their class time working on speaking skills.

But my students have continued to find ways around the obvious, and it's obvious to me that I've made some critical misjudgments. Consider the following problems, all of which have cropped up at some point or other this semester:

1. Many students have no qualms about cheating if that's what gets a correct answer onto the paper. My original instructions were simple: sit with one partner at a time, work only with that partner in the method described above, then move on to another partner. Any behavior not in line with those instructions is, quite simply, cheating. These conditions were made clear at the outset, but they have done nothing to prevent cheating. Students clump together in large groups, speak to each other in Korean, grab their own papers back and mark corrections on them, etc.

2. Students don't get the basic premise that they're supposed to talk their partners through each answer. Instead, I sometimes get what happened today: the partner receives a paper, writes her own answer on it, then hands it back to the paper's owner, who gives it an approving nod. Not a word spoken-- no attempt made by the paper's owner to delve into her active vocabulary and produce a meaningful utterance.

3. Because of (2), some students who are actually quite good are ending up with lower grades than less capable students. This is because the good students aren't policing their own papers well: they're trusting the partner to write whatever, and somehow hoping that that will suffice. Also: the conversation class isn't for credit, so no one feels any sense of ultimate urgency about their grades. This was unexpected. However, paradoxically:

4. Students will skip class on quiz day, probably because they have already built up such negative associations with tests and quizzes, based on years in the Korean system. They are aware that a poor grade on a quiz in my class will have no effect on their GPA, but the very prospect of a poor grade is enough to drive some students away. The whole thing is amazingly childish.

I'm viewing all this-- the ups and downs of this quizzing method-- as experimental data. The data lead me to conclude that the method, as it stands, needs to be either improved or completely rethought. Obviously, some sort of change is necessary, but I'm attached to the basic design of the mixer quiz. Some possible solutions present themselves:

1. Reduce the quiz to a series of highly regimented steps which the students must explicitly follow. As things stand, I hand each student a paper with ten questions. Perhaps I should do the quiz in "rounds"-- handing out only one or two questions per round. Or, instead of handing out quizzes to all the students, I should hand out quizzes to only half the class at a time, thereby reducing the illicit traffic in paper-trading.

2. Keep the basic idea of the mixer format, but use a totally different mixing procedure.

3. Chuck the whole thing.

The main reason I chose the mixer quiz format for this course (Level 2 English conversation) is that the textbook I'm using, Jazz English (pardon the corny title, but ESL literature is filled with corny titles), is billed as a mixer-style book. Because my mixer quizzes worked well with my French classes in America, I was convinced they'd work equally well here. While the mixer has met with some success here-- at least at the beginning of the term-- this hasn't panned out over time.

So maybe it's time to go back to the drawing board and find more effective, more culturally appropriate formats for quizzes and tests. Or maybe it's time to strip everything down and return to some simple, hardcore basics-- newfangled teaching methodologies be damned. Here's a rundown of the most popular conversation exam formats, along with their respective merits and demerits:

1. Multiple choice tests targeting isolated aspects of language-- e.g., listening comprehension, grammar points, etc.

Advantage: Easy to grade.

Disadvantages: Where to begin? First, there's the "luck" factor: a canny student can guess their way through part or all of such a test. Second, there's the question of language in isolation. Multiple choice tests don't confront students with language in context. Third, there's the passivity factor: no matter how cleverly you build your multiple choice test, there's more reliance on passive vocabulary than active vocabulary to get through. Finally, it's something of a joke to give conversation students a multiple choice exam. This kind of test simply isn't a serious option, in my opinion. Koreans, in particular, have been conditioned to take-- and ace-- tests of this nature since the beginning. A student learns almost nothing about his true ability from such a test.

2. One-on-one short interviews.

Advantages: When the student is asked to come to the front with no notes and no textbook allowed, there's almost no way for the student to fake her way through the interview. Because interviews can be conducted in a variety of ways, there's an inherent flexibility in the process. Interviews can also be used to gather a holistic impression of the student's abilities, or they can target specific areas. The holistic aspect of interviews makes them, in some ways, more realistic (keeping in mind that no test is completely realistic) than multiple choice exams: dialogue has a certain direction and flow, and creates its own context as it progresses.

Major disadvantages: First, such interviews take time and are cumbersome to conduct. The teacher will often need to occupy the rest of the class with other activities while each student awaits her turn. Another disadvantage is that, unless one has a precise, thorough, and consistent way of evaluating student output, the single grade assigned to the interview contains a large amount of "fuzziness"-- what, exactly, does a "B+" mean? Compare two "B+" students: one whose pronunciation leaves much to be desired but whose grammar and fluency are impeccable; another whose fluency and pronunciation are nearly flawless but whose utterances contain grammatical errors. How meaningful is a single grade? That grade hides as much as it reveals.

A possible solution to that problem is simply to assign multiple grades. When I took my Korean class at Korea University, I did end up with a single final grade, but my report card also showed what grades I'd received in each of the four language macroskills: listening, speaking, reading, writing. A colleague of mine has suggested a similar approach to our conversation classes-- as always, in the knowledge that these grades would be for informational purposes only, not for credit.

3. Write-and-recite-a-dialogue. In this exam format, somewhat similar in nature to the interview because it takes place in front of the teacher, students work in pairs or groups and create (i.e., write) dialogue using language elements from the material on which the students are being tested. They then recite their dialogue in front of the teacher. Dialogues can be either open-ended or rather directed in nature.

Advantages: students work cooperatively, which is a reflection of how conversation actually happens: it's an interchange. Students also have the chance to correct each other's mistakes before they even reach the teacher. While this may result in artificially "clean" grammar, etc., it "lowers the affective filter," as Stephen Krashen might argue, and still includes use of active vocabulary.

The obvious disadvantage is that certain important aspects of language, such as listening comprehension, can't be tested for, and the dialogue's lack of spontaneity is problematic. Furthermore, as with one-on-one interviews, there's the problem of time: students need time to create their dialogues, and even in pairs or groups, it will take time to interview all the students.

This sort of exam also presents a special problem: diffusion of responsibility can result in unfair exam results. Let's say that Students A and B create a dialogue. Because they've collaborated on it, it will be difficult for the teacher to know which student committed which grammatical (etc.) errors on paper. If the teacher is grading only the verbal utterances, then the student who utters the erroneous English will be penalized, even if she wasn't the one to commit the error to paper. This is obviously unfair. One way around this problem might be to ask students to write their own lines with no help from their partner, but this is nearly impossible to enforce in practice, as the partners have to work in tandem to create dialogue that makes sense.

4. Task-oriented activities. These come in all shapes and sizes, and it'd be impossible to discuss them all here. Like interview examinations, these sorts of activities would have a fairly holistic bent, which means they have some of the same advantages and disadvantages of other holistic evaluative methods.

5. Dictation-plus... In France, the dictée is an educational staple, especially in the early years of education, because so many French words have endings that sound alike, yet are spelled differently. An exam that couples dictation with other, more active aspects of language-- say, speaking-- can test a student's listening and speaking skills, but depending on how the speaking element is to be tested, the exam might fall into any of the traps noted above. Also, it's questionable as to whether dictation is really testing listening comprehension, per se.

One's theory of language learning will affect how one teaches and tests students. Is development of L2 (second language) more a matter of acquisition (implicit, osmotic, holistic), or of learning (explicit, self-conscious, atomistically rule-governed)? To what extent is language a matter of habit-formation? Rote memorization? Precise pronunciation and application of grammatical elements? Is the holistic maxim "Be understood!" a good enough criterion for grading students?

Is a communicative approach to language teaching-- in which students are encouraged to make generally sensible, creative utterances that convey their intentions, but are likely to develop bad speech habits-- superior to something more along the lines of an audiolingual approach, in which decontextualized elements of language are "plugged into" larger grammatical structures, then taught to the students according to some theory of language-as-habit-formation?

Some might look at the above choices and say, "The answer lies somewhere in between." I'm no longer convinced that the "compromise" solution offers any greater wisdom than explicit, distinct approaches to language teaching. Compromise approaches, which attempt to integrate what's best about various older methods into themselves, often end up as little more than a confusing mishmash of teaching techniques. With no clear idea as to how much of each method to integrate into one's compromise syllabus, one risks creating an impotent concoction that helps the student not at all.

There is, we all know, no "perfect method" for language teaching and learning. The process is as much art as it is science-- imprecise because there are so many human factors to consider. Statistics, which linguists use to bash each other's pet approaches to language, are susceptible to multiple interpretations and don't provide us with a trustworthy means to know for sure that such-and-such an approach is an absolute failure or clear success. How to explain, for example, the numerous European examples of English- and French-speakers from centuries ago, who wrote (and likely spoke) in impeccable Church Latin (not to mention other European languages) despite having little more than the clumsy grammar-translation method at their disposal? If that method-- now almost universally despised-- was so poor, how did these scholars, many of whom were linguists in their own right, ever come to be?

There was once an amusing story in some magazine or newspaper about an African marathoner who, by running-- and winning-- the marathon while shoeless, was making those fancy shoe companies nervous. It's a good analogy for language teaching: sometimes what "works" is the old, simple, unadorned formula of hard work, hard drills, and uncompromising focus on the basics, like grammar, pronunciation, and vocabulary. Newfangled techniques that preach communicative competence (itself a pretty vague notion when you stop to think about it) and stress keeping the students interested might provide language learners with some benefits, but ultimately, how much better are those new methods than the older ones? I'm not saying we should go back to the Stone Age, but I am wondering whether, given the kaleidoscopic, market-driven nature of the ESL field, the Emperor's new clothes aren't a bit too much like thin air.

Perhaps my problem isn't just a matter of quiz design. Perhaps I need to be looking at how I approach my whole curriculum.


_