Jaqueline Mull, Research Institute of English Language Education, Kobe, Japan
Mull, J. (2013). The learner as researcher: Student concordancing and error correction. Studies in Self-Access Learning Journal, 4(1), 43-55.
Download paginated PDF version
Abstract
The idea of language learners using a concordancer, to autonomously investigate vocabulary and structure in a target language was suggested over 30 years ago. Since then, some research has explored this idea further, but the potential benefit of concordancers in the hands of learners is still largely unexplored – especially with regards to structure. This study investigates what learners are able to accomplish when asked to investigate an English corpus with a concordancer in order to correct grammar errors in an essay. The study was conducted after only 30 minutes of training on a concordancer. Participants reactions to the software and to analyzing the target language autonomously are also shared. While participants’ reactions were mixed with regards to using a concordacer for error correction, all participants expressed an interest in using a concordancer during their writing process – something which was beyond the scope of this study – but which suggests a potential value for learner exposure to concordancers for autonomous language investigation.
Keywords: centred, writing centre, Japanese university, EFL writing
Proponents of autonomous learning have long encouraged educators to rethink the traditional, teacher-centered classroom and instead explore ways to help students build the skills and confidence they need to seek out answers on their own. According to Dam (2011), autonomy is not about learners studying completely on their own or doing all the work independently of an instructor. Rather, it is a partnership between learners and instructors with the purpose of encouraging the learner to pay attention to their own gaps in knowledge, create their own goals and find strategies that will address both. Of course, with proper support, guidance and tools to answer their own questions, a learner can do a great deal of language exploration and skill building on their own, as well.
Literature Review
Writing offers an excellent opportunity for students to reflect on their own language ability and notice gaps in their interlanguage. While struggling to create language, learners are likely to generate a wide variety of questions. If a student is not comfortable acting in an autonomous way, they will likely bring questions to a teacher or other “expert” for answers. Legenhausen (2011) argues that “learners need to be encouraged to also pay attention to formal structures but without being explicitly taught or instructed” (p.36). If learners are familiar with language structure, they are better equipped to analyze language when they see it. This ability to analyze opens doors for learners to research answers to their own questions as those questions are relevant.
Johns (1991) proposed teaching learners to analyze authentic target language sentences as a way to promote learner autonomy. The tool he used was a concordance. A concordancer is a simple piece of software designed to search through large, principled collections of text, pull out every instance of the search term, and list those instances for analysis. For example, a simple Key Word in Context (or KWIC) search for the term “who” produces a list of results such as the following (Figure 1): 
Figure. 1. An Example of a Concordancer
Concordancers have been used to gain insight into researchers’ first languages, and Johns was enthusiastic about the potential of such a research tool in the hands of second language learners. Tribble and Jones (1990) suggested a number of ways corpus results might be used in the classroom and Reppen (2010) gives an updated, easy-entry guide to using corpora in the classroom for beginners.
Since the early nineties a number of studies have shown the value of using a concordancer for vocabulary exploration (Cobb, 1997; Todd, 2001). Cobb’s study found that learning vocabulary with concordance results led to better retention than doing the same activities with a single example sentence. With concordancers, learners can quickly see their target vocabulary in authentic contexts. They can find out how frequent a particular word is and what other words tend to collocate with it. If the corpus is designed carefully of writing samples that match the target style of writing (academic essays or medical documents, for example) language variation can be taken into account and help inform writing choices (Reppen, 2010).
Along with vocabulary, it is also important to learn grammar in an authentic context. Ideally, learners could analyze concordance results to make conclusions about the target language structure, but this is a more complicated task than analyzing vocabulary. In order to use a concordancer to analyze language structure, learners must be able to find suitable language samples and look beyond the vocabulary to find patterns in the structure. To ease some of the difficulty when crossing over to structure analysis, instructors may find suitable concordance lines for the students, so that students need only focus on analyzing the language for a result that is certain (Tribble & Jones, 1990). While this certainly eases the burden on the student, it may hurt student autonomy.
Gaskell and Cobb (2004) attempted to mediate this challenge by starting their study by giving teacher-selected concordance results to students. Results were associated with a highlighted grammar error and the students then used these results to create a hypothesis about the target language that could correct their errors. Once students were familiar with analyzing the results, they were asked to use the concordancer to find their own samples to analyze. While the first half of the program was quite successful, fewer than half the students persisted in finding their own results in the second half of the program. Gaskell and Cobb noted that students who failed to persist tended to be lower level than the group who persisted, and also suggested that more training may have improved their numbers. Kennedy and Micelli (2011) conducted a case study where students were given extensive training with the concordancer and then used the tool to investigate vocabulary and grammar while writing. Two of their three participants succeeded in making the software their own by developing personal strategies for finding useful information through simple searches as described above. They both noted the value of searching openly through the corpus for expressions and phrasing that may not have occurred to them otherwise. The third participant, who struggled with the concordancer, noted difficulty with the user interface and a lack of confidence analyzing the resulting concordance lines.
While student use of concordancers still holds promise, what kind of training and how much training is needed to help students be fully successful with it is currently under investigation. Kennedy and Miceli (2010) had a full apprentice program that resulted in very promising use of a concordancer. Chambers (2005) argued against excessive training, pointing out that the goal was not to create full linguistic researchers. Even so, her own study into student use of concordancers required 9 hours of training before participants started her core project. Similarly, O’Sullivan and Chambers (2006) required 3 weeks of training before asking students to correct grammar errors in their own essays using a concordancer.
Unfortunately, studies investigating student use of concordancers for analyzing target language structure have been few and we are still a long way off from identifying what, if any training, can help students truly take command of the software to make their own inquiries into the target language. Another question that has not been asked is how much learners can do without training. While it is safe to assume training will not have a negative impact, and will likely improve learners’ effectiveness with a concordancer, there seems to be an unspoken assumption in student concordancing literature, like the studies mentioned above, that training is a prerequisite for using a concordancer. This case study calls that assumption into question. It asks the following questions:
Assuming virtually no prior training with a concordancer:
- What do students do intuitively with the software when peer editing for grammar with a concordancer?
- How do students react to the software as a tool for writing autonomy?
Methodology
Participants
This study involved two pairs of students from the upper-intermediate level of the Intensive English Language Program (IELP) at a University in the United States. All participants had TOEFL scores of 520 or better and eventually hoped to fully matriculate into a degree program at the university. All four participants were clearly motivated as evidenced by their participating in this study, which promised them nothing more than the opportunity to learn more about English.
For anonymity, pseudonyms (of the participants’ own choosing) were used. The first pair consisted of Edison (from South Korea) and Y (from Thailand). The second pair were Emmy (from South Korea) and Maria (from Thailand). First languages were mixed to promote English discussion during the work sessions.
Task
The study centered around grammar error correction in peer essays and therefore, before the study, three essays were prepared for the participants to correct. Each essay was written by the researcher but modeled after authentic persuasive essays that had come from students in the IELP and dealt with topics and essay styles the participants were familiar with. The introductory essay contained ten grammar errors styled after common errors found in authentic essays from students at the same level as the participants. The next two persuasive essays each contained five grammar errors styled after the most common grammar errors found in the level above the participants. These errors included articles, agreement, prepositions, tense and clause construction. All errors in all essays were highlighted but no indication was given as to the nature of the error. No other errors appeared in the essays.
Study
The participants took part in four sessions. The first session included an introduction to the task above and 30 minutes to practice using the concordancer while correcting the grammar errors in the introductory essay. The goal of the first session was to introduce the participants to the concordancer. Participants could ask the study coordinator if they had any questions about the technology or the task at hand. The coordinator showed them one term search at the very beginning of the study and pointed out how the resulting samples could be used to determine English structure. This initial introduction took no more than five minutes of the allotted 30 and the participants used the rest of the time to experiment. After the work session, participants had ten minutes to journal about their experience.
The next two sessions were the core of the study and were structured to match each other. Both began with a 15-minute interview to investigate journal comments more fully. Next, students were given an essay with five grammar errors (a different essay for each session) and asked to rate the errors for difficulty so as to gain insight into how much, if any, previous knowledge might be playing into their concordancer use. Once the errors were rated, the participants corrected the errors, using the concordancer if they chose. In the first of these two core sessions participants were given a worksheet to help guide them through the error correction process with the concordance. In practice, participants only filled in this worksheet after the error was fully corrected and their process was finished, so its intended use as a guide was null and did not figure into the results of this study. In the second session the worksheet was optional and participants chose not to use it. Both core sessions ended with another ten minutes for journaling. Finally, participants met for a fourth session. This final wrap-up session consisted of a final interview both about the previous journal writing and the experience as a whole.
Data collection
The results of this study were based on several different forms of data. The previously mentioned journal entries and video-tapes of the interviews were a primary source of participant opinions and insight into their work sessions. Video of the participants’ computer screen (showing their concordancer searches and results) along with audio of their discussion during work sessions was also used to track their thought process and strategies with the software itself.
Corpus and concordancer
A corpus of 80 persuasive essays was specially built for this study. All essays were taken from two free use essay sharing websites: 123HelpMe (www.123helpme.com) and All Free Essays (www.allfreeessays.com) and were verified to contain native-level English. The corpus itself was very small by modern standards but consisted of precisely the style of writing (though not necessarily the topic) the participants were working with. Participants used MonoConc 2.2 (Barlow, 2002) as their concordancer to search through the essays that had been collected.
Results
Perhaps not surprisingly, the participants were not at all intimidated by the simple search interface of the concordancer. Edison mentioned that it was not unlike doing Google searches, which he sometimes used when he had questions about English phrasing. All participants seemed comfortable with the software. Finding useful results and noticing useful patterns in the target language was considerably more difficult, however. Even so, between the two pairs of participants, 11 of a possible 20 errors were corrected with the concordance during the core work sessions. Keeping in mind that the students were not specially trained with the technology, participants used the concordancer to three different ends.
Student uses for the concordancer
1. Theory checking. The primary benefit participants received from the concordancer was as a theory-checking device. Both pairs of participants consistently discussed previous knowledge in order to predict how the highlighted error might be corrected. If they were at all uncertain about their answer, they would search in the concordancer for samples that backed up their theory. In this study, participants tended to be fairly confident of their answers before they checked, and used the concordancer successfully to confirm their theories a total of seven out of the 11 times between the two core work sessions.
In fact, the two pairs were able to confirm their theories a grand total of nine out of 11 times, but two of those theories (both for the same error) did not lead to a fully accurate correction in the end. The cause of this was unfortunately coordinator error. One grammar error highlighted in the first core work session, the couple live to an apartment contained two errors – agreement and preposition. Each pair noticed one of the two errors, but they had been told that each highlighted section contained one error, so they moved on after making one correction. Edison and Y noticed the preposition and submitted the correction the couple live in an apartment. Emmy and Maria noticed the agreement problem and submitted the correction the couple lives to an apartment. Even though their corrections were both improvements, the end sentences were not fully accurate.
2. Changing the original meaning. In one instance, Edison and Y were trying to correct the error they can save money for better future. Edison noticed the use of “future” and instantly thought of the chunk “in the future”. He was convinced that the sentence should be they can save money in the future. When Edison and Y checked this theory in the concordancer, they did find samples that used “in the future”. Y was uncomfortable with this answer, however, since it seemed to change the original meaning, but she couldn’t think of an alternate search to help her find a better correction. In the end, they decided to propose two different possible corrections. Edison stuck with they can save money in the future and Y proposed they can save money in the better future. Both changed the original intent of the writer.
3. Discovery learning. For the previously noted error corrections, the concordancer was successfully used to confirm what the participants believed they knew. While that may be helpful, for fully autonomous learning participants would ideally discover something new about the language as a result of their searches. For Edison and Y, this was never possible. However, Emmy and Maria had one error that led them into new territory. For the error they can save money for better future, the same error that had given Edison and Y trouble, Emmy and Maria thought they knew the answer when they turned to the concordancer. At first, they believed there should be an article, but when they typed in “future” there were no results. As it turned out, none of the articles in the corpus of essays used that word. Now, they were no longer confident that the problem was the article. They searched for “best future” and “better future”- these, too, with no results. Finally, no longer sure what the problem was, they typed in “better” alone. In the resulting sentences, they found the indefinite article “a” before better with a variety of other nouns following. They recognized the pattern and were able to accurately correct the error to read, they can save money for a better future.
Participant reactions
Participants had mixed reactions to the concordancer, though most of their complaints seemed to focus around using the concordancer for grammar error correction. After less than 90 minutes total using the concordancer, both pairs of participants commented that the concordancer seemed better suited as a reference during writing than for correction after the fact. Edison explained that while he was correcting errors, he had been focused on the narrow goal of finding a problem and fixing it, as opposed to exploring the target language. In retrospect, he felt that he had missed the point of the concordancer. Participants also likened the concordancer to other resources they use while they write, but they seemed to feel that the concordancer could fill a unique gap in their studies, especially since it represented authentic English from persuasive essays, which was the kind of writing they were expected to do for class. Edison noted that the results from the concordancer were similar to sentences in online dictionary searches but more useful. Interestingly, all participants asked if they could have a concordancer to take home after the study.
Even so, there was plenty of frustration with the technology. All four participants felt that it was challenging to interpret the concordancer results and they never felt confident they had come to the right conclusion, even if they did find results that supported their theories. They wished there were some way to get instant feedback on whether they had interpreted the results correctly. Edison suggested that it would be useful for students to use the concordancer at home, where they could relax, but then bring a journal of results to class to share with other students and get confirmation from a teacher.
All four participants were impressed that the corpus was made of persuasive essays of a native English level and more than one comment suggested that their faith in the source material was a key motivator for persisting with the technology. Emmy and Maria noted that when they did not get the results they expected for in the better future they both kept searching, even though they were frustrated, because they believed that if the proficient writers (represented by the corpus) had not written a particular term they were searching for, there must be another answer.
Discussion
The results of this study are tentatively positive. The participants benefitted from the concordancer by double-checking their theories and even doing a little exploration into the target language. Of course, for real autonomy, one would hope for more discovery learning to come from the interaction. It appears from the student reactions that part of the problem was the focus of the project, or rather, using the concordancer after writing for correction rather than during writing. It would be interesting to investigate in future studies how students might use a concordancer to inform their writing process, given a similarly basic level of training.
It is heartening to see participants showing so much enthusiasm for the technology in spite of frustrations. They clearly felt they were getting value from the concordancer even though they had little training. This suggests it is still worth investigating this technology for autonomous learning and especially as a tool during the writing process.
Even so, it is important to remember that these students were highly motivated and were already seeking technologies for autonomous exploration of the target language. They were also well versed in analyzing grammar structure. For this kind of student, simply providing access to or creating a relevant corpus of target language and providing them a concordancer may be enough. However, even for these highly motivated students, there were complaints that the technology was not guiding them enough. If students are not comfortable learning autonomously, they may need extra support to encourage them to investigate answers on their own, without an instructor to guide them. They may also need more structured activities in the beginning to illustrate what kind of information they can discover for themselves while using a concordancer (common collocates, expressions, structures, etc). It is also possible that some students may not be used to thinking of language as a structure. In this case, they may well need training and support in more basic grammar structure in order to understand the patterns they are uncovering below the lexical level when they investigate the target language with a concordancer.
It is also highly likely that with more sophisticated training with a concordance these highly motivated students would be able to get even more out of their corpus searches. The Kennedy and Miceli (2011) case study did provide significantly more training and did promote open searches while writing. As a result, two of their participants were able to delve into the concordancer results much more deeply than the participants in this study.
Conclusion
Student use of concordancers for autonomous investigation of the target language is not a new idea, but it is still under-researched. While the technology is not difficult to use, the results can be intimidating and off-putting. Sample sentences are composed of authentic language which may be complicated, and users must be able to recognize the structure below the sample sentences for themselves. However, learners need exposure to authentic language if they aspire to push their language ability to an advanced level. Learners also need the confidence to develop theories about the target language autonomously while still having an opportunity to check those theories later for accuracy.
As proponents of autonomous learning, we need to help learners find opportunities to answer their own questions and give them the confidence to find their own answers independently when they need them. A concordancer is one tool available that can expose learners to the target language in an organized fashion to help them answer some of these questions and inform their own manipulation of the target language. There are still many questions that remain to be answered as we try to find the best way to introduce concordancers and corpus searches to learners, but there are useful things learners can do with this technology with little training. There is still plenty of reason to be excited by the prospect of concordancers for autonomous language exploration, especially as it relates to writing.
Notes on the contributor
Jacqueline Mull is Vice Dean and Lecturer at the Research Institute of English Language Education in Kobe, Japan. Her research interests include corpus linguistics, materials development and assessment.
References
Barlow, M. (2002). MonoConc Pro (Version 2.2) [Computer software]. Houston, TX: Athelstan.
Chambers, A. (2005). Integrating corpus consultation in language studies. Language Learning & Technology, 9(2), 111-125. Retrieved from http://llt.msu.edu/vol9num2/pdf/chambers.pdf
Cobb, T. (1997). Is there any measurable learning from hands-on concordancing? System, 25(3), 301–315. doi:10.1016/S0346-251X(97)00024-9
Dam, L. (2011). Developing learner autonomy with school kids: Principles, practices, results. In D. Gardner (Ed.), Fostering autonomy in language learning (pp. 40-51). Retrieved from http://ilac2010.zirve.edu.tr/Fostering_Autonomy.pdf
Gaskell, D., & Cobb, T. (2004). Can learners use concordance feedback for writing errors? System, 32(3), 301-319. doi:10.1016/j.system.2004.04.001
Johns, T. (1991). Should you be persuaded: Two samples of data-driven learning materials. English Language Research Journal, 4, 1-16. Retrieved from http://wordsmithtools.net/wordsmith/corpus_linguistics_links/Tim%20Johns%20and%20DDL.pdf
Kennedy, C., & Miceli, T. (2010). Corpus-assisted creative writing: Introducing intermediate Italian learners to a corpus as a reference resource. Language Learning & Technology, 14(1), 28-44. Retrieved from http://llt.msu.edu/vol14num1/kennedymiceli.pdf
Legenhausen, L. (2011). The place of grammar in an autonomous classroom: Issues and research results. In D. Gardner (Ed.), Fostering autonomy in language learning (pp. 27-38). http://ilac2010.zirve.edu.tr/Fostering_Autonomy.pdf
O’Sullivan, I., & Chambers, A. (2006). Learners’ writing skills in French: Corpus consultation and learner evaluation. Journal of Second Language Writing, 15(1), 49–68. doi:10.1016/j.jslw.2006.01.002
Reppen, R. (2010). Using corpora in the language classroom. New York, NY: Cambridge University Press.
Todd, R. W. (2001). Induction from self-selected concordances and self-correction. System, 29(1), 91-102. doi:10.1016/S0346-251X(00)00047-6
Tribble, C., & Jones, G. (1990). Concordances in the classroom: A resource book for teachers. Harlow, UK: Longman.
SiSAL Journal 