Craig Yamamoto, Chugoku Gakuen University, Japan
Yoko Kinoshita, Sojo University, Japan
Yamamoto, C., & Kinoshita, Y. (2019). Self-assessment surveys – A tool for independent learning in lower-tier dependent classrooms. Studies in Self-Access Learning Journal, 10(3), 296-318. https://doi.org/10.37237/100306
Download paginated PDF version
Abstract
This project took place over an academic year, using self-assessment surveys as a unique tool for guiding students to engage in the learning process, with the goal of improving their motivation and overall awareness in second language studies. The participants (n=94) were first year students from a private Japanese university from the departments of Aerospace and Systems Engineering (n=11), Architecture (n=11), Computer Information Sciences (n=49), Life Sciences (13) and Mechanical Engineering (n=10). The surveys were bookended with two similar Likert-scaled questionnaires. The first was used as a baseline of participants’ initial beliefs, motivation and overall awareness regarding their L2 education, while the second was used as a comparison, through a paired sample T-test, to mark any changes. The self-assessment survey sought to elicit students’ knowledge and confidence in areas of vocabulary, use of English and usage of the target language, with a survey report given to everyone prior to unit assessments. The following is a report on the effect self-assessment surveys had on students’ motivation and overall awareness when used as a learning tool promoting autonomous learning for teachers and advisors wishing to empower students to take a more prominent role in their L2 education.
Keywords: self-assessment, motivation, learner autonomy, CALL
Japanese students are traditionally exposed to a passive learning environment from the beginning of their primary education through secondary education and even throughout much of their post-secondary education, a characteristic Aspinall (2008, p. 151) labeled “Deference to the authority of the teacher”. As a result of recent reforms by the Ministry of Education, Culture, Sports, Science and Technology, schools are working to “Enrich educational content in relation to nurturing individual’s sense of Japanese identity,” (MEXT, 2014, p. 1) and attempting to change the view of Japan being a collectivist society. However, students at the beginning of this project were incapable of performing task-based activities, such as writing a self-introduction to present to different partners, without heavy support from the teacher, suggesting they lacked the ability to work autonomously.
Self-assessment is a popular tool for individuals to recognize their achievements and failures, but more importantly for lower-tier EFL classrooms it can provide a better understanding of difficulties. Moreover, it allows for opportunities to develop learner autonomy by actively involving students in the learning process (Dam, 2009). Through previous research regarding autonomy, it was concluded that true beginners “lacked the ability to communicate in the L2 at a level where they could reflect on their effort and the content of the lessons/activities.” (Meilleur, Yamamoto, & Franz, 2014, p. 129). Hence, the idea of a self-assessment survey could prove to be more efficient in identifying students’ needs and abilities while reducing hindrance from the inability to communicate effectively in the L2, enabling teachers to give more appropriate guidance for those attempting to become independent or autonomous learners. As Chen stated, “Through the process of self-assessment, students learn to discern patterns of strength and weakness that can help them become better learners. Equipped with self-assessment skills, students gradually develop a critical attitude toward learning throughout their lives and in the long run achieve the fullest autonomy” (2008, pp. 237-238). In accordance with Chen (2008), the self-assessment surveys were used not only to assist students in identifying strengths and weaknesses, but also as a tool to take a more prominent role and to engage more directly in the learning process, with the ultimate goal of becoming autonomous learners.
In Ross’s (2006) study attempting to understand if self-assessment improves student performance, four stages for training students in self-assessment were applied:
(i) involve students in defining assessment criteria
(ii) teach students how to apply the criteria
(iii) give students feedback on their self-assessments
(iv) help students use assessment data to develop action plans
This project explores three of the stages (ii-iv) through self-assessment surveys, excluding the first stage, as it does not accurately align with the structure or the level of the participants in this project. Through this paper, the researchers will demonstrate how teachers can guide students to engage with the learning process through the use of self-assessment surveys, in an attempt to increase motivation and overall awareness, leading them towards becoming autonomous learners.
Research Questions
The researchers proposed the following questions:
- Can self-assessment surveys provide guidance to lower-tier students to become autonomous learners?
- Can student motivation to improve language skills and overall awareness of the learning process increase by accessing their metacognitive skills through the use of self-assessment surveys as a tool for learning?
Literature Review
Development of metacognitive skill in a classroom
Little defined autonomy as a “capacity – for detachment, critical reflection, decision-making, and independent action.” He also mentions that autonomy has been commonly referred to as a “measure of independence from the control of others,” giving a sense of freedom. Recognizing the possible misconception regarding autonomy, he emphasized that such freedom is “never absolute, always conditional and constrained” (1991, pp. 4-5). Self-assessment is one of the essential factors of metacognitive strategies in language learning and it is an indispensable part of the area of developing learner autonomy. For decades, educators and researchers have been discussing the definition of learner autonomy, with many theories emphasizing the importance of development of learners’ metacognitive skills in order to help learners to be independent (Anderson, 2008). Holec (1981, p. 3) noted that “taking charge of one’s own learning involves planning, selection of materials monitoring learning progress and self-assessment.” In other words, without metacognitive strategies, learners will not be able to take charge of their own learning. O’Malley and Chamot (1990, p. 8) also emphasize that, “students without metacognitive approaches are essentially learners without direction or opportunity to plan their learning, monitor their progress, or review their accomplishments and strategize future learning.” Therefore, implementation of a monitoring system and self-assessment to encourage a metacognitive approach could be deemed beneficial. Tholin (2008, p. 9) mentions that since the meaning of ‘Learner Autonomy’ became broad and vague, and came to mean different things for different teachers, some researchers, including himself, prefer to put their attention on ‘self-directed language learning’ instead. According to Tholin (2008, p. 9), self-directed language learning can be defined as “learners wanting to, and being able to, assume a great responsibility for their own learning.” By accessing metacognitive skills, learners can evaluate their progress and determine their needs, and work out a plan for improvement. Although there are some conflicts surrounding the terms “autonomy” and “self-directed language learning”, they both share the common objectives, which are developing learners to be more aware of their own learning and to have a responsibility for planning, monitoring and reflecting on their learning, and these form the basis of language learning.
Self-Assessment and learner development
Similar to Little’s (2005) idea that a learner-centered curriculum is incomplete without self-assessment and shared responsibility, Hirschel, Yamamoto, & Lee (2012, p. 293) state that, “an integral part of autonomous learning, regardless of how we may define the term, is some measure of autonomous assessment or self-assessment.” In Everhard’s (2018) study, she discussed research showing how assessment can have an effect or influence on autonomy, by giving customarily teacher responsibilities to learners to evaluate their achievements, changing their attitudes and roles as learners, or what the researchers call self-assessment. Tholin (2008, p. 10) describes ‘self-assessment’ as a natural element of ‘self-directed language learning’, which is also an essential tool for language acquisition. By consciously working on ‘self-assessment’, learners will be able to make choices that contribute to the development of their own learning. In addition, Todd (2002, p.17) mentions that self-assessment helps learners “increase learning motivation and goal orientation.”
In Japan, it has been considered quite normal for learners to depend on teachers regarding assessment processes, and is a probable cause for Japanese learners’ tendency to value the result of tests and final grade rather than their own learning process. Beginning in 2002, Japanese learners from grades 3-9 were required to take Integrated Studies classes, which are to encourage autonomous learning, through cooperation with teachers on projects and activities, but are not part of the high school curriculum (Ellington, 2005). Many educators in Japan consider these classes as negatively affecting education, causing the Ministry of Education to make reassessments (Ellington, 2005). Therefore, students have received little knowledge how to assess their own learning process and usually depend on the assessment from a teacher or a third-party. Munoz and Alvarez (2007, p. 4) commented that students lacking experience in self-assessment commonly have a lack of understanding of the assessment process; a lack of objectivity and reliability about their work; and reluctance to do something that they think is a teacher’s duty.
Although there seem to be a number of benefits regarding self-assessment, Cardoso (2010, p. 24) argues that, “…mainstream curricula are still very far from being learner-centered.” English education in Japan is not the exception. As he points out, although the implementation of a learner-centered environment in English classrooms has been flourishing among current language teachers, there often seems to be some degree of distance between reality and ideology. Such distance between reality and ideology may be caused by the educational system in Japan as well as the presence or absence of metacognitive strategies. Little (2005, p. 322) acknowledges that in these situations, it is difficult to expect students to accurately assess themselves, so “self-assessment depends on a complex of skills that must be mediated by the teacher, often in very small steps.” Therefore, to lead learners, regardless of their proficiency level, in the direction that is best for them as a language learner, it could be useful for teachers to incorporate self-assessment surveys.
Developing learner autonomy has been recognized as an indispensable part of language teaching in Japan. Many educational institutions have established Self-Access Learning Centers encouraging learners to access them regularly to study the L2 independently, but it is questionable whether or not learners who are not properly skilled in autonomous learning have the ability to take full advantage of such facilities. Sakai (2008, p. 48) mentioned that according to the study of the Test in Practical English Proficiency or EIKEN Test (英検 [Eiken], n.d.), learners at the 5th level to 3rd level are categorized as dependent learners with a Common European Framework of Reference (CEFR) (Council of Europe, 2007) equivalency level of A1-1 to A1-3, while learners from pre-2 (A2) to 2 (B1) levels are characterized as independent learners, and pre-1 (B2) to 1 (C1) levels are categorized as independent users (英検 [Eiken], n.d.). This suggests learners of higher proficiency have higher autonomy and learners without learning strategies or methods of learning tend to lose confidence in language learning since they cannot improve their performance effectively. Sakai (2008, p. 52) presumed that it is essential for learners to acquire “the foundation of learning English,” and without it, we cannot expect educational effectiveness. Therefore, these researchers believe through the introduction of self-assessment surveys, lower-level students can begin to develop effective language learning skills.
Research Method
Participants
In this project the participants were drawn from four classes of first-year university students from a private Japanese university of sciences and engineering. All participants were required to undertake two 90-minute classes per week per semester for two semesters, totaling thirty weeks. The participants were a combination of lower-tier classes from the Faculty of Aerospace and Systems Engineering, Architecture, Computer Information Sciences, Life Sciences and Mechanical Engineering totaling 94 students (6 female, 88 male), studying English communication as a compulsory course in the first four semesters at this particular university. Tiers were based on scores from their initial Visualizing English Language Competency (VELC) Placement Test taken upon entrance into the university. According to the results of the placement test, all students in the aforementioned classes scored in the 300-point band except one, who scored 298 points or 205-point TOEIC range (Visualizing English Language Competency Test, n.d.) Therefore, in comparison to the more familiar Common European Framework of Reference, the participants rated at an A1 or possibly a lower A2 level (Language Enhancement Center, n.d.). All the students voluntarily completed online agreements allowing the use of all information necessary to conduct the research with the understanding that all the participants would remain anonymous after receiving an explanation from the researchers as to the purpose of the research and the expectations of the students throughout the process.
Research design and procedures
This project was a year-long study of self-assessment surveys in English Communication courses I and II from April 2017 to January 2018, bookended with translated Likert scale pre- and post-questionnaires administered through Survey Monkey and Google Forms. The aim of this project was to investigate the implementation of self-assessment surveys in lower-tier classes and the ramifications in accordance with guiding students in autonomous learning. The pre-questionnaire was based on a 6-point Likert scale (1=Strongly disagree, 2=Disagree, 3=Somewhat disagree, 4=Somewhat agree, 5=Agree, 6=Strongly agree) to determine students’ perceptions of their own learner awareness and motivation. The post-questionnaire was also a 6-point Likert scale measurement. It was used to analyze any changes in students’ behavior and habits regarding becoming more proficient in English. This instrument included open-ended items to get students’ opinions on the effectiveness of self-assessment surveys as a tool for learning. Students were encouraged to leave comments and to comment openly in Japanese to eliminate any L2 interference, which would then be divided into categories based on positive, negative and unrelated responses. Students were also informed that their responses to the pre- and post-questionnaires along with the Self-Assessment Surveys would not be reflected in their class scores.
The self-assessment surveys were completed at the end of each content-based lesson, therefore excluding orientation lessons, test preparation and testing days. Each survey was designed to correspond to the specific lesson with five questions focused on vocabulary, grammar, usage of English, troublesome areas and complete homework, which will be discussed more in the next section. Lessons were planned to take 90-minutes, but occasionally a lesson took as much as 120 minutes to complete due to students’ inability to progress smoothly, which was left to the discretion of the teacher. The method for administering the surveys was through QR codes, developed in 1994 by Masahiro Hara of Denso Wave (formerly Denso Corporation) (QR code.com, n.d.). Students who have any type of mobile device including a traditional “flip-phone” have quick access to a QR code reader, allowing them to scan the QR code at the bottom of their handout in a matter of seconds, viewable in Appendix A.¥
Background – self-assessment survey contents
The five-question checkbox design of the self-assessment survey breaks down each lesson based on necessary vocabulary, grammar and communicative activities that would meet the objectives indicated in the course syllabus. Based on students’ limited knowledge of vocabulary words not included in the New General Service List (NGSL) most commonly used 2000 words (Browne, Culligan, & Phillips, n.d.) were mostly omitted. The NGSL is “based on a carefully selected 273 million word subsection of the 1.6 billion-word Cambridge English Corpus (CEC)”, (Browne, 2013, p. 14) which was viewed as the most appropriate guide for the researchers to determine the words to be used. Through the use of the NGSL the number of vocabulary words necessary at this level was reduced by approximately 50-70%. The researchers also identified one grammatical structure that could be used or added to previously covered material to meet the current communicative objective. To help simplify the use of English (or grammar), the researchers began referring to the Common European Framework of Reference – Japan (CEFR-J), an adaptation of the original Common European Framework of Reference focusing on English language teaching in Japan (Tono & Negishi, 2012).
Conditions
In conjunction with the first self-assessment survey the pre-questionnaire, shown in Appendix B, was administered after the first content-based lesson in May 2017, to set a baseline to determine students’ current motivation to improve their language skills and overall understanding of the learning process as well as get an understanding of the technology they were accustomed to using to best administer future surveys efficiently. The results would later be used in a paired sample T-test with the post-questionnaire (shown in Appendix C) given at the end of the final content-based lesson of the academic year in January 2018 to determine any significant differences regarding motivation, overall awareness and signs of independent learning. The self-assessment surveys, shown in Appendix D, were administered through the use of QR codes in the last 3-5 minutes of each lesson. Aligned with stages ii-iv of Ross’s four stages of training students in self-assessment (2006, p. 4-5), (ii) students were given the opportunity to reflect on the vocabulary and grammar points of each lesson as well as areas they wanted or needed to focus their studies on. The surveys were also designed to serve as a reminder to students of their completed homework from the previous week. At the end of each unit, approximately every third or fourth week, (iii) all the data were collated and shared individually in a “self-assessment report” (see Appendix E). (iv) individualized electronically generated feedback was given on each report for students to reflect on guiding them to take the next step in becoming autonomous learners. Feedback included initial positive support based on students’ responses, such as “Good job” or “Keep trying,” followed by suggested steps to improve, including using the available resources in the university self-access center and making appointments to see English learning advisors. This information was scrutinized and reflected on by the researchers to determine feasibility and consider ways to improve and adapt the surveys in the future for possible addition to English Communication curricula and/or development of new curricula.
When the research project began it was determined that each self-assessment survey would consist of five parts (see Appendix D). Q1 would focus on the particular lesson vocabulary; Q2 would focus on use of English (or grammar); Q3 was to determine completion of the lesson objective; Q4 was a brief reflection of individual performance; and Q5 was a reflection based on students’ ongoing homework. In order to answer the first research question, shown in Appendix B, all students were asked to scan a QR code linking them to a self-assessment survey.
The second research question targets student engagement in the learning process through the use of self-assessment surveys. Therefore, all participants were asked to complete one self-assessment survey for each lesson pertaining to the specified curriculum. At three separate points throughout the semester, students were given a self-assessment survey report, as shown in Appendix E, to assist them in preparing for their upcoming unit quiz without needing further coaching from the teacher regarding what to study. The researchers later used the reports as a measurement to help answer item 2 from Appendix B. The purpose of this report was for students to utilize the information from their self-assessment surveys to identify what parts of the particular unit they should focus their attention on in order to perform adequately on their upcoming unit quiz. All vocabulary words in column one are words each student identified as ones they did not know prior to beginning the specified lesson. Columns two and three consist of general feedback based on the individual marks (1: “Need to work on” to 5: “Very well”) students gave regarding their comprehension of the grammar and ability to use the vocabulary and grammar communicatively. Column four was used as a reminder to the students as to what area they had difficulty in for that particular lesson. Lastly, column five of the report was a second reminder indicating to students what ongoing homework activities they had done recently.
In order to assess the results of the project, two survey instruments would be given to the students, one at the end of the first content-based lesson in May and one at the end of the last lesson of the academic year in January. The researchers began the first content-based lesson doing the initial self-assessment survey with basic verbal instructions: take their time, ask for assistance with how to properly complete the survey and be honest about their answers with reassurance that their surveys will not affect their official scores for the course, and would be kept confidential, not to be shared among the class or with other teachers outside of this report.
Survey instrument one
The initial survey instrument consisted of nine statements — three focusing on personal electronic devices being used by the participants and six 6-point Likert scale statements to reflect on the participants’ interests pertaining to the research project contents. Similar to the Common European Framework of Reference ‘Can do’ Statements (Council of Europe, 2007), the survey items were written as statements to give all participants a feeling of ownership and personal opinion rather than a teacher assigned interview with expected goals and outcomes. The first three items were used to predetermine any issues that could arise from using personal electronic devices as the source for collecting data. Items 1-6 were designed to set a baseline of the participants’ initial beliefs, motivation and overall awareness (see Appendix B).
Survey instrument two
All participants completed the second survey instrument at the end of the semester, shown in Appendix C. Items 1-5 coincide with the items 1-5 of survey instrument one enabling the researchers to use the results as a comparison of participants’ own perception of their motivation and overall awareness regarding their learning. Item 6 was used to determine students’ initiative to autonomously complete semester long assignments, such as online vocabulary activities and activities involving the university self-access center. Items 7-14 focused on the self-assessment surveys’ effectiveness, with items 8 and items 11-14 being open-ended or having options for students to comment in order to gain further insight into student perceptions of this particular project.
Results
As shown in Table 1, the information collected from survey instrument one demonstrated 98.9% of students possessed smartphone devices, while one student owned a standard mobile phone. 96.8% of students stated their phones were capable of reading QR codes, though after guidance from the teachers, 100% of participants understood that their phones possessed the necessary capabilities. The participants also showed they only use QR codes in their daily lives to a minimal degree.
Table 1
Survey Instrument One Results
The two-tailed paired sample T-test descriptive statistical comparison for survey instrument one and survey instrument two is presented in Table 2. Degree of freedom varied in item 5 due to one student not completing both questions on each survey, eliminating those responses. Through a paired sample T-test the results indicate an increase in all but one item (Item 3). The increases indicated items 5 and 6 achieved gains that were statistically significant at the 0.05 level. Therefore, it can be concluded that the participants believed by the end of the project, their interest in improving their language skills increased and they were better able to comprehend the lesson objectives and progress through their homework to a significant degree through the use of self-assessment surveys.
Pedagogical Implications
Quantitative analysis
Information in the descriptive statistical comparison for item 2, regarding the self-assessment survey being a useful tool to help improve English, initially although positive, may seem insignificant (p-value=0.12). However further study may suggest students’ somewhat positive response in survey instrument one (μ=4.02, SD=1.24) was based on positive assumptions of the course. Survey instrument two results may be a reaffirmation of the initial positive students’ assumptions (μ=4.23, SD=0.93).
Items 3 and 4 were intended to reveal any increase in motivation to improve their language skills regarding studying vocabulary and grammar, with participants mostly taking a neutral position regarding their motivation to study. Although the responses in survey instrument one (μ=4.03, μ=3.87) were slightly higher than expected, response in survey instrument two (μ=3.96, μ=3.94) suggest the participants’ motivation to study the vocabulary through the use of the self-assessment surveys did show minimal decline, it also showed increased motivation to study grammar.
Survey instrument one, tended to elicit a somewhat neutral response from students indicating their lack of confidence in their ability to identify the main point of the daily lessons. In retrospect, this lack of understanding of the process as well as their lack of objectivity and confidence in their own assessment at the beginning of the project, correspond with Munoz and Alvarez (2007, p. 4) – traits typical of students’ inexperienced in self-assessment. Regardless, the results from survey instrument one were followed by significant statistical gains (p-value=0.01) in comparison with survey instrument two. Item 5, not only indicates students understanding of the lesson objectives, it may also indicate that students are more confident in their ability to accurately recognize the most important aspects of each lesson.
Table 2
Descriptive Statistical Comparison
Descriptive analysis
Analyzing item 6, shown in Table 3, tells us that students were slightly positive in their response to tracking and taking initiative to autonomously complete homework assignments, which also included students spending time in the university self-access center, to complete their homework, accessing available resources, or getting support from an English language advisor during their free time. The research does not prove that through the use of self-assessment surveys alone students are more autonomous, but the researchers can assume the possibility that through the feedback in the Self-Assessment Survey Report, students have been influenced to take a more prominent role by spending time in the self-access center outside of class.
Items 7-11 were used to analyze the survey’s usefulness, contents and students’ reflection of the overall project. Along with open-ended items 12-14, item 8 and item 11 included an open-ended comments section for students to reflect openly about the project (see Appendix C). Students were encouraged to write their comments in Japanese to eliminate any L2 interference. The comments were grouped into three categories, positive, negative and unrelated by the researchers. Positive responses were any comments suggesting a positive, as opposed to negative responses suggesting dissatisfaction. Unrelated responses were anything unable to be categorized as positive or negative or unrelated to the item, including responses that the researchers were unable to comprehend due to L2 interference.
In conjunction with the idea of self-assessment surveys to be used as a learning tool, items 7 and 8 demonstrate the participants’ responses as neutral to positive (μ=4.19, μ=3.68) when directly involving them in the learning process. Comments left by students in item 8 show 80% (32/40) illustrated a sense of motivation or satisfaction with responses such as “少しできるようになった気がする (I feel I can do it a little more); fantastic; 英語に対して自信がついた (I have gained more confidence in English); and 前より英語をやろうという気になれた (I feel more motivated to study English than in the past).” (translations provided by the researchers) Such responses are similar to Todd’s (2005, p.17) findings on self-assessment for evaluation and goal orientation in learning. Three comments were deemed negative in the sense that they did not reflect a positive outcome with comments “after all English is difficult and I am not very confident; まだまだ未熟です (I am still inexperienced); and …自己評価できるかわからなかった (…did not know if I could self-assess properly or not).” The remaining five comments were deemed as unrelated responses such as “English is difficult” and “ok.”
Item 11 included an open-ended comments option regarding students’ interest in continuing the use of the self-assessments. Based on 42 comments, 28 students gave positive responses, such as “自己評価で自分のダメな所がわかるのはいいと思います (Understanding my weak points through the self-assessment surveys was good)”, with four negative comments, two directly related to a formatting error in the last of 13 total surveys stating “最後のほうsalkなど複数やったけど１つしか選択できないことがあった (The last question, I did multiple activites such as SALC homework, but the question only allowed one option)”. The remaining 10 comments such as “Thank you,” were classified as neutral or unrelated.
Table 3
Survey Instrument Two – Descriptive Analysis
Viewable in Appendix C, item 12 asked students to give their input for improving the survey with 32 students commenting they felt it was best left in it’s current version, while three students suggested adding it to the class Moodle page or use paper versions and one student suggesting to do it more often, suggesting positive views of the usefulness of the self-assessment surveys used in the study. students’ When viewing the students’ comments for items 13 and 14, there was considerable overlap in regarding what they enjoyed and what they found useful such as, “knowing not understanding points; 結果が数字としてわかりやすく見える (easy to visualize my results); 自分の足りない部分がよく分かる (easily understand my weak points); and 勉強がしやすくなった (it became easier to study).”
Limitations
When students are required to access information individually using new or unfamiliar technology there will always be questions as to the reliability of their responses. In some instances this could be an issue in MALL (Mobile Assisted Language Learning) where students are unfamiliar with using their smartphone devices to answer an English survey or what Fageeh called a “cognitive burden” (2011, p. 23). Given the parameters of a typical classroom and the technology available, QR codes still seem to be the best available option for administering the survey to any number of students within a short period of time.
A second and more obvious limitation to such a study pertains to the students’ ability and objectivity when completing the surveys. Students may misinterpret the instructions of the activity, mistype answers, send incomplete information or mistakenly not send information at all. Some students might have an innate belief they should know all the information, because they practiced in class or have studied the material in the past causing them to incorrectly give themselves a very positive evaluation, while others may have a negative perception of themselves or the course improperly giving a lower evaluation than what may be true. There have also been studies concerning students from collectivist cultures such as Japan or other Asian countries, who share more modest or socially sensitive opinions, showing the tendency to avoid extreme options, hence choosing neutral responses over more extreme ones (Wang, Hempton, Dugan, & Komives, 2008, p. 2).
Future Research
Although there were positive outcomes, the researchers in this instance will need to investigate reasons for insignificant changes regarding motivation to study vocabulary and use of English, although students’ overall awareness improved. Through further analysis of the surveys, the researchers will discuss improvements to the Self-assessment Survey Report, regarding more clearly stated feedback for students that will hopefully increase motivation in the previously mentioned areas. It will also be necessary to study the effects on standardized examinations in comparison to a control group and the overall effectiveness of using electronically administered self-assessment surveys pedagogically.
Conclusion
Although the researchers have outlined a number of limitations, the project has garnered some statistically significant results when analyzing the descriptive statistical comparisons of the two survey instruments. Through these results an argument can be made regarding the effectiveness of self-assessment surveys as a learning tool and their implementation in EFL courses. In accordance with Todd’s (2002) research on self-assessment for evaluation, the development and implementation of this project has also culminated in a revisal of the curriculum creating a more level-appropriate course for the lowest band of students at this particular Japanese university, furthering the pedagogical importance of self-assessment surveys. Through the findings from this project, we would hope over a longer period of time the results would be similar or even improve particularly in students’ ability to self-assess and the surveys’ usefulness as a learning tool.
The introduction of self-assessment surveys as part of the lessons, was viewed positively by students as demonstrated by their interest to continue using self-assessment surveys in future English courses, which can be viewed as a positive response to Research Question 1 in providing guidance, along with previously stated comments such as, “自分の足りない部分がよく分かる (I can easily understand my weak points); and 勉強がしやすくなった (It became easier to study).” It is possible that this best exemplifies how the researchers effectively implemented self-assessment surveys in lower-tier classes. Although, the research did not produce significant gains in using self-assessment surveys as a tool for learning, through the positive comments given, it can be deduced that, regardless of the gains, students showed positive feelings in this area, further indicating the effectiveness of the surveys.
In accordance with theories on the importance of developing metacognitive skills to assist independent learning (Anderson, 2008), elicited comments in items 6-14 of post-questionnaire (shown in Appendix C) demonstrate students began accessing those skills and work autonomously, suggesting there is a relation, which helps to answer Research Question 2. Moreover, as O’Malley and Chamot (1990, p. 8) emphasized, use of the Self-Assessment Survey Report along with accessing their metacognitive skills was beneficial to students as a way to review and plan their steps to learning. Students were seemingly more aware and willing to engage in the learning process by taking a prominent role to improve their language skills, autonomously taking advantage of the university self access learning center for resources and services, steps which could begin to characterize these students as independent learners according to Sakai (2008, p. 48). The researchers suggest that, students’ motivation to improve their language skills and overall awareness to take control of the learning process could benefit through continued use of Self-Assessment Surveys and Self-Assessment Survey Reports, when used as a learning tool.
Notes on the Contributors
Craig Yamamoto is an associate professor at Chugoku Gakuen University in Okayama, Japan. He has been teaching and training teachers in EFL/ESL since 1996 at the primary, secondary, and postsecondary levels. His research interests include assessment, curriculum development, learner autonomy, motivation and teacher training.
Yoko Kinoshita is a senior assistant professor at Sojo University in Kumamoto, Japan. She has been teaching classes as well as working as a learning advisor in the university’s Self Access Learning Center since 2010. Her research interests include learner autonomy, motivation and development of self-access centers.
References
Anderson, N. J. (2008). Metacognition and good language learners. In C. Griffiths (Ed.), Lessons from good language learners (pp. 99-109). Cambridge, UK: Cambridge University Press. https://doi.org/10.1017/CBO9780511497667.010
Aspinall, R. (2008). Foreign language education policy in Japan and England: Problems of teaching foreign language communication in island nations. Japanese studies around the world 2007, 14, 147-169. http://doi.org/10.15055/00003732
Browne, C. (2013). The new general service list: Celebrating 60 years of vocabulary learning. The Language Teacher, 37(4), 13-16. Retrieved from https://jalt-publications.org/files/pdf-article/37.4tlt_featureds.pdf
Browne, C., Culligan, C., & Phillips, J. (n.d.). New General Service List: The most important list for second language learners of English. New General Service List. Retrieved from http://www.newgeneralservicelist.org
Cardoso, W. C. (2010). Learner autonomy and self-assessment: Indispensable tools for successful learning. Instituto Cervantes de Sao Paulo. Retrieved from https://authenticteaching.files.wordpress.com/2010/10/new-routes-learner-autonomy-and-self-assessment.pdf
CEFR-J. (2012). CEFR-based framework for ELT in Japan. Retrieved from http://www.tufs.ac.jp/ts/personal/tonolab/cefr-j/english/download.html
Chen, Y. (2008). Learning to self-assess oral performance in English: A longitudinal case study. Language Teaching Research, 12(2), 235-262. https://doi.org/10.1177/1362168807086293
Council of Europe. (2007). Common European Framework of Reference for Languages: Learning, teaching, assessment. Cambridge, UK: Cambridge University Press.
Dam, L. (2009). The use of logbooks — a tool for developing learner autonomy. In Pemberton R., Toogood, S., & Barfield, A. (Eds.), Maintaining control: Autonomy and language learning (pp. 125-144). Hong Kong, China: Hong Kong University Press.
英検 Eiken [English Examination]. (n.d.). Eiken grades level comparison. Retrieved from http://www.eiken.or.jp/eiken/en/grades/
英検 Eiken [English Examination]. (n.d.). Overview of the EIKEN tests. Retrieved from http://www.eiken.or.jp/eiken/en/eiken-tests/overview/
Ellington, L. (2005). Japan digest – Japanese education. Retrieved from https://spice.fsi.stanford.edu/docs/japanese_education
Everhard, C. J. (2018). Re-exploring the relationship between autonomy and assessment in language learning: A literature overview. Relay Journal, 1(1), 6-20. Retrieved from https://kuis.kandagaigo.ac.jp/relayjournal/issues/jan18/everhard/
Fageeh, A. (2011). EFL students’ readiness for e-learning: Factors influencing e-learners acceptance of the Blackboard™ in a Saudi university. The JALT CALL Journal, 7(1), 19-42.
Hirschel, R., Yamamoto, C., & Lee, P. (2012). Video self-assessment for language learners. Studies in Self-Access Learning Journal, 3 (3), 291-309. Retrieved from https://sisaljournal.org/archives/sep12/hirschel_yamamoto_lee
Holec, H. (1981). Autonomy in foreign language learning. Oxford, UK: Pergamon.
Language Enhancement Center. (n.d.). CEFR mapping of the Oxford Online Placement Test to international English tests. Retrieved from https://lec.payap.ac.th/more-about-the-cefr-and-oopt/
Little, D. (1991). Learner autonomy 1: Definitions, issues and problems. Dublin, Ireland: Authentik.
Little, D. (2005). The Common European Framework and the European Language Portfolio: Involving learners and their judgements in the assessment process. Language Testing, 22, 321-336. https://doi.org/10.1191%2F0265532205lt311oa
Meilleur, R., Yamamoto, C., & Franz, T. (2014). Manufacturing autonomy: An L2 reflective writing project. Sojo University Journal, 39, 123-131.
MEXT. (2014). English education reform plan corresponding to globalization. Retrieved from http://www.mext.go.jp/english/topics/__icsFiles/afieldfile/2014/01/23/1343591_1.pdf
Munoz, A., & Alvarez, M. (2007). Students’ objectivity and perception of self assessment in an EFL classroom. The Journal of Asia TEFL, 4(2), 1-25.
O’Malley, J. M., & Chamot, A. U. (1990). Learning strategies in second language acquisition. Cambridge, UK: Cambridge University Press. https://doi.org/10.1017/CBO9781139524490
QR code.com. (n.d.). History of the QR code. Retrieved from http://www.qrcode.com/en/history/
Ross, J. A. (2006). The reliability, validity, and utility of self-assessment. Practical Assessment Research & Evaluation, 11(10), 1-13. Retrieved from http://pareonline.net/getvn.asp?v=11&n=10
Sakai. S. (2008). 英語教育における自律した学習者養成とICT. [Eigo kyōiku niokeru jiritsu shita gakushū-sha yōsei to ICT]. [Independent Learner Training and ICT measured in English education]. Journal of Multimedia Aided Education Research 2008, 5(1), 45-56.
STEP. (2006). 英検合格者の「英語学習・英語使用状況調査」の報告. [Ei ken gōkaku-sha no 「eigo gakushū・eigo shiyō jōkyō chōsa 」no hōkoku]. [Report on English learning & English use of students who successfully past the English examination]. STEP 英語情報, 9(4), 25-29.
Tholin, J. (2008). Learner autonomy, self-directed learning and assessment: Lessons from Swedish experience. Independence, 43, 9-12. Retrieved from http://www.diva-portal.org/smash/get2:870506/FULLTEXT01.pdf
Todd, R. (2002). Using self-assessment for evaluation. English Teaching Forum Online, 40(1), 16-19. Retrieved from https://americanenglish.state.gov/files/ae/resource_files/02-40-1-f.pdf
Tono, Y., & Negishi, M. (2012). The CEFR-J: Adapting the CEFR for English language teaching in Japan. Framework and Language Portfolio (FLP) SIG Newsletter, 8, 5-12.
Visualizing English Language Competency Test. (n.d.). VELC スコア. [VELC Score]. Retrieved from https://www.velctest.org/outline/#outline5
Wang, R., Hempton, B., Dugan, J., & Komives, S. (2008). Cultural differences: Why do Asians avoid extreme responses? The Survey Practice, 1(3), 1-7. https://doi.org/10.29115/SP-2008-0011
Appendices: See PDF version
SiSAL Journal 