Daya Datwani-Choy, The Centre for Applied English Studies, The University of Hong Kong, Hong Kong
Datwani-Choy, D. (2016). Evaluating a self-access centre’s effectiveness and efficiency: A case study and reflection. Studies in Self-Access Learning Journal, 7(4), 397-411. https://doi.org/10.37237/070408
Download paginated PDF version
Abstract
This article discusses changes that occurred after a case study that examined the effectiveness and efficiency of a Self-access Centre (SAC) within a university in Hong Kong using Morrison’s (2003) Evaluation of Self-access Language Learning Centre framework. The case study addressed issues from a stakeholder’s perspective by collecting data from learners through an online questionnaire and conducting 15 semi-structured interviews, an advisers’ focus group and management interviews and reports. The outcome of the study showed that a wider perspective for senior managers and insights for evaluation of the support services is vital to making key decisions in context. Upon reflection of the findings with the new SAC manager, more acute decision-making and continuous improvement is needed to enhance effectiveness and efficiency of the running of the SAC. To achieve this, regular feedback from various stakeholders, also re-structuring human and non-human support systems is vital.
Context
The University of Hong Kong (HKU) is considered to be the top university in Hong Kong with all subject content (even Science, Technology, Engineering and Mathematics – STEM) taught in English (except Chinese language and translation). The Centre of Applied English Studies (CAES) runs English courses for all undergraduate students in the university two masters programme and supervision of Research Postgraduates. CAES Annual Report 2012/13 (see Figure 1) shows HKU’s student population in comparison with that of CAES. However, users of the self-access centre (SAC) can be any staff, research and taught postgraduates and undergraduates in HKU, not necessarily studying any CAES courses.
Figure 1. Hong Kong University and CAES Student Population in Year 2012/13
The services offered by our SAC include English advising, workshops, language learning activities and online resources. However, there are no full-time staff in the SAC. In fact, the SAC coordinator had a full teaching load with a reduction included for Master’s degree programme coordination and SAC coordination. At the time of the case study a group of about 10 lecturers were assigned one to two hours of SAC advising duties per week to make up a full teaching load. There was an induction meeting in the beginning of each semester to run through administrative matters but no training in advising was provided.
To date, top management has been narrowly defining the success of these programmes by using incomplete data that focuses on frequency of the services taken up by users as a measure of the efficient use of resources (human resources and facilities). Effectiveness in terms of enhancement of learning, what the end users (students) and service providers (human and non-human support) deem as effective, has seldom been considered.
The case study described in this paper aimed to investigate the effectiveness of the services we provide in enhancing students’ learning and developing ‘learner autonomy’ which is the ultimate goal of most self-access centres. The case study took place in 2012/13 and was guided by Morrison’s (2003) framework “Evaluation of Self-access Language Learning Centre”.
First, the case study and its findings will be presented, with a suggested revision to the framework, followed by steps taken after consideration of the results in the study.
The Case Study
Morrison’s (2003) framework emphasises the need to first identify the main aspects of the SAC context being evaluated through a “mapping” process, and then collect data from all major stakeholders. Space restrictions limit the amount of detail that can be given here, but methods used include: an advisers’ focus group, semi-structured interviews with 15 SAC users, an interview with the SAC coordinator, as well as the examination of internal reports from 2011 to 2014. CAES Annual Report 2012/13 is cited mostly in this article as the case study took place that year. The SAC users’ interviews were transcribed and coded using analysis software Weft QDA into four themes, namely: objectives, self-perception, interaction and learning culture. Similar themes were used in the analysis of advisers’ focus group and the SAC coordinator interview.
The Case Study Findings
The case study highlighted a number of aspects of the SAC which required improvement to improve the effectiveness of the services offered to students. Due to limited space, this paper will focus on two major findings.
Training of advisers and tutors
The case study revealed a lack of staff training and professional development. At the time of the case study there was close to no staff training except for one induction meeting at the beginning of the semester basically for administrative purposes. The SAC coordinator assumed that SAC advisers should have had teacher training while peer tutors (student volunteers) and writing support tutors (who are mostly full-time research postgraduates of CAES under scholarship) though not teacher-trained may be self-trained if necessary (and a training video for this purpose was produced (Voller, 1998)). Mozzon-McPherson (2007) emphasises the importance of training for language advisers, considering it a developmental process of professional growth in which advisers become learners themselves as participants in dialogue looking for answers and carrying out action research. However, the SAC coordinator’s and advisers’ (lecturers in CAES) priority is to teach undergraduate and higher education courses run by the CAES. There needs to be some collaborative learning even if advisers work autonomously, but language advisers claimed the focus group itself was the first time there was any sharing.
Cost-effectiveness
While the SAC Coordinator argued that self-access support services are becoming more effective, the advisers’ focus group and learner interviews revealed areas for improvement. The SAC coordinator stated that self-access support services should be less costly in recent years because online resources are freely available, the physical space is smaller and fewer books are purchased as they can be found in the library. He also pointed out that cost of language advising was 20 minutes back in the 1990s whereas currently it would be 12 minutes (calculated by a lecturer’s pay in proportion to time spent in each advising session), However, the advisory service is still the most expensive resource provided in the self-access centre. Moreover, advisers in the focus group mentioned that much time with learners was spent on tedious repetition of matters such as how to log into the booking system. Besides finding the registration cumbersome, 12 out of 15 learners interviewed needed clarification of services even though they were users who had filled in the questionnaire and volunteered to be interviewed. This validates advisers’ complaints that much of the 20-minute session was taken up by having to repeatedly introduce the services because learners tend to be unclear about their English learning objectives and the reason for consultation, which is not the most efficient use of advisers’ time.
It has also been difficult to get accurate data in terms of supply and demand of the services. The Internal CAES Self-access report (2012/13) stated that the need for self-access consultation services (Figure 1) was demonstrated by student uptake of the services, since it was entirely voluntary. It shows 2,436 self-access consultations hours, but it does not reflect real numbers as three consultations take place each hour. There needs to be a system to check the exact number of learners actually served. Writing Support and Peer Tutoring showed an even higher uptake but this was not shown in the report because the former was a new initiative for the current year and the latter was a collaborative effort with an external body. Moreover, there are many reasons learners were not able to book the services. Some sessions are booked by users who do not show up, so others lose the opportunity to benefit from the service. This shows that depending only on summative data is insufficient.
Adapting the Evaluation Framework
Morrison’s (2003) original SAC Mapping for HKU (as each SAC is unique, his evaluation framework first requires each institution to conduct a mapping process to identify key features) had five components. From the findings of the case study and in consultation with Dr. Morrison, it has been decided that the five components (adapted framework see Figure 2), will be reduced to three protagonists; the learner, human support and non-human support. I suggested the resources and environment components, from the original framework, should go under non-human support, which was agreed. Reports with feedback and data from various support services need to be collated every semester and closely monitored for evaluation and decision-making.
Figure 2. Adapted Evaluation Framework for SACs in Hong Kong (SAC mapping indicated by an arrow)
Figure 3. Recommended Version of Morrison (2003) Framework’s Hong Kong SAC Mapping
As Figure 3 shows for SAC mapping (newly adapted version), continuous improvement can be applied to the human and non-human support systems each semester. Since evaluation and feedback will take place regularly, decisions can then be made for adjustments of the services offered, and this will enhance effectiveness.
Main Changes that were Implemented
As a result of the case study, several changes have been implemented to improve the efficiency and effectiveness of the HKU SAC.
An enhanced non-human support system
In practice the new SAC Mapping proposes several systems under non-human support, including information and resources. The use of IT (non-human support) has helped eliminate some of the advisers’ tedious tasks, improved the flow of information and of finding resources with instructional videos. Sharing of “good practices” by interviews with students who appreciated SAC services which are then broadcasted on our Campus TV is also underway.
Human support system
Since the case study, the SAC has diversified the human support system, and now offers several different kinds of support, improving the cost-effectiveness of the services.
Language advisers. In order to free up much of the language advisers’ time (the most costly resource) an enhanced log-in and information system is now in place. Moreover, while learners previously had the option of filling in a 10-page document, Students’ English Development Plan (SDEP), this is now reduced to two pages (see appendix C), so students are clearer about their objective or sub-skills development when visiting a language adviser.
Advisers’ time-slots have been changed from “one size fits all” 20-minute sessions to 30-minute slots for learners who want to make a plan enhancing English and study skills and 10-minute slots for those who wanted a ‘quick fix’ such as test practice a day before the exam.
Although learners’ initial questions may be about how to find materials or use of equipment, language advisers who help learners will most likely build trust and encourage them to return for advisory sessions. However if the adviser lacks the practical skills to assist learners in the beginning, they will not return and the opportunity for further language advising will be lost (Gardner & Miller, 1999). The case study identified a lack of training and professional development opportunities for advisers. The glass panels in the Advisory Zone and weekly email updates enable advisers to know what kind of services are going on so they are able to direct students which activities to join. Through measures such as these, more sharing between experienced and new advisers and a community of practice is beginning to emerge in our centre.
English Teaching Assistants (ETAs). A one-off Teaching Development Grant made hiring four overseas ETAs (undergraduate students who had a Teacher’s Certificate) possible for one year (2012/13). This group of younger tutors helped expand our services (offering services at other campuses nearby, for example the Medical Campus and Toastmasters Club in the evenings) and were less expensive than the advisers. As the next academic year without ETAs saw a sudden drop in numbers in all the services offered by SAC, it was thus decided use CAES’s own funding to hire ETAs for academic years 2015/16 and 2016/17.
Administration. Given that advisers are the most costly part of maintaining the SAC, student helpers or clerical staff may be deployed to provide practical information to learners and administer the SDEP (appendix C) before meeting an adviser. A full-time clerk from the Learning Commons has been provided since 2015/16 without additional cost.
Peer Tutoring. The Centre of Development and Resources for Students (CEDARS) had a reward system for students’ activities. This was an opportunity to collaborate with external agencies in the university. Utilizing the reward system they had in place it was easy to set up a ‘peer tutoring’ system. The ‘peers’ who were exchange students, from various cultures and backgrounds, who chat in English for 30 minutes with learners who reserve a session. They proved to be most popular, according to students’ questionnaire feedback, and much less costly than advisers and ETAs. A narrative study conducted in Japan (Yamaguchi, 2011) found that student staff members could affect their fellow learners who visit the SACs, acting as role models and further activating their agency to gain voice in the Community of Practice (CoP). From 2016/17 these peer tutors increased from 20 (in 2012/13) to 50 and now conduct both individual and group sessions.
Writing support. Our research postgraduates became a resource for a new Writing Support service from 2013 (a new initiative at the time of the case study). Time slots of 20 minutes were offered for these postgraduates to detect common writing errors and give students advice on fossilized errors, without proofreading their work. More lecturers were added later to provide writing support for learners of higher levels such as Master’s and PhD students. In addition, an online programme for writing support is currently being developed to start in 2017/18.
Collaboration and space
Securing students’ space in the Learning Commons. One of the larger offices was used as a SAC when CAES was relocated to the Faculty of Arts new Composite Building in 2012. After a while we were successful in getting space integrated with the Chi Wah Learning Commons (the three floors with state-of-the-art décor was initially catered for students’ use only). Resources such as books, DVDs, and magazines were moved into that larger space for students to work independently. It has been transformed by name as part of the Advisory Zone. This is another effort to step up SAC support services and become more visible.
The new location of the SAC brought opportunities of collaboration with other sub-units such as CEDARS (for peer tutoring) and Learning Environment Services (which provides our one full-time SAC staff). These sub-units provide support for services university wide and have a positive relationship with students, especially undergraduates. Murray and Fujishima (2013) emphasised that social learning space or learning commons share a lot of features with SACs. The need for more emphasis on social interaction was identified in the learner interviews but the only instance (at the time of the case study) of social learning observed from the interviewees is when learners got each other’s contact after discussion groups. Such social learning opportunities have now increased in the Learning Commons, for example social learning with ETAs, being allowed to bring a friend for language advising and a Facebook page where users can interact in English online.
Implications for Practice
English support services are central to the university because English is the language used in subject-content teaching and not merely as a second language for daily conversations or travelling. Although the English support services are constantly fully utilized, as shown in summative reports, the fact is that much more could be done to improve effectiveness to allow a larger student population to benefit from such services, making their provision more efficient – indeed there are many opportunities for co-operation with other faculties and sub-units.
Supporting the University’s vision and mission
While CAES’ vision is that it aims to be recognized as a regional and international leader in the field of second language education through teaching, research and innovation, which makes a vital contribution to the university (CAES Annual report, 2012-3), SACs do more than that. English is not the only focus of SACs, it is also to help students become autonomous learners which clearly aligns with ‘lifelong learning’ and ‘nurtures students…in a culture that fosters creativity, learning and freedom of thought, enquiry and expression’ in line with the HKU’s vision (University of Hong Kong, 2016).
Gardner and Miller (2014) emphasize the importance of an organizational vision and the rationale for the sub-units’ existence that is visible through the mission and is aligned with a SAC’s strategic planning. The SAC does not have a clear mission statement, which has caused some confusion among students and advisers as they did not really understand the rationale behind the support services offered. In fact, three out of eight of the University’s mission points are related to SAC work, which show there is great potential in expanding SAC services to a wider student population.
Embracing a wider student population
The SAC’s effectiveness can be enhanced if there is more attention given to learner diversity, which in turn could widen our student base. Language advisers questioned why international students who also need English support were not using our services. In higher education, learners’ starting point should be their proficiency level and advice may be provided on the next level that he or she wishes to attain.
Figure 4. An Analogy of Maslow’s Hierarchy of Needs Applied to English Learners’ Needs
An analogy would be, using Maslow’s Hierarchy of Needs (see Figure 4). The basic need would be survival such as asking for directions when travelling to English-speaking countries (actually some postgraduates need more input especially if they are coming from the Mainland China). Those who visit the SAC to pass an IELTS or TOEFL exam, may be at the next level. Most Year 4 or postgraduate students are seeking employment, so will have a higher need in terms of professional English. Learners who want to climb up the social ladder and being able to communicate in English near native-like belong to Maslow’s two highest levels referring to self-esteem and self-actualisation. Currently, the SAC, while open to all, does not attract users at every level, so more need to be done to appeal to a more diverse range of learners.
A strengthened core team
Rapid changes in the external environment make it necessary for a core team to be responsive and make adjustments in a timely manner. The Community of Practice (CoP) needs to function and develop its own judgment regarding supporting learners effectively. Feedback is now carried out promptly for users and every semester for service providers (see Appendix A and B), and the data collected is visible to the core team of SAC advisers, enabling shared understanding and informed decision-making.
In conclusion, self-access plays a vital role in providing support services to campus-wide users and not only the students of CAES courses, especially in an English medium university. Therefore, it is necessary to examine the SAC’s role in the context of the university it serves, not only as a sub-unit of a particular faculty. In collaboration with other sub-units on campus, we were able to produce desired results without having to invest many more resources. Finally, and perhaps most importantly, evaluation of the effectiveness and efficiency of the self-access centre requires more than summative reports based on headcounts. There needs to be sharing of good practices, development of a core team which operates as a supportive Community of Practice and a quality “tool kit” of resources for continuous improvement through constant feedback and evaluation.
Notes on the Contributor
Daya Datwani-Choy is a Lecturer in Centre for Applied English Studies, at the University of Hong Kong. Her research interests include self-access management and learner autonomy. She has worked in The City University of Hong Kong as an SAC manager for five years before joining CAES in 2012 and has been a Language Adviser since then.
References
Centre for Applied English Studies (2013). CAES Annual Report: 2012-2013. Hong Kong: The University of Hong Kong.
Datwani-Choy, D. (2014). Evaluating a self-access centre’s effectiveness and efficiency: A case study (Doctoral dissertation, USA, Boston: Northeastern University).
Gardner, D., & Miller, L. (1999). Establishing self-access: From theory to practice. Cambridge. UK: Cambridge University Press.
Gardner, D., & Miller, L. (2014). Managing self-access language learning. Hong Kong, City University of HK Press.
Morrison, B. (2003). The development of a framework for the evaluation of a self-access language learning centre (Doctoral dissertation, The Hong Kong Polytechnic University).
Mozzon-McPherson, M. (2007). Supporting independent learning environments: An analysis of structures and roles of language learning advisers. System, 35(1), 66-92. doi:10.1016/j.system.2006.10.008
Murray, G., & Fujishima, N. (2013). Social language learning spaces: Affordances in a community of learners. Chinese Journal of Applied Linguistics, 36(1), 141-157. doi:10.1515/cjal-2013-0009
University of Hong Kong. (2016). Vision & Mission. Retrieved from http://www.hku.hk/about/vision.html
Voller, P. (1998). One to one consultations (Video and guidebook). University of Hong Kong: English Centre.
Yamaguchi, A. (2011). Fostering learner autonomy as agency: An analysis of narratives of a student staff member working at a self-access learning centre. Studies in Self-Access Learning Journal, 2(4), 268-280. Retrieved from https://sisaljournal.org/archives/dec11/yamaguchi/
Appendices
SiSAL Journal 