EVALUATING FLIPPED CLASSROOM APPROACH IN EFL STUDENTS’ READING CLASSES

This paper examines the implementation of flipped classroom approach in an EFL private university in Indonesia. It explores how this approach impacts on students’ reading comprehension and how students perceive the flipped classroom in their learning process. Under the mixed method design, data were collected from pre-test and post-test, classroom observations, and students’ reflective notes. There were 47 students in the experimental groups and 25 students in the control groups. Those 72 students from both groups took a compulsory 2-credit Intensive Reading Course in their first year in the pre-service teacher training. Both groups were taught the same reading skills and tasks comprising of the total 14 meetings for the whole semester and each meeting took 100 minutes. Results indicate that students in the traditional classrooms gained an increase in their post-test score compared to their counterpart in the flipped classrooms. This might relate with the teacher-led instructions and scaffolding which are commonly conducted in traditional classrooms where students listen to teachers’ explanation and students can ask directly. Other reasons are hesitation/inconvenience in using technology in learning, task-related time management, and technology-related workload. However, for many students flipped classrooms are perceived as promoting independent, responsible, active, and free learning. Both benefits and drawbacks of flipped classrooms in this context are also discussed.


Introduction
Flipped classroom has been extensively investigated in both ESL and EFL contexts in higher education with various classroom disciplines. It refers to the switch between activities in face-to-face traditional classrooms and what is commonly done at home (homework) after the face-to-face sessions. New learning is experienced by students prior classroom sessions whereas in the classroom students collaborate with their classmates or the teachers to develop ideas. In other words, learning takes place at homes and homework is brought and discussed in classrooms. With this kind of well-designed activities, students can control their own learning, develop their collaborative skills, and enhance communication among classmates and teachers. When supported with technology, flipped classroom facilitates student-centered learning and likely to generate active learning environment (Mehring, 2018). Hence, with more advanced technology flipped classroom has been a great choice for learning that give opportunities for students to develop their own learning.
A study on a modified flipped classroom (OPIRTAS-Objective, Preparation, Instructional video, Review, Test, Activity, Summary) for Chinese undergraduate psychology students finds that students indicate positive perceptions on the classroom engagement, teachers' teaching quality and examination performance (Guo, 2019). Another study on the implementation of flipped classroom in an Engineering course with collaborative learning suggests that flipped classroom has developed students' critical analysis skills, problem solving skills, and communication skills (Munir et al., 2018). Similar flipped classroom with cooperative learning is applied using a quasi-experimental study for business students in Norway. Results show that students trained using flipped classroom with cooperative learning activities perform much better compared to those learning under the traditional lecture (Foldnes, 2016). Hence, flipped classroom seems to work well in several disciplines and it is, therefore, not surprising that flipped classroom becomes a fruitful alternative for building interesting and engaging learning using technology.
The high interest in using this classroom instruction could be supported by a survey of university instructors in which 55% of teachers use flipped classroom instruction and the other 25% plan to try it (Schaffhauser & Kelly, 2016). A study about the influence of flipped classroom in a biology course in higher education reveals that flipped classroom had correlation with students' confidence, motivation, and engagement. In addition, students' learning experiences have also been empowered by recorded lectures and sessions conducted in the class (Awidi & Paynter, 2019). In Chinese higher education, factors that influence the continued use of flipped classroom lie on the teachers' perceived technological knowledge and organizational supports. Meanwhile, teachers' beliefs on the technology environment and knowledge on technology use make teachers encouraged to continue using this approach (Cai et al., 2019). In short, how flipped classroom has impacted on students' learning experiences through correlated studies, surveys, and case studies have been flourishing. However, aspects that play roles in the success of flipped classroom implementation continue to reveal new findings and insights, particularly in different learning contexts.
The implementation of flipped classroom in EFL context in the last five years, particularly in language learning has been limited to content based language learning environment (Leis, 2018), narrative inquiry (Aghaei et al., 2019), metacognitive strategies (Shih & Huang, 2019), and Computer Assisted Language learning for EFL pre-service teachers (Akayoğlu, 2019). As far as the literature reports, a study investigating the effectiveness of flipped classroom in EFL college reading course was conducted by (Mo & Mao, 2017). Results confirm that flipped classroom has positive effect on college reading abilities. As the research in this particular reading class is still limited, further studies are expected to be conducted.
It can be inferred that flipped classroom implementation has resulted in many positive learning achievement and processes in several different disciplines. However, studies using quasi experimental method in higher education reading course is still limited. This, consequently, indicates urgent need towards evidences of the flipped classroom practices in different context of study. Therefore, this study is aimed to evaluate the implementation of flipped classroom in a college reading class using experimental (flipped classroom) and control groups (traditional teaching). This is particularly aimed to explore how students perceive flipped classroom in comparison with traditional classroom teaching and how effective flipped classroom in a reading course is. This research is aimed to answer the two questions below.
1. How did students in the experiment groups differ from those in the control groups in term of reading comprehension skills? 2. How do students perceive flipped classrooms in Intensive Reading course?

Method
This study was designed using mixed-method inquiry in which a pre-test and a post-test were administered and the score was compared to check whether flipped classroom was more effective than the traditional classroom. Observations and students' reflective notes were used to provide qualitative evidences on how students perceive flipped classroom.
There were 47 students (2 experimental groups) and 25 students (2 control groups). The pre-test and post-test were administered using TOEFL Reading Comprehension section with the total of 50-item questions. The results of both tests were analyzed using t-test sample in SPSS instrument. Students' reflective notes were collected, categorized, and thematized. Thematic analyses were employed for the students' reflective notes while statistical descriptions were presented in significance tables.

Findings and Discussion
The following section mainly consists of the answers to the research questions as determined previously. The number of the participants in this study is 72 EFL pre-service teachers taking Intensive Reading course in the first year. They were 47 students from 2 experimental groups and 25 students from 2 control groups. The distribution of gender in each group is illustrated in the following charts. Students in the traditional classrooms and the flipped classrooms were taught the same topics, using the same syllabus, and tasks. While the traditional classrooms are conducted with the teacher teaching conventionally in the class and exercises are given both in the class and at home. Whereas, students in the flipped classrooms were required to learn at home by reading, finding additional relevant references from websites, and watching videos on the scheduled topics. Students were given prompts in the form of questions or opinions on a particular topic. They had to upload their responses on to Google Classroom as the Learning Management System and the teacher gave feedback online. During face-to-face sessions, the teacher stimulated students with group discussions and exercises to be completed. When students had questions and needed clarification, the teacher explained and clarified. Meanwhile, the result of Paired Differences in the pre-test and post-test for the experimental groups is 0.1 (M=0; SD=9.362), meaning that there is no significant difference between those tests. Table 2 illustrates the results of the ttest for the control group (traditional classroom). The result of the Paired Differences for the control groups indicates that there is a significant difference in the pre-test and post-test scores (P= 0.003; M=5.6; SD=8.64).

RQ2. How do students perceive flipped classrooms in Intensive Reading course?
To answer this question, students' reflective notes are analyzed and thematized. The most prominent themes are background knowledge activation, time management, independent learning, perceived problems of tasks and prompts, feedback, awareness raising, and navigation skills. As is commonly addressed in every classroom practices, benefits and drawbacks of a learning process is also included. Some of the benefits of flipped classroom are learners become active, classes are lively, flipped classroom forces the brain to think more, students are more prepared, students' potentials are optimized as well as ideas during the learning.
Whereas, the drawbacks include misunderstanding on materials, not all students are independent and convenient using flipped classroom approach, deadlines of tasks and panicking, and less teacher supervision.
This section elaborates the prominent themes of the findings which have been categorized into major issues/themes and sub-themes. The themes are analyzed and discussed in relation with previous studies.

Flipped classroom may not be for all learners
Referring to the results of the statistical data on pre-test and post-test scores, there is no significant difference in the t-test for the experimental groups (flipped classroom). Whereas, the control groups (traditional classroom) has significant difference in the pre-test and post-test scores. The majority of students in the experimental groups (75%) has experienced flipped classrooms in the previous semester of even when they were at high schools. Only 25% of the total population has flipped classroom as their first learning experience. Probable explanation for this ineffective flipped classroom implementation in a reading course in term of reading comprehension scores may root from individual discomfort in using technology, time-management, and technology-related workload.
Students who felt inconvenient with flipped classroom approach expressed their anxiety and unpreparedness when they had to do tasks posted on Google Classroom (Learning Management System). As prompts are important in learning to stimulate students' cognitive, metacognitive, motivational, and cooperative actions when learning (Bannert & Reimann, 2012), these prompts are regularly used to raise students' curiosity on a topic. The prompts are given to students with several questions that trigger them to browse, read, understand, and find answers to the questions. Due to the time constraints, in their opinions, they often felt panicked. Although the syllabus and materials were distributed to students in the beginning of the semester, they did not seem have the initial checking and started to learn. In contrast, they waited for the teacher to post prompts and they perceived this as the time to start reading and learning to find responses to the prompts. The questions or prompts were posted one day before the class was started.
Another source of discomfort is informed by a female student whose learning style is auditory while she argues that the task in this course is mainly verbal (linguistic) (FAM). In addition, students expected to have tasks and when the tasks have not been informed or the questions have not been published, they felt very anxious. A student confirms that she was waiting for the questions and overwhelmed because she had to find answers to the questions or responds to a prompt (RSN). This is likely to be caused by the tasks that require them to read and post the answers on to Google Classroom.
A student elaborates his confusion in his reflective note: I was confused because usually the materials were taught first… but in this class we directly have discussion (DH).
It can be inferred from the note that the student needs to adjust the culture of having explanation in the class and then do the discussion. As in flipped classrooms, discussion is conducted as the follow-up activity after students read, learn, understand, compare, and explore topics given for that learning session by themselves. More importantly, the activities stimulate independent learning (planning their own learning to understand materials, having more time to explore topics from various resources, constructing knowledge in their own phase, and so forth). It can also be concluded that such activities which are conducted outside the classroom by the student himself is assumed as a workload, therefore, the student feels confused, panicked, and anxious. In line with this circumstance, workload in flipped classroom tends to be a challenge as students are required to prepare more before the class (Mehring, 2018).
Another evidence is from a student as reflected in her reflective note: Not all students have the motivation to learn independently at home, and not all students feel convenient using flipped classroom (KRNP).
This supports the fact that not all students are fond of flipped classroom design which require students to learn independently and being active in collecting information on relevant topics. Such a condition may relate with the students' cognitive style-field-dependent learner who tends to become less autonomous in their cognitive restructuring skills (Liu & Ginther, 1999). It could be inferred that students of this cognitive style tend to require reinforcement and supervision from teachers or other extrinsic motivation that help them structure their cognitive skills.

Perceived flipped classroom definition and experiences
There are varied perceptions of flipped classroom from the student participants. Many of them regard flipped classroom as a great and challenging experiences. Few still feel hesitate to embrace the flipped classroom as an alternative for learning due to their previous traditional classroom experiences.

A student perceives that
Flipped classroom is a learning activity in which the learning materials are learned at home (through videos, summarizing, taking notes, making questions, conducting online discussion with peers, and reading relevant references) and then in the class the students do exercises.
But I don't like such a learning because I am used to the traditional classroom… Honestly, flipped classroom is good in that it trains our brain to think critically but I have been using the traditional classroom and I got panicked when the flipped classroom is implemented (KRNP).
This student does not feel convenient with flipped classroom approach although she feels that flipped classroom is a good way of learning besides the traditional classrooms. However, her habit of being in traditional classrooms has occupied her long enough so that it may need time to change the habit. It could also be the reason that the student needs to have more exposure and experiences of flipped classrooms to get the maximum benefits of it.
Three students share similar positive sense of flipped classrooms.
Personally, I like this flipped classroom because I can be more active and I have the willingness to learn the materials. If there is no flipped classroom, I don't think I make any preparation before the class. I feel every challenge every time I do the task because the materials have not been taught yet… (Na) Flipped classroom is a different method of learning because there are more discussions with classmates and students become more active in the class compared to the learning where teachers explain about the materials (MPR).
What I like from flipped classroom is that students can dig knowledge without any compulsion. This could happen because students keep searching until their curiosity is answered and their critical thinking works (CA).
The student participants believe that flipped classrooms have made them prepare the lessons before classes, give them chance to become self-regulated learners, and active learners.
There are split opinions on whether flipped classrooms are preferred more than the traditional classrooms. While many of the students love flipped classrooms, a few prefer traditional classrooms. This finding is also supported by Tomas, Evans, Doyle, & Skamp (2019) that most of the students is undecided and inform that they do not want the flipped classrooms to replace the traditional classrooms. It can be inferred that students are likely to become hesitate whether they could perform as well as in the traditional classrooms. Or else, they may think of how they need to adjust with the conversed learning situation in a technology-mediated instruction.

Perceptions of good feedback
In learning, feedback plays very important roles in helping students develop their skills. The student participants indicate that the teachers give feedback which is motivating, interesting, appreciating, complimenting, and addressing students' work. Several students' notes show this: Teachers give motivating feedback (DH).
The feedback is very good and encouraging. It really evaluates students' work (MIPW).
The feedback given by the teacher is quite interesting because the feedback not only contains motivation but also appreciation, compliment, and expectation for every student (MH).
The teachers' feedback in this study is in line with a study conducted in higher education context in Australia. Good feedback should be positive, constructive, appreciative, clear, and motivational. Besides, feedback should directly bridge the tasks, guidelines, assessment criteria and the relevant points given in the feedback (Ferguson, 2011). Hence, appropriate feedback is good for students to help them indicate what needs to be corrected and improved based on the assessment framework proposed before.

Perceived benefits and drawbacks of flipped classroom
Implementation of a learning approach always brings two-sided impacts: positive and negative. The following are students' perceived benefits and drawback of flipped classrooms.
Students perceive that flipped classroom has many benefits and some of them are described in the following excerpts.
Students are used to understand materials before they are explained, be responsible for tasks, appreciate the time more when doing the assignments, get work done faster (L).
Flipped classroom makes students eager to read and find resources for learning. If not given such a task, I think most students are lazy to read and find references (MH).
Compared with traditional classrooms in which teachers spend time giving more explanations, flipped classroom gives students freedom to discuss about a topic (ZAV).
Unlike traditional classrooms that spoil or spoon-feed students and make the students dependent, flipped classrooms do empower students (MH).
Flipped classroom makes students become independent learners and get used to learning materials before classes (RSN).
Students can optimize their potentials to build new ideas in term of language learning (MAS).
It can be inferred that flipped classrooms encourage students to become independent and responsible learners. Flipped classrooms are also perceived as giving freedom to discuss and learn materials prior to classroom activities and give chance to students to develop their potentials. However, there are also issues to be noticed as shown by the following students' notes.
Misunderstanding is likely to occur if students do not comprehend the materials while they have to answer some questions or discuss about the topic (MCSP).

Less supervision from teachers (MAS)
Because flipped classrooms are designed as conversed traditional classrooms, students who are used to rely on teachers' explanation may need adaptation. Consequently, students might find difficulties in understanding materials due to their learning processes that are not instructed or guided by teachers.
As discussed, it could be inferred that although flipped classrooms can be a good alternative for students' learning. However, it is important to note that flipped classrooms do not always guarantee a good success in all language skills or disciplines. It may also not be everybody's preferable learning approach. For students who have independent learning characteristics, flipped classroom is a great challenge and a good way to improve their skills. Nonetheless, for dependent learners, attending flipped classrooms could mean extra workload as they do not only use technology in learning but also learn by themselves and teachers may not explain every details from the beginning of the class. The pedagogical implication for this type of learners could be teacher-led instruction in flipped classroom (Tomas et al., 2019). Flipped classrooms should be carried out step by step with teachers giving continuous instruction and guidance. Then, gradually the guidance is lessened and when the students are ready, flipped classrooms are fully applied.

Conclusion
From the discussion, it can be concluded that although flipped classroom implementation in this reading course have not resulted in satisfying statistical results, students do have meaningful learning experiences. Measuring success does not completely rely on the figures or numbers in the form of scores, but it also lies in the meaning of learning that the students have been dealing with. Results of learning could also be in the form of affective and motivational drives; it is not always distinction mark. This approach generates more bright sides in engaging the students to independently prepare themselves before the learning process in class, yet it is necessary to anticipate the fact that it may result negative feelings such as hesitation and anxiety especially for those who tend to be dependent learners needing more guided learning and longer time to adapt themselves to be more active, responsible, and motivated. Students' ability in reading skills could also be evaluated from how they get other knowledge accompanying the reading knowledge such as using a Learning Management System, independent learning experiences, and online interaction.
This study has several limitations. First, the context of this study is limited to pre-service teachers in the beginning of their first year at college. This might influence the results due to their adapting and adjusting period from high school to college lives. Secondly, this study used TOEFL reading comprehension test to measure students' reading ability. This may also impact the results in a way that the reading skills taught to the students are not exactly in line with the test. Or else, scaffolding for understanding the reading skills in the test is not totally covered. Thirdly, a survey or case study could be recommended to be conducted for future research so that extensive range of respondents and participants are wider, and data are richer.