The Formative Assessment Backwash in English Instruction at Kristen Nusantara Vocational School

____________________ Abstract ___________________________________________________________________ This case study focuses on covert backwash of formative assessment in English instruction and aims to provide framework of teacher’s instruction realizations in administering formative assessment.The subject was a teacher selected by certain established categories. Data collection techniques are observational recording, interview, and questionnaire. The instruments of data collection are checklist of lesson plan, interview, observational recording, and questionnaire. The findings consist of twenty claims. The conclusions are (1) teacher’s elicitation as key point, (2) elicitations to develop cognitive, (3) numbers of elicitation depending on the existence of students’ responses, (4) deeper involvement by teacher’s feedbacks, (5) no gap during grammar class orientation, (6) slow response, (7) active and interactive demands for the teacher, and (8) life on-going process of learning. The research suggests English teachers (1) to implement formative assessment conversation (2) to implement the claim because it is helpful in developing student cognition; (4) to provide sometimes of FACC absence for students to get ready into the next step and (5) the non-verbal attributes seen on the teacher facilitated the realization of formative assessment conversation to be understood by the students. This research is only limited on teacher without seeing the backwash on the students’ sides. Since it was sought to see the covert backwash, then the unit of analysis was classroom activity, specifically in formative assessment conversation cycle, in which it was administered orally. Further investigation is expected to see the backwash on other types of formative assessment


INTRODUCTION
The use of assessment in learning is always connected each other, even to teaching and learning. It links between a state of content standard to realization of the content (Hakuta and Jacks, 2009). Assessment results may be used for various purposes, included to form students' understanding or knowledge construction (Sheerman et al., 2008). It is seen as diagnosis to find out students' learning difficulites and to improve students achieving better competences and masteries by using both incoming stimuli and existing knowledge instead of merely absorbing transmitted knowledge (Berlak, 1992;Gipps, 1994;Wiliam, 1994 in Bell andCowie, 2001a). The assessment having such benefit is called formative assessment (Evans, Zeun, and Stanier, 2014). Other benefit of formative assessment deals with its formal and informal natures without requiring any written recording during its implementation by eliciting students and immediating use of knowledge in instruction -denoted to create learning (Ruiz-Primo and Furtak, 2007;William, 2010in Vingsle, 2014. Ruiz and Furtak formulate a model to notify the use of informal formative assessment known as ESRU cycle to seek information during teaching an learning through interaction between teachers and students, and then to be used in the teaching and learning process simultaneously to foster knowledge of the student. It is frequently an unplanned occurrence done by teacher through his professional knowledge and skill to mediate, keep, help, and create situation for students to practice (Vingsle, 2014). Thus, it becomes very important for English teachers of vocational high school to find well-planned and organized way to facilitate learning process (Derakhshan & Shirmohammadli, 2015;Ministry of National education, 2006in Newhouse & Suryadarma, 2011. Situations due to formative assessment implementation as explained previously are called backwash of formative assessment. Backwash is defined as negative effect and or direct or indirect of certain assessment practices used to change students' learning or teaching methods, generally categorized into positive or negative (Luke, 1995;Othman, 2007;Watkins, Dahlin, and Ekholm, 2005). When the assessment is used to reflect the skills and content taught in classroom, it is considered as positive backwash (Bedford, 1995) The implementation of formative assessment is actually suggested by 2013 curriculum. Through the curriculum, vocational school students are educated to seek the knowledge by themselves through observing, questioning, exploring and experimenting, associating and analyzing, and communicating, known as scientific approach (Kemendikbud 2013 in Nidhra & Dondeti, 2012). Its learning activity can be done into some models such as discovery learning, problem based approach, and project based approach (Purnawarman, Ratnaningsih, & Gunawan, 2017).
Formative assessment is understood to be applicable in every kind of assessment inside of learning process, applied both written or oral inside of teacher -students' interaction (Weurlander, Söderberg, Scheja, Hult, and Wernerson, 2012). In classroom interaction, formative assessment dynamically involves teacher and student talks adjusted to best meet the current learning needs for their students through informal, discrete, and frequent interaction (Broussard, 2014). The quality of the assessment is strongly depended on the quality of classroom interaction (Antón, 2015).
In another hand, backwash is defined as negative effect and or direct or indirect of certain assessment practices used to change students' learning or teaching methods, generally categorized into positive or negative (Luke, 1995;Othman, 2007;Watkins, Dahlin, and Ekholm, 2005). When the assessment is used to reflect the skills and content taught in classroom, it is considered as positive backwash (Bedford, 1995).
In this thesis, formative assessment is assumed to have backwash toward English instructions of the teacher seen from formative assessment conversation and teacher's nonverbal communication. Formative assessment conversation is defined as daily instructional dialogues embedding assessment into an activity already occurred in the classroom, consisting of teacher elicit question, students' responses, teacher recognition on students' responses, and information collected uses to learning in which continuously modified while learning is taking place (Duschl, 2003;Duschl andGitomer, 1997 in Ruiz-Primo, Furtak, andAraceli Ruiz-Primo, 2006). This cycle will be used as methodological triangulation for selecting classroom talk categorization system called Fiander Interaction Analysis System (FIACS). In another hand, this thesis also anticipates any non-verbal behavior in the class contributing to the backwash of formative assessment. Therefore, an analysis system, Zoric's and Schmid's non-verbal category (2007), dealing with non-verbal behavior having the backwash will be also needed.

METHOD
This case study had purposes investigate certain phenomenon in depth and to provide framework, in this case, the backwash of formative assessment in English instruction (Given, 2008;Yin, 2014). Therefore, the researcher had role as a human instrument to observe, to interview, to mediate data collection, to analyze, to encode, and to report (Watson, 2010).
The main unit of analysis is classroom activity, such as instruction, classroom talks, teacher and student activities (Lom, 2012). The data are formative assessment conversation (Morrison, 2015) and nonverbal communication (Krauss, Chen, and Chawla, 1996).
The research subject was a teacher selected by established categories of good teacher from various combinations of possible traits (Scates and Douglas, 1950). To ease the process of selecting the teacher, there was a need to examine whether the subject fitted the study objectives (Martínez-Mesa, González-Chica, Duquia, Bonamigo, and Bastos, 2016). To protect her privacy, a pseudonym was given (Pan and Li, 2013).
Data collection techniques are observational recording, interview, and questionnaire. The instruments of data collection are checklist of lesson plan, interview, observational recording, and questionnaire. Checklist was used to confirm the essential elements and facilitated researcher ensuring compeleteness in carrying out task ( Garloch, 1947;Frels, Sharma, Onwuegbuzie, Leech, and Stark, 2011;Hagströmer, Ainsworth, Kwak, and Bowles, 2012).
Dealing with observation, a pilot study was initially done (Bartlett, 2013;J. Fraser, Fahlman, Arscott, and Guillot, 2018;Teijilingen and Hundley, 2001). On the pilot test, this thesis employed observer as participant to collect the information better in non-threatening situation (B. Kawulich, 2014). However, participant observation had weaknesses such as probability of the observer to lack of interest or to miss a certain occasion (B. B. Kawulich, 2005), then it was decided to use video recording. This observational recording was considered to be more flexible and facilitate the investigation to get detail and precise data (Bowman, 1994;Halimaa, 2001;Penn-Edwards, 2004 Standardized open-ended interview employs worded identical questions to allow participant contributing much detailed information as they desire and allow researcher asking probing questions as means of follow-up (Turner, 2010;Gall, Gall, andBorg, 2003 in Turner, 2010). Turner (2010) suggests to prepare interview by selecting participants, constructing effective research questions, and using useful questions without making earlier assumption.
The questionnaire was carefully designed started from its question wording, choices of response formats, question sequences, questionnaire formats, and aspects of presentation (McColl et al., 2001). In this research, the questionnaire was used to ease the process of selecting teacher as the subject.
Triangulation was done by using expert judgment adopted from Retnawati and Mujiyanto (2015). The content of this study is presented based on SRQR model by by O'Brien, Harris, Beckman, Reed, and Cook (2014).

RESULTS AND DISCUSSION
Based on observational recording, transcription, and interview, twenty claims were found: 1) the student readiness, 2) the student prior knowledge, 3) teacher recognition, 4) positive student engagement as feedback, 5) teacher personal feeling, 6) series of elicitation, 7) cognitive ability and cognitive source, 8) direct instruction, 9) discourse filler, 10) instructional orientation and teacher's attitudes to silent period, 11) student idea development, 12) teacher's question tag, 13) theme based instruction, 14) lowering complexity of elicitation question, 15) discourse marker, 16) purposes of elicitation, 17) purpose of calling student name, 18) revised language instruction, 19) absence of formative assessment conversation cycle, and 20) formative assessment conversation cycle to check current cognitive level.

The Students' Readiness
The teacher spent time directing, sharing information, and delivering instructional orientation to brainstorm the students about the given material (Observational recording, 03/09/18; Transcription, 23/10/18). When they know what they learn, they will be motivated and ready to learn. It is in line with study done by Kırmızı (2015) and the teacher statement based on interview (Interview, 05/11/18). This finding about students' readiness is in line with findings of Agherdien, Mey, and Poisat (2018) and Kapur (2015).

Students' Prior Knowledge
The teacher to find the current understanding level of the students after listening to her explanation so she knew what to do in the next teaching sequence by an elicit question(Observational recording, 03/09/18; Transcription, 23/10/18). It is in line with Aksan and Çelikler (2015) telling about preliminary knowledge of students before teaching. The teacher also argued that the students' prior knowledge influenced learning outcome in the end of the class (Interview, 05/11/18). It is also in line with Hailikari, Katajavuori, and Lindblom-ylanne (2008).

Teacher's Recognition
The teacher's recognition had purpose to positively reinforce the students, as for example, by saying Good (Observational recording, 03/09/18; Transcription, 23/10/18). This finding about positive reinforcement in teacher's recognition is also supported by Kelly and Pohl (2018) and Rumfola (2017), showing that positive reinforcement made students feeling better and being ready to learn.

Positive Student Engagement as Feedback
The teacher directed a student individually to answer her elicit question (Observational recording, 03/09/18; Transcription, 23/10/18). It is a positive feedback engagement by involving one of the student to think, making him as the model for the others to actively engage (Hattie and Timperley, 2007;Rivers et al., 2017).

Teacher Personal Feeling
The teacher accepted HR's feeling then she expressed her personal feeling by saying "You are brilliant" (Observational recording, 03/09/18; Transcription, 23/10/18). This realization of teacher's recognition is influenced by the previous step of formative assessment cycle by expressing her personal feeling (Saunders, 2013;Zembylas, 2003). It is also a positive feedback, reinforcement, and encouragement to an individual.

Series of Elicitation
When an elicitation was not answered by the students, the teacher tried to elicit again (Observational recording, 03/09/18; Transcription, 23/10/18). It influences the cycle of formative assessment conversation to have series of elicitations, proving well-structured questions to get a certain variable, the response of the student in this case (Gosling, 2018). In this case, the response of the students was uncertain and could not be predicted (Quigley, Colson, Aspinall, and Cooke, 2018).

Cognitive Ability and Cognitive Source
One student at a time shared his response while the others keeping silent (Observational recording, 03/09/18; Transcription, 23/10/18). It happened since the others had not developed cognitive abilities and resources to bear and to effect on the subject matter (Berkowitz and Stern, 2018; Gill and Prowse, 2014).

Direct Instruction
Based on (Observational recording, 03/09/18; Transcription, 23/10/18), the direct instruction was simply directing the students to look at the example consisting new information, maintaining her instructional goal and promoting the process of knowledge instruction (Boleware, n.d.; Kenny, 1980;Liem and Martin, 2013).

Instructional Orientation and Teacher's Attitude to Silent Period
The teacher expressed instructional orientation to review current knowledge, allowing the students to engage in learning and to understand more complex matter (Celal Akdeniz, 2016;Lazarides and Rubach, 2016;Marzano, Gaddy, and Dean, 2000). She asked question and directed the whole class to share their opinion (Observational recording, 03/09/18; Transcription, 23/10/18).

Developing Students' Idea
There was a formative assessment conversation part -using students' response -also functioned as elicitation in the end of teacher's utterance, it influenced to the subsequent instructions (Observational recording, 03/09/18; Transcription, 23/10/18). The English instructions had purpose to encourage one of the students volunteering himself, to get his identity, and to develop his own idea and to provide chance to be active in the class (Noor, 2014;Sharpe, 2008).

Tag Question of the Teacher
On observational recording (03/09/18) and Transcription (23/10/18), the teacher asked a tag question -a follow up based on the teacher's previous statement -to confirm the students' understanding by providing space for student to contribute (Jovanović and Pavlović, 2014;Kimps, Davidse, and Cornillie, 2014). The tag question in which was influenced by the formative assessment conversation had purposes to develop critical thinking skills and construct the students' knowledge (Atwood, Turnbull, and Carpendale, 2010).

Theme Based Instruction
After the teacher assumed the students to have already understood the linguistic features of the material, she then continued to construct it into a complete text (Observational recording, 03/09/18; Transcription, 23/10/18). However, before she got into the complete text, she showed Prambanan temple picture as a theme of the learning material (Observational recording, 03/09/18; Transcription, 23/10/18). According to Alptekin, Erçetin, and Bayyurt (2007), this kind of instruction, known as theme based instruction or content based instruction, focuses on meaning rather than form so it will develop cognitive and linguistic factors simultaneously. Moreover, theme based instruction also provide more integrated activities and tasks (Cheung & Yang, 2009).

Discourse Marker
Another backwash was in the form of discourse marker of the teacher (Observational recording, 03/09/18; Transcription, 23/10/18). It imposed a relationship between some aspects of separated discourse segment, such as a turn taking signal or beneficially guiding for explaining the intention in communication (Alraddadi, 2016;Chapet, 2009;B. Fraser, 1991). This embedded discourse marker in English instruction also has textual function, aiding the speaker to structure and organize the sequence of teaching, contributing to the development of a higher-level explicature and encoding a procedural constrain on context selection (Schourup, 2011;Talebinejad and Namdar, 2011).

Purposes of Elicitation
The backwash of formative assessment are mostly seen after initial elicitation of each cycle (Observational recording, 03/09/18; Transcription, 23/10/18). An elicitation is not only a teacher-posted question to stimulate, provoke, and make students listening carefully, analyzing their thought, thinking critically, initiating discussion, and reviewing materials but also to incorporate students' thought into teachers' instructions realized in five categories (Fitriati, Isfara, and Tristanti, 2017;Jafari, 2013;Kopf and Effelsberg, 2007;Qi and Sykes, 2016;Sahin, 2007). Thus, it allows teacher to express more instructions until the purposes of elicitation categorized as: to inform, confirm, agree, commit, repeat, and clarify are achieved (Jafari, 2013).

Purposes of Calling Student Name
This claim could be seen when there was a student raising his hand, the teacher's talk had intention to instruct the student to start speaking (Observational recording, 03/09/18; Transcription, 23/10/18). This instruction was done by calling the student's name to foster the learning and to make the student feeling appreciated and exist (Glenz, 2014;Middendorf and Osborn, 2002).

Revised Language Instruction
This claim could be seen when the teacher gave chance to the student by asking a question, any other? (Observational recording, 03/09/18; Transcription, 23/10/18) Then, in this elicitation, the teacher indirectly criticized the whole class, by expecting the students to be sensitive to the criticism, by purpose to let them coping with the problem and to improve (Mizokawa, 2013(Mizokawa, , 2014Skipper and Douglas, 2015). There was also unsuccessful bit of language instruction revision (Observational recording, 03/09/18; Transcription, 23/10/18). It had indirect purpose to let the students recognizing the correct language structure in the question (Tomková, 2013;W. Yang, 2010).
While revising her instruction, it was accounted to have spaced-repeated question, separated by other turn takings, provided processing time to boost their performance (Observational recording, 03/09/18; Transcription, 23/10/18; Horness, 2016;Rouhi, Nabavi, and Mohebbi, 2014;Smith, 2012). This repetition was responded well by the student's answer although after that the teacher requested a clarification to rephrase the answer clearly by encouraging him to speak again (Baradeyah and Farrah, 2017;Purver, 2004).

Absence of Formative Assesment Conversation Cycle
The condition where the teacher gave the students work to perform and demonstrate their past experience story in front of the class (Observational recording, 03/09/18; Transcription, 23/10/18). In this part, there was not any FACC observed and accounted but some teacher corrections, teacher -student silent periods, and the use of discourse marker existed (Observational recording, 03/09/18; Transcription, 23/10/18).

FACC to Check Current Cognitive Level
Based on Observational recording (03/09/18) and Transcription (23/10/18), there was only one FACC to check the current cognitive level of the student. Since, there was a slight of silent period, the teacher uttered an instruction as a clue for the student. Then, after the student answered as expected, the teacher recognized the response by encouraging him, inviting the whole class to give applause (Observational recording, 03/09/18; Transcription, 23/10/18). In this cycle, it was not found any using student's response step since no gaps were found during previous steps of the cycle (Observational recording, 03/09/18; Transcription, 23/10/18).

The Realization of Teacher's Instruction in Administering Formative Assessment
The teacher's instruction realizations in administering formative assessment were done indirectly. It was initially done to the whole class in each FACC. The forms of these indirect instructions were elicit questions. Those questions were mostly in the form of WHquestions, functioning to get the information from the students. However, when the first elicitations were not responded well, the teacher added more-simpler indirect questions, or would change the instructions into direct instructions to get the responses. Then, when the responses were as expected, the teacher appreciated the students by using indirect instructions. All of those episodes were backwash of formative assessment conversation in teacher's English instruction to decrease students' apathy emergence (Abbas and Thaheem, 2018).
There were also some FACCs in which were entailed by other FACCS. They were not considered to have gaps but instead the teacher intended to involve the students into deeper discussion. By playing this role as what Winters and America (n.d.) refers as a circle keeper could students connecting ideas by thinking in a new ways or insight. The used of discourse markers were also noted in the findings. They were uttered by the teacher to give her chance in moving to the next sequence of teaching, activity, or giving her chance to think.

Teacher's Management in Following Up the Gaps
There was no gap or unexpected responses found in the findings. It happened due to what Marashi and Tehran (2018) called as traditional-grammar oriented class, causing no content information exchange. However, there were some slow responses of the students and silent periods. To overcome these, the teacher facilitated and eased the students by orally uttering clues.

The Realization of Nonverbal Communication Attributes
According to Zoric, Smid et al (2007), there are 10 categories of nonverbal communication. After analyzing, there were found: chronemics, kinesics, oculesics, physical appearance, proxemics, silence, and vocalics. Among those categories, the most frequent appearing categories were chronemics and kinesics. In terms of chronemics, the finding of this thesis aligns with Muchemwa (2013) showing that the teacher perceived time, structured it, and reacted to it to set the stage of classroom communication.
The teacher admitted she intentionally spoke with normal speech rate to let the student understand the instruction and the delivered material. Meanwhile, in case of kinesics, she admitted it was her habit to do so. Therefore, she did not realize for what specific purposes she did it. However, due to teacher's kinesics, it was believed she could manage the class well and formed students' communicative competence in nature of paralinguistic (Antes, 1996;Barabar and Caganaga, 2015).

CONCLUSION
The conclusions are (1) they key point of formative assessment was teacher's elicitation because teacher knew the ideal cognitive level of students to have understood the material given; (2) the elicitations were not only merely asking questions but they were intended to involve the students thinking and developing their cognitive level; (3) the numbers of elicitation might vary, depending on the existence of students' responses. When the was an absence of the response or students' apathy, it might trigger more elicitations to keep the students following the learning; (4) the feedbacks given by teacher were not only limited by gaps found during formative assessment cycle, but they were also given to involve the students into deeper discussion; (5) the gap was no gap found during the learning since it implemented traditionalgrammar oriented class; (6) instead of gaps, the problems appearing on students' responses were slow responses; (7) in this thesis, formative assessment demands the teacher to be active and interactive to communicate with the students; and (8) the formative assessment was a life ongoing process of learning.
The research suggests (1) to implement formative assessment conversation, according to the findings, teacher should be more aware and literate to deal with the on-going cognitive level, preliminary knowledge, and readiness of the students; (2) positive student engagement, teacher recognition, teacher's personal feeling, direct instruction, instructional orientation, teacher's attitudes to silent period, theme based instruction, lowering complexity of elicitation question, calling student name, developing student's idea, tag question of the teacher, revising language instruction, teacher's criticism, space-repeated question, and teacher's request at student to clarify could help teacher to deal with the on-going learning problems in the class; (3) question was not only used to be answered but it could be used to help the students in developing their cognition; (4) moments of lack or absence of FACC or the emergence of discourse filler provided time for students to get ready into the next step of teaching as well as providing teacher's time to move to the next sequence of teaching; and (5) the non-verbal attributes seen on the teacher facilitated the realization of formative assessment conversation to be understood by the students.
This article investigates the covert backwash of formative assessment toward English instructions by analyzing the teacher's formative assessment conversation regardless from positive and negative backwash. It was done by using formative assessment conversation cycle called as ESRU cycle to see teacher's feedbacks. However, it was too broad so that FIACS was needed to analyze what was going within the cycle. Unfortunately, these research instruments were not tested first to check their reliability. However, by using interview to crosscheck the claims, it could bridge and close the gap.
In another hand, there is also limitation by using Zorich's model (2007) to investigate non-verbal attributes entailing English instructions of the teacher. This model only covers the attributes attached on the teacher but it does not give further categories to indicate the purposes of the attributes. To overcome the problem, the interview is used to reveal what is going on within the attributes.
The use of interview is important for this research to keep consistency of the findings and claims of the research. Therefore, it is believed this research still reliable to contribute.
This article focuses on covert backwash of formative assessment in English instructions. Therefore, it is only limited on teacher without seeing the backwash on the students' sides. Since it was sought to see the covert backwash, then the unit of analysis was classroom activity, specifically in formative assessment conversation cycle, in which it was administered orally. Further investigation is expected to see the backwash on other types of formative assessment administration.