In the Research Teachers Can Use blog series, Jeff Rozelle, Knowles President and CEO, discusses research that has important implications for science and math teachers and teacher leaders with the researchers themselves. Hosun Kang is an Associate Professor of science education at the University of California, Irvine, where she also serves as the Faculty Director of Teacher Education in the School of Education. This interview was conducted on November 4, 2022, and has been edited for clarity and length.
Article citation: Kang, H., Talafian, H., & Tschida, P. (2022). Expanding opportunities to learn in secondary science classrooms using unconventional forms of classroom assessments. Journal of Research in Science Teaching, 1-39. https://doi.org/10.1002/tea.21824
Jeff: Thanks for joining me, Hosun. Let’s start with the title of your paper. What do you mean by unconventional forms of classroom assessments?
Hosun: By unconventional assessment, I mean an assessment that creates opportunities for students to express their thinking and do science that deviates from the existing norm of what it means to be good at and do science in the particular context and at a particular time. There was actually a lot of discussion and debate with the peer reviewers of this manuscript. We initially called the assessments in our paper “traditional” and “nontraditional” assessments. For the reviewers, a traditional assessment is a multiple choice question, and a non-traditional assessment is a performance assessment.
Jeff: I had a similar reaction when I first read the paper, because the “conventional” assessment in your research is actually a Claim, Evidence, and Reasoning (CER) prompt, which struck me as somewhat “unconventional.”
Hosun: Right. We realized what we are trying to communicate with this paper—the crux of the idea—is that if you want to promote equity through science instruction, we have to move beyond the notion of equity from the access paradigm. Promoting equity in science classrooms involves expanding what it means to do science and learn science while challenging the existing norms and expectations that mostly reflect western, White, Eurocentric views. It moves past just providing additional scaffolds without disrupting the conventional way of doing science. Providing various scaffolds to make sure students are successful in traditional expectations of school sciences is important; however, we don’t think that this approach addresses the heart of the problem. So in order to truly expand the opportunity for students who have been marginalized in science class, we have to expand what it means to do science and what it means to learn science and then what it means to be good at science–all of which have been constructed in a social, cultural, and political context reflecting a particular group’s views, ways of thinking, doing, and talking.
Jeff: So what’s different about an unconventional assessment from other things teachers might hear about like authentic or alternative or performance assessments?
Hosun: The unconventional assessment in our study—which I’ll talk about in a minute—might have all of those features. It is authentic, and it is in a performance space. One thing that distinguishes our assessment from those other ideas is that unconventional assessments, through the processes of designing and using them, function to disrupt the existing norm of what it means to be good at science. So if it is simply just replicating what has been done forever, whether it’s multiple choice or performance assessment, there are students who might not be getting opportunities to do science despite their capacity for doing it.
Jeff: Can you describe for me what the two assessments in your study were, and what you were trying to figure out about those two assessments?
Hosun: In this particular school district where we worked, CER has been the norm for doing science. Every student when they think about doing science, for them, it was a CER. We have an interesting quote from one student saying, “Oh, I was actually surprised that my teacher is not doing the CER in this science class” because that’s the way they have come to think about science in this school by the time they get to physics in 11th grade, which is where we were working.
We have been working with the physics teachers to co-design a unit about momentum, and we tried to be intentional in attending to language, race, and identities while designing this unit. The essential question of the whole unit was “How are modern cars designed to keep you and your loved ones safe in a collision?”
We carefully designed our CER assessment—the conventional assessment—by leveraging all the research knowledge. We contextualized the unit in a local phenomenon. Students compared California 1918 car fatality rates with 2018 car fatality rates, and changes to car schematics of the best selling cars at those times (Model-T vs. Toyota Camry). In our CER assessments, students were tasked to evaluate three proposed design changes to the Camry, in light of safety. Specifically, students were tasked to select one of the proposed design changes, draw and explain its impact before, during and after a collision. Students had to make their final claim with support of evidence from one of the class activities, and explain their reasoning.
For the unconventional assessment, we sought to provide different forms of opportunity for students to express who they are and how they think. We had them draw their dream car for a particular loved one of theirs. Students wrote a letter to the loved one, using the language that the loved one can understand (mostly Spanish). In the letter, students were tasked to describe the key features of the car that they designed, why they wanted to give this car to the loved one and how the design features will protect their loved one’s safety in a collision. Students were encouraged to actually deliver both their beautiful drawings of the car and the letters. The majority of our students in the classroom were of Mexican descent, so they’re speaking Spanish at home and we tried to create opportunities for students to do science using their home language, which was not a norm in physics class at all. Although the CER and Letter assessment look different, students were tasked to do essentially very similar kinds of intellectual work—using physics ideas to explain how the design features of the car affect the safety of passengers in a collision.
Jeff: So what did you find?
Hosun: We analyzed student performance on the two different assessments. And we also collected qualitative data from our case study students to more deeply understand their experiences with the assessments. We have a quantitative analysis of students’ performance as well as qualitative analysis of a student’s experiences. Our sample was 76 students in 11th and 12th grade, who were students of the teacher who participated in our project.
What we found—if I were to summarize—is that overall student performance improved throughout the unit. Latinx, multilingual students from economically disadvantaged communities showed statistically significant progress from initial to final CER assessments, which is super exciting. Students from diverse backgrounds performed similarly in the Letter assessment that was implemented toward the end of the unit. Interestingly, female students explained the safety features of the car better in the context of writing a letter than writing explanations in the format of CER. We took a close look at two case study students’ experiences qualitatively. The case study students were Latinx students from immigrant families, one male and one female. They were in the same period in the same teacher’s classroom. And we found that these two students had quite different experiences with the CER vs. Letter assessments. One of our case students, a girl, told us: “I was scared to take the physics class because I’ve heard that physics is all about math. Math was not my strong suit. I used to like math when I was a little kid, but during the middle school years, I lost interest in math.” But she also said: “I was surprised because I’m not only good at and successful in this class, but also I found myself enjoying doing physics!” And she told us that she thinks physics is about helping people. That’s the way she’s now thinking about it.
I have to make it very clear. The point of this study is not comparing conventional and unconventional assessments and which one is better. Our initial hypothesis was that we have to expand the opportunities so instead of just a CER conventional assessment, creating this unconventional assessment will expand the opportunity for students who have been marginalized. And this is exactly what we found from the analysis of the students. However, we also found that it is not just using one more assessment or a different form of assessment that made the difference. The assessments need to be well designed in a way that students can deeply connect themselves to it.
Jeff: So through the qualitative data you could see that the assessment was part of the identity work that she was undergoing as a physics learner and in her relationship to physics.
Hosun: Yes, exactly. In the quantitative analysis, we see that the students make progress from the initial assessment to the final assessment, which suggests that the unit helped students get better at explaining how design features of the car worked in terms of safety using the scientific terms and language. All the students, including the female students and the Latinx students of color, made great progress during the unit.
But we also looked at how the students do both on the conventional and unconventional assessments. The boys did not show a lot of difference, but for the female students, there’s suggestive evidence that they are doing better on the deeply personalized assessment. So that’s why we looked closely at the case studies. The girl in the case study I just talked about, she told us, “I don’t really make a big deal out of CERs.” And her CER assessment didn’t really show as much progress from the initial to the final. But on the letter—the unconventional assessment—she wrote a beautiful letter to her mom expressing a lot of thinking and understanding about physics.
Jeff: That’s one of the things you talk about in the article—if that teacher in your study had only done the conventional CER assessment, what would’ve been different for the teacher? Or the students?
Hosun: I appreciate that question a lot. So it’s kind of a thought experiment, but at the same time we do have data because one of the teachers is a co-author of this paper. At the first level, if we didn’t do this unconventional assessment, these teachers would not know that these girls in their classes can do physics at this level. That’s what the quantitative data tells us from the differences in the scores on the two assessments. Our case study girl didn’t do well on the CER assessment. Maybe she’s just a B student or a C student in his mind. He was surprised that some of those students that he didn’t think were particularly good at physics were doing so well.
But more than that, this unconventional assessment was about identifying a loved one, designing a car for them, and then writing about why they wanted to give that car to the loved one, and how this car would protect them in a collision. So the teachers come to learn so much about the students and their families, something that they would never, ever know. Some teachers found that “Oh, my students are such eloquent Spanish writers,” which they didn’t know before. So the biggest benefit for teachers is that it creates an opportunity for teachers to get to know those students very deeply and their families and culture, and also build a new relationship with the students. That is so important from an equity standpoint, because equity often has a lot to do with the relational work that we do between teachers and students. From the student’s point of view, it is very clear when I look at the data, there are some students who don’t necessarily need an alternative or unconventional assessment, like the male case study student. Some students might do well no matter what. However, there are clearly students who won’t see what they can do if they don’t have this kind of expanded opportunities to show who they are and how they think.
I don’t argue that either one or the other assessment is better or we only should do unconventional assessments. But we need to be very mindful and intentional about the kinds of opportunities that we provide by using particular assessment tasks. Assessment sets the expectations of mastery. And then if we truly want to expand the opportunity for students who have been historically and currently marginalized at school, we need to make a conscious, very intentional effort. We need to be aware of what we are asking students to do and how we can challenge ourselves to expand what it means to be good at and do science. It involves asking yourself what has been normalized in your school site or in your classroom context? What has been constructed as a norm for being good at science? We need to be aware of that, and need to think about how that might constrain some students from connecting themselves and relating themselves to science. When and under which conditions are you willing to let those norms go and allow them to express themselves using their home language in a meaningful and powerful way? That is the key. It is not writing a letter; that’s not the point.
Jeff: In my experience, writing good assessments is really challenging, whatever the kind of assessment. What can teachers do if they want to design or use unconventional assessments?
Hosun: My first recommendation is finding a good partner. And then finding a really good example. That’s what I do. Take a good example as a subject of inquiry. It’s easy to overlook everything going on in an assessment, and some things might not be visible when you first look at it. An assessment can look very good or not so good, but it’s important to dig into the intentions behind the design of the assessment. Even the framing of the essential question is so important and difficult. So finding somebody who can help and then finding a good example, and take a look at those good examples together. I think that will be a good starting place.
Jeff: Are there resources online that you recommend for teachers who want to do this work?
Hosun: We have a current website where we are posting all of the assessments that we design in chemistry and physics. And we are also developing another version of the website that is more teacher friendly. I also wrote a paper on scaffolding in assessments that teachers really seem to like that is very practical and includes a lot of examples in that.
Jeff: Is there anything else you’d want to talk about with this paper that you think teachers ought to hear?
Hosun: One thing that I want to emphasize is that a teacher was a big part of this work. Often when people think about equitable assessments, they go looking for the assessment task. The job becomes finding an assessment task, adding more scaffolds to make sure this assessment is equitable, adding more translation or adding more something, which puts the focus all about the designing the assessment task itself.
However, addressing equity through classroom assessment cannot be done by simply changing the form of the assessment task itself. We have to think about the assessment system, not just a task. And in this study we tried this; we co-designed this unconventional assessment with our collaborating teachers to challenge the culture of being good at science at this school. We cannot do this work without the teacher who understands the school, students and the local context. As much as we have to redesign the curriculum and assessments, the new curriculum and assessments should be enacted in a principled way by thoughtful teachers who understand and share the commitment. There are a lot of teachers, just like my collaborating teacher who’s so dedicated, who works tirelessly to support the students who have been marginalized. I have been so lucky to work with and learn from these amazing teachers. I want to honor and recognize those people and I want to say thank you.
Jeff: Thanks so much, Hosun.
