Brokk Toggerson – Page 8

Lab groups and peer evaluations

This past year, I have been working to develop a series of labs that focus on scientific skills, as opposed to teaching physics content. These changes are motivated in part by the pandemic: I want to have authentic laboratory experiences that students can complete at home with limited resources. However, these reforms are also motivated by the literature which suggests that lab is better suited to the teaching of such skills as opposed to content:

Holmes, Natasha G., and Carl E. Wieman. “Introductory Physics Labs: We Can Do Better.” Physics Today 71, no. 1 (January 1, 2018): 38–45. https://doi.org/10.1063/PT.3.3816.
MacIsaac, Dan. “Report: AAPT Recommendations for the Undergraduate Physics Laboratory Curriculum.” The Physics Teacher 53, no. 4 (April 1, 2015): 253–253. https://doi.org/10.1119/1.4914580.

Lab groups are one of the necessities of such a large class. In order to respect the TA’s time and keep the grading load manageable, students must turn in reports as groups. Fortunately, I also think that learning to work in a scientific team is also an important goal of the lab experience.

This past semester, I have been trying to use Moodle to manage the lab groups and CATME to do peer evaluations. However, this has yielded two problems:

The TAs must keep the lists in Moodle up to date and there is an unclear chain of command with regards to group management. Also, this requires a rather sophisticated understanding of Moodle and makes changing/managing groups difficult.
The CATME protocol, while fantastic, is, I think, insufficiently transparent. Moreover, I must manage it. This is, frankly, too much load for me. I need a system that the TAs can successfully manage on their own.

I really like the multiplicative nature of the CATME results. A plan with which I am currently toying involves:

Have a number of points equal to the number of members in the team.
Each team member would distribute these points to their team members. Perhaps this would be done for a few different categories.
There would also be one optional point that could be given to someone who really deserves an extra boost. This would be a bonus: if everyone in the team neglects to do it, they will still all get ones (i.e. their score would be equal to their actual grade).
The result would be scaled in such a way that the final multipliers are between 0.7 or so and 1.05.

Obviously, this needs to be flushed out, but there are some key points for improvement here.

Graduate Students’ Teaching Experiences Improve Their Methodological Research Skills

Feldon, David F., James Peugh, Briana E. Timmerman, Michelle A. Maher, Melissa Hurst, Denise Strickland, Joanna A. Gilmore, and Cindy Stiegelmeyer. “Graduate Students’ Teaching Experiences Improve Their Methodological Research Skills.” Science 333, no. 6045 (2011): 1037–39.

I was meeting with Colleen Kuusinen, a new member of our Center for Teaching and Learning on a new project I am pursing as an Honors Thesis mentor. During our conversation, she mentioned this paper from 2011 which indicates that teaching experiences are beneficial to developing graduate students’ research skills. In this paper 95 graduate students’ research proposals were graded in accordance with a peer-reviewed “‘universal’ rubric for assessing undergraduates’ scientific reasoning skills using scientific writing” from B. Timmerman et al., Assess. Eval. High. Educ. 36, 509 (2011). The results were quite impressive as shown in the figures below. I think that these results only further the importance of developing good TA training.

Results from the paper showing that students with teaching experience wrote more testable hypotheses and had higher-quality experimental design as measured from their research proposals. The total sample is n = 95 with n = 49 having both teaching and research experience.

The machinery of very large courses

Well, a new semester has begun. Once again, my Physics 132 course is remote. Due to the unique challenges of this pandemic and remote learning, I am also completely in charge of the lab portion of the course for the first time this semester. This added responsibility, 14 lab sections and 11 TAs, plus my efforts to make the course as flexible as possible in response to the pandemic really has me reflecting on the machinery of very large courses.

This first week-and-a-half has been all about getting things moving. There really are so many pieces: the zoom schedule, the forum, the recordings, the lab manual, the homework, the TA schedule, the TA help sessions, getting all the TAs knowing what they need to do, then helping the 600 students understand it all! It really is like running a small company.

Personnel management should be a required thing for all graduate students.

Fishbone Root Cause Analysis Protocol

This document from the Minnesota Department of Education describes this interesting protocol which describes a procedure for really determining the fundamental causes of a problem (such as student struggle) under the assumption that treating the cause (as best as possible) is more effective than treating just the symptoms. The basic idea is to work to you find a “significant cause that can, in fact, be changed.”

I find this to be an interesting perspective to share when we consider the myriad of unique challenges that our students are facing during this time of COVID-19.

Reflecting on Remote Physics 132

I know the blog has been quiet lately. Like so many others, I have been learning how to juggle everything in this new reality. What time I have found to share with others online has been spent on the page of remote teaching resources I have been curating.

Well, now the semester has finished and I am doing my usual reflecting on how it went and what I can do better for the next round of remote learning in the Spring 2021 semester. A lot happened, so the thoughts are long, but here they are.