The Renegade Coder – Code First. Ask Questions Later.

This semester I experimented with mid-semester feedback, and I wanted to take this article to encourage you to do the same. Standard Disclaimer About Student Evaluations In this article, I talk a lot about the importance of seeking feedback from students. Naturally, I am going to discuss the many ways of receiving feedback from students. One of the common techniques for getting student feedback is through institutional evaluation surveys, and there are many other third-party standardized tools as well. However, despite their "standardized" nature, these evaluations are notoriously biased and many scholars have argued for their exclusion from tenure and promotion decisions. That does not mean these tools have no value, but it's very important to interpret whatever you might see in an evaluation through that particular lens. I am very fortunate to not have to worry about these concerns as someone who fits the stereotypical description of a professor (i.e., a white guy). And if you do not believe me, I encourage you to explore this data visualization tool by Ben Schmidt for comparing the occurrences of certain words or phrases in Rate My Professors (RMP) evaluations. Again, this is by no means a foolproof way of analyzing the data (as even Ben says himself), but the gender disparities are pretty stark. For instance, try searching up adjectives like "smart" or "hot." The gaps can be disturbing, especially in STEM fields like engineering and computer science. At any rate, if you want to look at a more rigorous study, there are plenty out there. Certainly, there are similar studies that look at race or other traits that could bias evaluations. The main point I'm trying to lay out here is that we know about the bias in evaluations, but I think there is still value in hearing out your students. With that said, let's get into it! The Norm of End-of-Semester Evaluations You may recall that I previously ranted a bit about our cultural obsessions with "not being left behind" and "FOMO" as they pertain to our rapid adoption of generative AI. Well, what caused that rant was actually an opportunity to get some mid-semester feedback from my students. Rather than continuing to rant, I wanted to take an opportunity talk about how helpful mid-semester feedback is, and why you might want to engage in it. For most faculty, the only feedback we get from our students is either spontaneous, self-driven, or at the end of each semester. For example, where I work, student feedback comes in the form of a "student evaluation of instruction." It's basically a survey with ten question that do not change from semester to semester, and it also includes an open-ended section for comments. You may recall seeing me talk about it at other points on this site. It use to be a major topic when I would reflect after each semester—such as in Reflecting on My 8th Semester of Teaching: Autumn 2022. One of the cool things about these evaluations is that they share how you compare against the average scores of your department, your college, and the broader university. I've used this fact to create some wonderful plots showing my trajectory as an educator: One of the limitations of only getting feedback at the end of the semester is that it's not immediately actionable. After all, by the time you get this feedback, the students are already enjoying their vacation. Naturally, I expect this to be more-or-less the norm across institutions. There may be some more teaching-focused institutions that prioritize more forms of evaluation, but that's at least not the norm where I work. I will say that since I became faculty, I also get observed once each spring for the purpose of end-of-year reviews. I suspect that is also the norm across institutions. Informal Evaluations Just because an institution only requires that you get feedback from students at the end of each semester does not mean that you can't get feedback more informally. Personally, I get feedback from students in a lot of different ways. One very common way that faculty get feedback is through third-party sites that allow students to review professors. For example, RMP is the one-stop shop for students trying to put together their schedules. I am fortunate enough to have a high rating, but many professors do not. Outside of third-party tools, I also collect feedback every class with an anonymous discussion tool. Basically, I leave a discussion window open before class starts, so students can ask questions or share comments. These days, students don't really use it, but I relied on it pretty heavily in my early teaching to figure out what students weren't understanding. Likewise, I have a Discord server for my students, and there are channels that exist solely for sharing suggestions. Again, it's rare for students to take advantage of this kind of environment for feedback, but I like to make it clear that I am open to feedback in as many places as possible. As I briefly mentioned earlier, being open to spontaneous feedback is always great. What I mean by that is being open to suggestions from students in class, labs, or office hours. Recently, I had a student pitch me an idea for bettering their learning in the middle of lab. Rather than shutting them down, I bought their argument and made the change. Of course, I also shared my initial thoughts and rationale for why I thought they made a good argument. One thing I haven't tried is to put together a survey in the middle of the semester to get some feedback. I imagine this wouldn't be too hard. Put together a couple of open-ended questions and offer some minor incentive to have it filled out. Finally, I have partnered with our teaching and learning institute to run what they call a SGID (i.e., a small group instructional diagnostic). I forget exactly how it worked, but I think somebody came in and talked to my students with me out of the room. Then, they synthesized the feedback for me and held a little debrief meeting with me. Surely, I've written about SGIDs before. Apparently, I've also written about ways to get feedback before, so perhaps this section is redundant. All of that is to say, feedback is not something that has to be incredibly formal. I think putting together a small survey, hosting a focus group, or asking someone to observe you are all reasonable ways of getting some mid-semester feedback. A Review of My Mid-Semester Evaluations Now to the good stuff! This semester I took part in an optional mid-semester feedback survey, which included a handful of Likert-scale questions as well as four open-ended questions. While I didn't really bother to check the Likert-scale questions, the open-ended questions were as follows: What aspects of this course and your instructors' teaching help you learn best? What steps could you take to improve your own learning in this course? What specific advice would you give to help your instructors improve learning in this course? What specific advice would you give your instructor(s) that would help improve the environment of this course? I quite like these questions. The first question is really meant to offset the critiques that you would get from your students. Then, there's a nice reflection question in there for students, which I also like a lot. Finally, the last two get at the feedback: one on learning and the other on the environment. Because this is qualitative data, I actually went through and read every comment. As a part of my process, I tried to isolate the bits of feedback into categories for counting purposes. So if you looked at my reflection, you would see something like (this is an actual excerpt from my reflection): Students like flexible due dates (x5) Students like the care I put into them and the course (x3) Students like my hands-on approach to learning code (x2) Then, I went back through my categories—which are called "codes" in qualitative research—and looked for themes. The themes are what I'll briefly share next. Themes From Aspects that Help Learning Discussions were a major theme: students seemed to really like being able to discuss the topics with their peers. This showed up more times than any other form of feedback. Flexibility was a major theme: students like having more control over how they learn and when they submit their work. Real-world was a minor theme: students like that I try to relate what they’re doing to their futures. Resources were a minor theme: students like the different tools that they can use in the course, like VS Code, Git, Discord, the exam study guide, the homework keys, and office hours. Vibes were a minor theme: students described me as approachable, kind, energetic, and trusting. Themes From Steps to Improve Own Learning Working outside of class was a major theme: students largely felt that they should be spending more time studying outside of class, whether that be reviewing their notes and slides or completing practice problems. In fact, this coincides with many comments discussing the importance of doing homeworks and labs, and generally anything that is optional. Understanding vs. memorization was a minor theme: many students mentioned that memorization isn’t really an effective tool for learning in this course and that they should spend more time learning. Continuing the grind was a minor theme: many students felt they were doing just fine as-is. Multiple students stated that there wasn’t anything they could come up with that would improve their own learning. Taking advantage of resources was a minor theme: several students talked about the importance of going to office hours or coming to labs and lecturers. Many students also talked about the importance of asking questions. Themes From Advice for Improving Learning General satisfaction is a major theme: most students indicated that they’re happy with the course as-is. Structure is a major theme: while students enjoy the course, some students find the lack of structure to be a challenge—especially in lab. Some areas of improvement include high-level summaries of what to get out of the slides. Generally, students can feel lost in the self-guided nature of the course. Examples are a major theme: students want to be able to visualize the concepts better, whether that be through pictures, diagrams, or source code. The explicit connection between theory and practice needs to be made. Themes From Advice for Improving Environment Satisfaction is a major theme: students are generally very satisfied with the classroom environment. Advice ranged from “N/A” to “Keep doing you.” Isolation is a minor theme: while students are generally happy with the course, there are some students who feel isolated. They want some way to feel like they’re a part of the group and struggle to find anyone to discuss with. Changes I Plan to Make You may recall that in my old reflection articles I used to brainstorm changes. Well, changes are sort of built-in to mid-semester evaluations. You can't take advice from students and not try to implement some of it during the semester. As a result, I put together some ideas for changes (I'm leaving out some of the details because they don't really add anything for the casual reader): Add Key Takeaways to Slides: At the end of my slides, I'm going to start putting a key takeaways slide. Students seem to benefit from being explicitly told what they should be learning, and I think this is a nice way of wrapping up a lecture with a summary. Introduce Peer Instruction: I used to use peer instruction extensively, but it takes a lot of time to come up with questions. As a result, when I switched over to a new course, I left peer instruction behind. I'm thinking that may have been the structure my current students are missing. Polish Up Study Guides: I have study guides for the exams, but they're not really uniform. I would like to clean them up. Tighten Up Discussions: Discussion are a heavy part of my course, but some students hate how they drag out. I'm not the biggest fan of caving to maximum efficiency, but I think tightening up the discussions could go a long way for some students. Naturally, this probably too much work to get done by the end of the semester, but I think having some accountability from the students will push me to do some of it and leave the rest for the summer. With that said, I really ought to get some sleep! I hope this was helpful to you in some way. If so, there's plenty more where that came from: Lessons Learned From Two Years of College Teaching Crafting My Fall Teaching Plans Based on Assignment Feedback Looking Back on My First Year of Teaching And if you're feeling generous, check out my list of ways to grow the site. Anything helps! Otherwise, take care.

Welcome to The Renegade Coder, a site for people interested in Python programming and college teaching! Join me on Patreon for even more content.

Latest Articles