Academia – Page 4 – The Renegade Coder

Student evaluations of teaching are a norm across college campuses, but just about every ounce of research on their use has shown negative outcomes. No more is this true than in my place of work, engineering. Let's talk about it! Examining a Twitter Thread As is often the case when I'm brainstorming my article for the week, I turn to Twitter. This week, there were a lot of wonderful topics to choose from. For instance, there was a ton of drama around the "correct" way to implement grade calculation, which I might cover next week. For now, however, I want to move into the teaching space again because I saw this really interesting thread about teaching evaluations. Because the thread was so hotly debated, I would not be surprised if some of the more controversial bits will be later deleted. As a result, rather than embedding the tweets directly, I'll just transcribe them here. To start, the original tweet I saw was the following: what's up with engineering students? @KamerynJW Which was a retweet in response to the following: This shows the frequency of the word "unqualified" in teaching reviews. Women in orange, men in blue. @Komaniecki_R Before you take a moment to digest this figure, I think it's important to know the origins of the data. These plots were pulled from a tool developed by Ben Schmidt, who pulled the data about reviews from RateMyProfessor. I think that context is important here because there is a very different type of teaching review that makes its way to RateMyProfessor than the kinds that go directly to the institution. I couldn't really characterize the difference, but I would imagine that RateMyProfessor ratings are more polarizing, given that students have to go out of their way to make reviews (i.e., they have to be motivated either very positively or very negatively to make a review). Though, I would also argue that RateMyProfessor is probably more candid, given that it's a third-party tool with anonymous reviews. However, RateMyProfessor is also susceptible to review botting and other rating manipulation techniques. See my recent article on Shewchuk for what I mean. All of that is to say to take the data above with a grain of salt. What's Up With Engineering Students? Going back to the original tweet, I found the question "what's up with engineering students?" to be an interesting one. After all, if you glance at the graph quickly, it almost looks like there is no data on women in engineering. The reality is much, much worse. The disparity between men being referred to as unqualified and women being referred to as unqualified is almost comically large, with women in engineering being roughly 50% more likely than any another discipline to be referred to as unqualified. Now, I think it's also important to note the metric we're dealing with: uses per millions of words of text. So, I'm not certain 6+ uses per million is that much different than 2 or 3 uses per million. Others made a similar point on the post by showing the plots for qualified, but this raised questions around the methodology of the tool. Would we have to also include the phrase "not qualified"? For the sake of argument, I did and the relationship for engineering held: With that said, as Robert points out in his thread, there's plenty of evidence that women face harsher reviews from students on average, stating: With another semester ending, here's your annual reminder that teaching evaluations systemically disadvantage women. Even when controlling for grades and other factors, students (esp. males) consistently give female professors lower scores. This can have serious ramifications. @Komaniecki_R As someone in engineering education, I am hardly surprised by this reality. Women and people of color have been systemically excluded from the space for a long time. Yet, that's not exactly what you'll hear in reaction to the plots above. Browsing the Hottest Takes If you explore the threads above, you'll see a lot of interesting takes. I'll start with one of my favorites: engineering must be the purest ontology of sexism @SHEVANGELION I love this take because it calls into question the very nature of engineering. Is the discipline itself sexist? I think it's somewhat goofy to conclude that, and this user is obviously joking. However, it's not unreasonable to look at the history of the discipline and conclude that sexism forms some of its foundations. A great paper to look at in reference to this idea is "Rigor/Us: Building Boundaries and Disciplining Diversity with Standards of Merit" by Donna Riley. Another take I really enjoyed was this one (and many like it): Shoutout to cs and mathematics for manifesting sexism elsewhere @sloppyhomotopy Something you might find as surprising as the gap between dots in engineering is the near-perfect alignment of dots in computer science. As someone in CS myself, it's still very much white male dominated (source: I am a white guy). That's why I like this take so much: CS has just as many marginalization problems, if not more. Therefore, the sexism must be "manifesting" elsewhere. That said, plenty have joked that CS is made up of equal opportunity haters. Though, I would argue that the issue is that computer science education is still in very rough shape compared to engineering, so the hating is probably deserved. Finally, I have to share the following take, which leads me into the next section: one of society's most reactionary demographics tbh @Cam_Oflage Fundamental Flaws in Engineering Mindset All of the commentary above got me thinking about a time I was in an engineering education seminar and a speaker came in to discuss service learning and humanitarian engineering. They did this wonderful presentation on all the things engineers can do to support refugee communities, such as providing solutions for clean water and plumbing. However, as I sat there, something was nagging at me about their approach to humanitarian aid. Why do engineers always feel like they have to "engineer" a solution for someone? Do engineers ever consider the communities that they actually serve and their needs? In other words, do refugees really want us to show up and design them waste treatment systems, or do they just want us to help them find permanent living situations? So, I basically asked them in the nicest way possible: what are some things engineers can do that isn't engineering related to support folks in refugee camps? To me, it was a layup! Surely since engineers make pretty good money on average, we could make donations to organizations that are helping refugees find stable living conditions. Similarly, there has to be political campaigns looking for donations to help push for important policies, like providing safe living conditions for refugees or making the visa and asylum processes quicker and easier. Unfortunately, the speaker very quickly sort of snapped back at me, as if I hit some nerve with this question. From what I remember of their response, they simply confirmed my assumption: engineers should engineer things. And to me, that response reflects the dominant mindset in engineering. There is no consideration of how a design might affect a community's privacy and safety, among a myriad of other concerns. We see this all the time in the tech space. For example, AI development continues to ignore the concerns of artists, musicians, and other creatives. To me, this is just an evolution of the way social media kicked off the sales of private data for the purposes of advertising. Engineers, either consciously or not, continue to exploit regular people for personal gain. Fundamentally, engineers do not care about the broader impacts of their work. Just read the work of Erin Cech: the dominant ideologies in engineering are meritocracy, individualism, and value neutrality. Therefore, to bring it all back to the start of this discussion, it's not a total surprise to me that engineers would have such a hard time reflecting on sexism—it's just completely beyond the scope of the discipline. Student Evaluations of Teaching Regardless of discipline, it seems like the general consensus is that we need to get rid of student evaluations of teaching, at least from a promotion and tenure standpoint. This is a somewhat upsetting revelation for me as I have historically gotten really positive reviews: While I like to think this reflects my teaching, I know that part of my success is due to my positionality as a white guy. I can say and do almost anything I want in the classroom without a lot of pushback. Again, I'll point to Shewchuk as further evidence of this. Therefore, I think it's probably time to abolish the concept of student evaluations of teaching. Otherwise, we are going to continue to marginalize folks otherwise wonderful educators. Perhaps in the future, they can return. Until then, however, we need to find other ways to assess teaching. With all that said, I think it's time to wrap things up for the day! As always, I hope you got some value out of this piece. If so, I'd recommend you check out some of the following articles as well: Yet Another Instance of Misogyny in the Tech Field It’s Time to Rethink the Hackathon Why I’m a Certified AI Hater Otherwise, thanks for continuing to support the site! See you next time.