The Ohio State University – Page 3

As we near the end of the semester, I have to say: managing people is hard. As it turns out, letting people just do their job isn't a recipe for success, and I want to talk about that. Undergraduate Grader Behavior At this point in my teaching career, I've moved away from being the sole provider of instruction, and I've started to divide up the duties with a teaching team. Specifically, I picked up two graders this semester. Unfortunately, no one could have prepared me for the challenge that is managing undergraduate teaching assistants. For the most part, they've done an excellent job. For instance, the always get grades in on time, and they also give great feedback. On the other hand, we don't necessarily agree on teaching philosophy. As a result, I've had to deal with a few frustrating situations. Grading to Impossible Standards At the beginning of the semester, I had an expectations meeting with my graders because I wanted to get off on the right foot. In that meeting, I shared a lot of logistics like when I expected to get grades in and how I expected labs to run. Unfortunately, I had a bit of an oversight; I didn't explain to them how to grade. Instead, I passed off a rubric that I didn't even write. Naturally, they followed that rubric rather than their own instincts. As a result, students were quite frustrated. Despite having a working program, some were routinely scoring less than 50%. In other words, students were very unhappy and quite discouraged. To make matters worse, I was having a hard time striking a balance between student and grader satisfaction. Every time I told my students I'd address the issue, the issue would resurface on the next project. As students grew increasingly frustrated, I tried to take more drastic measures like holding meetings, sending expectations updates over email, and even offering to regrade projects. Of course, I could just step in and start micromanaging my graders, but they need to stay motivated as well. If I can't help them enjoy their work, it'll reflect in the grading. Whether that means grades dropped or assignments are graded late, the outcomes are all bad. As a result, I have to strike a balance. Addressing Behavior Over Time When I first heard about the bad scores, I was a bit shocked. Honestly, I wasn't sure how anyone could be comfortable handing out scores that bad. Luckily, the grader had enough self-awareness to reach out about the issue. As it turns out, they were basically following "my" rubric to the letter. Unfortunately, that rubric ran on a demerit system. In other words, if a project was worth ten points, it would start there and go down for each mistake. Obviously, a rubric like that cannot possibly account for all possibly mistakes. Even worse, there's no way those mistakes are valued at the proper rate. In other words, if each mistake is worth a point and there are 15 possible mistakes, couldn't a student get a negative grade? That's clearly not fair. Personally, I take a more holistic or big picture approach to grading. If a solution works as intended, it get 7 points. Then, the remaining 3 points are reserved for good design choices. As a result, I reward students for prioritizing proper behavior. Unfortunately, I haven't been able to convince my graders to follow this sort of approach. To me, it seems like they can't see big picture. They prefer to nitpick minor "mistakes" based on arbitrary preferences of the course coordinator (i.e. no multiple returns). And honestly, I think that's a consequence of the culture of the department; it's how they were taught. Sharing Information with Only the Worthy While grading is the bulk of my problem, there are actually other issues I have to worry about as well. For instance, I recently let one of the graders share their concerns with the class. Personally, I felt the concerns were valid but a little petty. That said, I'm always happy when there's open communication, so I let it go. Unfortunately, things turned ugly when one of the graders mentioned that students missing the class wouldn't be able to see their suggestions. At that point, the other grader essentially said "who cares?" which was absolutely astounding to me. Apparently, withholding information from people who miss class is standard toxic practice in the department. Shortly after their comment, they began to explain themselves. Naturally, they felt that students who missed class didn't deserve to be "spoon fed" information. Think about that. Sure, part of me agrees with that. After all, I shouldn't have to give extra lessons to students because they missed our agreed upon lecture time. However, this information was vital to getting a decent grade, and it was short enough to package into an email or an announcement. At that point, you're just being manipulative. After all, you don't know why those students aren't there, and it's not up to you to play gatekeeper. To me, it seems like this grader resents people who miss class but still manage to do well. For whatever reason, they appear to think they deserve better just for showing up. Or rather, students that miss deserve worse when they're probably just better utilizing their time. All that said, it's not a competition. Whether or not you get a degree has almost nothing to do with your classmates in that exact moment. Sure, classmates can have an effect on curves and whatnot, but it's more complicated than that. Instead of villainizing your peers, ask yourself: why does it matter so much to you how your peers are doing? This toxic culture is what makes tech such a frustrating field for me. Attending Class Isn't Important Personally, I don't care if a student misses class, and I'm not alone in that belief. In fact, I actively promote missing class because some students just aren't going to benefit from my lecture. That's not because I'm a bad instructor. That's because each student has different needs. For instance, some students already know the material, so it does them little good to attend class. Sure, they could benefit from peer instruction, but they might benefit more by saving those 2-4 hours a week for other courses. It's not my job to decide what's good for each individual student. As an added bonus for me, smaller class sizes allow me to work more with students who need extra help. Having a full class limits my ability to work one-on-one with students, and I think the higher achieving students are doing the rest of the class a service by skipping class from time to time. Also, I'm just a fan of life lessons. If a student thinks they can skip class and it doesn't work out for them, that's on them. They need to learn that lesson. I don't think enticing them with credit is the right way to get them to go to class. They need to be intrinsically motivated. Addressing the Issue On three separate occassions this semester, I attempted to address the grading issues. As someone who prefers to have a paper trail, I made sure to share my expectations with them over email. Initially, I didn't send any information about grading expectations as I assumed it was pretty straightforward. That said, I did pass off a rubric that someone else had written. When grades initially came back bad, I decided to share a set of averages from a previous semester: Also, I wanted to share some grade distributions (averages) from over the summer, so you can get an idea of what you can expect. In particular, projects are meant as an opportunity to make mistakes, so I wouldn't grade them too harshly. Of course, I'm open to discussion on this: - Project 1: 9.32 - Project 2: 8.78 - Project 3: 8.62 - Project 4: 8.19 - Project 5: 8.7 - Project 6: 9.78 - Project 7: 8.65 - Project 8: 7.41 - Project 9: 8.73 - Project 10: 9.35 - Project 11: 8.49 All that said, feedback has been excellent so far. If you want to go the extra mile, I recommend placing TODO comments in the source code and reuploading it. Of course, if that takes too much time, don't worry about it.Jeremy Grifski - 09/19/19, 4:31 PM Then, when nothing changed, I sent out an even larger email with significantly more details: I wanted to take some time to discuss project grading a little bit, so I put together this email to share my thoughts. Feel free to follow up on this email if you'd like. Otherwise, we can set some time up to discuss later this week.At any rate, I've been getting a lot of feedback from students who have been frustrated with their project grades. At the moment, I've been doing my best to hear them out while also defending your grading, but I think we can meet somewhere in the middle.Grading Tips From what I've seen from students, I think the two of you have done an excellent job of providing feedback. My only critique would be to provide point values associated with specific errors or error groups, so students know exactly why they're losing points. In addition, it might be helpful to split comments by correctness vs. software quality (more on that later). When I grade, I tend to place comments directly in the source code with a "// TODO". Then, I give a summary of the grade with an explanation behind the final grade in the comment box on Carmen. Finally, I attach their commented solution to the summary comment. Obviously, you're not expected to do that, but you may find that students appreciate it. Grading PhilosophyPerhaps it might help if I share my grading philosophy for projects. Since the students only get one shot at each project (in other words, we don't allow regrades for half credit or something like that), I tend to be more forgiving in terms of grading. On average, you should expect most students to score about an 8 for each project. In fact, I tend to use 7 as the cutoff for correctness. Then, I reserve the last 3 points for code quality. Of course, correctness can sometimes be subjective. As a result, I prefer to be more lenient. For example, many of the assignments are vague and open to interpretation, so I try not to punish students for coming up with solutions that don't match what we expect (i.e. adding extra columns in the RSS projects might not be a bad thing). In fact, in many cases, the assignments are open-ended. In other words, students are allowed to improve on what they're asked, so we shouldn't punish them for that. In terms of code quality points, I would avoid taking away almost any of those at this early in the course. As long as they're using Check Style and Spot Bugs, their code should be pretty solid (compared to stuff you might see in the real world). It's not until later that they have to start adhering to our discipline a bit more (i.e. no null, no repeated arguments, etc.). Of course, you're welcome to comment on anything you see since they'll benefit from it.Personally, I think my grading philosophy falls in line nicely with the course rubric. I would really only expect to see a score below a 6 (i.e.The solution is just satisfactory; it meets some criteria but there is significant room for improvement.) if the solution was egregiously bad. In other words, the assignment was incomplete or completely unrelated (i.e. not using OSU components). That said, if you see something egregious, don't be afraid to give a bad grade. I'm not promoting grade inflation or anything like that. Instead, I'm looking for grades that are a bit more fair and representative of the class. On the flip side, it's okay to give a 10. In the last three projects, I think only 3 solutions (out of almost 120) received a 10. To me, that seems like we're nitpicking a little bit, but I could be wrong.Keep in mind that the two of you are high achieving students. Otherwise, you wouldn't have been hired. Unfortunately, most of your students aren't going to be as high achieving, so you might feel like a lot of the code you're seeing isn't all that great. That's totally fine! Don't worry about projecting your personal standards onto them. Grading ReviewAt this point, I've had the chance to peek at the project averages, and they're all fairly low (~6). Personally, I don't think that's representative of the students' abilities, and I think a lot of students are feeling frustrated as a result. However, I did peek at the distributions, and it looks like there are a few students just not doing work (i.e. pulling the average down). That said, I don't think it's right for me to challenge your work since I value what you've done. After all, grading is hard, and I'm happy to not be doing it. However, I think that there's some room for improvement. I was wondering if it would be alright if I reviewed your grading of projects 2, 3, and 4. I'll probably handpick like 5 solutions from each project and grade them in private then share the results with you. If there's a significant contrast in the way I grade, maybe we can talk about it. Otherwise, I'll just be confirming your work.Does that seem fair? Let me know! Also, feel free to challenge anything I said here. Jeremy Grifski - 10/01/19, 8:49 PM Finally, the straw that broke the camel's back was when project 8 came back with an average below 50%. At that point, I received a particularly frustrated email from a student which read: I understand that you are going to be curving the project grades for everyone at the end of the semester, but there is no way I can’t complain about the grading of Project 8. Below I have attached the comments left by [the grader]. According to the project grading rubric, a 4/10 is a failing grade, and the project is barely acceptable and doesn’t work properly. My project ran successfully with every case I tested it with, I understand there are some shortcomings with the project and I should be deducted for those, but a 6 point deduction on a 10 point project is simply unacceptable. Last time I took this class, I received an 8.5/10 on this project, and all I have done is make my assignment better. I understand there have been grading issues in past projects with [the grader], but the grading for my project 8 is abysmal. I would really appreciate if you could take a second look at my project and provide the appropriate grading.I’m sorry to be sending this email because I know this isn’t something you want to deal with, but this affected my grade very heavily and I simply cannot bear to not say something about it.Thank you for your time, and I really appreciate how serious you take these matters.Anonymous - 11/12/19, 6:39 PM In frustration, I decided to reach out for help from my course coordinator who assured me that I had final say. As a result, I sent yet another email: This email is a gentle reminder that project grades should ultimately be assessed using the rubric in the course policies. In other words, projects should be graded holistically rather than through fixed deductions. Scores below a 6 are considered unsatisfactory and should therefore only be assessed if the solution is missing parts, clearly doesn't do most of what is asked (> 50%), or egregiously violates the discipline (e.g. not using our components, not using CheckStyle/FindBugs, etc.). Confusing or convoluted solutions (to you) warrant excessive comments but shouldn't receive major penalties. In other words, good faith attempts will likely meet the satisfactory criteria.Also, don't be afraid to give positive feedback as well. If students don't know what they're doing right, how can we expect them to keep those practices going forward?Overall, projects are an opportunity for students to make mistakes and learn from them. If it were possible, I'd allow students to iterate on their solutions, so they'd learn how to strive for excellence. As a bonus, they'd get used to the code review process in the field. Since we don't do that, I ask that you be a bit more forgiving in the final project grades. I don't intend to promote grade inflation, but I also don't see our role as tech gatekeepers. If a student isn't ready to move on, it will no doubt be reflected in their exam scores. Thanks again for all your hard work this semester. Overall, I think you've done a great job giving feedback, and I think the students are really appreciating it. Also, keep in mind that I'm planning to grade project 10, the glossary project. Sorry I haven't been able to provide any examples of grading up to this point. Jeremy Grifski - 11/13/19, 10:06 AM Unfortunately, I don't believe this email solved my problem, but it felt good to outline my values once again. Reflection At this point in the semester, the issue has gone on so long that I can't really correct it. Unfortunately, whoever gets these graders in the future will have to deal with the fallout. That said, most folks in the department are on the same wavelength, so it probably won't be an issue. On my end, I'm hoping to have a more thorough expectations meeting in the spring. That way, I can address this issue better in the future. In addition, I think I'll do some grading with them as well. In the meantime, I'll continue to document my frustrations with toxic tech culture. Part of my wishes I was still teaching intro programming because there weren't any toxic behaviors yet. Students just wanted to learn! If you're like my intro students and love to learn, I have a whole series dedicated to computer science misconceptions. Likewise, you can always read up on my PhD journey which hasn't exactly been linear. Since you're already a subscriber, I won't bombard you with requests. That said, I would love it if you shared your favorite article on social media. Your support helps grow this site, and I appreciate it. Thanks!