The Ohio State University – Page 3

Recently, I read The Skills of Argument by Deanna Kuhn, a book about argumentation as a key to beliefs, and it got me thinking about some of my own beliefs in tech. So I wondered, have you reflected on some of your own tech beliefs? In this article, I take some time to share my background. Specifically, I outline my transition from tech into education and how that shaped my desire to fix up some the cultural issues in tech. Then, I share an interview protocol for getting at a person's beliefs before flipping the script on you! Are you comfortable with challenging some of your own beliefs? Influencing the Culture of Tech Through Education As some of you may know, I'm in the process of transitioning from a Computer Science and Engineering PhD program to an Engineering Education PhD program. The reason for this change is a bit complicated, but it basically boils down to culture. Specifically, I've had a hard time finding my place in the tech community, and I felt like this was the right change for me. Of course, now my life is less focused on Machine Learning and Data Visualization and instead focused on finding ways to help folks feel more welcome in the tech community. In other words, just because I haven't felt all that welcomed that hasn't stopped me from making sure that others don't end up jaded like me. Unfortunately, that's a lot easier said than done. Sure, as an educator, I do my best to welcome students in the classroom. For instance, I try to reach students who feel ostracized and find ways to provide them support. However, those students are only supported for as long as they're in my class. What about when they move on to their next class or their first job? As much as I love teaching, it can only change the culture of tech at a small scale. It does nothing to prepare students to deal with some of the more abrasive parts of our field like hiring practices and work culture—among a plethora of other issues. Instead, it burdens them with the idea that things could be better, and they're almost too powerless to enact change. After all, what kind of junior developer is going to walk into a company and challenge their superiors? To make bigger changes, we need to learn more about the injustices in our community. That's why I decided to complement my teaching with research in tech culture. Ultimately, I'm interested in finding out what people in the community believe about the culture of tech. This article is my first step on that journey. Exploring Tech Culture Through Beliefs Last semester, I took a research design course. Oddly enough, this was my first exposure to research despite being nearly two years into the degree. Again, to me, this is a reflection of what the folks value in Computer Science versus Education. Had I not switched departments, I may never have gotten that formal research training. What does that say about the quality of research in tech? Rant aside, I used that class to write my first research proposal which was designed to answer the following question: what do second-year computer science students believe about the cultural strengths of their department? Answering this question would have given us plenty of valuable information. For example, focusing on second-year students gives us an idea of how the culture was retaining these students. Likewise, understanding the cultural strengths gives us an idea how we can lean into what the department is already doing well. Of course, at the time, I had no research design experience, so I wrote up a method to the best of my ability. Specifically, I decided to conduct interviews using a maximum variation sample. That basically meant that I'd be selecting participants based on certain criteria (in my case gender, ethnicity, and programming experience). The goal here was to ensure that the interviews captured as much of the experience of the department as possible. Ultimately, that proposal ended up receiving a good grade, and I walked away from it. In other words, I never actually conducted that study, but my interests still lie in the same area. Now, I'm officially a part of the Engineering Education department, and I've transitioned from teaching to research (much to my dismay). That said, one of the benefits of being a research assistant is that I get on-the-job training. In other words, I don't have to guess at what good research looks like; I can live it! On my new team, I'm doing research in beliefs in engineering. Mainly, we study deeply-held beliefs by faculty and students in engineering. Naturally, this is a great space for someone like me who wants to learn a bit more about tech culture beliefs. For example, I'm interested in investigating beliefs about tech stereotypes, developer self-efficacy (i.e. impostor syndrome), gender roles, work culture, smartness, etc. Ultimately, I want to uncover where these sort of "toxic" beliefs come from and how deeply these beliefs are held. Until recently, I hadn't really had a mechanism for exploring these beliefs. After all, it's one thing just to ask someone "what are your beliefs on x?" It's another entirely to uncover someone's true beliefs. Studying Beliefs Through Argumentation Recently, I was fortunate enough to be recommended an older book (1991) titled The Skills of Argument by Deanna Kuhn. Although it's a bit dense, it provides a framework for uncovering beliefs. Specifically, the author uses roughly 300 pages to pitch that idea that asking a participant to construct an argument is an excellent way to uncover their beliefs. Think about that! If I asked you to tell me what your beliefs were about whiteboard interviews, you'd probably give me some surface level belief and some justification. For example, you might say that you believe whiteboard interviews are the best way to select candidates. If I were to ask you why, you might justify it by saying that it's the only way to see if they "really know their stuff." Clearly, this response hints at some meritocracy beliefs, but it doesn't really give us an idea of how deeply held this belief is. In other words, how certain is this individual that they're right? And, have they considered any other possibilities? The Skills of Argument Framework Fortunately, The Skills of Argument provides a framework for getting at a deeper understanding. Specifically, the interview is structured around the argumentation process. For example, instead of asking the participant what their beliefs are about whiteboard interviews, we might structure the interview as follows—based on Appendix 1 of The Skills of Argument: What causes some candidates to fail whiteboard interviews?(If multiple causes mentioned) Which of these would you say is the major cause of candidates failing interviews?How do you know that this is the cause?If you were trying to convince someone else that your view is right, what evidence would you give to try to show this?Suppose now that someone disagreed with your view that this is the cause. What might they say to show that you were wrong?What evidence might this person give to try to show that you were wrong?Would you be able to prove this person wrong? And, what would you say to show that your own view is the correct one? Sample Interview: Whiteboard Interviews If I were presented this interview, I might answer as follows: What causes some candidates to fail whiteboard interviews? As with most issues, there's a level of complexity that makes it difficult to suggest a single cause. Part of failure likely rests on the individual for not adequately preparing for the interview. However, there's also the disconnect between what interviews expect and the types of knowledge that students are getting from their institutions. In other words, part of the reason why the individual isn't prepared is because their education hasn't prepared them. That said, I have a more fundamental issue with the whiteboard interviews: there's no way to prepare for them except to learn from the collective experience of others who have failed. In other words, there is no way to guarantee passing an interview because interviewers don't provide any interview expectations ahead of time. How can the individual prepare if they don't know what the interviewer is planning to ask? Likewise, there are a lot of other issues involved like the irrationality of the process. How is the individual to know how they're being graded? How do they know if they're on the right track? And, will the questions being asked ever be related to the actual job? In addition, there's the social justice angle. For instance, perhaps the individual failed because the interviewer was racist/sexist. Hell, there's probably 1000 other variables that go into the success of the interview which could include anything from the individual's programming talent to the weather that day. (If multiple causes mentioned) Which of these would you say is the major cause of candidates failing interviews? Being forced to choose a single cause of failure is a bit tough. That said, if I had to choose one cause, it would have to be the whiteboard interview itself. The nature of the interview doesn't give every participant an equal opportunity to demonstrate their skill. As a result, success in the interview is largely random. How do you know that this is the cause? I know that this is the cause to a certain extent because I've experienced the interviews myself. In many cases, I've been blindsided by an obscure data structures problem that I was forced to work through on a short time limit. At the time, I didn't feel like it adequately assessed my ability, and I didn't feel like it gave me a fair shake. If you were trying to convince someone else that your view is right, what evidence would you give to try to show this? Honestly, I'm not familiar with any research in this area, so I'd probably have to pull from my experience. That said, I'd suspect that you'd find developers at other companies that don't use whiteboard interviews who were just as good as (or even better than) developers at companies that do. This would show that the whiteboard interview process does basically nothing to select for the best candidates. In other words, candidates who fail these interviews aren't necessarily bad candidates. They're just victims of the randomness of the process. Of course, this sort of study doesn't really account for every sort of scenario, so I might be more interested in looking specifically at people who have failed whiteboard interviews to see if they had career success. Likewise, it would be important to look at the flip side; do people who pass whiteboard interviews have career success? I'd probably compare those two groups to see if there were any differences. If it turned out that the failing group had as much success or more than the passing group, I believe that would undermine the integrity of the interview process. If so, I'd be even more inclined to believe that failing is a result of the process. Suppose now that someone disagreed with your view that this is the cause. What might they say to show that you were wrong? I think one possible counterargument to my claim that "people fail interviews because the interview process is inherently flawed" is to say that the interview is testing valuable skills. In other words, this person might say "sure, the interview process has some kinks, but the skills being tested translate directly to the job. Therefore, it's important that the interviewee know the information." As a result, they might argue that people actually fail interviews because they aren't a good fit for the company—not because the process itself is flawed. In other words, they're assuming that if the interviewee can't pass the interview, they can't succeed in the job. What evidence might this person give to try to show that you were wrong? To prove the "company fit" argument, they would probably need to conduct a study where people who would normally have failed an interview got a chance to work at the company. Then, if those people failed as expected, it would provide some evidence to their claim. However, it would be important to compare that population against the group that passed the interview process. The relative success of the group that passed would help justify the integrity of the process. Therefore, taking a swipe at my argument. Would you be able to prove this person wrong? And, what would you say to show that your own view is the correct one? I don't think I could prove this person wrong, but I do think I'd have a stronger argument. After all, the "company fit" argument assumes that people who fail couldn't learn and adapt after being hired. It also assumes that the small amount of material tested is enough to measure "company fit". Unfortunately, disproving the first assumption wouldn't be enough to disprove the entire argument. After all, even if individuals are capable of adapting, it's likely that companies don't want to spend the resources on the transition period. As a result, I'd probably tackle the second assumption by finding a way to show that current whiteboard interview practices aren't enough to establish "company fit." To do that, I would once again compare companies to see if hiring practices play a role in employee retention. If whiteboard interviews truly establish "company fit", then I'd expect to see those companies have better retention. If not, their argument wouldn't have much left to stand on (as far as I can anticipate).That said, I'm aware that there are some flaws in my single cause argument. For example, I don't believe that 100% of the people who failed whiteboard interviews were all plagued by the system. Some of those people rightfully failed due to a lack of preparation. However, if I hard to argue for a single cause, I think the interview process itself is problematic at best. Now, you might think this is a bit messy—and it is! That's because this is framed as an interview. In other words, this is the type of response I would expect from someone who has no access to outside resources for concrete evidence. In other words, they're constructing an argument based on their knowledge and beliefs. Argumentation As the Key to Beliefs Naturally, we can get a better understanding of beliefs with this style of interview because the individual is asked to do the following State a beliefDefend that belief using evidenceImagine a counterargument and/or an alternative beliefImagine evidence that could be used against their argumentConstruct a rebuttal to that counterargument Along the way, we can learn a lot about an individual's beliefs. For example, if the individual can't come up with genuine evidence or a counterargument, it's likely they've never actually reflected on their belief. The deeper we go in the interview process, the more we can learn about the nature of this individual's beliefs. As it turns out, I actually simplified the interview process quite a bit. In the original book, the process involves 26 interviews questions not including probes. If you're interested in learning more about this process, I recommend checking out the book (ad). Reflecting on Tech Beliefs That said, I decided to demonstrate this process because I'm interested in seeing if other folks are willing to get a little vulnerable with me for a moment. In other words, have you reflected on some of your own tech beliefs? The main reason I ask this because I worry that the field purposefully avoids these sort of discussions. As a result, no one ever critically reflects on their beliefs. Instead, we all form camps and spew hate at each other on the other side. So today, I'll ask you to reflect on some the following scenarios. Specifically, ask yourself what causes the following: A candidate to fail a whiteboard interviewA person to earn the title of software developer/engineer/etc.A software system to be badA junior developer to become a senior developerDevelopers to work on personal projects in their spare time In addition, while you're thinking about these scenarios, I'd ask you to also consider some of your beliefs around different technologies and tools. For example, is your love/hate for a programming language justified? Have you considered what some of your critics might say? Again, the goal of this exercise is to give the tech community an opportunity to show some growth. If we can create a space for some of our most deeply held beliefs as a community to be challenged, we can begin to make the space more welcoming for new folks. One last note: not all opinions are equal. I know it's common in the United States (and possibly other places) for people to say "everyone is entitled to their opinion," but that only leads to pseudoscience garbage like climate change deniers, anti-vaxxers, and flat Earthers. Hell, the fact that there's even a group of people in Florida claiming that masks are killing people is a testament to this dangerous belief that all opinions are created equal. With that said, I'll call it quits here! If you liked this sort of article, and you want to see more like it, I recommend checking out some of the following articles: Lessons Learned From Two Years of College TeachingA Look at a Typical Computer Science Lecture If you'd like to take your support a step further, you're welcome to check out my list of ways to help grow this site. In particular, you'll find links to my Patreon, Newsletter, and YouTube channel—among other things. Otherwise, thanks for stopping by! I appreciate it.