The Renegade Coder – Code First. Ask Questions Later.

I have a bit of a different one today. Periodically, I will see a story or a video that "proves" some of the takes I've had on this site around generative AI, and I figured I'd share a few of them. Just a Few Stories For every dozen hype articles, there seems to be one or two that vindicate me. Let's get into them. Preprint Outlet arXiv Is Tired of Fake Articles What kicked off the idea for this article was actually an email I received from someone in my department. It read: arXiv has announced that it will no longer accept review or position papers in computer science amid a flood of low-quality submissions, some of which appear to have been written using artificial intelligence tools. “What we are seeing is many surveys that are just annotated bibliographies without analysis, synthesis or road mapping,” says computer scientist and chair of arXiv’s computer-science section Thomas Dietterich. If you're not familiar with arXiv, it's basically a place where you can publish your work before it goes through the lengthy peer review and publication process. Obviously, anything on arXiv is meant to be met with a healthy dose of skepticism, but in fields that move quickly like tech, arXiv helps keep academia moving. If you're able to access the original source, it's here, but I'll share what I've skimmed. Broadly, it seems that arXiv is shutting their doors to computer science papers for the time being because so many of those papers have been garbage. However, it should be noted that the type of paper being banned is just "narrative reviews," which include surveys and position papers. They also mention that there may be some sort of citation purchasing ring happening, which is why these types of papers are springing up. Of course, they also blame chat bots because generated papers now evade basic plagiarism checks, making bad papers harder to immediately detect. With all that said, research has long been broken since universities equated publication volume with funding, but adding generative AI to the mix only seems to be accelerating the madness in academia. On a side note, this brings about a particularly dystopian thought to me. Folks are obviously worried about AI displacing workers, but now I'm wondering if workers will continue to be needed but for soul-crushing jobs like moderating the massive influx of slop. We already see this with AI-generated pull requests on repos and AI-generated bug reports on software. Why not have a huge editing team reviewing AI-generated research papers? That seems like a great time. GitHub Copilot Extension Breaks Markdown on VS Code Earlier this month, I had several students asking for deadline extensions because they couldn't do the homework assignments. For context, I give all of my assignments in markdown (perhaps unsurprisingly as the guy who made a markdown generation library called SnakeMD). Apparently, students were unable to open the markdown files at all. Ignoring any commentary I could make about students' unwillingness to solve this problem for themselves (i.e., learned helplessness) in the age of generative AI, I ran a quick google search and found the following GitHub issue. Markdown files stuck on infinite loading - source code editor won't open #277599 Does this issue occur when all extensions are disabled?: Yes VS Code Version: 1.106.0 OS Version: macOS (Darwin 24.6.0) Steps to Reproduce: Open any .md file in VSCode workspace Try to view the source code in the editor (not the preview) Editor shows loading spinner indefinetly File content never appears Expected Behavior:The markdown file should open normally in the source code editor, so I can edit the raw markdown content. Actual Behavior:All markdown files are stuck on infinite loading when trying to view the source code. The loading spinner just keeps spinning and the file never opens. However, the markdown preview works perfectly fine and I can see the rendered content without any issues. Workaround:When I rename the file from .md to .txt, it loads instantly. As soon as I rename it back to .md, it's going back to infinite loading again. Additional Context:This issue affects ALL markdown files in my workspace, including files that previously worked fine. I've already tried: Disabling all extensions (including built-in markdown ones) Clearing VSCode cache completely Killing VSCode helper processes Restarting VSCode multiple times Various markdown-related settings changes Nothing helped. The issue persists even with completely disabled extensions and clean setup. It seems to be specifically tied to the .md file extension, since renaming to .txt makes the exact same file content load immediatly. Two comments into the thread, one user says: Following this thread: #277450 I disabled copilot extension and it is working again. To me, this signals a massive concern. Obviously, I don't run any generative AI tools on my own system, but now I'm wondering what could possibly be going wrong here. Why would or should Copilot ever be interfering with the opening of a file? In fact, I am a bit concerned that coding in the IDE itself is no longer "free," as others have pointed at in this thread. What's to stop Microsoft from simply scraping your code as you write it for their models? After all, that's the only way they're going to get around model collapse: oops it's all surveillance. Of course, if you dig slightly deeper, you'll see that the bug was related to markdown files triggering the wrong editor to open, specifically if you had the "GitHub Copilot app modernization" extension installed. I guess it may have been included with the Java extension pack (which I have my students install). Gmail Trains Gemini on Your Emails, Maybe Last week, an article came out that claimed Gmail was scanning emails to train its generative AI models. The source on that article was Malwarebytes, and they since had to walk the article back. However, I'm not sure I would trust Google's word on this given there is already a class action lawsuit in the works. Not only that, it's already well-known that Google programmatically scans your private information in emails. While it might be reasonable to scan an email for malware or spam, Google goes much further. For example, your emails can be "read" to provide targeted advertisements, though Google claims they don't do that anymore. However, they do "read" your emails to provide you summaries, response suggestions, and actions. Now, you might not be weirded out by this. You might think it's fine as long as this is entirely automated and only to benefit you, the user. I don't know that I would put that much faith in Google. Discord literally just got caught storing government IDs in a database that they should have only been holding for mere moments for verification purposes (while at the same time having the audacity to say the hacker "stole" them as if they weren't also stealing them). Are we supposed to believe that Google—who has access to search, watch, prompt, and email history—is the benevolent tech company in comparison? In the end, it's only a matter of time before generative AI scraping becomes so normalized that we willingly let Google train their models on our private conversations. After all, maybe then Gemini can get to know us even better. We'll have our own personal Clippy helping us converse with our peers, and we won't ever need to have an original thought again. Ultimately, I would argue that all email should be encrypted and unable to be scanned or read. There is no feature you could possibly show me that would make me willing to expose my private conversations, and I don't have anything to hide. I am not so desperate for "convenience" that I will give up my privacy. It's why I've been slowly transitioning out of the Google ecosystem. ChatGPT Seeks Profit Through Ads In the least surprising turn of events, OpenAI has opened up the possibility of running ads in ChatGPT. What would that look like, you ask? It's unclear from the linked article, but I think there are several obvious ways this could go about. First, you run ads alongside the service. These would be normal ads you might see on a webpage (e.g., something in the sidebar). Alternatively, they could show ads in the prompt bar as "suggestions" to ask ChatGPT. That seems like one way to quietly promote products. Finally, there's the nuclear option: have ChatGPT bake product placement directly in its output. I think we all see where this is going. OpenAI is bleeding cash, so they need other ways to fund their lossy compression machine. While we're certain to see subscription prices climb (as well as external costs rise like GPUs, RAM, and electricity), the bread and butter of the internet is advertisement. Again, this is the next step in the enshittification of a product. Get your user hooked. Then, spend as much time as possible extracting value out of them. There is no step three. LLMs Give Users Psychosis Recently, I saw this wonderful video by Dr. K going over early research on the effects of LLMs on human psychology. The results are hardly surprising: LLMs even cause normal folks to spiral into psychosis. The general idea seems to be that LLMs feed into your delusions. As you become more delusional, so does it. This replicates an interesting psychological phenomenon known as Folie à deux (not the Fall Out Boy album), where two people share a delusion. It's apparently quite powerful and can drive someone into psychosis. I think I also saw some work that showed that LLMs amplify the Dunning-Kruger effect. Actually, the article kind of frames it as people of any ability demonstrating overconfidence, rather than just beginners. I suspect that delusions from LLM usage go hand-in-hand with overestimating our abilities and knowledge. It's like the LLM bypasses all of our cognitive safeguards. Speaking of bypassing our cognitive safeguards, I keep seeing stories of folks dying after using ChatGPT. There have been teens who have fallen in love with it and committed suicide when things went wrong. There have been folks who were in a dark place and were guided down the path to suicide. There have even been accidental deaths by people using the tech to get advice. That last one was featured in a Chubbyemu video (and the person didn't actually die), and I was a little disappointed with his takeaway at the end of the video. He uses the libertarian "guns don't kill people" argument to absolve ChatGPT of sin. More specifically, he states "ChatGPT didn't hold this guy down and force feed him bromide." I find this a bit irresponsible given that we don't know what ChatGPT said to the man, and he very well could have been deluded into taking bromide. It may have even told him outright to take it. Assuming that the person who went out of their way to take it was just some fool seems to lack some empathy, even if I agree that people do stupid things from time to time. We give technology far too much grace and living, breathing people far too little. Side Note: I find the whole obsession with personal responsibility to be a bit annoying. Sure, I would like people to think for themselves and make good decisions, but the environment is tough out there. If someone is working hard and can't pay rent in the loneliness era, there's no amount of personal responsibility that is going to save the day. In fact, there's no amount of personal responsibility that is going to prevent a person from making bad decisions under that kind of stress (i.e., turning to drugs and alcohol, chatting with a bot, etc.). So to be like, "this guy is a fucking idiot and that has nothing to do with ChatGPT" is reductive at best. Generative AI Is a Disease I think a lot of normal folks can point to image, video, (and maybe, audio) generation and come to the conclusion that it shouldn't exist. No one wants it, and no one wants to live in a society where it exists. Somehow, it's a bit harder to convince people that text generation also shouldn't exist. I find that puzzling because our entire society is built on language. At our core, we communicate. That said, I think folks are going to slowly be turned off by text-based generative AI as it pollutes comment sections, ruins classrooms, and exacerbates loneliness. Hopefully, the examples in this article start to convert you as well. I was reflecting recently on my mother's passing. She was really into technology. In fact, both of my parents were. My dad still repairs electronics and is really into ham radio. He also still games from time to time. My mom spent much of her time at the computer reading and playing games. Fortunately, as far as I know, my dad hasn't started playing with ChatGPT. I would hate for him to fall into its trap while feeling lonely. At the same time, I'm somewhat glad my mom never got to experience ChatGPT. She was always on the front lines with new tech, so she certainly would have played with it. I feel like she was the type of person to really enjoy it, as she often used to read us news articles and Facebook posts. To me, that's really frightening. I have to wonder how many folks have family members whose personalities are changing from LLM usage. It really weirds me out how quickly we adopted this tech and how quickly you're labeled an idiot for not taking part in it. The last thing I want is to be compared to antivaxxers and flat earthers, but I have a strong feeling I'm in the right on this one. I'm genuinely concerned about where we're headed as a society.

Welcome to The Renegade Coder, a site for people interested in Python programming and college teaching! Join me on Patreon for even more content.

Latest Articles