LLMs

My relationship with LLMs is much like smokers' with cigarettes^[1]: I know they're bad for me, and I'm not proud of using them, but I still reach for them when things get tough. Socially taboo (at least in hacker circles), but also universally accepted.

TL;DR: I don't like LLMs and I don't like LLM boosters. I do occasionally throw coding questions at Claude and maths questions at Gemini, though it's a bit of an adversarial relationship. I refuse to use genAI for anything else, including writing text or code.

My view on LLMs#

I find LLMs interesting (especially from the perspective of “we have built a machine that can parrot human text”, which is equal parts scary and fascinating), but they suck in a lot of ways (hallucinations and inaccuracies, kickstarting the “genAI bubble”, generating fake news, environmental impact and ethics of AI training, reliance on major companies, vibecoding leading to poor quality code with no oversight, causing an avalanche of web scrapers tormenting sysadmins and the annoyance of randos in FOSS support chats that copypaste ChatGPT output) that prevent me from endorsing them in any way.

Most of these problems are never going to be resolved; they either stem from technological limitations or societal reasons.

I don't really go out of my way to shun all LLM users, though. My personal policy is this: if you consider yourself an AI engineer vibecoder 10x developer or similar, there's a good chance we won't mesh together^[2]. If you just occasionally use LLMs as a helper (like I do), I don't really care.

My own LLM usage#

For all of the research I've been doing for this garden and projects described herein, I've been making a conscious effort to avoid using LLMs.

I don't use LLMs to write code, because I don't trust their output enough to give them free reign, and I like to understand my code. I did use them for small snippets before, but I can count the amount of cases on the fingers of my hands.

(A part of this, I think, is also related to me not really knowing how to review code or work with other people - something I need to work on. But LLM output is a particularily insidious example, since the probabilistic weights machine knows how to be convincing, even if it's not right.)

I don't use LLMs to write, be it documentation or prose or garden entries or blog posts. No exceptions.

I maintain that if I wanted to read what an LLM thinks about a subject, I'd just ask it directly, and AI-generated writing is just annoying.

I did, in a few cases, use LLMs for code review. I've found that they're pretty good at spotting simple bugs, and in my kernel work, I've been able to have them compare the vendor driver with the mainline driver to see what is missing and what is implemented.

In kernel development use-cases, though, it greatly helps to have a proper understanding of the actual hardware. LLMs still tend to hallucinate and focus on the wrong problems.

I do, occasionally, use LLMs for sysadmin work, but I generally haven't had much luck with these topics - I've had LLMs hallucinate config options at me about 70% of the time I asked them for something.

Granted, the few times I tried using LLMs for that stuff, it was things that I couldn't find documentation on... which is something they struggle with. See my later point about “LLMs as a last resort”.

I do, occasionally, use LLMs for troubleshooting bugs. They make a pretty good talking rubber duck, though you shouldn't always take their advice at face value. I've had LLMs get me very close to the problem area, only to fall apart at finding the exact problem. I've had LLMs focus on the wrong thing, that I knew for a fact wasn't the issue. I've had LLMs miss the mark completely and give me advice that does not apply to my use case.

They really aren't kidding, your mileage does vary.

I do use LLMs for studying maths for uni, because I usually cram for exams only a short while before them, and need something that can explain a maths problem step-by-step quickly, and answer questions about my understanding, should any arise.

(I know some folks who tried to use LLMs for studying multiple times, and got bogus output every time. I seem to have lucked out (though much of that is because I primarily rely on sources from my lecturers (slides, etc.), rather than asking the LLM to generate everything for me.))

My approach to LLMs#

Generally speaking, I don't trust LLMs to generate correct content. I prefer to write/understand things myself, and only ask the LLM for pointers which I look up in other sources. I don't think that this is an infallible solution, because LLMs lie all the time.

Very often, I'll only throw a subject at an LLM if I'm completely stumped on it, which usually happens when:

• The documentation is tough to find or unreadable (unfortunately LLMs struggle with this even harder than I do, since they'll often just hallucinate things instead);
• There's too many things to look into and I don't know where to start (this is actually something that LLMs are good at, since they'll immediately throw a bunch of trustworthy-sounding keywords at you. I usually follow it up with a DuckDuckGo search (or grep the code if I'm asking about a particular project) for those keywords).

Using an LLM to power through code feels good in the moment, but it leaves me with a deep dissatisfaction afterwards. A part of it is that it's not my work.

I think my brain is already fried enough as-is from the modern internet's attention-span-rotting design (not to mention other struggles with attention deficit and executive dysfunction), and I don't need another tool to deepen those problems.

Plus, there's already research linking AI usage to decreasing durable skill acquisition and/or cognitive decline. People who “vibecode” (that is, offload their code-writing to AI) describe burnout and feeling overloaded. I've witnessed this with my own friends who have fallen into heavy vibecoding, though I also have friends who use LLMs and seem to be doing fine.

I suspect if I started to rely on LLMs (that is, use them any more than I do now), I'd very quickly fall into vibecoding.

I prefer not to take the risk.

LLMs in open source#

A few thoughts on LLMs in open source projects:

• Using LLMs to respond to somebody's issues, be it on an issue tracker or support chat, is absolutely horrible and antisocial behavior. If I wanted to know what an LLM thinks about the subject, I'd just ask it directly. Wikis and support channels are for human help.
• Using LLMs to write code raises red flags for the quality of the projects for me, and for many others as well.
  • Vibecoded (fully written by AI) projects are something I avoid completely.
• Strangely enough, I don't particularily care about “licensing” concerns. I'm fine with my code being laundered through the LLM machine. I don't particularily care about the licensing of LLM output. I agree with the take that the genAI movement is too pro-copyright.
  • Nonetheless, I get that there are people who don't want their code in AI training datasets (and it's probably the majority), and I think there should be opt-outs (or ideally opt-ins but we know that's never going to happen)
• LLM policies are… tough. Just thinking about them makes my brain twist and turn. I think LLM code review is fine, but LLM generated code isn't. I think LLM translations are fine, but writing with LLMs isn't. A lot of policies either outright ban all forms of generative AI or go in the other extreme and embrace its usage (see AGENTS.md files).
  • At the end of the day, though, the purpose of anti-LLM policies is to scare off vibecoders, not to explicitly vet any and all potential AI-generated code, but it still feels heavy-handed.

GenAI in art#

I don't like AI generated art. Seeing it in commercial products makes me lose a little bit of hope. We have invented machines to do the fun things for us so we can go back to doing boring things.

I think AI-generated art counts as its own, separate category of art. There are people who enjoy it and who will continue to both generate and consume it. I'm not within that group, but more power to them.

External links#

Links to various blog posts on the subject of LLMs that I find interesting:

• Elissa Black - “Very Average Prototypes”; relevant to my point about LLM output being unsatisfying. The author brings up an anecdote - how she created 3 game prototypes in a short span of time with LLMs, but found them to be soulless, average and not fun; and how a fourth game prototype, this time done without LLMs, was much more succesful in these goals.
  "[…] it’s about ceding control. It happens in increments, and suddenly you’re making fewer decisions. LLMs are averaging machines. Every answer they produce, everything they do, fundamentally comes down to the most likely result. Meaning for every tiny creative decision you cede, however small, the decision will be “made” by the LLM and given the most average possible answer."
• On AI and skill erosion, a Nature article shared on the Fediverse:
  “A study of physicians in Poland who specialize in endoscopy […] shows how quickly AI tools can erode human abilities. The physicians […] were given access to an AI system that analyses colonoscopy images in real time and flags a type of precancerous intestinal lesion called an adenoma. The tool was available to the specialists on some days but not on others. Once physicians began using it, their performance dropped significantly whenever the system was unavailable.”
• On AI in open-source: “What if maintainer burnout isn't burnout?”
  “The thing that made open source maintenance joyful was that someone cared enough about your work to try to improve it. You mentored them. They learned. The code got better. The community grew. That's the social contract.”
  "AI slop doesn't add work. It replaces human connection with mechanized noise."

1. ^{^}

   Despite this metaphor, I don't smoke.

2. ^{^}

   I mostly mean the generic “firstname lastname 10x engineer 50 buzzwords in bio add me on LinkedIn” kind of person. I have some friends who have a special interest in AI and LLMs (which I assume would make most people think of this category), but they are cool (and do a lot of other cool stuff outside of these things).