And to make sure that agent doesn’t make mistakes I have a different agent review that agents work!

I can’t tell if you’re being sarcastic or not, but if you are, the real world is not far behind. I can imagine a world where a mixture of AI agents (some doing hypercritical code review) can return you tested and idiomatic PRs faster than you can describe the new architecture in issues.I think a lot of people are unfamiliar with the (expensive) SOTA.

That’s nothing, I was a 4X programmer and now I’m a 500x programmer!I don’t just run one agent, I run all of them!My time to close tickets is measured in minutes!I don’t even review code, I have a different agent review it for me!

Nerds got taken aside and talked to about how it&#x27;s not nice or cool to brag about IQ score so they invented a new artificial metric to brag about.

It means cranking out hello world even faster i guess. I wonder how complex all these projects are people are proud to have completed with the help of AI.

It’s a riff on the “10x programmer” concept. People who haven’t worked with 10x programmers tend to not believe they exist.I’m nowhere near that, but even unaided I’m quite a bit faster than most people I’ve hired or worked with. With LLMs my high quality output has easily tripled.Writing code may be easier than reading it - but reading it is FASTER than writing it. And that’s what matters.

It depends on the value of x. I think it&#x27;s safe to assume x &lt;= 0.75, else they&#x27;d contribute negatively to their teams (happens from time to time, but let&#x27;s be generous). Previously they&#x27;d be anywhere from a 0&#x2F;10 to 3&#x2F;10 programmer, and now they get up to 9&#x2F;10 on a good day but sometimes are a net negative, as low as -2.25&#x2F;10 on a bad day. I imagine that happens when tired or distracted and unable to adequately police LLM output.

&gt; I was a 3-4x programmer before. Now I’m a 9-15x programmerWhat the fuck does this mean?

I&#x27;m not sure about giving specific metrics or kpis of efficiency or performanceIt definitely feels different to develop using LLMs, especially things from scratch. At this point, you can&#x27;t just have the LLM do everything. Sooner or later you need to start intervening more often, and as the complexity of the project grows, so does the attention you need to give to guiding the LLM. At that point the main gains are mostly in typing and quickly looking some things up, which are still really nice gains

Copilot is a bad yardstick. The article literally addresses exactly this. It’s not just “cool technology”, that’s the point. It enables things that were previously impossible.I spent $600 on claude via cursor last month and it was easily worth 2-3x that.

I don&#x27;t know. I think 9-12 months ago I&#x27;d agree with you. But I feel like the last 6 months my productivity has vastly improved. Not only that, it&#x27;s also brought back a little bit of passion for the field.It&#x27;s easy to come up with some good ideas for new project, but then not want to do a lot of the garbage work related to the project. I offload all that shit to the LLM now.Seriously, the LLMs have increased my productivity 2-4x.

I hate how the discourse around LLM-assisted programing is so polarized. It&#x27;s either detractors saying it&#x27;s &quot;a fad that&#x27;s useless and going nowhere, wasting billions of megawatts every year&quot; or it&#x27;s true believers calling it &quot;the most miraculous sea change technological advancement in my lifetime&quot; or &quot;more important than fire and electricity[1].&quot; There just doesn&#x27;t seem to be any room in the middle.I tried out Copilot a few months back to see what all the fuss was about and so that I could credibly engage with discussions having actually used the technology. I&#x27;d rate it as &quot;kind of neat-o&quot; but not earth shattering. It was like the first time I used an IDE with auto-complete. Oh, cool, nice feature. Would I pay monthly for it? No way. Would I integrate it into my development workflow if it were free? Maybe, I guess? Probably wouldn&#x27;t bother unless it came literally set up for me out of the box like autocomplete does nowadays.Don&#x27;t get me wrong--it&#x27;s cool technology. Well done, AI people. Is it &quot;the 2nd most important thing to happen over the course of my career&quot; as OP wrote? Come on, let&#x27;s come down to earth a little.1: <a href="https:&#x2F;&#x2F;www.cnbc.com&#x2F;2018&#x2F;02&#x2F;01&#x2F;google-ceo-sundar-pichai-ai-is-more-important-than-fire-electricity.html" rel="nofollow">https:&#x2F;&#x2F;www.cnbc.com&#x2F;2018&#x2F;02&#x2F;01&#x2F;google-ceo-sundar-pichai-ai-...</a>

THANK YOU.I was a 3-4x programmer before. Now I’m a 9-15x programmer when wrangling LLMs.This is a sea change and it’s already into “incredible” territory and shows no signs of slowing down.&gt; Think of anything you wanted to build but didn’t. You tried to home in on some first steps. If you’d been in the limerent phase of a new programming language, you’d have started writing. But you weren’t, so you put it off, for a day, a year, or your whole career.I have been banging out little projects that I have wanted to exist for years but always had on the back burner. Write a detailed readme and ask the agent to interrogate you about the missing parts of the spec then update the README. Then have it make a TODO and start implementing. Give it another code base for style guide.I’ve made more good and useful and working code in the last month than I have in the last two years.

And of course, a lot of modern &quot;dvd players&quot; do not properly transmit closed captions as subtitles over HDMI, so that sure isn&#x27;t helpingA slightly off topic but interesting video about this <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=OSCOQ6vnLwU" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=OSCOQ6vnLwU</a>

Many DVDs of old movies and TV shows may contain the closed captions, but they are not visible through HDMI. You have to connect your DVD player to your TV via the composite video analogue outputs.This video explains all about it: <a href="https:&#x2F;&#x2F;youtu.be&#x2F;OSCOQ6vnLwU" rel="nofollow">https:&#x2F;&#x2F;youtu.be&#x2F;OSCOQ6vnLwU</a>

Old TV series should have closed captions available (which are apparently different from subtitles), however the question of where to obtain aside from VHS copies them might be difficult.

I feel you. In the late 00&#x27;s&#x2F;early 10&#x27;s, downloading and getting American movies were fairly easy but getting the subtitles was a challenge. It was even worse with movies from other regions. Even now I know people that record conversations to be replayed using Whisper so they can get 100% the info from it.Disclaimer: I&#x27;m not praising piracy but outside of US borders is a free for all.

It&#x27;s not the speech recognition model alone that&#x27;s fantastic. It&#x27;s coupling it to an LLM for cleanup that makes all the difference.See <a href="https:&#x2F;&#x2F;blog.nawaz.org&#x2F;posts&#x2F;2023&#x2F;Dec&#x2F;cleaning-up-speech-recognition-with-gpt&#x2F;" rel="nofollow">https:&#x2F;&#x2F;blog.nawaz.org&#x2F;posts&#x2F;2023&#x2F;Dec&#x2F;cleaning-up-speech-rec...</a>(This is not the best example as I gave it free rein to modify the text - I should post a followup that has an example closer to a typical use of speech recognition).Without that extra cleanup, Whisper is simply not good enough.

What translation models are better than LLMs?The problem with Google-Translate-type models is the interface is completely wrong. Translation is not sentence-&gt;translation, it&#x27;s (sentence,context)-&gt;translation (or even (sentence,context)-&gt;(translation,commentary)). You absolutely have to be able to input contextual information, instructions about how certain terms are to be translated, etc. This is trivial with an LLM.

&gt; However, LLMs are obviously more powerful in what they can do despite just speech recognitionUnfortunately, one of those powerful features is &quot;make up new things that fit well but nobody actually said&quot;, and... well, there&#x27;s no way to disable it. :p

That leaderboard omits the current SOTA which is GPT-4o-transcribe (an LLM)

That&#x27;s not quite true. State of the art both in speech recognition and translation is still a dedicated model only for this task alone. Although the gap is getting smaller and smaller, and it also heavily depends on who invests how much training budget.For example, for automatic speech recognition (ASR), see: <a href="https:&#x2F;&#x2F;huggingface.co&#x2F;spaces&#x2F;hf-audio&#x2F;open_asr_leaderboard" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;spaces&#x2F;hf-audio&#x2F;open_asr_leaderboard</a>The current best ASR model has 600M params (tiny compared to LLMs, and way faster than any LLM: 3386.02 RTFx vs 62.12 RTFx, much cheaper) and was trained on 120,000h of speech. In comparison, the next best speech LLM (quite close in WER, but slightly worse) has 5.6B params and was trained on 5T tokens, 2.3M speech hours. It has been always like this: With a fraction of the cost, you will get a pure ASR model which still beats every speech LLM.The same is true for translation models, at least when you have enough training data, so for popular translation pairs.However, LLMs are obviously more powerful in what they can do despite just speech recognition or translation.

&gt; it’s still leaps and bounds ahead of traditional translators alreadyTraditional machine translators, perhaps. Human translation is still miles ahead when you actually care about the quality of the output. But for getting a general overview of a foreign-language website, translating a menu in a restaurant, or communicating with a taxi driver? Sure, LLMs would be a great fit!

&gt; It seems as though an LLM would truly have no issues integrating societal concepts, obscure references, pop culture, and more, and be able to compare it across culture to find a most-perfect translation.Somehow LLMs can&#x27;t do that for structured code with well defined semantics, but sure, they will be able to extract &quot;obscure references&quot; from speech&#x2F;text

Translation seems like the ideal application. It seems as though an LLM would truly have no issues integrating societal concepts, obscure references, pop culture, and more, and be able to compare it across culture to find a most-perfect translation. Even if it has to spit out three versions to perfectly communicate, it’s still leaps and bounds ahead of traditional translators already.

Have they solved the problem of Whisper making up plausible sounding junk (e.g. such that reading it you would have no idea it was completely hallucinated) when there is any silence or pause in the audio?

&gt; Machine translation and speech recognition.Yes, yes and yes!I tried speech recognition many times over the years (Dragon, etc). Initially they all were &quot;Wow!&quot;, but they simply were not good enough to use. 95% accuracy is not good enough.Now I use Whisper to record my voice, and have it get passed to an LLM for cleanup. The LLM contribution is what finally made this feasible.It&#x27;s not perfect. I still have to correct things. But only about a tenth of the time I used to. When I&#x27;m transcribing notes for myself, I&#x27;m at the point I don&#x27;t even bother verifying the output. Small errors are OK for my own notes.

What is the relevance of this comment? The post is about LLMs in programming. Not about translation or NLP, two things transformers do quite well and that hardly anyone contests.

Hi Andrew, I&#x27;ve been trying to get a similar audio language support app hacked together in a podcast player format (I started with Anytime Player) using some of the same principles in your project (transcript generation, chunking, level &amp; obscurity aware timestamped hints and translations).I really think support for native content is the ideal way to learn for someone like me, especially with listening.Thanks for posting and good luck.

Definitely not. I took this same basic idea of feeding videos into Whisper to get SRT subtitles and took it a step further to make automatic Anki flashcards for listening practice in foreign languages [1]. I literally feel like I&#x27;m living in the future every time I run across one of those cards from whatever silly Finnish video I found on YouTube pops up in my queue.These models have made it possible to robustly practice all 4 quadrants of language learning for most common languages using nothing but a computer, not just passive reading. Whisper is directly responsible for 2 of those quadrants, listening and speaking. LLMs are responsible for writing [2]. We absolutely live in the future.[1]: <a href="https:&#x2F;&#x2F;github.com&#x2F;hiandrewquinn&#x2F;audio2anki">https:&#x2F;&#x2F;github.com&#x2F;hiandrewquinn&#x2F;audio2anki</a>[2]: <a href="https:&#x2F;&#x2F;hiandrewquinn.github.io&#x2F;til-site&#x2F;posts&#x2F;llm-tutored-writing-practice-for-secondary-language-acquisition&#x2F;" rel="nofollow">https:&#x2F;&#x2F;hiandrewquinn.github.io&#x2F;til-site&#x2F;posts&#x2F;llm-tutored-w...</a>

I don&#x27;t think you are also including having AI lie of &quot;hallucinating&quot; to us which is an important point even if the article is only about having AI write code for an organization.

&gt;The other day, alone in a city I&#x27;d never been to before, I snapped a photo of a bistro&#x27;s daily specials hand-written on a blackboard in Chinese, copied the text right out of the photo, translated it into English, learned how to pronounce the menu item I wanted, and ordered some dinner.To be fair apps dedicated apps like Pleco have supported things like this for 6+ years, but the spread of modern language models has made it more accessible

&gt; I snapped a photo of a bistro&#x27;s daily specials hand-written on a blackboard in Chinese, copied the text right out of the photo, translated it into English, learned how to pronounce the menu item I wanted, and ordered some dinner.&gt; Two years agoThis functionality was available in 2014, on either an iPhone or android. I ordered specials in Taipei way before Covid. Here&#x27;s the blog post celebrating it:<a href="https:&#x2F;&#x2F;blog.google&#x2F;products&#x2F;translate&#x2F;one-billion-installs&#x2F;" rel="nofollow">https:&#x2F;&#x2F;blog.google&#x2F;products&#x2F;translate&#x2F;one-billion-installs&#x2F;</a>This is all a post about AI, hype, and skepticism. In my childhood sci-fi, the idea of people working multiple jobs to still not be able to afford rent was written as shocking or seen as dystopian. All this incredible technology is a double edges sword, but doesn&#x27;t solve the problems of the day, only the problems of business efficiency, which exacerbates the problems of the day.

I completely agree that technology in the last couple years has genuinely been fulfilling the promise established in my childhood sci-fi.The other day, alone in a city I&#x27;d never been to before, I snapped a photo of a bistro&#x27;s daily specials hand-written on a blackboard in Chinese, copied the text right out of the photo, translated it into English, learned how to pronounce the menu item I wanted, and ordered some dinner.Two years ago this story would have been: notice the special board, realize I don&#x27;t quite understand all the characters well enough to choose or order, and turn wistfully to the menu to hopefully find something familiar instead. Or skip the bistro and grab a pre-packaged sandwich at a convenience store.

Last time I used Whisper with a foreign language (Chinese) video, I’m pretty sure it just made some stuff up.The captions looked like they would be correct in context, but I could not cross-reference them with snippets of manually checked audio, to the best of my ability.

We have the tools to do this, and will have commercial products for everything you listed in the next couple years.

this is very dismissive and binary and thats what this whole article is about. AI skeptic expect that&#x27;s either AGI or perfect with all use cases or otherwise its useless. SST, translation and TTS went really far away in last 2 years. My mother who doesn&#x27;t speak english find it very useful when she my sister in US. I find it super useful while travelling in asia. Definitely much more useful than what we had in Google Translate.

&gt; Am I the only one who remembers when that was the stuff of science fiction?Would you go to a foreign country and sign a work contract based on the LLM translation ?Would you answer a police procedure based on the speech recognition alone ?That to me was the promise of the science fiction. Going to another planet and doing inter-species negotiations based on machine translation. We&#x27;re definitely not there IMHO, and I wouldn&#x27;t be surprised if we don&#x27;t quite get there in our lifetime.Otherwise if we&#x27;re lowering the bar, speech to text has been here for decades, albeit clunky and power hungry. So improvements have been made, but watching old movies is a way too low stake situation IMHO.

&gt; Is it smart enough to insert the time codes such that the subtitle is well enough synchronised to the spoken line?Yes, Whisper has been able to do this since the first release. At work we use it to live-transcribe-and-translate all-hands meetings and it works very well.

Using AI to generate subtitles is inventive. Is it smart enough to insert the time codes such that the subtitle is well enough synchronised to the spoken line?As someone who has started losing the higher frequencies and thus clarity, I have subtitles on all the time just so I don&#x27;t miss dialogue. The only pain point is when the subtitles (of the same language) are not word-for-word with the spoken line. The discordance between what you are reading and hearing is really distracting.This is my major peeve with my The West Wing DVDs, where the subtitles are often an abridgement of the spoken line.

I tried whisper with a movie from the 60&#x27;s and it was a disaster.Not sure if it was due to the poor quality of the sound, the fact people used to speak a bit differently 60 years ago or that 3 different languages were used (plot took place in France during WW2).

I started watching Leverage, the TV show, on Amazon, and the subtitles in the early series are clearly AI generated or just bad by default.I use subtitles because I don’t want to micromanage the volume on my TV when adverts are forced on me and they are 100x louder than than what I was watching.

would be interesting if court transcriptions can be handled by these models.

Machine translation and speech recognition. The state of the art for these is a multi-modal language model. I&#x27;m hearing impaired veering on deaf, and I use this technology all day every day. I wanted to watch an old TV series from the 1980s. There are no subtitles available. So I fed the show into a language model (Whisper) and now I have passable subtitles that allow me to watch the show.Am I the only one who remembers when that was the stuff of science fiction? It was not so long ago an open question if machines would ever be able to transcribe speech in a useful way. How quickly we become numb to the magic.

A well articulated blog, imo. Touches on all the points I see argued about on LinkedIn all the time.I think leveling things out at the beginning is important. For instance, I recently talked to a senior engineer who said &quot;using AI to write programming is so useless&quot;, but then said they&#x27;d never heard of Cursor. Which is fine - but I so often see strong vocal stances against using AI tools but then referring to early Copilot days or just ChatGPT as their experience, and the game has changed so much since then.

It’s surreal to me been using ChatGPT everyday for 2 years, makes me question reality sometimes like ‘howtf did I live to see this in my lifetime’I’m only 39, really thought this was something reserved for the news on my hospital tv deathbed.

Today I had a dentist appointment and the dentist suggested I switch toothpaste lines to see if something else works for my sensitivity better.I am predisposed to canker sores and if I use a toothpaste with SLS in it I&#x27;ll get them. But a lot of the SLS free toothpastes are new age hippy stuff and is also fluoride free.I went to chatgpt and asked it to suggest a toothpaste that was both SLS free and had fluoride. Pretty simple ask right?It came back with two suggestions. It&#x27;s top suggestion had SLS, it&#x27;s backup suggestion lacked fluoride.Yes, it is mind blowing the world we live in. Executives want to turn our code bases over to these tools

I remember the first time I played with GPT and thought “oh, this is fully different from the chatbots I played with growing up, this isn’t like anything else I’ve seen” (though I suppose it is implemented much like predictive text, but the difference in experience is that predictive text is usually wrong about what I’m about to say so it feels silly by comparison)

I find it sad how normalized it&#x27;s become.  Yes, the technology is imperfect in very meaningful ways.  Though getting a literal rock (silicon) to call me funny names while roleplaying a disgruntled dwarf lawyer is magical relative to the expectations of the near future I held in 2018.

It&#x27;s almost exactly one of the stories in Stanislaw Lem&#x27;s The Cyberiad.

Been vibe coding for the past couple of months on a large project. My mind is truly blown. Every day it&#x27;s just shocking. And it&#x27;s so prolific. Half a million lines of code in a couple of months by one dev. Seriously.Note that it&#x27;s not going to solve everything. It&#x27;s still not very precise in its output. Definitely lots of errors and bad design at the top end. But it&#x27;s a LOT better than without vibe coding.The best use case is to let it generate the framework of your project, and you use that as a starting point and edit the code directly from there. Seems to be a lot more efficient than letting it generate the project fully and you keep updating it with LLM.

It&#x27;s mind blowing. At least 1-2x&#x2F;week I find myself shocked that this is the reality we live in

Honestly LLMs are a great canary if your documentation &#x2F; language &#x2F; whatever is &#x27;good&#x27; at all.I wish I would have kept it around but had ran into an issue where the LLM wasn&#x27;t giving a great answer. Look at the documentation, and yea, made no sense. And all the forum stuff about it was people throwing out random guessing on how it should actually work.If you&#x27;re a company that makes something even moderately popular and LLMs are producing really bad answers there is one of two things happening.1. Your a consulting company that makes their money by selling confused users solutions to your crappy product
2. Your documentation is confusing crap.

I&#x27;ve just got good at reading code, because that&#x27;s the one constant you can rely one (unless you&#x27;re using some licensed library). So whenever the reference is not enough, I just jump straight to the code (one of my latest examples is finding out that opendoas (a sudo replacement) hard code the persist option for not asking password to 5 minutes).

Though I haven’t embraced LLM codegen (except for non-functional filler&#x2F;test data), the fuzziness is why I like to use them as talking documentation. It makes for a lot less of fumbling around in the dark trying to figure out the magic combination of search keywords to surface the information needed, which can save a lot of time in aggregate.

It will, or it might? Because if every time you use an LLM is misinterprets your input as something easier to solve, you might want to brush up on the fundamentals of the tool(I see some people are quite upset with the idea of having to mean what you say, but that&#x27;s something that serves you well when interacting with people, LLMs, and even when programming computers.)

When you have a precise input, why give it to an LLM? When I have to do arithmetic, I use a calculator. I don&#x27;t ask my coworker, who is generally pretty good at arithmetic, although I&#x27;d get the right answer 98% of the time. Instead, I use my coworker for questions that are less completely specified.Also, if it&#x27;s an important piece of arithmetic, and I&#x27;m in a position where I need to ask my coworker rather than do it myself, I&#x27;d expect my coworker (and my AI) to grab (spawn) a calculator, too.

Well said, these things are actually in a tradeoff with each other. I feel like a lot of people somehow imagine that you could have the best of both, which is incoherent short of mind-reading + already having clear ideas in the first place.But thankfully we do have feedback&#x2F;interactiveness to get around the downsides.

The other side of the coin is that if you give it a precise input, it will fuzzily interpret it as something else that is easier to solve.

right: we don&#x27;t use programming languages instead of natural language simply to make it hard. For the same reason, we use a restricted dialect of natural language when writing math proofs -- using constrained languages reduces ambiguity and provides guardrails for understanding. It gives us some hope of understanding the behavior of systems and having confidence in their outputsThere are levels of this though -- there are few instances where you actually need formal correctness. For most software, the stakes just aren&#x27;t that high, all you need is predictable behavior in the &quot;happy path&quot;, and to be within some forgiving neighborhood of &quot;correct&quot;.That said, those championing AI have done a very poor job at communicating the value of constrained languages, instead preferring to parrot this (decades and decades and decades old) dream of &quot;specify systems in natural language&quot;

It sounds like you think I don&#x27;t find value in using machines in their precise way, but that&#x27;s not a correct assumption. I love code! I love the algorithms and data structures of data science. I also love driving 5-speed transmissions and shooting on analog film – but it isn&#x27;t always what&#x27;s needed in a particular context or for a particular problem. There are lots of areas where a &#x27;good enough solution done quickly&#x27; is way more valuable than a 100% correct and predictable solution.

&quot;I got into this profession precisely because I wanted to give precise instructions to a machine and get exactly what I want.&quot;So you didn&#x27;t get into this profession to be lead then eh?Because essentially, that&#x27;s what Thomas in the article is describing (even if he doesn&#x27;t realize it). He is a mini-lead with a team of a few junior and lower-mid-level engineers - all represented by LLM and agents he&#x27;s built.

That’s interesting. I got into computing because unlike school where wrong answers gave you indelible red ink and teachers had only finite time for questions, computers were infinitely patient and forgiving. I could experiment, be wrong, and fix things. Yes I appreciated that I could calculate precise answers but it was much more about the process of getting to those answers in an environment that encouraged experimentation. Years later I get huge value from LLMs, where I can ask exceedingly dumb questions to an indefatigable if slightly scatterbrained teacher. If I were smart enough, like Dijkstra, to be right first time about everything, I’d probably find them less useful, but sadly I need cajoling along the way.

&gt;simple fact that you can now be fuzzy with the input you give a computer, and get something meaningful in returnI got into this profession precisely because I wanted to give precise instructions to a machine and get exactly what I want. Worth reading Dijkstra, who anticipated this, and the foolishness of it, half a century ago&quot;Instead of regarding the obligation to use formal symbols as a burden, we should regard the convenience of using them as a privilege: thanks to them, school children can learn to do what in earlier days only genius could achieve. (This was evidently not understood by the author that wrote —in 1977— in the preface of a technical report that &quot;even the standard symbols used for logical connectives have been avoided for the sake of clarity&quot;. The occurrence of that sentence suggests that the author&#x27;s misunderstanding is not confined to him alone.) When all is said and told, the &quot;naturalness&quot; with which we use our native tongues boils down to the ease with which we can use them for making statements the nonsense of which is not obvious.[...]It may be illuminating to try to imagine what would have happened if, right from the start our native tongue would have been the only vehicle for the input into and the output from our information processing equipment. My considered guess is that history would, in a sense, have repeated itself, and that computer science would consist mainly of the indeed black art how to bootstrap from there to a sufficiently well-defined formal system. We would need all the intellect in the world to get the interface narrow enough to be usable&quot;Welcome to prompt engineering and vibe coding in 2025, where you have to argue with your computer to produce a formal language, that we invented in the first place so as to not have to argue in imprecise language<a href="https:&#x2F;&#x2F;www.cs.utexas.edu&#x2F;~EWD&#x2F;transcriptions&#x2F;EWD06xx&#x2F;EWD667.html" rel="nofollow">https:&#x2F;&#x2F;www.cs.utexas.edu&#x2F;~EWD&#x2F;transcriptions&#x2F;EWD06xx&#x2F;EWD667...</a>

See I try that sort of thing, like asking Gemini about a science fiction book I read in 5th grade that (IIRC) involved people living underground near&#x2F;under a volcano, and food in pill form, and it immediately hallucinates a non-existent book by John Christopher named &quot;The City Under the Volcano&quot;

Wake me up when LLMs render the world a better place by simply prompting them &quot;make me happy&quot;. Now that&#x27;s gonna be a true win of fuzzy inputs!

You can be fuzzier than a soft fluff of cotton wool. I’ve had incredible success trying to find the name of an old TV show or specific episode using AIs. The hit rate is surprisingly good even when using the vaguest inputs.“You know, that show in the 80s or 90s… maybe 2000s with the people that… did things and maybe didn’t do things.”“You might be thinking of episode 11 of season 4 of such and such snow where a key plot element was both doing and not doing things on the penalty of death”

Thanks, love this – it&#x27;s something I&#x27;ve thought about as well!

&gt; The writers were always adamant that Data had no emotions... but quite often made faces reflecting the mood of the room or indicating he understood jokes made by other crew members.This doesn&#x27;t seem too different from how our current AI chatbots don&#x27;t actually understand humor or have emotions, but can still explain a joke to you or generate text with a humorous tone if you ask them to based on samples, right?&gt; &quot;Hey computer, we&#x27;re all about to die, solve the plot for us so we make it to next week&#x27;s episode!&quot;I&#x27;m curious, do you recall a specific episode or two that reflect what you feel boiled down to this?

&gt;&quot;Being a robot&#x27;s great, but we don&#x27;t have emotions and sometimes that makes me very sad&quot;.From Futurama in a obvious parody of how Data was portrayed

ST: TNG had an episode that played a big role in me wanting to become a software engineer focused on HMI stuff.It&#x27;s the relatively crummy season 4 episode Identity Crisis, in which the Enterprise arrives at a planet to check up on an away team containing a college friend of Geordi&#x27;s, only to find the place deserted. All they have to go on is a bodycam video from one of the away team members.The centerpiece of the episode is an extended sequence of Geordi working in close collaboration with the Enterprise computer to analyze the footage and figure out what happened, which takes him from a touchscreen-and-keyboard workstation (where he interacts by voice, touch and typing) to the holodeck, where the interaction continues seamlessly. Eventually he and the computer figure out there&#x27;s a seemingly invisible object casting a shadow in the reconstructed 3D scene and back-project a humanoid form and they figure out everyone&#x27;s still around, just diseased and ... invisible.I immediately loved that entire sequence as a child, it was so engrossingly geeky. I kept thinking about how the mixed-mode interaction would work, how to package and take all that state between different workstations and rooms, have it all go from 2D to 3D, etc. Great stuff.

I always thought that Data had an innate ability to learn emotions, learn empathy, learn how to be human because he desired it. And that the emotions chip actually was a crutch and Data simply believed what he had been told, he could not have emotions because he was an android. But, as you say, he clearly feels close to Geordi and cares about him. He is afraid if Spot is missing. He paints and creates music and art that reflects his experience. Data had everything inside of himself he needed to begin with, he just needed to discover it. Data, was an example to the rest of us. At least in TNG. In the movies he was a crazy person. But so was everyone else.

I was a big fan of Star Trek: The Next Generation as a kid and one of my favorite things in the whole world was thinking about the Enterprise&#x27;s computer and Data, each one&#x27;s strengths and limitations, and whether there was really any fundamental difference between the two besides the fact that Data had a body he could walk around in.The Enterprise computer was (usually) portrayed as fairly close to what we have now with today&#x27;s &quot;AI&quot;: it could synthesize, analyze, and summarize the entirety of Federation knowledge and perform actions on behalf of the user. This is what we are using LLMs for now. In general, the shipboard computer didn&#x27;t hallucinate except during most of the numerous holodeck episodes. It could rewrite portions of its own code when the plot demanded it.Data had, in theory, a personality. But that personality was basically, &quot;acting like a pedantic robot.&quot; We are told he is able to grow intellectually and acquire skills, but with perfect memory and fine motor control, he can already basically &quot;do&quot; any human endeavor with a few milliseconds of research. Although things involving human emotion (art, comedy, love) he is pretty bad at and has to settle for sampling, distilling, and imitating thousands to millions of examples of human creation. (Not unlike &quot;AI&quot; art of today.)Side notes about some of the dodgy writing:A few early epsiodes of Star Trek: The Next Generation treated the Enterprise D computer as a semi-omniscient character and it always bugged me. Because it seemed to &quot;know&quot; things that it shouldn&#x27;t and draw conclusions that it really shouldn&#x27;t have been able to. &quot;Hey computer, we&#x27;re all about to die, solve the plot for us so we make it to next week&#x27;s episode!&quot; Thankfully someone got the memo and that only happened a few times. Although I always enjoyed episodes that centered around the ship or crew itself somehow instead of just another run-in with aliens.The writers were always adamant that Data had no emotions (when not fitted with the emotion chip) but we heard him say things _all the time_ that were rooted in emotion, they were just not particularly strong emotions. And he claimed to not grasp humor, but quite often made faces reflecting the mood of the room or indicating he understood jokes made by other crew members.

I literally pasted these two lines into ChatGPT that were sent to me by one of sysadmin and it told me exactly what I needed to know:<pre><code> App1: requestedAccessTokenVersion&quot;: null
 App2: requestedAccessTokenVersion&quot;: 2
</code></pre>
I use it like that all time. In fact, I&#x27;m starting to give it less and less context and just toss stuff at it. It&#x27;s more efficient use of my time.

I want to be fuzzy and I want the LLM to generate something exact.

It can get you 80% of the way there, you can still be exacting in telling it where it went wrong or fine tuning the result by hand.

But when I&#x27;m doing my job as a software developer, I don&#x27;t want to be fuzzy. I want to be exact at telling the computer what to do, and for that, the most efficient way is still a programming language, not English. The only place where LLMs are an improvement is voice assistants. But voice assistants themselves are rather niche.

That&#x27;s interesting to me, because saying &quot;Delete all the screenshots on my Desktop&quot; is not at all how I want to be using my computer. When I&#x27;m getting breakfast, I don&#x27;t instruct the banana to &quot;peel yourself and leap into my mouth,&quot; then flop open my jaw like a guppy. I just grab it and eat it. I don&#x27;t want to tell my computer to delete all the screenshots (except for this or that that particular one). I want to pull one aside, sweep my mouse over the others, and tap &quot;delete&quot; to vanish them.There&#x27;s a &quot;speaking and interpreting instructions&quot; vibe to your answer which is at odds with my desire for an interface that feels like an extension of my body. For the most part, I don&#x27;t want English to be an intermediary between my intent and the computer. I want to do, not tell.

&gt; I want to be able to say &quot;Delete all the screenshots on my Desktop&quot;, instead of going into a terminal and typing &quot;rm ~&#x2F;Desktop&#x2F;*.png&quot;.Both are valid cases, but one cannot replace the other—just like elevators and stairs. The presence of an elevator doesn&#x27;t eliminate the need for stairs.

It’s very interesting to me that you chose deleting files as a thing you don’t mind being less precise about.

I personally can&#x27;t see this example working out. I&#x27;ll always want to get some kind of confirmation of which files will be deleted, and at that point, just typing the command out is much easier than reading.

The junior will repeatedly ask the AI to delete the screenshots. Until he forgets what is the command to delete a file.The engineer will wonder why his desktop is filled his screenshots, change the settings that make it happen, and forget about it.That behavior happened for years before AI, but AI will make that problem exponentially worse. Or I do hope that was a bad example.

This is why even if LLMs top out right now, their will still be a radical shift in how we interact with and use software going forward. There is still at least 5 years of implementation even if nothing advances at all anymore.No one is ever going to want to touch a settings menu again.

&gt; I want to be able to say &quot;Delete all the screenshots on my Desktop&quot;, instead of going into a terminal and typing &quot;rm ~&#x2F;Desktop&#x2F;*.png&quot;.But why? It takes many more characters to type :)

It&#x27;s a radical change in human&#x2F;computer interface. Now, for many applications, it is much better to present the user with a simple chat window and allow them to type natural language into it, rather than ask them to learn a complex UI. I want to be able to say &quot;Delete all the screenshots on my Desktop&quot;, instead of going into a terminal and typing &quot;rm ~&#x2F;Desktop&#x2F;*.png&quot;.

not if you consider how confused our ideas are today

Now with LLMs, you can put in the right figures and the wrong answers might come out.

&gt;On two occasions, I have been asked [by members of Parliament], &#x27;Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?&#x27; I am not able to rightly apprehend the kind of confusion of ideas that could provoke such a question.
- Charles BabbageThis quote did not age well

&gt; The big change with LLMs seems to be that everyone now has an opinion on what programming&#x2F;AI is and can doTrue, but I think this is just the zeitgeist. People today want to share their dumb opinions about any complex subject after they saw a 30 second reel.

For me this moment came when Google calendar first let you enter fuzzy text to get calendar events added, this was around 2011, I think. In any case, for the end user this can be made to happen even when the computer cannot actually handle fuzzy inputs (which is of course, how an LLM works).The big change with LLMs seems to be that everyone now has an opinion on what programming&#x2F;AI is and can do. I remember people behaving like that around stocks not that long ago…

We wanted to check the clock at the wrong time but read the correct time. Since a broken clock is right twice a day, we broke the clock, which solves our problem some of the time!

It is fun to watch. I&#x27;ve sometimes indeed seen the LLM say something like &quot;I&#x27;m assuming you meant [X]&quot;.

It&#x27;s very impressive that I can type misheard song lyrics into Google, and yet still have the right song pop up.But, having taken a chance to look at the raw queries people type into apps, I&#x27;m afraid neither machine nor human is going to make sense of a lot of it.

We&#x27;re talking about God function.function God (any param you can think of) {}

theseday,s i ofen donot correct my typos even wheni notice them while cahtting with LLMS. So far 0 issues.

&gt; This has been an obviously absurd question for two centuries now. Turns out the people asking that question were just visionaries ahead of their time.This is not the point of that Babbage quote, and no, LLMs have not solved it, because it cannot be solved, because &quot;garbage in, garbage out&quot; is a fundamental observation of the limits of logic itself, having more to with the laws of thermodynamics than it does with programming. The output of a logical process cannot be more accurate than the inputs to that process; you cannot conjure information out of the ether. The LLM isn&#x27;t the logical process in this analogy, it&#x27;s one of the inputs.

Well, you can enter 4-5 relatively vague keywords into google and first&#x2F;second stackoverflow link will probably provide plenty of relevant code. Given that, its much less impressive since &gt;95% of the problems and queries just keep repeating.

There&#x27;s the old quote from Babbage:&gt; On two occasions I have been asked, &#x27;Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?&#x27; I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.This has been an obviously absurd question for two centuries now. Turns out the people asking that question were just visionaries ahead of their time.It is kind of impressive how I&#x27;ll ask for some code in the dumbest, vaguest, sometimes even wrong way, but so long as I have the proper context built up, I can get something pretty close to what I actually wanted. Though I still have problems where I can ask as precisely as possible and get things not even close to what I&#x27;m looking for.

It can give back code though. It might be wrong, but it won’t be ambiguous.

Sure, you can now be fuzzy with the input you give to computers, but in return the computer will ALSO be fuzzy with the answer it gives back. That&#x27;s the drawback of modern AI.

Computers finally work they way they were always supposed to work :)

If anything we now need to unlearn the rigidity - being too formal can make the AI overly focused on certain aspects, and is in general poor UX. You can always tell legacy man-made code because it is extremely inflexible and requires the user to know terminology and usage implicitly lest it break, hard.For once, as developers we are actually using computers how normal people always wished they worked and were turned away frustratedly. We now need to blend our precise formal approach with these capabilities to make it all actually work the way it always should have.

In my opinion, most of the problems we see now with LLMs come from being fuzzy ... I&#x27;m used to getting very good code from claude o gemini (copy and paste without any changes that just works) but I have to be very specific, sometime it takes longer to write the prompt than writing the code itself.If I&#x27;m fuzzy, the output quality is usually low and I need several iterations before getting an acceptable result.At some point, in the future, there will be some kind of formalization on how to ask swe question to llms ... and we will get another programming language to rule the all :D

It invalidates this CinemaSins nitpick on Alien completely<a href="https:&#x2F;&#x2F;youtu.be&#x2F;dJtYDb7YaJ4?si=5NuoXaW0pkGoBSJu&amp;t=76" rel="nofollow">https:&#x2F;&#x2F;youtu.be&#x2F;dJtYDb7YaJ4?si=5NuoXaW0pkGoBSJu&amp;t=76</a>

One thing that I find truly amazing is just the simple fact that you can now be fuzzy with the input you give a computer, and get something meaningful in return. Like, as someone who grew up learning to code in the 90s it always seemed like science fiction that we&#x27;d get to a point where you could give a computer some vague human level instructions and get it more or less do what you want.

Maybe make a video of how you&#x27;re vibecoding a valuable project in an existing codebase, and how agents are saving you time by running your tools in a loop.

I’m uninterested in giving you content. In particular because of your past behavior.Thanks for the offer though.

PLEASE write your response. We&#x27;ll publish it on the Fly.io blog. Unedited. If you want.

Can you direct me somewhere with superior counterarguments? I&#x27;m quite curious

&gt;&gt; with poor arguments every week&gt;This roughly matches my experience too, but I don&#x27;t think it applies to this one.I&#x27;m not so sure. The argument that any good programming language would inherently eliminate the concern for hallucinations seems like a pretty weak argument to me.

&gt; with poor arguments every weekThis roughly matches my experience too, but I don&#x27;t think it applies to this one. It has a few novel things that were new ideas to me and I&#x27;m glad I read it.&gt; I’m ready to write a boilerplate response because I already know what they’re going to sayIf you have one that addresses what this one talks about I&#x27;d be interested in reading it.

Which of the arguments in this post hasn&#x27;t occurred on HN in the past month or so?

There&#x27;s also the reverse genre: valid criticism of absolutely strawman arguments that nobody makes.

&gt;I feel the opposite, and pretty much every metric we have shows basically linear improvement of these models over time.Wait, what kind of metric are you talking about? When I did my masters in 2023 SOTA models where trying to push the boundaries by minuscule amounts. And sometimes blatantly changing the way they measure &quot;success&quot; to beat the previous SOTA

Models are absolutely not improving linearly. They improve logarithmically with size, and we&#x27;ve already just about hit the limits of compute without becoming totally unreasonable from a space&#x2F;money&#x2F;power&#x2F;etc standpoint.We can use little tricks here and there to try to make them better, but fundamentally they&#x27;re about as good as they&#x27;re ever going to get. And none of their shortcomings are growing pains - they&#x27;re fundamental to the way an LLM operates.

&quot;pretty much every metric we have shows basically linear improvement of these models over time.&quot;They&#x27;re also trained on random data scraped off the Internet which might include benchmarks, code that looks like them, and AI articles with things like chain of thought. There&#x27;s been some effort to filter obvious benchmarks but is that enough? I cant know if the AI&#x27;s are getting smarter on their own or more cheat sheets are in the training data.Just brainstorming, one thing I came up with is training them on datasets from before the benchmarks or much AI-generated material existed. Keep testing algorithmic improvements on that in addition to models trained on up to date data. That might be a more accurate assessment.

I feel the opposite, and pretty much every metric we have shows basically linear improvement of these models over time.The criticisms I hear are almost always gotchas, and when confronted with the benchmarks they either don’t actually know how they are built or don’t want to contribute to them. They just want to complain or seem like a contrarian from what I can tell.Are LLMs perfect? Absolutely not. Do we have metrics to tell us how good they are? YesI’ve found very few critics that actually understand ML on a deep level. For instance Gary Marcus didn’t know what a test train split was. Unfortunately, rage bait like this makes money

What valid AI criticisms? Most criticisms of AI are not very deep nor founded in complexity theoretic arguments, whereas Yann LeCun himself gave an excellent 1 slide explanation of the limits of LLMs. Most AI criticisms are low quality arguments.

“Valid” criticism rarely come from the people barely capable of understanding the difference between AI and LLMs, and using them interchangeably.

I feel like we get one of these articles that addresses valid AI criticisms with poor arguments every week and at this point I’m ready to write a boilerplate response because I already know what they’re going to say.Interns don’t cost 20 bucks a month but training users in the specifics of your org is important.Knowing what is important or pointless comes with understanding the skill set.

Is there a term for “skeptics just haven’t used it enough” argument?Because it frequently got rolled out in crypto-currency arguments too.

Exactly, seems much skepticism comes from only scratching the surface of what&#x27;s possible.

It&#x27;s been so much more rewarding playing with AI coding tools on my own than through the subtle and not so subtle nudges at work. The work AI tools are a walled garden, have a shitty interface, feel made to extract from me than to help me. In my personal stuff, downloading models, playing with them, the tooling, the interactions, it all been so much more rewarding to give me stable comfortable workflows I can rely on and that work with my brain.The dialog around it is so adversarial it&#x27;s been hard figuring out how to proceed until dedicating a lot of effort to diving into the field myself, alone, on my personal time and learned what&#x27;s comfortable to use it on.

An important note is that the models you can host at home (e.g. without buying ten(s of) thousand dollar rigs) won&#x27;t be as effective as the proprietary models. A realistic size limit is around 32 billion parameters with quantisation, which will fit on a 24GB GPU or a sufficiently large MBP. These models are roughly on par with the original GPT-4 - that is, they will generate snippets, but they won&#x27;t pull off the magic that Claude in an agentic IDE can do. (There&#x27;s the recent Devstral model, but that requires a specific harness, so I haven&#x27;t tested it.)DeepSeek-R1 is on par with frontier proprietary models, but requires a 8xH100 node to run efficiently. You can use extreme quantisation and CPU offloading to run it on an enthusiast build, but it will be closer to seconds-per-token territory.

You can self host an open-weights LLM. Some of the AI-powered IDEs are open source. It does take a little more work than just using VSCode + Copilot, but that&#x27;s always been the case for FOSS.

It&#x27;s unfortunate that AMD isn&#x27;t in on the AI stuff, because they are releasing a 96GB card ($10k so it&#x27;s pricey currently) which would drop the number you need.

I mean it depends on the model; some people running deepseek report they have better performance at home running on a CPU with lots of ram (think a few hundred gigabytes). Even when running locally vram is more relevant than the performance of the GPU. That said I&#x27;m really not the person to ask about this, as I don&#x27;t have AI agents running amuck on my machine yet

Yeah, I&#x27;m ready to jump in, but I need an agent running on my hardware at home without internet access.How far away are we from that? 
How many RYX 50s do I need?This is a serious question btw.

Damn. Well I&#x27;ll spend a few bucks trying it out and I&#x27;ll ask my employer if they&#x27;re okay with me using agents on company time, butBut I&#x27;m not thrilled about centralized, paid tools. I came into software during a huge FOSS boom. Like a huge do it yourself, host it yourself, Publish Own Site, Syndicate Elsewhere, all the power to all the people, borderline anarchist communist boom.I don&#x27;t want it to be like other industries where you have to buy a dog shit EMR and buy a dog shit CAD license and buy a dog shit tax prep license.Maybe I lived through the whale fall and Moloch is catching us. I just don&#x27;t like it. I rage against dying lights as a hobby.

Some of us thrive in tedium, and also do it better than bots.

Unrelated to your friends, but a big part of learning is to do tedious tasks. Maybe once you master a topic LLMs can be better, but for many folks out there, using LLMs as a shortcut can impede learning.

I don&#x27;t know the full answer to this question, but I have a partial answer: they should at least stop doing tedious tasks that LLMs do better.

&gt; Extraordinarily talented people are doing work that LLMs already do better, out of spite.So what, people should just stop doing any tasks that LLMs do subjectively better?

you are giving it instructions but it&#x27;s running a while loop with a list of tools and it can poke around in your code base until it thinks it&#x27;s done whatever you ask for.See Claude Code, windsurf, amp, Kilcode, roo, etc.I might describe a change I need to have made and then it does it and then I might say &quot;Now the tests are failing. Can you fix them?&quot; and so on.Sometimes it works very great. sometimes you find yourself arguing with the computer.

This example seems to keep coming up.  Why do you need an AI to run linters?  I have found that linters actually add very little value to an experience programmer, and actually get in the way when I am in the middle of active development.  I have to say I&#x27;m having a hard time visualizing the amazing revolution that is alluded to by the author.

I run Cursor in a mode that starts up shell processes, runs linters, tests etc on its own, updates multiple files, runs the linter and tests again, fixes failures, and so on. It auto stops at 20 iterations through the feedback loop.Depending on the task it works really well.

I&#x27;m sorry but this doesn&#x27;t explain anything. Whatever it is you have in your mind, I&#x27;m afraid it&#x27;s not coming across on the page. There is zero chance that I&#x27;m going to let an AI start running arbitrary commands on my PC, let alone anything that resembles a commit.What is an actual, real world example?

I cut several paragraphs from this explaining how agents work, which I wrote anticipating this exact comment. I&#x27;m very happy to have brought you to this moment of understanding --- it&#x27;s a big one. The answer is &quot;yes, that&#x27;s exactly what people are doing&quot;: &quot;turning LLMs loose&quot; (really, giving them some fixed number of tool calls, some of which might require human approval) to do stuff on real systems. This is exactly what Cursor is about.I think it&#x27;s really hard to undersell how important agents are.We have an intuition for LLMs as a function blob -&gt; blob (really, token -&gt; token, but whatever), and the limitations of such a function, ping-ponging around in its own state space, like a billion monkeys writing plays.But you can also get go blob -&gt; json, and json -&gt; tool-call -&gt; blob. The json-&gt;tool interaction isn&#x27;t stochastic; it&#x27;s simple systems code (the LLM could indeed screw up the JSON, since that process is stochastic --- but it doesn&#x27;t matter, because the agent isn&#x27;t stochastic and won&#x27;t accept it, and the LLM will just do it over). The json-&gt;tool-call-&gt;blob process is entirely fixed system code --- and simple code, at that.Doing this grounds the code generation process. It has a directed stochastic structure, and a closed loop.

&gt; You&#x27;re directing the work, but at a higher level of abstraction.The article likens this to a Makefile. I gotta say, why not just use a Makefile and save the CO2?

&gt; Is this what people are really doing?Some people are, and some people are not. This is where some of the disconnect is coming from.&gt; Who is just turning AI loose to modify things as it sees fit?In the advent of source control, why not? If it does something egregiously wrong, you can throw it away easily and get back to a previous state with ease.&gt; If I&#x27;m not directing the work, how does it even know what to do?You&#x27;re directing the work, but at a higher level of abstraction.

This is how I work:I use Cursor by asking it exactly what I want and how I want it. By default, Cursor has access to the files I open, and it can reference other files using grep or by running specific commands. It can edit files.It performs well in a fairly large codebase, mainly because I don’t let it write everything. I carefully designed the architecture and chose the patterns I wanted to follow. I also wrote a significant portion of the initial codebase myself and created detailed style guides for my teammates.As a result, Cursor (or you can say models you selecting because cursor is just a router for commercial models) handles small, focused tasks quite well. I also review every piece of code it generates. It&#x27;s particularly good at writing tests, which saves me time.

Zed has a great four minute demo showing how it works: <a href="https:&#x2F;&#x2F;zed.dev&#x2F;agentic" rel="nofollow">https:&#x2F;&#x2F;zed.dev&#x2F;agentic</a>I personally have my Zed set up so the agent has to request every command be manually reviewed and approved before running.

I let an agent upgrade some old C code that wouldn’t compile and had 100’s of warnings. It was running builds on its own, looking at new errors, etc. It even wrote some tests! I could’ve done this myself but it was a hobby project and tedious work. I was impressed.

Can someone explain to me what this means?&gt; People coding with LLMs today use agents. Agents get to poke around your codebase on their own. They author files directly. They run tools. They compile code, run tests, and iterate on the results. ...Is this what people are really doing? Who is just turning AI loose to modify things as it sees fit? If I&#x27;m not directing the work, how does it even know what to do?I&#x27;ve been subjected to forced LLM integration from management, and there are no &quot;Agents&quot; anywhere that I&#x27;ve seen.Is anyone here doing this that can explain it?

&gt; But all day, every day, a sizable chunk of the front page of HN is allocated to LLMs: incremental model updates, startups doing things with LLMs, LLM tutorials, screeds against LLMs. It’s annoying!You forgot the screeds against the screeds (like this one)

It is false and you&#x27;re proving it. Smarter means smarter.Smarter does not mean &quot;better at writing and shipping infrastructure code.&quot;Some of the smartest people I know are also infra engineers and none of them are AI skeptics in 2025.

It&#x27;s not false. He&#x27;s talking about people smarter than him (at writing and shipping infrastructure code).Thomas is the smartest at other things.

&quot;they’re smarter than me&quot; feels like false humility and an attempt to make the medicine go down better.1. Thomas is obviously very smart.2. To be what we think of as &quot;smart&quot; is to be in touch with reality, which includes testing AI systems for yourself and recognizing their incredible power.

I find the AI proponents have an insane level of egocentrismThey cannot possibly imagine someone has a different use case where the AI didn&#x27;t work&quot;I crank out shitty webapps all day, therefore every single other dev does. Everyone obviously has the same use case as me because I am the center of the universe&quot;

A writing for the ages. I&#x27;ve found most of the LLM skeptics are either being hypocritical or just being gate-keepy (we dont want everyone to write code)

this behaviour is literally removable with proper prompting.this is a strawmans argument... of whatever your are arguing

Yeah I tried Copilot&#x27;s automatic commit messages and they&#x27;re trash, but the agent-based ones are much better.

If you&#x27;ve ever run or been part of a team that does thorough, multi-party, pull request reviews you know what I am talking about.The only part I don&#x27;t automate is the pull request review (or patch review, pre-commit review, etc. before git.), thats always been the line to hold for protecting codebases with many contributors of varying capability, this is explicitly addressed in the article as well.You can fight whatever straw man you want. Shadowbox the hypotheticals in your head, etc. I don&#x27;t get all these recent and brand new accounts just straight up insulting and insinuating all this crap all over HN today.

Try Cubic, which is a Github add-on. Really good at writing GH commit messages and also surfaces bugs fairly reliably (adds PR comments). Not affiliated, just a user.

&gt; I haven&#x27;t written a commit summary in monthsI don&#x27;t know what you&#x27;re posting, but if it&#x27;s anything like what I see being done by GitHub copilot, your commit messages are junk. They&#x27;re equivalent to this and you&#x27;re wasting everyone&#x27;s time:<pre><code> &#x2F;&#x2F; Sets the value
 const value = &quot;red&quot;</code></pre>

For me, the electric bike analogy works differently: it enables people to ride, regularly, who would not be able to do that with traditional bikes. That&#x27;s totally fine. But electric bikes don&#x27;t threaten to take away our normal bikes.

I think the hardest part is not spending the next 3 months of my life in a cave finishing all the hobby&#x2F;side projects I didn&#x27;t quite get across the line.It really does feel like I&#x27;ve gone from being 1 senior engineer to a team that has a 0.8 Sr. Eng, 5 Jrs. and one dude that spends all his time on digging through poorly documented open source projects and documenting them for the team.Sure I can&#x27;t spend quite as much time working on hard problems as I used to, but no one knows that I haven&#x27;t talked to a PM in months, no one knows I haven&#x27;t written a commit summary in months, it&#x27;s just been my AI doppelgangers. Compared to myself a year ago I think I now PERSONALLY write 150% more HARD code than I did before. So maybe, my first statement about being 0.8 is false.I think of it like electric bikes, there seems to be indication that people with electric assist bikes actually burn more calories&#x2F;spend more time&#x2F;go farther on an electric bike than those who have manual bikes <a href="https:&#x2F;&#x2F;www.sciencedirect.com&#x2F;science&#x2F;article&#x2F;abs&#x2F;pii&#x2F;S2214140516303930" rel="nofollow">https:&#x2F;&#x2F;www.sciencedirect.com&#x2F;science&#x2F;article&#x2F;abs&#x2F;pii&#x2F;S22141...</a>.

It&#x27;s not much different without the AI. Managers don&#x27;t care about efficient code, they care about code that meets the business goals - whether that&#x27;s good or bad is debatable. Agencies duct-taping together throwaway code isn&#x27;t new. The classic &quot;just buy more AWS resources&quot; &amp; such have been around for quite a while.

&gt;Yeah, who cares about the quality, the mediocrity, the craft..Just about no-one in the F100 unless they are on very special teams.If you care about the craft you&#x27;re pushed out for some that drops out 10x LOC a day because your management has no ability to measure what good software is. Extra bonus points for including 4GB of node_modules in your application.

What this boils down to is an argument for slop. Yeah, who cares about the quality, the mediocrity, the craft... get the slop, push it in, call it done. It mostly works in the golden path, it&#x27;s about 6 or 7 orders of magnitude slower than hand-written software but that&#x27;s ok, just buy more AWS resources, bill the client, whatever.I can maybe even see that point in some niches, like outsourcing or contracting where you really can&#x27;t be bothered to care about what you leave behind after the contract is done but holy shit, this is how we end up with slow and buggy crap that no one can maintain.

My hunch is that to exploit LLMs one should lean on data driven code more. LLMs seem to have a very easy time to generate data literals. Then it&#x27;s far less of an issue to write in a niche language.Not familiar with Elixir but I assume it&#x27;s really good at expressing data driven code, since it&#x27;s functional and has pattern matching.

I think for some languages like Clojure and Elixir, it&#x27;s just so easy to get to the level of abstraction you need to write your business logic that everyone does so. So the code does not have any commonalty with each other. Even when using the same framework&#x2F;library.But for Python, JS, etc,... it&#x27;s the same down to earth abstraction that everyone is dealing with, like the same open a file, parse a csv, connect to the database patterns.

I agree with this. People who are writing Python, Javascript, or Typescript tell me that they get great results. I&#x27;ve had good results using LLMs to flesh out complex SQL queries, but when I write Elixir code, what I get out of the LLM often doesn&#x27;t even compile even when given function and type specs in the prompt. As the writer says, maybe I should be using an agent, but I&#x27;d rather understand the limits of the lower-level tools before adding other layers that I may not have access to.

There&#x27;s a huge caveat i don&#x27;t see often, which is that it depends on your language for programming. IE. AI is reallllly good at writing Next.js&#x2F;Typescript apps, but not so much Ruby on Rails. YMMV

Seconded. I am very much still in the mode of copying from the chat window and then editing. I would like to have whatever she is having.

&gt; I&#x27;m just starting to use Codex Web for asynchronous agents because I have a friend who swears by queueing up a dozen async prompts every morning and sifting through them in the afternoonBunch of async prompts for the same task? Or are you parallelizing solving different issues and just reviewing in the afternoon?Sounds intriguing either way.

I&#x27;m curious how much you paid in the past month for API fees generated by these tools. Or at least what order of magnitude we&#x27;re talking about.

&gt; I use Zed as my primary interface to &quot;actually doing project work&quot; LLM stuff, because it front-ends both OpenAI and Google&#x2F;Gemini models, and because I really like the interface. I still write code in Emacs; Zed is kind of like the Github PR viewer for me.You&#x27;re not concerned about OpenAI or Google stealing your code? I won&#x27;t use VSCode for that reason, personally, but I do use VSCodium.

I use Codex CLI for casual stuff, because of the ergonomics of just popping open another terminal tab.I use Zed as my primary interface to &quot;actually doing project work&quot; LLM stuff, because it front-ends both OpenAI and Google&#x2F;Gemini models, and because I really like the interface. I still write code in Emacs; Zed is kind of like the Github PR viewer for me.I&#x27;m just starting to use Codex Web for asynchronous agents because I have a friend who swears by queueing up a dozen async prompts every morning and sifting through them in the afternoon. The idea of just brainstorming a bunch of shit --- I can imagine keeping focus and motivation going long enough to just rattle ideas off! --- and then making coffee while it all gets tried, is super appealing to me.

tptacek, curious question: what agent &#x2F; stack do you currently use?

&gt; while people working on DoD contracts will be typing things into vim like a foolForget LLMs, try getting Pandas approved. Heck I was told by some AF engineers they were banned from opening Chrome Dev Tools by their security office.FWIW I think the LLM situation is changing quite fast and they&#x27;re appearing in some of our contracts. Azure-provided ones, of course.

&gt; The speed at which LLM stuff is progressing is insaneYou clearly haven&#x27;t been following the space or maybe following too much.Because the progress has been pretty slow over the last years.Yes modals are cheaper and faster but they aren&#x27;t substantially better.

Frankly, as someone who is engaged in fields where LLMs can be used heavily.I would stay in any high danger&#x2F;high precision&#x2F;high regulation role.The speed at which LLM stuff is progressing is insane, what is cutting edge today wasn&#x27;t available 6 months ago.Keep up as a side hobby if you wish, I would definitely recommend that, but I just have to imagine that in 2 years a turnkey github project will get you pretty much all the way there.Idk, that&#x27;s my feeling fwiw.I love LLMs but I&#x27;m much less confident that people and regulation will keep up with this new world in a way that benefits the very people who created the content that LLMs are built on.

I develop space-borne systems, so I can&#x27;t use the best LLM&#x27;s for ITAR&#x2F;etc reasons, but this article really makes me feel like I&#x27;m missing out. This line in particular makes me wonder if my skills are becoming obsolete for general private industry:&gt; People coding with LLMs today use agents. Agents get to poke around your codebase on their own. They author files directly. They run tools. They compile code, run tests, and iterate on the results. They also:Every once in a while I see someone on X posting how they have 10 agents running at once building their code base, and I wonder if in 3 years most private industry coders will just be attending meetings to discuss what their agents have been working on, while people working on DoD contracts will be typing things into vim like a fool

But you definitely don&#x27;t understand it nearly as well as if you wrote it. And you&#x27;re the one that needs to take responsibility for adding it to your codebase.

Are you, though? Reading code is harder, potentially much harder.[1]And I suspect the act of writing it yourself imparts some lower level knowledge you don&#x27;t get by skimming the output of an AI.[1] <a href="https:&#x2F;&#x2F;www.joelonsoftware.com&#x2F;2000&#x2F;05&#x2F;26&#x2F;reading-code-is-like-reading-the-talmud&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.joelonsoftware.com&#x2F;2000&#x2F;05&#x2F;26&#x2F;reading-code-is-li...</a>

Reviewing code is often slower than writing it. You don&#x27;t have to be an exceptionally fast coder or slow reviewer for that to be true.

In this thread, we pretend that the difficult and time-consuming part of a code review is all the reading you have to do.

You are either a very fast producer or a very slow reader. Claude and Gemini are much faster at producing code than I am, and reviewing their code - twice over, even - still takes less time than writing it myself.

&gt; Which takes just as long.This has never once been my experience. Its definitely less fun but it takes way less time.

Indeed, instead of writing code to shave a Yak, we&#x27;re now instead reviewing how the Yak was (most-shittily) shaved.

and probably results in a greater net energy consumption&#x2F;carbon output

It&#x27;s because these people don&#x27;t know how to write it, think they know how to review it. Ship a todo list app in a day, and then write blog posts about how they are changing the world.

I&#x27;m not familiar with the author&#x27;s work. What % of their time is spent writing code?

The thought alone makes me want to hang up my (professional) keyboard and open a nursery&#x2F;gardening center.

All of these people advocating for AI software dev are effectively saying they would prefer to review code instead of write it. To each their own I guess but that just sounds like torture to me.

&gt;So we replace the task of writing tedious boilerplate with the task of reading the AI&#x27;s tedious boilerplate. Which takes just as long. And leaves you with less understanding. And is more boring.These all sound like your projected assumptions. 
No, it generally does not take longer to review sizable code changes than it does to write it. This is further alleviated if the code passes tests, either existing or new ones created by the ai.

So we replace the task of writing tedious boilerplate with the task of reading the AI&#x27;s tedious boilerplate. Which takes just as long. And leaves you with less understanding. And is more boring.

&gt; pull in arbitrary code from the tree, or from other trees online, into their context windows,I guess this presupposes that it is ok for 3rd parties to slurp up your codebase? And possibly (I guess it ostensibly depends on what plan you are on?) using that source code for further training (and generating that same code for others)?I imagine in some domains this would not be ok, but in others is not an issue.

Programmers who think front end is &quot;easier&quot; than backend have been wrong for well over a decade. <a href="https:&#x2F;&#x2F;simonwillison.net&#x2F;2012&#x2F;Feb&#x2F;13&#x2F;why-are-front-end&#x2F;" rel="nofollow">https:&#x2F;&#x2F;simonwillison.net&#x2F;2012&#x2F;Feb&#x2F;13&#x2F;why-are-front-end&#x2F;</a>

i feel like surprisingly, front end work which used to be viewed by programmers as &quot;easier&quot; is now more difficult of the two, because it&#x27;s where LLMs suck the mostyou get a link to a figma design and you have to use your eyes and common sense to cobble together tailwind classes, ensure responsiveness, accessibility, try out your components to make sure they&#x27;re not janky, test out on a physical mobile device, align margins, padding, truncation, wrapping, async loading states, blah blah you get itLLMs still suck at all that stuff that requires a lot of visual feedback, after all, you&#x27;re making an interface for humans to use, and you&#x27;re a humanin contrast, when i&#x27;m working on a backend ticket ai feels so much more straightforward and useful

And potentially make incredibly risky mistakes while the AI assures you it’s fine.

&gt; Often, it will drop you precisely at that golden moment where shit almost works, and development means tweaking code and immediately seeing things work better. That dopamine hit is why I code.Only if you are familiar with the project&#x2F;code. If not, you were throw into a foreign codebase and have no idea how to tweak it.

Me too -- actually, I&#x27;d say that the LLMs I use these days (Sonnet 4 and GPT4.1, o4, etc) are pretty good at rust.

No only this, but I would challenge the OP to see if he really knows Rust but turning off LLM and see “how much you truly know”.This is akin to be on tutorial hell and you “know the language “

I&#x27;ve been writing Rust code in production for 4+ years, and I can write Rust pretty well, and I&#x27;ve learned a lot from using chatgpt and co-pilot&#x2F;cursor.In particular, it helped me write my first generic functions and macros, two things that were pretty intimidating to try and get into.

How does anyone self learning know they&#x27;re learning the &quot;right things and the best way to do things&quot;? By putting the stuff they&#x27;ve learned into practice and putting it up against the real world. How many Rust tutorials are out there that teach things incorrectly, non-idomatically or just inefficiently? How does anyone not already an expert know except by trying it out?

I have to say, my ability to learn Rust was massively accelerated via LLMs.How would you know?If you didn&#x27;t know Rust already, how would you know the LLM was teaching you the right things and the best way to do things?Just because it compiles doesn&#x27;t mean it works. The world is full of bad, buggy, insecure, poor code that compiles.

Quite a lot, but hey, feel free to put me to the test

How much of that proficiency remains once you switch it off?

It is not bad at rust.  I don&#x27;t think I could even function well as a Rust programmer without chatgpt and now Cursor.  It removes a lot of the burden of remembering how to write generic code and fixing borrow checking stuff.  I can just write a generic function with tons of syntax errors and then tell cursor to fix it.

&gt;but it’s bad at rustI have to say, my ability to learn Rust was massively accelerated via LLMs. I highly recommend them for learning a new skill. I feel I&#x27;m roughly at the point (largely sans LLMs) now where I can be nearly as productive in Rust as Python. +1 to RustRover as well, which I strongly prefer to any other IDE.

My AI skeptic friends are all nuts