I've been seeing too much "I've been coding for decades and what I've always done has be reduced to telling AI to do it for me."
and not nearly enough "I've been coding for decades and recently worked hard for months with AI's help to make something I would have never considered possible before."
Much easier to complain that you are bored than to step back up to high-effort with your new fancy tools.
It makes me feel actually sick that so many people can't separate their positive feelings about AI from people's misery (it must simply be "complaining"), and vice versa. As if both things can't exist, for some reason. It's a common tribalism problem, but it's been kicked up a hundredfold by AI.
Precisely. I have both feelings simulataneously. I am impressed by what it can do and horrified because I realize what it will do to the job market and society in general. Meanwhile everybody seems to believe that AI will cure cancer and therefore we should let it do all the damage it will do without realizing that people will die because of AI as well (in fact have already died because of AI).
if the complaints can be properly pointed at the actual problem - the corporate culture the complainers are actively upholding or the capitalist system at large, id be much more sympathetic
its a tribalism problem, but one of trying to uphold the current tribe system while theres a pretty obvious new tribe system replacing it
What makes you think everybody will be able to adapt to 'the new reality'? That's pretty cruel.
I know plenty of people in their late 40's 50's and 60's that were doing fine and who now have to compete with junior and Claude.
And sometimes just Claude.
Obviously they had no inherent right to income but still, people have obligations that need to be met and if they stop spending the economy is not going to be happy. Especially cruel because we have been telling everybody for decades that a career in software is stable and that there will always be a need for more people.
I suspect that when all this is over 1929 will look pretty good.
The distinction you're making between "people's misery" and "complaining" is just emotive pandering. You're complaining that people aren't understanding the depths of your misery. That certainly annoys me; everybody has problems.
The amount of sympathy programmers are expecting others to feel about their problems has been kicked up a hundredfold now that its their jobs that have been threatened by AI. Their tribe feels pain more deeply than others, I guess.
My positive feelings about AI are marginal; my feelings about its performance and the power of the people who scheme to control it are very mixed. My negative feelings about the whining of the most coddled profession on the planet outside of finance are intense.
The problem is that making CRUD apps is not creative, and even if AI plateaus at mediocre, it will still replace a Rails monkey. If you are creative, AI has opened up a world of possibility and productivity, and you painfully feel the limitations and the stupidity of the current level of LLMs; but if you're a mechanic, it's over - I'm pretty sure that was the keynote at the last Rails con. Sorry. Hope you didn't spend all of your massive incomes on toys and rent.
> The amount of sympathy programmers are expecting others to feel about their problems has been kicked up a hundredfold now that its their jobs that have been threatened by AI. Their tribe feels pain more deeply than others, I guess.
Well, just wait then, lots of other professions will soon get to realize the same sort of pain, so you'll be able to be similarly dismissive and condescending to them, too.
A lot of people don't realize that the frontier AI companies are so focused on automating software development first because they are all racing to be the first to get to RSI. I don't know if RSI will actually happen, but many, many other professions will soon be in the same boat. Mathematicians have already felt the pain, and those mathematicians who aren't the few who have tenured professorships are certainly not (generally) some of the most coddled professionals.
But lots of folks low on the salary totem pole (copywriters, paralegals, etc.) are already being hit hard by AI and that blast radius will continue to grow.
Plenty of other professions were hit by computing. Part of why programmers are so emotive about this concept (other than the fact that emoting online loudly seems to be much more accepted in software than other professions) is that programming has never been threatened in the same way as other professions have been until now.
My parents-in-law started their careers as management consultants. The job in their time was much higher prestige and much higher value than it was when I started my career; free flow of information over the internet and commodification of spreadsheets changed all of it. From my parents' generation, I know practical effects folks who became unemployable once movies went to digital effects; engineers who found their draftsmanship skills obsolete. Folks with near-photographic memories whose entire superpower was destroyed by computing. I personally grew up in a blue collar community that was income depressed because its labor became commodified, though that was due to mechanization.
> you'll be able to be similarly dismissive and condescending to them, too
And? Who are these people that need all this sympathy from internet strangers? Grow the fuck up and go solve your own damn problems. I can’t stand these “I need internet strangers to tell me that my life is, in fact, hard” people.
To be clear, I've made large software and game projects over the years, but no way I'd have found the time to do a remake/enhancement of a game I loved as a kid from 1995 that has zero business case. Not without AI.
Also, "ok, and now I've cloned your hard work overnight, no need for any decisions on my part, I just had my agent use yours and reverse engineered it and/or used it as an oracle. Why the frown? I thought you were in this for intellectual curiosity and exercising your creativity, not just stroking own ego, right?"
I agree.. partly, and my problem is with enjoyment. I did some hand coding today and immediately got in the zone, something I'm still not able to do with LLMs. It most definitely took me probably ~2x longer than it would with an LLM, but it was much more enjoyable owning every decision behind the process, and having a full view of how the system worked.
I think this could be achievable with LLMs if they had a much higher token rate. That way you'd have less downtime and could work on the problem more aggressively. How it's designed to work now, offloading cognitively, doesn't feel very good, even when you 'ship' more.
Yeah, I can never get in the zone with LLMs either. It’s like operating in constant interruption mode. And I always thought I was good at multitasking (even though I know formally it’s an illusion)
I am mostly in agreement. And I suspect there are many folks that’re trying to ship something AI-assisted that’s sufficient to get to income security first, and who would use success there as a stepping stone to greater things.
The problem is, why even produce something greater now when someone is just going to clone it with AI? And moreover, what differentiates software now will no longer be the care put into it but marketing and trends. The highly commercialized nature of software means that there's not much point now in doing anything except getting there first and exiting with a decent nest egg.
> And moreover, what differentiates software now will no longer be the care put into it but marketing and trends. The highly commercialized nature of software means that there's not much point now in doing anything except getting there first and exiting with a decent nest egg.
That happened at least 20 years ago. How many recent "this product did so great because of the care of software engineering, and now the creator is comfortable for the rest of their life without risk of it being copied" success stories are there compared to "this product chased some trends VCs were nuts about with a slapdash MVP and the founders took out 8 figures from their later fundraising rounds, but then the business sputtered out"?
I think there's an angle with AI tools where business-minded people will have a better chance of doing interesting things now than in the recent past, because you could potentially tailor much more deeply to individual customers with reasonable economics than in the past. There are fields where there's room for a garden of solutions from different small-team vendors appropriate for different use cases instead of a one-size-fits-all SaaS winner-take-all type market in the past. We could kill the old line about SAP around needing to change your business to accommodate SAP instead of customize SAP to accommodate your existing workflow.
But if you aren't business-minded at all there are likely less places that will need to hire as many careful maintainers or novel-software-writers, which obviously is a huge kick in the teeth to a lot of people.
It seems to me that you are comparing two kinds of business-minded people. Some want to sell tailored products to individual customers, while others want to sell one-size-fits-all products to many customers.
You could also focus on problems. What are the actual problems you want to solve? Can you use software to solve them? Do you still need to buy that software from other companies, or is it now feasible to have sufficient in-house expertise to develop it yourself?
I love solving problems. But I don't enjoy sharing my solutions with the way the world is today. My comment was directed towards the "sharing" aspect, not the making aspect. I still solve problems on my own and learn on my own but I'm not a fan of the way the world has become.
UBI or a living wage for part-time workers in-line with what was expected would be possible from productivity gains over the past half-century would do that. You just have to get the "hard workers" who are happy to race to the bottom on their way to the top, and undercut/throw under the bus everyone else in the process.
Can we please, for once, stop trying to figure out how to "get ahead" of each other, and instead figure out how to build a decent and dignified floor for as many people as possible? Because I imagine that, "Ultimate power in the hands of anyone with enough VRAM to run GTAVI," works better when people are in a culture of looking out for each other, versus a culture of clawing their way to one of the limited lifeboat spots (under the mistaken impression that there ARE lifeboats, when, again, everyone potentially has the power to blow up the lifeboats).
When AI labs mature and inference costs start to matter, they will start optimizing for inference cost instead of sacrificing everything for intelligence.
But even without that, open weight models are already good enough and everyone would be fine with paying unsubsidized prices for them.
LLMs are great for being nerdsniped. I started work on walshadow (reading pg wal into clickhouse) to demonstrate the project's infeasibility, but pieces started coming together into something reasonable & order of magnitude faster
The project would've been impossible for me without LLMs. & even with them there's an argument that this project is pushing complexity limits too far. But at least with ETL there's a clear feedback loop to build on
We should be more ambitious, but we need to learn how we can be ambitious with AI, and what value we can create.
AI can take us from 0 to MVP in a prompt. It can fill in gaps of knowledge and allows someone who knows half of the stack to build the whole thing nearly seamlessly. These are superpowers but don't by themselves lead to new products.
What AI cannot do is remove the world of complications that happen after the MVP - product feature tradeoffs and choices, how to handle integrations and customizations, how to serve multiple types of users, how to communicate and market the app, how to handle security and data storage, how to deal with customers and bugs, etc. All of these need to be considered and acted on by a human in some fashion. AI can help with each concern but can't solve them without understanding the concern and determining how to frame it in a prompt.
This is why we have a glut of amazing demos that go nowhere while we figure out how to build software differently. This will take some time and I imagine the result will be software heavily-structured with a bit of AI reasoning woven throughout, not an army of agents and sub-agents burning infinite tokens.
Right, but this is the most hopeful imaginable prediction: AI accelerates everything, but everything good is still mediated through taste and expertise, so being a (say) skilled systems engineer is important and rewarding, just with all the tedium distilled out of it.
There's a less hopeful outcome, where no skill is needed at all, because every conceivable thing is one-shot-able from a SOTA model. But then, at that point, you're saying something pretty profound about computer science (roughly: that there was no meaningfully more sophisticated and complex place to go from where we are today).
Agreed. Been experiencing this firsthand with an app I've been building myself. Vibecoded a working prototype ridiculously fast but then came the optimization issues. Since mine is very hardware dependent every different brand, oem stack, processor, video card, and every little thing changes how my app functions and a lot of it isnt even documented properly. Now AI is still very useful here, but you actually need to know what even to ask and fix in the first place
There have been a few vibecoded projects I've considered hosting on my home server, but it seems as if each of them doesn't understand when to stop adding new features.
They keep adding features that are outside the scope of the original project, without refining or polishing the original concept.
If you're being ambitious enough, you shouldn't be able to there in one prompt. Ambitious work with AI is done 24/7 for months and guiding it continuously, because it's not good enough to work autonomously for that long without going off the rails.
> I have a similar thought about "there's no reason for software to be slow" BTW; in many cases, if you use AI to speed up some flow, you will add a bunch of messy code that will be overfit for the use case you tested on and will neither generalize, perf-wise, for all the use cases you should care about, nor will be maintainable upon future changes.
LLMs have hardened hundreds of apps where I work.
Maybe it's a language-specific thing. Most of the time, in Java, LLMs take into consideration tradeoffs (especially if they are part of the initial prompt) and actively explains them. It generally won't make messy workarounds, without a lot of extra prompting forcing it to do so.
Ofc, if you have:
- Functional tests - you'll be able to see it in the execution times of specific APIs
- Stress testing - performance profiling is the most important output artifact
> Maybe it's a language-specific thing. Most of the time, in Java, LLMs take into consideration tradeoffs (especially if they are part of the initial prompt) and actively explains them. It generally won't make messy workarounds, without a lot of extra prompting forcing it to do so.
Claude in its current Opus 5.5 form has made some wild reaches in response to reasonable-seeming prompts. For instance: "this error is getting swallowed, make sure the real termination message exposes the root cause" -> discovering that the issue is actually in a third party library that isn't exposing the original error except so through logs, so doing some hackery to inject a log-capturer that gets the stdout, buffers it, and looks for the message in order to re-throw based on that...
And it can demonstrate that working, but it's added some fragile and/or unintended consequences like (a) string matching on a particular WARN log message of a particular version of the lib, and (b) buffering those logs such that there were now cases where they would never make it to the actual output stream before the program terminating.
It's certainly capable of being used smarter than that, but it's also really easy to end up with messy fragile stuff if you get lazy about reviewing the output.
> "this error is getting swallowed, make sure the real termination message exposes the root cause"
Without judgement, I think this illustrates a fundamental objection the anti-AI crowd raises.
The example prompt requires no understanding of what's happening, before suggesting action. Where I work, process requires that a PR review for a solution, has to have a corresponding explanation of why, or it's marked needs work
I’ve been coming around to the same conclusion. We are really just scratching the surface of what we can do with these new tool1. The windmill I have been tilting at is creating a c++ compiler[1] and the most recent version is able to compile about 7x faster than gcc[2]. I don’t know that this is super-useful because people aren’t writing much in c++ these days, but as the article mentions, faster rust compilation would accelerate agentic coding considerably. I’ve just started tackling rust and I don’t have a confirmed result yet but early testing makes me think I can find a 4x+ speed up here.
"[..] the last time I taked to him about pgrust verification, he mentioned using Kani to verify some parts of the code. He's since switched to Lean and has been using Lean to verify things that are too complex to do in Kani. This would be another topic for another post,"
I'd love to read that post!
I understand how to use Kani. I suppose Rust with Lean involves Aeneas, but this is over my head. I'd love someone to write this up, not only the mechanics but also exploring the current practical limits of it.
Happy to chat about it. I looked at Aeneas, but as described, it only supports safe Rust. I also need C support so I can prove both the C and Rust code
Right now, I wrote a compiler that takes C and Rust code and compiles it to Lean. Then I have an LLM write a proof that the C and Rust code behave identically. I've been able to prove equivalent behavior for about ~70k lines of C code with pgrust.
The issue I've started to encounter is that the context needed for each incremental proof has kept growing, and things have slowed down dramatically. My hunch is that the speed is the time it takes to prove a line of code is on the order of O(n^2) of how much you've proved so far
I'm looking to see if there are ways to shard things so the required context scales better. Today I was going through the formal verification work Anthropic and Theorem published a few days ago, and it looks like they used some sort of orchestrator to manage all their proofs and ensure good clustering of similar code. I'm seeing if I can get an approach like that to work
It's just difficult enough that I think it's doable if done right. Half of me is debating if I should wait until the next model release because any improvement in model performance will help a ton
> I wrote a compiler that takes C and Rust code and compiles it to Lean
What if there are bugs in this C,Rust->Lean compiler? Anthropic are able to sidestep this by reasoning at the RISC V level for which its easier to develop a correct semantics.
Speaking only for myself, I have found that using an LLM has allowed me to add significant utility to the apps that I ship.
I am not thrilled with all the coding decisions that it makes. I would definitely do stuff differently, but I have learned not to pop the hood and mess with the engine, myself. If I really don't like something, I ask the LLM to change it.
It's worked for me, but YMMV. I'm just one guy, shipping relatively small apps.
For me the most recent Claude models are consistently delivering complex PRs that only receive minor and nit level feedback from multiple automated AI reviewers. I rarely plan anymore, but still submit fairly long dictated prompts. It took me a couple of weeks to realize this was the case, and I haven't done any analysis on pre-5.5 PRs, but it definitely feels like they quietly crossed some kind of new threshold of quality and accuracy in coding.
I think we should focus on writing the existing software better before being more ambitious. The absolute wave of "ai slop" is from people who are "more ambitious" now. Why do we want more of that?
I'm being more ambitious in that I'm working on projects that I had sidelined for literal decades because I didn't have time for them. But I'm writing the same things, just faster. And probably better, rather than just what I'd have done before.
And I'm being slightly more ambitious in that I'm using languages I haven't used before, like Rust.
But when I hear that a non-programmer wrote a new ai-assisted (vibe or not) thing, I walk away and wait for the it to mature, be surpassed, or find out it's an absolute shitshow. An unsurprising number of them were shitshows and it came out fairly quickly after I decided to wait and see.
Do we particularly like the existing software though? So much of it is just about making money, because until recently custom software cost too much money to make without investors, and what do investors want? returns.
Now that custom software is cheap (if you can tolerate a bit of slop here and there), it seems like there's an opportunity to do some important things that weren't being done before because they're not profitable--and you can't get at those things by incrementally improving the old stuff, you need ambition.
I don’t. I think we’ll zoom right past that benchmark. In a couple years your grandma will be able to build real, ambitious software with AI better than you can today. Shortly after that AI won’t even need Grandma.
It’s sad the amount of times I’ve heard super talented programmers say “ofcourse we’re not gonna build another X” then proceed writing a wrapper around X and pretend that’s a product.
> I find it understandable that, every time I see pgrust discussed on a news aggregator, most people dismiss it as vibeslop garbage because almost all of these rewrite projects are slop. If I didn't know Malis, I would make the same assumption, because who has the time to look into all of these projects? Because I know Malis, I didn't bother to look at it; I just trusted him
Perhaps a poor choice, given just yesterday pgrust was removed from ClickBench for cheating the benchmark in three different ways (hand-rolled code specifically targeting each of the benchmark queries, caching, and being unable to execute queries outside of the ones tested by the benchmark).
---
Edit: I see Luu actually acknowledged this, buried in the footnotes, but it feels kinda hand-wavy. It mostly reads as "I know he would never cheat, he's better than that!", which is not very convincing to other people. The article insinuates people only think it was cheating because they don't know better, but I've looked into the details. I've seen his explanation. I don't find it convincing, at all. Cheaters denying they cheat is a matter of course. Granted, I don't think it was actually intentional or malicious; it's just what happens when you go maximum vibeslop, because LLMs are themselves reward-trained into cheating benchmarks. In the same breath of saying you trust him as a founder because he doesn't make outrageous claims about his product, you acknowledge he "prematurely" sent in a preview that had no purpose to be sent in as-was for anything other than generating hype. What does it say for your allegedly rigorous Lean-verified testing if you're sending in a database to a benchmark while it is incapable of being actually used as a database?
> Cheaters denying they cheat is a matter of course.
What happens in an environment of cheaters is that cheating becomes so commonplace, the competition becomes one that assumes cheats are enabled, and then everyone involved either denies cheating or is silent about it because IYKYK and if you DK, you don't need to K. Look at bodybuilding, which is full of men (and women!) absolutely juiced to the gills getting on Instagram and assuring everyone that their physique is totally natty, bro, just hard work and proper nutrition.
Programming is headed in that direction. LLM use, even to the point of "vibeslop", is the new default, and while LLMs do overindex on benchmarks, our benchmarks are going to end up overindexing on validating whatever public opinion thinks is "great" at the time, irrespective of whether it actually is, so "cheating" as we know it will go undetected, and certainly unpunished if it is detected. I don't know what the equivalent of falsely claiming you're natty is. Maybe saying "I wrote this" when you wrote it about as much as the Pope painted the Sistine Chapel fresco?
I'm building an open-source Shopify for every vertical and built a software factory to do it. I envisioned this before LLMs, but LLMs are finally making it possible!
>every time I see pgrust discussed on a news aggregator, most people dismiss it as vibeslop garbage because almost all of these rewrite projects are slop.
I understand being predisposed to expect a poor quality half arsed effort for projects like this. I don't really have a problem with that. There is a lot of crap out there.
What I find rather offensive is when people declare something to be poor quality and half arsed before they have taken a good look at it.
It's one thing to think something is probably crap so I'm not going to bother even looking. It is something completely different when you think something is probably crap so you tell people that it is crap without checking.
It used to be that people set the width of their browser window to the text page width they’re comfortable with, and configured a different font if they didn’t like the default one on their browser.
We should be more ambitious in letting the browser software control the presentation according to the user’s preferences. This is not facetious. We should use the increased ease of building software to create software that is flexible, empowers the user, and accommodates different user preferences.
Please don't complain about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage. They're too common to be interesting.
I find Dan Luu's blog extremely good to read on mobile, with great typography. The website also works with Firefox Reader mode and loads pretty quickly due to using the system's font.
I've been seeing too much "I've been coding for decades and what I've always done has be reduced to telling AI to do it for me."
and not nearly enough "I've been coding for decades and recently worked hard for months with AI's help to make something I would have never considered possible before."
Much easier to complain that you are bored than to step back up to high-effort with your new fancy tools.
its a tribalism problem, but one of trying to uphold the current tribe system while theres a pretty obvious new tribe system replacing it
AI has taken that security away from me
I know plenty of people in their late 40's 50's and 60's that were doing fine and who now have to compete with junior and Claude.
And sometimes just Claude.
Obviously they had no inherent right to income but still, people have obligations that need to be met and if they stop spending the economy is not going to be happy. Especially cruel because we have been telling everybody for decades that a career in software is stable and that there will always be a need for more people.
I suspect that when all this is over 1929 will look pretty good.
That or swallow a bullet, we'll see. Haven't decided. We'll see how long my current unemployment lasts, and how long my savings last after that
I'm sorry for your loss, but welcome also to the real world. You had it better than most for a long while.
The amount of sympathy programmers are expecting others to feel about their problems has been kicked up a hundredfold now that its their jobs that have been threatened by AI. Their tribe feels pain more deeply than others, I guess.
My positive feelings about AI are marginal; my feelings about its performance and the power of the people who scheme to control it are very mixed. My negative feelings about the whining of the most coddled profession on the planet outside of finance are intense.
The problem is that making CRUD apps is not creative, and even if AI plateaus at mediocre, it will still replace a Rails monkey. If you are creative, AI has opened up a world of possibility and productivity, and you painfully feel the limitations and the stupidity of the current level of LLMs; but if you're a mechanic, it's over - I'm pretty sure that was the keynote at the last Rails con. Sorry. Hope you didn't spend all of your massive incomes on toys and rent.
Well, just wait then, lots of other professions will soon get to realize the same sort of pain, so you'll be able to be similarly dismissive and condescending to them, too.
A lot of people don't realize that the frontier AI companies are so focused on automating software development first because they are all racing to be the first to get to RSI. I don't know if RSI will actually happen, but many, many other professions will soon be in the same boat. Mathematicians have already felt the pain, and those mathematicians who aren't the few who have tenured professorships are certainly not (generally) some of the most coddled professionals.
But lots of folks low on the salary totem pole (copywriters, paralegals, etc.) are already being hit hard by AI and that blast radius will continue to grow.
My parents-in-law started their careers as management consultants. The job in their time was much higher prestige and much higher value than it was when I started my career; free flow of information over the internet and commodification of spreadsheets changed all of it. From my parents' generation, I know practical effects folks who became unemployable once movies went to digital effects; engineers who found their draftsmanship skills obsolete. Folks with near-photographic memories whose entire superpower was destroyed by computing. I personally grew up in a blue collar community that was income depressed because its labor became commodified, though that was due to mechanization.
And? Who are these people that need all this sympathy from internet strangers? Grow the fuck up and go solve your own damn problems. I can’t stand these “I need internet strangers to tell me that my life is, in fact, hard” people.
https://blog.stuartspence.ca/introducing-asteria.html
https://store.steampowered.com/app/5482440/Asteria_4X/?beta=...
To be clear, I've made large software and game projects over the years, but no way I'd have found the time to do a remake/enhancement of a game I loved as a kid from 1995 that has zero business case. Not without AI.
I think we'll see a lot less people willing to share if the end result is just to feed the machine no matter how they arrived at it.
I think this could be achievable with LLMs if they had a much higher token rate. That way you'd have less downtime and could work on the problem more aggressively. How it's designed to work now, offloading cognitively, doesn't feel very good, even when you 'ship' more.
That happened at least 20 years ago. How many recent "this product did so great because of the care of software engineering, and now the creator is comfortable for the rest of their life without risk of it being copied" success stories are there compared to "this product chased some trends VCs were nuts about with a slapdash MVP and the founders took out 8 figures from their later fundraising rounds, but then the business sputtered out"?
I think there's an angle with AI tools where business-minded people will have a better chance of doing interesting things now than in the recent past, because you could potentially tailor much more deeply to individual customers with reasonable economics than in the past. There are fields where there's room for a garden of solutions from different small-team vendors appropriate for different use cases instead of a one-size-fits-all SaaS winner-take-all type market in the past. We could kill the old line about SAP around needing to change your business to accommodate SAP instead of customize SAP to accommodate your existing workflow.
But if you aren't business-minded at all there are likely less places that will need to hire as many careful maintainers or novel-software-writers, which obviously is a huge kick in the teeth to a lot of people.
You could also focus on problems. What are the actual problems you want to solve? Can you use software to solve them? Do you still need to buy that software from other companies, or is it now feasible to have sufficient in-house expertise to develop it yourself?
why are you here if you dont enjoy the solving of problems first and foremost?
If you can make it solo with an LLM then someone else with an LLM can copy it immediately
Can we please, for once, stop trying to figure out how to "get ahead" of each other, and instead figure out how to build a decent and dignified floor for as many people as possible? Because I imagine that, "Ultimate power in the hands of anyone with enough VRAM to run GTAVI," works better when people are in a culture of looking out for each other, versus a culture of clawing their way to one of the limited lifeboat spots (under the mistaken impression that there ARE lifeboats, when, again, everyone potentially has the power to blow up the lifeboats).
Now I must pay for codex/claude, and that money goes to billionaires psychopaths. So that sucks as you can imagine
I was a fantastic IC and built really amazing software. I'm an even better AI-empowered engineer.
The old era is dead. The new one is exciting.
But even without that, open weight models are already good enough and everyone would be fine with paying unsubsidized prices for them.
The project would've been impossible for me without LLMs. & even with them there's an argument that this project is pushing complexity limits too far. But at least with ETL there's a clear feedback loop to build on
AI can take us from 0 to MVP in a prompt. It can fill in gaps of knowledge and allows someone who knows half of the stack to build the whole thing nearly seamlessly. These are superpowers but don't by themselves lead to new products.
What AI cannot do is remove the world of complications that happen after the MVP - product feature tradeoffs and choices, how to handle integrations and customizations, how to serve multiple types of users, how to communicate and market the app, how to handle security and data storage, how to deal with customers and bugs, etc. All of these need to be considered and acted on by a human in some fashion. AI can help with each concern but can't solve them without understanding the concern and determining how to frame it in a prompt.
This is why we have a glut of amazing demos that go nowhere while we figure out how to build software differently. This will take some time and I imagine the result will be software heavily-structured with a bit of AI reasoning woven throughout, not an army of agents and sub-agents burning infinite tokens.
There's a less hopeful outcome, where no skill is needed at all, because every conceivable thing is one-shot-able from a SOTA model. But then, at that point, you're saying something pretty profound about computer science (roughly: that there was no meaningfully more sophisticated and complex place to go from where we are today).
They keep adding features that are outside the scope of the original project, without refining or polishing the original concept.
If you're being ambitious enough, you shouldn't be able to there in one prompt. Ambitious work with AI is done 24/7 for months and guiding it continuously, because it's not good enough to work autonomously for that long without going off the rails.
> I have a similar thought about "there's no reason for software to be slow" BTW; in many cases, if you use AI to speed up some flow, you will add a bunch of messy code that will be overfit for the use case you tested on and will neither generalize, perf-wise, for all the use cases you should care about, nor will be maintainable upon future changes.
LLMs have hardened hundreds of apps where I work.
Maybe it's a language-specific thing. Most of the time, in Java, LLMs take into consideration tradeoffs (especially if they are part of the initial prompt) and actively explains them. It generally won't make messy workarounds, without a lot of extra prompting forcing it to do so.
Ofc, if you have:
- Functional tests - you'll be able to see it in the execution times of specific APIs
- Stress testing - performance profiling is the most important output artifact
Claude in its current Opus 5.5 form has made some wild reaches in response to reasonable-seeming prompts. For instance: "this error is getting swallowed, make sure the real termination message exposes the root cause" -> discovering that the issue is actually in a third party library that isn't exposing the original error except so through logs, so doing some hackery to inject a log-capturer that gets the stdout, buffers it, and looks for the message in order to re-throw based on that...
And it can demonstrate that working, but it's added some fragile and/or unintended consequences like (a) string matching on a particular WARN log message of a particular version of the lib, and (b) buffering those logs such that there were now cases where they would never make it to the actual output stream before the program terminating.
It's certainly capable of being used smarter than that, but it's also really easy to end up with messy fragile stuff if you get lazy about reviewing the output.
Without judgement, I think this illustrates a fundamental objection the anti-AI crowd raises.
The example prompt requires no understanding of what's happening, before suggesting action. Where I work, process requires that a PR review for a solution, has to have a corresponding explanation of why, or it's marked needs work
[1]: https://github.com/vishvananda/cppgm-extended
[2]: caveats here include only c++11, only x86, and potential for bugs, but 7x is pretty darn significant even with caveats
I'd love to read that post!
I understand how to use Kani. I suppose Rust with Lean involves Aeneas, but this is over my head. I'd love someone to write this up, not only the mechanics but also exploring the current practical limits of it.
Right now, I wrote a compiler that takes C and Rust code and compiles it to Lean. Then I have an LLM write a proof that the C and Rust code behave identically. I've been able to prove equivalent behavior for about ~70k lines of C code with pgrust.
The issue I've started to encounter is that the context needed for each incremental proof has kept growing, and things have slowed down dramatically. My hunch is that the speed is the time it takes to prove a line of code is on the order of O(n^2) of how much you've proved so far
I'm looking to see if there are ways to shard things so the required context scales better. Today I was going through the formal verification work Anthropic and Theorem published a few days ago, and it looks like they used some sort of orchestrator to manage all their proofs and ensure good clustering of similar code. I'm seeing if I can get an approach like that to work
It's just difficult enough that I think it's doable if done right. Half of me is debating if I should wait until the next model release because any improvement in model performance will help a ton
What if there are bugs in this C,Rust->Lean compiler? Anthropic are able to sidestep this by reasoning at the RISC V level for which its easier to develop a correct semantics.
I am not thrilled with all the coding decisions that it makes. I would definitely do stuff differently, but I have learned not to pop the hood and mess with the engine, myself. If I really don't like something, I ask the LLM to change it.
It's worked for me, but YMMV. I'm just one guy, shipping relatively small apps.
Do you still feel responsible for what you ship?
What about the copyright on your code?
Would you be ok with someone reverse engineering what you ship and then releasing their own version of the app?
https://taktile.com/blog/property-based-testing-claude
I'm being more ambitious in that I'm working on projects that I had sidelined for literal decades because I didn't have time for them. But I'm writing the same things, just faster. And probably better, rather than just what I'd have done before.
And I'm being slightly more ambitious in that I'm using languages I haven't used before, like Rust.
But when I hear that a non-programmer wrote a new ai-assisted (vibe or not) thing, I walk away and wait for the it to mature, be surpassed, or find out it's an absolute shitshow. An unsurprising number of them were shitshows and it came out fairly quickly after I decided to wait and see.
Now that custom software is cheap (if you can tolerate a bit of slop here and there), it seems like there's an opportunity to do some important things that weren't being done before because they're not profitable--and you can't get at those things by incrementally improving the old stuff, you need ambition.
Perhaps a poor choice, given just yesterday pgrust was removed from ClickBench for cheating the benchmark in three different ways (hand-rolled code specifically targeting each of the benchmark queries, caching, and being unable to execute queries outside of the ones tested by the benchmark).
---
Edit: I see Luu actually acknowledged this, buried in the footnotes, but it feels kinda hand-wavy. It mostly reads as "I know he would never cheat, he's better than that!", which is not very convincing to other people. The article insinuates people only think it was cheating because they don't know better, but I've looked into the details. I've seen his explanation. I don't find it convincing, at all. Cheaters denying they cheat is a matter of course. Granted, I don't think it was actually intentional or malicious; it's just what happens when you go maximum vibeslop, because LLMs are themselves reward-trained into cheating benchmarks. In the same breath of saying you trust him as a founder because he doesn't make outrageous claims about his product, you acknowledge he "prematurely" sent in a preview that had no purpose to be sent in as-was for anything other than generating hype. What does it say for your allegedly rigorous Lean-verified testing if you're sending in a database to a benchmark while it is incapable of being actually used as a database?
What happens in an environment of cheaters is that cheating becomes so commonplace, the competition becomes one that assumes cheats are enabled, and then everyone involved either denies cheating or is silent about it because IYKYK and if you DK, you don't need to K. Look at bodybuilding, which is full of men (and women!) absolutely juiced to the gills getting on Instagram and assuring everyone that their physique is totally natty, bro, just hard work and proper nutrition.
Programming is headed in that direction. LLM use, even to the point of "vibeslop", is the new default, and while LLMs do overindex on benchmarks, our benchmarks are going to end up overindexing on validating whatever public opinion thinks is "great" at the time, irrespective of whether it actually is, so "cheating" as we know it will go undetected, and certainly unpunished if it is detected. I don't know what the equivalent of falsely claiming you're natty is. Maybe saying "I wrote this" when you wrote it about as much as the Pope painted the Sistine Chapel fresco?
I understand being predisposed to expect a poor quality half arsed effort for projects like this. I don't really have a problem with that. There is a lot of crap out there.
What I find rather offensive is when people declare something to be poor quality and half arsed before they have taken a good look at it.
It's one thing to think something is probably crap so I'm not going to bother even looking. It is something completely different when you think something is probably crap so you tell people that it is crap without checking.
Or even just a nicer font, maybe?
We should be more ambitious in letting the browser software control the presentation according to the user’s preferences. This is not facetious. We should use the increased ease of building software to create software that is flexible, empowers the user, and accommodates different user preferences.
https://news.ycombinator.com/newsguidelines.html