This. Once you're building something that genuinely hasn't been built before, LLMs cannot be trusted with any architectural decisions. I'm building a product based around various physics simulations, so it's purely first principles, but without active research, thinking, and challenging, it produces computational code literally hundreds of orders of magnitude slower WHILE implementing absurd fallbacks and shortcuts that effectively result in a useless calculation.
This is the case perhaps 95% of the time.
Oversight is very important, and architectural thinking cannot yet be outsourced, only execution.
Hopefully everyone? Else your job could have been outsourced or replaced by a junior with access to Google and StackOverflow way before LLMs (it just wasn’t due to zero interest rates and proliferation of bullshit jobs in tech companies).
Sure. But where do you think AI will be in a year? Or do you think that AI is just an advanced Markov chain? Like “AI will never be able to write code. Ok AI will never be able to debug code. ok ai will never be able to write design docs. Ok AI will never be able to architecture. Ok ai will never be able to do distributed systems architecture. Ok ai will never be able to design new products completely from scratch. Ok AI will never be able to run a company. Ok AI will never be able to run a city. Ok AI will never be able to run a government. Ok ai will never be able to run the world economy…” It’s Robin Williams Gaddadi sketch “Ok you cross this line you die!” [1].
And to close the loop - there is no architectural thinking without experience in execution. The highly productive people who are all-in on agentic coding today are powered by their previous experience doing implementation. As time goes on their powers will wane unless they make a point to keep them sharp by doing enough hands-on implementation.
It’s the same as a “non-coding architect” role (remember those). Most of them are absolutely full of shit architecture astronauts.
I think this is a question of how much control the user is able to have over the end product. Music creation in particular is very difficult... I've produced music for 4-5 years, and the granularity with which one has to control the finest pieces is often mindblowingly frustrating. It takes years to develope a decent ear for mixing.
By giving up that control, you do get to a quality end result sooner, but that end result can only be an approximation to your original vision, since you're giving up the control required to shape the sound to that granular level.
Additionally, without the knowledge of how you got from A to B, you don't know what else is possible (or impossible.) In the process of doing something manually, you may stumble across a particular setting or effect that creates something you never even considered. And now, that is knowledge you can use on the next project.
Does the woodworker who shape using a handsaw use less "soul" than the one who uses a machine?
Does the musician who use a DAW and VSTs instead of analogue tape recorders create music with less "soul"?
Does the painter who buys acryllic paint instead of synthesizing their own dye from plants use less "soul"?
As technological innovation progresses, the barrier to creation falls. The process of creating something is not to be conflated with the final piece of art itself.
Your analogies are flawed. DAWs and skill saws generate nothing. They take skill to operate, and a novice cannot use these tools at all unless they know the craft.
Compare to this to prompting an LLM: “Generate a third person where game with a view from above where you can steal cars, shoot at people, run from the police, etc.” Anybody with access to the tool can do this, and the results are just another uninspiring GTA clone that you would imagine.
The latter is more like a carpenter ordering their “work” from alibaba then it is like using a skill saw.
I'm critical of many "AI" developments but I can't and don't want to argue with this. I say we still need to struggle for humanity and we do need to save our souls, but that "it's a machine" is not where the battlefield is.
Hmm, but AI isn't just a really-good tool, it's also doing creative work too. As an aspiring composer, music generators are in my opinion really quite good, and often matches what composers and song writers can do. So if you ask me does a person who creates music with gen AI miss out on the soul of creating music? In my opinion at least, for the most part, yes.
To write a piece of music, you're working at so many different levels, the analytical, emotional, and structural as well as drawing on years of training and experience. When a person with little (or even just a moderate amount of) music training generates a piece of music in a couple hours, are they actually a composer? I personally would say no. I mean it does take a good ear (which is important) to use a music generator well, but still, would say they are more of an editor or an evaluator or a practical critic of music instead of a composer.
Yeah, at what point in a discipline does the increasing skill of AI overtake the need for our contributions from the working being done? In music, it's happened already. Looks like it's happening in coding too.
Does the carpenter who used to build custom fit cabinets with hand and power tools put in the same creativity when he just carries around a scanner, scans the area, the customers use software to select the layout, approve the work, then the CNC cuts out the wood, then all that's left is to put the screws in the holes and go home.
This isn't like the step from hand saws to power saws, and it's disingenuous to pretend like it is. This is what the startup machine has been doing to every industry... finding... "inefficiencies" and "optimizing" them.
Not _my_ opinion, but I just wanted to share that many people (in the Midwest) do believe that anything synthetic that it not readily made from simple materials has "less soul". It's a sorta test of "if I dropped you off in the jungle, can you still produce works of soul? Or are you just another cog in the machine.".
It's when a woodworker, musician or painter completely outsources their work and just marks what's wrong, sending those parts back. Yes, the final art piece might be the same, but the artist definitely uses less of their "soul".
This is the case perhaps 95% of the time.
Oversight is very important, and architectural thinking cannot yet be outsourced, only execution.
reply