I've really come around to trusting OpenAI a lot more than Anthropic the past few months. Reading between the lines of his own output, Dario Amodei comes off as both a dogmatic believer in ASI as a perfect, infallible ruler for humanity and quite an extreme American nationalist. The company, likewise, looks to be in ideological lockstep. I could see them, say, allowing or consciously creating runaway ASI they believed was ideologically aligned with them.
OpenAI seems generally less dogmatic and more practically oriented. There's really nothing particularly good about them, but you can at least predict how a normal company will act.
This is a fun peek into the economic implications of RSI/ASI. Because it's so infinitely valuable that it basically destroys all markets, labs will eventually do stuff like stop releasing models completely and skipping out on contracted commitments because they'll have the power to just drive their competitors out of business before the legal battle gets expensive.
Cloud providers - at first smaller ones, then the hyperscalers - will follow suit, completely closing sales to anyone but the labs and demanding payment in equity/direct decision-making power rather than cash. There's no particular reason why the inference/training split has to be 80/20, and no amount of willingness to pay can help you in an event that turns your money worthless.
I don't think this scenario makes sense. It's one of a class of scenarios I've seen several of, that simultaneously assume:
A) ASI is developed and massively overshadows the rest of the world economy
B) the world still has rule of law, contracts, business, well-developed finance, etc
You can get to a lot of weird conclusions if you assume both A and B, but I think the much more likely scenario is that if A happens, B stops being true in short order. If you are a company and you have ASI, you just stop caring about business and money and economics, and your outcomes instead start looking like "you conquer the world" or "you upload the board of directors to a fleet of von Neumann probes" or "you messed up, everyone dies".
There will be a brief(or, depending on the underlying rules of reality ASI uncovers, not-so-brief) period where A and B do overlap - we have superintelligence but still have to run experiments, manufacture robots, test new drugs in vivo, etc. That period is in and of itself dangerous for the labs, because many entities can just stop them by denying necessary inputs. For the labs to conquer the world, they'll need cooperation - from the state, from robotics companies, from compute companies, from the mining and energy and agriculture sectors.
There will be a period of time where markets attempt to run in a business-as-usual way while the transactions that matter happen as power-sharing arrangements - spots on the "AI Governance Board" or the "uploaded to von neumann probe" club. Markets will still matter in that the labs will need the state to overturn market obstacles to control of the world.
The existence of the A-B overlap also suggests to me that the US-China gap is less dire for China than it appears - they may be able to use their superior industrial, robotics, and scientific base to win the second leg of the race despite losing the first.
The combination of A and B is cyberpunk at its core, it takes off in the form of corporate consolidation and then control of the government. Large corporations will still have the rule of law between each other because they'll have both money and hard power. The average individual that wants to rise up against said corps will quickly be identified by ubiquitous surveillance and imprisoned/slave labor camped.
A market with only a single good makes that good infinitely valuable for exchange purposes. No trade can happen - everyone is just sitting on a big pile of the only thing that matters and they can only trade it for less of the only thing that matters.
I don't think one Claude can replace ten engineers of the caliber it takes to build a billion dollar company.
I also don't think that every set of ten engineers of that level builds a billion dollar company every time.
There is also a limit to the number of billion dollar companies that can be built before being a "billion dollar company" no longer means much (see: Zimbabwe).
Not really. It's possible they could, but in practice they cannot. Creating a billion dollar company requires a good idea, good timing, and a lot of luck, the engineers are the least important part.
More than that it takes things like the right social connections, strong marketing, insight into customer demand, infrastructure spend, etc. You can't normally just convert engineering effort into profit in the way implied.
10 engineers build a billion dollar company and immediately realise they need 1000 people to run it. Getting 1000 Claude licenses will still be expensive, let alone maintaining those swarms of agents.
this wont be possible by the time its possible. there would be massive deflation. why would i care about 10 engineeers prompts when i can prompt it myself
There's literally no indication that this is the case, or will ever be. Unless you're a completely naive person who's impressed with all output of an LLM because you don't know what you're talking about. These models aren't impressive, and the people who think they are impressive are even less impressive.
Especially when you can actively choose to not use Anthropic. They think they have a moat from all of the IP they've stolen. Just wait until there's nothing more to steal and the laws eventually turn against them. And let's be honest about these companies. It is very much Dario and Sam and Sundar and Mark and Peter and Elon and... These are the choices they are making and hopefully they are held accountable both legally and within society as a whole.
No, you pretty obviously didn't understand it, at least in the sense of ASI being talked about. The whole "oh don't buy it" stops mattering. Humans are no longer the sole creators of information and intelligence. That is AI no longer has to steal, but humans will have to beg, borrow, or steal the information/products that ASI creates.
I didn't say "buy" anything. You've clearly bought into the narrative that ASI is even remotely possible with current architectures. You've chosen to ignore all of the lies that have thus far failed to come to fruition. The assumption is an insane leap from models in a loop that have no intelligence to models taking over the "sole creators of information and intelligence". Anthropic marketing is doing a phenomenal job for a large portion of those in the bubble.
it'd be really funny if we got the RSI -> ASI world, all human labor became worthless, etc., but everyone with any money in the labs also lost their shirts because OSS is maximally good for most inference anyway.
The token consumption concern is real but I think the framing misses a key trend: specialized models for specific domains.
In legal tech, we run domain-specific models for contract review that use 90% fewer tokens than general-purpose LLMs because they understand legal document structure natively. The token cost per document dropped from dollars to cents.
The real "tokenpocalypse" is for use cases that try to do everything with one general model. As the ecosystem matures toward specialized tools (similar to how we got specialized IDEs for different programming languages), token efficiency improves dramatically.
The analogy holds: general-purpose models are like Swiss Army knives — useful but inefficient. Domain-specific models are like proper tools — more expensive upfront but vastly more efficient for their domain.
A US-China AGI ban treaty could prevent superintelligence indefinitely. Data centers are hard to hide. Have fun buying GPUs when you're cut off from all global payments. America would have to make some unpleasant concessions but that seems like a solid trade for preventing a wide variety of nightmare futures.
Not the article's main point but I've never liked the "google killing products" complaints. People always talk about how big companies fail because they're unwilling to take risks and just recommit to their areas of strength, but this is what risk-taking looks like - you blast out products, see what sticks, and kill what doesn't. People who think it's a quality product won't be wary of whether it'll get killed - the quality itself is insurance against that. How many DAUs would stadia or hangouts or even reader have today?
This is not about stickiness. People complain because they liked the dead product. Do you hear complaints about Google+ dying? Reader wasn't a risk, it was a product people loved that wasn't hard to run. It was just too boring to maintain, didn't support the ad monopoly, and Google dropped it for the next shiny monetizable object.
Anyway, enterprise products are an entirely different ballgame where product support, and the reliability thereof, is measured in decades. The consumer product attitude is just a bad look, but things like the Railway incident are deal killers.
Well, actually, Google+ circa 2014 was the only social network that had enough of the interesting technical content for me, and I did use it in exactly the way how one would use a decent social network: I posted my content, got comments, and read what other people posted. Some formatting weirdness aside, it was not bad at all, and surely better and quieter, in a good sense, than Facebook back then, and especially than Facebook now.
I still have the Takeout archive of my posts, might be a nice time machine experience to parse that huge JSON and read through it.
I thought the “circles” idea made total sense and it could have been a good social network, if they had let it do that instead of forcing it into every other part of Google.
Reader was dropped in the run up to G+. I believe there was a strategic decision to try and get people to move to G+ and move both personal news and organisational news together.
The leadership was never completely honest about why Reader was shutdown, and the stated reasons didn't pass some basic sniff checks. But it was easy to read between the lines: the executives' attention was on other things, and Reader was a threat to their growth. But also it was a passion project that a company like Google would struggle to keep updating since it brought in little revenue (even though there were hordes of people volunteering to maintain it for free in their 20% time).
The decline of RSS among a certain audience was only peripherally related to the elimination of Reader; it's not like there aren't other RSS readers out there even if Google makes a convenient villain. But Google did whiff on social in general and certainly indexing the world's information became a very secondary goal especially after failing with some efforts on the copyright front.
Google+ couldn't handle spams. Inbox was an excellent execution. one needed the tech that made Gmail, and the other couldn't co-exist with Gmail? we will never truly know.
I'm sure that Google internally is well aware of the negative press that comes with product shutdowns, and is doing them regardless as a deliberate strategic tradeoff where they believe the benefits outweigh the costs.
But it's very difficult to measure the costs, bc the #1 cost is lost trust, and how do you measure that? Many people simply won't sign up for a Google product bc they don't trust it'll be around long enough to justify the investment. These people don't show up in any metrics that you can reason about, and they're the least likely to take any surveys you might send out. At best, Google can guess what the impact is, and they might be wildly underestimating.
I think a different strategic decision they could've made (and still could make!) would be to the do the opposite, and prioritize the benefits of keep projects alive over the costs of ruthlessly sunsetting then.
They could say, "You know what, we have considerable resources. When we release something new, we're going to dedicate ourselves to keeping it running indefinitely." They wouldn't have to market them, or advertise them, or connect them to every new part of the evolving Google ecosystem, or make them particularly easy to find, or even keep them open to new signups. But just keeping them running as-is, indefinitely, and having customers tell each other, "It's Google, you can trust it, it's not going away," would be such a great PR win.
It’s not just the price of keeping the servers humming. You have to pay people to maintain the software. So now you’re paying hundreds or thousands of people to maintain zombie software.
Yeah, there are costs for sure. But this is "zombie software" with millions, probably tens of millions of users. And Google has ~80,000 engineers? And the company prints an incredible amount of money.
I think the real cost/risk here is not financial, but strategic, i.e. preventing a loss of focus.
The common critique of Googles actions - the organization has profits, therefore there is no problem engaging in less profitable activities — strikes me as superficial.
It’s not about investing any given portion of revenues, it’s investing optimally. There are opportunity costs that must be considered in investments (and that means Net Present Value calculations).
Google’s revenue and profits are for the shareholders. When revenue is directed back into the business the question simply isn’t if the whole business will make money, it’s if this investment is optimally profitable considering all the alternatives. If a support engineer on Google+ generates $X over 5 years, but that same resource would generate $3X working on Gemini then dictating eternal Google+ support is robbing future Google of revenue.
Investments need to be individually justified, but also better than the alternatives to make fiscal sense. Even though that sucks for pleased consumers.
Yes, but this is exactly my point. You can't perfectly calculate exactly how much revenue is lost due to breaking trust with users because you repeatedly sunset projects that they've relied on.
Maybe if Google had those support engineers on Google+ for 5 years generating $X, that would've created enough trust that Gemini could now generate $4X.
> People who think it's a quality product won't be wary of whether it'll get killed - the quality itself is insurance against that.
Not true in the slightest. Google has had some quality hardware products where they killed the cloud service and rendered them useless. Products that were stable and working, but Google decided to pull the plug and make the hardware worthless.
Quality of the product means nothing. You’re at the mercy of the whims of Google’s decision making. The thing you like may stop working in a couple months.
Unfortunately quality and earning money are only loosely correlated - especially for Google when they can pour money into making something good, but baby necessarily figure out how to make it earn.
And it also seems if it's not a $100 m+ business they lose interest very quickly. So even good things that could run somewhat cheap of optimized end up with no long term place in the Google ecosystem when they fail to make it big.
... That all said Google kills perfectly good things because they have a few internal rules that encourage it: that there can only be one of a system (dunno if that's applies to product but they seem to always want to consolidate every few years) and that stuff cannot be unowned so reorgs => kill products that don't match the new org structure. That and they incentivize pumping out new products with their promotion process - whether or not they really needed yet another chat/vc/etc product, someone probably figured they could climb the career ladder by shipping another one.
> Not the article's main point but I've never liked the "google killing products" complaints. ... you blast out products, see what sticks, and kill what doesn't
Except that Google kills the products we use! Google Reader had basically the entire community.
No one cares when they kill something niche.
I don't touch new Google products, they're toxic.
And now with AI I can finally stop using their search too. All that will be left is email because it's too much of a commodity.
If anything, email is the #1 priority for de-googling, lest they slop you with a ban. Losing access to your email is so disruptive that migrating to a more trustworthy provider with custom domain support is worth the time and effort of updating addresses.
Most people would argue that Stadia would have many. Many people loved Google Reader. There are numerous examples of things that were great and were killed, because they hadn't monetized enough or "fast enough", and when you are chasing results on a quarterly basis, you can't always get things that will generate tremendous value with more time.
I think the complaint about Google specifically is that they seem to do these things and commit to what seem to be whole lines of business without an actual business plan to make it viable.
It’s one thing to take risks. It’s another thing to just guess without a plan.
It's interesting to imagine if there's some kind of middle ground where products could be launched without the pretense of them being permanent? I suspect at least some of people's frustration is that X or Y was pitched as something serious, which then grates some when it gets canceled.
But maybe you can't launch a product without pretending it's going to be real because it'll be dead on arrival?
> Not the article's main point but I've never liked the "google killing products" complaints
Exactly. I mean, you can argue forever about the stuff that got killed. But you know what didn't get killed? All the stuff they're turning into $4.6T of market cap.
In particular, they didn't kill AI. They're the only pre-AI tech giant with a successful frontier LLM, everyone else failed (Meta) or missed the boat (Apple, Microsoft[1]). They're the only tech giant[2] to produce working in-house datacenter AI hardware.
Making a lot of bets means that you make a lot of bad bets too. But Google has made a lot of good bets.
[1] Though MS got near-exclusive access to OpenAI via a very expensive late investment, which sorta counts.
[2] NVIDIA is giant now, they certainly weren't when these bets were being placed.
> you blast out products, see what sticks, and kill what doesn't.
They killed their chat app. Then had their pants on fire after Discord became the thing. The same can happen with any app they killed. So no, that's not 'taking risks'. Taking risks is shutting down perfectly working apps that can suddenly become 'the new thing'.
I've come to believe that Big Tech's fondness for launch-and-kill approaches to new products is closely related to the career incentives for both PMs and finance chiefs. Both of them are acting rationally to advance their careers. In tandem, they create a horrifying cycle of death.
Product managers win their chops by launching something and winning applause for getting fast adoption in the first few months. Once the boss says "Wow," it's on to a bigger new chance. It's habit-forming to outrun your failures in a system where personal incentives reward ephemeral success.
Finance folks win their chops by identifying bits of the company that aren't pulling their (economic) weight. If you haven't delivered $$$ of efficiency savings, why are you even on the payroll? Pulling the plug on small/mid-sized products that have plateaued is the way you get recognition and promotions.
CEOs, of course, could change this. But they're the ultimate plate-spinners in a big company, trying to keep up the appearance that they've got everything in control and making sure that neither shareholders nor employees mutiny. As long as they've got a new sizzle story to sell, no one (except users) is there to grieve about what's dying. And users of a product that's less than 1% of revenue can be safely ignored.
Can you name a good new Google product then? I just can't remember anything recent. I can't even remember any good recent _improvements_ to their core products.
If anything, recent changes are more like downgrades than upgrades.
Waymo was originally called the Google self-driving car. If it's only come to your city this year, I'd say that's pretty new. If it's not in your city yet, it's so new it's not even there!
Is it any sort of leap over the product's it's copying?
IBM made some decent (sometimes extremely good, even!) products in a lot of segments for a long time after losing their relevance as "driving the future of computing." But rarely as a segment-definer or introducer.
You're entitled to your opinion, but you're literally the first person I've ever heard say that. Even people who like LLMs seem to think that GPT and Claude are the good ones, with Gemini being B tier at best.
#6 in usage by tokens. It obviously has some use case. My experience is that Gemini is much more token efficient than other models. Measuring tokens is like measuring distance travelled by measuring fuel consumption.
I mostly use Codex though, I can't be bothered to have more than one subscription.
But it still great today, no? I mean professionally I am in the MS 365 world, which is definitely not better than gmail. We have some mail servers for services and until now we weren't able (for instance) to find an open source alternative to gmail and office 365.. any recommendation?
I think AI companies have actually broadly adopted the right strategy. There's no way to sugarcoat or hide that your company's pitch is "your salary is our TAM, btw our product is so powerful it might cause human extinction". Deploying as widely as possible while steamrolling opposition before it can get its bearings is the only viable option for the technology as they describe it.
I call bullshit. They got greedy. The right way would have been to work with both knowledge and culture workers instead of making it an antagonistic relationship. These are the people ai companies need to operate their agents. Its stupid to threaten them and foment resentment
Also dont be ridiculous. We didnt go extinct from the nuke, we wont go extinct from a next token predictor
Yet. We've come very close (see e.g. Daniel Ellsberg's The Doomsday Machine or https://en.wikipedia.org/wiki/Nuclear_close_calls). A catastrophic mistake or miscalculation could trigger a massive exchange at any time; it could happen on any given Thursday.
I have no experience in the CTF scene so I'm curious - why not lean in and design the puzzles with an AI harness like the one top teams use in the loop and use the(presumably) expert skills of the designers to patch up the holes until the AI can't find them? Do you just end up designing ~perfectly secure systems that no human can break without finding monetizable 0days?
I think that misses the point - it's a little bit like asking why FPS game developers don't lean into aimbot usage. You could, but by default it's a bit boring, and a different type of game.
Agent-assisted programming is fundamentally the skill of directing and supervising agents. I don't see any reason to believe that working a job where you direct and supervise agents will make you any worse at directing and supervising agents long term.
OpenAI seems generally less dogmatic and more practically oriented. There's really nothing particularly good about them, but you can at least predict how a normal company will act.
reply