AI vs. software companies

[Spekulatius]

I did a few queries in Gemini about  gross margins for OpenAI and Anthropic and the answer is that it about 40% and may right answer is 42 . Thats pretty low and the reason is that unlike traditional software, their service is expensive to provide because of the compute cost for tokens. This may go up over time but it’s not a certainty  because competition may eat into margins. So in any case, these companies will be valued lower than software cos (which can easily exceed 80% gross margins).

 

My guess is that business models will have some attributes similar to chemical processors because in the end they process energy via silicon into tokens that get consumed with by their customers. So besides model quality which determines the value of the tokens, the cost to produce these tokens is of immense importance which is likely why OpenAI gets into producing their own chips.

Edited June 27 by Spekulatius

[whatstheofficerproblem]

https://a16z.com/the-cost-of-cloud-a-trillion-dollar-paradox/

 

LOL.

[whatstheofficerproblem]

On 5/29/2026 at 3:12 AM, backtothebeach said:

Lol. There is no real intelligence in LLMs.

 

Perplexity has learned from this prompt being posted on social networks and gets it right: 

 

"Drive there. The car needs to be physically at the car wash to get cleaned, so walking would leave it at home. ..."

 

ChatGPT however:

 

"If it’s only 100 meters and the weather’s nice, walking is probably the better move. It’ll take about 1–2 minutes on foot, you avoid the hassle of starting the car just to move it a very short distance, and you get a bit of fresh air in the sun."

 

Excellent point. Should also note the inference/API costs which the AI bulls say are coming down 1000x are not really calculated costs. It's cost to reach a benchmark. These benchmarks have been public for a long time now and it's very easy to train future models or existing models to optimize for the bench. Like the folks on Shitter say, "benchmaxxing". So inference/token costs coming down is an illusion.

Edited June 29 by whatstheofficerproblem

[treasurehunt]

1 hour ago, whatstheofficerproblem said:

https://a16z.com/the-cost-of-cloud-a-trillion-dollar-paradox/

 

LOL.

 

This article was published over 5 years ago. Surely that's enough time for companies to move workloads off the cloud en masse if that is beneficial, right? The article concludes thus: "either the public clouds will start to give up margin, or, they’ll start to give up workloads". Has either of these things happened?

 

Or are you making some other point that I missed?

[whatstheofficerproblem]

1 hour ago, treasurehunt said:

This article was published over 5 years ago. Surely that's enough time for companies to move workloads off the cloud en masse if that is beneficial, right? The article concludes thus: "either the public clouds will start to give up margin, or, they’ll start to give up workloads". Has either of these things happened?

 

Or are you making some other point that I missed?

 

My point was that this sounded familiar.. AI parallel. To your point, yes it was indeed happening. AWS went from growing 40% YoY to 20% and then 16 and it's lowest ever at 12% in the span of a year in 2023. Same thing in that timeframe with Azure from 50% to 30% and MSFT's CFO even warned on the earnings call that further slowdown is coming and remember the stock tanking on that. GCP went from 45% to 32% also, slowest ever since they started reporting that segment.

 

What happened according to Jassy's words was that FinOps became a thing and since people were spending too much on cloud, they started optimizing within cloud very aggressively. Sell-side notes, and I recall Barclays or Goldman reporting that over 80% CIOs planned on moving workloads off cloud to see how it goes while IDC found ~80% expecting to repatriate some compute or storage within a year.

 

Then, magic happened. AI came in, total market grew, server lives expanded increasing margins by couple hundred basis points. None of the points identified in the article are wrong. This paradox they mention simply moved a layer up. Why pay NVDA or frontier labs money, when AWS/GCP/Azure give you access to the orchestration layer? That's going to be the thinking going forward imo.

Edited June 29 by whatstheofficerproblem

[backtothebeach]

On 5/7/2026 at 4:12 PM, backtothebeach said:

It’s been a week since the last post in this thread. The whole AI-is-going-to-destroy-software narrative is getting tired.
Something about the bounce in software stocks today feels strange to me. The CSU universe up bigly. NOW, ADBE, too. Maybe the pod shops are getting back into software?

 

On 5/7/2026 at 4:36 PM, whatstheofficerproblem said:

No. They will fade semis or more so opticals to rotate into the CPU play now. See ARM, AMD & INTC. This is a zero sum flow game and software thus far has nothing to show for as to attract their capital. For them, easier ways to make money in AI verticals.

 

The best activity you'll see here is likely these names being up on folks covering their shorts.

 

Quite the sector rotation again the last few days, with everything that had been struggling (software, insurance brokers, even consumer staples) getting a boost and semis/AI struggling. Just a blip/short covering again? Looks like a much wider rotation this time, not only software.
 

Edited July 2 by backtothebeach

[whatstheofficerproblem]

Wow. The "too dangerous to use" Fable refuses to tell me why "What are you listening to" by Chris Stapleton is pure genius lyricism. We're about 3.50 GW away from AGI. Can't wait for Dario to whine at the White House on how AI quoting lyrics of a country song would be detrimental to the western civilization.

 

[EgonKuhn]

3 hours ago, whatstheofficerproblem said:

Wow. The "too dangerous to use" Fable refuses to tell me why "What are you listening to" by Chris Stapleton is pure genius lyricism. We're about 3.50 GW away from AGI. Can't wait for Dario to whine at the White House on how AI quoting lyrics of a country song would be detrimental to the western civilization.

 

How can open-source and open-weight LLMs not win in the long run, given that we are already tinkering with this level of self-censorship in commercial models at this early stage?

 

Unless perhaps they simply get banned at some point?

[gfp]

7 hours ago, whatstheofficerproblem said:

Wow. The "too dangerous to use" Fable refuses to tell me why "What are you listening to" by Chris Stapleton is pure genius lyricism. We're about 3.50 GW away from AGI. Can't wait for Dario to whine at the White House on how AI quoting lyrics of a country song would be detrimental to the western civilization.

 

 

All of the US frontier models are programmed to refuse anything approaching reproducing original copy written material.  Song lyrics, an entire novel, etc...  Its a blanket choice based on legal realities and maybe something negotiated as part of their agreements that give them access to all of that material in the first place.

 

Even an "open weight" model from China has to be hosted somewhere and if Kimi ends up being served on AWS, AWS might very well tinker with it to make sure it doesn't run afoul of the legal system.  Just not worth the headache.

 

I don't think this tells us anything about model capability or how dangerous a model is.  Its just the legal department

[whatstheofficerproblem]

2 hours ago, gfp said:

I don't think this tells us anything about model capability or how dangerous a model is.  Its just the legal department

 

Isn't that stupid though? They have violated all sorts of IP laws and ToS' to train these models in the first place, but now merely quoting a pre-existing material is taboo? Jailbroken models will soon gain more traction than frontier.

[EgonKuhn]

4 hours ago, gfp said:

 

All of the US frontier models are programmed to refuse anything approaching reproducing original copy written material.  Song lyrics, an entire novel, etc...  Its a blanket choice based on legal realities and maybe something negotiated as part of their agreements that give them access to all of that material in the first place.

 

Even an "open weight" model from China has to be hosted somewhere and if Kimi ends up being served on AWS, AWS might very well tinker with it to make sure it doesn't run afoul of the legal system.  Just not worth the headache.

 

I don't think this tells us anything about model capability or how dangerous a model is.  Its just the legal department

 

As far as I know, LLMs do not operate in a strictly deterministic way. This means that any form of artificial self-censorship or self-restriction will inevitably lead to unintended side effects that are, at least in part, undesirable. It wouldn't surprise me if, in the long run, this resulted in a competitive disadvantage compared to LLMs that implement little to no self-censorship or self-restriction.

 

If those latter models are on top freely available to anyone as open-source code, it will likely be nearly impossible to put the genie back in the bottle, maybe short of harsh repressive measures. Ultimately it’s just code, if push comes to shove, I can simply run it on my own AWS instance or in doubt on my own local device (provided I have the right hardware).

[gfp]

so kind of like how talking with a super woke politically correct snowflake type human being can be a bit frustrating even when you are talking about "safe" topics?

[Spekulatius]

1 hour ago, EgonKuhn said:

 

As far as I know, LLMs do not operate in a strictly deterministic way. This means that any form of artificial self-censorship or self-restriction will inevitably lead to unintended side effects that are, at least in part, undesirable. It wouldn't surprise me if, in the long run, this resulted in a competitive disadvantage compared to LLMs that implement little to no self-censorship or self-restriction.

 

If those latter models are on top freely available to anyone as open-source code, it will likely be nearly impossible to put the genie back in the bottle, maybe short of harsh repressive measures. Ultimately it’s just code, if push comes to shove, I can simply run it on my own AWS instance or in doubt on my own local device (provided I have the right hardware).

Yes, I agree. If we self censor, users in other countries that can these cheaper models are at an advantage.

In any case, the business model for LLM providers like OpenAI becomes highly questionable. They would need to recoup their entire R&D expense in the roughly 3 month lead they have to open source models.

Edited 3 hours ago by Spekulatius

[LC]

Any output filters are provided as an overlay to the underlying model. It should not effect frontier model development.