AI vs. software companies

[Spekulatius]

I did a few queries in Gemini about  gross margins for OpenAI and Anthropic and the answer is that it about 40% and may right answer is 42 . Thats pretty low and the reason is that unlike traditional software, their service is expensive to provide because of the compute cost for tokens. This may go up over time but it’s not a certainty  because competition may eat into margins. So in any case, these companies will be valued lower than software cos (which can easily exceed 80% gross margins).

 

My guess is that business models will have some attributes similar to chemical processors because in the end they process energy via silicon into tokens that get consumed with by their customers. So besides model quality which determines the value of the tokens, the cost to produce these tokens is of immense importance which is likely why OpenAI gets into producing their own chips.

Edited Saturday at 09:52 PM by Spekulatius

[whatstheofficerproblem]

https://a16z.com/the-cost-of-cloud-a-trillion-dollar-paradox/

 

LOL.

[whatstheofficerproblem]

On 5/29/2026 at 3:12 AM, backtothebeach said:

Lol. There is no real intelligence in LLMs.

 

Perplexity has learned from this prompt being posted on social networks and gets it right: 

 

"Drive there. The car needs to be physically at the car wash to get cleaned, so walking would leave it at home. ..."

 

ChatGPT however:

 

"If it’s only 100 meters and the weather’s nice, walking is probably the better move. It’ll take about 1–2 minutes on foot, you avoid the hassle of starting the car just to move it a very short distance, and you get a bit of fresh air in the sun."

 

Excellent point. Should also note the inference/API costs which the AI bulls say are coming down 1000x are not really calculated costs. It's cost to reach a benchmark. These benchmarks have been public for a long time now and it's very easy to train future models or existing models to optimize for the bench. Like the folks on Shitter say, "benchmaxxing". So inference/token costs coming down is an illusion.

Edited 3 hours ago by whatstheofficerproblem

[treasurehunt]

1 hour ago, whatstheofficerproblem said:

https://a16z.com/the-cost-of-cloud-a-trillion-dollar-paradox/

 

LOL.

 

This article was published over 5 years ago. Surely that's enough time for companies to move workloads off the cloud en masse if that is beneficial, right? The article concludes thus: "either the public clouds will start to give up margin, or, they’ll start to give up workloads". Has either of these things happened?

 

Or are you making some other point that I missed?

[whatstheofficerproblem]

1 hour ago, treasurehunt said:

This article was published over 5 years ago. Surely that's enough time for companies to move workloads off the cloud en masse if that is beneficial, right? The article concludes thus: "either the public clouds will start to give up margin, or, they’ll start to give up workloads". Has either of these things happened?

 

Or are you making some other point that I missed?

 

My point was that this sounded familiar.. AI parallel. To your point, yes it was indeed happening. AWS went from growing 40% YoY to 20% and then 16 and it's lowest ever at 12% in the span of a year in 2023. Same thing in that timeframe with Azure from 50% to 30% and MSFT's CFO even warned on the earnings call that further slowdown is coming and remember the stock tanking on that. GCP went from 45% to 32% also, slowest ever since they started reporting that segment.

 

What happened according to Jassy's words was that FinOps became a thing and since people were spending too much on cloud, they started optimizing within cloud very aggressively. Sell-side notes, and I recall Barclays or Goldman reporting that over 80% CIOs planned on moving workloads off cloud to see how it goes while IDC found ~80% expecting to repatriate some compute or storage within a year.

 

Then, magic happened. AI came in, total market grew, server lives expanded increasing margins by couple hundred basis points. None of the points identified in the article are wrong. This paradox they mention simply moved a layer up. Why pay NVDA or frontier labs money, when AWS/GCP/Azure give you access to the orchestration layer? That's going to be the thinking going forward imo.

Edited 57 minutes ago by whatstheofficerproblem