Energy for AI

[AlwaysDay1]

Nvidia is benefitting from supply constraints in the face of growing demand. The next phase seems to be energy which will face supply constraints with growing demand. Which companies will this benefit the most in your opinion? Some will benefit but require increased capex. Others may be straight money printers like Nvidia.  Let's discuss! 

[ValueArb]

Do you really think AI is going to materially affect the demand for energy? I doubt it will ever surpass crypto in that respect. There are a couple things that I think we can agree on.

 

1) Mankind will always produce and use more energy over time. This is a trend that hasn't changed for many thousands of years.

2) Computer calculations will always get cheaper and use less energy per calculation over time. This is a trend that hasn't changed since before the invention of integrated circuits.

 

So in the long run even if we agree AI use will grow at high rates whether its going to significantly increase energy demand is unclear. One example is the Apple Neural Engine, which is their machine learning procesor embedded in their iPhone and Mac Apple Silicon. In 2017 Apple introduced the A11 SOC in the iPhone 8 & X with a Neural Engine that could perform 600 billion operations per second. In 2023 Apple Introduced the A17 with a neural engine that can perform 35 Trillion operations per second. So in 6 years they've increased AI performance by nearly 600 times while using roughly the same amount of power.

 

Now that doesn't directly apply to nVidia as they aren't selling mobile devices but instead offering the highest possible performance that can fit in a reasonable power and heat limited package for PCs and Servers. But still they benefit from the same reductions in process size (TSMC's 10 nm to 3 nm in the case of the A11 to A17) that drives Apple's performance and efficiency gains. So while their increase in performance per watt is likely a bit less given how close to the edge they already were and how power usage is a secondary goal for them, it's still increased a huge amount over the same period and will continue to do so in the future. 

[ValueArb]

Sabine almost always thinks through things better than I do, so there is this.

 

 

[LearningMachine]

1 hour ago, ValueArb said:

1) Mankind will always produce and use more energy over time. This is a trend that hasn't changed for many thousands of years.

 

I think this is true.  I think we'll overestimate what AI could provide in the short-term, but will underestimate what AI could provide in the long term. 

 

Before industrial revolution, people couldn't have imagined all the things we take for granted today that take energy. 

 

Similarly here, imagine human desires were not bottlenecked by availability of human labor.  For example, think about all the things that people in top 1 percentile can afford with human labor today, e.g. concierge to take kids around, help to keep home clean, human labor to rebuild a house on the lake and another cottage in woods, repair roads, build more roads, build more housing, jets to fly around the world, etc. Now, imagine all these things were not bottlenecked by availability of human labor for the remaining 99% of humans. 

 

All that would need more and more energy over time, and it is possible that growth rate in energy use will go up when we are not bottlenecked by human labor needed today to fulfill a lot of the human needs that top 1 percentile are able to satisfy today. 

Edited March 26 by LearningMachine

[Saluki]

I wonder if this will be a winner take all (or most) like other tech.  MSFT is the winner take most in the business software (word, outlook, excel) and AMZN is the winner take most in online sales and cloud, NVDIA is the winner take most in AI chips, and GOOG and META are the winner take most in advertising.  

 

GOOG and MSFT seem to be the frontrunners in AI and Apple trying to license the tech rather than build their own seems to be acknowledgement of the idea that it's better to license the best one than to make a pretty good one.  If that's the state it is now, and AI works best with more users, more data, and more iterations, then maybe the lead will grow bigger over time and the lead will eventually be like what Alphabet has with Google.  Someone can spend $100bln on Bing for a single digit market share.  https://www.neowin.net/news/microsoft-says-it-spent-100-billion-on-bing-and-is-stopping-google-from-having-a-monopoly/#:~:text=In the hearing%2C Nadella mentioned,in terms of market share.  If that's not proof of an incredible moat, then what is? 

 

And a search engine is a lot easier to make than AI.  Search engines have been around since the internet has been around. Maybe if more people start using GOOG's AI than MSFT's or vice versa, then the learning and constant reiterations will make it so that you can't catch up.  I.e. if you kept using the crappier AI it would get better, but you won't because it's like BING.  So the better one just keeps getting better.  

[LearningMachine]

On 3/27/2024 at 8:41 AM, Saluki said:

I wonder if this will be a winner take all (or most) like other tech.  MSFT is the winner take most in the business software (word, outlook, excel) and AMZN is the winner take most in online sales and cloud, NVDIA is the winner take most in AI chips, and GOOG and META are the winner take most in advertising.  

 

GOOG and MSFT seem to be the frontrunners in AI and Apple trying to license the tech rather than build their own seems to be acknowledgement of the idea that it's better to license the best one than to make a pretty good one.  If that's the state it is now, and AI works best with more users, more data, and more iterations, then maybe the lead will grow bigger over time and the lead will eventually be like what Alphabet has with Google.  Someone can spend $100bln on Bing for a single digit market share.  https://www.neowin.net/news/microsoft-says-it-spent-100-billion-on-bing-and-is-stopping-google-from-having-a-monopoly/#:~:text=In the hearing%2C Nadella mentioned,in terms of market share.  If that's not proof of an incredible moat, then what is? 

 

And a search engine is a lot easier to make than AI.  Search engines have been around since the internet has been around. Maybe if more people start using GOOG's AI than MSFT's or vice versa, then the learning and constant reiterations will make it so that you can't catch up.  I.e. if you kept using the crappier AI it would get better, but you won't because it's like BING.  So the better one just keeps getting better.  

 

I think LLMs is only one of the benefits of AI.  For short-term for LLMs, moat is in data and existing user experiences that carbon based neural nets (humans) are trained on already.  Tech itself will spread around with good engineers getting high bids from those who have the moat.  META has already benefited from it in increasing relevance of its ads. 

 

Nvidia's customer base are companies with big pockets and when those customers are paying 10s of billions of dollars, they can easily spend a few billion dollars each to continue to invest to try to create their own AI chips.  Will one of them be successful eventually?  Time will tell.  I think probability is significant that at least one of them will be successful eventually. 

 

Next huge value-add won't be in increasing relevance of ads, but it will be in another scenario.  One of those will be to reduce # of hours of high-hourly rate knowledge workers.  If you can save 50% of hours of 10M workers getting paid $100K per year, you have created $500B in value, and that is per year.  At 10-20X multiple, that is worth $5-10 Trillion.   To enable this scenario, you need access to data that those knowledge workers are consuming and producing already.  You can deduce from there, who has access to that data and who already owns the user experience that these knowledge workers have been trained on. 

 

Longer term, who all will benefit is a little like trying to predict which 3 car manufacturers will be winner out of 1000s of car manufacturers in 1900, where almost all went bankrupt. LLMs is only a start. 

• Another big industry to be disrupted is biotech, in discovery of new treatments and drugs.   Counterintuitively, I think value of drug exclusivity rights will actually go down if innovation starts to happen at such rapid pace that a better drug is found by someone else that doesn't infringe an existing drug's exclusivity rights.
• Autonomous driving has been talked about a lot, but when level 5 really arrives, it will cause a sea change. 
• Entertainment is ripe for disruption with AI generated short-videos, movies and even games. 

Longer term, innovation will continue to happen.  Some assets will be bottlenecks and will continue to get valued higher, e.g. energy, and even commodities that have a monopoly controlling the limited supply, e.g. oil. 

 

The demand for these commodities could rise to unthinkable levels in terms of multiples of today as commodities will still be needed by the remaining 99% to increase their standard of living to match top 1% of today, when labor is no longer bottleneck. 

 

But, the value will only accrue to those commodities that have a monopoly over controlling supply. 

 

 

Edited March 28 by LearningMachine

[sholland]

On 3/26/2024 at 12:05 PM, ValueArb said:

2) Computer calculations will always get cheaper and use less energy per calculation over time. This is a trend that hasn't changed since before the invention of integrated circuits.

I believe that point #2 is irrelevant to the discussion because of Jevon’s paradox. 
 

https://en.wikipedia.org/wiki/Jevons_paradox

[bargainman]

On 3/28/2024 at 10:46 AM, LearningMachine said:

 

Next huge value-add won't be in increasing relevance of ads, but it will be in another scenario.  One of those will be to reduce # of hours of high-hourly rate knowledge workers.  If you can save 50% of hours of 10M workers getting paid $100K per year, you have created $500B in value, and that is per year.  At 10-20X multiple, that is worth $5-10 Trillion.   To enable this scenario, you need access to data that those knowledge workers are consuming and producing already.  You can deduce from there, who has access to that data and who already owns the user experience that these knowledge workers have been trained on. 

 

Already happening to some degree with coding assistants like GitHub copilot and others.  MS Office copilot possibly as well..

 

 

On 3/28/2024 at 10:46 AM, LearningMachine said:

 

Longer term, who all will benefit is a little like trying to predict which 3 car manufacturers will be winner out of 1000s of car manufacturers in 1900, where almost all went bankrupt. LLMs is only a start. 

• Another big industry to be disrupted is biotech, in discovery of new treatments and drugs.   Counterintuitively, I think value of drug exclusivity rights will actually go down if innovation starts to happen at such rapid pace that a better drug is found by someone else that doesn't infringe an existing drug's exclusivity rights.

 

Question as it is with LLMs is how do you evaluate?  testing is generally still done with human trials which are slow and very expensive. It will be a while before it moves to pure simulation if ever.

 

On 3/28/2024 at 10:46 AM, LearningMachine said:

Longer term, innovation will continue to happen.  Some assets will be bottlenecks and will continue to get valued higher, e.g. energy, and even commodities that have a monopoly controlling the limited supply, e.g. oil. 

 

I wonder about this. If what Tony Seba projects comes to pass, I wonder if oil will completely collapse along with the dairy industry among others.

 

 

 

[LearningMachine]

6 hours ago, bargainman said:

Already happening to some degree with coding assistants like GitHub copilot and others.  MS Office copilot possibly as well..

@bargainman, agreed.  That is exactly what I was implying.

 

 

6 hours ago, bargainman said:

Question as it is with LLMs is how do you evaluate?  testing is generally still done with human trials which are slow and very expensive. It will be a while before it moves to pure simulation if ever.

Yes, testing still needs human trial, but effectively "duplicating" carbon-based human research-scientists onto silicon-based intelligence will make it way more efficient to find new drugs, and simulations should help pick those drugs that will pass human trials so that more of human trials would result in success.  Also, a competitor doesn't have to wait until another competitor's human trial is done to start their trial if they have truly found something better within next 6 months because of increased productivity.  So, if both competitors pass human trial, first one gets exclusivity only for 6 months.  Also, it is not an all or nothing benefit.  I think we should be able to see some incremental improvement in this direction over time. 

 

 

6 hours ago, bargainman said:

I wonder about this. If what Tony Seba projects comes to pass, I wonder if oil will completely collapse along with the dairy industry among others.

Thanks for sharing. You will get lots of heads up on oil.  If 100% of 15M car sales in the US were electric today, it will take 18-19 years to replace all of 283M cars in the US on the road today.  Even if Level 5 automation came today, it will take time for transportation as a service to take off fully.  I understand it has started in SF and Phoenix, but we are no-where near level 5 automation yet.  

 

Also, 99% of the population in the world wants to live like the top 1%, i.e. have big mansions with great oil-based flooring, walls, roofs, carpets, oil based furniture, materials mined using oil, heated homes, heated water, cooking gas, etc.  They also want asphalt paved roads, and want those roads to be cleaned using oil-based machinery.  Even if all cars were electric today, to give bottom 99% of the population same life as that top 1% have after labor is not bottleneck, you need multiple times more oil just for petrochemicals. 

 

Vicki Hollub claims oil supply will go down before oil demand would.  I think what she means is that you will get so much heads up on demand coming down that oil companies will stop investing more capex to keep production flat, and production will start falling, while leaving more cash for shareholders in the end game - cash that would have otherwise gone into capex.

Edited April 9 by LearningMachine