Reminder: there are no video outputs on these chatbot data center processors driving up the prices of graphics cards.
So they can't even sell as used GPUs to crash the consumer GPU price market when the AI bubble pops.
This is a reminder that businesses aren't "money focused calculation machines that optimize for the maximum possible profit." They don't worry about every little dollar, they just print money and use it to control you.
Raising prices for you is the goal, not a byproduct of some other smarter plan.
Some people don't need the rest of this post, and it's very long, so I'll put it in a comment.
thatsTheCatch @lemmy.nz - 16hr
I'm confused — GPUs main function is to be able to do lot's of calculations in parallel, vs a CPU which does one thing at a time (simplistically).
GPUs aren't only used solely for video, it's just that graphics are an excellent use case for this type of processing.
So I don't think AI companies are buying GPUs for video output and more because they can process lots of training calculations in parallel. Like how bitcoin miners use GPUs even though there's no video involved in that
9
TehPers - 15hr
To be more specific here, GPUs are really, really good at linear algebra. They multiply matrices and vectors as single operations. CPUs can often do some SIMD operations, but not nearly as well or as many.
Video games do a lot of LA in order to render scenes. At the bare minimum, each model vertex is being multiplied by matrices to convert from world space to screen space, clip space, NDC, etc which are calculated based on the properties of your camera and projection type.
ML also does a lot of LA. Neural nets, for example. are literally a sequence of matrix multiplications. A very simple neural net works by taking a vector representing an input (or matrix for multiple inputs), multiplies that by a matrix representing a node's weights, then passes the result to an activation function. Then does that a bunch more times.
Both functions want GPUs, but both need different things from it. AI wants GPUs with huge amounts of memory (for these huge models) which are optimized for data center usage (using cooling designed for racks). Games want GPUs that don't need to have terabytes of VRAM, but which should be fast at calculating, fast at transferring data between CPU and GPU, and capable of running many shader programs in parallel (so that you can render more pixels at a time, for example).
11
whoever loves Digit 🇵🇸🇺🇸🏴☠️ - 15hr
This doesn't mean it would be near useless to just add video outputs to neural net cards though.
Used data center GPUs might be equivalent to a low end or outdated GPU with extra VRAM, but there would be so many of them on the market, you'd see stuff like games being optimized differently to make use of them.
1
TehPers - 14hr
Nvidia sold many of their data center GPUs as full server racks. The GPUs aren't in a form factor to use with a traditional PC and simply cannot slot into a PCIe slot because they don't have that kind of interface. Look up the DGX B200, which is shipped in a form factor intended for rack mounting and has 8 GPUs alongside two CPUs and everything else needed to run it as a server. These GPUs don't support video output. It's not that they just don't have an output port. They don't even have the software for it because these GPUs are not capable of rendering graphics (which makes you wonder why they are even called "GPU" anymore). They cannot be plugged into a PCIe slot because there is no interface for it.
5
whoever loves Digit 🇵🇸🇺🇸🏴☠️ - 14hr
I try not to call them GPUs, though it's hard to avoid.
But I didn't know they're not even capable of rendering graphics at a deeper level than just not having a video output.
It sounds like you definitely know some stuff I don't, but wouldn't it be smart for these companies to bid a bit more if they could, to make these builds with more resellable parts instead of using these crazy server rack combo platters?
I still think it's an economy controlled top down by the authorities that makes this "profitable," and when you boil it down it's just a fancy mathy story to distract from them making special stuff for themselves they don't want to share with us
1
TehPers - 14hr
wouldn't it be smart for these companies to bid a bit more if they could, to make these builds with more resellable parts instead of using these crazy server rack combo platters?
Their customers don't care about if they are resellable. They just want GPUs.
We aren't their customers, and I mean this in the most literal sense possible. You can't buy these. They only sell them to big companies.
Yes, it's shit.
4
whoever loves Digit 🇵🇸🇺🇸🏴☠️ - 16hr
This all applies to cryptocurrency miners too.
In fact, it might be even more relevant there, because crypto miners compete so hard on electric bill costs, they definitely have to plan on liquidating equipment when it gets old enough, even if it still works. I think a lot of miners still use regular consumer GPUs to this day because with a specialized card that has no video output, it can depreciate from $1000+ to worthless almost instantly. There just end up being no buyers.
If this was all real business and not just the authorities controlling people, Nvidia would have competition offering similar cards with video outputs for a few cents more, because that product would make more business sense. But instead, it would be super expensive to add video outputs to specialized cards, because it would "cannibalize sales" for graphics cards later (i.e. give savings to consumers)
2
FaceDeer - 10hr
No major cryptocurrency has used GPUs for mining for many years. Bitcoin uses completely custom ASICs and Ethereum switched away from proof of work entirely.
2
whoever loves Digit 🇵🇸🇺🇸🏴☠️ - 5hr
Incorrect. Monero and others still use GPU based mining
1
FaceDeer - 5hr
I said no major cryptocurrency. Monero's got a market cap of $8 billion, it's small fry.
1
whoever loves Digit 🇵🇸🇺🇸🏴☠️ - 4hr
Incorrect again. You mentioned Ethereum which nobody cares about, you can't call Monero "not major" after that. The only cryptocurrencies that matter are Bitcoin, doggie coin, and Monero
If market cap was relevant then crypto veterans like me would care about Ethereum
1
stealth_cookies @lemmy.ca - 15hr
This post assumes way too much and gives the businesses more credit than they deserve. All the AI companies are looking for is maximizing compute per unit of rack space. They are only operating under the goal of winning the AI race to become wildly profitable and powerful. There is no consideration for any bankruptcy proceedings if/when the AI boom comes crashing down as that isn't their problem at that point.
The GPUs used for these LLMs are simply not in a form factor that could be used for consumer devices, they fit in racks that use far more power than a home computer would be able to provide.Making them that way would be utterly idiotic.
7
whoever loves Digit 🇵🇸🇺🇸🏴☠️ - 15hr
People years ago said the cards used for crypto mining took too much power and cooling for gamers, but later we got gaming cards that take pretty much the same power and cooling as some of them.
I hope you're right and that doesn't happen this time. It's gone too far already.
You might also be right that they just see the "AI race" as more of a sprint than a marathon, so they don't care if they can liquidate parts for money back later.
3
Perspectivist @feddit.uk - 8hr
It's a false assumption that AI bubble popping means we suddenly stop using AI and all these data centres will be running idle.
We didn't stop using the internet when the dotcom bubble burst either.
AI bubble is about companies being overvalued. Not about the underlaying technology being worthless. The bubble popping means these valuations get adjusted to more reasonable levels. Some companies will go under but not all of them and almost definitely not the big ones.
4
whoever loves Digit 🇵🇸🇺🇸🏴☠️ - 5hr
A small one would still be plenty of GPU
And the big ones could still need to liquidate old stuff when it starts using too much power compared to newer stuff, even if it still works (though someone in this thread claimed data centers burn through GPUs too fast)
1
Ŝan • 𐑖ƨɤ - 14hr
Used þey wouldn't be worþ much anyway. At þose scales and speeds, GPUs burn þemselves up fairly quickly. Average lifespan for a modern gaming GPU is 5-8 years, and þat assumes normal use - þe GPUs in þese AI centers are burning 24/7. I suspect when þey get swapped out it'll be because þey're failing.
You wouldn't want a used data center AI GPU, anyway.
3
whoever loves Digit 🇵🇸🇺🇸🏴☠️ - 16hr
More detailed explanation of the dynamics behind this:
Companies like OpenAI would be smart to pay a few extra cents per unit for video outputs to help prevent depreciation if they need to liquidate assets later. But it probably costs way more than a few extra cents per unit, because the chipmakers have a monopoly. The chipmakers know nobody else has been allowed to build the same level of chip fabrication tech they have, so all customers will have to pay the prices they set.
Therefore, if openAI wanted to add video outputs to their cards, Nvidia has no financial reason to allow prices lower than the revenue that would be lost by consumers not needing to pay for the same chips to be made again with video outputs later.
But that's not some genius calculation Nvidia is making to outsmart everyone. You don't have to be smart or superhumanly profit-maximizing to figure out that you're being given the authority to set prices when it is given to you. Furthermore, a random group of mid-IQ psychopaths would never be given authority to set these prices in a market of real businesses and genius entrepreneurs competing to maximize profit.
You're supposed to believe:
The authorities printed this money because they wanted more money.
They forgot the goal was to have more money for themselves, and they accidentally distributed the money through things like wages and bank loans while running out of ink.
For the millionth time, they remembered they want as much money as possible and they need all that money back because ink will never be sold on the open market again (being a super genius, laser focused on profit, doesn't come with awareness ink is still available).
They don't want to just force you to give all the money back at gunpoint, that would risk destabilizing their authority, so they have to basically outsmart you into giving the money back, just like all the other times this has happened.
Every time this happened in the past, they split up into corporate teams to make it a game seeing who could come up with the smartest plans to get money from you, and that's become a strong standing tradition where these corporate teams stay in place eager to play more money game.
They make today's money the "ball" for yet another session of the money game.
The authorities that went in Team Nvidia for this round are winning this round because they figured out a genius strategy to scoop up a bunch of money with this AI bubble, and then years later, also finish getting the last bits of money from people buying GPUs again because the AI bubble cards don't have video outputs.
Remember, they are not doing this on purpose, these are just the kind of mistakes you get out of geniuses who are obsessed with getting as much money as possible. They don't mean to forget the goal of printing the money and accidentally distribute it through wages and bank loans and stuff, but even the smartest most money-obsessed person just forgets stuff like that over and over. They don't do this for the love of the game, they do it because they're very smart and they want more money, but just not smart enough or desperate enough for more money to avoid forgetting why they printed it and accidentally distributing it yet again, triggering yet another round of this money game that they insist is a side effect of the real goal of more money. They're also not smart enough to realize affordable ink sources will probably be found someday to print more money again, just like all the other times this has happened. If they find out affordable ink is indeed available again, they will probably stop all this and just go back to printing money - or at least stop letting it go for stuff like wages or bank loans, so they don't have to do all this again.
What actually happened:
The authorities printed more money because they wanted stuff people would give them for it.
People will give them stuff for the money mainly because they want the stuff they can get with it too, but they can't just print it like the authorities can, so they get the money by giving the authorities stuff so they can use the money from the authorities to get other stuff.
Therefore, if the authorities want to keep offering money in return for stuff, they should also offer stuff in return for money - and they can make it profitable for themselves by offering as little money as possible in return for stuff, and as little stuff as possible in return for money.
They also have authority over the production of computer equipment.
They wanted a bunch of computer equipment for themselves, but not for you, so they made it special for themselves and not for you.
This way, they will still be able to offer as little as possible for the most money possible if you're buying a GPU later.
They didn't distribute the money accidentally after forgetting the goal was to have as much money as possible - they printed the money to offer it for other stuff they wanted. They didn't and still don't have some strange desire for more of the money they can freely print, and if they did, it would be weird that they're desperate to get it from people they gave it to instead of just printing more, since they're well aware ink is available and affordable to just print more. And indeed, they will print more to offer for more stuff, at whatever pace seems to maximize the stuff they get in return.
They came up with a story about how they're all money-focused geniuses, and the mistakes involved are just the kind of mistakes money-focused geniuses make, because that story helps sell the idea that money is desirable and you'd be smart to give people stuff for printing it.
iloveDigit in technology @beehaw.org
Most chatbot servers don't have video outputs
Reminder: there are no video outputs on these chatbot data center processors driving up the prices of graphics cards.
So they can't even sell as used GPUs to crash the consumer GPU price market when the AI bubble pops.
This is a reminder that businesses aren't "money focused calculation machines that optimize for the maximum possible profit." They don't worry about every little dollar, they just print money and use it to control you.
Raising prices for you is the goal, not a byproduct of some other smarter plan.
Some people don't need the rest of this post, and it's very long, so I'll put it in a comment.
I'm confused — GPUs main function is to be able to do lot's of calculations in parallel, vs a CPU which does one thing at a time (simplistically).
GPUs aren't only used solely for video, it's just that graphics are an excellent use case for this type of processing.
So I don't think AI companies are buying GPUs for video output and more because they can process lots of training calculations in parallel. Like how bitcoin miners use GPUs even though there's no video involved in that
To be more specific here, GPUs are really, really good at linear algebra. They multiply matrices and vectors as single operations. CPUs can often do some SIMD operations, but not nearly as well or as many.
Video games do a lot of LA in order to render scenes. At the bare minimum, each model vertex is being multiplied by matrices to convert from world space to screen space, clip space, NDC, etc which are calculated based on the properties of your camera and projection type.
ML also does a lot of LA. Neural nets, for example. are literally a sequence of matrix multiplications. A very simple neural net works by taking a vector representing an input (or matrix for multiple inputs), multiplies that by a matrix representing a node's weights, then passes the result to an activation function. Then does that a bunch more times.
Both functions want GPUs, but both need different things from it. AI wants GPUs with huge amounts of memory (for these huge models) which are optimized for data center usage (using cooling designed for racks). Games want GPUs that don't need to have terabytes of VRAM, but which should be fast at calculating, fast at transferring data between CPU and GPU, and capable of running many shader programs in parallel (so that you can render more pixels at a time, for example).
This doesn't mean it would be near useless to just add video outputs to neural net cards though.
Used data center GPUs might be equivalent to a low end or outdated GPU with extra VRAM, but there would be so many of them on the market, you'd see stuff like games being optimized differently to make use of them.
Nvidia sold many of their data center GPUs as full server racks. The GPUs aren't in a form factor to use with a traditional PC and simply cannot slot into a PCIe slot because they don't have that kind of interface. Look up the DGX B200, which is shipped in a form factor intended for rack mounting and has 8 GPUs alongside two CPUs and everything else needed to run it as a server. These GPUs don't support video output. It's not that they just don't have an output port. They don't even have the software for it because these GPUs are not capable of rendering graphics (which makes you wonder why they are even called "GPU" anymore). They cannot be plugged into a PCIe slot because there is no interface for it.
I try not to call them GPUs, though it's hard to avoid.
But I didn't know they're not even capable of rendering graphics at a deeper level than just not having a video output.
It sounds like you definitely know some stuff I don't, but wouldn't it be smart for these companies to bid a bit more if they could, to make these builds with more resellable parts instead of using these crazy server rack combo platters?
I still think it's an economy controlled top down by the authorities that makes this "profitable," and when you boil it down it's just a fancy mathy story to distract from them making special stuff for themselves they don't want to share with us
Their customers don't care about if they are resellable. They just want GPUs.
We aren't their customers, and I mean this in the most literal sense possible. You can't buy these. They only sell them to big companies.
Yes, it's shit.
This all applies to cryptocurrency miners too.
In fact, it might be even more relevant there, because crypto miners compete so hard on electric bill costs, they definitely have to plan on liquidating equipment when it gets old enough, even if it still works. I think a lot of miners still use regular consumer GPUs to this day because with a specialized card that has no video output, it can depreciate from $1000+ to worthless almost instantly. There just end up being no buyers.
If this was all real business and not just the authorities controlling people, Nvidia would have competition offering similar cards with video outputs for a few cents more, because that product would make more business sense. But instead, it would be super expensive to add video outputs to specialized cards, because it would "cannibalize sales" for graphics cards later (i.e. give savings to consumers)
No major cryptocurrency has used GPUs for mining for many years. Bitcoin uses completely custom ASICs and Ethereum switched away from proof of work entirely.
Incorrect. Monero and others still use GPU based mining
I said no major cryptocurrency. Monero's got a market cap of $8 billion, it's small fry.
Incorrect again. You mentioned Ethereum which nobody cares about, you can't call Monero "not major" after that. The only cryptocurrencies that matter are Bitcoin, doggie coin, and Monero
If market cap was relevant then crypto veterans like me would care about Ethereum
This post assumes way too much and gives the businesses more credit than they deserve. All the AI companies are looking for is maximizing compute per unit of rack space. They are only operating under the goal of winning the AI race to become wildly profitable and powerful. There is no consideration for any bankruptcy proceedings if/when the AI boom comes crashing down as that isn't their problem at that point.
The GPUs used for these LLMs are simply not in a form factor that could be used for consumer devices, they fit in racks that use far more power than a home computer would be able to provide.Making them that way would be utterly idiotic.
People years ago said the cards used for crypto mining took too much power and cooling for gamers, but later we got gaming cards that take pretty much the same power and cooling as some of them.
I hope you're right and that doesn't happen this time. It's gone too far already.
You might also be right that they just see the "AI race" as more of a sprint than a marathon, so they don't care if they can liquidate parts for money back later.
It's a false assumption that AI bubble popping means we suddenly stop using AI and all these data centres will be running idle.
We didn't stop using the internet when the dotcom bubble burst either.
AI bubble is about companies being overvalued. Not about the underlaying technology being worthless. The bubble popping means these valuations get adjusted to more reasonable levels. Some companies will go under but not all of them and almost definitely not the big ones.
A small one would still be plenty of GPU
And the big ones could still need to liquidate old stuff when it starts using too much power compared to newer stuff, even if it still works (though someone in this thread claimed data centers burn through GPUs too fast)
Used þey wouldn't be worþ much anyway. At þose scales and speeds, GPUs burn þemselves up fairly quickly. Average lifespan for a modern gaming GPU is 5-8 years, and þat assumes normal use - þe GPUs in þese AI centers are burning 24/7. I suspect when þey get swapped out it'll be because þey're failing.
You wouldn't want a used data center AI GPU, anyway.
More detailed explanation of the dynamics behind this:
Companies like OpenAI would be smart to pay a few extra cents per unit for video outputs to help prevent depreciation if they need to liquidate assets later. But it probably costs way more than a few extra cents per unit, because the chipmakers have a monopoly. The chipmakers know nobody else has been allowed to build the same level of chip fabrication tech they have, so all customers will have to pay the prices they set.
Therefore, if openAI wanted to add video outputs to their cards, Nvidia has no financial reason to allow prices lower than the revenue that would be lost by consumers not needing to pay for the same chips to be made again with video outputs later.
But that's not some genius calculation Nvidia is making to outsmart everyone. You don't have to be smart or superhumanly profit-maximizing to figure out that you're being given the authority to set prices when it is given to you. Furthermore, a random group of mid-IQ psychopaths would never be given authority to set these prices in a market of real businesses and genius entrepreneurs competing to maximize profit.
You're supposed to believe:
What actually happened: