Skip to content(if available)orjump to list(if available)

How 'inference' is driving competition to Nvidia's AI chip dominance

jsemrau

I think the key takeaway quotes are these:

“The amount of inference compute needed is already 100x more” than it was when large language models started out, Huang said on last month’s earnings call. “And that’s just the beginning.”

The cost of serving up responses from LLMs has fallen rapidly over the past two years, driven by a combination of more powerful chips, more efficient AI systems and intense competition between AI developers such as Google, OpenAI and Anthropic.

null

[deleted]