Microsoft Unveils A New AI Inference Accelerator Chip, Maia 200

Microsoft’s new Maia 200 inference accelerator chip enters this overheated market with a new chip that aims to cut the price to serve AI responses.
NextBigFuture

Distributed AI Inference Will Capture Most of the LLM Value

Using the AIs will be way more valuable than AI training. AI training – feed large amounts of data into a learning algorithm to produce a model that can make predictions. AI Training is how we make ...
The Next Platform

SoC-Driven Inference Datacenters Becoming New Reality

The mighty SoC is coming for the datacenter with inference as a prime target, especially given cost and power limitations. With multiple form factors stretching from edge to server, any company that ...
Semiconductor Engineering

AI Inference Memory System Tradeoffs

When companies describe their AI inference chip they typically give TOPS but don’t talk about their memory system, which is equally important. What is TOPS? It means Trillions or Tera Operations per ...