AI and Machine Learning

Training and inference · GPUs, ASICs, LPUs, NPUs · the buyers

Inference now takes about two thirds of AI compute, and the market for chips built for it is passing $50 billion. Training still runs on TSMC wafers, CoWoS and HBM, but inference is opening other paths: NVIDIA's Groq 3 LPU is made by Samsung and uses no HBM at all.

Share of AI compute used for inference rather than training, Deloitte: about one third in 2023, about half in 2025 and about two thirds projected for 2026. The market for inference-optimized chips grows from over 20 billion dollars in 2025 to over 50 billion in 2026. SHARE OF AI COMPUTE: INFERENCE VS TRAINING (DELOITTE) 2023 Inference, about one third Training 2025 Inference, about half 2026F Inference, about two thirds INFERENCE-OPTIMIZED CHIP MARKET, $ BILLION (DELOITTE) 2025 over $20B 2026F over $50B Inference chips no longer all follow the training path of TSMC wafers, CoWoS packaging and HBM: NVIDIA's Groq 3 LPU is made by Samsung on 4nm and uses on-chip SRAM, not HBM. DELOITTE TMT PREDICTIONS 2026. SHARES ARE APPROXIMATE.
Inference share of compute
About two thirds (2026)
Inference chip market
Over $50B
Big four capex 2026
Over $700B
NVIDIA AI chip outlook
$1T through 2027
Groq 3 LPU
Samsung 4nm
Cambricon H1 revenue
+108%

Workload

Inference Now Takes Most AI Compute

Training versus inference share of AI compute

Training builds a model once; inference runs it every time someone uses it. Deloitte estimates inference used about one third of AI compute in 2023, about half in 2025 and about two thirds in 2026, and puts the 2026 market for chips designed for inference at over $50 billion, up from over $20 billion in 2025. At GTC in March 2026 NVIDIA put its AI chip revenue opportunity at $1 trillion or more through 2027, double its earlier $500 billion projection through 2026.

Inference changes what a chip needs. Training needs the most memory bandwidth and the biggest packages, which is why it runs on GPUs with HBM on CoWoS. Inference values cost per answer and latency, which opens room for custom ASICs, SRAM-based chips and, at the edge, small NPUs.

Supply

Inference Is Pulling Some Chips Off the GPU Path

Wafer, package and memory source for each AI chip family

Every leading training chip runs through TSMC wafers, CoWoS in Taiwan and HBM from Korea; the supplier rings trace that path for ten chips. Inference chips are starting to leave it. In December 2025 NVIDIA paid $20 billion to license Groq's technology, and its Groq 3 LPX, added to Vera Rubin servers in the third quarter of 2026, is made by Samsung Foundry on 4nm at Pyeongtaek. Samsung began production about five months before NVIDIA announced full volume in August 2026. The chip keeps model weights in on-chip SRAM and needs neither HBM nor CoWoS.

Five paths from an AI chip design to silicon. Training GPUs and custom ASICs both run through TSMC wafers in Taiwan, CoWoS packaging in Taiwan and HBM from Korea, with some NVIDIA logic dies now made in Arizona. NVIDIA's Groq 3 LPU for inference is made by Samsung on 4nm in Pyeongtaek and uses on-chip SRAM instead of HBM. Edge NPUs sit inside phone and PC chips made by TSMC with LPDDR memory. China's Ascend and Cambricon chips run on SMIC 7nm-class wafers with stockpiled or domestic memory. FIVE PATHS FROM AI CHIP DESIGN TO SILICON LOGIC WAFER PACKAGE MEMORY Training GPU NVIDIA RUBIN, AMD MI455X TSMC N3P TAIWAN CoWoS-L TAIWAN HBM4 SK HYNIX, SAMSUNG, MICRON Custom ASIC GOOGLE TPU, AWS TRAINIUM3 MAIA 200, META MTIA TSMC N3 class TAIWAN CoWoS TAIWAN HBM3E and HBM4 SRAM inference LPU NVIDIA GROQ 3 LPX (LP30) Samsung 4nm PYEONGTAEK Standard package On-chip SRAM NO HBM Edge NPU APPLE A19 PRO, QUALCOMM MEDIATEK TSMC N3 and N2 TAIWAN InFO and flip-chip LPDDR5X China domestic HUAWEI ASCEND, CAMBRICON SMIC 7nm class CHINA Domestic packaging Stockpiled HBM2E CXMT RIGHT-HAND BAR: US STATUS OF THE LOGIC WAFER. NVIDIA BLACKWELL LOGIC IS ALSO MADE AT TSMC ARIZONA.
Each AI chip family and where its wafer, package and memory come from. The bar at right shows the US status of the logic wafer.
  • Some US volume
  • Made abroad

The training path is covered on AI Accelerators, GPUs and HBM; the custom chips on ASICs.

Demand

Four US Cloud Companies Fund Most of It

2026 capital spending guidance and own chips

AI chip demand comes from a short list of buyers. By late July 2026 the four largest US cloud companies had guided more than $700 billion of 2026 capital spending, about three quarters of it for AI infrastructure. Each also designs its own chip, which is how custom ASICs reach volume.

2026 capital spending guidance of the four largest US cloud companies (as of late July 2026)
Company2026 guidanceMain AI chips boughtOwn chip
Alphabet$195B to $205B (raised July 22)NVIDIA GPUs, Google TPUTPU Ironwood
AmazonAbout $200BNVIDIA GPUs, TrainiumTrainium3
MicrosoftAbout $190BNVIDIA GPUs, AMD GPUs, MaiaMaia 200
Meta$125B to $145BNVIDIA GPUs, AMD GPUs, MTIAMTIA
TotalMore than $700B; about 75% for AI infrastructure

Gartner estimates the AI data center ecosystem takes 36.5% of all semiconductor revenue in 2026; the effect on the rest of the chip market is on the Sectors hub.

Mix

Custom ASICs Are Expected to Pass GPUs in Units

The unit split between ASICs and GPUs

TrendForce expected AI server shipments to grow more than 20% in 2026 and custom ASIC shipments to surpass GPU shipments as US cloud companies and Chinese firms scale their own chips, with NVIDIA's share of AI chips easing from about 70% in 2025. GPUs keep most of the value: each ASIC is typically cheaper, and NVIDIA's own inference parts (Rubin CPX, Groq 3 LPX) compete for the same racks.

The design work for most ASICs runs through Broadcom, Marvell, Alchip, GUC and MediaTek, and almost all of the wafers through TSMC (see ASICs). Moving from GPUs to ASICs changes the chip designer, not the factory.

China

China Builds a Separate AI Stack on SMIC

China's AI chip makers and their wafer source

Export controls keep NVIDIA's leading chips and TSMC's leading nodes out of China, so Chinese AI chips run on SMIC's 7nm-class process, which uses older DUV scanners and multiple patterning. Cambricon's revenue doubled in the first half of 2026, but its shipments depend on SMIC yields that were reported at about 20% in April 2026. Memory is the tighter limit: Chinese accelerators use stockpiled HBM2E and early CXMT parts.

China's AI chip makers, 2026
CompanyChipWafer sourceLatest figure
Huawei HiSiliconAscend 910C, 950 seriesSMIC 7nm classLargest Chinese supplier; see the Huawei spotlight
CambriconSiyuan seriesSMIC 7nm classRevenue 5.996B yuan in the first half of 2026, up 108%; net profit 2.311B yuan
Hygon, Moore Threads, MetaX, BirenGPUs and acceleratorsSMIC and stockpiled TSMC wafers (reported)Smaller volumes

The split between the two stacks is traced on China Bifurcation.

Edge

Edge Inference Stays Small Next to the Data Center

Where inference runs outside the data center

Phones and PCs ship with NPUs, but Deloitte expects almost all AI compute in 2026 to run in data centers or enterprise servers: on-device NPUs handle short, simple tasks. It puts 2026 AI chip spend in robots, drones and autonomous vehicles at under $5 billion. Edge AI chips are made on the same TSMC nodes as phone processors, so their supply rides on the consumer chain, not the AI one.

Edge chips are covered on AI Inference and Edge Compute SoCs, cars on Automotive and robots on Robotics and IoT.

Record

Page Record

Covers
AI chip demand by workload, inference chip market, the wafer, package and memory path for each AI chip family, cloud capital spending, the ASIC and GPU split, China's AI chips and edge inference. Folded in: AI Inference Semiconductors: Edge to Cloud.
Sources
Deloitte TMT Predictions 2026, compute power for AI; Bits&Chips via Techleap on NVIDIA's GTC outlook and the Groq deal (March 2026); SamMobile on Groq 3 LPX production at Samsung (August 25, 2026); Cryptobriefing on 2026 capital spending guidance (July 24, 2026); TrendForce on AI server shipments (October 31, 2025, via EMSNow); Gartner (August 25, 2026); Implicator on Cambricon first-half results (August 7, 2026 filing) and Tom's Hardware on SMIC yields (reported). Wafer and memory sources in the paths diagram come from the SemiconductorX chip supplier rings.
Last verified
October 2026