AI memory: SRAM, DRAM, HBM & NAND →
Compare the technologies, their suppliers and the business indicators to track.
AI glossary
Explore a category, or search for a term or company.
Topic guides
Chip design & manufacturing
ASIC · Foundry · EUV · Advanced packaging · CoWoS · Yield
ASIC · Application-specific integrated circuit
A chip designed for a particular application.
Technical distinctions and research indicators
Specialization can improve efficiency but reduces flexibility; an ASIC is not necessarily an AI chip.
Research question: Check customer concentration, design wins and production timing.
Definition source · Intel ↗Foundry · Contract chip manufacturing
Manufactures chips from customers’ designs.
Technical distinctions and research indicators
Chip design and manufacturing are different businesses; some firms do both.
Research question: Compare utilization, technology mix and customer commitments.
Definition source · TSMC ↗EUV · Extreme ultraviolet lithography
Uses very short-wavelength light to print fine chip patterns.
Technical distinctions and research indicators
Lithography is one manufacturing step; equipment wavelength is not the chip’s marketed node size.
Research question: Track tool deliveries, customer spending and production readiness.
Definition source · ASML ↗Advanced packaging
Connects multiple chips or memory dies closely within a package.
Technical distinctions and research indicators
Different packaging approaches solve different connection, heat and cost problems.
Research question: Check qualified capacity, yield and customer-specific demand.
Definition source · TSMC ↗CoWoS · Chip-on-Wafer-on-Substrate
TSMC’s packaging platform connecting logic chips and high-bandwidth memory.
Technical distinctions and research indicators
CoWoS is a TSMC brand, not the name for all advanced packaging.
Research question: Separate announced expansion from usable, qualified output.
Definition source · TSMC ↗Yield · Manufacturing yield
The share of manufactured chips that meet requirements.
Technical distinctions and research indicators
More wafer capacity does not guarantee more good chips.
Research question: Look for good output and unit costs, not only wafer starts.
Definition source · ASML ↗Compute & storage
Compute · FLOPS · DRAM · HBM · SRAM · DDR5 · NAND · SSD · GPU · CPU
Compute · Computing capacity
The ability of computers to carry out calculations. For AI, it powers both learning from data (training) and generating answers (inference).
Technical distinctions and research indicators
A GPU is a computing chip; HBM feeds it data; a data center houses the systems. More GPUs do not guarantee proportionally faster work: memory, networking and software also matter.
Research question: Distinguish chip sales from rented compute services. Track utilization, customer revenue and costs including electricity, depreciation and financing; announced capacity alone does not establish profit.
Definition source · IBM ↗FLOPS · Floating-point operations per second
A measure of calculation rate. 1 TFLOPS is one trillion operations per second; 1 PFLOPS equals 1,000 TFLOPS; 1 EFLOPS equals 1,000 PFLOPS.
Technical distinctions and research indicators
Peak specifications are not achieved application performance. Compare the same numerical precision (such as FP16 or FP32), sparsity assumptions and workload. FLOPS is a rate; FLOPs can refer to the total operations in a task.
Research question: Compare measured workload performance and cost, not just the largest advertised number. Data-center megawatts measure power, not FLOPS; there is no universal conversion.
Definition source · IBM ↗DRAM · Dynamic random-access memory
Working memory that temporarily holds data while a computer runs.
Technical distinctions and research indicators
Needs refreshing and power to retain data. HBM is a type of DRAM.
Research question: Compare selling prices, inventories and product mix.
Definition source · IBM ↗HBM · High-bandwidth memory
Stacked DRAM built to move large amounts of data to an accelerator.
Technical distinctions and research indicators
It is memory, not a processor or long-term storage. Bandwidth and capacity are different measures.
Research question: Look for customer qualification, usable output and margins—not just announced capacity.
Definition source · Micron ↗SRAM · Static random-access memory
Fast working memory often used in small caches close to a processor.
Technical distinctions and research indicators
Unlike DRAM, it needs no periodic refresh, but still loses data without power.
Research question: Check the chip architecture; more SRAM is not automatically more revenue for a listed memory supplier.
Definition source · IBM ↗DDR5 · Fifth-generation DDR memory
A DRAM generation commonly used for computer and server main memory.
Technical distinctions and research indicators
DDR5 and HBM serve different memory arrangements; one does not simply replace the other.
Research question: Separate product generations when comparing supply and prices.
Definition source · IBM ↗NAND · Flash memory
Storage chips that keep data when power is off.
Technical distinctions and research indicators
NAND is a component; an SSD is a complete storage device that typically uses NAND.
Research question: Track price cycles, inventory and cost per bit.
Definition source · IBM ↗SSD · Solid-state drive
A storage device, usually combining NAND chips with a controller.
Technical distinctions and research indicators
Data-center drives have different endurance and reliability requirements from consumer drives.
Research question: Check enterprise sales mix, customer qualification and margins.
Definition source · IBM ↗GPU · Graphics processing unit
A processor that runs many similar calculations in parallel.
Technical distinctions and research indicators
Useful for AI matrix calculations; memory and networking can still limit speed.
Research question: Compare usable performance, software support and cost per workload.
Definition source · Intel ↗CPU · Central processing unit
The general-purpose processor coordinating a computer’s work.
Technical distinctions and research indicators
A GPU accelerates selected tasks; it does not eliminate the CPU.
Research question: Check server demand and the number and type of processors per system.
Definition source · Intel ↗Networking
Bandwidth · Ethernet · InfiniBand · Optical transceiver · Scale-up · Scale-out
Bandwidth
How much data a connection can move per unit of time.
Technical distinctions and research indicators
Like road capacity, not the time one car takes to arrive. Memory and network bandwidth are separate.
Research question: Identify the actual constraint before assuming a faster component improves the whole system.
Definition source · Micron ↗Ethernet
A widely used networking technology linking computers and switches.
Technical distinctions and research indicators
AI Ethernet needs system-level congestion control; headline link speed alone is insufficient.
Research question: Compare complete network performance and adoption, not port speeds alone.
Definition source · NVIDIA ↗InfiniBand
A networking technology used in tightly coordinated computing clusters.
Technical distinctions and research indicators
It is a different ecosystem from Ethernet, with compatibility and operating trade-offs.
Research question: Watch customer architecture choices and switch/adaptor deployments.
Definition source · NVIDIA ↗Optical transceiver
Converts electrical data signals into light and back for fiber links.
Technical distinctions and research indicators
Reach, power use and compatibility matter alongside speed.
Research question: Check qualification, product-generation mix and margins.
Definition source · NVIDIA ↗Scale-up
Connects accelerators tightly so they work as a larger computing unit.
Technical distinctions and research indicators
In AI systems this often means a fast accelerator fabric, rather than simply buying a bigger server.
Research question: Watch fabric size, memory access and actual application scaling.
Definition source · NVIDIA ↗Scale-out
Adds networked compute nodes to increase cluster capacity.
Technical distinctions and research indicators
More nodes increase coordination and networking needs.
Research question: Check throughput gained per added node and network cost.
Definition source · NVIDIA ↗Power & data centers
Rack power density · Liquid cooling · CDU · PUE · Grid connection · MW vs MWh
Rack power density
The power needed by equipment in a rack, commonly expressed in kW per rack.
Technical distinctions and research indicators
Design maximum and measured average load are different.
Research question: Compare cooling and electrical design with actual deployment load.
Definition source · Vertiv ↗Liquid cooling
Uses liquid to carry heat away from equipment.
Technical distinctions and research indicators
Direct-to-chip and immersion are different approaches; air cooling may still be needed.
Research question: Check customer adoption, retrofit costs and service requirements.
Definition source · Vertiv ↗CDU · Coolant distribution unit
Manages coolant flow and heat exchange for liquid-cooled equipment.
Technical distinctions and research indicators
A CDU is not automatically a refrigeration chiller.
Research question: Check system compatibility, deployment volumes and maintenance costs.
Definition source · Vertiv ↗PUE · Power usage effectiveness
Total facility energy divided by IT equipment energy over the same period.
Technical distinctions and research indicators
Closer to 1 means less overhead, not necessarily more profitable or cleaner computing.
Research question: Compare measurement boundaries, climate and utilization.
Definition source · The Green Grid ↗Grid connection
Connecting a facility to the electricity system with sufficient usable power.
Technical distinctions and research indicators
An agreement or requested capacity is not proof that power is available today.
Research question: Track approvals, substations and confirmed energization milestones.
Definition source · US DOE ↗MW vs MWh · Power vs energy
MW measures power; MWh measures energy used or delivered over time.
Technical distinctions and research indicators
A constant 1 MW for one hour equals 1 MWh. IT load and total site power are also different.
Research question: Check units and whether a number is planned, operating or measured.
Definition source · US EIA ↗Models & cloud
Training · Inference · Latency · Token · Context window · Throughput · Neocloud
Training
Learning patterns from data by adjusting a model’s parameters.
Technical distinctions and research indicators
Training builds the model; inference uses it.
Research question: Ask whether spending is a one-off build or supports recurring workloads.
Definition source · NVIDIA ↗Inference
Using a trained model to produce an answer, prediction or other output.
Technical distinctions and research indicators
More requests do not necessarily mean more profit: serving costs matter.
Research question: Compare revenue per request with compute cost and utilization.
Definition source · NVIDIA ↗Latency
How long an operation takes, such as the wait for a model’s first response.
Technical distinctions and research indicators
Low latency means a shorter wait; high throughput means more work completed per second.
Research question: Compare the same workload and service quality, not isolated benchmark claims.
Definition source · NVIDIA ↗Token
A unit of text processed by a language model, often a word or part of a word.
Technical distinctions and research indicators
A token is not always one character or one word; tokenizers differ.
Research question: Compare input/output pricing and tokens per completed task.
Definition source · IBM ↗Context window
The amount of information a model can consider in one interaction.
Technical distinctions and research indicators
A large window is not permanent memory or proof of accurate reasoning.
Research question: Compare accuracy, latency and cost at realistic context lengths.
Definition source · IBM ↗Throughput
The amount of work a system completes per unit of time.
Technical distinctions and research indicators
Tokens per second across many users differs from one user’s response speed.
Research question: Hold model, workload and service-quality targets constant.
Definition source · IBM Research ↗Neocloud · AI-focused cloud provider
A market term for cloud providers focused on AI compute.
Technical distinctions and research indicators
Not a formal accounting category; owning facilities and renting them have different economics.
Research question: Track utilization, customer concentration, funding and power delivery.
Definition source · Forrester ↗Investment research
CapEx · Gross margin · FCF · Backlog · RPO
CapEx · Capital expenditure
Spending to acquire or improve long-lived assets such as servers and buildings.
Technical distinctions and research indicators
Cash purchases and leased assets may be presented differently; spending is not immediate profit.
Research question: Compare spending with cash generation, asset life and revenue delivery.
Definition source · SEC ↗Gross margin
Revenue minus cost of revenue, divided by revenue.
Technical distinctions and research indicators
Gross profit is an amount; gross margin is a percentage before other operating costs.
Research question: Compare consistent accounting and changes in product mix.
Definition source · SEC ↗FCF · Free cash flow
Commonly operating cash flow minus capital expenditure.
Technical distinctions and research indicators
A non-GAAP measure with no uniform definition; read the company’s reconciliation.
Research question: Check lease payments, working capital and recurring investment needs.
Definition source · SEC filing correspondence ↗Backlog · Order backlog
A company’s measure of work or orders still to be fulfilled.
Technical distinctions and research indicators
Definitions vary and may include options; backlog is not booked revenue or guaranteed cash.
Research question: Check cancellation terms, delivery schedule and conversion to cash.
Definition source · Booz Allen Hamilton · SEC filing ↗RPO · Remaining performance obligations
Contract revenue allocated to obligations not yet satisfied.
Technical distinctions and research indicators
Not the same as all future sales or backlog; disclosures may have accounting exclusions.
Research question: Check expected recognition timing, contract terms and customer concentration.
Definition source · Microsoft Investor Relations ↗