Resources
AI hardware glossary
Plain definitions of the technical terms used across our product and solution pages. Written for procurement, finance and IT decision-makers as much as for the engineers who specify the system.
GPU + memory
The compute and memory architecture inside an AI server.
- GPU (Graphics Processing Unit)
- A processor with thousands of small cores built for the parallel matrix arithmetic behind neural network training and inference. We build with NVIDIA cards from the RTX PRO 4500 Blackwell up to DGX-class data centre GPUs.SeeAll systems
- HBM3e (High Bandwidth Memory 3 extended)
- Stacked DRAM mounted next to the GPU die, with several times the bandwidth of GDDR7. Used on NVIDIA H200, B200, B300, GB200 and GB300; the H100 used plain HBM3. Bandwidth, not capacity, is usually what limits the largest models.SeeH200 vs B200 vs B300
- GDDR7
- The seventh-generation graphics DDR standard, used on cards like the RTX PRO 4500 and RTX PRO 6000 Blackwell. Less bandwidth per card than HBM3e, and cheaper. Larger models are spread across several cards, as on the eight-card AI Server.SeeEMARQUE AI Server
- VRAM (Video RAM)
- The memory pool on a GPU. A model that does not fit in VRAM fails to load, or spills to the CPU and slows down. A 4-bit 70B model needs about 40 GB; at full precision, about 140 GB.
- PCIe Gen5
- The fifth-generation PCI Express bus, double Gen4's bandwidth at 64 GB/s per ×16 slot, which is what keeps a GPU fed from the CPU and storage.
- NVLink
- NVIDIA's interconnect between GPUs in one chassis, many times faster than PCIe Gen5. GPUs share memory coherently, so a model larger than one card's VRAM runs as though on a single device. DGX and HGX systems have it; PCIe-only RTX PRO 6000 Server Edition builds do not.SeeNVIDIA DGX B300EMARQUE AI HGX B300
- NVLink-C2C (NVLink Chip-to-Chip)
- NVIDIA's coherent CPU-to-GPU interconnect on Grace platforms: DGX Spark, DGX Station, GB200 and GB300. CPU and GPU share one memory address space at NVLink bandwidth, with no copy step across PCIe.SeeNVIDIA DGX SparkNVIDIA DGX Station GB300
- NVL72
- An NVIDIA rack-scale platform, used by GB300 NVL72 and Vera Rubin NVL72: 72 GPUs and 36 CPUs in one liquid-cooled cabinet on NVLink switching, so the rack appears to software as a single large GPU. We source them through NVIDIA's OEM partners and coordinate the install.SeeNVIDIA GB300 NVL72NVIDIA Vera Rubin NVL72
- DGX and HGX
- DGX is NVIDIA's complete system: DGX B200 and B300 come from NVIDIA, while DGX Spark and DGX Station are also built by NVIDIA's partners, under the partner's warranty. HGX is the GPU baseboard NVIDIA makes for server makers, who build the server around it. The same eight GPUs reach you either way; the question is who supports the machine.SeeNVIDIA DGX B300EMARQUE AI HGX B300NVIDIA DGX B200
- Blackwell and Blackwell Ultra
- NVIDIA's data-centre GPU generations after Hopper (H100, H200). B200 is Blackwell; B300 is Blackwell Ultra, which NVIDIA lists with 2.1 TB of GPU memory across eight GPUs against B200's 1.4 TB, and 1.5 times B200's dense FP4 compute.SeeH200 vs B200 vs B300NVIDIA DGX B300
- GB300
- NVIDIA's Grace Blackwell Ultra superchip: a Grace CPU and a Blackwell Ultra GPU joined by NVLink-C2C. One superchip powers DGX Station, with 748 GB of coherent memory; GB300 NVL72 puts 72 Blackwell Ultra GPUs and 36 Grace CPUs in one rack.SeeNVIDIA DGX Station GB300NVIDIA GB300 NVL72
- MIG (Multi-Instance GPU)
- NVIDIA's hardware partitioning: one physical GPU splits into up to seven isolated instances, each with its own memory, compute and cache. One card then serves several workloads that each need little VRAM.SeeNVIDIA DGX Station GB300
CPU + system
The host platform around the GPUs: CPU, memory, storage and power.
- EPYC
- AMD's server CPU family. Our AI HGX B300 runs dual EPYC 9005 or 9004 series processors, for the core count and the PCIe Gen5 lanes that eight GPUs need.SeeEMARQUE AI HGX B300
- Threadripper PRO
- AMD's workstation CPU family: fewer cores than the EPYC server parts, with full ECC support and 128 PCIe 5.0 lanes. Our AI PRO 500 uses the 9000 WX series, usually the 9995WX at 96 cores.SeeAI PRO 500
- Xeon Platinum
- Intel's flagship server CPU line. The Xeon Platinum 8570 is an Emerald Rapids part and the host CPU in NVIDIA DGX B200; our AI Server runs dual 5th-generation Xeon Scalable processors.SeeNVIDIA DGX B200EMARQUE AI Server
- ECC memory (Error-Correcting Code memory)
- Server-grade DRAM that detects and corrects single-bit errors, so one glitch cannot end a long training run. Our AI PRO 500 and AI Server use registered ECC; the AI Work 100 uses consumer DDR5 without ECC.
- RDIMM (Registered DIMM)
- ECC server memory with a register chip that keeps the bus stable as capacity rises. It is what lets a server carry terabytes of RAM.
- NVMe Gen5
- Storage on a PCIe Gen5 link, twice the bandwidth per lane of Gen4. It keeps the GPUs fed while datasets load and checkpoints are written.
- RDMA (Remote Direct Memory Access)
- A protocol that lets one server read or write another's memory without involving the host CPU, which cuts the latency between nodes that multi-node training depends on. Implemented on InfiniBand and on RoCE (RDMA over Converged Ethernet).
- InfiniBand
- High-throughput networking used to scale AI training across servers. Per link: HDR 200 Gb/s, NDR 400 Gb/s, XDR 800 Gb/s. NVIDIA GB300 NVL72 scales out over Quantum-X800 InfiniBand at XDR, or over Spectrum-X Ethernet.
Models + inference
The software side: model architectures, quantisation, fine-tuning and serving.
- LLM (Large Language Model)
- A neural network trained on large collections of text to generate or interpret language. We work with open-weight families you can run on your own premises: Qwen, Gemma, DeepSeek, gpt-oss and Mistral.
- RAG (Retrieval-Augmented Generation)
- A pattern where the model is given relevant excerpts from your own documents at query time, instead of relying on its training data. It answers questions about private material without retraining the model.SeePrivate RAG reference architecture
- Fine-tuning
- Continuing to train a pre-trained model on a smaller, domain-specific dataset so it picks up your vocabulary, style or task. We do full fine-tuning, which rewrites every weight, and adapter methods like LoRA.
- LoRA (Low-Rank Adaptation)
- A fine-tuning method that freezes the base model weights and trains a small adapter alongside them, typically 0.1% to 1% of the base model's size. One model can then carry several swappable adapters.
- QLoRA
- LoRA with the base model weights quantised to 4-bit during training. It cuts memory use further, bringing fine-tuning of a 70B-class model within reach of one workstation GPU, which is what our AI PRO 500 is sized for.
- Quantisation
- Compressing model weights from 16-bit to 8-bit, 4-bit or smaller, so a model that needed a server runs on a workstation. The accuracy cost depends on the model and the task.
- Inference
- Running a trained model to produce text or predictions: the production phase, as opposed to training. Most of the GPU time in a deployment goes here, so it is what we size for.
- vLLM
- An open-source inference server for LLMs. Its throughput advantage over plain PyTorch serving comes from paged attention, continuous batching and KV-cache management.
- TGI (Text Generation Inference)
- Hugging Face's inference server, an alternative to vLLM. The choice usually follows what your team already runs.
- Triton
- NVIDIA's general-purpose inference server. It serves several frameworks (PyTorch, TensorFlow, ONNX, TensorRT), which suits a deployment serving mixed model types from one endpoint: vision, voice and language together.
- TFLOPS / PFLOPS (tera / peta floating-point operations per second)
- One TFLOPS is a trillion floating-point operations per second, and one PFLOPS is a thousand TFLOPS. Theoretical throughput, useful for a first comparison between GPUs. NVIDIA quotes Tensor Core figures with sparsity unless it marks them dense, so check which.
Software + platform
The systems software underneath an on-premises AI deployment.
- DGX OS
- NVIDIA's Ubuntu-based operating system, with drivers, CUDA, NCCL and Docker configured. Included on DGX Spark, DGX B200 and DGX B300; DGX Station ships Ubuntu with NVIDIA AI Developer Tools. NVIDIA AI Enterprise is a separate licence on those four, and included with NVIDIA's DGX GB300 and DGX Vera Rubin racks.SeeNVIDIA DGX Spark
- CUDA (Compute Unified Device Architecture)
- NVIDIA's programming platform: every AI framework (PyTorch, TensorFlow, JAX) reaches GPU compute through it. Framework, CUDA and driver versions have to match each other.
- NCCL (NVIDIA Collective Communications Library)
- The GPU-to-GPU communication library distributed training uses, implementing collective operations (all-reduce, broadcast, all-gather) over NVLink and InfiniBand. Any multi-GPU training job needs it.
- Kubernetes
- A container orchestration platform. A single-team deployment usually does not need it; one shared across teams usually does.
Tell us about your workload.
Tell us what it needs to do and who will use it. A few lines is enough. We reply by email, and every quote comes in writing, in MYR.
