High Bandwidth Memory (HBM)
3D-stacked DRAM architecture providing massive bandwidth for GPUs and AI accelerators
11 min readConcept
Tagged with
gpu
Explore machine learning concepts related to gpu. Clear explanations and practical insights.
15
Concepts Found
Concepts Related to gpu
GPU Memory Hierarchy & Optimization
Master GPU memory hierarchy from registers to global memory, understand coalescing patterns, bank conflicts, and optimization strategies for maximum performance
8 min readConcept
NVIDIA Unified Virtual Memory
Automatic memory management between CPU and GPU through page faulting and on-demand migration
12 min readConcept
Page Migration & Fault Handling
Understanding virtual memory page migration, fault handling, and TLB management in CPU-GPU systems
9 min readConcept
NVIDIA Device Files in /dev/
Understanding character devices, major/minor numbers, and the device file hierarchy created by NVIDIA drivers for GPU access in Linux.
11 min readConcept
Understanding nvidia-modeset: Kernel Mode-Setting for NVIDIA GPUs
Deep dive into nvidia-modeset, the NVIDIA kernel module that handles display mode-setting, monitor configuration, and DRM integration in Linux systems.
11 min readConcept
CUDA Multi-Process Service (MPS)
Understand NVIDIA CUDA Multi-Process Service (MPS), a client-server architecture that enables multiple CUDA processes to share a single GPU context for concurrent kernel execution and better utilization.
7 min readConcept
Flash Attention: IO-Aware Exact Attention
Interactive visualization of Flash Attention - the breakthrough algorithm that makes attention memory-efficient through tiling, recomputation, and kernel fusion.
7 min readConcept
Understanding NVIDIA Kubernetes GPU Operator
Explore how the NVIDIA GPU Operator automates GPU infrastructure management in Kubernetes, transforming manual GPU setup into a declarative, cloud-native system.
8 min readConcept
Understanding CUDA Contexts
Explore the concept of CUDA contexts, their role in managing GPU resources, and how they enable parallel execution across multiple CPU threads.
2 min readConcept
SoA vs AoS: Data Layout Optimization
Master Structure of Arrays (SoA) vs Array of Structures (AoS) data layouts for optimal cache efficiency, SIMD vectorization, and GPU memory coalescing with interactive visualizations.
11 min readConcept
Understanding NVIDIA Persistence Daemon
Eliminating GPU initialization latency through nvidia-persistenced - a userspace daemon that maintains GPU driver state for optimal startup performance.
11 min readConcept
NCCL: High-Performance Multi-GPU Communication
Understanding NVIDIA's Collective Communications Library for distributed deep learning and multi-GPU training
8 min readConcept
Tensor Cores: Accelerating Deep Learning
Understanding NVIDIA's specialized matrix multiplication hardware for AI workloads
6 min readConcept
GPU Streaming Multiprocessor (SM)
Deep dive into the fundamental processing unit of modern GPUs - the Streaming Multiprocessor architecture, execution model, and memory hierarchy
8 min readConcept