gpu-practice

📚 Study GPU programming with CUDA and PyTorch

GPU Mode lecture

• GPU Mode: [Youtube Channel],[github],

1. How to profile CUDA kernels in PyTorch
   • See the profile directory for examples on how to use PyTorch Profiler with CUDA kernels.
2. (Todo) Getting Started with CUDA
3. (Todo) Understanding Computation and Memory Architecture
4. (Todo) Fusing PyTorch Optimizer (optimizing optimizer)

... And so on

Some usefule articles

• The ML/AI Engineer's starter guide to GPU Programming
• Getting Started With CUDA for Python Programmers - Jeremy Howard. (2024, January 28)
• Optimizing MatMul from Naive to CUBLAS - Simon Boehm (Anthropic)
• Tensor Parallelism with CUDA - Multi-GPU Matrix Multiplication - Dennis Kennetz (Oracle)
• 100 days of CUDA challenge