Post on 22-Dec-2015
transcript
Graphics Processing UnitsREFERENCES:
•COMPUTER ARCHITECTURE 5TH EDITION, HENNESSY AND PATTERSON, 2012
•HTTP://WWW.NVIDIA.COM/CONTENT/PDF/FERMI_WHITE_PAPERS/NVIDIA_FERMI_COMPUTE_ARCHITECTURE_WHITEPAPER.PDF
•HTTP://WWW.REALWORLDTECH.COM/PAGE.CFM?ARTICLEID=RWT093009110932&P=1
•HTTP://WWW.MODERNGPU.COM/INTRO/PERFORMANCE.HTML
•HTTP://HEATHER.CS.UCDAVIS.EDU/PARPROCBOOK
CPU vs. GPU
http://chip-architect.com/news/2003_04_20_Looking_at_Intels_Prescott_part2.html
• CPU: small fraction of chip used for arithmetic
CPU vs GPU
http://www.pcper.com/reviews/Graphics-Cards/NVIDIA-GT200-Revealed-GeForce-GTX-280-and-GTX-260-Review/NVIDIA-GT200-Archite
• GPU: large fraction of chip used for arithmetic
CPU vs GPU
Intel Haswell 170 GFlops on quad-core at 3.4GHz
AMD Radeon R9 290 4800 GFlops at 9.5GHz
Nvidia GTX 970 5000 Gflops at 1.05GHz
GPGPU
General Purpose GPU programming Massively parallel
Scientific computing, brain simulations, etc
In supercomputers 53 of top500.org supercomputers used
NVIDIA/AMD GPUs (Nov 2014 ranking)
Including 2nd and 6th places
OpenCL vs CUDA
Both for GPGPU
OpenCL Open standard
Supported on AMD, NVIDIA, Intel, Altera, …
CUDA Proprietary (Nvidia)
Losing ground to OpenCL?
Similar performance
CUDA
Programming on Parallel Machines, Norm Matloff, Chapter 5
http://www.nvidia.com/content/PDF/fermi_white_papers/NVIDIA_Fermi_Compute_Architecture_Whitepaper.pdf
Uses a thread hierarchy Thread
Block
Grid
Thread
Executes an instance of a kernel (program)
ThreadID (within block), program counter, registers, private memory, input and output parameters
Private memory for register spills, function calls, array variables
Nvidia Fermi Whitepaper pg 6
Block
Set of concurrently executing threads
Cooperate via barrier synchronization and shared memory (fast but small)
BlockID (within grid)
Nvidia Fermi Whitepaper pg 6
Grid
Array of thread blocks running same kernel
Read and write global memory (slow – hundreds of cycles)
Synchronize between dependent kernel calls
Nvidia Fermi Whitepaper pg 6
Hardware Mapping
GPU executes 1+ kernel (program) grids
Streaming Multiprocessor (sm) executes 1+ thread blocks
CUDA core executes thread
Fermi Architecture
Debuted in 2010
512 CUDA cores executes 1 FP or integer instruction per cycle
32 CUDA cores per SM
16 SMs per GPU
6 64-bit memory ports
PCI-Express interface to CPU
GigaThread scheduler distributes blocks to SMs each SM has a thread scheduler (in hardware)
fast context switch
3 billion transistors
Nvid
ia F
erm
i W
hit
epaper
pg 7
CUDA core
pipelined integer and FP units
IEEE 754-2008 FP fused multiply-add
integer unit boolean, shift, move, compare, ...
Nvidia Fermi Whitepaper pg 8
Streaming Multiprocessor (SM) 32 CUDA cores
16 ld/st units calculate source/destination
addresses
Special Function Units sin, cosine, reciprocal, sqrt
Nvidia Fermi Whitepaper pg 8
Warps
32 threads from a block are bundled into warps which execute the same instr/cycle
this becomes the minimum size of SIMD data
warps are implicitly synchronized if threads branch in different directions, they step
through both using predicated instructions
two warp schedulers select 1 instruction from a warp each to issue to 16 cores, 16 ld/st units or 4 SFUs
Maxwell Architecture
2014
16 streaming multiprocessors * 128 cores/SM = 2048 cores
Programming CUDA
C code
daxpy(n,2.0,x,y); // invoke
void daxpy(int n, double a, double *x double *y) { for(int i=0; i<n; i++) y[i] = a*x[i] + y[i];}
Programming CUDA
CUDA code
__host__int nblocks=(n+511)/512; // grid sizedaxpy<<<nblocks,512>>(n,2.0,x,y);// 512 threads/block
__global__void daxpy(int n, double a, double *x double *y) { int i=blockIdx.x*blockDim.x + threadIdx.x; if(i<n) y[i] = a*x[i] + y[i];}
n=8192, 512 threads/blockgrid block0 warp0 Y[0]=A*X[0]+Y[0]
...
Y[31]=A*X[31]+Y[31]
...
warp15 Y[480]=A*X[480]+Y[480]
...
Y[511]=A*X[511]+Y[511]
...
block15 warp0 Y[7680]=A*X[7680]+Y[7680]
...
Y[7711]=A*X[7711]+Y[7711]
...
warp15 Y[8160]=A*X[8160]+Y[8160]
...
Y[8191]=A*X[8191]+Y[8191]
Moving data between host and GPU
int main() {double *x, *y, a, *dx, *dy;x = (double *)malloc(sizeof(double)*n);y = (double *)malloc(sizeof(double)*n);// initialize x and y…cudaMalloc(dx, n*sizeof(double));cudaMalloc(dy, n*sizeof(double));cudaMemcpy(dx, x, n*sizeof(double), cudaMemcpyHostToDevice); …daxpy<<<nblocks,512>>(n,2.0,x,y);cudaThreadSynchronize();cudaMemcpy(y, dy, n*sizeof(double), cudaMemcpyDeviceToHost);cudaMemFree(dx); cudaMemFree(dy);free(x); free(y);
}