Enabling Efficient Intra-Warp Communication for Fourier Transforms in a Many Core Architecture

synergy.cs.vt.edu

Enabling Efficient Intra-Warp Communication for Fourier Transforms in a Many Core ArchitectureStudent: Carlo C. del Mundo*, Virginia Tech (Undergrad)Advisor: Dr. Wu-chun Feng*§, Virginia Tech* Department of Electrical and Computer Engineering, § Department of Computer Science, Virginia Tech

synergy.cs.vt.eduEnabling Efficient Intra-Warp Communication for Fourier Transforms in a Many-Core Architecture

Forecast: Hardware-Software Co-Design

Software(Transpose)

Hardware(K20c and shuffle)

NVIDIA Kepler K20c Shuffle Mechanism


Q: What is shuffle?


Q: What is shuffle?

Cheaper data movement


Q: What is shuffle?


• Faster than shared memory


Q: What is shuffle?


• Faster than shared memory• Only in NVIDIA Tesla Kepler GPUs


Q: What is shuffle?


• Faster than shared memory• Only in NVIDIA Tesla Kepler GPUs• Limited to a warp


Q: What is shuffle?


• Faster than shared memory• Only in NVIDIA Tesla Kepler GPUs• Limited to a warp>>> Idea: reduce data communication between threads <<<


Q: What are you solving?


Q: What are you solving?• Enable efficient data

communication



communication– Shared Memory (the “old” way)







– Shuffle (the “new” way)


Approach• Evaluate shuffle using matrix transpose

– Matrix transpose is a data communication step in FFT







• Devised Shuffle Transpose Algorithm– Consists of horizontal (inter-thread shuffles) and vertical

(intra-thread)


Analysis• Bottleneck: Intra-thread data movement


Analysis

Register File

t0 t1 t2 t3

• Bottleneck: Intra-thread data movement

Stage 2: Vertical


Analysis

Register File

t0 t1 t2 t3

for (int k = 0; k < 4; ++k) dst_registers[k] = src_registers[(4 - tid + k) % 4];

Code 1: (NAIVE)


Stage 2: Vertical


Analysis

Register File

t0 t1 t2 t3


Code 1: (NAIVE)


Stage 2: Vertical


Analysis

Register File

t0 t1 t2 t3


Code 1: (NAIVE)


Stage 2: Vertical

15x


Code 1 (NAIVE)63 for (int k = 0; k < 4; ++k)64 dst_registers[k] = src_registers[(4 -

tid + k) % 4];

General strategies• Registers are fast.• CUDA local memory is slow.

– Compiler is forced to place data into CUDA local memory if array indices CANNOT be determined at compile time.

Analysis15x



tid + k) % 4];

Code 2 (DIV)int tmp = src_registers[0];if (tid == 1){

src_registers[0] = src_registers[3];src_registers[3] = src_registers[2];src_registers[2] = src_registers[1];src_registers[1] = tmp;

}else if (tid == 2){

src_registers[0] = src_registers[2];src_registers[2] = tmp;tmp = src_registers[1];src_registers[1] = src_registers[3];src_registers[3] = tmp;



}



Analysis15x



tid + k) % 4];







}



Divergence

Divergence

Divergence

Analysis15x

6%



tid + k) % 4];







}



Code 3 (SELP OOP)

65 dst_registers[0] = (tid == 0) ? src_registers[0] : dst_registers[0]; 66 dst_registers[1] = (tid == 0) ? src_registers[1] : dst_registers[1]; 67 dst_registers[2] = (tid == 0) ? src_registers[2] : dst_registers[2]; 68 dst_registers[3] = (tid == 0) ? src_registers[3] : dst_registers[3]; 69 70 dst_registers[0] = (tid == 1) ? src_registers[3] : dst_registers[0]; 71 dst_registers[3] = (tid == 1) ? src_registers[2] : dst_registers[3]; 72 dst_registers[2] = (tid == 1) ? src_registers[1] : dst_registers[2]; 73 dst_registers[1] = (tid == 1) ? src_registers[0] : dst_registers[1]; 74 75 dst_registers[0] = (tid == 2) ? src_registers[2] : dst_registers[0]; 76 dst_registers[2] = (tid == 2) ? src_registers[0] : dst_registers[2]; 77 dst_registers[1] = (tid == 2) ? src_registers[3] : dst_registers[1]; 78 dst_registers[3] = (tid == 2) ? src_registers[1] : dst_registers[3]; 79 80 dst_registers[0] = (tid == 3) ? src_registers[1] : dst_registers[0]; 81 dst_registers[1] = (tid == 3) ? src_registers[2] : dst_registers[1]; 82 dst_registers[2] = (tid == 3) ? src_registers[3] : dst_registers[2]; 83 dst_registers[3] = (tid == 3) ? src_registers[0] : dst_registers[3];

Divergence

Divergence

Divergence

Analysis15x

6%

44%


Results


Results


Conclusion• Overall Performance

– Max. Speedup (Amdahl’s Law): 1.19-fold– Achieved Speedup: 1.17-fold


Conclusion• Overall Performance

– Max. Speedup (Amdahl’s Law): 1.19-fold– Achieved Speedup: 1.17-fold

• Surprise Result– Goal: Accelerate communication (“gray bar”)– Result: Accelerated the computation also (“black

bar”)


Thank You!• Enabling Efficient Intra-Warp Comunication for

Fourier Transforms in a Many-Core Architecture– Student: Carlo del Mundo, Virginia Tech

(undergrad)– Overall Performance

• Theoretical Speedup: 1.19-fold• Achieved Speedup: 1.17-fold


tid + k) % 4];


Appendix


Motivation• Goal

– Accelerating an application based on hardware-specific mechanisms (e.g., “the hardware-software co-design process”)

• Case Study– Application: Matrix transpose as part of a 256-pt

FFT– Architecture: NVIDIA Kepler K20c

• Use shuffle to accelerate communication

• Results– Max. Theoretical Speedup: 1.19-fold– Achieved Speedup: 1.17-fold


Background: The New and Old• Shuffle

– Idea:• Communicate data

within a warp w/o shared memory

– Pros• Faster (1 cycle to

perform load and store)• Eliminate the use of

shared memory higher thread occupancy

– Cons• Poorly understood• Only available in Kepler

GPUs

• Only limited to 32 threads

• Shared Memory– Idea

• Scratchpad memory to communicate data

– Pros• Easy to program• Scales to a block (up to

1536 threads)– Cons

• Prone to bank conflicts

Date post:	25-Feb-2016
Category:	Documents
Upload:	alta
View:	31 times
Download:	0 times

Enabling Efficient Intra-Warp Communication for Fourier Transforms in a Many Core Architecture

Documents