paul@sep.stanford.edu

weilib: wave-equation imaging

Paul Sava

paul@sep.stanford.edu

My goal

• Quick introduction to the library

• Not much about the imaging science

• Ask the right questions

paul@sep.stanford.edu

Library goals

• Flexible research– Code recycling – Efficiency vs. flexibility

• Cluster access– Parallelization with MPI, OMP

• Large problems– 3-D data

paul@sep.stanford.edu

Characteristics

• Programming– Fortran 90– OO

• Problem types– Shot-receiver migration– Shot-profile migration

• Versions

paul@sep.stanford.edu

Linear operators

dm L

call operator_init(…)

stat=operator(adj,add,m,d)

m d L

paul@sep.stanford.edu

WE imaging

dm L• Large datasets (m,d)

– Data partition

• Complicated operators (L)– Operator partition

paul@sep.stanford.edu

The problem

DATA (d)

IMA

GE

(m

)

dm *L

z

paul@sep.stanford.edu

Cluster distribution

MPI

z

ii dm *L

paul@sep.stanford.edu

Outline

• Overview

• Objects– Partitioning– I/O

• Operators– WE migration– WE MVA

• Examples

paul@sep.stanford.edu

WEI initialization

…

call sep_init()

call weipar_init()

call weipar_report()

call weitag_init()

call weipro_init()

call weipro_bcast()

…

paul@sep.stanford.edu

WEI objects & procedures

• Objects– Data– Image– Slowness– Wavefield

• Procedures– file tag– dimensions– I/O– MPI communication

paul@sep.stanford.edu

WEI data object

type wedata

character(len=128) :: tag

character(len=128) :: from

integer :: bites,esize

type(axis) :: amx,amy,ahx,ahy,aw

complex, pointer :: bin(:,:,:,:,:)

end type wedata

paul@sep.stanford.edu

WEI data object

DD%tag=wtag%D

call weiD_init(DD)

call weiD_report(DD)

call weiD_allocate(DD)

call weiD_read(DD)

call weiD_write(DD)

call weiD_mpisprd(DD)

call weiD_mpigthr(DD)

…

paul@sep.stanford.edu

WEI migration workflow

data

slowness

image

image

image

image

slowness

slowness

slowness

slowness

data

data

data

data

imageNode 2

Node 4

Node 3

Node 1MPI

MPI

paul@sep.stanford.edu

WEI workflow example

…

call weiS_mpicopy(SS)

call weiR_mpizero(RR)

call weiD_mpisprd(DD)

…

call weimigof(…)

…

call weiR_mpisumm(RR)

…

paul@sep.stanford.edu

I/O blocking

z

paul@sep.stanford.edu

I/O blocking example

…

do iwb=1,nwb

call weiD_read(bW,iwb)

do izb=1,nzb

call weiR_read(RR,izb)

call weiS_read(SS,izb)

…

call weiR_write(RR,izb)

end do

end do

…

paul@sep.stanford.edu

Shared memory parallelization

OMP

z

paul@sep.stanford.edu

Shared memory parallelization

!$OMP PARALLEL

!$OMP DO &

!$OMP$ PRIVATE(iws,izs,ith)

do iws=1,nws

ith=omp_get_thread_num()+1

do izs=1,nzs

…

end do

end do

!$OMP END DO

!$OMP END PARALLEL

paul@sep.stanford.edu

Outline

• Overview

• Objects– Partitioning– I/O

• Operators– WE migration– WE MVA

• Examples

paul@sep.stanford.edu

WE migration

extr

apol

atio

n

imaging

depth

frequencyDATA (d)

IMA

GE

(m

)

paul@sep.stanford.edu

Extrapolation

zikzzz

zeWW

depth

frequency

Wz

Wz+z 00

0ss

ds

dkkk

s

zzz

paul@sep.stanford.edu

Imaging

depth

frequency

Wz+z

Rz+z

zzzz RW

paul@sep.stanford.edu

zssds

dki

zik

zzzs

z

z eeWW

000

WE migration

zzzz RW

paul@sep.stanford.edu

WCop: wavefield continuation operator

• Single reference slowness• Multi reference slowness

– Wavefield interpolation

zssds

dki

zik

zzzs

z

z eeWW

000

zzzz RW

paul@sep.stanford.edu

SLop: slowness selection operator

• Min/Mean/Median• Lloyd

zssds

dki

zik

zzzs

z

z eeWW

000

zzzz RW

paul@sep.stanford.edu

FKop: phase-shift operator

• Common-azimuth• Narrow-azimuth • Full prestack

zssds

dki

zik

zzzs

z

z eeWW

000

zzzz RW

paul@sep.stanford.edu

FXop: finite-difference operator

• Phase-shift• Split-step Fourier• Generalized Screen Propagator

zssds

dki

zik

zzzs

z

z eeWW

000

zzzz RW

paul@sep.stanford.edu

IGop: imaging operator

• Offset• Ray parameter

zssds

dki

zik

zzzs

z

z eeWW

000

zzzz RW

paul@sep.stanford.edu

Outline

• Overview

• Objects– Partitioning– I/O

• Operators– WE migration– WE MVA

• Examples

paul@sep.stanford.edu

WE MVA

zsds

dki

zik

zzzs

z

z eeWW

00

zs

ds

dkiWW

s

zzzzz

0

10

szds

dkiWW

s

z 0

0

paul@sep.stanford.edu

SCop: scattering operator

zsds

dki

zik

zzzs

z

z eeWW

00

szds

dkiWW

s

z 0

0

paul@sep.stanford.edu

Outline

• Overview

• Objects– Partitioning– I/O

• Operators– WE migration– WE MVA

• Examples

paul@sep.stanford.edu

WEI datuming (survey sinking)

f-x operator

Slowness operator

f-k operator

Continuation operator

stat = weidtm(adj,add, D, U,SLop=weiop_slo1,

WCop=weiop_mwc1,FKop=weiop_wem3,

FXop=weiop_ssf3)

call weidtm_init(SLin=weiop_slo1_init,

WCin=weiop_mwc1_init,FKin=weiop_wem3_init,FXin=weiop_ssf3_init)

paul@sep.stanford.edu

WEI migration: example 1

stat = weimig(adj,add, R, D,SLop=weiop_slo1,WCop=weiop_mwc1,FKop=weiop_wem3,FXop=weiop_ssf3,IGop=weiop_pcig)

Imaging operator

call weimig_init(SLin=weiop_slo1_init,

WCin=weiop_mwc1_init,FKin=weiop_wem3_init,FXin=weiop_ssf3_init,IGin=weiop_pcig_init)

paul@sep.stanford.edu

WEI migration: example 2

stat = weimig(adj,add, R, D,SLop=weiop_sllN,WCop=weiop_mwcN,FKop=weiop_cam3,FXop=weiop_ssf3,IGop=weiop_hcig)

call weimig_init(SLin=weiop_sllN_init,

WCin=weiop_mwcN_init,FKin=weiop_cam3_init,FXin=weiop_ssf3_init,IGin=weiop_hcig_init)

paul@sep.stanford.edu

WEI MVA

stat = weimva(adj,add, dS,dR,SLop=weiop_sllN,SCop=weiop_bor1,WCop=weiop_mwcN,FKop=weiop_wem3,FXop=weiop_ssf3,IGop=weiop_hcig)

Scattering operator

call weimva_init(SLin=weiop_sllN_init,SCin=weiop_bor1_init,

WCin=weiop_mwcN_init,FKin=weiop_wem3_init,FXin=weiop_ssf3_init,IGin=weiop_hcig_init)

paul@sep.stanford.edu

Summary

• Flexible, reusable f90 code

• Cluster ready (MPI,OMP)

• Standard operator interface

• Applications– WE datuming– WE migration– WE MVA

paul@sep.stanford.edu

Resources (name@sep.stanford.edu)

• Developers– Paul, Bob

• Manual– Marie

• Users– Biondo, Antoine, Morgan, Brad, Nick, Alejandro