Date post: | 08-Jan-2018 |
Category: |
Documents |
Upload: | charlotte-cummings |
View: | 222 times |
Download: | 0 times |
Lofar Information System on GRID
A.N.Belikov
Lofar Long Term Archive
Prototypes: EGEE Astro-WISE
Requirements to data storage
Tiers
Astro-WISE adaptation
Problem: optimal data placement
Lofar: Science
Lofar: Network
10 Gbps optical
single data processing center for raw data (CIT, Groningen)
Lofar: Data Processing
GbE
sw
itch
GbE
switc
hG
bEsw
itch
GbE
switc
hG
bEsw
itch
GbE
switc
h
Auxiliary processingsection
Stor
age
syst
em
Off-line processing/user software
Off-lineStorageOn-Line
Long term Archiving& Data ExportG
bEsw
itch
On-Line pipeline processingBlue Gene/L
WAN
Blue Gene/L
linux cluster
centralized data processing for raw data
long-term data storage
Lofar: LTA requirements
Lofar: LTA requirements
20 PB for 5 yearsdecentralized data storagedecentralized data reprocessinginternational partners
LHC & EGEE
15 PB data per year grid computing
EGEE: ~250 sites, >45000 CPU
www.eu-egee.org EGEE-III
EGEE: Tiers approach
Astro-Wise: grid
Astro-WISE: dataservers (1)
Astro-WISE: dataservers (2)
Astro-WiseLoWise
dataserverdata retrieved by URI0.5 TB 7 sec
3.5 TB 20 sec 1 PB 8 h
LoWISE: URI storage
LoWISE: Tiers
Astro-WISE-like interfaces user access (from PM to anonymous) Tier 1 as core element
LoWISE: data storage
distributed file systems: GPFS (IBM) Lustre (Sun)
dCAcheCASTOR
Lofar-WISE: Data Storage
initial configuration
distributed FS & EGEE
Oracle for metadata
Problem: users
users: observer, project managerprivileged users: operators
Problem: CE & SE geographical location file transfer user access research group driven
Problem: Access policy
Problem: Optimal Data Placement
priorities groups pool of queries TCF monitor system
Task
create and implement a system for an optimal data placement