Date post: | 11-Jan-2016 |
Category: |
Documents |
Upload: | ernest-lamb |
View: | 213 times |
Download: | 0 times |
Active Folder : Integrating All Activities of Simulation on File System
Active Folder : Integrating All Activities of Simulation on File System
Suntae Hwang ( [email protected] )Daeyoung Heo ( [email protected] )
School of Computer ScienceKookmin University
Apr 10 2014
2
Active Folder: Integrating All Activities of Simula-tion on File System Active Folder: Integrating All Activities of Simula-tion on File System
⊙Active Folder – good for case comparative study●Tasks○Described as regular folders and files
●Products○Input or output of simulation ○Can be handled like regular file by using legacy software○Contains provenance information (meta data, task info,
etc)○Can be reproduced by the task which is extracted from the
provenance information●Apps(Computing Resource)○Computing server(Local, Grid, Cloud, what ever, …) is reg-
istered as regular folders and files○To submit a Job(task), just Drag&Drop the task folder to the
folder which represents computing server
ActiveFolder
3
Active Folder: Integrating All Activities of Simula-tion on File System Active Folder: Integrating All Activities of Simula-tion on File System
Storage(Cloud, Local, ..)
Producing
Sharing
Analizing
• Products• Tasks• Apps
User’s own environment
Computing(Cloud, Grid, Local, …)
Or Searching(DB, IR, …)
4
Formal Definition of Spaces in Active Folder Formal Definition of Spaces in Active Folder
Definition 1. Set P for Product
Definition 2. Set T for taskDefinition 3. Set A for producing/reproducingDefinition 4. Set M for monitoring.
5
Active Folder: Product and ReproducingActive Folder: Product and Reproducing
DATA
History
TaskProduct
Fall3D
Visualize & Analysis
Procedure
𝒍𝒆𝒕 𝒕=𝒕𝒂𝒔𝒌 (𝒑)𝒂𝒑𝒑(𝒕 ,𝑲 ,𝑷𝒌)
Differentparame-ters
Task can be ex-tracted from prod-ucts and used for producing similar results with differ-ent parameters
Product consists of data, history info and task(provenance)
6
Active Folder: Products Searching and ComparisonActive Folder: Products Searching and Comparison
⊙Similar or interested products can be searched by their provenance information and compared by var-ious legacy tools on conventional file system
7
Case Study : Volcano Eruption Simulation using Ac-tive FolderCase Study : Volcano Eruption Simulation using Ac-tive Folder
8
Short history of Active FolderShort history of Active Folder
⊙Active Folder on DropBox+EC2●Cost & Performance Problem with very large files
⊙Active Folder on NAS⊙NAS(Network Attached Storage)● Large Volume Storage● Network File System ( NFS,
SMB/CIFS, AFP … )● Most vendors support Cloud
solution like DropBox
at PRAGMA 25
at PRAGMA 24
http://www.youtube.com/watch?v=vXTh3tCHEPE
9
Spin
1000
Temperature273K
Planning
File & Folder
Everything is
Applications
Executio
nDrag & Drop
SSH
CHARMM
AMBER
NAMD
GAUSSIAN
ActiveFolder . . . ...
CHARM
M
SimulationScript
Activity
CHARMMT-SPACE
ssh.server.com
Server
A-SPACE
Active Folder Features 1:Computing by manipulating files or folders, which makes only files visible by hiding correlation of computing method, parameters, result files and computing re-sources
10
Spin
1000
Temperature273K
Activ
eFol
der . .
. ...
CHARM
M
Sim
ulat
ion
Script
How to create this product?
Products
P-SPACE
Product
Product
Data Provenance
Relationship
PROVIDED / USED
History Board
Created by CHARMM Activity
From ssh.server.com
At 2014.m.d
StageOut : 2014.m.d 12:30
StageOut : 2014.m.d 13:02
Active Folder Features 2:Keep the way of computing meth-ods and parameters with result files silently
11
Activ
eFol
der . .
. ...
CHARM
M
Sim
ulat
ion
Script
Temperature300K
Temperature293K
Searching
Data Provenance
Parameters
Para
mete
rs
Para
met
ers
Para
met
ers
Parameters
Para
mete
rs
Para
mete
rs
Para
mete
rs
Active Folder Features 3:Searching result files and other core-lated ones
12
Activ
eFol
der . .
. ...
CHARM
M
Sim
ulat
ion
Script
Reproduction
Using data provenance
Extracting
Activity
CHARMMT-SPACE
Parameters
Para
mete
rs
Para
mete
rs
Temperature
293K
ApplyingNew Methods/Parameters
Para
mete
rs
Para
mete
rs
Para
mete
rs
Para
mete
rs
Para
mete
rs
Para
mete
rs
Execution
Active Folder Features 4:Computing by manipulating files or folders, Re-generating either the same or similar ones by existing re-sult files
13
ssh.server.com
Server
A-SPACE
CHARMMSSH
Products
Charmm.dcd file created
StageOut : 2014.m.d 12:30
Charmm.dcd file modified
StageOut : 2014.m.d 1:13
stdout.afsx.log file created
StageOut : 2014.m.d 1:14
stdout.afsx.log file modified
StageOut : 2014.m.d 1:15
Tim
eli
ne
Data Synchronization
Between server and your active folder
• Automatic Stage-Out to ActiveFolder• Incremental update.
Active Folder Features 5:Returns computing results by progressive synchronization without connecting computing server
14
File Navigator
Products
P-SPACE
Activity
CHARMMT-SPACE
ssh.server.com
Server
A-SPACE
SAMBA
FTP
NFS
Data Delivery
Support Legacy File ServiceYour Desktop, Laptop or Mobile
Mic
rosoft
Win
dow
s
MacO
SX
Lin
ux
Smartphone
Clo
ud
Sto
rag
e
ActiveFolder
Apps Activities Products
Open this files by your favorite softwares
Active Folder Features 6:Support various file service such as NFS, Samba(CIFS) and FTP
15
Active Folder: Overall ViewActive Folder: Overall View
Products
P-SPACE
Activity
CHARMMT-SPACE
ssh.server.com
Servers
A-SPACE
Searching and Analysis
ActiveFolder
16
Active Folder: Software ArchitectureActive Folder: Software Architecture
17
ActiveFolder
ComputingEnvironment(Large scale
service)
ActiveFolder
Satellite Service
* Private Space - File Management - Private Archiving* For Power-User
Satellite Service
ActiveFolder
Satellite Service
Usage of Active FolderUsage of Active Folder
⊙Good for managing & sharing data in a small group of Lab
⊙For Power-users us-ing legacy SW in their own computing environment (with-out GUI sometimes)
18
Active Folder SiteActive Folder Site
⊙Active Folder is released
●http://www.activefolder.org
⊙Downloads ●Binary file for general installation(Synology(NAS) tested)●RPM for Redhat 6/CentOS 6
⊙Coming soon●Product searching interface●Synology packaging for NAS●Integrated using NAS and DropBox together
Attention! .org only, not .com
19
Thank you!
Questions?