MySpace: Personalized MySpace: Personalized Work Space in AstroGridWork Space in AstroGrid

Catherine Ling QinCatherine Ling Qin

2626thth Nov 2003 Nov 2003

26/Nov/200326/Nov/2003 IVOA Small Projects, BeiJingIVOA Small Projects, BeiJing

BackgroundBackground

• Vast amount of DataVast amount of Data

• Sharing Data (Limited Storage for Sharing Data (Limited Storage for each astronomer)each astronomer)

• Distributed Across The GlobeDistributed Across The Globe

26/Nov/200326/Nov/2003 IVOA Small Projects, BeiJingIVOA Small Projects, BeiJing

OverviewOverview

• MySpace is a component of AstroGridMySpace is a component of AstroGrid

• A system for accessing astronomical A system for accessing astronomical archives which is being developed in the archives which is being developed in the UKUK

• MySpace provides AstroGrid users with MySpace provides AstroGrid users with scratch space for storing scratch space for storing temporary/permanent datasets, typically temporary/permanent datasets, typically the results of queries submitted to large the results of queries submitted to large databases, and other transient files. databases, and other transient files.

26/Nov/200326/Nov/2003 IVOA Small Projects, BeiJingIVOA Small Projects, BeiJing

DesignDesign

• Open-source software products Open-source software products

• Distributed, collaborative and Distributed, collaborative and centrally-controlled data centrally-controlled data storage/processing system storage/processing system

• Simple and user-friendly interface Simple and user-friendly interface

• Easy Installation (Plug and Play). Easy Installation (Plug and Play). Eventually be deployed at institutions Eventually be deployed at institutions around the globe.around the globe.

26/Nov/200326/Nov/2003 IVOA Small Projects, BeiJingIVOA Small Projects, BeiJing

MySpace ComponentsMySpace Components

• MySpaceManagerMySpaceManager

Invoked by users to access or manipulate data sources in MySpace systems. It incorporates a registry comprising the Meta Data describing the data sources in the MySpace system. Typically there will be one MySpace Manager per Community.

26/Nov/200326/Nov/2003 IVOA Small Projects, BeiJingIVOA Small Projects, BeiJing

MySpace Components MySpace Components (Continue)(Continue)

• MySpaceServerMySpaceServer

Act like repositories and are where the Data Sources are kept.

•Cache Server -- Stores Transient DataCache Server -- Stores Transient Data

•Community Server – Stores Data Sources Community Server – Stores Data Sources within the Community.within the Community.

26/Nov/200326/Nov/2003 IVOA Small Projects, BeiJingIVOA Small Projects, BeiJing

MySpace Components MySpace Components (Continue)(Continue)

• MySpace ExplorerMySpace ExplorerEasy to use.Easy to use.Web Based. The User Interface Web Based. The User Interface displayed displayed

within a web browser that allows within a web browser that allows astronomers to interactively browse/edit astronomers to interactively browse/edit their data sources within MySpace Systems. their data sources within MySpace Systems.

MySpace Explorer displays a simple and MySpace Explorer displays a simple and intuitive representation of hierarchical data intuitive representation of hierarchical data sources. The notional hierarchy of containers sources. The notional hierarchy of containers does not correspond to the actual directory does not correspond to the actual directory structure on MySpace Servers. Rather, the structure on MySpace Servers. Rather, the structure is stored and maintained by structure is stored and maintained by MySpace managerMySpace manager

26/Nov/200326/Nov/2003 IVOA Small Projects, BeiJingIVOA Small Projects, BeiJing

MySpace Components MySpace Components (Continue)(Continue)• MySpace ExplorerMySpace Explorer

26/Nov/200326/Nov/2003 IVOA Small Projects, BeiJingIVOA Small Projects, BeiJing

MySpace Components (Continue)MySpace Components (Continue)

• MySpace ManagerMySpace Manager

• MySpace ServerMySpace Server•Cache ServerCache Server

•Community ServerCommunity Server

MyS

pace M

anager

MyS

pace

Server

MyS

pace S

erverMyS

pace S

erver

Internet

Data Transfer between servers

DatasetAccess

Collocated data transfer

DatasetAccess

Collocated data transfer DatasetAccess

Collocated data transfer

Leicester Community

BeiJing Community

26/Nov/200326/Nov/2003 IVOA Small Projects, BeiJingIVOA Small Projects, BeiJing

Technology ChoicesTechnology Choices

• Web servicesWeb services

• JavaJava

• Cocoon (MVC)Cocoon (MVC)

26/Nov/200326/Nov/2003 IVOA Small Projects, BeiJingIVOA Small Projects, BeiJing

Current Implementation and Current Implementation and Supported FunctionalitiesSupported Functionalities

• Create/Delete File or ContainerCreate/Delete File or Container

• Upload/Download FileUpload/Download File

• Copy/Move FileCopy/Move File

• Look up details of a single fileLook up details of a single file

• Look up details of all files matching an Look up details of all files matching an optionally wild-carded file name.optionally wild-carded file name.

• Extend LeaseExtend Lease

• A rudimentary MySpace Explorer.A rudimentary MySpace Explorer.

26/Nov/200326/Nov/2003 IVOA Small Projects, BeiJingIVOA Small Projects, BeiJing

Enhancements and Open Enhancements and Open IssuesIssues• File Storing System Combined with DBMS File Storing System Combined with DBMS

Database.Database.• Access ControlAccess Control• File Transferring Protocols File Transferring Protocols

(Http/FTP/GridFTP) (Http/FTP/GridFTP) • In order that free access to (sometimes In order that free access to (sometimes

huge) distributed data sources does not huge) distributed data sources does not cause network resources to become cause network resources to become overloaded, we would need to investigate overloaded, we would need to investigate thethe possibility of smart algorithms for possibility of smart algorithms for controlled replication of (parts of) data controlled replication of (parts of) data sources. sources.