Date post: | 25-Dec-2015 |
Category: |
Documents |
Upload: | frederick-beasley |
View: | 218 times |
Download: | 4 times |
Director, NSF Planning I/UCRC for Spatiotemporal Thinking,
Computing and Applications
Co-Director, Center of Intelligent Spatial Computing for
Water/Energy Sciences
Associate Professor, Geography and GeoInformation Science
George Mason Univ., Fairfax, VA, 22030-4444
http://cisc.gmu.edu/
http://cpgis.gmu.edu/homepage/
Page 2
What is Cloud Computing
Why Cloud Computing
What are the Issues
Cloud Computing Future
Outline
Cloud Computing Research
Background
Background I
Background II
Background III
What if we can• Integrate all geospatial data, information,
knowledge, processing in a few minutes• Generate and send the right information in real time
to the people including decision makers, first responders, victims
This dream requires a computing platform that • can be ready in a few minutes• can reach out to all people needed• only cost for the amount of computing used• won’t cost to maintain after the emergency
responseThis requires spatiotemporal thinking and computing, and was somehow envisioned by cloud computing
Cloud ComputingCloud Computing
Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. This cloud model promotes availability and is composed of five essential characteristics.
NIST 2010
Cloud ComputingCloud ComputingFive essential characteristics, which differentiate cloud computing from grid computing and other distributed computing paradigms: oOn-demand self-service. provision computing capabilities as needed automatically. oBroad network access. available over the network and accessed through standard mechanisms.oResource pooling. computing resources are pooled with location independenceoRapid elasticity. Capabilities can be rapidly and elastically provisioned.oMeasured Service. automatically control and optimize resource
NIST 2010
Page 8
Cloud Computing Service Model
•On-demand sharing physical infrastructures • Users: System Administrator
•Platform for developing and delivering applications, abstracted from infrastructures • Users: Developer
• Almost any IT services• Users: End-user
Page 9
Clouds Type
Commercial Clouds
Private/Community Clouds
Hybrid CloudsCommercial clouds and private clouds: EC2 Vs Eucalyptus, EC2 Vs OpenNebular
Build by commercial or open-source Solutions
Page 10
Framework
Page 11
Why Cloud Computing
Flexible price model: Pay-as –you-go No ongoing operational expenses No upfront capital
On demand scale up and down
Economics Elasticity
Accessed from anywhere and anytime
with any device
Self-Service Accessibility
User Perspective
Simpler and faster to use cloud service Minimum interaction with the service
provider
Page 12
Improved UtilizationEconomics
Easier for application vendors to reach new customersLowest cost way of delivering and supporting applicationsAbility to use commodity server and storage hardwareAbility to drive down data center operational costServer and storage utilization increased from 10-20% to 70-80%
Why Cloud Computing
Page 13
What are the issues
Many customers don’t wish to trust their data to be in “the cloud” Data must be locally retained for regulatory reasons
Cannot easily switch from existing legacy applicationsEquivalent cloud applications do not exist
Virtualized computing power and networkNot suitable for real-time applications
Page 14
What if something goes wrong? What is the true cost of providing SLAs?
Customers want intuitive GUI, open, standardarized, interoperable APIs Need to continuously add value
SaaS/PaaS models are challenging Much lower upfront revenue
What are the issues
Page 15
Cloud Research
Cloud definition, services
Management
Cloud technologies, solutions, issues,
cost model
Web applicationBig data
HPC applications
General issues
Cloud Optimization
Cloud migration
Future Direction
Across-Cloud implementations
Tools and middleware will be available to enable
interoperability and portability across different clouds
IaaS Become
standardized and
commoditized Add new
utilities and PaaS
capabilities
SaaSIntegrate with
applications
utilizing mobile
devices and
sensors
PaaS
Battleground for
determining the
future of Cloud
Computing
Page 17
VirtualizationWeb service &
SOA, APIs
World-wide
distributed storage
& file system
Parallel & distributed
programming model
Enabling Technology
Architecture
Page 18
VIM (OpenNebula, Eucalyptus,CloudStack)
HypervisorHypervisorHypervisorHypervisor HypervisorHypervisor HypervisorHypervisor
Virtual Machine
Physical Infrastructure
Virtual Infrastructure Middleware (VIM) VM lifecycling Scheduling & monitoring Networking
Cloud Computing for GIScience
Outline
1.Background
2.Case Study 1: Web application
3.Case Study 2: Big data application
4.Conclusion
Background
Many scientific problems are concurrent, data and computational intensive
Case 1: Web application (GEOSS Clearinghouse)
GEOSS Clearinghouse Metadata catalogues search facility for the
Intergovernmental Group on Earth Observation (GEO).
EO data, services, and related resources can be discovered and accessed.
GeoCloud I Governmental cloud initiative Common operating system and software suites Deployment and management strategies Usage and costing of Cloud services Security (certification and accreditation)
Amazon EC2 Cloud
EC2 Instances
XEN Virtualization
Physical Server
Simple StorageService
(S3)
Elastic Block
Storage(EBS)
Hosting of Virtual machine
images(AMI)
Hosting of Virtual machine
images(AMI)
A “Web service that provides resizable compute capacity in the cloud”
Deployment of GEOSS Clearinghouse on EC2 Cloud
0
200
400
600
800
1000
1 20 40 60 80 100 120
Ave
rage
Rep
onse
Tim
e(s)
Concurrent Request Number
GetCapabilities
m1.small m1.large m1.xlarge m2.xlargem2.2xlarge m2.4xlarge c1.medium c1.xlarge
m2.xlarge
m1.xlarge
c1.xlargem2.4xlarge m2.2xlarge
m1.small m1.large c1medium
8/2 8/217:00 17:30
100
50
08/2 8/2
17:00 17:308/2 8/2
17:00 17:308/2 8/2
17:00 17:30
100
50
0
100
50
0
100
50
0
8/2 8/217:00 17:30
8/2 8/217:00 17:30
8/2 8/217:00 17:30
8/2 8/217:00 17:30
100
50
0
100
50
0
100
50
0
100
50
0
Performance in the EC2 Cloud
Lucene (used for indexing while
searching) might be the reason
behind the virtual CPUs under-
utilization.0.38s : 0 record3s: 26, 130 records
MapReduce for indexingSpatiotemporal indexing
Only One Core of the VM is utilized
Usage/Costs in EC2 CloudsTable 6. Monthly Costs of AWS services
Usage chart from July to Nov, 2011
Monthly cost from July to Oct , 2011
Case 2: Big data -> Climate@Home
Input: 150 MBOutput: 2G1 Year, 1
Scenario
100 Year, 1000
Scenarios
10 Year, 100 Scenario
Input: 15 GOutput: 750 G
Computing time per scenario: 45 minutes
Computing time per scenario: 4 days and 16
hours
Run on Community Clouds(NASA Eucalyptus)
Scenario: 300 model configurationVM: 4 – 8 (20 CPU Cores, 64 GB memory) Start date: Dec 1949 End date: Jan 1961
Model Simulation Information
Cloud Computing Information
Platform: Eucalyptus VMs: 4 – 8 (20 CPU Cores, 64 GB memory) Task scheduler: Condor
System CPU Utilization
Provides high-capacity and scalable computing, storage and network connectivity for GIScience applications
Create new opportunities for national, international, state, and local partners to leverage research easily
Conclusion
Acknowledgements
Collaborators: Doug Nebert, Myra Bambacus, Yan Xu,
Daniel Fay, Karl Benedict, Songqing Chen
Team: Qunying Huang, Kai Liu, Jizhe Xia, Zhipeng Gui,
Chen Xu, and all CISC members
I/UCRC for Spatiotemporal Thinking, Computing, and Applications (STC)
Chaowei Yang, Director, GMU SiteKeith Clarke, Co-Director, UCSB SitePeter Bol, Co-Director, Harvard Site
Industry/University Cooperative Research Centers: National Scope, Impact
59 Centers172 I/UCRC Sites
Plus Participating International Sites
ENG CISE
Over 760 Member Organizations (2010)
Academic-Industry partnerships meeting industry sector research needs
Planning Grant Meeting with University Partners, Students, Center Evaluator, Prospective Members and
NSF I/UCRC Program Directors
Planning Grant Meeting with University Partners, Students, Center Evaluator, Prospective Members and
NSF I/UCRC Program Directors
Step 6Step 6Step 6Step 6LOI Step 6Step 6Step 6Step 6Planning Grant Proposal
Events Occuring at the Meeting
Day 1 Day 1
Events Pre Meeting
Events Pre Meeting
Events Post Meeting
Events Post Meeting
Day 2 Day 2
I/UCRC Planning Process
Purpose: Maximize the potential for a successful Center Proposal.
33
Su
cces
sfu
l Pro
po
sal &
1st
IAB
Mee
tin
g
LOI, Planning Grant Pending or
Awarded, what now?
Getting the proposal ready to go!
Planning Meeting Approaching…
Objective
1. Capture and advance human intelligence2. Enable and improve machine processing and
applications 3. Start from geographic science and technologies
for spatiotemporal issues and solutions4. Expand to other domains, such as Earth
science, political science, economics, biology, public health, energy and environment, K-16 education, and others in the future if things went well
Target
1. Improve the US and international spatiotemporal research infrastructure base;
2. Advance the intellectual capacity of the future science, engineering and workforce;
3. Establish the national and international leadership in spatiotemporal thinking, computing, and applications.
Approaches
1. Explore new solutions to our 21st century challenges, such as natural disasters, by investigating the spatiotemporal principles within the challenges with national and international leaders.
2. Advance human knowledge and intelligence by combining spatiotemporal principles and computing thinking to form spatiotemporal thinking as a new methodology and innovative thinking process to enable physical and social science discoveries, and to conduct the next generation computing.
3. Improve interoperability and infrastructure building using the spatiotemporal methods formed to enable the discoverability, accessibility, and usability of big data.
4. Facilitate better understanding of physical and social sciences through phenomena simulation and visualization improved by spatiotemporal thinking.
5. Developing new spatiotemporal computing products in collaboration amongst the center’s members to establish national and international leadership in the field, and transferring the new technologies to companies to improve center members’ efficiency and competitiveness.
NSF I/UCRC Typical Organization
Gray 1998
To ensure the success and sustainability of the center.
•University Management includes VP for Research, Dean for COS, and GGS Chair
•Science Advisory Committee includes international renowned scientists from industry, agencies, and academia
•Industry advisory board comprises sponsor representatives
•Research programs will be dynamic according to progress in the center life cycle
•Each project will include a PI, IAB/sponsor member, and students participating in projects
•A center director assistant or operational director will be assigned at each site
Membership and Benefits
1. Free access to R&D results worth 10+ times by investing $50k+ each year.
2. Increase company and agency’s competitiveness through deliverable oriented partnership with academia and agencies.
3. Access to student talent cultivated through the collaborative research and development projects.
4. Collaborate in an academia, government, and industry environment.
New Proposals
IAB Portfolio
Engagement
IAB Portfolio Engagement
New ProjectsCompleted Projects
Industry/Agency
Advisory Board Needs
Refined
Projects Initial Results
Center Site Strengths
ReviewDiscussAdaptL.I.F.E.
Biannual IAB MeetingBiannual IAB Meeting
Biannual IAB MeetingBiannual IAB Meeting
L.I.F.EReviewDiscuss
AdaptSelect
The co-operative
process rapidly aligns the
Center’s Portfolio with
Member Needs and University
strengths
The IUCRC Research Portfolio CycleL.I.F.E.: Level of Interest and Feedback Evaluation Form
Advancing spatiotemporal computing to enable 21st century geospatial sciences and applications
Experimental Plan, Industrial Relevance and Appropriateness for the center: With the massive amount of spatiotemporal data now available, novel, more efficient approaches for data modeling and management are needed to enable 21st century geospatial sciences and applications. This project aims at developing the theoretical and technical foundations for spatiotemporal computing with a focus on exploiting spatiotemporal principles to build new approaches for data and scientific modeling, indexing, search, and retrieval.Objectives: Develop a novel approach for spatiotemporal computing. This is a four step approach including 1) design and implementation of data structures; 2) algorithms (e.g. indexing methods); 3) spatiotemporal enabled optimized ontology and reasoning methods and 4) search strategy. Team: PIs: Dr. Yang, Dr. Clarke, Dr. Bol, and interested members from agencies and industry, one graduate student at each site. Dr. Rezgui will work as the manager and integrator at the GMU site.
Sample Projects
Four Dimensional space time visualization of tracked movement
Objective: Better visualizing enormous quantities of tracking data collected through innovative geospatial technology developing/using a host of new display techniques have emerged from computer vision, graphics and information visualization that show promise for space-time data. Approaches: or this research project, visualization environments (software programs, tools, code libraries and standards) will be combined with display environments (flat, stereo, augmented virtual and immersive virtual) such that moving objects and fields can be explored. Team: PI: Keith Clarke and Michael Goodchild at UCSB, Phil Yang at GMU, two students with one from each side; Prof. Janowitz will coordinate the research and development from the UCSB site.
Sample Projects
Temporal Gazetteer and Place Name Resolution Service with Temporal Awareness
Objective: develop a new temporal gazetteer and place name resolution service with temporal awareness that (1) compiles and integrate data stored within existing gazetteer systems; (2) enables new crowd-sourced gazetteer entries through a standardized schema; and (3) provide an Application Programming Interface (API). Approaches: 1) Design and implement a comprehensive gazetteer structure; 2) Integrate information from multiple existing gazetteers; 3) Build a web-based entry system to allow crowd-sourced contributions; 4) Design and implement a conflation rule-base to resolve duplicated entries; 5) Publish a user interface for crowd-sourced quality assessment, authorized adjustment of gazetteer entries, and iterative improvement of conflation rules; 6) Build a temporal place name resolution service accessible through API and an online user interface. Team: PI: Peter K. Bol, two technical staff, Dr. Wendy Guan will coordinate the research and development at Harvard University.
Sample Projects
Project objectives: SCC is to develop a middleware that can best arrange and optimize the computing resources and task scheduling by fully considering the spatiotemporal patterns of data, users, cloud computing resources, and geospatial science phenomena. Such an effort would greatly help to construct a better spatial cloud computing (SCC) platform (Yang et al. 2011b) and geospatial cyberinfrastructure (Yang et al., 2010a). We will conduct extensive experiments to explore the spatiotemporal patterns involved in the forecasting of land and atmospheric phenomena, e.g., air quality. We will also experiment with spatiotemporal patterns of users and computing resources, including computing nodes, network and storage. These experiments would provide basic guidelines on how to design the computing platform architecture, select and arrange the geographically distributed computing resources to handle the computations, how to organize and store the data for fast model initialization and output delivery. Team: PI: Drs. Yang, Houser, and two students
Spatial Cloud Computing (SCC) Middleware
Sample Projects
Discussion
RelevancePotential ProjectsCollaboration for customized project