Date post: | 01-Apr-2015 |
Category: |
Documents |
Upload: | clarence-lones |
View: | 217 times |
Download: | 1 times |
Presenter:
Cindi Dunn, Principal Evaluator
Office of Educational Innovation & Evaluation
College of Education, Kansas State University
June 13, 2013
EVALUATION AND ASSESSMENT
3RD PARTY EXTERNAL EVALUATION
NORTH GULF COASTAL HAZARDS COLLABORATORY TRACK II
“AREAS OF SUCCESS” METRICS
1. Cyberinfrastructure Improvement
(Cyberinfrastructure Development; Simulation
Experiments)
2. Research Production
3. Research Collaboration
4. Human Resource Development (Outreach
and Education Activities; Diversity)
5. Sustainability for Future Cyber-Enabled
Research and Innovation
DATA SOURCES
1. DRAFT Third Annual Report to NSF
2. Project Documents on Portal
3. NG-CHC Retreat held October 2012
4. NG-CHC Annual Conference June 2013
5. Collaboration Survey Spring 2014
6. Portal Analysis Spring 2014
7. Reports and other documentation
“AREAS OF SUCCESS” METRICS
1. Cyberinfrastructure Improvement
(Cyberinfrastructure Development;
Simulation Experiments)
2. Research Production
3. Research Collaboration
4. Human Resource Development (Outreach
and Education Activities; Diversity)
5. Sustainability for Future Cyber-Enabled
Research and Innovation
Number of Researchers and Students Supported by NG-CHC
1. CYBERINFRASTRUCTURE AND EXPERIMENTS
Researchers Post-Doctoral Researchers
Graduate Students Undergraduate Students
0
10
20
30
40
50
38
6
15
1
48
6
20
4
34
7
27
8
Year 1 Year 2 Year 3
1. CYBERINFRASTRUCTURE AND EXPERIMENTS
Metric Data Status Year 3
Number of high performance computing services and tools in NG-CHC
Operational Storm Surge Forecast System (MSU)
Data Grabber to Download Coastal Data (LSU)
Data Merging Tool (MMRI) Data Extraction Tool (MMRI) MODIS Daily Imagery Download Too (MMRI) ASGS-NHC Text Advisories to ATCF format ASGS-ADCIRC ASCII to NetCDF script Sulis Metadata Creator <beta> EnVis Spatial Watershed Erosion Tool FloodVix ADCIRC to ESRI Shape File Scripts Data to NetCDF Scripts Sulis Portal Viewer Sulis Severe Weather Viewer Sulis Informatics Services Portal Weather Underground Web Data Extraction
Tool (WUDEXT) Coastal Datafactory Multi-layer Data Infrastructure for System of
Systems
1. CYBERINFRASTRUCTURE AND EXPERIMENTS
Need to define and develop measures
Metric Data Status
Functionality of each cyber-enabled tool and service
Year 1 = Stage 1 (organizing)Year 2 = Stage 2 (defined workflow)Year 3 = Stage 3 (product)
• What can users do now that they couldn’t do prior?
• What feedback have users provided?
Usability of each cyber-enabled tool and service
Functionality of the ASGS, SULIS, and Flood Risk Models
Usability of the ASGS, SULIS, and Flood Risk Models
Usability of Technologies developed from the ASGS, SULIS, and Flood Risk Models
1. CYBERINFRASTRUCTURE AND EXPERIMENTS
Need to define and develop measures
Metric Data Status
Quantitative performance of each cyber-enabled tool and service
No data reported any year
Number of data analysis services developed and deployed in NG-CHC
No data reported any year
Capacity and performance of distributed storage systems in NG-CHC
Developed by Mississippi in Year 1
Number of virtual organizations and communities established in NG-CHC
3 simulation experiments
Number of cyber-enabled applications supported by NG-CHC
No data reported any year
Number of courses supported by NG-CHC
One reported in Year 1; no data reported in subsequent years
“AREAS OF SUCCESS” METRICS
1. Cyberinfrastructure Improvement
(Cyberinfrastructure Development; Simulation
Experiments)
2. Research Production
3. Research Collaboration
4. Human Resource Development (Outreach
and Education Activities; Diversity)
5. Sustainability for Future Cyber-Enabled
Research and Innovation
NSF No NSF NSF No NSF NSF No NSFYear 1 Year 2 Year 3
0
5
10
15
20
25
30
35
64
10
6
14
3
3
4
2
4
12
4
14
Alabama Louisiana Mississippi
Number of Publications in Major
Scientific Journals
2. RESEARCH PRODUCTION
Published or accepted
1
9 8
25
14
30
Presentation Presenters Presentation Presenters Presentation PresentersYear 1 Year 2 Year 3
0
5
10
15
20
25
30
35
40
45
50
4 4
17
610
3
9
13
15
3
7
3
13
4
19
4
Alabama Louisiana Mississippi
2
Number of Presentations at Major
Peer-reviewed Conferences
2. RESEARCH PRODUCTION
20
9
43
12
44
10
Need to identify peer-reviewed
2
Number of Products and Other Types of Intellectual Property
2. RESEARCH PRODUCTION
Year 1 Year 2 Year 3
Website Website Website
Portal Portal Portal
Structured Data Set via THREDDS
Server
Tools applicable?
Number and Types of Media Exposure
2. RESEARCH PRODUCTION
Year 1 Year 2 Year 30
1
2
3
4
5
6
7
8
9
2
3
1
4
2
1
2 2
3
1
Radio TV News print Web
5
8
Awards Awardees Awards Awardees Awards AwardeesYear 1 Year 2 Year 3
0
2
4
6
8
10
12
14
16
18
9
4
9
24
3
3
1
4
1
11
4
4
4
1
Alabama Louisiana Mississippi
Number and Types of Awards and Honors
2. RESEARCH PRODUCTION
12
5
17
79
5
Advisory Boards
Editorial Boards
Fellowships
“AREAS OF SUCCESS” METRICS
1. Cyberinfrastructure Improvement
(Cyberinfrastructure Development; Simulation
Experiments)
2. Research Production
3. Research Collaboration
4. Human Resource Development (Outreach
and Education Activities; Diversity)
5. Sustainability for Future Cyber-Enabled
Research and Innovation
Year 1 Year 2 Year 3 Total0
20
40
60
80
100
120
140
160
180
200
19 1642
27 2923
79
12 2231
65
Multi-State Multi-Institutional External
7
Number of Multi-State/Institution Research Projects (per year and total)
3. COLLABORATION
58 67 61
186
Number of Researchers from Multi-State/Institutions (per strategy* and total)
3. COLLABORATION
66
*Faculty could be in more than one strategy
38
86
Year 1 Year 2 Year 30
10
20
30
40
50
60
70
80
90
100
10
30
1813
32
138
6
134
8
17
10
CI1 CI2 CI3 CI4 CI5
53
Quality of Collaboration (per strategy* and total Year 1 = 3.8; Year 2 = 4.1)
3. COLLABORATION
Governance , Administration, Mutuality (Y2H), Norms (Y1H)
CI5:E&O
CI4
CI3
CI2.4 River
CI2.3 ERFRR
CI2.2 SULIS
CI2.1ADCIRC
CI1: DDEM
0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.003.16
3.73
3.72
4.59
3.98
4.02
3.52
3.80
3.91
4.11
3.88
4.57
4.00
4.05
4.01
4.02
Year 2 Year 1
Overall Quality of Collaboration (per strategy* and total)
3. COLLABORATION
Satisfaction Commitment Efficiencies Productivity Overall Collaboration1.00
1.50
2.00
2.50
3.00
3.50
4.00
4.50
5.00
3.944.38 4.18 4.09 4.15
3.72
4.444.17
3.87 4.04
Year 2 Year 1
Number of Publications Produced by Collaborative Efforts (NSF acknowledged)
3. COLLABORATION
Year 1 = 67%; Year 2 = 73%; Year 3 = 89%
No collaboration reported
Another department at the institution
Another institution in the state
Institution outside the state (in U.S)
Institution abroad0
1
2
3
4
5
6
7
8
9
10
322
6
2
5 5
2
9
5
3
Year 1 Year 2 Year 3
Number of Collaborative Research Proposals Developed by NG-CHC
3. COLLABORATION
Year 1 Year 2 Year 3
NG-CHC 1 Not evident Not evident
Multiple Partners 2 Not evident Not evident
External Partners 3 3 4
Need to define and develop measures
“AREAS OF SUCCESS” METRICS
1. Cyberinfrastructure Improvement
(Cyberinfrastructure Development; Simulation
Experiments)
2. Research Production
3. Research Collaboration
4. Human Resource Development (Outreach
and Education Activities; Diversity)
5. Sustainability for Future Cyber-Enabled
Research and Innovation
*Graduate Students in NG-CHC Research Program
*Number of Students Recruited for Project that Persist in STEM
4. HUMAN RESOURCES
Year 1 Year 2 Year 3
NG-CHC Graduate Students
15 20 35
Earned Ph.D. 3 2 4
EarnedMasters 0 4 3
Earned Bachelors 0 2 0
Numbers of Minorities in Outreach and Education Activities
4. HUMAN RESOURCES
Year 1 Year 2 Year 3
Education Activities
(UG/GRA)
3 women4 URM
8 women7 URM
8 women3 URM
OutreachSTEM Bowl
32 URM No report No report
Need values from Outreach Activities
*Number of Summer Workshops for Junior/Senior High Schools
4. HUMAN RESOURCES
Year 1 Year 2 Year 3
• Beowulf (40)• Northern Gulf
Institute
• STEM Bowl Scholarship Competition (37)
• Lockheed Martin IT Day/STEM Bowl Scholarship Competition (1000)
• STEM Bowl Scholarship Competition
• Teachers Exploring Coastal Hazards and Resilience (20)
• Teachers Exploring Coastal Hazards and Resilience
• Dauphin Island Sea Lab (19 teachers)
# of participants and satisfactory levels not reported or limited.
*Number of Summer Workshops for Undergraduate/Graduate Students
4. HUMAN RESOURCES
Year 1 Year 2 Year 3
• Evidence not explicitly stated
• Delft3D Tutorial
• Evidence not explicitly stated
• USGS National Wetlands Research Center Modeling Teams
• NG-CHC Student Poster Session (15)
# of participants and satisfactory levels not reported or limited.
*Number of faculty and staff trained in NG-CHC.
*Number of students trained in NC-CHC.
*Number of new courses.
4. HUMAN RESOURCES
Define and obtain existent data.
“AREAS OF SUCCESS” METRICS
1. Cyberinfrastructure Improvement
(Cyberinfrastructure Development; Simulation
Experiments)
2. Research Production
3. Research Collaboration
4. Human Resource Development (Outreach
and Education Activities; Diversity)
5. Sustainability for Future Cyber-Enabled
Research and Innovation
*Number of funded research clusters & other organizations established.
*Number of researchers and graduate students in each established center.
5. SUSTAINABILITY
Define and obtain existent data.
Year 1 Year 2 Year 30
1
2
3
4
5
6
7
8
9
10
4
1
2
2
3
2
2
Alabama Louisiana Mississippi
Number of Successful Awards from NSF and Other Agencies
5. SUSTAINABILITY
$2,604,374
$1,083,000$210,000
EPAFESD
IndustryNASA
NSFNOAA
“AREAS OF SUCCESS” METRICS
1. Cyberinfrastructure Improvement
(Cyberinfrastructure Development; Simulation
Experiments)
2. Research Production
3. Research Collaboration
4. Human Resource Development (Outreach
and Education Activities; Diversity)
5. Sustainability for Future Cyber-Enabled
Research and Innovation