Date post: | 13-Dec-2015 |
Category: |
Documents |
Upload: | jordan-reed |
View: | 212 times |
Download: | 0 times |
US Particle Accelerator School
Review of
Scientific User Facilities Division
Committee of Visitors
Presented by W. A. Barletta, Chair
US Particle Accelerator School
Top level charge
Assess processes used during FY2010, 2011, & 20121. to solicit, review, recommend, & document proposal actions
2. to monitor active projects and programs.
Program elements to assess1. Light Sources including the Accelerator & Detector Research
Program
2. Neutron Sources
3. Nanoscale Science Research Centers & Electron Beam Micro-Characterization Centers
4. Construction Projects & MIEs
Report to the Summer 2013 BESAC meeting
US Particle Accelerator School
Charge specifics
(1) For scientific user facilities including the accelerator & detector program, Assess the efficacy Assess the quality of processes used to:
• (a) Solicit, review, recommend, & document proposal actions (b) Monitor active projects, programs & facilities
(2) Within boundaries set by DOE missions & funding, Comment on how the award process has affected:
• (a) Breadth & depth of portfolio elements• (b) National & international standing of portfolio elements
US Particle Accelerator School
BES SUFD Committee of VisitorsDr. William Barletta (MIT/USPAS), Chair
Construction Projects James Krupnick (LBNL, Ret.) Angus Bampton (PNNL) Jeff Hoy (Trident Service LLC) Maria Dikeakos (DOE PPPL)
Nano-Science & E-Beams Prof. Donald Tennant (Cornell) Prof. Beatriz Roldan Cuenya
(UCF) Dr. Ernie Hall (GE) Dr. James Liddle (NIST)
Light Sources, Accelerator & Detector Research
Dr. Simon Bare (UOP) Prof. Nora Berrah (WMU) Dr. Gene Ice (ORNL) Dr. Joel Ullom (NIST) Dr. David Robin (LBNL)
Neutron Facilities Prof. Sunil Sinha (UCSD) Dr. Robert Dimeo (NIST) Prof. Thomas Russell (UMass) Dr. John Tranquada (BNL)
Chairman in RedBESAC Member in Italics
Chairman in RedBESAC Member in Italics
US Particle Accelerator School
Overarching conclusions
We commend SUFD for effective use of its available funding to construct & operate a system of facilities which deliver world-leading science
The efficacy of SUFD’s processes to review, recommend & document proposal actions are excellent
The efficacy of the processes to monitor & review active projects, programs & facilities are also excellent
SUFD staff members are to be commended for rigorous & effective program management
Comment: BES growth over the decade is impressive
US Particle Accelerator School
Overarching conclusions
Within the scope of DOE missions & available funding, SUFD’s award processes continue to enhance the breadth & depth of portfolio elements as well as their national and international standing
International competition in scientific user facilities is fierce; maintaining scientific leadership will require increased investments for the user facilities for user support
US Particle Accelerator School
SUFD-wide findings & recommendations - 1
A. Implementation of previous COV recommendations FINDING: SUFD has effectively addressed the majority of the
recommendations made in the previous COV report FINDING: SUFD travel budget continues to be incommensurate
with most effective oversight of its program
RECOMMENDATION: Enhance the effectiveness of program oversight by increasing the flexibility of SUFD managers
• to interact with facility managers • to communicate with the facilities staff via increased on-site
presence
US Particle Accelerator School
SUFD-wide findings & recommendations - 2
B. Assessment of COV process effectiveness FINDING: Providing all data on computers to each reviewer is
highly responsive to previous recommendations & greatly improves the COV process.
• The electronic documentation was thorough and well organized
RECOMMENDATION: The move toward a full, searchable database is commendable. It should be available to the next COV
US Particle Accelerator School
SUFD-wide findings & recommendations - 3
C. Facility review process description & effectiveness FINDING: The 3-year reviews of the facilities are well organized
and well executed with appropriate review teams FINDING: A uniform definition of high impact publications has
been established for light sources & neutron sources. RECOMMENDATION: Finalize the set of uniform definitions
of high impact publications for the nanoscience centers
US Particle Accelerator School
SUFD-wide findings & recommendations - 4
D. General Issues FINDING: The quality of the scientists at the facilities is the
critical asset that ensures excellence & success. RECOMMENDATION: Place added emphasis on career
development as well as on maintaining state-of-the-art experimental apparatus, sample environment & software at all facilities to maximize scientific productivity
FINDING: Different types of facilities serve different scientific communities. They are all needed & important.
RECOMMENDATION: New metrics that account for scientific impact should apply to all the types of scientific user facilities
US Particle Accelerator School
SUFD-wide findings & recommendations - 5
Recommendations: If an increased fraction of industrial users is desired, this fact should be clearly communicated to the management of facilities Proposal evaluation should include criteria that value factors such
as economic & technological impact Thorough review of user agreements should be undertaken with a
view to removing barriers to industry users
US Particle Accelerator School
Light sources
and
Accelerator & Detector Research
US Particle Accelerator School
Light source findings - 1
The review process is comprehensive, balanced, fair, and transparent to all facilities They are all reviewed using the same major criteria.
The number and breadth of the reviewers for each facility was appropriate
Reviews provided important comments & specific actionable recommendations where appropriate.
The reviews were uniformly forthright, detailed, & contained a summary section at the beginning of the review The detail in review comments indicated a deep appreciation of
important issues that went beyond the documentation solicited. This indicates the importance of the on-site review.
US Particle Accelerator School
Light source findings - 2
Review follow-up was not documented between reviews RECOMMENDATION: A formal yearly follow up of
facility recommendations should be documented each year Ideally this could be a short response saying all issues had been
previously addressed when appropriate
Recommendations from the prior reviews were not sent to the facility reviewers
RECOMMENDATION: Make previous review recommendations & facility response available to the facility reviewers at the outset to allow the review committee to assess how the facility has
responded to the prior recommendations.
US Particle Accelerator School
Accelerator & Detector Research - Findings
Resources available are so limited ~$10M (~1% of total
SUFD) that the breadth is necessarily small (focusing primarily on FELs) This figure is too small to maintain scientific leadership
Projects are generally excellent & matched to SUFD needs At present the portfolio does not include x-ray optics
US Particle Accelerator School
Accelerator & Detector Research - Recommendations
Increase the ADR portfolio to $20M to $30M per year (2-3% of SUFD budget)
Consider the concept of a HUB to advance ADO technology in support of its scientific mission
Add X-ray optics to the ADR portfolio
As part of increasing the portfolio, formalize the proposal solicitations
Continue using workshop reports to guide research initiatives & to shape investment priorities
Develop topic-specific metrics to assess / characterize US capabilities in accelerators, detectors & optics
US Particle Accelerator School
Neutron sources
US Particle Accelerator School
Neutron sources
Finding : Few instruments in ORNL’s Neutron Sciences Directorate operate in the steward-partner model.
RECOMMENDATION: Strongly encourage neutron scattering facilities to explore forming partnerships on instruments with potential partners from other agencies in the cooperative stewardship model Exploit neutron scattering capabilities to benefit the broadest
possible scientific community
Finding: It is unreasonable to frontload facilities with understaffed instrumentation or beamlines
Recommendation: When an MIE is being considered, a the facility should have well-designed plan to ensure its robust, long-term operation for users
US Particle Accelerator School
Neutron sources & neutron science
Finding: Trends in the science enabled by neutrons have evolved significantly since last studies (1993, 2002) on the future of neutron science in the US
Recommendation: We recommend that BES join with other agencies, such as DOC, NSF, & NIH, to assess current status & future directions for U.S. neutron science Include factors, such as neutron measurement capacity &
capabilities and present & future needs of the U.S. scientific community
US Particle Accelerator School
Nanoscience Research Centers &
E-beam Microcharacterization Centers
US Particle Accelerator School
NSRCs & EBCs - General Observations
Finding: NSRCs are entering a post-ramp up phase in which they face issues of high demand for facilities & instruments & over subscribed staff
Recommendation: Provide guidance to NSRCs to plan for expansion of facilities or expanded operating hours Alternatively adopt higher rejection rates of user proposals
US Particle Accelerator School
Finding: SUFD program managers have done a commendable job at establishing thorough & transparent processes for evaluating ongoing projects The 3-year review cycle might be excessive for timely corrective actions
Recommendation: Increase face-to-face to time between DOE officials & management, scientific staff, & user community of the NSRCs &EBCs, including regular (yearly) on-site visits
Methods of assessing NSRCs have not evolved since the centers were first commissioned
Recommendation: NSRCs (& EBCs) are sufficiently different from light sources to warrant tailored metrics & assessment methods (user surveys)
NSRCs & EBCs - General Observations
US Particle Accelerator School
NSRCs & EBCs - General Observations
BES recommended merger of EBMCs & NSRCs in response to a triennial review finding that EBMCs were understaffed & underfunded to carry out their mission. This recommendation focuses on improvements in synergy &
operational efficiency.
Recommendation: SUFD should ensure that facility merger plans focus clearly on these improvements
US Particle Accelerator School
High Utilization Instruments
Finding: The unique, world-leading instruments associated with the EBMCs are in high demand but are not utilized optimally Staff funding is currently for 40 hour week, yet the labs are
open more than 8 hours per day.
Recommendation: Merger plans should include expanded staffing (>8 hr/day) on select tools
US Particle Accelerator School
Industry Participation
NSRCs have not attracted a significant number of industry users
Recommendations: If an increase in the fraction of industrial users is desired, this fact should be clearly communicated to the NSRCs Proposal evaluation should include criteria that value
potential technological or economic impact Thorough review of user agreements should be undertaken
with a view to removing legal barriers to industry users
US Particle Accelerator School
CONSTRUCTION PROJECTS
US Particle Accelerator School
Findings regarding project management
Comment: PM processes are robust & successful in delivering projects on schedule, within budget & meeting baseline technical performance parameters SUFD met, and often exceeded, BES goal of remaining within +
/ - 10% for cost/schedule performance Project management performance metrics have become even more
visible indicators of DOE performance for external stakeholders
Recommendation: Ensure that CD4 requirements are reasonable, broadly understood by all stakeholders, & fully achievable within the project budget Effort should be made to manage & align expectations for
what constitutes successful initial scientific operations
US Particle Accelerator School
Impact of travel reduction
Comment: SUFD management should determine the correct level of field presence for Program Managers to provide adequate Federal oversight, operational awareness, & fosters strong and open communication between field and HQ elements On-site presence, graded to project risk, should be appropriately
balanced with use of remote communication tools to live within constrained budgets
Recommendation: Mitigate negative impacts of reduced travel funds Balance field presence with communication technology to
maintain robust communication between program managers & the on-site members of IPTs
US Particle Accelerator School
The COV thanks
Jim Murphy and the entire SUFD staff
for their candor & responsiveness
Special thanks to Linda Cerrone
for her help in making this review
easier and more pleasant