NARSTO Review Panel Report

Subcommittee on Air Quality Research

March 19, 2009

William T Pennell

NARSTO Management Coordinator

Motivation for Review

A request from NARSTO’s Executive Assembly that the Management Coordinator convene a panel to determine whether or not NARSTO provided sufficient value to justify its continuation, and if so, what improvements were warranted.

Panel Members• Carol Henry, USA, chair (consultant in toxicology and risk

assessment, formerly Vice President, Industry Performance Programs, American Chemistry Council)

• Jane Barton, Canada (consultant in environmental management and public policy, formerly Chief, North American Smog Programs, Environment Canada)

• Agustin Garcia, Mexico (researcher, Centro de Ciencias de la Atmósfera-UNAM and professor of environmental engineering)

• John Kinsman, USA (Senior Director, Environment, Edison Electric Institute)

• Ed Piché, Canada (formerly Senior Science Advisor to the Assistant Deputy Minister for Environmental Sciences and Standards, Ontario Ministry of the Environment)

• Rich Poirot, USA (Air Quality Planner, Vermont Department of Environmental Conservation)

Charge to Review Committee

• Assess the quality and value of past and current NARSTO activities

• Assess NARSTO’s distinctive capabilities and their value to NARSTO’s constituencies

• Assess the future need for and value of NARSTO

• Provide recommendations as to a) whether or not NARSTO should continue, and b) if so, what changes or improvements are needed

Methodology

• Interviews: 18 individuals representing EPA, DOE, NAS, NOAA, Environment Canada, INE, a regional AQM organization, and industrial organizations

• Questionnaire: 53 individuals responded representing a broad cross-section of NARSTO constituents

• Citation Analysis: Review Panel conducted research of its own for references to NARSTO in policy actions and scientific articles

Q: 4 NARSTO PARTICIPATION

0

10

20

30

40

50

60

70

80

90

Workshop

Assessments

Author/Contrib

utor

Steering Committe

e

Data Archive User

Other products

Other

In Kind Contributor

Financial Contrib

utor

Field Research

Reactivity

Work

ACTIVITY

PER

CEN

TA

GE

Quality and Value of Past and Current NARSTO Activities

• Survey participants and interviewees value unique opportunity to collaborate with colleagues from other institutions and countries

• Survey participants and interviewees indicate NARSTO assessments and reports useful in planning future research and communicating scientific issues to management

• NARSTO products have impact on policy decisions in all three countries

In the United States

• Frequently cited in standard setting– Criteria documents and Staff Papers for ozone and PM– Clean Air Interstate Rule– New Source Review for PM2.5

– Clean air PM2.5 Implementation Rule and Regional Haze Regulations

• Reactivity Research Working Group results cited in– EPA guidance on control of VOCs in ozone SIPs– Approval of California SIP revisions for control of aerosol

coatings– National reactivity-based VOC emission standards for

aerosol coatings

United States (Cont)

• NARSTO products cited by State and regional AQM groups– National Association of Clean Air Administrators (NACAA)– Ozone Transport Assessment Group (OTAG)– Ozone Transport Commission (OTC)– Regional air quality management organizations (e.g.,

MARAMA, NESCAUM)

From a presentation to OTAGon NARSTO NE by Don BlumenthallIncluded in the Executive Summary Report,OTAG Air Quality Analysis Workgroup

In Mexico

• NARSTO has organized a number of meeting and capacity-building activities in Mexico, including– The international technical symposium, Tropospheric Aerosols:

Science and Decisions in an International Community, Queretaro, Mexico (2000)

– MIT, CEC, SEMARNAT Workshop on Mexico Emissions Inventory (2003)

– Mexican researchers and government officials participated in the NARSTO workshop Innovative Methods for Emissions Inventory Development and Verification (2003)

– Mexico City Technical Symposium on Multipollutant Air Quality Management in Mexico (2008)

• NARSTO has also supported field research projects in Mexico City in 1997, 2003, and 2006

Mexico (Continued)

• Mexican researchers have had significant involvement in the NARSTO Emissions Inventory Assessment and the Multipollutant Air Quality Management Assessment

• The Emissions Inventory Assessment, for example,– Provided important support to the development of Mexico’s

National Emissions Inventory (Inventario Nacional de Emisiones de México)

– Cited in SEMARNAT’s Manual para el Curso de Elaboración y uso de Inventaros de Emisiones

– And in the Information Catalogue for the Mexicali-Imperial Valley Border Region

In Canada

• Ozone Assessment provided support for the addition of ozone and its precursors to the list of Toxic Substances in the 1999 Canadian Environmental Protection Act

• NARSTO results used in the Canadian Council of Ministers of the Environment review of Canada-Wide Standards for Ozone and PM and for review of Canadian Measurement Networks

• Province of Ontario cited NARSTO products in developing its Proposed Performance Indicators for Ontario’s Anti-Smog Action Plan

• And in preparing Ontario’s legal submissions to the US EPA on transboundary air pollution

In Bilateral and Trilateral Activities• Transboundary air pollution is a fact affecting air quality

management in all three NARSTO countries• NARSTO’s contributions to understanding to improving

understanding of this problem was recognized as valuable (and even unique by some respondants), for example– Canada-US Air Quality Committee relied heavily on the NARSTO PM

Assessment in developing the Canada-United States Transboundary Particulate Matter Science Assessment and in negotiating a Particulate Matter Annex to the 1991 US-Canada Air Quality Agreement

– Canada-US Air Quality Committee cited NARSTO activities and assessments in each of its biennial progress reports 1998-2008.

– NARSTO’s work on methods and formats (e.g., in emissions inventories and modeling) have set the stage for establishing uniform continental-wide conclusions regarding air quality

– The 2006 MILAGRO and MAXMex field campaigns initiated assessment on Mexico City’s effect on regional climate

In the Scientific Literature

NARSTO has been referenced in more than 500 publications in more than 100 journals between 1997 and 2009

0

10

20

30

40

50

60

70

80

1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009

Articles Conferences Reviews

NARSTO Citations in the Scientific Literature, by year

And Elsewhere

A web search on “NARSTO” yields– 23,200 hits on Google– 119,000 hits on AlltheWeb– 4,330 hits on Search.com

Ranking of NARSTO Activities

Ranking of 11 NARSTO activities: 1 is most valuable 11 least valuable

NARSTO ActivityNumber

RespondingAverage

RankHighest Rank

Lowest Rank

Assessments 44 1.8 1 5

Reports 37 3.5 1 11

Meetings/Workshops 38 3.7 1 9

Networking Opportunities 31 4.6 1 10

Regional Emission Inventory Workshops 23 5.1 1 11

Data Archive 30 5.3 1 11

Atmospheric Model Inter-comparisons 26 5.3 2 10

Website 28 5.8 1 9

Executive Assemblies 24 6.2 2 11

Measurement Methodologies 23 6.2 2 10

Reactivity Working Group 15 6.5 1 11

05

10152025303540455055606570

VeryValuable

SomewhatValuable

NotValuable

Don'tKnow

EMISSIONSINVENTORY 2005

PARTICULATEMATTER 2004

TROPOSPHERICOZONE 2000

NARSTO’s Distinctive Capabilities and Their Value

• NARSTO unique in its tri-national scientific capabilities • Enables collaboration among government, industry,

and academic experts in all three countries• NARSTO’s organization is streamlined, non

bureaucratic, flexible and transparent• Data archive is seen as useful• No other organization in North America encompasses

NARSTO’s scope, for example– NAS covers some of the same US issues, but involves

process that is more time-consuming and costly– NAFTA’s CEC is tri-national, but not equipped to take on in-

depth scientific work

05

101520253035404550556065707580859095

100

Yes No No Opinion

ABCDEFGHIJKL

A Improves communication B Tri -national organizationC Enhances cooperationD Improves Credibility E Encourages interactions F Minimal bureaucracyG Enhances technology transferH Resource leveragingI Timely /responsive

J Improves Flexibility K US centric focusL Minimizes Bias

Future Need for NARSTO

• Most interviewed or surveyed stated that NARSTO should be continued. No one called for its abolition

• A substantial number of interviewees and questionnaire respondents identified problems if NARSTO ceased to exist

S C I E N T I F I C I S S U E S O F F U T U R E I M P O R T A N C E I D E N T I F I E D

B Y R E S P O N D E N T S A N D I N T E R V I E W E E S

0

5

1 0

1 5

2 0

2 5

3 0

Air Quality - Climate Interface

Long Range Transport

Measurement/Monitoring

Ozone and Particulate Matter

Air Quality Management - Process

Air Quality Modeling

Emission Inventories

Toxic Chemicals

Nitrogen/Reactive Nitrogen

Ecosystem/Ecological Effects

GHG Policy

Biofuels

Climate Models

Exposure AssessmentAcid Rain and Visibility

Atmospheric Processes

NUMBER

Issues Raised

• Funding model needs to be addressed -- too dependent upon DOE, unbalanced among three countries

• Without core funding, NARSTO cannot function• Policy-makers who set priorities are less involved in NARSTO

than before• Institutions and other clients are not aware of the value and

relevance to their work of NARSTO products• Involvement of key clientele in planning is necessary to ensure

that NARSTO projects remain relevant to NARSTO members• As individuals involved in the founding of NARSTO retire or

move on, their level of commitment has not always been sustained by their institutions

Recommendations

• NARSTO should be continued• Formalize institutional support for NARSTO• Increase participation by policy-makers• Strengthen NARSTO’s strategic work

planning• Assure funding for core activities• Review and assess NARSTO’s organizational

structure and partnerships