SEMINAR ON STATUS OF MEASURE LIFETIME WORK

RTF EUL Kickoff Meeting, August 2011

Lisa A. Skumatz, Ph.D.Skumatz Economic Research Associates

skumatz@serainc.com, 303/494-1178

SERA

PERSISTENCE / RETENTION

SERA

RETENTION / PERSISTENCE Persistence, measure life, EULs, 50% median,

in place and operable Protocols, best practices summaries on

samples, data collection, analysis, modeling, comparisons, documentation

Some variations in considering adaptations for behavioral programs

SERA

4

REMAINING USEFUL LIFETIME (RUL)

Conceptual issue – early replacement Intervene at replacement standard vs. efficient Intervene early original vs. efficient for “early” part standard vs. efficient for later period Question is, length of time from end of life? Few studies; 1/3 ad hoc Welch & Rogers 2010;

Survey, Weibull, curves for residential equipment System dynamics / stocks cohorts

Strong application for behavioral adoption curves / timing / lifetimes? Measurement issue

SERA

5

TECHNICAL DEGRADATION Technical degradation (TDFs)

Addressed in CA-EM&V protocols Differences in decay? Very few primary studies 2 effects - Technical degradation & behavioral /

operational based on quality of use & upkeep – need studies on combination

Behavioral very parallel; no studies

SERA

LIFETIMES / EULs STUDIES CPUC

Protocols, results, measures with gaps Updated lifetimes in report

All sectors, many measures incorporated into DEER (database for energy efficient resources, energy.ca.gov/deer), protocols

Reviewed program, savings, measures, sampling, field work, data validation, analysis, justifiability Strengths, weaknesses, “score” Hundreds of millions in shareholder claims

SCE / DEER CPUC / CIEE – nationwide review Individual studies

SERA

SCORING CRITERION ON CPUC RETENTION PROJECT-SUMMARY

Criterion Description

CPUC Protocol

Did they meet the CPUC protocol of using the top 10 measures or measures that account for 50% of the program savings? A score of 3 means that they just met the criteria, but the discussion of the measures selected and the program savings may have been weak; 5 was the highest score possible.

Sampling Strategy

Describe the sampling strategy whether it was stratified, random, etc. Describe the sampling and stratification basis and process. (i.e., customers, rebates, installations, etc.) Is there adequate discussion of the sampling process? Discuss problems with the strategy / basis if appropriate. A score of 3 indicated that the sampling strategy and basis was sufficient.

Fieldwork and Validation

What type of fieldwork was used to collect the data?How well is the data collection process explained? Describe the field work and validation of the data. This score reflected the adequacy of the field work done in collecting data and the thoroughness with which the data was validated.

Methodology Is the description of the methodology adequate? Discuss the methodology implemented in the study. Were alternative models estimated or considered? Was the final selection of models well justified? Was data attrition sufficiently explained? Does the study adequately discuss the results of its model?

SERA

EUL STUDY – BEST PRACTICES ISSUES Small sample size

Population list problems; needs vary by lifetime of measure

Omitted other models Potential variations by measure type; easy to

add; improves potential fit; not a priori Ambiguous failure dates

Frequent surveys; “bound”, labeling, phone survey for some

Poor documentation in report Methodology, weighting, failures, conclusions

Results not put in context Over time; other studies

SERA

9

EUL VALUES USED IN US - RESIDENTIAL

Lighting HVAC Shell & OtherLighting – CFL Bulbs: 6-8 years, with some recent work starting to incorporate variations based on assumptions about hours per day that the bulb operates

Hardwired fixtures – 15-20 years for interior and exterior fixtures

Lamps (table or touchier) – 5-10 years for most studies[1], depending on type

Occupancy sensors – 10-15 years

HVAC replacement – 15-25 years

HVAC and water heating in Energy Star – 15-25 years

Room A/C – 11-15 years

Programmable thermostat – 10-12 years

Whole house fans – 25 years

Attic ventilation fans with thermostat controls – 19 years

Duct sealing and air sealing – each 15-20 years

Insulation 20-25 years

Duct insulation – 20 years

Windows – 20-35 years

Weatherization – combined measures – 20-25 years

Pipe wrap – 10-20 years

Tank temperature turn down – 4-7 years

Note – some differences in values between CA, NW, NE values – Especially usage / weather based…

SERA

10

EUL VALUES USED IN US - COMMERCIAL

Lighting HVACLighting – CFL Bulbs: 3.4-6 years with some recent work starting to incorporate variations based on assumptions about hours per day that the bulb operates in business locations

Fluorescent fixture – 11-16 years

Hardwired CFL – 10-15 years

HID (interior & exterior) 13-15 years

Occupancy sensors – 8-15 years

Daylighting dimming – 9-10 years

Packaged AC / HP – 12-15 years

Chillers – 19-23 years

Economizers – 7-15 years

Energy Management Systems (EMS) – 10-15 years

Motors – 13-20 years

SERA

11

ISSUES IN EULs Process values lacking (small sample size) Some dependent on operating assumptions Some end-uses missing / gaps

CFLs, lighting – updated with operating hours Measures with limited (unreliable or zero) studies – some with

much attributable savings Models of air compress, A/C, cook, hvac, refrig/freezer,

process, shell, (motors/pumps, dryers, lighting), ASD/VSD in some sectors

Missing for plug loads Building shell – at least verify Priority depends on future savings, rarity, variations; waiting

hurts EUL data Trend toward simplified tables, BUT research shows strong

variations in turnover by business type

SERA

12

EULs FOR BEHAVIORAL PROGRAMS Missing for behavioral / educational programs

2 studies CBSM Best practices with nuances - Partial retention;

frequency of data collection; large surveys and random assignment

Retention of “upstream” complicated

SERA

MEASURE LIFETIME ESTIMATES / RETENTION

Behavioral / Outreach Programs

Examples Only 1-2 examples (Harrigan), short term; a few recent additions (Peters)

Gaps / Issues Missing for behavioral, education/ training, commissioning, othersImportant –parallel concept to technical degradationNeed “partial” retention conceptLack of exploration; Months? Years?Lack of existing research on behavioral retentionRemaining useful lifetime / baseline issues

Best Practices /Rec’m for Behavioral / Outreach

Presumably similar approach, but consider data acquisition differences and data uncertainties

SERA14

VARIATIONS IN PERSISTENCEEULs

General results Early work in NW – gravitation to CA values. CA requires ex post statistical verification, but for subset of measures – led to refinements. Some measures with inadequate / missing – especially behavioral

Variations by Program type

Almost all EUL results are by measure, not by program design or incentive provided. Not clear if they should vary by program type.

Variations for behavioral vs. measure-based programs

Missing for behavioral programs including education / training, commissioning training, and similar programs.

SERA

15

ISSUES / PROBLEMS Best practices Results / gaps TDF RUL Behavioral Key component of program savings

Potential bias away from new, creative Risk Complexities for behavioral

Little primary research / dormant / agreement

SERA

16

CONCLUSIONS – MEASURE LIFETIMES

Measure lifetimes are a key element in the calculation of energy savings from energy efficiency programs

Measure lifetimes (and methods) are fairly consistent for many measure-based programs in residential and commercial sectors Issue of simplified EUL tables / caution

Shortage of primary research on technical degradation (TDF); shortage of research on RULs

Virtual absence of studies addressing retention or persistence of energy savings from behavioral and education programs

Identifying the measure lifetimes of behavioral and education programs is complicated as more media messages on behavior and education “bleed” across territories

SERA

17

RECOMMENDATIONS – MEASURE LIFETIMES

Conduct measure lifetime studies on: Process equipment, some shell measures, cooking, refrigeration,

and air compressors

Conduct technical degradation studies that account for mechanical and behavioral performance-related changes

Conduct studies on retention or persistence of energy savings from behavioral and education programs

Require new behavioral programs to conduct retention assessments every year or two

Apply different evaluation methods to a variety of behavioral programs

SERA

NEXT STEPS / RESEARCH NEEDED

Behavioral / Outreach Programs

Retention Studies of retention of behavioral Consider data acquisition differences and data uncertainties (date?, data collection frequency)Best practices with nuances – partial retention, retention of “upstream” complicatedRUL researchComparison of results for key factors affecting retention before values can be adopted / transferred as with measures currently (quality? Audience?, other?)

SERA

CONTEXT DISCUSSION Investment dollars at risk Debate over precision

Granger – evaluation to avoid making wrong decisions Multiple applications… varying precision needs?

Program decisions to be advised include: Public dollars responsibly spent Apportionment of dollars between strategies

Precision based on value / cost of possibility of wrong decision… Yes/No vs. precise level of shareholder dollars…

SERA

20

KEY USES OF EVALUATION RESULTS / PRECISION NEEDS

By Use / Application By Considerations

Assessing progress Uncertainty small, low cost, small value implications, e.g.small resid programs

Information on C/E

DSM planning input (tradeoff)

Paying participant

Paying utility incentive

Alternative supply High certainty needed, large impacts, large cost

Deemed

Increasing Rigor(& cost)

Detailed M&V, site verification, +

SERA

21

Persistence

SERA

QUESTIONS?

Project Manager contact:Lisa A. Skumatz, Ph.D. Skumatz Economic Research Associates (SERA)762 Eldorado Drive, Superior, CO 80027Phone: 303/494-1178Email: skumatz@serainc.com