Consumer price index manualT h e o r y a n d p r a c t i c e

Price: 200 francs

Consumer pr ice index manual :T h e o r y a n d p r a c t i c e

The consumer price index (CPI) measures

the rate at which the prices of consumer goods

and services are changing over time.

It is a key statistic for purposes of economic

and social policy-making, especially monetary

policy and social policy, and has substantial

and wide-ranging implications for governments,

businesses and workers, as well as households.

This important and comprehensive manual

provides guidelines for statistical offices and

other agencies responsible for constructing

CPIs and explains in depth the methods that

are used to calculate a CPI. It also examines

the underlying economic and statistical

concepts and principles needed for making

choices in efficient and cost-effective ways

and for appreciating the full implications

of those choices.

The following international organizations,

concerned both with the measurement of

inflation and with policies designed to control

it, have collaborated on the preparation

of this manual: the International Labour

Office; the International Monetary Fund; the

Organisation for Economic and Co-operation

and Development; the Statistical Office

of the European Communities (Eurostat);

the United Nations Economic Commission

for Europe; and the World Bank.

ISBN 92-2-113699-X

3 165140 163361

Co

nsu

me

r p

ric

e i

nd

ex

ma

nu

al

:T

he

or

ya

nd

pr

ac

ti

ce

International Labour Office

International Monetary Fund

Organisation for Economic Co-operation and Development

Statistical Office of the European Communities (Eurostat)

United Nations

The World Bank

Copyright # 2004International Labour Organization/International Monetary Fund/Organisation for Economic Co-operation andDevelopment/Statistical Office of the European Communities/United Nations/The International Bank forReconstruction and Development/The World Bank

First published 2004

Publications of the International Labour Office, the International Monetary Fund, the Organisation forEconomic Co-operation and Development, the Statistical Office of the European Communities (Eurostat), theUnited Nations Economic Commission for Europe and the World Bank (the publishing organizations) enjoycopyright under Protocol 2 of the Universal Copyright Convention. Nevertheless, short excerpts from themmay be reproduced without authorization, on condition that the source is indicated.

For rights of reproduction of this English original, or of translation into languages other than French andSpanish, application should be made to the Publications Bureau (Rights and Permissions), InternationalLabour Office, CH-1211 Geneva 22, Switzerland. The International Labour Office welcomes suchapplications.

For rights of reproduction of the French and Spanish translations, application should be made to theInternational Monetary Fund, 700 19th Street, N.W. Washington, DC, 20431, United States.Libraries, institutions and other users registered in the United Kingdom with the Copyright Licensing Agency,90 Tottenham Court Road, London W1P 4LP [Fax: (+44) (0) 207 631 5500; email: cla@cla.co.uk], in theUnited States with the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923 [Fax (+1)(978) 750 4470; email: info@copyright.com], or in the other countries with associated Reproduction RightsOrganizations, may make photocopies in accordance with the licences issued to them for this purpose.

ILO/IMF/OECD/UNECE/Eurostat/The World BankConsumer price index manual: Theory and practiceGeneva, International Labour Office, 2004

Guide, consumer price index, data collecting, statistical method, calculation, methodology, developed country,developing country. 09.02

ISBN 92-2-113699-X

ILO Cataloguing in Publication Data

The designations employed in this publication, which are in conformity with the practice of the publishingorganizations, and the presentation of material therein do not imply the expression of any opinion whatsoeveron the part of the publishing organizations concerning the legal status of any country, area or territory or of itsauthorities, or concerning the delimitation of its frontiers.

The responsibility for opinions expressed in signed articles, studies and other contributions rests solely withtheir authors, and publication does not constitute an endorsement by the publishing organizations of theopinions expressed in them.

Reference to names of firms and commercial products and processes does not imply their endorsement by thepublishing organizations, and any failure to mention a particular firm, commercial product or process is not asign of disapproval.

ILO publications can be obtained through major booksellers or ILO local offices in many countries, or directfrom ILO Publications, International Labour Office, CH-1211 Geneva 22, Switzerland. Catalogues or lists ofnew publications are available free of charge from the above address or by email: pubvente@ilo.orgVisit our website: www.ilo.org/publns

Printed in Hong Kong

iv

FOREWORD

This volume is an expanded revision of Consumer price indices: An ILO manual, published in 1989. Throughthe mechanism of the Intersecretariat Working Group on Price Statistics (IWGPS), the revision has beenundertaken under the joint responsibility of six international organizations: the International Labour Office(ILO); the International Monetary Fund (IMF); the Organisation for Economic Co-operation andDevelopment (OECD); the Statistical Office of the European Communities (Eurostat); the United NationsEconomic Commission for Europe (UNECE); and the World Bank. It is also being published jointly by theseorganizations.

The manual contains detailed comprehensive information and explanations on compiling a consumerprice index (CPI). It provides an overview of the conceptual and theoretical issues that statistical officesshould consider when making decisions on how to deal with the various problems in the compilation of aCPI, and is intended for use by both developed and developing countries. The chapters cover many topics;they elaborate on the different practices currently in use, propose alternatives whenever possible, and discussthe advantages and disadvantages of each alternative. Given the comprehensive nature of the manual, weexpect it to satisfy the needs of many users.

The main purpose of the manual is to assist producers of consumer price indices, particularly in countriesthat are revising or setting up their CPIs. It draws on a wide range of experience and expertise in an attemptto describe practical and suitable measurement methods. It should also help countries to produce their CPIsin a more comparable way so that statistical offices and international organizations can make meaningfulinternational comparisons. Bringing together a large body of knowledge on the subject, the manual may beused for self-learning, or as a teaching tool for training courses on the CPI.

Other CPI users, such as employers, workers, policy-makers and researchers, are also targeted. Themanual will inform them not only about the different methods that are employed in collecting data andcompiling such indices, but also of the limitations, so that the results may be interpreted correctly.

The drafting and revision have entailed many meetings over a five-year period, in which CPI experts fromnational statistical offices, international and regional organizations, universities and research institutes haveparticipated. The new manual owes much to their collective advice and wisdom.

The electronic version of the manual is available on the Internet at www.ilo.org/stat. The IWGPS viewsthe manual as a ‘‘living document’’ that it will amend and update to address particular points in more detail.This is especially true for emerging discussions and recommendations made by international groupsreviewing the CPI, such as the International Conference of Labour Statisticians (ICLS), meetings of theInternational Working Group on Price Indices (the ‘‘Ottawa Group’’), and the Joint UNECE/ILO Meetingson Consumer Price Indices.

Comments on the manual are welcomed by the IWGPS, and should be sent to the ILO Bureau ofStatistics (e-mail: stat@ilo.org). They will be taken into account in any future revisions.

International Labour Office (ILO): A. Sylvester Young, Director, Bureau of StatisticsInternational Monetary Fund (IMF): Horst Koehler, Managing Director

Organisation for Economic Co-operation and Development (OECD): Enrico Giovanini, Director,Statistical Directorate

Statistical Office of the European Communities (Eurostat): Bart Meganck, Director, Economic Statistics,and Economic and Monetary Convergence

United Nations Economic Commission for Europe (UNECE): Heinrich Br€uungger, Director,Statistics Division

World Bank: Shaida Badiee, Director, Development Data Group

v

CONTENTS

Foreword ............................................................................................................................................................ v

Preface ............................................................................................................................................................... xix

Acknowledgements ............................................................................................................................................. xxv

Reader’s guide .................................................................................................................................................... xxix

1 An introduction to consumer price index methodology .................................................................................... 1

The origins and uses of consumer price indices ............................................................................................ 1

Choice of index number ................................................................................................................................ 2

Price indices based on baskets of goods and services ................................................................................... 2

Lowe indices .......................................................................................................................................... 2Laspeyres and Paasche indices .............................................................................................................. 3Decomposing current value changes using Laspeyres and Paasche indices .......................................... 3Ratios of Lowe and Laspeyres indices .................................................................................................. 4Updated Lowe indices ........................................................................................................................... 4Interrelationships between fixed basket indices ..................................................................................... 4Young index ........................................................................................................................................... 5Geometric Young, Laspeyres and Paasche indices ................................................................................ 5Symmetric indices .................................................................................................................................. 5Fixed base versus chain indices ............................................................................................................. 6

Axiomatic and stochastic approaches to index numbers .............................................................................. 7

First axiomatic approach ....................................................................................................................... 7Ranking of indices using the first axiomatic approach ......................................................................... 8Some further tests .................................................................................................................................. 8The stochastic approach and a second axiomatic approach ................................................................. 9The unweighted stochastic approach ..................................................................................................... 9The weighted stochastic approach ......................................................................................................... 9A second axiomatic approach ............................................................................................................... 9

Cost of living index ....................................................................................................................................... 10

Upper and lower bounds on a cost of living index ............................................................................... 11Some special cases ................................................................................................................................. 11Estimating COLIs by superlative indices ............................................................................................... 11Representativity bias .............................................................................................................................. 12Data requirements and calculation issues .............................................................................................. 12Allowing for substitution ....................................................................................................................... 13

Aggregation issues ......................................................................................................................................... 13

Illustrative numerical data ............................................................................................................................ 13

Seasonal products .......................................................................................................................................... 13

Elementary price indices ............................................................................................................................... 14

Weights within elementary aggregates ................................................................................................... 15Interrelationships between different elementary index formulae ........................................................... 15Axiomatic approach to elementary indices ............................................................................................ 16Economic approach to elementary indices ............................................................................................ 16

Concepts, scope and classifications ............................................................................................................... 17

Acquisitions and uses ............................................................................................................................. 17Unconditional and conditional cost of living indices ............................................................................ 19Specific types of transactions ................................................................................................................. 19Household production ........................................................................................................................... 20Coverage of households and outlets ...................................................................................................... 21Price variation ........................................................................................................................................ 21Classifications ......................................................................................................................................... 22Consumer price indices and national accounts price deflators .............................................................. 22

Expenditure weights ...................................................................................................................................... 22

Household expenditure surveys and national accounts ......................................................................... 22Other sources for estimating expenditure weights ................................................................................. 23

Collection of price data ................................................................................................................................. 23

Random sampling and purposive sampling .......................................................................................... 24

vii

Methods of price collection ................................................................................................................... 25Continuity of price collection ................................................................................................................ 25Resampling ............................................................................................................................................ 26

Adjusting prices for quality changes ............................................................................................................. 26

Evaluation of the effect of quality change on price .............................................................................. 27Implicit methods for adjusting for quality changes ............................................................................... 27Explicit quality adjustments ................................................................................................................... 29

Item substitution and new goods .................................................................................................................. 29

New goods and services ......................................................................................................................... 30

Calculation of consumer price indices in practice ........................................................................................ 30

Elementary price indices ........................................................................................................................ 31Higher-level indices ................................................................................................................................ 31

Organization and management ..................................................................................................................... 32

Publication and dissemination ...................................................................................................................... 32

2 Uses of consumer price indices ........................................................................................................................ 33

A range of possible consumer price indices .................................................................................................. 33

Indexation ..................................................................................................................................................... 33

Indexation of wages ............................................................................................................................... 33Indexation of social security benefits ..................................................................................................... 33The type of index used for indexation ................................................................................................... 34Indexation of interest, rents and other contractual payments ............................................................... 34Taxation ................................................................................................................................................. 34

Real consumption and real income ............................................................................................................... 35

Consistency between price indices and expenditure series ..................................................................... 35Purchasing power parities ...................................................................................................................... 35

Use of the consumer price index for accounting under inflation ................................................................. 36

Current purchasing power accounts ...................................................................................................... 36Current cost accounting ......................................................................................................................... 36

Consumer price indices and general inflation ............................................................................................... 36

Consumer price indices and inflation targets ........................................................................................ 36Consumer price indices and international comparisons of inflation ..................................................... 36

Popularity of consumer price indices as economic statistics ......................................................................... 37

The need for independence and integrity of consumer price indices ............................................................ 37

3 Concepts and scope ......................................................................................................................................... 39

Introduction .................................................................................................................................................. 39

Alternative consumption aggregates ............................................................................................................. 39

Acquisitions and expenditures ............................................................................................................... 40Monetary versus non-monetary expenditures ........................................................................................ 40

Acquisitions and uses .................................................................................................................................... 40

Durables and non-durables .................................................................................................................... 41Consumer price indices based on acquisitions and uses ........................................................................ 41

Basket indices and cost of living indices ....................................................................................................... 42

Lowe indices .......................................................................................................................................... 42Cost of living indices ............................................................................................................................. 42

Expenditures and other payments outside the scope of consumer price indices .......................................... 43

Transfers ................................................................................................................................................ 43Insurance ................................................................................................................................................ 44Gambling ............................................................................................................................................... 44Transactions in financial assets .............................................................................................................. 44

Purchases and sales of foreign currency ....................................................................................................... 44

Payments, financing and credit ..................................................................................................................... 44

Financial transactions and borrowing ................................................................................................... 45The creation of a financial asset/liability ............................................................................................... 45Hire purchase ......................................................................................................................................... 45Interest payments ................................................................................................................................... 46

Household production .................................................................................................................................. 46

Business activities ................................................................................................................................... 46Consumption of own produce ............................................................................................................... 47

Coverage of households and outlets ............................................................................................................. 48

Definition of household ......................................................................................................................... 48Types of household ................................................................................................................................ 49

CONTENTS

viii

Geographical coverage ........................................................................................................................... 49Outlet coverage ...................................................................................................................................... 50

Price variation ............................................................................................................................................... 50

Price discrimination ............................................................................................................................... 51Price variation between outlets .............................................................................................................. 51Outlet rotation ....................................................................................................................................... 51

Treatment of some specific household expenditures ..................................................................................... 52

Fees of agents and brokers .................................................................................................................... 52Undesirable or illegal goods and services .............................................................................................. 52Luxury goods and services .................................................................................................................... 52Second-hand goods ................................................................................................................................ 52Imputed expenditures on goods and services ........................................................................................ 53

Price coverage ................................................................................................................................................ 53

Taxes and subsidies ................................................................................................................................ 53Discounts, rebates, loyalty schemes and ‘‘free’’ products ..................................................................... 54

Classification ................................................................................................................................................. 55

Criteria for classifying consumption expenditure .................................................................................. 55Classification by product type ............................................................................................................... 55Classification by purpose ....................................................................................................................... 56Classifications for consumer price indices ............................................................................................. 56Publication level ..................................................................................................................................... 56Classification of Individual Consumption according to Purpose (COICOP) ........................................ 57

Appendix 3.1 Consumer price indices and national accounts price deflators ........................................ 58

4 Expenditure weights and their sources ............................................................................................................. 59

Introduction .................................................................................................................................................. 59

The weighting structure of the consumer price index ................................................................................... 59

Group, class and sub-class weights ........................................................................................................ 61Regional weights .................................................................................................................................... 61Outlet or outlet-type weights ................................................................................................................. 61Elementary aggregate weights ................................................................................................................ 61

Data sources .................................................................................................................................................. 62

Household expenditure surveys ............................................................................................................. 62National accounts .................................................................................................................................. 63Retail sales data ..................................................................................................................................... 63Point-of-purchase surveys ...................................................................................................................... 63Scanner data .......................................................................................................................................... 63Population censuses ............................................................................................................................... 63

Deriving the weights in practice .................................................................................................................... 64

Payments that are not consumption expenditures ................................................................................. 64Unimportant expenditures ..................................................................................................................... 64Products that are difficult to price ......................................................................................................... 64Use and combination of different sources ............................................................................................. 64Adjusting the weights derived from household expenditure surveys ..................................................... 64Weight reference period ......................................................................................................................... 65Need for revising the weights ................................................................................................................ 65Frequency of updating the weights ....................................................................................................... 65Classification .......................................................................................................................................... 66Items requiring special treatment .......................................................................................................... 66Errors in weighting ................................................................................................................................ 68

5 Sampling ......................................................................................................................................................... 69

Introduction .................................................................................................................................................. 69

Probability sampling techniques ................................................................................................................... 69

Implementing probability sampling in consumer price indices ..................................................................... 70

Sampling techniques based on probability proportional to size ........................................................... 70Sampling methods used by the US Bureau of Labor Statistics ............................................................ 71

Non-probability sampling techniques ........................................................................................................... 71

Reasons for using non-probability sampling ......................................................................................... 71Cut-off sampling .................................................................................................................................... 72Quota sampling ...................................................................................................................................... 73The representative item method ............................................................................................................ 73Sampling in time .................................................................................................................................... 73

CONTENTS

ix

Choice of sampling method .......................................................................................................................... 73

Estimation procedures ................................................................................................................................... 74

Implementing estimation procedures for consumer price indices ................................................................. 75

Variance estimation ....................................................................................................................................... 76

Variances of elementary index formulae ............................................................................................... 76The United States approach .................................................................................................................. 76The Swedish approach ........................................................................................................................... 76The French approach ............................................................................................................................ 77The Luxembourg approach ................................................................................................................... 77Other approaches ................................................................................................................................... 78

Optimal allocation ......................................................................................................................................... 78

Summary ....................................................................................................................................................... 78

6 Price collection ................................................................................................................................................ 81

Introduction .................................................................................................................................................. 81

Frequency and timing of collection .............................................................................................................. 81

Taking account of hyperinflation .......................................................................................................... 83

Item specification .......................................................................................................................................... 83

Collection procedures .................................................................................................................................... 84

Price collection techniques ..................................................................................................................... 85Questionnaire design .............................................................................................................................. 87Field procedures ..................................................................................................................................... 90Central and head office collection ......................................................................................................... 90Price reductions ...................................................................................................................................... 91Price bargaining ..................................................................................................................................... 93Forced replacements, product substitution and quality adjustment ..................................................... 94

Related issues ................................................................................................................................................ 95

Electronic reporting ............................................................................................................................... 95Purchasing power parities ...................................................................................................................... 96Data quality and quality assurance ....................................................................................................... 97Documentation ...................................................................................................................................... 97

Appendix 6.1 Extract from a simple price collection form ........................................................................... 98

7 Adjusting for quality change ........................................................................................................................... 99

Introduction .................................................................................................................................................. 99

Why the matched models method may fail ................................................................................................. 99

Missing items ......................................................................................................................................... 99Sampling concerns ................................................................................................................................. 100New products ......................................................................................................................................... 101

The nature of quality change ........................................................................................................................ 101

A utility-based approach ....................................................................................................................... 102Conditional indices ................................................................................................................................ 103

An overview of methods of quality adjustment when matched items are unavailable .................................. 104

Additive versus multiplicative adjustment ............................................................................................. 105Base versus current period adjustment .................................................................................................. 105Long-run versus short-run comparisons ................................................................................................ 106

Implicit methods of quality adjustment ........................................................................................................ 106

Overlap ................................................................................................................................................... 106Overall mean or targeted mean imputation .......................................................................................... 108Class mean imputation .......................................................................................................................... 111Comparable replacement ....................................................................................................................... 111Linked to show no price change ............................................................................................................ 112Carry-forward ........................................................................................................................................ 112

Explicit methods of quality adjustment ........................................................................................................ 112

Expert judgement ................................................................................................................................... 112Quantity adjustment .............................................................................................................................. 113Differences in production or option costs ............................................................................................. 114Hedonic approach .................................................................................................................................. 116Limitations of the hedonic approach .................................................................................................... 120

Choice between quality adjustment methods ................................................................................................ 122

High-technology and other sectors with a rapid turnover of models ........................................................... 124

Some examples ....................................................................................................................................... 124Hedonic price indices ............................................................................................................................. 125

CONTENTS

x

The difference between hedonic indices and matched indices ............................................................... 128Chaining ................................................................................................................................................. 129

Long-run and short-run comparisons ........................................................................................................... 130

Quality adjustment methods in short-run comparisons ........................................................................ 130Implicit short-run comparisons using imputations ................................................................................ 131Single-stage and two-stage indices ......................................................................................................... 132

Appendix 7.1 Data on personal computers, obtained from United Kingdom Compaq and Dell web sites,July 2000, to illustrate hedonic regression ................................................................................................. 134

8 Item substitution, sample space and new products ........................................................................................... 137

Introduction .................................................................................................................................................. 137

Matched samples ........................................................................................................................................... 137

Sample space and item replacement or substitution ..................................................................................... 138

Sample rotation, chaining and hedonic indices ............................................................................................ 140

Information requirements for a quality adjustment strategy ........................................................................ 140

Statistical metadata system .................................................................................................................... 140

New products and how they differ from products with quality changes ..................................................... 141

Incorporation of new products ..................................................................................................................... 142

Sample rebasing and rotation ................................................................................................................ 143Directed replacements and sample augmentation ................................................................................. 144Reservation prices .................................................................................................................................. 146

Summary ....................................................................................................................................................... 146

Appendix 8.1 Appearance or disappearance of products or outlets ............................................................. 148

Appendix 8.2 New goods and substitution ................................................................................................... 151

9 Calculating consumer price indices in practice ................................................................................................ 153

Introduction .................................................................................................................................................. 153

The calculation of price indices for elementary aggregates .......................................................................... 153

Construction of elementary aggregates .................................................................................................. 153Construction of elementary price indices .............................................................................................. 154Chain versus direct indices for elementary aggregates .......................................................................... 159Consistency in aggregation .................................................................................................................... 160Missing price observations ..................................................................................................................... 160Other formulae for elementary price indices ......................................................................................... 162Unit value indices .................................................................................................................................. 164Formulae applicable to scanner data .................................................................................................... 164

The calculation of higher-level indices ........................................................................................................... 164

Consumer price indices as weighted averages of elementary indices ............................................................ 165

A numerical example ............................................................................................................................. 166Young and Lowe indices ....................................................................................................................... 166Factoring the Young index .................................................................................................................... 167Price-updating from the weight reference period to the price reference period .................................... 167The introduction of new weights and chain linking .............................................................................. 168Decomposition of index changes ........................................................................................................... 171Some alternatives to fixed weight indices .............................................................................................. 172

Data editing ................................................................................................................................................... 173

Identifying possible errors and outliers ................................................................................................. 174Verifying and correcting data ................................................................................................................ 176

10 Some special cases ........................................................................................................................................ 179

Introduction .................................................................................................................................................. 179

Owner-occupied housing ............................................................................................................................... 179

Use ......................................................................................................................................................... 179Payments ................................................................................................................................................ 181Acquisitions ........................................................................................................................................... 183

Clothing ......................................................................................................................................................... 185

The clothing market ............................................................................................................................... 185Approaches to constructing indices for non-seasonal clothing .............................................................. 185Replacement of items and quality change ............................................................................................. 186Approaches to including seasonal clothing in the consumer price index .............................................. 187Summary comments ............................................................................................................................... 190

Telecommunication services .......................................................................................................................... 191

Representative items – matched samples ............................................................................................... 192Representative items – unit values ......................................................................................................... 193

CONTENTS

xi

Customer profiles ................................................................................................................................... 193Sample of bills ....................................................................................................................................... 194

Financial services .......................................................................................................................................... 194

Currency exchange ................................................................................................................................. 195Stockbroking services ............................................................................................................................. 195Deposit and loan facilities ..................................................................................................................... 196

Real estate agency services ............................................................................................................................ 198

Property insurance services ........................................................................................................................... 198

Payments ................................................................................................................................................ 199Use ......................................................................................................................................................... 199Acquisitions ........................................................................................................................................... 200Pricing gross insurance premiums ......................................................................................................... 200Using gross premiums as a proxy for the net insurance service ........................................................... 200

Appendix 10.1 Calculation of a price index for a deposit product .............................................................. 202

11 Errors and bias .............................................................................................................................................. 207

Introduction .................................................................................................................................................. 207

Types of error ............................................................................................................................................... 207

Sampling error ....................................................................................................................................... 207Non-sampling error ............................................................................................................................... 207

Measuring error and bias .............................................................................................................................. 208

Estimation of variance ........................................................................................................................... 208Qualitative descriptions of non-sampling errors .................................................................................... 209

Procedures to minimize errors ...................................................................................................................... 209

Types of bias ................................................................................................................................................. 210

Components of bias ...................................................................................................................................... 211

Upper-level substitution bias ................................................................................................................. 211Elementary aggregate bias ..................................................................................................................... 212Quality change and new products bias .................................................................................................. 213New outlet bias ...................................................................................................................................... 213

Summary of bias estimates ............................................................................................................................ 214

Conclusion ..................................................................................................................................................... 214

12 Organization and management ...................................................................................................................... 215

Introduction .................................................................................................................................................. 215

Local collection ............................................................................................................................................. 215

Contracting out ...................................................................................................................................... 215Central collection ................................................................................................................................... 217

Quality in the field ........................................................................................................................................ 217

Descriptions ........................................................................................................................................... 217Continuity .............................................................................................................................................. 217Data entry queries ................................................................................................................................. 218Feedback ................................................................................................................................................ 218

Quality checks in local collection: The role of auditors ............................................................................... 218

Monitoring ............................................................................................................................................. 218Backchecking ......................................................................................................................................... 219Other auditor functions ......................................................................................................................... 219

Quality checks in head office ......................................................................................................................... 219

Reports ................................................................................................................................................... 220Algorithms ............................................................................................................................................. 221

Producing and publishing the index ............................................................................................................. 221

Monthly compilation ............................................................................................................................. 221Spreadsheets ........................................................................................................................................... 222Introducing changes ............................................................................................................................... 222Disaster recovery ................................................................................................................................... 222

Quality management and quality management systems ............................................................................... 223

Quality management systems ................................................................................................................. 223Scope for greater use of quality management techniques ..................................................................... 224

Performance management, development and training .................................................................................. 225

Training requirements ............................................................................................................................ 225Specific training for compilers and collectors ........................................................................................ 225Documentation ...................................................................................................................................... 225Reviews .................................................................................................................................................. 226

CONTENTS

xii

13 Publication, dissemination and user relations ................................................................................................ 227

Introduction .................................................................................................................................................. 227

Time series presentation of level and change ................................................................................................ 227

Seasonal adjustment and smoothing of the index ........................................................................................ 228

Analysis of contributions to change ............................................................................................................. 228

Economic commentary and interpretation of the index ............................................................................... 228

Presentation of related or alternative measures ............................................................................................ 229

Core inflation ......................................................................................................................................... 229Alternative indices ................................................................................................................................. 229Sub-aggregate indices ............................................................................................................................. 229

Press release, bulletin and methodological statement ................................................................................... 230

International standards concerning the dissemination of consumer price indices ........................................ 231

Timing of dissemination of the consumer price index .................................................................................. 231

Timeliness of release versus data accuracy ................................................................................................... 231

Access to data ............................................................................................................................................... 232

Confidentiality ............................................................................................................................................... 232

Electronic dissemination ............................................................................................................................... 232

User consultation .......................................................................................................................................... 232

Different uses of consumer price indices ............................................................................................... 232Presentation of methodology ................................................................................................................. 233Role of advisory committees ................................................................................................................. 233Explaining index quality ........................................................................................................................ 233

14 The system of price statistics ........................................................................................................................ 235

Introduction .................................................................................................................................................. 235

National accounts as a framework for the system of price statistics ........................................................... 236

Aggregate supply and use of goods and services .................................................................................. 236Institutional units and establishments ................................................................................................... 237Accounts of institutional units .............................................................................................................. 238

The consumer price index among major price indices .................................................................................. 252

Scope of the expenditure aggregates of the consumer price index ........................................................ 252The consumer price index as a measure of inflation in market transactions ........................................ 255Treatment of cross-border shopping in the consumer price index ........................................................ 255

Other price indicators in the national accounts ............................................................................................ 255

Price indices for total supply ................................................................................................................. 255Price indices for intermediate consumption ........................................................................................... 256Price indices for final uses ..................................................................................................................... 256Price indices for gross domestic product ............................................................................................... 256Price indices for labour services ............................................................................................................. 257

Framework for a system of price statistics for goods and services .............................................................. 257

International comparisons of expenditure on goods and services ................................................................ 261

15 Basic index number theory ............................................................................................................................ 263

Introduction .................................................................................................................................................. 263

The decomposition of value aggregates into price and quantity components .............................................. 264

The decomposition of value aggregates and the product test ............................................................... 264The Laspeyres and Paasche indices ....................................................................................................... 265

Symmetric averages of fixed basket price indices ......................................................................................... 266

The Fisher index as an average of the Paasche and Laspeyres indices ................................................. 266The Walsh index and the theory of the ‘‘pure’’ price index .................................................................. 268

Annual weights and monthly price indices ................................................................................................... 270

The Lowe index with monthly prices and annual base year quantities ................................................ 270The Lowe index and mid-year indices ................................................................................................... 274The Young index ................................................................................................................................... 275

The Divisia index and discrete approximations to it .................................................................................... 278

The Divisia price and quantity indices .................................................................................................. 278Discrete approximations to the continuous time Divisia index ............................................................ 279Fixed base versus chain indices ............................................................................................................. 280

Appendix 15.1 The relationship between the Paasche and Laspeyres indices .............................................. 285

Appendix 15.2 The relationship between the Lowe and Laspeyres indices .................................................. 285

Appendix 15.3 The relationship between the Young index and its time antithesis ...................................... 286

Appendix 15.4 The relationship between the Divisia and economic approaches ......................................... 287

CONTENTS

xiii

16 The axiomatic and stochastic approaches to index number theory ................................................................ 289

Introduction .................................................................................................................................................. 289

The levels approach to index number theory ............................................................................................... 291

An axiomatic approach to unilateral price indices ................................................................................ 291A second axiomatic approach to unilateral price indices ...................................................................... 292

The first axiomatic approach to bilateral price indices ................................................................................. 292

Bilateral indices and some early tests .................................................................................................... 292Homogeneity tests .................................................................................................................................. 293Invariance and symmetry tests .............................................................................................................. 294Mean value tests .................................................................................................................................... 295Monotonicity tests ................................................................................................................................. 296The Fisher ideal index and the test approach ....................................................................................... 296The test performance of other indices ................................................................................................... 297The additivity test .................................................................................................................................. 297

The stochastic approach to price indices ...................................................................................................... 299

The early unweighted stochastic approach ............................................................................................ 299The weighted stochastic approach ......................................................................................................... 301

The second axiomatic approach to bilateral price indices ............................................................................ 304

The basic framework and some preliminary tests ................................................................................. 304Homogeneity tests .................................................................................................................................. 305Invariance and symmetry tests .............................................................................................................. 306A mean value test .................................................................................................................................. 306Monotonicity tests ................................................................................................................................. 306Weighting tests ....................................................................................................................................... 307The Tornqvist–Theil price index and the second test approach to bilateral indices ............................. 307

The test properties of the Lowe and Young indices ..................................................................................... 309

Appendix 16.1 Proof of the optimality of the Tornqvist–Theil price index in the second bilateral testapproach ................................................................................................................................................... 311

17 The economic approach to index number theory: The single-household case ................................................. 313

Introduction .................................................................................................................................................. 313

The Konus cost of living index and observable bounds ............................................................................... 314

The true cost of living index when preferences are homothetic ................................................................... 316

Superlative indices: The Fisher ideal index ................................................................................................... 318

Quadratic mean of order r superlative indices .............................................................................................. 320

Superlative indices: the Tornqvist index ....................................................................................................... 322

The approximation properties of superlative indices .................................................................................... 324

Superlative indices and two-stage aggregation .............................................................................................. 325

The Lloyd–Moulton index number formula ................................................................................................. 327

Annual preferences and monthly prices ........................................................................................................ 328

The Lowe index as an approximation to a true cost of living index .................................................... 329A first-order approximation to the bias of the Lowe index .................................................................. 330A second-order approximation to the substitution bias of the Lowe index ......................................... 330The problem of seasonal commodities .................................................................................................. 332

The problem of a zero price increasing to a positive price ........................................................................... 334

18 The economic approach to index number theory: The many-household case .................................................. 337

Introduction .................................................................................................................................................. 337

Plutocratic cost of living indices and observable bounds ............................................................................. 337

The Fisher plutocratic price index ................................................................................................................ 339

Democratic versus plutocratic cost of living indices ..................................................................................... 341

19 Price indices using an artificial data set ........................................................................................................ 345

Introduction .................................................................................................................................................. 345

The artificial data set .................................................................................................................................... 345

Early price indices: The Carli, Jevons, Laspeyres and Paasche indices ........................................................ 346

Asymmetrically weighted price indices .......................................................................................................... 346

Symmetrically weighted indices: Superlative and other indices .................................................................... 348

Superlative indices constructed in two stages of aggregation ....................................................................... 349

Lloyd–Moulton price indices ........................................................................................................................ 349

Additive percentage change decompositions for the Fisher ideal index ....................................................... 351

The Lowe and Young indices ....................................................................................................................... 352

Mid-year indices based on the Lowe formula .............................................................................................. 353

Young-type indices ........................................................................................................................................ 353

xiv

CONTENTS

20 Elementary indices ........................................................................................................................................ 355

Introduction .................................................................................................................................................. 355

Ideal elementary indices ................................................................................................................................ 355

Aggregation and classification problems for elementary aggregates ............................................................ 358

Elementary indices used in practice .............................................................................................................. 360

Numerical relationships between the frequently used elementary indices .................................................... 361

The axiomatic approach to elementary indices ............................................................................................. 363

The economic approach to elementary indices ............................................................................................. 364

The sampling approach to elementary indices .............................................................................................. 366

The use of scanner data in constructing elementary aggregates ................................................................... 367

A simple stochastic approach to elementary indices ..................................................................................... 369

Conclusion ..................................................................................................................................................... 370

21 Quality change and hedonics ......................................................................................................................... 373

Introduction .................................................................................................................................................. 373

New and disappearing items ......................................................................................................................... 373

Hedonic prices and implicit markets ............................................................................................................. 374

Items as tied bundles of characteristics ................................................................................................. 374The consumer or demand side ............................................................................................................... 375The producer or supply side ................................................................................................................. 376Equilibrium ............................................................................................................................................ 377What hedonic prices mean ..................................................................................................................... 377An alternative, consumer-based hedonic theoretical formulation ......................................................... 379

Hedonic indices ............................................................................................................................................. 381

Theoretical characteristics price indices ................................................................................................. 381Hedonic regressions and dummy variables of time ............................................................................... 382Hedonic imputation indices ................................................................................................................... 382Superlative and exact hedonic indices ................................................................................................... 383Unweighted hedonic indices and unweighted matched index number formulae ................................... 385

New goods and services ................................................................................................................................ 385

Appendix 21.1 Some econometric issues ....................................................................................................... 388

22 The treatment of seasonal products ............................................................................................................... 393

Introduction .................................................................................................................................................. 393

A seasonal commodity data set .................................................................................................................... 394

Year-over-year monthly indices .................................................................................................................... 396

Year-over-year annual indices ....................................................................................................................... 400

Rolling year annual indices ........................................................................................................................... 402

Predicting a rolling year index using a current period year-over-year monthly index ................................. 405

Maximum overlap month-to-month price indices ........................................................................................ 407

Annual basket indices with carry forward of unavailable prices .................................................................. 411

Annual basket indices with imputation of unavailable prices ...................................................................... 412

Bean and Stine Type C or Rothwell indices ................................................................................................. 413

Forecasting rolling year indices using month-to-month annual basket indices ............................................ 414

Conclusion ..................................................................................................................................................... 416

23 Durables and user costs ................................................................................................................................. 419

Introduction .................................................................................................................................................. 419

The acquisitions approach ............................................................................................................................ 420

The rental equivalence approach .................................................................................................................. 421

The user cost approach ................................................................................................................................. 422

The relationship between user costs and acquisition costs ........................................................................... 424

Alternative models of depreciation ............................................................................................................... 426

A general model of depreciation for (unchanging) consumer durables ................................................ 426Geometric or declining balance depreciation ........................................................................................ 427Straight line depreciation ....................................................................................................................... 428‘‘One hoss shay’’ or light bulb depreciation .......................................................................................... 428

Unique durable goods and the user cost approach ..................................................................................... 429

The user cost of owner-occupied housing ..................................................................................................... 430

The treatment of costs that are tied to owner-occupied housing ................................................................. 433

The treatment of mortgage interest costs .............................................................................................. 433The treatment of property taxes ............................................................................................................ 434The treatment of property insurance ..................................................................................................... 434

xv

CONTENTS

The treatment of maintenance and renovation expenditures ............................................................... 435The treatment of the transactions costs of home purchase ................................................................... 437

User costs for landlords versus owners ......................................................................................................... 437

Damage costs ......................................................................................................................................... 437Non-payment of rent and vacancy costs ............................................................................................... 437Billing and maintenance costs ............................................................................................................... 437The opportunity cost of capital ............................................................................................................ 437The supply of additional services for rented properties ....................................................................... 438

The payments approach ................................................................................................................................ 438

Alternative approaches for pricing owner-occupied housing ........................................................................ 439

The acquisitions approach ..................................................................................................................... 439The rental equivalence approach ........................................................................................................... 439The user cost approach .......................................................................................................................... 440

A glossary of main terms ............................................................................................................................... 443

Appendix to the glossary. Some basic number formulae and terminology .......................................... 450

Annex 1 Harmonized Indices of Consumer Prices (European Union) ............................................................ 453

Annex 2 Classification of Individual Consumption according to Purpose (COICOP)-Extract ....................... 465

Annex 3 Resolution concerning consumer price indices adopted by the Seventeenth International Conference ofLabour Statisticians, 2003 ................................................................................................................ 483

Annex 4 Spatial comparisons of consumer prices, purchasing power parities and the International Comparison

Program ........................................................................................................................................... 495

Bibliography ................................................................................................................................................... 507

Index .............................................................................................................................................................. 521

List of tables

4.1 Example of weights by region and outlet type for the sub-class ‘‘fresh fruit’’ ............................... 61

5.1 Systematic sample of 3 out of 10 outlets, based on probability proportional to size .................... 70

5.2 Pareto sample of 3 out of 10 outlets, based on probability proportional to size .......................... 71

6.1 Example of a survey form showing the number of price quotations by shop or stall ................... 94

6.2 Example illustrating the method for determining the actual price paid by the purchaser whenbargaining takes place .................................................................................................................... 95

7.1 Example of the implicit methods of quality adjustment ................................................................ 106

7.2 Example of the bias from implicit quality adjustment when the (mean) price change of quality-adjusted new items compared with the items they are replacing is assumed not to change(r2=1.00) ......................................................................................................................................... 110

7.3 Example of size, price and unit price of bags of flour ................................................................... 114

7.4 Hedonic regression results for Dell and Compaq personal computers .......................................... 117

7.5 Example of long-run and short-run comparisons ........................................................................... 130

8.1 Example of sample augmentation ................................................................................................... 145

9.1 Calculation of price indices for an elementary aggregate ............................................................... 156

9.2 Imputation of temporarily missing prices ....................................................................................... 161

9.3 Disappearing items and their replacements with no overlap .......................................................... 162

9.4 Disappearing and replacement items with overlapping prices ........................................................ 163

9.5 The aggregation of elementary price indices .................................................................................. 166

9.6 Price-updating of weights between the weight and price reference periods ................................... 168

9.7 Calculation of a chain index ........................................................................................................... 170

9.8 Decomposition of index changes .................................................................................................... 172

10.1 Calculation of a mortgage debt series ............................................................................................ 183

10.2 Calculation of a mortgage interest charges series ........................................................................... 183

10.3 Synthetic price data to illustrate approaches to constructing clothing price indices ...................... 188

10.4 Alternative price indices for summer seasonal clothing ................................................................. 189

10.5 Alternative price indices for winter seasonal clothing .................................................................... 189

10.6 Alternative price indices for total clothing ..................................................................................... 190

10.7 An illustrative index structure for telecommunication services (representative item approach) .... 192

10.8 Examples of specifications of telecommunication services ............................................................. 192

10.9 Example of a user profile for mobile phone services ...................................................................... 194

CONTENTS

xvi

10.10 Illustration of the impact of taxes on measures of insurance services ............................................. 201

11.1 A taxonomy of errors in a consumer price index ........................................................................... 207

14.1 Production account for an establishment, institutional unit or institutional sector ....................... 239

14.2 Production account with product detail for an establishment or local kind of activity unit ......... 241

14.3 Use of income account for institutional units and sectors ............................................................. 243

14.4 Use of income account with product detail for institutional units and sectors ............................. 245

14.5 Use of income account with product detail for the total economy ................................................ 246

14.6 Capital account ............................................................................................................................... 247

14.7 Capital account with product detail ............................................................................................... 248

14.8 External account of goods and services .......................................................................................... 250

14.9 External account of goods and services with product detail .......................................................... 250

14.10 The supply and use table (SUT) ..................................................................................................... 251

14.11 Location and coverage of major price indices: Columns in the supply and use table ................... 253

14.12 Definition of scope, price relatives, coverage and weights for major price indices ........................ 254

14.13 Generation of income account for establishment, institutional unit or institutional sector .......... 256

14.14 Generation of income account for establishment and industry with labour services (occupational)detail ............................................................................................................................................... 258

14.15 A framework for price statistics ..................................................................................................... 258

19.1 Prices for six commodities .............................................................................................................. 346

19.2 Quantities for six commodities ....................................................................................................... 346

19.3 Expenditures and expenditure shares for six commodities ............................................................. 346

19.4 The fixed base Laspeyres, Paasche, Carli and Jevons indices ........................................................ 347

19.5 Chain Laspeyres, Paasche, Carli and Jevons indices ...................................................................... 347

19.6 Asymmetrically weighted fixed base indices ................................................................................... 347

19.7 Asymmetrically weighted indices using the chain principle ............................................................ 347

19.8 Asymmetrically weighted fixed base indices for commodities 3–6 ................................................. 348

19.9 Asymmetrically weighted chained indices for commodities 3–6 ..................................................... 348

19.10 Symmetrically weighted fixed base indices ..................................................................................... 349

19.11 Symmetrically weighted indices using the chain principle .............................................................. 349

19.12 Fixed base superlative single-stage and two-stage indices .............................................................. 350

19.13 Chained superlative single-stage and two-stage indices .................................................................. 350

19.14 Chained Fisher and fixed base Lloyd–Moulton indices ................................................................. 350

19.15 Chained Fisher and chained Lloyd–Moulton indices ..................................................................... 351

19.16 Diewert’s additive percentage change decomposition of the Fisher index ..................................... 351

19.17 Van Ijzeren’s decomposition of the Fisher price index .................................................................. 352

19.18 The Lowe and Young indices, the fixed base Laspeyres, Paasche and Fisher indices, and thechained Laspeyres, Paasche and Fisher indices .............................................................................. 352

19.19 The five Lowe indices, the mid-year index, and the Tornqvist and Fisher chain indices .............. 354

19.20 The five Young-type indices and the Tornqvist and Fisher chain indices ..................................... 354

20.1 Proportion of transactions in 2000 that could be matched to 1998 ............................................... 360

20.2 Laspeyres price indices by type of classification, September 1998–September 2000 ...................... 360

20.3 Fisher price indices by type of classification, September 1998–September 2000 ............................ 360

22.1 An artificial seasonal data set: Prices ............................................................................................. 395

22.2 An artificial seasonal data set: Quantities ...................................................................................... 395

22.3 Year-over-year monthly fixed base Laspeyres indices .................................................................... 398

22.4 Year-over-year monthly fixed base Paasche indices ....................................................................... 398

22.5 Year-over-year monthly fixed base Fisher indices .......................................................................... 398

22.6 Year-over-year approximate monthly fixed base Paasche indices .................................................. 398

22.7 Year-over-year approximate monthly fixed base Fisher indices ..................................................... 399

22.8 Year-over-year monthly chained Laspeyres indices ........................................................................ 399

22.9 Year-over-year monthly chained Paasche indices ........................................................................... 399

22.10 Year-over-year monthly chained Fisher indices ............................................................................. 399

22.11 Year-over-year monthly approximate chained Laspeyres indices .................................................. 400

22.12 Year-over-year monthly approximate chained Paasche indices ..................................................... 400

22.13 Year-over-year monthly approximate chained Fisher indices ........................................................ 400

22.14 Annual fixed base Laspeyres, Paasche and Fisher price indices .................................................... 402

22.15 Annual approximate fixed base Laspeyres, Paasche, Fisher and geometric Laspeyres indices ...... 402

xvii

CONTENTS

22.16 Annual chained Laspeyres, Paasche and Fisher price indices ........................................................ 402

22.17 Annual approximate chained Laspeyres, Paasche and Fisher price indices ................................... 402

22.18 Rolling year Laspeyres, Paasche and Fisher price indices ............................................................. 404

22.19 Rolling year approximate Laspeyres, Paasche and Fisher price indices ........................................ 405

22.20 Rolling year fixed base Laspeyres and seasonally adjusted approximate rolling year price indices 406

22.21 Month-to-month maximum overlap chained Laspeyres, Paasche and Fisher price indices ........... 409

22.22 Month-to-month chained Laspeyres, Paasche and Fisher price indices ......................................... 409

22.23 Lowe, Young, geometric Laspeyres and centred rolling year indices with carry forward prices ... 411

22.24 Lowe, Young, geometric Laspeyres and centred rolling year indices with imputed prices ............ 413

22.25 The Lowe with carry forward prices, Rothwell and normalized Rothwell indices ........................ 414

22.26 Seasonally adjusted Lowe, Young and geometric Laspeyres indices with carry forward prices andthe centred rolling year index ......................................................................................................... 415

22.27 Seasonally adjusted Lowe, Young and geometric Laspeyres indices with imputed prices, seasonallyadjusted Rothwell and centred rolling year indices ........................................................................ 416

List of figures

4.1 Typical aggregation structure of a consumer price index (CPI) ..................................................... 60

6.1 Price collection procedures ............................................................................................................. 86

7.1 Quality adjustment for different-sized items ................................................................................... 113

7.2 Scatter diagram showing prices and processing speeds of personal computers ............................. 116

7.3 Flowchart for making decisions on quality change ........................................................................ 123

9.1 Typical aggregation structure of a consumer price index (CPI) ..................................................... 155

12.1 Price collection procedures ............................................................................................................. 216

17.1 The Laspeyres and Paasche bounds to the true cost of living index ............................................. 315

21.1 Consumption and production decisions for combinations of characteristics ................................. 375

22.1 Rolling year fixed base and chained Laspeyres, Paasche and Fisher indices ................................. 404

22.2 Rolling year approximate fixed base and chained Laspeyres, Paasche and Fisher indices ............ 405

22.3 Fixed base Laspeyres, seasonally adjusted approximate and approximate rolling year indices .... 407

22.4 Lowe, Young, geometric Laspeyres and centred rolling year Laspeyres indices ........................... 412

22.5 Lowe, Young and geometric Laspeyres with imputed prices and centred rolling year indices ..... 413

22.6 The Lowe and Rothwell price indices ............................................................................................ 414

22.7 Seasonally adjusted Lowe, Young and geometric Laspeyres indices with carry forward prices andthe centred rolling year index ......................................................................................................... 415

22.8 Seasonally adjusted Lowe, Young and geometric Laspeyres indices with imputed prices, seasonallyadjusted Rothwell and centred rolling year indices ........................................................................ 416

A4.1 A minimum spanning tree: Europe ............................................................................................... 502

A4.2 Price data for CPI and ICP activities ............................................................................................ 504

A4.3 A sequence of price comparisons ................................................................................................... 505

List of boxes

13.1 Model presentation of consumer price index ................................................................................. 230

13.2 Model note on methodology – to be included in press releases on consumer price indices .......... 231

14.1 Institutional sectors in the System of National Accounts 1993 ....................................................... 238

14.2 Coverage of industries or activities by the producer price index in terms of aggregate output value 242

14.3 Treatment of housing and consumer durables in the system of national accounts and in consumerprice indices .................................................................................................................................... 244

CONTENTS

xviii

PREFACE

The International Labour Office (ILO), the International Monetary Fund (IMF), the Organisation forEconomic Co-operation and Development (OECD), the Statistical Office of the European Communities(Eurostat), the United Nations Economic Commission for Europe (UNECE) and the World Bank, togetherwith experts from a number of national statistical offices and universities, have collaborated since 1998 ondeveloping this manual. The sponsoring organizations endorse the principles and recommendations con-tained in it as good practice for statistical agencies in compiling their consumer price indices (CPIs). Becauseof practical and resource constraints, however, some of the current recommendations may not be imme-diately attainable by all statistical offices, and they should therefore serve as guidelines or targets for agenciesas they revise their CPIs and improve their CPI programmes. There are not always clear-cut solutions tospecific conceptual and practical problems such as sample design, choice of index formula, adjustment ofprices for quality changes, and the treatment of new products. Statistical offices must therefore rely on theunderlying economic and statistical principles laid out in this manual to arrive at practical solutions.

The consumer price indexThe CPI is an index that measures the rate at which the prices of consumption goods and services are

changing from month to month (or from quarter to quarter). The prices are collected from shops or otherretail outlets. The usual method of calculation is to take an average of the period-to-period price changes forthe different products, using as weights the average amounts that households spend on them. CPIs areofficial statistics that are usually produced by national statistical offices, ministries of labour or centralbanks. They are published as quickly as possible, typically about ten days after the end of the most recentmonth or quarter.

The manual is intended for the benefit of users of CPIs, as well as for the statistical agencies that compile theindices. It is designed to do two things. First, it explains in some detail the methods that are actually used tocalculate a CPI. Second, it explains the underlying economic and statistical theory on which the methods arebased.

A CPI measures the rate of price inflation as experienced and perceived by households in their role asconsumers. It is also widely used as a proxy for a general index of inflation for the economy as a whole,partly because of the frequency and timeliness with which it is produced. It has become a key statistic forpurposes of economic policy-making, especially monetary policy. It is often specified in legislation and in awide variety of private contracts as the appropriate measure of inflation for the purposes of adjustingpayments (such as wages, rents, interest and social security benefits) for the effects of inflation. It cantherefore have substantial and wide-ranging financial implications for governments and businesses, as wellas for households.

This manual provides guidelines for statistical offices or other agencies responsible for constructing a CPI,bearing in mind that the resources available for this purpose are limited. Calculating a CPI cannot be reducedto a simple set of rules or standard set of procedures that can be mechanically followed in all circumstances.While there are certain general principles that may be universally applicable, the procedures followed inpractice, whether they concern the collection or processing of the prices or the methods of aggregation, haveto take account of particular circumstances. These include the main use of the index, the nature of themarkets and pricing practices within the country, and the resources available to the statistical office. Sta-tistical offices have to make choices. The manual explains the underlying economic and statistical conceptsand principles needed to enable statistical offices to make their choices in efficient and cost-effective ways andto be aware of the full implications of their choices.

The manual draws upon the experience of many statistical offices throughout the world. The proceduresthey use are not static, but continue to evolve and improve in response to several factors. First, researchcontinually refines and strengthens the economic and statistical theory underpinning CPIs. For example,clearer insights have recently been obtained on the relative strengths and weaknesses of the various formulaeand methods used to process the basic price data collected for CPI purposes. Second, recent advances ininformation and communications technology have affected CPI methods. Both of these theoretical and datadevelopments can impinge on all the stages in compiling a CPI. New technology can affect the methods usedto collect prices and transmit them to the central statistical office. It can also improve the processing andchecking, including the methods used to adjust prices for changes in the quality of the goods and services

xix

covered. Finally, improved formulae help in calculating more accurate and reliable higher-level indices,including the overall CPI itself.

International standards for CPIsSome international standards for economic statistics have evolved primarily in order to enable inter-

nationally comparable statistics to be compiled. However, individual countries also stand to benefit frominternational standards. The CPI standards described in this manual draw upon the collective experience andexpertise accumulated in many countries. All countries can benefit by having easy access to this experienceand expertise.

In many countries, CPIs were first compiled mainly in order to be able to adjust wages to compensate forthe loss of purchasing power caused by inflation. Consequently, the responsibility for compiling CPIs wasoften entrusted to ministries, or departments, of labour. The International Conference of Labour Statis-ticians (ICLS), convened by the Governing Body of the ILO, therefore provided the natural forum in whichto discuss CPI methodology and develop guidelines.

The first international standards for CPIs were promulgated in 1925 by the Second ICLS. The first set ofstandards referred to ‘‘cost of living’’ indices rather than CPIs. A distinction is now drawn between twodifferent types of index. A consumer price index can be defined simply as measuring the change in the cost ofpurchasing a given ‘‘basket’’ of consumption goods and services, whereas a cost of living index is defined asmeasuring the change in the cost of maintaining a given standard of living, or level of utility. For this reason,the Tenth ICLS in 1962 decided to adopt the more general term ‘‘consumer price index’’, which should beunderstood to embrace both concepts. There need be no conflict between the two. As explained in themanual, the best-practice methods are likely to be very similar, whichever approach is adopted.

The international standards have been revised three times, in 1947, 1962 and 1987, in the form of reso-lutions adopted by the ICLS. The 1987 standards on CPI were followed by a manual on methods (Turvey,1989), which provided guidance to countries on the practical application of the standards.

The background to the present revisionA few years after the publication of the 1989 ILO manual, it became clear that a number of outstanding

and controversial methodological problems needed further investigation and analysis. An expert group wasformed consisting of specialists in price indices from national statistical offices, international organizationsand universities from around the world. It met for the first time in Ottawa in 1994, and became known asthe ‘‘Ottawa Group’’, one of the city groups established by the United Nations Statistical Commission toaddress selected problems in statistical methods. During the course of seven meetings of the Ottawa Groupbetween 1994 and 2003, over 100 research papers on the theory and practice of price indices were presentedand discussed. One outcome was that it became apparent that existing CPI methods could be improved andstrengthened in a number of ways.

At the same time, the control of inflation had become a high-priority policy objective in most countries.Not only is the CPI widely used to measure and monitor inflation, but inflation targets in many countriesare set specifically in terms of a precise rate of change in the CPI. The slowing down of inflation in manyparts of the world in the 1990s, as compared with the 1970s and 1980s, far from reducing interest in CPImethodology, actually stimulated a demand for more accurate, precise and reliable measures of inflation.When the rate of inflation slows to only 2 or 3 per cent per year, even a small error or bias in the CPIbecomes relatively significant.

In order to be sure about the accuracy of CPIs, governments or research institutes in a few countriescommissioned special groups of experts to investigate and evaluate the methods used. The methodology usedto calculate CPIs was subjected to public interest and scrutiny unknown in the past. One conclusion reachedwas that existing methods might lead to some upward bias. Many academic and government economists andother users of CPIs became convinced of this, believing that insufficient allowance was being made forimprovements in the quality of many goods and services. In fact, the extent and sometimes even the directionof such bias are uncertain. It will also, of course, vary between different types of consumption goods andservices, and its total effect on the overall CPI will vary between countries. However, the bias is potentiallylarge. For this reason, this manual addresses in some detail the issue of adjusting prices for changes inquality, drawing upon the most recent research in this area. There are other sources of possible bias, such asthat resulting from working with an out-of-date and unrepresentative basket of goods and services. Bias mayalso result from the sampling and price collection methods used. Several chapters deal with these issues, withan overall summary of possible errors and biases given in Chapter 11.

CPIs are widely used for the index linking of social benefits such as pensions, unemployment benefits andother government payments, and also as escalators for adjusting prices in long-term contracts. Thecumulative effects of even a small bias could be substantial over the long term and could have considerablefinancial consequences for government budgets. Government agencies, especially ministries of finance, have

PREFACE

xx

therefore taken a renewed interest in CPIs, examining their accuracy and reliability more closely andcarefully than in the past.

In response to the various developments outlined above, the need to revise, update and expand the 1989ILO manual was gradually recognized and accepted during the late 1990s. A formal recommendation to revisethe manual was made at the joint UNECE/ILO Meeting on Consumer Price Indices, held in Geneva at theend of 1997. Responsibility for the revision was entrusted to the main international organizations interested inthe measurement of inflation. This strategy was endorsed in 1998 by the United Nations Statistical Com-mission, which also agreed to the conversion of the Ottawa Group into a formal Intersecretariat WorkingGroup on Price Statistics (IWGPS). The Sixteenth ICLS, meeting in 1998, also recommended that theFourteenth ICLS resolution concerning consumer price indices, adopted in 1987, should be revised. Thepreparation of the draft revised resolution discussed at the Seventeenth ICLS (24 November–3 December2003) was carried out by the ILO Bureau of Statistics in parallel with the preparation of this revised manual.Every effort has been made to ensure that the two documents are consistent and mutually supportive.1

Some concerns about current index methodsThis new manual takes advantage of the wealth of new research on index number theory and methods in

the last decade to address the kinds of concerns referred to above. It recommends some new practices and itspurpose is not simply to codify existing statistical agency practices. It is useful to highlight a few of the mainconcerns that have led to many topics being dealt with in some depth in the manual.

The traditional standard methodology underlying a typical CPI is based on the concept of a Laspeyresprice index. A Laspeyres index measures the change between two periods of time in the total cost ofpurchasing a basket of goods and services that is representative of the first, or base, period. The base periodbasket of consumer purchases is priced first at base period prices and then repeatedly priced at the prices ofsuccessive time periods. This methodology has at least three practical advantages. It is easily explained tothe public; it can make repeated use of the same data on consumer purchases that date from some pasthousehold survey or administrative source (rather than requiring new data each month); and it need not berevised, assuming users are satisfied with the Laspeyres concept. Another notable advantage is that theLaspeyres is consistent in aggregation down to the lowest level of aggregation. The index can be brokendown into sub-aggregates that are interrelated in a simple way.

Statistical agencies actually calculate their CPIs by implementing the Laspeyres index in its alterna-tive form as a weighted average of the observed price changes, or price relatives, using the base periodexpenditure shares as weights. Unfortunately, although the Laspeyres is a simple concept, it is difficult tocalculate a proper Laspeyres index in practice. Consequently, statistical agencies have to resort toapproximations:

� It is generally impossible to obtain accurate expenditure shares for the base period at the level of individualcommodities, so statistical agencies settle for getting base period expenditure weights at the level of100–1,000 product groups.

� For each of the chosen product groups, agencies collect a sample of representative prices from outletsrather than attempting to collect every single transaction price. They use equally weighted (rather thanexpenditure-weighted) index formulae to aggregate these elementary product prices into an elementaryaggregate index, which will in turn be used as the price relative for each of the 100–1,000 product groupswhen calculating the higher-level Laspeyres index. It is recognized that this two-stage procedure is notentirely consistent with the Laspeyres methodology (which requires weighting at each stage of aggre-gation). However, for a number of theoretical and practical reasons, statistical agencies judge theresulting elementary index price relatives to be sufficiently accurate to insert into the Laspeyres formulaat the higher stage of aggregation.

This methodology dates back to the work of Mitchell (1927) and Knibbs (1924), and other pioneers whointroduced it 80 or 90 years ago, and it is still used today.

Although most statistical agencies have traditionally used the Laspeyres index as their target index, botheconomic and index number theory suggest that some other types of indices may be more appropriatetarget indices to aim at: namely, the Fisher, Walsh or Tornqvist–Theil indices. As is well known, theLaspeyres index has an upward bias compared with these target indices. Of course, these target indices maynot be achievable by a statistical agency, but it is necessary to have some sort of theoretical target to aim at.Having a target concept is also necessary so that the index that is actually produced by a statistical agencycan be evaluated to see how close it comes to the theoretical ideal. In the theoretical chapters of the manual,four main approaches to index number theory are described:

1The 2003 resolution concerning consumer price indices is reproduced in Annex 3. It can also be found on the ILO Bureau of Statisticsweb site: http://www.ilo.org/public/english/bureau/stat.

xxi

PREFACE

(1) fixed basket approaches and symmetric averages of fixed baskets;(2) the stochastic (statistical estimator) approach to index number theory;(3) test (axiomatic) approaches; and(4) the economic approach.

Approaches (3) and (4) will be familiar to many price statisticians and expert users, but perhaps a few wordsabout approaches (1) and (2) are in order.

The Laspeyres index is an example of a basket index. The concern from a theoretical point of view is thatthere is an equally valid alternative for the two periods being compared: the Paasche index, which uses thebasket of quantities from the current period. If there are two equally valid estimators for the same concept,then statistical theory suggests taking an average of the two. However, there is more than one kind ofaverage and the question of which average to take is not trivial. The manual proposes that the ‘‘best’’average is the geometric average of the Laspeyres and Paasche indices (the Fisher ideal). Alternatively, the‘‘best’’ basket is one whose quantities are geometric averages of the quantities in both periods (the Walshindex). From the statistical estimation perspective, the ‘‘best’’ index number is a geometric average of theprice relatives that uses the (arithmetic) average expenditure share in two periods as weights (the Tornqvist–Theil index).

One additional result from index number theory should be mentioned here: the problem of defining theprice and quantity of a product that should be used for each period in the index number formula. Theproblem is that the same product may be sold at a number of different prices. So the question arises, whatprice would be most representative of the sales of this product for the period? The answer is the unit value,since this price multiplied by the total quantity sold during the period equals the value of sales. Of course,the manual does not endorse taking unit values over heterogeneous products; unit values should only becalculated for identical products.

Six main areas of concern with the standard methodology are listed below. They are not ranked in orderof importance, and all are considered to be important:

1. At the final stage of aggregation, a conventional CPI is not a true Laspeyres index since the expenditureweights pertain to a reference base year that is different from the base month (or quarter) for prices. Thus,the expenditure weights are annual whereas the prices are collected monthly. To be a true Laspeyresindex, the period that provides the expenditure weights must coincide with the reference period for theprices. In fact, the index actually calculated by many statistical agencies at the last stage of aggregationhas a weight reference period that precedes the base price period. Indices of this type are likely to havesome upward bias compared to a true Laspeyres index, especially if the expenditure weights are price-updated from the weight reference period to the Laspeyres base period. It follows that they must havedefinite upward biases compared to theoretical target indices such as the Fisher, Walsh or Tornqvist–Theil indices.

2. At the early stages of aggregation, unweighted averages of prices or price relatives are used. Untilrecently, when scanner data from electronic points of sale became more readily available, it was thoughtthat the biases that might result from the use of unweighted indices were not particularly significant.However, recent evidence suggests that there is potential for significant upward bias at lower levels ofaggregation compared to results that are generated by the preferred target indices mentioned above.

3. The third major concern with standard CPI methodology is that, although statistical agencies generallyrecognize that there is a problem with the treatment of quality change and new goods, it is difficult towork out a coherent methodology for these problems in the context of a Laspeyres index that uses a fixedset of quantities. The most widely received good practice in quality adjusting price indices is ‘‘hedonicregression’’, which characterizes the price of a product at any given time as a function of its physical andeconomic characteristics as compared with substitutes. In fact, there is a considerable amount of con-troversy on how to integrate hedonic regression methodology into the CPI’s theoretical framework. Boththe theoretical and the more practically oriented chapters in the manual devote a lot of attention to thesemethodological issues. The problems created by the disappearance of old, and the appearance of new,products are now much more severe than they were when the traditional CPI methodology was developedsome 80 years ago (when the problem was mostly ignored). For many categories of products, such asmodels of consumer durables, those priced at the beginning of the year are simply no longer available bythe end of the year. Sample attrition creates tremendous methodological problems. At lower levels ofaggregation, it becomes necessary (at least in many product groups) to use chained indices rather thanfixed base indices. Certain unweighted indices are liable to have substantial bias when chained.

4. A fourth major area of concern is related to the first: that is, the treatment of seasonal commodities. The useof annual quantities or annual expenditure shares is justified to a certain extent if one is interested in thelonger-run trend of price changes. However, some users, such as central banks, focus on short-term,month-to-month changes, in which case the use of annual weights can lead to misleading signals. Monthlyprice changes for products that are out of season (i.e., the seasonal weights for the product class are small

xxii

PREFACE

for those months) can be greatly magnified by the use of annual weights. The problem is worse when theproducts are not available at all at certain months of the year. There are solutions to these seasonalityproblems, but they may not appeal to many CPI compilers and users since they involve the construction oftwo indices: one for the short-term measurement of price changes and another (more accurate) longer-term index that is adjusted for seasonal influences.

5. A fifth concern with standard CPI methodology is that, in common with most economic statistics,services have been comparatively neglected in CPIs, notwithstanding the fact that they have becomeextremely important. A typical CPI will collect many more goods prices than services prices and willhave many more product groups for goods rather than services. Traditionally, there has not been muchfocus on the problems involved in measuring price and quantity changes for services, even though theyraise serious conceptual and practical problems. Some examples of difficult-to-measure services are:insurance, gambling, financial services, advertising, telecommunications, entertainment and housingservices. In many cases, statistical agencies simply do not have the resources or methodologies at theirdisposal to deal adequately with these difficult measurement problems.

6. A final concern with existing CPI methodology is that it tends not to recognize that more than one CPImay be required to meet the needs of different users. For example, some users may require informationon the month-to-month movement of prices in a timely fashion. This requires a basket index withpredetermined (even though possibly inappropriate and out-of-date) weights that are instantly available.However, other users may be more interested in a more accurate or representative measure of pricechange and may be willing to sacrifice timeliness for increased accuracy. For this reason, the UnitedStates Bureau of Labor Statistics provides, on a retrospective basis, a superlative index that uses bothcurrent and base period weight information in a symmetrical way. This is an entirely reasonabledevelopment, recognizing that different users have different needs. A second example where more thanone index might be compiled relates to owner-occupied housing. Good cases have been made for threedifferent treatments: the acquisitions approach, the rental equivalence approach and the user costapproach. However, these three approaches may give quite different numerical results in the short run. Astatistical agency has to opt for one approach, but since all three command support, indices using theother two approaches could be made available as analytical series for interested users. A third exampleof where more than one index would be useful occurs when, because of seasonal commodities, themonth-to-month index may not be based on the same set of products as one that compares the monthwith the same month a year earlier.

The above kinds of concern are addressed in this manual. Frank discussions of these matters shouldstimulate the interest of professional economists and statisticians in universities, government departments,central banks, and so on, to address these measurement problems and to provide new solutions that can beused by statistical agencies. Public awareness of these areas should also heighten awareness of the need foradditional resources to be allocated to statistical agencies so that economic measurement will be improved.

The Harmonized Indices of Consumer PricesWithin the European Union (EU), the convergence of inflation in Member States was an important

prerequisite for the formation of a monetary union in 1999. This required a precisely defined measure ofinflation and an agreed methodology to ensure that the different countries’ price indices are comparable. Adetailed and systematic review of all aspects of the compilation of CPIs was therefore undertaken duringthe 1990s by all the national statistical offices of the EU Member States in collaboration with Eurostat, theStatistical Office of the EU. This work culminated in the elaboration of a new EU standard for the 29Member and candidate States, and led to the development of the EU’s Harmonized Indices of ConsumerPrices (HICPs). A summary of HICP methodology is given in Annex 1 to this manual.

Work on the HICPs proceeded in parallel with that of the IWGPS, many of whose experts also parti-cipated both in work on the HICPs and in the present revision of this manual. Although the methodologyelaborated here has much in common with that adopted for the HICPs, there are also differences. TheHICPs were developed for a very specific purpose, whereas the methodology developed in this manual isintended to be flexible, multi-purpose and applicable to all countries, whatever their economic circum-stances and level of development. The manual also provides considerably more detail, information,explanation and rationalization of CPI methodology and the associated economic and statistical theorythan is to be found in the HICP standards.

The organization of the revisionThe six international organizations listed at the beginning of this preface, concerned with both the

measurement of inflation and policies designed to control it, have collaborated on the revision of thismanual. They have provided, and continue to provide, technical assistance on CPIs to countries at all levels

xxiii

PREFACE

of development, including those in transition from planned to market economies. They joined forces for therevision of this manual, establishing the IWGPS for the purpose. The role of the IWGPS was to organize andmanage the process rather than act as an expert group.

The responsibilities of the IWGPS were to:

� appoint the various experts on price indices who participated in the revision process, either as members ofthe Technical Expert Group (TEG/CPI), providing substantive advice on the content of the manual, oras authors;

� provide the financial and other resources needed;

� arrange meetings of the TEG/CPI, prepare the agendas and write the reports of the meetings; and

� arrange for the publication and dissemination of the manual.

Members of the IWGPS were also members of the TEG/CPI. It is important to note that the expertsparticipating in the TEG/CPI were invited in their personal capacity as experts and not as representatives, ordelegates, of the national statistical offices or other agencies in which they might be employed. Participantswere able to give their expert opinions without in any way committing the offices from which they came.

The revision of the manual took five years, and involved multiple activities:

� the development of the manual outline and the recruitment of experts to draft the various chapters;

� the review of the draft chapters by the members of the TEG/CPI, the IWGPS and other experts;

� the posting of the draft chapters on a special web site for comment by interested individuals andorganizations;

� discussions by a small group of experts from statistical agencies and universities on the finalization of allthe chapters;

� final copy-editing of the whole manual.

Links with the Producer price index manualOne of the first decisions of the IWGPS was that a new international manual on producer price indices

(PPIs) should be produced simultaneously with this manual. Whereas there have been internationalstandards for CPIs for over 70 years, the first international manual on producer price indices was notproduced until 1979 (United Nations, 1979). Despite the importance of PPIs for measuring and analysinginflation, the methods used to compile them have been comparatively neglected, at both national andinternational levels.

A new Producer price index manual (ILO, IMF, OECD, Eurostat, UNECE and the World Bank, forth-coming) has therefore been developed and written in parallel with this manual. The IWGPS established asecond Technical Expert Group on PPIs whose membership overlapped with that of the Technical ExpertGroup on CPIs. The two groups worked in close liaison with each other. The methodologies of PPIs andCPIs have much in common. Both are based on essentially the same underlying economic and statisticaltheory, except that the CPI draws on the economic theory of consumer behaviour whereas the PPI draws onthe economic theory of production. However, the two economic theories are isomorphic and lead to thesame kinds of conclusions with regard to index number compilation. The two manuals have similar contentsand are fully consistent with each other conceptually, sharing common text when appropriate.

Most members of the Technical Expert Groups on CPIs and PPIs also participated as active members ofthe Ottawa Group. The two manuals were able to draw upon the contents and conclusions of all thenumerous papers presented at meetings of the Group.

xxiv

PREFACE

ACKNOWLEDGEMENTS

The organizations of the IWGPS wish to thank all those involved in the drafting and production of themanual. Particular thanks go to Peter Hill, the editor, W. Erwin Diewert, who contributed extensively tothe theoretical chapters of the manual, and Bert Balk, who acted as referee for all the theoretical chapters.Their efforts greatly enhanced the quality of the manual.

The authors of the chapters are as follows:

Preface Peter Hill, Paul Armknecht and W. Erwin Diewert

Reader’s guide Peter Hill

1 An introduction to consumer price index methodology Peter Hill

2 Uses of consumer price indices Peter Hill

3 Concepts and scope Peter Hill and Fenella Maitland-Smith

4 Expenditure weights and their sources Valentina Stoevska and Carsten Boldsen

5 Sampling Jorgen Dalen, A. Sylvester Young and Bert Balk

6 Price collection David Fenwick

7 Adjusting for quality change Mick Silver

8 Item substitution, sample space and new products Mick Silver

9 Calculating consumer price indices in practice Carsten Boldsen and Peter Hill

10 Some special cases Keith Woolford, David Fenwick, contributors from several statistical offices

11 Errors and bias John Greenlees and Bert Balk

12 Organization and management David Fenwick

13 Publication, dissemination and user relations Tom Griffin

14 The system of price statistics Kimberly Zieschang

15 Basic index number theory W. Erwin Diewert

16 The axiomatic and stochastic approaches to index number theory W. Erwin Diewert

17 The economic approach to index number theory: The single-household case W. Erwin Diewert

18 The economic approach to index number theory: The many-household case W. Erwin Diewert

19 Price indices using an artificial data set W. Erwin Diewert

20 Elementary indices W. Erwin Diewert

21 Quality change and hedonics Mick Silver

22 The treatment of seasonal products W. Erwin Diewert

23 Durables and user costs W. Erwin Diewert

A glossary of main terms and annex to the glossary Peter Hill and Bert Balk

Annexes1 Harmonized Indices of Consumer Prices (European Union) Alexandre Makaronidis, Keith Hayes

2 Classification of Individual Consumption according to Purpose (COICOP)-Extract United Nations

3 Resolution concerning consumer price indices adopted by the Seventeenth International Conference ofLabour Statisticians, 2003 ILO

4 Spatial comparisons of consumer prices, purchasing power parities and the International ComparisonProgram Prasada Rao

xxv

The manual has also benefited from valuable contributions by many other experts, including: Martin Boon(Statistics Netherlands); Heber Camelo and Ernestina Perez (Economic Commission for Latin America andthe Caribbean); Denis Fixler (United States Bureau of Economic Analysis); Leendert Hoven (StatisticsNetherlands); Michel Mouyelo-Katoula (African Development Bank); Carl Obst (formerly OECD); Bou-chaib Thich (Departement de la prevision economique et du plan, Morocco); and Ralph Turvey (expert). Thefollowing also gave helpful advice and comments: Statistics Austria; Statistics Singapore; United States BLS;Michael Anderson (ABS); Rob Edwards (ABS); Eivind Hoffmann (ILO); participants at the InternationalWorkshop on Consumer Price Indices, Singapore, June 2001; and the members of the Ottawa Group.

The IWGPS established the Technical Expert Group on the CPI (TEG/CPI) for the revision of themanual. Members of the IWGPS were also members of the TEG/CPI, whose individual members were:

David Fenwick Chair, United Kingdom ONSPaul Armknecht TEG/PPI chair, IMFJohn Astin* EurostatBert Balk Statistics NetherlandsW. Erwin Diewert University of British Columbia, CanadaYoel Finkel Israel Central Bureau of StatisticsCarsten Boldsen Statistics DenmarkJohn Greenlees United States BLSPaul Haschka Statistics AustriaPeter Hill EditorJean-Claude Roman* EurostatBohdan Schultz* Statistics CanadaMick Silver Cardiff University, United KingdomKimberly Zieschang IMF

The UNECE (Jan Karlsson, Lidia Bratanova*, Miodrag Pesut*, Tihomira Dimova*) and the ILO(Valentina Stoevska) jointly acted as the Secretariat of the TEG/CPI.

The TEG/CPI met seven times: 11–12 February 1999 (Geneva), 2 November 1999 (Geneva), 5–6 February2001 (Washington, DC), 25–26 June 2001 (Geneva), 31 October 2001 (Geneva), 19–21 March 2002 (Lon-don) and 14–15 October 2002 (London).

The IWGPS met formally five times: 24 September 1998 (Paris), 11 February 1999 (Geneva), 2 November1999 (Geneva), 21–22 March 2002 (London) and 5 December 2003 (Geneva). A number of informalmeetings were also held.

The ILO was the Secretariat of the Group and A. Sylvester Young the chairperson of the IWGPS.During the revision of the manual, the CPI manual editor (Peter Hill), the TEG-CPI chairperson (DavidFenwick), the PPI manual editor and the TEG/PPI chairperson (Paul Armknecht) participated in themeetings of the IWGPS.

The affiliations of the authors are as follows:

Bert Balk Statistics NetherlandsCarsten Boldsen Statistics DenmarkJorgen Dalen ExpertW. Erwin Diewert University of British Columbia, CanadaDavid Fenwick United Kingdom Office of National Statistics (ONS)John Greenlees United States Bureau of Labor Statistics (BLS)Tom Griffin ExpertKeith Hayes EurostatPeter Hill Expert, manual editorFenella Maitland-Smith OECDAlexandre Makaronidis EurostatPrasada Rao University of Queensland, AustraliaMick Silver Cardiff University, United KingdomValentina Stoevska ILOKeith Woolford Australian Bureau of Statistics (ABS)A. Sylvester Young ILOKimberly Zieschang IMF

* These members served for only part of the period.

xxvi

ACKNOWLEDGEMENTS

The final publication of the English version of this manual was coordinated, with the involvement of theIWGPS member organizations, by Valentina Stoevska of the ILO Bureau of Statistics. The ILO Bureau ofPublications provided extensive editorial and support services for the production process. We should alsolike to thank Angela Haden and Barbara Campanini for their thorough copy-editing of the final draft.

xxvii

ACKNOWLEDGEMENTS

READER’S GUIDE

International manuals in the field of economic statistics have traditionally been intended to provide guidanceabout concepts, definitions, classifications, coverage, valuation, the recording of data, aggregation pro-cedures, formulae, and so on. They have been intended mainly to assist compilers of the relevant statistics inindividual countries. This manual has the same principal objective.

The manual is also intended for the benefit of users of consumer price indices (CPIs), such as governmentand academic economists, financial experts and other informed users. The CPI is a key statistic for policypurposes. It attracts a great deal of attention from the media, governments and the public at large in mostcountries. Despite its apparent simplicity, the CPI is a sophisticated concept that draws upon a great deal ofeconomic and statistical theory, and requires complex data manipulation. This manual is therefore alsointended to promote greater understanding of the properties of CPIs.

In general, compilers and users of economic statistics must have a clear perception of what the statisticsare supposed to measure, in principle. Measurement without theory is unacceptable in economics, as inother disciplines. The manual therefore contains a thorough, comprehensive and up-to-date survey of therelevant economic and statistical theory. This makes the manual completely self-contained on both thetheory and practice of CPI measurement.

The resulting manual is large. As different readers may have different interests and priorities, it is notpossible to devise a sequence of chapters that suits everyone. Indeed, because this manual is intended toserve as a reference source, it will not necessarily be read from cover to cover. Many readers may beinterested in only a selection of chapters. The purpose of this reader’s guide is to provide a map of thecontents of the manual that will assist readers with different interests and priorities.

An overview of the sequence of chaptersChapter 1 is a general introduction to CPI methodology, and is intended for all readers. It provides the

basic information needed to understand the subsequent chapters. It summarizes index number theory, asexplained in detail in Chapters 15 to 23, and outlines the main steps involved in the actual compilation of aCPI, drawing on material in Chapters 3 to 9. It does not provide a summary of the manual as whole, as itdoes not cover some specific topics and special cases that are not of general relevance.

Chapter 2 explains how CPIs have evolved in response to the demands made upon them and how the usesof CPIs affect the choice of methodology to be used. Chapter 3 is concerned with a number of basicconcepts, principles and classifications, as well as with the scope or coverage of an index. The scope of aCPI can vary significantly from country to country.

Chapters 4 to 9 form an interrelated sequence describing the various steps involved in the compilation ofa CPI from the collection and processing of the price data through to the calculation of the final index.Chapter 4 explains how the expenditure weights attached to the price changes for different goods andservices are derived. These weights are typically based on household expenditure surveys supplemented bydata from other sources.

Chapter 5 deals with sampling issues. A CPI is essentially an estimate based on a sample of prices.Chapter 5 considers sampling design, and the pros and cons of random versus purposive sampling. Chapter6 is devoted to the procedures actually used to collect the prices from a selection of retail outlets or othersuppliers. It deals with topics such as questionnaire design, the specification of the items selected, the use ofscanner data and the use of hand-held computers.

Chapter 7 addresses the difficult question of how to adjust prices for changes over time in the quality ofthe goods or services selected. Changes in value resulting from changes in quality count as changes inquantity, not price. Disentangling the effects of quality change poses serious theoretical and practicalproblems for compilers. Chapter 8 covers the closely related question of how to deal with new goods orservices not previously purchased and for which there are no prices in earlier periods.

Chapter 9 pulls together the material contained in the preceding five chapters and gives a step-by-stepsummary of the various stages of CPI calculation. It describes both the elementary price indices calculatedfrom the raw prices collected for small groups of products and the subsequent averaging of the elementaryindices to obtain indices at higher levels of aggregation up to the overall CPI itself.

Chapter 10 deals with a number of cases that require special treatment: for example, goods and servicesfor which prices are not quoted separately, being embedded within composite transactions covering more

xxix

than one item. It also examines the case of owner-occupied housing. Chapter 11 considers the errors andbiases to which CPIs may be subject.

Chapter 12 deals with issues of organization and management. Conducting the price surveys and pro-cessing the results is a massive operation that requires careful planning and organization, and also efficientmanagement. Chapter 13 is concerned with the publication or dissemination of the results.

Chapter 14 marks a break in the sequence of chapters, as it not concerned with the compilation of a CPI.Its purpose is different, namely to examine the place of the CPI in the general system of price statistics. TheCPI should not be treated as an independent, isolated statistic. The flow of consumer goods and services towhich it relates is itself only one of a set of interdependent flows within the economy as a whole.The analysis of inflation requires more than one index, and it is essential to know exactly how the CPIrelates to the producer price index (PPI) and to other price indices, such as indices of export and importprices. The supply and use matrix of the System of National Acounts provides the appropriate conceptualframework within which to examine these interrelationships.

Chapters 15 to 18 provide a systematic and detailed exposition of the index number and economic theoryunderlying CPIs. Five different approaches to index number theory are examined that between them coverall aspects of index number theory. Collectively, they provide a comprehensive and up-to-date survey ofindex number theory, including recent methodological developments as reported in journals and conferenceproceedings.

Chapter 15 provides an introduction to index number theory focusing on the decomposition of valuechanges into their price and quantity components. Chapter 16 examines the axiomatic and stochasticapproaches to CPIs. The axiomatic, or test, approach lists a number of properties that it is desirable forindex numbers to possess and tests specific formulae to see whether or not they possess them.

Chapter 17 explains the economic approach based on the economic theory of consumer behaviour. Onthis approach, a CPI is defined as a cost of living index (COLI). Although COLIs cannot be calculateddirectly, a certain class of index numbers, known as superlative indices, can be expected to approximateCOLIs in practice. An increasing number of economists and other users have concluded that, in principle,the preferred, ideal index for CPI purposes should be a superlative index, such as the Fisher index. This isreinforced by the fact that the Fisher also emerges as a very desirable index on axiomatic grounds.

Chapter 18 deals with aggregation issues. Chapter 19 uses a constructed data set to illustrate thenumerical consequences of using different index number formulae. It demonstrates that, in general, thechoice of index number formula can make a considerable difference, but that different superlative indices alltend to approximate each other.

Chapter 20 addresses the important question of what is the theoretically most appropriate form ofelementary price index to calculate at the first stage of CPI compilation when no information is available onquantities or expenditures. This has been a comparatively neglected topic until recently, even though thechoice of formula for an elementary index can have a significant impact on the overall CPI. The elementaryindices are the basic building blocks from which CPIs are constructed.

Chapters 21 to 23 deal with difficult issues. Chapter 21 discusses adjusting for quality change, includingthe hedonic approach, from a theoretical viewpoint. Chapter 22 examines the treatment of seasonal pro-ducts. Finally, Chapter 23 considers the treatment of durable goods. There is some tension in both nationalaccounts and CPIs resulting from the fact that owner-occupied houses are treated as assets, whereasconsumer durables are not. These treatments are not easy to reconcile conceptually and Chapter 23 dis-cusses the theoretical issues involved.

The manual concludes with a glossary of terms, a bibliography, and four annexes on the following topics:

� the Harmonized Indices of Consumer Prices (HICPs) of the European Union;

� the Classification of Individual Consumption according to Purpose (COICOP), a household expenditureclassification;

� the resolution concerning consumer prices indices adopted by the Seventeenth International Conferenceof Labour Statisticians, 2003;

� spatial comparisons of consumer prices, using purchasing power parities and the International Com-parison Program.

Suggested reading plansDifferent readers may have different needs and priorities. Readers interested mainly in the compilation of

CPIs may not wish to pursue all the finer points of the underlying economic and statistical theory. Con-versely, readers interested more in the use of CPIs for analytic or policy purposes may not be so interestedin reading about the technicalities of conducting and managing price surveys.

Not all readers will want to read the entire manual but all readers, whatever their preferences, will find ituseful to read the first three chapters. Chapter 1 provides a general introduction to the whole subject bygiving an overview of the CPI theory and practice that is presented in the manual. It covers the basic

READER’S GUIDE

xxx

knowledge required for understanding subsequent chapters. Chapter 2 explains why CPIs are calculated andwhat they are used for. Chapter 3 examines a number of fundamental concepts and the scope of a CPI.

A reading plan for compilersChapters 4 to 13 are primarily for compilers. They follow a logical sequence that roughly matches the

various stages of the actual compilation of a CPI, starting with the derivation of the expenditure weightsand the collection of the price data, and finishing with the publication of the final index.

Chapter 14 is intended equally for compilers and users of CPIs. It places CPIs in perspective within theoverall system of price indices.

The remaining chapters from 15 to 23 are mainly theoretical. Compilers may find it necessary to pursuecertain theoretical topics in greater depth, in which case they have immediate access to the relevant material.It would be desirable for compilers to be acquainted with at least the basic index number theory set out inChapter 15 and the numerical example developed in Chapter 19. The material in Chapter 20 on elementaryprice indices is also particularly important for compilers.

A reading plan for usersAlthough all readers will find Chapters 1 to 3 useful, the subsequent ten chapters are designed primarily

for compilers. Two topics that have, however, aroused considerable interest among many users are thetreatment of quality change and new products. These are discussed at some length in Chapters 7 and 8.Users may also find Chapter 9 particularly helpful as it provides a concise description of the various stagesof compiling a CPI.

Chapter 11 on errors and bias, and Chapter 14 on the system of price statistics are also of equal interestto users and compilers.

Chapters 15 to 23 covering the underlying economic and statistical theory are likely to be of interest tomany users, especially professional economists and students of economics.

ReferencesIn the past, international manuals on economic statistics have not usually provided references to the

associated literature. It was not considered helpful to cite references when the literature was mostly confinedto printed volumes, including academic journals or proceedings of conferences, located only in university ormajor libraries. Compilers working in many statistical offices were unlikely to have ready access to suchliterature. This situation has been completely transformed by the Internet and the Web, which make all suchliterature readily accessible. Accordingly, this manual breaks with past tradition by including a compre-hensive bibliography on the very large literature that exists on index number theory and practice.

READER’S GUIDE

xxxi

1AN INTRODUCTION TO CONSUMER PRICEINDEX METHODOLOGY

1.1 A price index is a measure of the proportionate,or percentage, changes in a set of prices over time. Aconsumer price index (CPI) measures changes in theprices of goods and services that households consume.Such changes affect the real purchasing power of con-sumers’ incomes and their welfare. As the prices of dif-ferent goods and services do not all change at the samerate, a price index can only reflect their average move-ment. A price index is typically assigned a value of unity,or 100, in some reference period and the values of theindex for other periods of time are intended to indicatethe average proportionate, or percentage, change in pricesfrom this price reference period. Price indices can alsobe used to measure differences in price levels betweendifferent cities, regions or countries at the same point intime.1.2 Much of this manual and the associated eco-

nomic literature on price indices is concerned with twobasic questions:

� Exactly what set of prices should be covered by theindex?

� What is the most appropriate way in which to averagetheir movements?

These two questions are addressed in the early sectionsof this introduction.1.3 Consumer price indices (CPIs) are index num-

bers that measure changes in the prices of goods andservices purchased or otherwise acquired by households,which households use directly, or indirectly, to satisfytheir own needs and wants. Consumer price indices canbe intended to measure either the rate of price inflationas perceived by households, or changes in their cost ofliving (that is, changes in the amounts that the house-holds need to spend in order to maintain their standardof living). There need be no conflict between these twoobjectives. In practice, most CPIs are calculated asweighted averages of the percentage price changes for aspecified set, or ‘‘basket’’, of consumer products, theweights reflecting their relative importance in householdconsumption in some period. Much depends on howappropriate and timely the weights are.1.4 This chapter provides a general introduction to,

and overview of, the methodology for compiling CPIs.It provides a summary of the relevant theory andpractice of index number compilation that is intendedto facilitate the reading and understanding of thedetailed chapters that follow, some of which are inevi-tably quite technical. It describes all the various stepsinvolved in CPI compilation starting with the basicconcept, definition and purpose of a CPI, followed bythe sampling procedures and survey methods used to

collect and process the price data, and finishing with asummary of the actual calculation of the index and itsdissemination.

1.5 An introductory presentation of CPI methodol-ogy has to start with the basic concept of a CPI and theunderlying index number theory, including the proper-ties and behaviour of the various kinds of index numberthat are, or might be, used for CPI purposes. In prin-ciple, it is necessary to settle what type of index to cal-culate before going on to consider the best way in whichto estimate it in practice, taking account of the resourcesavailable.

1.6 The main topics covered in this chapter are asfollows:

– the origins and uses of CPIs;

– basic index number theory, including the axiomaticand economic approaches to CPIs;

– elementary price indices and aggregate CPIs;

– the transactions, activities and households covered byCPIs;

– the collection and processing of the prices, includingadjusting for quality change;

– the actual calculation of the CPI;

– potential errors and bias;

– organization, management and dissemination policy.

In contrast, in this manual, the chapters dealing withindex theory come later on; thus the presentation in thischapter does not follow the same order as the corre-sponding chapters of the manual.

1.7 It is not the purpose of this introduction toprovide a complete summary of the contents of themanual. The objective is rather to provide a short pre-sentation of the core methodological issues with whichreaders need to be acquainted before tackling thedetailed chapters that follow. Some special topics, suchas the treatment of certain individual products whoseprices cannot be directly observed, are not consideredhere as they are not central to CPI methodology.

The origins and uses ofconsumer price indices

1.8 CPIs must serve a purpose. The precise way inwhich they are defined and constructed depends verymuch on what they are meant to be used for, and bywhom. As explained in Chapter 15, CPIs have a longhistory dating back to the eighteenth century. Laspeyresand Paasche indices, which are still widely used today,were first proposed in the 1870s. They are explained

1

below. The concept of the cost of living index wasintroduced early in the twentieth century.

1.9 Traditionally, one of the main reasons for com-piling a CPI was to compensate wage-earners for infla-tion by adjusting their wage rates in proportion to thepercentage change in the CPI, a procedure known asindexation. For this reason, official CPIs tended to be-come the responsibility of ministries of labour, but mostare now compiled by national statistical offices. A CPIthat is specifically intended to be used to index wages isknown as a compensation index.

1.10 CPIs have three important characteristics. Theyare published frequently, usually every month but some-times every quarter. They are available quickly, usuallyabout two weeks after the end of the month or quarter.They are also usually not revised. CPIs tend to be closelymonitored and attract a lot of publicity.

1.11 As CPIs provide timely information about therate of inflation, they have also come to be used for awide variety of purposes in addition to indexing wages.For example:

� CPIs are widely used to index pensions and socialsecurity benefits.

� CPIs are also used to index other payments, such asinterest payments or rents, or the prices of bonds.

� CPIs are also commonly used as a proxy for the gen-eral rate of inflation, even though they measure onlyconsumer inflation. They are used by some govern-ments or central banks to set inflation targets forpurposes of monetary policy.

� The price data collected for CPI purposes can also beused to compile other indices, such as the price indicesused to deflate household consumption expendituresin national accounts, or the purchasing power paritiesused to compare real levels of consumption in differ-ent countries.

1.12 These varied uses can create conflicts of inter-est. For example, using a CPI as an indicator of generalinflation may create pressure to extend its coverage toinclude elements that are not goods and services con-sumed by households, thereby changing the nature andconcept of the CPI. It should also be noted that becauseof the widespread use of CPIs to index a wide variety ofpayments – not just wages, but social security benefits,interest payments, private contracts, etc. – extremelylarge sums of money turn on their movements, enoughto have a significant impact on the state of governmentfinances. Thus, small differences in the movements ofCPIs resulting from the use of slightly different formulaeor methods can have considerable financial implications.CPI methodology is important in practice and not justin theory.

Choice of index number1.13 The first question is to decide on the kind of

index number to use. The extensive references dealingwith index theory in the bibliography reflect the fact thatthere is a very large literature on this subject. Manydifferent kinds of mathematical formulae have beenproposed over the past two centuries. While there may

be no single formula that would be preferred in all cir-cumstances, most economists and compilers of CPIsseem to be agreed that, in principle, the index formulashould belong to a small class of indices called super-lative indices. A superlative index may be expected toprovide an approximation to a cost of living index. Acharacteristic feature of a superlative index is that ittreats the prices and quantities in both periods beingcompared symmetrically. Different superlative indicestend to have similar properties, yield similar results andbehave in very similar ways. Because of their propertiesof symmetry, some kind of superlative index is alsolikely to be seen as desirable, even when the CPI is notmeant to be a cost of living index.

1.14 When a monthly or quarterly CPI is first pub-lished, however, it is invariably the case that there is notsufficient information on the quantities and expendituresin the current period to make it possible to calculate asymmetric, or superlative, index. While it is necessary toresort to second-best alternatives in practice, being ableto make a rational choice between the various possibil-ities means having a clear idea of the target index thatwould be preferred in principle. The target index canhave a considerable influence on practical matters suchas the frequency with which the weights used in theindex should be updated.

1.15 A comprehensive, detailed, rigorous and up-to-date discussion of the relevant index number theory isprovided in Chapters 15 to 23 of the manual. The fol-lowing sections provide a summary of this material.Proofs of the various propositions or theorems stated inthis chapter are to be found in the later chapters, towhich the reader may refer for further explanation.

Price indices based on basketsof goods and services

1.16 The purpose of an index number may beexplained as comparing the values of households’ expen-ditures on consumer goods and services in two timeperiods. Knowing that expenditures have increased by5 per cent is not very informative if we do not know howmuch of this change is attributable to changes in theprices of the goods and services, and how much tochanges in the quantities purchased. The purpose of anindex number is to decompose proportionate or per-centage changes in value aggregates into their overallcomponents of price and quantity change. A CPI isintended to measure the price component of the changein households’ consumption expenditures. One way todo this is to measure the change in the value of anaggregate, holding the quantities constant.

Lowe indices1.17 One very wide, and popular, class of price

indices is obtained by defining the index as the percent-age change, between the periods compared, in the totalcost of purchasing a given set of quantities, generallydescribed as a ‘‘basket’’. The meaning of such an indexis easy to grasp and to explain to users. This class ofindex is called a Lowe index in this manual, after the

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

2

index number pioneer who first proposed it in 1823 (seeChapter 15). Most statistical offices make use of somekind of Lowe index in practice.1.18 Let there be n products in the basket with prices

pi and quantities qi, and let the two periods compared be0 and t. The Lowe index, PLo, is defined as follows:

PLo �

Pni=1

ptiqiPp0i qi

1.19 In principle, any set of quantities could serve asthe basket. The basket does not have to be restricted tothe quantities purchased in one or other of the two peri-ods compared, or indeed any actual period of time. Thequantities could, for example, be arithmetic or geometricaverages of the quantities in the two periods. For prac-tical reasons, the basket of quantities used for CPI pur-poses usually has to be based on a survey of householdconsumption expenditures conducted in an earlier periodthan either of the two periods whose prices are com-pared. For example, a monthly CPI may run from Jan-uary 2000 onwards, with January 2000=100, but thequantities may be derived from an annual expendituresurvey made in 1997 or 1998, or even spanning boththose years. As it takes a long time to collect and processexpenditure data, there is usually a considerable time lagbefore such data can be introduced into the calculationof CPIs. The basket may also refer to a year, whereas theindex may be compiled monthly or quarterly.1.20 The period whose quantities are actually used

in a CPI is described as the weight reference period and itwill be denoted as period b. Period 0 is the price refer-ence period. As just noted, b is likely to precede 0, atleast when the index is first published, and this isassumed here, but b could be any period, including onebetween 0 and t, if the index is calculated some timeafter t. The Lowe index using the quantities of period bcan be written as follows:

PLo �

Pni=1

ptiqbi

Pni=1

p0i qbi

�Pni=1

( pti=p0i )s

0bi

where s0bi =p0i qbiPn

i=1

p0i qbi

(1.1)

The index can be written, and calculated, in two ways:either as the ratio of two value aggregates, or as anarithmetic weighted average of the price ratios, or pricerelatives, pti=p

0i , for the individual products using the

hybrid expenditure shares s0bi as weights. The expendi-

tures are described as hybrid because the prices andquantities belong to two different time periods, 0 and brespectively. The hybrid weights may be obtained byupdating the actual expenditure shares in period b,namely pbi q

bi =Ppbi q

bi , for the price changes occurring

between periods b and 0 by multiplying them by theprice relatives b and 0, namely p0

i =pbi . Lowe indices are

widely used for CPI purposes.

Laspeyres and Paasche indices1.21 Any set of quantities could be used in a Lowe

index, but there are two special cases which figure veryprominently in the literature and are of considerableimportance from a theoretical point of view. When thequantities are those of the price reference period, that iswhen b=0, the Laspeyres index is obtained. Whenquantities are those of the other period, that is whenb=t, the Paasche index is obtained. It is necessary toconsider the properties of Laspeyres and Paasche indi-ces, and also the relationships between them, in moredetail.

1.22 The Laspeyres price index, PL, is defined as:

PL=

Pni=1

ptiq0i

Pni=1

p0i q

0i

�Pni=1

( pti=p0i )s

0i (1.2)

where s0i denotes the share of the actual expenditure oncommodity i in period 0: that is, p0

i q0i =Pp0i q

0i .

1.23 The Paasche index, PP, is defined as:

PP=

Pni=1

ptiqti

Pni=1

p0i qti

�Pni=1

( pti=p0i )�1sti

� ��1

(1.3)

where sti denotes the actual share of the expenditureon commodity i in period t; that is, ptiq

ti=Pptiq

ti .

Notice that the Paasche index is a weighted harmonicaverage of the price relatives that uses the actualexpenditure shares in the later period t as weights. Itfollows from equation (1.1) that the Paasche indexcan also be expressed as a weighted arithmetic ave-rage of the price relatives using hybrid expenditureweights, in which the quantities of t are valued at theprices of 0.

Decomposing current valuechanges using Laspeyres andPaasche indices

1.24 Laspeyres and Paasche quantity indices aredefined in a similar way to the price indices, simply byinterchanging the p and q values in formulae (1.2) and(1.3). They summarize changes over time in the flow ofquantities of goods and services consumed. A Laspeyresquantity index values the quantities at the fixed pricesof the earlier period, while the Paasche quantity indexuses the prices of the later period. The ratio of the valuesof the expenditures in two periods (V ) reflects thecombined effects of both price and quantity changes.When Laspeyres and Paasche indices are used, the valuechange can be exactly decomposed into a price indextimes a quantity index only if the Laspeyres price(quantity) index is matched with the Paasche quantity(price) index. Let PLa and QLa denote the Laspeyresprice and quantity indices and let PPa and QPa denotethe Paasche price and quantity indices: then, PLaQPa:Vand PPa QLa:V.

AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY

3

1.25 Suppose, for example, a time series of house-hold consumption expenditures at current prices in thenational accounts is to be deflated by a price index toshow changes in real consumption. To generate a seriesof consumption expenditures at constant base periodprices (whose movements are identical with those of theLaspeyres volume index), the consumption expendituresat current prices must be deflated by a series of Paascheprice indices.

Ratios of Lowe and Laspeyres indices1.26 The Lowe index is transitive. The ratio of two

Lowe indices using the same set of qb values is alsoa Lowe index. For example, the ratio of the Lowe indexfor period t+1 with price reference period 0 divided bythat for period t also with price reference period 0 is:

Pni=1

pt+1i qbi

.Pni=1

p0i qbi

Pni=1

ptiqbi

.Pni=1

p0i qbi

=

Pni=1

pt+1i qbi

Pni=1

ptiqbi

=Pt, t+1Lo (1.4)

This is a Lowe index for period t+1 with period t as theprice reference period. This kind of index is, in fact,widely used to measure short-term price movements,such as between t and t+1, even though the quantitiesmay date back to some much earlier period b.

1.27 A Lowe index can also be expressed as the ratioof two Laspeyres indices. For example, the Lowe indexfor period t with price reference period 0 is equal to theLaspeyres index for period t with price reference periodb divided by the Laspeyres index for period 0 also withprice reference period b. Thus,

PLo=

Pni=1

ptiqbi

Pni=1

p0i qbi

=

Pni=1

ptiqbi

.Pni=1

pbi qbi

Pni=1

p0i qbi

.Pni=1

pbi qbi

=PtLaP 0La

(1.5)

Updated Lowe indices1.28 It is useful to have a formula that enables a

Lowe index to be calculated directly as a chain index, inwhich the index for period t+1 is obtained by updatingthe index for period t. Because Lowe indices are tran-sitive, the Lowe index for period t+1 with price refer-ence period 0 can be written as the product of the Loweindex for period t with price reference period 0 multi-plied by the Lowe index for period t+1 with pricereference period t. Thus,

Pni=1

pt+1i qbi

Pni=1

p0i qbi

=

Pni=1

ptiqbi

Pni=1

p0i qbi

2664

3775Pni=1

pt+1i qbi

Pni=1

ptiqbi

2664

3775

=

Pni=1

ptiqbi

Pni=1

p0i qbi

2664

3775 Pn

i=1

pt+1i

pti

� �stbi

�(1.6)

where the expenditure weights stbi are hybrid weightsdefined as:

stbi � ptiqbi.Pni=1

ptiqbi (1.7)

1.29 Hybrid weights of the kind defined in equation(1.7) are often described as price-updated weights. Theycan be obtained by adjusting the original expenditureweights pbi q

bi =Ppbi q

bi by the price relatives pti=p

bi . By

price-updating the expenditure weights from b to t inthis way, the index between t and t+1 can be calculateddirectly as a weighted average of the price relativespt+1i =pti without referring back to the price reference

period 0. The index can then be linked on to the value ofthe index in the preceding period t.

Interrelationships between fixedbasket indices

1.30 Consider first the interrelationship between theLaspeyres and the Paasche indices. A well-known resultin index number theory is that if the price and quantitychanges (weighted by values) are negatively correlated,then the Laspeyres index exceeds the Paasche index.Conversely, if the weighted price and quantity changesare positively correlated, then the Paasche index exceedsthe Laspeyres index. The proof is given in Appendix 15.1of Chapter 15.

1.31 As consumers are usually price-takers, theytypically react to price changes by substituting goods orservices that have become relatively cheaper for thosethat have become relatively dearer. This is known as thesubstitution effect, a phenomenon that figures promi-nently in this manual and the wider literature on indexnumbers. Substitution tends to generate a negativecorrelation between the price and quantity relatives, inwhich case the Laspeyres index is greater than thePaasche index, the gap between them tending to widenover time.

1.32 In practice, however, statistical offices do notcalculate Laspeyres or Paasche indices but insteadusually calculate Lowe indices as defined in equation(1.1). The question then arises of how the Lowe indexrelates to the Laspeyres and Paasche indices. It is shownin the text of Chapter 15, and also in Appendix 15.2, thatif there are persistent long-term trends in relative pricesand if the substitution effect is operative, the Lowe indexwill tend to exceed the Laspeyres, and therefore also theFisher and the Paasche indices. Assuming that period bprecedes period 0, the ranking under these conditionswill be:

Lowe�Laspeyres�Fisher�Paasche

Moreover, the amount by which the Lowe exceeds theother three indices will tend to increase, the further backin time period b is in relation to period 0.

1.33 The positioning of period b is crucial. Given theassumptions about long-term price trends and substitu-tion, a Lowe index will tend to increase as period b ismoved backwards in time, or to decrease as period b ismoved forwards in time. While b may have to precede 0

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

4

when the index is first published, there is no suchrestriction on the positioning of b as price and quantitydata become available for later periods with passage oftime. Period b can then be moved forwards. If b ispositioned midway between 0 and t, the quantities arelikely to be equi-representative of both periods, assumingthat there is a fairly smooth transition from the relativequantities of 0 to those of t. In these circumstances, theLowe index is likely to be close to the Fisher and othersuperlative indices, and cannot be presumed to haveeither an upward or a downward bias. These points areelaborated further below, and also in Chapter 15.1.34 It is important that statistical offices take these

relationships into consideration in deciding upon theirpolicies. There are obviously practical advantages andfinancial savings from continuing to make repeated useover many years of the same fixed set of quantities tocalculate a CPI. However, the amount by which such aCPI exceeds some conceptually preferred target index,such as a cost of living index (COLI), is likely to getsteadily larger the further back in time the period b towhich the quantities refer. Most users are likely tointerpret the difference as upward bias. A large bias mayundermine the credibility and acceptability of the index.

Young index1.35 Instead of holding constant the quantities of

period b, a statistical office may calculate a CPI as aweighted arithmetic average of the individual pricerelatives, holding constant the revenue shares of periodb. The resulting index is called a Young index in thismanual, again after another index number pioneer. TheYoung index is defined as follows:

PYo �Pni=1

sbiptip0i

� �where sbi �

pbi qbiPn

i=1

pbi qbi

(1.8)

In the corresponding Lowe index, equation (1.1), theweights are hybrid revenue shares that value the quan-tities of b at the prices of 0. As already explained, theprice reference period 0 is usually later than the weightreference period b because of the time needed to collectand process and revenue data. In that case, a statisticaloffice has the choice of assuming that either the quan-tities of period b remain constant or the expenditureshares in period b remain constant. Both cannot remainconstant if prices change between b and 0. If theexpenditure shares actually remained constant betweenperiods b and 0, the quantities must have changedinversely in response to the price changes, which impliesan elasticity of substitution of unity.1.36 Whereas there is a presumption that the Lowe

index will tend to exceed the Laspeyres index, it is moredifficult to generalize about the relationship between theYoung index and the Laspeyres index. The Young couldbe greater or less than the Laspeyres depending on howsensitive the quantities are to changes in relative prices.It is shown in Chapter 15 that with high elasticities ofsubstitution (greater than unity) the Young will tend toexceed the Laspeyres, whereas with low elasticities theYoung will tend to be less than the Laspeyres.

1.37 As explained later in this chapter, the Loweindex may be preferred to the Young index because theYoung index has some undesirable properties that causeit to fail some critical index number tests (see alsoChapter 16).

Geometric Young, Laspeyres andPaasche indices

1.38 In the geometric version of the Young index, aweighted geometric average is taken of the price relativesusing the expenditure shares of period b as weights. It isdefined as follows:

PGYo �Yni=1

ptip0i

� �sbi(1.9)

where sbi is defined as above. The geometric Laspeyres isthe special case in which b=0; that is, the expenditureshares are those of the price reference period 0. Similarly,the geometric Paasche uses the expenditure shares ofperiod t. It should be noted that these geometric indicescannot be expressed as the ratios of value aggregates inwhich the quantities are fixed. They are not basketindices and there are no counterpart Lowe indices.

1.39 It is worth recalling that for any set of positivenumbers the arithmetic average is greater than, or equalto, the geometric average, which in turn is greater than,or equal to, the harmonic average, the equalities holdingonly when the numbers are all equal. In the case ofunitary cross-elasticities of demand and constant expen-diture shares, the geometric Laspeyres and Paascheindices coincide. In this case, the ranking of the indicesmust be the ordinary Laspeyres� the geometric Las-peyres and Paasche� the ordinary Paasche, because theindices are, respectively, arithmetic, geometric and har-monic averages of the same price relatives which all usethe same set of weights.

1.40 The geometric Young and Laspeyres indiceshave the same information requirements as their ordin-ary arithmetic counterparts. They can be produced ona timely basis. Thus, these geometric indices must betreated as serious practical possibilities for purposes ofCPI calculations. As explained later, the geometricindices are likely to be less subject than their arithmeticcounterparts to the kinds of index number biases dis-cussed in later sections. Their main disadvantage may bethat, because they are not fixed basket indices, they arenot so easy to explain or justify to users.

Symmetric indices1.41 A symmetric index is one that makes equal use

of the prices and quantities in both the periods com-pared and treats them in a symmetric manner. There arethree particular symmetric indices that are widely usedin economic statistics. It is convenient to introduce themat this point. As already noted, these three indices arealso superlative indices.

1.42 The first is the Fisher price index, PF, defined asthe geometric average of the Laspeyres and Paasche

AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY

5

indices; that is,

PF �ffiffiffiffiffiffiffiffiffiffiffiffiPLPP

p(1.10)

1.43 The second is the Walsh price index, PW. This isa basket index whose quantities consist of geometricaverages of the quantities in the two periods; that is,

PW �

Pni=1

ptiffiffiffiffiffiffiffiffiffiqtiq

0i

pPni=1

p0i

ffiffiffiffiffiffiffiffiffiqtiq

0i

p (1.11)

By taking a geometric rather than an arithmetic averageof the quantities, equal weight is given to the relativequantities in both periods. The quantities in the Walshindex can be regarded as being equi-representative ofboth periods.

1.44 The third index is the Tornqvist price index, PT,defined as a geometric average of the price relativesweighted by the average expenditure shares in the twoperiods.

PT=Yni=1

( pti=p0i )

si (1.12)

where si is the arithmetic average of the share ofexpenditure on product i in the two periods.

si=Sti+S0

i

2(1.13)

where the si values are defined as in equations (1.2) and(1.3) above.

1.45 The theoretical attractions of these indicesbecome more apparent in the following sections on theaxiomatic and economic approaches to index numbers.

Fixed base versus chain indices1.46 This topic is examined in Chapter 15. When a

time series of Lowe or Laspeyres indices is calculatedusing a fixed set of quantities, the quantities becomeprogressively out of date and increasingly irrelevant tothe later periods for which prices are being compared.The base period, in which quantities are set, has to beupdated sooner or later and the new index series linkedto the old. Linking is inevitable in the long run.

1.47 In a chain index, each link consists of an indexin which each period is compared with the preceding one,the weight and price reference periods being movedforward each period. Any index number formula can beused for the individual links in a chain index. Forexample, it is possible to have a chain index in whichthe index for t+1 on t is a Lowe index defined asPpt+1qt�j=

Pptqt�j. The quantities refer to some period

that is j periods earlier than the price reference period t.The quantities move forward one period as the pricereference period moves forward one period. If j=0, thechain Lowe becomes a chain Laspeyres, while if j ¼�1,it becomes a chain Paasche.

1.48 The CPIs in some countries are, in fact, annualchain Lowe indices of this general type, the quantitiesreferring to some year or years that precede the price

reference period 0 by a fixed period. For example, the 12monthly indices from January 2000 to January 2001,with January 2000 as the price reference period, could beLowe indices based on price-updated expenditures for1998. The 12 indices from January 2001 to January 2002are then based on price updated expenditures for 1999,and so on.

1.49 The expenditures lag behind the January pricereference period by a fixed interval, moving forward ayear each January as the price reference period movesforward one year. Although, for practical reasons, therehas to be a time lag between the quantities and the priceswhen the index is first published, it is possible to recal-culate the monthly indices for the current year later,using current expenditure data when they eventuallybecome available. In this way, it is possible for the long-run index to be an annually chained monthly index,with contemporaneous annual weights. This method isexplained in more detail in Chapter 9. It is used by onestatistical office.

1.50 A chain index has to be ‘‘path dependent’’. Itmust depend on the prices and quantities in all theintervening periods between the first and last periods inthe index series. Path dependency can be advantageousor disadvantageous. When there is a gradual economictransition from the first to the last period, with smoothtrends in relative prices and quantities, chaining willtend to reduce the index number spread between theLowe, Laspeyres and Paasche indices, thereby makingthe movements in the index less dependent on the choiceof index number formula.

1.51 If there are fluctuations in the prices andquantities in the intervening periods, however, chainingmay not only increase the index number spread but alsodistort the measure of the overall change between thefirst and last periods. For example, suppose all the pricesin the last period return to their initial levels in period 0,which implies that they must have fluctuated in between;a chain Laspeyres index will not return to 100. It willtend to be greater than 100. If the cycle is repeated withall the prices periodically returning to their originallevels, a chain Laspeyres index will tend to drift furtherand further above 100 even though there may be nolong-term upward trend in the prices. Chaining istherefore not advised when prices fluctuate. Whenmonthly prices are subject to regular and substantialseasonal fluctuations, for example, monthly chainingcannot be recommended. Seasonal fluctuations causeserious problems, which are analysed in Chapter 22.While a number of countries update their expenditureweights annually, the 12-monthly indices within eachyear are not chain indices but Lowe indices using fixedannual quantities.

1.52 The Divisia index. If the prices and quantitiesare continuous functions of time, it is possible to par-tition the change in their total value over time intoprice and quantity components following the method ofDivisia. As shown in Chapter 15, the Divisia index maybe derived mathematically by differentiating value (i.e.price multiplied by quantity) with respect to time toobtain two components: a relative-value-weighted pricechange and a relative-value-weighted quantity change.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

6

These two components are defined to be price andquantity indices, respectively. The Divisia is essentially atheoretical index. In practice, prices can be recordedonly at discrete intervals, even if they vary continuouslywith time. A chain index may, however, be regarded as adiscrete approximation to a Divisia. The Divisia indexitself offers limited practical guidance about the kind ofindex number formula to choose for the individual linksin a chain index.

Axiomatic and stochasticapproaches to index numbers1.53 Various axiomatic approaches to index num-

bers are explained in Chapter 16. These approaches seekto determine the most appropriate functional form foran index by specifying a number of axioms, or tests, thatthe index ought to satisfy. They throw light on theproperties possessed by different kinds of indices, someof which are not intuitively obvious. Indices that fail tosatisfy certain basic or fundamental axioms, or tests,may be rejected completely because they are liable tobehave in unacceptable ways. An axiomatic approachmay also be used to rank indices on the basis of theirdesirable, and undesirable, properties.

First axiomatic approach1.54 The first approach is the traditional test

approach pioneered by Irving Fisher. The price andquantity indices are defined as functions of the twovectors of prices and two vectors of quantities relating tothe two periods compared. The prices and quantities aretreated as independent variables, whereas in the eco-nomic approach to index numbers considered later inthis chapter the quantities are assumed to be functionsof the prices.1.55 Chapter 16 starts by considering a set of 20

axioms, but only a selection of them is given here by wayof illustration.

T1: positivity – the price index and its constituentvectors of prices and quantities should be positive.

T3: identity test – if the price of every product isidentical in both periods, then the price index shouldequal unity, no matter what the quantity vectors are.

T5: proportionality in current prices – if all prices inperiod t are multiplied by the positive number l, thenthe new price index should be l times the old price index;i.e., the price index function is (positively) homogeneousof degree one in the components of the period t pricevector.

T10: invariance to changes in the units of measure-ment (commensurability test) – the price index does notchange if the units in which the products are measuredare changed.

T11: time reversal test – if all the data for the twoperiods are interchanged, then the resulting price indexshould equal the reciprocal of the original price index.

T14: mean value test for prices – the price index liesbetween the highest and the lowest price relatives.

T16: Paasche and Laspeyres bounding test – the priceindex lies between the Laspeyres and Paasche indices.

T17: monotonicity in current prices – if any period tprice is increased, then the price index must increase.

1.56 Some of the axioms or tests can be regarded asmore important than others. Indeed, some of the axiomsseem so inherently reasonable that it might be assumedthat any index number actually in use would satisfythem. For example, test T10, the commensurability test,says that if the unit of quantity in which a product ismeasured is changed, say, from a gallon to a litre, theindex must be unchanged. One index that does notsatisfy this test is the Dutot index, which is defined as theratio of the arithmetic means of the prices in the twoperiods. As explained later, this is a type of elementaryindex that is in fact widely used in the early stages ofCPI calculation.

1.57 Consider, for example, the average price of saltand pepper. Suppose it is decided to change the unit ofmeasurement for pepper from grams to ounces whileleaving the units in which salt is measured (for example,kilos) unchanged. As an ounce is equal to 28.35 grams,the absolute value of the price of pepper increases byover 28 times, which effectively increases the weight ofpepper in the Dutot index by over 28 times.

1.58 When the products covered by an index areheterogeneous and measured in different physical units,the value of any index that does not satisfy the com-mensurability test depends on the purely arbitrary choiceof units. Such an index must be unacceptable con-ceptually. If the prices refer to a strictly homogeneous setof products that all use the same unit of measurement,the test becomes irrelevant.

1.59 Another important test is T11, the time reversaltest. In principle, it seems reasonable to require that thesame result should be obtained whichever of the twoperiods is chosen as the price reference period: in otherwords, whether the change is measured forwards in time,i.e., from 0 to t, or backwards in time from t to 0. TheYoung index fails this test because an arithmetic averageof a set of price relatives is not equal to the reciprocal ofthe arithmetic average of the reciprocals of the pricerelatives. The fact that the conceptually arbitrary deci-sion to measure the change in prices forwards from 0and t gives a different result from measuring backwardsfrom t to 0 is seen by many users as a serious dis-advantage. The failure of the Young index to satisfy thetime reversal test needs to be taken into account bystatistical offices.

1.60 Both the Laspeyres and Paasche fail the timereversal test for the same reasons as the Young index.For example, the formula for a Laspeyres calculatedbackwards from t to 0, PBL, is:

PBL=

Pni=1

p0i qti

Pni=1

ptiqti

� 1

PP(1.14)

This index is identical to the reciprocal of the (for-wards) Paasche, not to the reciprocal of the (forwards)Laspeyres. As already noted, the (forwards) Paaschetends to register a smaller increase than the (forwards)Laspeyres so that the Laspeyres index cannot satisfy the

AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY

7

time reversal test. The Paasche index also fails the timereversal test.

1.61 In contrast, the Lowe index satisfies the timereversal test provided that the quantities qbi remain fixedwhen the price reference period is changed from 0 to t.The quantities of a Laspeyres index are, however, thoseof the price reference period by definition, and mustchange whenever the price reference period is changed.The basket for a forwards Laspeyres is different fromthat for a backwards Laspeyres, and the Laspeyres failsthe time reversal test in consequence.

1.62 Similarly, the Lowe index is transitive whereasthe Laspeyres and Paasche indices are not. Assumingthat a Lowe index uses a fixed set of quantities, qbi ,whatever the price reference period, it follows that

Lo0, t=Lo0, t�kLot�k, t

where Lo0, t is the Lowe index for period t with period 0as the price reference period. The Lowe index thatcompares t directly with 0 is the same as that calculatedindirectly as a chain index through period t�k.

1.63 If, on the other hand, the Lowe index is definedin such a way that quantities vary with the price refer-ence period, as in the index

Ppt+1qt�j=

Pptqt�j con-

sidered earlier, the resulting chain index is not transitive.The chain Laspeyres and chain Paasche indices arespecial cases of this index.

1.64 In the real world, the quantities do change andthe whole point of chaining is to enable the quantities tobe continually updated to take account of the changinguniverse of products. Achieving transitivity by arbitrarilyholding the quantities constant, especially over a verylong period of time, does not compensate for the poten-tial biases introduced by using out-of-date quantities.

Ranking of indices using the firstaxiomatic approach

1.65 In Chapter 16 it is shown not only that theFisher price index satisfies all the 20 axioms listed butalso, more remarkably, that it is the only possible indexthat can satisfy all 20 axioms. Thus, on the basis of thisparticular set of axioms, the Fisher clearly dominatesother indices.

1.66 In contrast to Fisher, the other two symmetric(and superlative) indices defined in equations (1.11) and(1.12) above do not emerge so well from the 20 tests. InChapter 16, it is shown that the Walsh price index failsfour tests while the Tornqvist index fails nine tests.Nevertheless, the Tornqvist and the Fisher may be ex-pected to approximate each other quite closely numeri-cally when the data follow relatively smooth trends, asshown in Chapter 19.

1.67 One limitation of the axiomatic approach isthat the list of axioms is inevitably somewhat arbitrary.Some axioms, such as the Paasche and Laspeyresbounding test failed by both Tornqvist and Walsh, couldbe regarded as dispensable. Additional axioms or testscan be envisaged, and two further axioms are consid-ered below. Another problem with a simple applicationof the axiomatic approach is that it is not sufficient toknow which tests are failed. It is also necessary to know

how badly an index fails. Failing badly one majortest, such as the commensurability test, might be con-sidered sufficient to rule out an index, whereas failingseveral minor tests marginally may not be very dis-advantageous.

Some further tests1.68 Consider a further symmetry test. Reversing the

roles of prices and quantities in a price index yields aquantity index of the same functional form as the priceindex. The factor reversal test requires that the product ofthis quantity index and the original price index should beidentical with the change in the value of the aggregate inquestion. The test is important if, as stated earlier, priceand quantity indices are intended to enable changes inthe values of aggregates over time to be factored intotheir price and quantity components in an economicallymeaningful way. Another interesting result given inChapter 16 is that the Fisher index is the only price indexto satisfy four minimal tests: T1 (positivity), T11 (timereversal test), T12 (quantity reversal test) and T21 (factorreversal test). As the factor reversal test implicitly assumesthat the prices and quantities must refer either to period 0or to period t, it is not relevant to a Lowe index in whichthree periods are involved, b, 0 and t.

1.69 As shown earlier, the product of the Laspeyresprice (quantity) index and the Paasche quantity (price)index is identical with the change in the total value of theaggregate in question. Thus, Laspeyres and Paascheindices may be said to satisfy a weak version of thefactor reversal test in that dividing the value change by aLaspeyres (Paasche) price index does lead to a mean-ingful quantity index, i.e., the Paasche (Laspeyres), eventhough the functional forms of the price and quantityindices are not identical.

1.70 Another test discussed in Chapter 16 is theadditivity test. This is more important from the per-spective of quantity than price indices. Price indices maybe used to deflate value changes to obtain implicitquantity changes. The results may be presented for sub-aggregates such as broad categories of household con-sumption. Just as expenditure aggregates at currentprices are, by definition, obtained simply by summingindividual expenditures, it is reasonable to expect thatthe changes in the sub-aggregates of a quantity indexshould add up to the changes in the totals – the addi-tivity test. Quantity indices such as Laspeyres andPaasche that use a common set of prices to valuequantities in both periods must satisfy the additivitytest. Similarly, the Lowe quantity index defined asPp jqt=

Pp jq0 is also additive. The Geary–Khamis

quantity index (see Annex 4) used to make internationalcomparisons of real consumption and gross domesticproduct (GDP) between countries is an example ofsuch a Lowe quantity index. It uses an arithmeticallyweighted average of the prices in the different countriesas the common price vector p j to compare the quantitiesin different countries.

1.71 Similarly, an average of the prices in two peri-ods can be used to value the quantities in intertemporalindices. If the quantity index is also to satisfy the time

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

8

reversal test, the average must be symmetrical. Theinvariance to proportional changes in current prices test(which corresponds to test T7 listed in Chapter 16, exceptthat the roles of prices and quantities are reversed)requires that a quantity index depend only on the rela-tive, not the absolute, level of the prices in each period.The Walsh quantity index satisfies this test, is additiveand satisfies the time reversal test as well. It emerges as aquantity index with some very desirable properties.1.72 Although the Fisher index itself is not additive,

it is possible to decompose the overall percentage changein a Fisher price, or quantity, index into additive com-ponents that reflect the percentage change in each priceor quantity. A similar multiplicative decomposition ispossible for a Tornqvist price or quantity index.

The stochastic approach and asecond axiomatic approach1.73 Before considering a second axiomatic ap-

proach, it is convenient to take the stochastic approachto price indices. The stochastic approach treats the ob-served price changes or relatives as if they were a randomsample drawn from a defined universe whose mean canbe interpreted as the general rate of inflation. There can,however, be no single unique rate of inflation. Manypossible universes can be defined, depending on whichparticular sets of expenditures or transactions the user isinterested in. Clearly, the sample mean depends on thechoice of universe from which the sample is drawn.Specifying the universe is similar to specifying the scopeof a CPI. The stochastic approach addresses issues suchas the appropriate form of average to take and the mostefficient way to estimate it from a sample of price rela-tives, once the universe has been defined.1.74 The stochastic approach is particularly useful

when the universe is reduced to a single type of product.Because of market imperfections, there may be con-siderable variation in the prices at which the same pro-duct is sold in different outlets and also in the pricechanges observed. In practice, statistical offices have toestimate the average price change for a single productfrom a sample of price observations. Important meth-odological issues are raised, which are discussed in somedetail in Chapter 7 and Chapter 20.

The unweighted stochastic approach1.75 In Chapter 16, the unweighted stochastic

approach to index number theory is explained. If simplerandom sampling has been used, equal weight may begiven to each sampled price relative. Suppose each pricerelative can be treated as the sum of two components: acommon inflation rate and a random disturbance with azero mean. Using least squares or maximum likelihood,the best estimate of the common inflation rate is theunweighted arithmetic mean of price relatives, an indexformula known as the Carli index. This index is theunweighted version of the Young index and is discussedfurther below, in the context of elementary price indices.1.76 If the random component is multiplicative, not

additive, the best estimate of the common inflation rateis given by the unweighted geometric mean of price

relatives, known as the Jevons index. The Jevons indexmay be preferred to the Carli on the grounds that itsatisfies the time reversal test, whereas the Carli doesnot. As explained below, this fact may be decisive whendetermining the functional form to be used to estimatethe elementary indices compiled in the early stages ofCPI calculations.

The weighted stochastic approach1.77 As explained in Chapter 16, a weighted sto-

chastic approach can be applied at an aggregative levelcovering sets of different products. As the products maybe of differing economic importance, equal weightshould not be given to each type of product. The pro-ducts may be weighted on the basis of their share in thetotal value of the expenditures, or other transactions, insome period or periods. In this case, the index (or itslogarithm) is the expected value of a random sample ofprice relatives (or their logarithms) whose probability ofselection is proportional to the expenditure on that typeof product in some period, or periods. Different indicesare obtained depending on which expenditure weightsare used and on whether the price relatives or theirlogarithms are used.

1.78 Suppose a sample of price relatives is randomlyselected with the probability of selection proportional tothe expenditure on that type of product in the pricereference period 0. The expected price change is then theLaspeyres price index for the universe. Other indicesmay, however, also be obtained using the weighted sto-chastic approach. Suppose both periods are treatedsymmetrically and the probabilities of selection are madeproportional to the arithmetic mean expenditure sharesin both periods 0 and t. When these weights are appliedto the logarithms of the price relatives, the expected valueof the logarithms is the Tornqvist index, also known asthe Tornqvist–Theil index. From an axiomatic viewpoint,the choice of a symmetric average of the expenditureshares ensures that the time reversal test is satisfied, whilethe choice of the arithmetic mean, as distinct from someother symmetric average, may be justified on the groundsthat the fundamental proportionality in current pricestest, T5, is thereby satisfied.

1.79 By focusing on price changes, the Tornqvistindex emerges as an index with some very desirable prop-erties. This suggests a second axiomatic approach toindices, in which the focus is shifted from the individualprices and quantities used in the traditional axiomaticapproach, to price changes and values shares.

A second axiomatic approach1.80 A second axiomatic approach is examined in

Chapter 16 in which a price index is defined as a func-tion of the two sets of prices, or their ratios, and two setsof values. Provided the index is invariant to changes inunits of measurement, i.e., satisfies the commensu-rability test, it makes no difference whether individualprices or their ratios are specified. A set of 17 axioms ispostulated which are similar to the 20 axioms consideredin the first axiomatic approach.

AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY

9

1.81 It is shown in Appendix 16.1 that the Tornqvist,or Tornqvist–Theil, is the only price index to satisfy all17 axioms, just as the Fisher price index is the only indexto satisfy all 20 tests in the first approach. However, theTornqvist index does not satisfy the factor reversal test,so that the implicit quantity index obtained by deflatingthe change in value by the Tornqvist price index is notthe Tornqvist quantity index. The implicit quantity indexis therefore not ‘‘best’’ in the sense of satisfying the 17axioms when these are applied to the quantity, ratherthan price, indices.

1.82 Zero prices may cause problems for indicesbased on price ratios, and especially for geometricaverages of price ratios. In particular, if any price tendsto zero, one test that may be applied is that the priceindex ought not to tend to zero or plus infinity. TheTornqvist does not meet this test. It is therefore proposedin Chapter 16 that when using the Tornqvist index, careshould be taken to bound the prices away from zero inorder to avoid a meaningless index number.

1.83 Finally, Chapter 16 examines the axiomaticproperties of the Lowe and Young indices. The Loweindex emerges quite well from the axiomatic approach,satisfying both the time reversal and circularity tests. Onthe other hand, the Young index, like the Laspeyres andPaasche indices, fails both tests. As already explained,however, the attractiveness of the Lowe index dependsmore on how relevant the fixed quantity weights are tothe two periods being compared, that is on the posi-tioning of period b, than its axiomatic properties.

1.84 Although the ‘‘best’’ indices emerging from thetwo axiomatic approaches, namely Fisher and Tornqvist,are not the same, they have much in common. As alreadynoted, they are both symmetric indices and they are bothsuperlative indices. Although their formulae are different,they may be expected to behave in similar ways andregister similar price movements. The same type of indi-ces keep emerging as having desirable properties which-ever approach to index theory is adopted, a conclusionthat is reinforced by the economic approach to indexnumbers, which is explained in Chapter 17.

Cost of living index1.85 Approaching the consumer price index from

the standpoint of economic theory has led to thedevelopment of the concept of a cost of living index(COLI). The theory of the COLI was first developed byKonus (1924). It rests on the assumption of optimizingbehaviour on the part of a rational consumer. The COLIfor such a consumer has been defined succinctly as theratio of the minimum expenditures needed to attain thegiven level of utility, or welfare, under two differentprice regimes. A more precise definition and explanationare given in Chapter 17.

1.86 Whereas a Lowe index measures the change inthe cost of purchasing a fixed basket of goods and ser-vices resulting from changes in their prices, a COLImeasures the change in the minimum cost of maintain-ing a given level of utility, or welfare, that resultsfrom changes in the prices of the goods and servicesconsumed.

1.87 A COLI is liable to possible misinterpretationbecause households’ welfare depends on a variety ofphysical and social factors that have no connection withprices. Events may occur that impinge directly on wel-fare, such as natural or man-made disasters. When suchevents occur, households may need to increase theirconsumption of goods and services in order to com-pensate for the loss of welfare caused by those events.Changes in the costs of consumption triggered by eventsother than changes in prices are irrelevant for a CPI thatis not merely intended to measure changes in the pricesof consumer goods and services but is generally inter-preted by users as measuring price changes, and onlyprice changes. In order to qualify as a CPI, a COLI musttherefore hold constant not only the consumer’s pre-ferences but all the non-price factors that affect theconsumer’s welfare and standard of living. If a CPI isintended to be a COLI it must be conditional on:

– a particular level of utility or welfare;

– a particular set of consumer preferences;

– a particular state of the physical and social environ-ment.

Of course, Lowe indices are also conditional as theydepend on the particular basket of goods and servicesselected.

1.88 Lowe indices and COLIs have in common thefact that they may both be defined as the ratios ofexpenditures in two periods. However, whereas, bydefinition, the quantities are fixed in Lowe indices, theyvary in response to changes in relative prices in COLIs.In contrast to the fixed basket approach to index theory,the economic approach explicitly recognizes that thequantities consumed are actually dependent on the pri-ces. In practice, rational consumers may be expected toadjust the relative quantities they consume in responseto changes in relative prices. A COLI assumes that aconsumer seeking to minimize the cost of maintaininga given level of utility will make the necessary adjust-ments. The baskets of goods and services in thenumerator and denominator of a COLI are not there-fore exactly the same.

1.89 The observed expenditure of a rational con-sumer in the selected base period may be assumed to bethe minimum expenditure needed to achieve the level ofutility enjoyed in that period. In order to calculate aCOLI based on that period, it is necessary to know whatwould be the minimum expenditure needed to attainprecisely the same level of utility if the prices prevailingwere those of the second period, other things remainingequal. The quantities purchased under these assumedconditions are likely to be hypothetical. They will not bethe quantities actually consumed in the second period ifother factors, including the resources available to theconsumer, have changed.

1.90 The quantities required for the calculation ofthe COLI in at least one of the periods are not likelyto be observable in practice. The COLI is not anoperational index that can be calculated directly. Thechallenge is therefore to see whether it is possible to findmethods of estimating a COLI indirectly or at least tofind upper and lower bounds for the index. There is also

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

10

considerable interest in establishing the relationshipbetween a COLI and Lowe indices, including Laspeyresand Paasche, that can be calculated.

Upper and lower bounds on acost of living index1.91 It follows from the definition of a Laspeyres

index that, if the consumer’s income were to change bythe same proportion as the change in the Laspeyresindex, the consumer must have the possibility of pur-chasing the same basket of products as in the baseperiod. The consumer cannot be worse off. However, ifrelative prices have changed, a utility-maximizing con-sumer would not continue to purchase the same quan-tities as before. The consumer would be able to achieve ahigher level of utility by substituting, at least marginally,products that have become relatively cheaper for thosethat have become dearer. As a COLI measures thechange in the minimum expenditures needed to maintaina constant level of utility, the COLI based on the firstperiod will increase by less than the Laspeyres index.1.92 By a similar line of reasoning, it follows that

when relative prices change, the COLI based on thesecond period must increase by more than the Paascheindex. As explained in more detail in Chapter 17, theLaspeyres index provides an upper bound to the COLIbased on the first period and the Paasche a lower boundto the COLI based on the second period. It should benoted that there are two different COLIs involved here:one based on the first period and the other based on thesecond period. In general, however, the two COLIs areunlikely to differ much.1.93 Suppose that the theoretical target index is a

COLI, but that, for practical reasons, the CPI is actuallycalculated as a Lowe index in which the quantities refer tosome period b that precedes the price reference period 0.One important conclusion to be drawn from this pre-liminary analysis is that as the Lowe may be expected toexceed the Laspeyres, assuming long-term price trendsand substitution, while the Laspeyres may in turn beexpected to exceed the COLI, the widely used Lowe indexmay be expected to have an upward bias. This point hashad a profound influence on attitudes towards CPIs insome countries. The bias results from the fact that, bydefinition, fixed basket indices, including Laspeyres, donot permit any substitution between products in responseto changes in relative prices. It is therefore usually de-scribed as ‘‘substitution bias’’. A Paasche index would beexpected to have a downward substitution bias.

Some special cases1.94 The next step is to establish whether there are

special conditions under which it may be possible tomeasure a COLI exactly. In Chapter 17 it is shown that ifthe consumer’s preferences are homothetic – that is, eachindifference curve has the same shape, each being auniform enlargement, or contraction, of each other –then the COLI is independent of the utility level on whichit is based. The Laspeyres and Paasche indices provideupper and lower bounds to the same COLI.

1.95 One interesting special case occurs when thepreferences can be represented by the so-called ‘‘Cobb–Douglas’’ function in which the cross-elasticities ofdemand between the various products are all unity. Con-sumers adjust the relative quantities they consume inver-sely in proportion to the changes in relative prices so thatexpenditure shares remain constant. With Cobb–Douglaspreferences, the geometric Laspeyres provides an exactmeasure of the COLI. As the expenditure shares remainconstant over time, all three geometric indices – Young,Laspeyres and Paasche – coincide with each other andwith the COLI. Of course, the arithmetic versions of theseindices do not coincide in these circumstances, because thebaskets in periods b, 0 and t are all different as substitu-tions take place in response to changes in relative prices.

1.96 One of the more famous results in index num-ber theory is that if the preferences can be representedby a homogeneous quadratic utility function, the Fisherindex provides an exact measure of the COLI (seeChapter 17). Even though consumers’ preferences areunlikely to conform exactly with this particular func-tional form, this result does suggest that, in general, theFisher index is likely to provide a close approximationto the underlying unknown COLI and certainly a muchcloser approximation than either the arithmetic Las-peyres or Paasche indices.

Estimating COLIs by superlative indices1.97 The intuition – that the Fisher index approx-

imates the COLI – is corroborated by the following line ofreasoning. Diewert (1976) noted that a homogeneousquadratic is a flexible functional form that can provide asecond-order approximation to other twice-differentiablefunctions around the same point. He then described anindex number formula as superlative when it is exactlyequal to the COLI based on a certain functional form andwhen that functional form is flexible, e.g., a homogeneousquadratic. The derivation of these results, and furtherexplanation, is given in detail in Chapter 17. In contrast tothe COLI based on the true but unknown utility function,a superlative index is an actual index number that can becalculated. The practical significance of these results isthat they provide a theoretical justification for expecting asuperlative index to provide a fairly close approximationto the underlying COLI in a wide range of circumstances.

1.98 Superlative indices as symmetric indices. TheFisher is by no means the only example of a superlativeindex. In fact, there is a whole family of superlative indices.It is shown in Chapter 17 that any quadratic mean of orderr is a superlative index for each value of r=0. A quadraticmean of order r price index P r is defined as follows:

Pr �

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPni=1

s0ipti

p0i

!r

vuutr=2

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPni=1

stip0i

pti

!r

vuutr=2

(1.15)

where s0i and sti are defined as in equations (1.2) and (1.3)above.

AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY

11

1.99 The symmetry of the numerator and denomi-nator of equation (1.15) should be noted. A distinctivefeature of equation (1.15) is that it treats the pricechanges and expenditure shares in both periods sym-metrically, whatever value is assigned to the parameter r.Three special cases are of interest:

– when r=2, equation (1.1) reduces to the Fisher priceindex;

– when r=1 it is equivalent to the Walsh price index;

– in the limit as r! 0, it equals the Tornqvist index.

These indices were introduced earlier as examples ofindices that treat the information available in both peri-ods symmetrically. Each was originally proposed longbefore the concept of a superlative index was developed.

1.100 Choice of superlative index. Chapter 17 ad-dresses the question of which superlative formula tochoose in practice. As each may be expected to approx-imate to the same underlying COLI, it may be inferredthat they ought also to approximate to each other. Thefact that they are all symmetric indices reinforces thisconclusion. These conjectures tend to be borne out inpractice by numerical calculations. So long as the pa-rameter r does not lie far outside the range 0 to 2,superlative indices tend to be very close to each other. Inprinciple, however, there is no limit on r and it hasrecently been shown that as r becomes larger, the formulatends to assign increasing weight to the extreme pricerelatives and the resulting superlative indices may divergesignificantly from each other. Only when the absolutevalue of r is small, as in the case of the three commonlyused superlative indices (Fisher, Walsh and Tornqvist), isthe choice of superlative index unimportant.

1.101 Both the Fisher and the Walsh indices dateback nearly a century. The Fisher index owes its pop-ularity to the axiomatic, or test, approach, which Fisherhimself was instrumental in developing. As alreadynoted, it dominates other indices using the first axiom-atic approach, while the Tornqvist dominates using thesecond axiomatic approach outlined above. The factthat the Fisher and the Tornqvist are both superla-tive indices whose use can be justified on economicgrounds suggests that, from a theoretical point of view,it may not be possible to improve on them for CPIpurposes.

Representativity bias1.102 The fact that the Walsh index is a Lowe index

that is also superlative suggests that the bias in otherLowe indices depends on the extent to which theirquantities deviate from those in the Walsh basket. Thiscan be viewed from another angle.

1.103 As the quantities in the Walsh basket aregeometric averages of the quantities in the two periods,equal importance is assigned to the relative, as distinctfrom the absolute, quantities in both periods. The Walshbasket may therefore be regarded as being the basketthat is most representative of both periods. If equalimportance is attached to the consumption patterns inthe two periods, the optimal basket for a Lowe indexought to be the most representative basket. The Walsh

index then becomes the conceptually preferred targetindex for a Lowe index.

1.104 Suppose that period b, for which the quan-tities are actually used in the Lowe index, lies midwaybetween 0 and t. In this case, assuming fairly smoothtrends in the relative quantities, the actual basket inperiod b is likely to approximate to the most repre-sentative basket. Conversely, the further away thatperiod b is from the midpoint between 0 and t, the morethe relative quantities of b are likely to diverge fromthose in the most representative basket. In this case, theLowe index between periods 0 and t that uses period bquantities is likely to exceed the Lowe index that usesthe most representative quantities by an amount thatbecomes progressively larger the further back in timeperiod b is positioned. The excess constitutes ‘‘bias’’ ifthe latter index is the target index. The bias can beattributed to the fact that the period b quantities tend tobecome increasingly unrepresentative of a comparisonbetween 0 and t the further back period b is positioned.The underlying economic factors responsible are, ofcourse, exactly the same as those that give rise to biaswhen the target index is the COLI. Thus, certain kindsof indices can be regarded as biased without invokingthe concept of a COLI. Conversely, the same kinds ofindices tend to emerge as being preferred, whether or notthe objective is to estimate a cost of living bias.

1.105 If interest is focused on short-term pricemovements, the target index is an index between con-secutive time periods t and t+1. In this case, the mostrepresentative basket has to move forward one period asthe index moves forward. Choosing the most repre-sentative basket implies chaining. Similarly, chaining isalso implied if the target index is a COLI between t andt+1. In practice, the universe of products is continuallychanging as well. As the most representative basketmoves forward, it is possible to update the set of productscovered, as well as take account of changes in the relativequantities of products that were covered previously.

Data requirements and calculation issues1.106 As superlative indices require price and

expenditure data for both periods, and as expendituredata are usually not available for the current period, it isnot feasible to calculate a superlative CPI, at least at thetime that a CPI is first published. In practice, CPIs tendto be Lowe indices with fixed quantities or annuallyupdated chain Lowe indices. In the course of time,however, the requisite expenditure data may becomeavailable, enabling a superlative CPI to be calculatedsubsequently. Users will find it helpful for superlativeCPIs to be published retrospectively as they make itpossible to evaluate the properties and behaviour of theofficial index. Superlative CPIs can be treated as sup-plementary indices that complement, rather thanreplace, the original indices, if the policy is not to revisethe official index.

1.107 Chapter 17 notes that, in practice, CPIs areusually calculated in stages (see also Chapters 9 and 20)and addresses the question of whether indices calculatedthis way are consistent in aggregation: that is, have the

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

12

same values whether calculated in a single operation orin two stages. The Laspeyres index is shown to beexactly consistent, but the superlative indices are not.The widely used Fisher and Tornqvist indices arenevertheless shown to be approximately consistent.

Allowing for substitution1.108 Chapter 17 examines one further index pro-

posed recently, the Lloyd–Moulton index, PLM, definedas follows:

PLM �Pni=1

s0iptip0i

� �1�s( ) 11�s

s 6¼ 1 (1.16)

The parameter s, which must be non-negative, is theelasticity of substitution between the products covered.It reflects the extent to which, on average, the variousproducts are believed to be substitutes for each other.The advantage of this index is that it may be expected tobe free of substitution bias to a reasonable degree ofapproximation, while requiring no more data than aLowe or Laspeyres index. It is therefore a practicalpossibility for CPI calculation, even for the most recentperiods, although it is likely to be difficult to obtain asatisfactory, acceptable estimate of the numerical valueof the elasticity of substitution, the parameter used inthe formula.

Aggregation issues1.109 It has been assumed up to this point that the

COLI is based on the preferences of a single repre-sentative consumer. Chapter 18 examines the extent towhich the various conclusions reached above remainvalid for CPIs that are actually compiled for groups ofhouseholds. The general conclusion is that essentially thesame relationships hold at an aggregate level, althoughsome additional issues arise which may require addi-tional assumptions.1.110 One issue is how to weight individual house-

holds. Aggregate indices that weight households by theirexpenditures are called ‘‘plutocratic’’, while those thatassign the same weight to each household are called‘‘democratic’’. Another question is whether, at any onepoint of time, there is a single set of prices or whetherdifferent households face different prices. In general,when defining the aggregate indices it is not necessary toassume that all households are confronted by the sameset of prices, although the analysis is naturally simplifiedif there is only a single set.1.111 A plutocratic aggregate COLI assumes that

each individual household minimizes the cost of attain-ing a given level of utility when confronted by two dif-ferent sets of prices, the aggregate COLI being defined asthe ratio of the aggregate minimum costs over all house-holds. As in the case of a single household, it is recog-nized that the aggregate COLI that is appropriate forCPI purposes must be conditional on the state of a par-ticular set of environmental variables, typically those inone or other of the periods compared. The environmentmust be understood in a broad sense to refer not only to

the physical environment but also to the social and po-litical environment.

1.112 Like the index for a single representative con-sumer, an aggregate COLI cannot be calculated directly,but it may be possible to calculate aggregate Laspeyresand Paasche indices that bound their respective COLIsfrom above or below. If there is only one single set of na-tional prices, the aggregate plutocratic Laspeyres indexreduces to an ordinary aggregate Laspeyres index. As theaggregate plutocratic Laspeyres and Paasche can, inprinciple, be calculated, so can the aggregate plutocraticFisher index. It is argued in Chapter 18 that this shouldnormally provide a good approximation to the aggregateplutocratic COLI.

1.113 Chapter 18 finally concludes that, in principle,both democratic and plutocratic Laspeyres, Paasche andFisher indices could be constructed by a statisticalagency, provided that information on household-specificprice relatives and expenditures is available for bothperiods. If expenditure information is available only forthe first period, then only the Laspeyres democratic andplutocratic indices can be constructed. The data require-ments are rather formidable, however. The required dataare unlikely to be available for individual households inpractice and, even if they were to be, they could be sub-ject to large errors.

Illustrative numerical data1.114 Chapter 19 presents some numerical examples

using an artificial data set. The purpose is not to illus-trate the methods of calculation as such, but rather todemonstrate how different index number formulae canyield very different numerical results. Hypothetical buteconomically plausible prices, quantities and expendi-tures are given for six commodities over five periods oftime. In general, differences between the different for-mulae tend to increase with the variance of the pricerelatives. They also depend on the extent to which theprices follow smooth trends or fluctuate.

1.115 The numerical results are striking. For exam-ple, the Laspeyres index over the five periods registers anincrease of 44 per cent while the Paasche falls by 20 percent. The two commonly used superlative indices,Tornqvist and Fisher, register increases of 25 per centand 19 per cent respectively, an index number spread ofonly 6 points compared with the 64-point gap betweenthe Laspeyres and Paasche. When the indices arechained, the chain Laspeyres and Paasche indices reg-ister increases of 33 per cent and 12 per cent respectively,reducing the gap between the two indices from 64 to 21points. The chained Tornqvist and Fisher indices regis-ter increases of 22.26 per cent and 22.24 per centrespectively, being virtually identical numerically.These results show that the choice of index formula andmethod does matter.

Seasonal products1.116 As explained in Chapter 22, the existence of

seasonal products poses some intractable problems and

AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY

13

serious challenges for CPI compilers and users. Seasonalproducts are products that are either:

– not available during certain seasons of the year; or

– are available throughout, but their prices or quantitiesare subject to regular fluctuations that are synchro-nized with the season or time of the year.

There are two main sources of seasonal fluctuations: theclimate and custom. Month-to-month movements in aCPI may sometimes be so dominated by seasonal influ-ences that it is difficult to discern the underlying trends inprices. Conventional seasonal adjustment programmesmay be applied, but these may not always be satisfactory.The problem is not confined to interpreting movementsin the CPI, as seasonality creates serious problems for thecompilation of a CPI when some of the products in thebasket regularly disappear and reappear, thereby break-ing the continuity of the price series from which the CPIis built up. There is no panacea for seasonality. A con-sensus on what is best practice in this area has not yetbeen formed. Chapter 22 examines a number of differentways in which the problems may be tackled using anartificial data set to illustrate the consequences of usingdifferent methods.

1.117 One possibility is to exclude seasonal productsfrom the index, but this may be an unacceptable reduc-tion in the scope of the index, as seasonal products canaccount for a significant proportion of total householdconsumption. Assuming seasonal products are retained,one solution is to switch the focus from month-to-monthmovements in the index to changes between the samemonth in successive years. In some countries, it is com-mon for the media and other users, such as central banks,to focus on the annual rate of inflation between the mostrecent month and the same month in the previous year.This year-on-year figure is much easier to interpret thanmonth-to-month changes, which can be somewhatvolatile, even in the absence of seasonal fluctuations.

1.118 This approach is extended in Chapter 22 tothe concept of a rolling year-on-year index that com-pares the prices for the most recent 12 months with thecorresponding months in the price reference year. Theresulting rolling year indices can be regarded as sea-sonally adjusted price indices. They are shown to workwell using the artificial data set. Such an index can beregarded as a measure of inflation for a year that iscentred around a month that is six months earlierthan the last month in the rolling index. For somepurposes, this time lag may be disadvantageous, but inChapter 22 it is shown that under certain conditionsthe current month year-on-year monthly index, to-gether with the previous month’s year-on-year monthlyindex, can successfully predict the rolling year indexthat is centred on the current month. Of course, rollingyear indices and similar analytic constructs are notintended to replace the monthly or quarterly CPI butto provide supplementary information that can be ex-tremely useful to users. They can be published alongsidethe official CPI.

1.119 Various methods of dealing with the breaks inprice series caused by the disappearance and reappear-ance of seasonal products are examined in Chapter 22.

However, this remains an area in which more researchneeds to be done.

Elementary price indices1.120 As explained in Chapters 9 and 20, the calcu-

lation of a CPI proceeds in stages. In the first stage, ele-mentary price indices are estimated for the elementaryexpenditure aggregates of a CPI. In the second stage, theseelementary indices are aggregated, or averaged, to obtainhigher-level indices using the elementary expenditureaggregates as weights. An elementary aggregate consistsof the expenditures on a small and relatively homoge-neous set of products defined within the consumptionclassification used in the CPI. As explained in Chapter 6,statistical offices usually select a set of representative pro-ducts within each aggregate and then collect samples oftheir prices from a number of different outlets. The ele-mentary aggregates serve as strata for sampling purposes.

1.121 The prices collected at the first stage are typi-cally not prices observed in actual transactions betweendifferent economic units, but the prices at which theproducts are offered for sale in retail outlets of onekind or another. In principle, however, a CPI measureschanges in the prices paid by households. These pricesmay actually vary during the course of a month, which istypically the time period to which the CPI relates. Inprinciple, therefore, the first step should be to average theprices at which some product is sold during the period,bearing in mind that the price may vary even for the sameproduct sold in the same outlet. In general, this is not apractical possibility. However, when the outlet is anelectronic point of sale at which all the individual pro-ducts are ‘‘scanned’’ as they are sold, the values of thetransactions are actually recorded, thereby making itpossible to calculate an average price instead of simplyrecording the offer price at a single point of time. Someuse of scanner data is already made for CPI purposes andit may be expected to increase over the course of time.

1.122 Once the prices are collected for the repre-sentative products in a sample of outlets, the questionarises of what is the most appropriate formula to use toestimate an elementary price index. This topic is con-sidered in Chapter 20. It was comparatively neglecteduntil a number of papers in the 1990s provided muchclearer insights into the properties of elementary in-dices and their relative strengths and weaknesses. Thequality of a CPI depends heavily on the quality of theelementary indices which are the building blocks fromwhich CPIs are constructed.

1.123 Prices are collected for the same product in thesame outlet over a succession of time periods. An ele-mentary price index is therefore typically calculatedfrom two sets of matched price observations. Here it isassumed that there are no missing observations and nochanges in the quality of the products sampled so thatthe two sets of prices are perfectly matched. The treat-ment of new and disappearing products, and of qualitychange, is a separate and complex issue in its own right.It is outlined below, and discussed in detail in Chapters7, 8 and 21.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

14

Weights within elementary aggregates1.124 In most cases, the price indices for elementary

aggregates are calculated without the use of explicit ex-penditure weights. Whenever possible, however, weightsshould be used that reflect the relative importance of thesampled items, even if the weights are only approximate.In many cases, the elementary aggregate is simply thelowest level at which any reliable weighting informationis available. In this case, the elementary index has tobe calculated without the use of weights. Even in thiscase, however, it should be noted that when the itemsare selected with probabilities proportional to the sizeof some relevant variable such as sales, for example,weights are implicitly introduced by the sampling selec-tion procedure.1.125 For certain elementary aggregates, informa-

tion about sales of particular items, market shares andregional weights may be used as explicit weights withinan elementary aggregate. Weights within elementaryaggregates may be updated independently, and possiblymore often than the elementary aggregates themselves(which serve as weights for the higher-level indices).1.126 For example, assume that the number of

suppliers of a certain product, such as petrol, is limited.The market shares of the suppliers may be known frombusiness survey statistics and can be used as weights inthe calculation of an elementary aggregate price indexfor petrol. As another example, prices for water may becollected from a number of local water supply serviceswhere the population in each local region is known. Therelative size of the population in each region may thenbe used as a proxy for the relative consumption expen-ditures to weight the price in each region to obtain theelementary aggregate price index for water.

Interrelationships between differentelementary index formulae1.127 Useful insights into the properties of various

formulae that have been used, or considered, for ele-mentary price indices may be gained by examining themathematical interrelationships between them. Chapter20 provides a detailed analysis of such relationships.As it is assumed that there are no explicit weights avail-able, the various formulae considered all make use ofunweighted averages: that is, simple averages in whichthe various items are equally weighted. There are twobasic options for an elementary index:

– some kind of simple average of the price ratios orrelatives;

– the ratio of some kind of simple average of the pricesin the two periods.

In the case of a geometric average, the two methodscoincide, as the geometric average of the price ratios orrelatives is identical to the ratio of the geometric averageprices.1.128 Using the first of the above options, three

possible elementary price indices are:

– a simple arithmetic average of the price relatives,known as the Carli index, or PC; the Carli is theunweighted version of the Young index;

– a simple geometric average of the price relatives,known as the Jevons index, or PJ; the Jevons is theunweighted version of the geometric Young index;

– a simple harmonic average of the price relatives, orPH.

As noted earlier, for any set of positive numbers thearithmetic average is greater than, or equal to, thegeometric average, which in turn is greater than, orequal to, the harmonic average, the equalities holdingonly when the numbers are all equal. It follows thatPC�PJ�PH.

1.129 It is shown in Chapter 20 that the gapsbetween the three indices widen as the variance of theprice relatives increases. The choice of formula becomesmore important the greater the diversity of the pricemovements. PJ can be expected to lie approximatelyhalfway between PC and PH.

1.130 Using the second of the options, three possibleindices are:

– the ratio of the simple arithmetic average prices,known as the Dutot index, or PD;

– the ratio of the simple geometric averages, again theJevons index, or PJ;

– the ratio of the simple harmonic averages, or PH.

The ranking of ratios of different kinds of average arenot predictable. For example, the Dutot, PD, could begreater or less than the Jevons, PJ.

1.131 The Dutot can also be expressed as a weightedaverage of the price relatives in which the prices ofperiod 0 serve as the weights:

PD �

Pni=1

pti

.n

Pni=1

p0i

.n

=

Pni=1

p0i

ptip0i

� �Pni=1

p0i

(1.17)

As compared with the Carli, which is a simple averageof the price relatives, the Dutot gives more weight to theprice relatives for the products with high prices in per-iod 0. It is nevertheless difficult to provide an economicrationale for this kind of weighting. Prices are notexpenditures. If the products are homogeneous, veryfew quantities are likely to be purchased at high prices ifthe same products can be purchased at low prices. If theproducts are heterogeneous, the Dutot should not beused anyway, as the quantities are not commensurateand not additive.

1.132 While it is useful to establish the inter-relationships between the various indices, they do notactually help decide which index to choose. However, asthe differences between the various formulae tend toincrease with the dispersion of the price relatives, it isclearly desirable to define the elementary aggregates insuch a way as to try to minimize the variation in the pricemovements within each aggregate. The less variationthere is, the less difference the choice of index formulamakes. As the elementary aggregates also serve as stratafor sampling purposes, minimizing the variance in theprice relatives within the strata will also reduce thesampling error.

AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY

15

Axiomatic approach toelementary indices

1.133 One way to decide between the various ele-mentary indices is to exploit the axiomatic approachoutlined earlier. A number of tests are applied to theelementary indices in Chapter 20.

1.134 The Jevons index, PJ, satisfies all the selectedtests. It dominates the other indices in the way that theFisher tends to dominate other indices at an aggregativelevel. The Dutot index, PD, fails only one, the com-mensurability test. This failure is critical, however. Itreflects the fundamental point made earlier that whenthe quantities are not additive from an economic view-point, the prices are also not additive and hence cannotbe meaningfully averaged. However, PD performs wellwhen the sampled products are homogeneous. The keyissue for the Dutot is therefore how heterogeneous arethe products within the elementary aggregate. If theproducts are not sufficiently homogeneous for theirquantities to be additive, the Dutot should not be used.

1.135 Although the Carli index, PC, has been widelyused in practice, the axiomatic approach shows it tohave some undesirable properties. In particular, as theunweighted version of the Young index, it fails the timereversal and transitivity tests. This is a serious disadvan-tage, especially as elementary indices are often monthlychain indices. A consensus has emerged that the Carlimay be unsuitable because it is liable to have a sig-nificant upward bias. This is illustrated by numericalexample in Chapter 9. Its use is not sanctioned for theHarmonized Indices of Consumer Prices used within theEuropean Union. Conversely, the harmonic average ofthe price relatives, PH, is liable to have an equally sig-nificant downward bias; anyway, it does not seem to beused in practice.

1.136 Based on the axiomatic approach, the Jevonsemerges as the preferred index, but its use may not beappropriate in all circumstances. If one observation iszero, the geometric mean is zero. The Jevons is sensitiveto extreme falls in prices; it may be necessary to imposeupper and lower bounds on the individual price relativeswhen using the Jevons.

Economic approach toelementary indices

1.137 The economic approach to elementary indicesis explained in Chapter 20. The sampled products forwhich prices are collected are treated as if they con-stituted a basket of goods and services purchased byrational utility-maximizing consumers. The objective isthen to estimate a conditional cost of living index cov-ering the set of products in question.

1.138 It should be noted, however, that the differ-ences between the prices of the sampled products do notnecessarily mean that the products are qualitativelydifferent. If markets were perfect, relative prices shouldreflect relative costs of production and relative utilities.In fact, price differences may occur simply because ofmarket imperfections. For example, exactly the sameproducts may be bought and sold at different prices indifferent outlets simply because consumers lack infor-

mation about the prices charged in other outlets. Pro-ducers may also practise price discrimination, chargingdifferent customers different prices for exactly the sameproducts. Price discrimination is widespread in manyservice industries. When the price differences are a resultof market imperfections, consumers cannot be expectedto react to changes in the relative prices of products inthe same way as they would if they were well informedand had free choice.

1.139 In any case, assuming there is no informationabout quantities or expenditures within an elementaryaggregate, it is not possible to calculate any kind ofsuperlative index. So the conditional cost of living index atthe level of an elementary aggregate can be estimated onlyon the assumption that certain special conditions apply.

1.140 There are two special cases of some interest.The first case is where the underlying preferences are so-called Leontief preferences. With these preferencesrelative quantities remain fixed whatever the relativeprices. No substitutions are made in response to changesin relative prices. The cross-elasticities of demand arezero. With Leontief preferences, a Laspeyres index pro-vides an exact measure of the cost of living index. In thiscase, the Carli calculated for a random sample wouldprovide an estimate of the cost of living index providedthat the items were selected with probabilities propor-tional to the population expenditure shares. It mightappear that if the items were selected with probabilitiesproportional to the population quantity shares, thesample Dutot would provide an estimate of the popu-lation Laspeyres. However, assuming that the basket forthe Laspeyres index contains a number of heterogeneousproducts whose quantities are not additive, the quantityshares, and hence the probabilities, are undefined.

1.141 The second case is one already consideredabove, namely when the preferences can be representedby a Cobb–Douglas function. As already explained, withthese preferences, the geometric Laspeyres would pro-vide an exact measure of the cost of living index. In thiscase, the Carli calculated for a random sample wouldprovide an unbiased estimate of the cost of living index,provided that the items were selected with probabilitiesproportional to the population expenditure shares.

1.142 On the economic approach, the choice betweenthe sample Jevons and the sample Carli rests on which islikely to approximate the more closely to the underlyingCOLI: in other words, on whether the demand cross-elasticities are likely to be closer to unity or zero, onaverage. In practice, the cross-elasticities could take onany value ranging up to plus infinity for an elementaryaggregate in which the sampled products were strictlyhomogeneous, i.e., perfect substitutes. It should be notedthat in the limiting case in which the sampled productsare homogeneous, there is only a single kind of productand therefore no index number problem: the price indexis given by the ratio of the unit values in the two periods.It may be conjectured that, on average, the cross-elasti-cities are likely to be closer to unity than zero for mostelementary aggregates so that, in general, the Jevonsindex is likely to provide a closer approximation to thecost of living index than the Carli. In this case, the Carlimust be viewed as having an upward bias.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

16

1.143 It is worth noting that the use of the Jevonsindex does not imply, or assume, that expenditure sharesremain constant. Obviously, a geometric average of theprice relatives can be calculated whatever changes do ordo not occur in the expenditure shares, in practice. Whatthe economic approach shows is that if the expenditureshares remain constant (or roughly constant), then theJevons can be expected to provide a good estimate of theunderlying cost of living index. The insight provided bythe economic approach is that the Jevons is likely toprovide a closer approximation to the cost of livingindex than the Carli because a significant amount ofsubstitution is more likely than no substitution, espe-cially as elementary aggregates should be deliberatelyconstructed in such a way as to group together similaritems that are close substitutes for each other.1.144 An alternative to the Jevons, PJ, would be a

geometric average of PC and PH, an index labelledPCSWD in Chapter 20. This could be justified on groundsof treating the data in both periods symmetricallywithout invoking any particular assumption about theform of the underlying preferences. It is also shown inChapter 20 that the geometric average of PC and PH islikely to be very close to PJ, so that the latter may bepreferred on the grounds that it is a simpler concept andeasier to compile.1.145 It may be concluded that, based on the eco-

nomic approach, as well as the axiomatic approach, theJevons emerges as the preferred index in general,although there may be cases in which little or no sub-stitution takes place within the elementary aggregateand the Carli might be preferred. The index compilermust make a judgement on the basis of the nature of theproducts actually included in the elementary aggregate.1.146 The above discussion has also thrown light on

some of the sampling properties of the elementaryindices. If the products in the sample are selected withprobabilities proportional to expenditures in the pricereference period:

– the sample (unweighted) Carli index provides anunbiased estimate of the population Laspeyres;

– the sample (unweighted) Jevons index provides anunbiased estimate of the population geometric Las-peyres.

These results hold irrespective of what the underlyingcost of living index may be.

Concepts, scope andclassifications1.147 The purpose of Chapter 3 of the manual is to

define and clarify a number of basic concepts underlyinga CPI and to explain the scope of the index: that is, theset of goods and services and the set of households thatthe index is intended to cover, in principle. Chapter 3also examines the structure of the classification of con-sumer goods and services used.1.148 While the general purpose of a CPI is to

measure changes in the prices of consumption goods andservices, there are a number of concepts that need to bedefined precisely before an operational definition of a

CPI can be arrived at. The concept of consumption is animprecise one that can be interpreted in several differentways, each of which may lead to a different CPI. It isalso necessary to decide whether the index is meant tocover all consumers, i.e., all households, or just a par-ticular group of households. The scope of a CPI isinevitably influenced by what is intended, or believed, tobe the main use of the index. Compilers also need toremember that the index may be used as proxy for ageneral price index and used for purposes other thanthose for which it is intended.

1.149 The word ‘‘consumer’’ can be used to referboth to a type of economic unit and to a type of pro-duct. To avoid confusion here, the term consumptiongood or service will be used where necessary, rather thanconsumer good or service. A consumption good or ser-vice provides utility to its user. It may be defined as agood or service that members of households use, directlyor indirectly, to satisfy their own personal needs andwants. ‘‘Utility’’ should be interpreted in a broad sense.It is simply the generic, technical term preferred byeconomists for the benefit or welfare that individuals orhouseholds derive from the use of a consumer good orservice.

1.150 A CPI is generally understood to be a priceindex that measures changes in the prices of consump-tion goods and services acquired and used by house-holds. In principle, more broadly based price indices canbe defined whose scope extends beyond consumptiongoods and services to include the prices of physical assetssuch as land or dwellings. Such indices may be useful asbroad measures of inflation as perceived by households,but most CPIs are confined to consumption goodsand services. These may include the prices of the flowsof services provided by assets such as dwellings, eventhough the assets themselves may be excluded. In anycase, the prices of financial assets such as bonds, sharesor other marketable securities purchased by house-holds are generally regarded as being outside the scope ofa CPI.

Acquisitions and uses1.151 The times at which households acquire and

use consumption goods or services are generally not thesame. Goods are typically acquired at one point in timeand used at some other point in time, or even usedrepeatedly over an extended period of time. The time ofacquisition of a good is the moment at which the legal oreffective economic ownership of the good passes to theconsumer. In a market situation, this is the point atwhich the purchaser incurs a liability to pay. A service isacquired at the time that the producer provides it, nochange of ownership being involved. The time at whichacquisitions are recorded, and their prices, should alsobe consistent with the way in which the same transac-tions are recorded in the expenditure data used forweighting purposes.

1.152 The time at which payment is made may bedetermined mainly by institutional arrangements andadministrative convenience. When payments are notmade in cash, there may be a significant lapse of timebefore the consumer’s bank account is debited for a

AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY

17

purchase paid for by cheque, by credit card or similararrangements. The time at which these debits are even-tually made is irrelevant for the recording of the acqui-sitions and the prices. On the other hand, when theacquisition of a good or service is financed by the cre-ation of a new financial asset at the time of acquisition,such as a loan to the purchaser, two economicallyseparate transactions are involved, the purchase/sale ofthe good or service and the creation of the asset. Theprice to be recorded is the price payable at the time ofacquisition, however the purchase is financed. Of course,the provision of finance may affect the price payable. Thesubsequent repayments of any debt incurred by the pur-chaser and the associated interest payments are financialtransactions that are quite distinct from the purchase ofthe good or service whose price has to be recorded. Theexplicit or implicit interest payments payable on theamount depend on the capital market, the nature of theloan, its duration, the creditworthiness of the purchaser,and so on. These points are explained in more detail inChapter 3.

1.153 The distinction between the acquisition andthe use of a consumer good or service outlined above hasled to two different concepts of a CPI being proposed:

� A CPI may be intended to measure the averagechange between two time periods in the prices of theconsumer goods and services acquired by households.

� Alternatively, a CPI may be intended to measure theaverage change between two time periods in the pricesof the consumer goods and services used by house-holds to satisfy their needs and wants.

The distinction between time of acquisition and time ofuse is particularly important for durable goods andcertain kinds of services.

1.154 Durable and non-durable goods. A ‘‘non-durable’’ good might be better described as a single usegood. For example, food or drink are used once only tosatisfy hunger or thirst. Many so-called non-durableconsumer goods are in fact extremely durable physically.Households may hold substantial stocks of non-durables, such as many foodstuffs and fuel, for longperiods of time before they are used.

1.155 The distinguishing feature of a durable con-sumption good is that it is durable under use. Consumerdurables can be used repeatedly or continuously tosatisfy the needs or wants of consumers over a longperiod of time, possibly many years: for example, fur-niture or vehicles. For this reason, a durable is oftendescribed as providing a flow of services to the consumerover the period it is used (see also Box 14.3 of Chapter14). There is a close parallel between the definitions ofconsumer durables and fixed assets. Fixed assets aredefined in national accounts as goods that are usedrepeatedly or continuously over long periods of time inprocesses of production: for example, buildings or otherstructures, machinery and equipment.

1.156 A list of the different kinds of consumer dura-bles distinguished in the Classification of IndividualConsumption according to Purpose (COICOP) is givenin Chapter 3. Of course, some durables last much longerthan others, the less durable ones being described as

‘‘semi-durables’’ in COICOP: for example, clothing. Itshould be noted that dwellings are classified as fixedassets, not durable consumption goods, and are there-fore not included in COICOP. Dwellings are used toproduce housing services. These services are consumedby tenants or owner-occupiers, as the case may be, andare therefore included in COICOP.

1.157 Many services are durable and are also notfully consumed, or used up, at the time they are acquired.Some services bring about long-lasting improvementsfrom which the consumers derive enduring benefits. Thecondition and quality of life of persons receiving med-ical treatments such as hip replacements or cataractsurgery, for example, are substantially and permanentlyimproved. Similarly, consumers of educational servicescan derive lifetime benefits from them. Expenditures oneducation and health also share with durable goods thecharacteristic that they are also often so costly that theyhave to be financed by borrowing or by running downother assets.

1.158 Expenditures on durable goods and durableservices are liable to fluctuate, whereas using up suchgoods and services is likely to be a fairly steady process.However, the using up cannot be directly observed andvalued. It can only be estimated by making assumptionsabout the timing and duration of the flows of benefits.Partly because of the conceptual and practical difficul-ties involved in measuring uses, statistical offices tend toadopt the acquisitions approach to consumer durablesin both their national accounts and CPIs.

1.159 A consumer price index based on the acquisi-tions approach. Households may acquire goods andservices for purposes of consumption in four main ways.They may:

– purchase them in monetary transactions;

– produce them themselves for their own consumption;

– receive them as payments in kind in barter transac-tions, particularly as remuneration in kind for workdone;

– receive them as free gifts, or transfers, from othereconomic units.

1.160 The broadest possible scope for goods andservices based on the acquisitions approach would beone covering all four categories, irrespective of who bearsthe costs. It would therefore include all social transfers inkind in the form of education, health, housing and othergoods and services provided free of charge, or at nomi-nal prices, to individual households by governments ornon-profit institutions (NPIs). Total acquisitions areequivalent to the total actual individual consumption of(non-institutional) households, as defined in the SNA(see Chapter 14). Collective services provided by govern-ments to the community as whole, such as public admin-istration and defence, are not included and are outsidethe scope of a CPI.

1.161 From the point of view of the government orNPI that provides and pays for them, social transfers arevalued either by the market prices paid for them or by thecosts of producing them. From the point of view of thereceiving households they have zero or nominal prices.For CPI purposes, the appropriate price is that paid

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

18

by the household. The price paid by the governmentbelongs in a price index for government expenditures.When households incur zero expenditures, the servicesprovided free carry zero weight in a CPI. However, whengovernments and NPIs introduce charges for goods orservices that were previously provided free, the increasefrom a zero to a positive price could be captured by aCPI, as explained in Chapter 3.1.162 Expenditures versus acquisitions. Expenditures

need to be distinguished from acquisitions. Expendituresare incurred by the economic units that bear the costs.Households do not incur expenditures on social trans-fers in kind, so the scope of households’ expenditures isgenerally narrower than the scope of their acquisitions.Moreover, not all expenditures are monetary. A mone-tary expenditure occurs when a household pays in cash,by cheque or credit card, or otherwise incurs a financialliability to pay. Only monetary expenditures generatemonetary prices that can be observed and recorded forCPI purposes.1.163 Non-monetary expenditures occur when house-

holds pay, but in other ways than cash. There are threeimportant categories of non-monetary expenditures:

� In barter transactions, households exchange con-sumption goods and services among themselves. Asthe values of the goods and services surrendered aspayments constitute negative expenditures, the expen-ditures should cancel out so that barter transactionsbetween households carry zero weight on aggregate.They can be ignored in practice for CPI purposes.

� When employees are remunerated in kind, they pur-chase the goods or services, but pay with their labour,not cash. Monetary values can be imputed for theexpenditures implicitly incurred by the households.

� Similarly, when households produce goods and ser-vices for themselves, they incur the costs, some ofwhich may be monetary in the form of purchasedinputs. The monetary values of the implicit expendi-tures on the outputs produced can be imputed on thebasis of the corresponding market prices. If suchimputed prices were to be included in the CPI, theprices of the inputs would have to be excluded toavoid double counting.

1.164 A hierarchy of consumption aggregates. Ahierarchy of possible consumption aggregates may beenvisaged, as explained in Chapter 14:

– total acquisitions of goods and services by house-holds;

– less social transfers in kind=households’ totalexpenditures;

– less non-monetary expenditures=households’ mone-tary expenditures.

The choice of consumption aggregate is a policy matter.For example, if the main reason for compiling a CPI isto measure inflation, the scope of the index might berestricted to household monetary expenditures on con-sumption, inflation being essentially a monetary phe-nomenon. Prices cannot be collected for the consumergoods and services involved in non-monetary expendi-tures, although they can be estimated on the basis of the

prices observed in corresponding monetary transactions.The European Union’s Harmonized Indices of Con-sumer Prices, which are specifically intended to measureinflation within the EU, are confined to monetaryexpenditures.

Unconditional and conditional cost ofliving indices

1.165 Cost of living indices, or COLIs, are explainedin Chapters 15 and 17. As also noted in Chapter 3, thescope of a COLI depends on whether it is conditional orunconditional. The welfare of a household depends notonly on the utility derived from the goods and services itconsumes, but on the social, political and physicalenvironment in which the household lives. An uncondi-tional cost of living index measures the change in theminimum cost of maintaining a given level of welfare inresponse to changes in any of the factors that affectwelfare, whereas a conditional cost of living index mea-sures the change in the minimum cost of maintaining agiven level of utility or welfare resulting from changes inconsumer prices, holding the environmental factorsconstant.

1.166 An unconditional COLI may be a morecomprehensive cost of living index than a conditionalCOLI, but it is not a more comprehensive price index.An unconditional index does not include any more priceinformation than a conditional index and it does notgive more insight into the impact of price changes onwelfare. On the contrary, the impact of the price changesis diluted and obscured the more environmental vari-ables are included within the scope of an unconditionalindex. In order to qualify as a price index, a COLI mustbe conditional.

Specific types of transactions1.167 Given that conceptually, a CPI is an index

that measures changes in the prices of consumptiongoods and services, expenditures on items that are notconsumption goods and services fall outside the scope ofthe CPI; for example, expenditures on assets such asland or bonds, shares and other financial assets. Simi-larly, payments that do not involve any flows of goodsor services in return for the payments are outside thescope; for example, payments of income taxes or socialsecurity contributions.

1.168 Transfers. A transfer occurs when one eco-nomic unit provides a good, service or asset, includingmoney, to another without receiving any counterpartgood, service or asset in return. As no good or serviceis acquired when a household makes a transfer, thetransfer must be outside the scope. For this reason,compulsory cash transfers, such as payments of directtaxes on income or wealth, must be outside the scope ofa CPI. It is not always clear, however, whether certainpayments to government are transfers or purchases ofservices. For example, payments to obtain certain kindsof licences are sometimes taxes under another name,whereas in other cases the government may provide aservice by exercising some kind of supervisory, reg-ulatory or control function. Gifts or donations must be

AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY

19

transfers and therefore outside the scope. On the otherhand, subscriptions to clubs and societies which providetheir members with some kind of service in return areincluded. Tips and gratuities can be borderline cases.When they are effectively an expected, even obligatory,part of the payment for a service they are not transfersand should be treated as part of the price paid.

1.169 Undesirable or illegal goods or services. Allgoods and services that households willingly buy on themarket to satisfy their own needs and wants should beincluded, even if most people might regard them asundesirable or even if they are prohibited by law. Ofcourse, illegal goods and services may have to be ex-cluded in practice because the requisite data cannot becollected.

1.170 Financial transactions. Financial transactionsoccur when one kind of financial asset is exchanged foranother, bearing in mind that money is itself a financialasset. For example, the purchase of a bond or share is afinancial transaction. Borrowing is a financial transac-tion in which cash is exchanged, the counterpart beingthe creation of a financial asset or liability.

1.171 No consumption occurs when a financialtransaction takes place, even though financial transac-tions may be undertaken in order to facilitate futureconsumption. Financial transactions as such are notcovered by CPIs because, by definition, no goods areexchanged, nor services provided, in financial transac-tions. However, some ‘‘financial’’ transactions may notbe entirely financial because they may include anexplicit or implicit service charge in addition to theprovision of an asset, such as a loan. As a service chargeconstitutes the purchase of a service by the household,it should be included in a CPI, although it may bedifficult to separate out the service charge in some cases.For example, foreign exchange transactions are finan-cial transactions in which one financial asset is ex-changed for another. Changes in the price of a foreigncurrency in terms of the domestic currency resultingfrom changes in the exchange rate are outside the scopeof a CPI. On the other hand, the commission chargesassociated with the exchange of currencies are includedas payments for the services rendered by the foreignexchange dealers.

1.172 Households may borrow in order to makelarge expenditures on durables or houses, but also tofinance large educational or health expenses, or evenexpensive holidays. Whatever the purpose of the bor-rowing, the financial transaction in which the loan iscontracted is outside the scope of a CPI. The treatmentof the interest payable on loans is a separate issue con-sidered below.

1.173 Composite transactions. As just noted, sometransactions are composite transactions containingtwo or more components whose treatment may bequite different for CPI purposes. For example, part ofa life insurance premium is a financial transactionleading to the creation of a financial claim and istherefore outside the scope, whereas the remainderconsists of a service charge which should be covered by aCPI. The two components are not separately itemized,however.

1.174 As explained in Chapter 3, the treatment ofpayments of nominal interest is difficult because it mayhave four conceptually quite different components:

– a pure interest payment;

– a risk premium that depends on the creditworthinessof the borrower;

– a service charge payable to the bank, moneylender orother financial institution engaged in the business ofmaking loans;

– a payment to compensate the creditor for the realholding loss incurred on the principal of the loanduring inflation.

The fourth component is clearly outside the scope of aCPI as it is a capital flow. Conversely, the third com-ponent, the service charge, should clearly be included.The treatment of the first two components is con-troversial. When there is significant inflation or a veryimperfect capital market, payments of nominal interestmay be completely dominated by the last two compo-nents, both of which are conceptually quite differentfrom the concept of pure interest. For example, the‘‘interest’’ charged by a village moneylender may bemostly a high service charge. It may be impossible todecompose the various components of nominal interestin practice. The treatment of nominal interest as a wholeremains difficult and somewhat controversial.

Household production1.175 When households engage in production for

the market, the associated business transactions are alloutside the scope of a CPI. Expenditures incurred forbusiness purposes are excluded, even though theyinvolve purchases of goods and services that might beused to satisfy the personal needs and wants of membersof the household instead.

1.176 Households also produce goods and servicesfor their own consumption, mainly service productionsuch as the preparation of meals, the care of children,the sick or the elderly, the cleaning and maintenance ofdurables and dwellings, the transportation of householdmembers, and so on. Owner-occupiers produce housingservices for their own consumption. Households alsogrow vegetables, fruit, flowers or other crops for theirown use.

1.177 Many of the goods or services purchased byhouseholds do not provide utility directly but are usedas inputs into the production of other goods and servicesthat do provide utility: for example, raw foodstuffs,fertilizers, cleaning materials, paints, electricity, coal,oil, petrol, and so on.

1.178 In principle, a CPI should record changes inthe prices of the outputs from these production activities,as it is the outputs rather than the inputs that are actuallyconsumed and provide utility. However, as the outputsare not themselves purchased, no prices can be observedfor them. Prices could be imputed for them equal to theprices they would fetch on the market, but this wouldmake a CPI heavily dependent on assumed rather thancollected prices. The pragmatic solution recommended inChapter 3 is to treat all goods and services purchased on

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

20

the market to be used exclusively as inputs into theproduction of other goods and services that are directlyconsumed by households as if they were themselvesconsumption goods and services. On this principle,goods such as insecticides and electricity are treated asproviding utility indirectly and included in CPIs. This is,of course, the solution usually adopted in practice notonly for CPIs but also in national accounts, where mostexpenditures on inputs into household production areclassified as final consumption expenditures.1.179 In some countries, there is an increasing ten-

dency for households to purchase prepared, take-awaymeals rather than the ingredients. As the prices of suchmeals cost more than the sum of the ingredients that thehouseholds previously purchased, the weight attached tofood consumption increases. This partly reflects the factthat the costs of the households’ own labour inputs intothe preparation of meals were previously ignored. Var-ious kinds of household service activities that were pre-viously outside the scope of a CPI may be brought withinthe scope if households choose to pay others to performthe services.1.180 Subsistence agriculture and owner-occupied

housing. In the case of two important types of productionfor own consumption within households, namely agri-cultural production for own consumption and housingservices produced by owner-occupiers, the national ac-counts do actually try to record the values of the outputsproduced and consumed rather than the inputs. Simi-larly, CPIs may also try to price the outputs rather thanthe inputs in these two cases.1.181 In principle, the prices of the outputs from

own-account agricultural production may be included inCPIs, even though they are imputed. On the other hand,for households relying on subsistence agriculture, theprices of inputs of agricultural materials purchased onthe market may be their main exposure to inflation. Twopoints may be noted. First, the imputed market valueof the output should usually be greater than the costsof the purchased inputs, if only because it shouldcover the costs of the labour inputs provided by thehousehold. Thus, pricing the purchased inputs ratherthan the outputs may mean that the consumption of ownagricultural production in CPIs does not receive suf-ficient weight. Second, double counting should be avoi-ded. If the imputed prices of the outputs are included, theactual prices of the inputs consumed should not beincluded as well.1.182 In the case of owner-occupied housing, the

situation is complicated by the fact that the productionrequires the use of the capital services provided by alarge fixed asset in the form of the dwelling itself. Even ifthe inputs into the production of housing services arepriced for CPI purposes, it is still necessary to imputeprices for the inputs of capital services (mainly depre-ciation plus interest) provided by the dwelling. Somecountries therefore prefer to impute the prices of theoutputs of housing services actually consumed on thebasis of the rents payable for the same kind of dwellingsrented on the market. The treatment of owner-occupiedhousing is complex, and somewhat controversial, and isconsidered in Chapters 3, 9, 10 and 23, among others.

Coverage of households and outlets1.183 As explained in Chapter 3, households may be

either individual persons or groups of persons livingtogether who make common provision for food or otheressentials for living. A CPI may be required to cover:

– either the consumption expenditures made by house-holds resident in a particular area, usually a countryor region, whether the expenditures are made inside oroutside the area – this is called the ‘‘national’’ conceptof expenditure;

– or the consumption expenditures that take placewithin a particular area, whether made by householdsresident in that area or residents of other areas – thisis called the ‘‘domestic’’ concept.

Adopting the domestic concept may make it more dif-ficult to collect the relevant disaggregated expendituredata in household surveys. A CPI may also be defined tocover a group of countries, such as the European Union.

1.184 Not all kinds of households have to be in-cluded. As explained in Chapter 3, some countries chooseto exclude particular categories of households such as verywealthy households or households engaged in agricul-ture. Some countries also compile different indices de-signed to cover different groups of households, such ashouseholds resident in different regions. Another possi-bility is to compile a general CPI designed to cover all ormost households and, in addition, one or more specialindices aimed at particular sections of the community,such as households headed by pensioners. The precisecoverage of households is a matter of choice. It is inev-itably influenced by what are believed to be the main usesof the index. The set of households actually covered bythe CPI is described as the ‘‘reference population’’.

Price variation1.185 Prices for exactly the same good or service

may vary between different outlets, while different pricesmay sometimes be charged to different types of custo-mers. Prices may also vary during the course of themonth to which the index relates. Conceptually, it isnecessary to distinguish such pure price variation fromprice differences that are attributable to differences inthe quality of the goods or services offered, although it isnot always easy to distinguish between the two inpractice. The existence of pure price differences reflectssome form of market imperfections, such as consumers’lack of information or price discrimination.

1.186 When pure price differences exist, a change inmarket conditions may make it possible for somehouseholds to switch from purchasing at higher prices topurchasing at lower prices, for example if new outletsopen that offer lower prices. The resulting fall in theaverage price paid by households counts as a price fallfor CPI purposes, even though the price charged by eachindividual outlet may not change. If the prices are col-lected from the outlets and switches in households’purchasing habits remain unobserved, the CPIs are saidto be subject to outlet substitution bias, as explained inmore detail in Chapter 11. On the other hand, when theprice differences reflect differences in the quality of the

AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY

21

goods and services sold in the different outlets, switchingfrom outlets selling at higher prices to outlets selling atlower prices simply means that households are choosingto purchase lower-quality goods or services. In itself,this does not imply any change in price.

Classifications1.187 As explained in Chapter 3, the classification of

household expenditures used in a CPI provides the nec-essary framework for the various stages of CPI compi-lation. It provides a structure for purposes of weightingand aggregation, and also a basis for stratifying thesamples of products whose prices are collected. Thegoods and services covered by a CPI may be classifiedin several ways: not simply on the basis of their physicalcharacteristics but also by the purposes they serveand the degree of similarity of their price behaviour.Product-based and purpose-based classifications differbut can usually be successfully mapped onto each other.In practice, most countries use a hybrid classificationsystem in which the breakdown at the highest level isby purpose while the lower-level breakdowns are byproduct type. This is the case for the recently revisedinternationally agreed Classification of Individual Con-sumption according to Purpose (COICOP), which pro-vides a suitable classification for CPI purposes.

1.188 The first level of classification in COICOPconsists of 12 divisions covering total consumption ex-penditures of households. As just noted, the breakdowninto divisions is essentially by purpose. At the secondlevel of disaggregation, the 12 divisions are divided into47 groups of products, which are in turn divided into 117classes of products at the third level. Chapter 3 providesa listing of ten classes of goods defined as durables inCOICOP. It also gives a list of seven classes describedas semi-durables, such as clothing, footwear and house-hold textiles.

1.189 The 117 classes at the lowest level of aggrega-tion of COICOP are not sufficiently detailed for CPIpurposes. They can be divided into sub-classes using thesub-classes of the internationally agreed Central ProductClassification (CPC). Even some of these may requirefurther breakdown in order to arrive at some of the ele-mentary aggregates used for CPI purposes. In order tobe useful for CPI purposes, expenditure weights must beavailable for the various sub-classes or elementary aggre-gates. From a sampling perspective, it is desirable forthe price movements of the individual products withinthe elementary aggregates to be as homogeneous as pos-sible. The elementary aggregates may also be divided intostrata for sampling purposes, on the basis of location orthe type of outlet in which the products are sold.

Consumer price indices and nationalaccounts price deflators

1.190 Appendix 3.1 of Chapter 3 explains the dif-ferences between the overall CPI and the deflator fortotal household consumption expenditures in nationalaccounts. In practice, CPIs may be designed to coveronly a subset of the households and a subset of the

expenditures covered by the national accounts. More-over, the index number formulae needed for CPIs andnational accounts deflators may be different. These dif-ferences mean that the overall CPI is generally not thesame as the deflator for total household consumptionexpenditures in the national accounts. On the otherhand, the basic price and expenditure data collected andused for CPI purposes are also widely used to build upthe price indices needed to deflate the individual com-ponents of household consumption in the nationalaccounts.

Expenditure weights1.191 As already noted, there are two main stages in

the calculation of a CPI. The first is the collection of theprice data and the calculation of the elementary priceindices. The second is the averaging of the elementaryprice indices to arrive at price indices at higher levels ofaggregation up to the overall CPI itself. Expendituredata are needed for the elementary aggregates that canbe used as weights in the second stage. These weights areneeded whatever index number formula is used foraggregation purposes. Chapter 4 is concerned with thederivation, and sources, of the expenditure weights.

Household expenditure surveys andnational accounts

1.192 The principal data source for household con-sumption expenditures in most countries is a householdexpenditure survey (HES). An HES is a sample surveyof thousands of households that are asked to keep rec-ords of their expenditures on different kinds of con-sumer goods and services over a specified period of time,such as a week or longer. The size of the sampleobviously depends on the resources available, but alsoon the extent to which it is desired to break down thesurvey results by region or type of household. HESs arecostly operations. This manual is not concerned with theconduct of HESs or with general sampling survey tech-niques or procedures. There are several standard textson survey methods to which reference may be made.Household expenditure surveys may be taken at speci-fied intervals of time, such as every five years, or theymay be taken each year on a continuing basis.

1.193 HESs can impose heavy burdens on therespondents, who have to keep detailed expenditurerecords of a kind that they would not normally keep,although this may become easier when supermarkets orother retail outlets provide detailed printouts of pur-chases. HESs tend to have some systematic biases. Forexample, many households either deliberately, or uncon-sciously, understate the amounts of their expenditures oncertain ‘‘undesirable’’ products, such as gambling, alco-holic drink, tobacco or drugs. Corrections can be madefor such biases. Moreover, the data collected in HESsmay also need to be adjusted to bring them into line withthe concept of expenditure required by the CPI. Forexample, the imputed expenditures on the housing ser-vices produced and consumed by owner-occupiers arenot collected in HESs.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

22

1.194 As explained in Chapter 14, the use of thecommodity flow method within the supply and usetables of the SNA enables data drawn from differentprimary sources to be reconciled and balanced againsteach other. The commodity flow method may be used toimprove estimates of household consumption expendi-tures derived from expenditure surveys by adjustingthem to take account of the additional informationprovided by statistics on the sales, production, importsand exports of consumer goods and services. By draw-ing on various sources, the household expenditure datain the national accounts may provide the best estimatesof aggregate household expenditures, although theclassifications used may not be fine enough for CPIpurposes. Moreover, because HESs may be conductedonly at intervals of several years, the expenditure data inthe national accounts may be more up to date, asnational accounts are able to draw upon other kinds ofmore recent data, such as retail sales and the productionand import of consumer goods and services. It is impor-tant to note, however, that national accounts shouldnot be viewed as if they were an alternative, indepen-dent data source to HESs. On the contrary, HESs pro-vide one of the main sources for the expenditure dataon household consumption used to compile nationalaccounts.1.195 Household expenditure surveys in many

countries may not be conducted as frequently as mightbe desired for CPI, or national accounts, purposes.National HESs can be very costly and onerous for thehouseholds, as already noted. They may be conductedonly once every five or ten years, or even at longerintervals. In any case, conducting and processing HESsis time-consuming, so the results may not be availablefor CPI purposes until one or two years after the surveyshave been conducted. It is for these practical reasonsthat CPIs in many countries are Lowe indices that usethe quantities of some base period b that may precedethe time reference period 0 by a few years and period tby many years.1.196 Some countries conduct continuous HESs not

only in order to update their CPI weights but also toimprove their national accounts. Of course, the samepanel of households does not have to be retained indef-initely; the panel can be gradually rotated by droppingsome households and replacing them by others. Coun-tries that conduct continuous expenditure surveys areable to revise and update their expenditure weights eachyear so that the CPI becomes a chain index with annuallinking. Even with continuous expenditure surveys,however, there is a lag between the time at which the dataare collected and the time at which the results are pro-cessed and ready for use, so that it is never possible tohave survey results that are contemporaneous with theprice changes. Thus, even when the weights are updatedannually, they still refer to some period that precedes thetime reference period. For example, when the pricereference period is January 2000, the expenditure weightsmay refer to 1997 or 1998, or both years. When the pricereference period moves forward to January 2001, theweights move forward to 1998 or 1999, and so on. Suchan index is a chain Lowe index.

1.197 Some countries prefer to use expenditureweights that are the average rates of expenditure overperiods of two or three years in order to reduce ‘‘noise’’caused by errors of estimation (the expenditure surveysare only samples) or erratic consumer behaviour overshort periods of time resulting from events such asbooms or recessions, stock market fluctuations, oilshocks, or natural or other disasters.

Other sources for estimatingexpenditure weights

1.198 If expenditures need to be disaggregated byregion for sampling or analytical purposes, it is possibleto supplement whatever information may be availableby region in HESs by using data from population cen-suses. Another data source may be food surveys. Theseare special surveys, conducted in some countries, thatfocus on households’ expenditures on food products.They can provide more detailed information on foodexpenditures than that available from HESs.

1.199 Another possible source of information con-sists of points of purchase (POP) surveys, which areconducted in some countries. A POP survey is designedto provide information about the retail outlets at whichhouseholds purchase specified groups of goods and ser-vices. Households are asked, for each item, about theamounts spent in each outlet and the names andaddresses of the outlets. The main use for a POP survey isfor selecting the sample of outlets to be used for pricecollection purposes.

Collection of price data1.200 As explained in Chapter 9, there are two levels

of calculation involved in a CPI. At the lower level,samples of prices are collected and processed to obtainlower-level price indices. These lower-level indices arethe elementary indices, whose properties and behaviourare studied in Chapter 20. At the higher level, the ele-mentary indices are averaged to obtain higher-levelindices using expenditures as weights. At the higherlevel, all the index number theory elaborated in Chap-ters 15 to 18 comes into play.

1.201 Lower-level indices are calculated for ele-mentary aggregates. Depending on the resources avail-able and procedures adopted by individual countries,these elementary aggregates could be sub-classes ormicro-classes of the expenditure classification describedabove. If it is desired to calculate CPIs for differentregions, the sub-classes or micro-classes have to bedivided into strata referring to the different regions. Inaddition, in order to improve the efficiency of the sam-pling procedures used to collect prices, it will usuallybe desirable, if feasible, to introduce other criteria intothe definitions of the strata, such as the type of outlet.When the sub-classes or micro-classes are divided intostrata for data collection purposes, the strata themselvesbecome the elementary aggregates. As a weight needsto be attached to each elementary aggregate in order tocalculate the higher-level indices, an estimate of theexpenditure within each elementary aggregate must be

AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY

23

available. Expenditure or quantity data are typically notavailable within an elementary aggregate, so the ele-mentary indices have to be estimated from price dataalone. This may change if scanner data from electronicpoints of sale become more available.

1.202 Chapter 5 is concerned with sampling strate-gies for price collection. Chapter 6 is concerned with themethods and operational procedures actually used tocollect prices. In principle, the relevant prices for a CPIshould be the purchasers’ prices actually paid byhouseholds, but it is generally neither practical nor cost-effective to try to collect prices each month or quarterdirectly from households, even though expenditure dataare collected directly from households in householdexpenditure surveys. In practice, the prices that arecollected are not actual transaction prices, but rather theprices at which goods and services are offered for sale inoutlets such as retail shops, supermarkets or serviceproviders. However, it may become increasingly feasibleto collect actual transactions prices as more goods andservices are sold through electronic points of sale thatrecord both prices and expenditures.

Random sampling andpurposive sampling

1.203 Given that the prices are collected from thesellers, there are two different sampling problems thatarise. The first is how to select the individual productswithin an elementary aggregate whose prices are to becollected. The second is how to select a sample of outletsselling those products. For some products, it may not benecessary to visit retail outlets to collect prices becausethere may be only a single price applying throughout thecountry. Such prices may be collected from the centralorganization responsible for fixing the prices. The fol-lowing paragraphs refer to the more common situation inwhich prices are collected from a large number of outlets.

1.204 As explained in Chapter 5, the universe ofproducts from which the sample is taken has severaldimensions. The products may be classified not only onthe basis of the characteristics and functions thatdetermine their place in COICOP, but also according tothe locations and outlets at which they are sold and thetimes at which they are sold. The fact that the universe iscontinually changing over time is a major problem, notonly for CPIs but also for most other economic statis-tics. Products disappear to be replaced by other kinds ofproducts, while outlets close and new ones open. Thefact that the universe is changing over time creates bothconceptual and practical problems, given that the mea-surement of price changes over time requires somecontinuity in the products priced. In principle, the pricechanges recorded should refer to matched products thatare identical in both time periods. The problems createdwhen products are not identical are considered in somedetail later.

1.205 In designing the sample for price collectionpurposes, due attention should be paid to standard sta-tistical criteria to ensure that the resulting sample esti-mates are not only unbiased and efficient in a statisticalsense, but also cost-effective. There are two types of bias

encountered in the literature on index numbers, namelysampling bias as understood here and the non-samplingbiases in the form of substitution bias or representativitybias, as discussed in Chapter 10. It is usually clear fromthe context which type of bias is meant.

1.206 There is a large literature on sampling surveytechniques to which reference may be made and whichneed not be summarized here. In principle, it would bedesirable to select both outlets and products using ran-dom sampling with known probabilities of selection.This ensures that the sample of products selected is notdistorted by subjective factors and enables samplingerrors to be calculated. Many countries neverthelesscontinue to rely heavily on the purposive selection ofoutlets and products, because random sampling may betoo difficult and too costly. Purposive selection is be-lieved to be more cost-effective, especially when thesampling frames available are not comprehensive andnot well suited to CPI purposes. It may also be cost-effective to collect a ‘‘cluster’’ of prices on differentproducts from the same outlet, instead of distributingthe price collection more thinly over a larger number ofoutlets.

1.207 Efficient sampling, whether random or pur-posive, requires comprehensive and up-to-date samplingframes. Two types of frames are needed for CPI pur-poses: one listing the universe of outlets, and the otherlisting the universe of products. Examples of possiblesampling frames for outlets are business registers, cen-tral or local government administrative records or tele-phone directories. When the sampling frames containthe requisite information, it may be possible to increaseefficiency by selecting samples of outlets using prob-abilities that are proportional to the size of some rele-vant economic characteristic, such as the total value ofsales. Sampling frames for products are not alwaysreadily available in practice. Possible frames are cata-logues or other product lists drawn up by major man-ufacturers, wholesalers or trade associations, or lists ofproducts that are specific to individual outlets such aslarge supermarkets.

1.208 Depending on the information available in thesampling frame, it may be possible to group the outletsinto strata on the basis of their location and size, asindicated by sales or employees. When there is infor-mation about size, it may be possible to increase effi-ciency by taking a random sample of outlets withprobabilities proportional to size. In practice, however,there is also widespread use of purposive sampling.

1.209 In most countries, the selection of most of theindividual items to be priced within the selected outletstends to be purposive, being specified by the centraloffice responsible for the CPI. The central office drawsup lists of products that are deemed to be representativeof the products within an elementary aggregate. The listscan be drawn up in collaboration with managers ofwholesale or large retail establishments, or other expertswith practical experience and knowledge. The actualprocedures are described in more detail in Chapter 6.

1.210 It has been argued that the purposive selectionof products is liable to introduce only a negligibleamount of sampling bias, although there is not much

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

24

conclusive evidence on this matter. In principle, randomsampling is preferable and it is also quite feasible. Forexample, the United States Bureau of Labor Statisticsmakes extensive use of random selection procedures toselect both outlets and products within outlets. Whenthe selection of products is delegated to the individualprice collectors, it is essential to ensure that they arewell trained and briefed, and closely supervised andmonitored.

Methods of price collection1.211 The previous section focused on the sampling

issues that arise when prices have to be collected for alarge number of products from a large number of out-lets. This section is concerned with some of the moreoperational aspects of price collection.1.212 Central price collection. Many important

prices can be collected directly by the central officeresponsible for the CPI from the head office of the or-ganization responsible for fixing the prices. When pricesare the same throughout the country, collection fromindividual outlets is superfluous:

� Some tariffs or service charges are fixed nationally andapply throughout the country. This may be the case forpublic utilities such as water, gas and electricity, postalservices and telephone charges, or public transport.The prices or charges can be obtained from the rele-vant head offices.

� Some national chains of stores or supermarkets maycharge the same prices everywhere, in which case theprices can be obtained from their head offices. Evenwhen national chains do not charge uniform prices,there may be only a few minor regional differences inthe prices and all the relevant information may beobtainable centrally.

� Many of these prices determined centrally may changevery infrequently, perhaps only once or twice or year,so they do not have to be collected monthly. More-over, many of these prices can be collected by tele-phone, fax or email and may not require visits to thehead offices concerned.

1.213 Scanner data. One important new develop-ment is the increasing availability in many countries oflarge amounts of very detailed ‘‘scanner’’ data obtainedfrom electronic points of sale. Such data are collated bycommercial databases. Scanner data are up to date andcomprehensive. An increasingly large proportion of allgoods sold are being scanned as they pass throughelectronic points of scale.1.214 The potential benefits of using scanner data

are obviously considerable and could ultimately have asignificant impact on the way in which price data arecollected for CPI purposes. Not enough experience is yetavailable to provide general guidelines about the use ofscanner data. Clearly, statistical offices should monitordevelopments in this field closely and explore the pos-sibility of exploiting this major new source of data.Scanner data also increase the scope for using improvedmethods of quality adjustment, including hedonic meth-ods, as explained in Chapter 7.

1.215 Local price collection. When prices are col-lected from local outlets, the individual products selectedfor pricing can be determined in two ways. One way isfor a specific list of individual products to be determinedin advance by the central office responsible for the CPI.Alternatively, the price collector can be given the dis-cretion to choose from a specified range of products.The collector may use some kind of random selectionprocedure, or select the products that sell the most orare recommended by the shop owner or manager. Anindividual product selected for pricing in an individualoutlet may be described as a sampled product. It may bea good or a service.

1.216 When the list of products is determined inadvance by the central office, the objective is usually toselect products that are considered to be representativeof the larger group of products within an elementaryaggregate. The central office also has to decide howloosely or tightly to describe, or specify, the repre-sentative products selected for pricing. In theory, thenumber of different products that might be identified isto some extent arbitrary, depending on the number ofeconomic characteristics that are deemed to be relevantor important. For example, ‘‘beef ’’ is a generic term fora group of similar but nevertheless distinct products.There are many different cuts of beef, such as mincedbeef, stewing steak or rump steak, each of which can beconsidered a different product and which can sell at verydifferent prices. Furthermore, beef can also be classifiedaccording to whether it is fresh, chilled or frozen, andcross-classified again according to whether it comesfrom domestic or imported animals, or from animals ofdifferent ages or breeds.

1.217 Tightening the specifications ensures that thecentral office has more control over the items actuallypriced in the outlets, but it also increases the chance thatsome products may not actually be available in someoutlets. Loosening the specifications means that moreitems may be priced but leaves the individual price col-lectors with more discretion with regard to the itemsactually priced. This could make the sample as a wholeless representative.

Continuity of price collection1.218 A CPI is intended to measure pure price

changes. The products whose prices are collected andcompared in successive time periods should ideally beperfectly matched; that is, they should be identical inrespect of their physical and economic characteristics.When the products are perfectly matched, the observedprice changes are pure price changes. When selectingrepresentative products, it is therefore necessary toensure that enough of them can be expected to remainon the market over a reasonably long period of time inexactly the same form or condition as when first select-ed. Without continuity, there would not be enough pricechanges to measure.

1.219 Having identified the items whose prices are tobe collected, the normal strategy is to continue pricingexactly those same items for as long as possible. Pricecollectors can do this if they are provided with veryprecise, or tight, specifications of the items to be priced.

AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY

25

Alternatively, they must keep detailed records them-selves of the items that they have selected to price.

1.220 The ideal situation for a price index would beone in which all the products whose prices are beingrecorded remain on the market indefinitely without anychange in their physical and economic characteristics,except of course for the timing of their sale. It is worthnoting that many theorems in index number theory arederived on the assumption that exactly the same set ofgoods and services is available in both the time periodsbeing compared. Most products, however, have only alimited economic life. Eventually, they disappear fromthe market to be replaced by other products. As theuniverse of products is continually evolving, the repre-sentative products selected initially may graduallyaccount for a progressively smaller share of total pur-chases and sales. As a whole, they may become less andless representative. As a CPI is intended to cover allproducts, some way has to be found to accommodate thechanging universe of products. In the case of consumerdurables whose features and designs are continuallybeing modified, some models may have very short livesindeed, being on the market for only a year or less beforebeing replaced by newer models.

1.221 At some point the continuity of the series ofprice observations may have to be broken. It maybecome necessary to compare the prices of some pro-ducts with the prices of other new ones that are verysimilar but not identical. Statistical offices must then tryto eliminate from the observed price changes the esti-mated effects of the changes in the characteristics of theproducts whose prices are compared. In other words,they must try to adjust the prices collected for anychanges in the quality of the products priced, as ex-plained in more detail below. At the limit, a completelynew product may appear that is so different from thoseexisting previously that quality adjustment is not fea-sible and its price cannot be directly compared with thatof any previous product. Similarly, a product maybecome so unrepresentative or obsolete that it has to bedropped from the index because it is no longer worthtrying to compare its price with those of any of theproducts that have displaced it.

Resampling1.222 One strategy to deal with the changing uni-

verse of products would be to resample, or reselect, atregular intervals the complete set of items to be priced.For example, with a monthly index, a new set of itemscould be selected each January. Each set of items wouldbe priced until the following January. Two sets haveto be priced each January in order to establish a linkbetween each set of 12 monthly changes. Resamplingeach year would be consistent with a strategy ofupdating the expenditure weights each year.

1.223 Although resampling may be preferable tomaintaining an unchanged sample or selection, it is notused much in practice. Systematically resampling theentire set of products each year would be difficult tomanage and costly to implement. Moreover, it does notprovide a complete solution to the problem of the

changing universe of products, as it does not captureprice changes that occur at the moment of time whennew products or new qualities are first introduced.Many producers deliberately use the time when productsare first marketed to make significant price changes.

1.224 A more practical way in which to keep thesample up to date is to rotate it gradually by droppingcertain items and introducing new ones. Items may bedropped for two reasons:

� The product is believed by the price collector or cen-tral office to be no longer representative. It appears toaccount for a steadily diminishing share of the totalexpenditures within the basic categories in question.

� The product may simply disappear from the marketaltogether. For example, it may have become obsoleteas a result of changing technology or unfashionablebecause of changing tastes, although it could dis-appear for other reasons.

1.225 At the same time, new products or new qual-ities of existing products appear on the market. At somepoint, it becomes necessary to include them in the list ofitems priced. This raises the general question of thetreatment of quality change and the treatment of newproducts.

Adjusting prices forquality changes

1.226 The treatment of quality change is perhaps thegreatest challenge facing CPI compilers. It is a recurringtheme throughout this manual. It presents both con-ceptual and practical problems for compilers of CPIs.The whole of Chapter 7 is devoted to the treatment ofquality change, while Chapter 8 addresses the closelyrelated topic of new goods and item substitution.

1.227 When a sampled product is dropped from thelist of products priced in some outlet, the normal prac-tice is to find a new product to replace it in order toensure that the sample, or selection, of sampled productsremains sufficiently comprehensive and representative. Ifthe new product is introduced specifically to replace theold one, it is necessary to establish a link between theseries of past price observations on the old item andthe subsequent series for the new item. The two series ofobservations may, or may not, overlap in one or moreperiods. In many cases, there can be no overlap becausethe new quality, or model, is only introduced afterthe one which it is meant to replace is discontinued.Whether or not there is an overlap, the linking of thetwo price series requires some estimate of the change inquality between the old product and the product selec-ted to replace it.

1.228 However difficult it is to estimate the con-tribution of the change in quality to the change in theobserved price, it must be clearly understood that someestimate has to be made either explicitly or, by default,implicitly. The issue cannot be avoided or bypassed. Allstatistical offices have limited resources and many maynot have the capacity to undertake the more elaborateexplicit adjustments for quality change described in

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

26

Chapter 7. Even though it may not be feasible to under-take an explicit adjustment through lack of data orresources, it is not possible to avoid making some kind ofimplicit adjustment. Even apparently ‘‘doing nothing’’necessarily implies some kind of implicit adjustment, asexplained below. Whatever the resources available tothem, statistical offices must be conscious of the impli-cations of the procedures they adopt.1.229 Three points are stressed in the introductory

section of Chapter 7:

� The pace of innovation is high, and possibly increas-ing, leading to continual changes in the characteristicsof products.

� There is not much consistency between countries inthe methods they use to deal with quality change.

� A number of empirical studies have demonstratedthat the choice of method does matter, as differentmethods can lead to very different results.

Evaluation of the effect of qualitychange on price1.230 It is useful to try to clarify why one would wish

to adjust the observed price change between two itemsthat are similar, but not identical, for differences in theirquality. A change in the quality of a good or serviceoccurs when there is a change in some, but not most, ofits characteristics. For purposes of a CPI, a qualitychange must be evaluated from the consumer’s perspec-tive. As explained in Chapter 7, the evaluation of thequality change is essentially an estimate of the additionalamount that a consumer is willing to pay for the newcharacteristics possessed by the new quality. This addi-tional amount is not a price increase because it representsthe monetary value of the additional satisfaction orutility that is derived from the new quality. Of course, ifthe old quality is preferred to the new one, consumerswould only be willing to buy the new quality if its pricewere lower.1.231 Consider the following hypothetical experi-

ment in which a new quality appears alongside an oldone. Assume that the two products are substitutes andthat the consumer is familiar with the characteristics ofthe old and the new qualities. Use lower case p to referto prices of the old quality and upper case P for theprices of the new quality. Suppose that both qualities areoffered to the consumer at the same price, namely theprice Pt at which the new quality is actually being sold inperiod t. The consumer is then asked to choose betweenthem and prefers the new quality.1.232 Suppose next that the price of the old quality

is progressively reduced until it reaches p�t , at whichpoint the consumer becomes indifferent between pur-chasing the old quality at p�t and the new quality at Pt.Any further decrease below p�t causes the consumer toswitch back to the old quality. The difference between Ptand p�t is a measure of the additional value that theconsumer places on the new quality as compared withthe old quality. It measures the maximum amount thatthe consumer is willing to pay for the new quality overand above the price of the old quality.

1.233 Let pt�1 denote the actual price at which theold quality was sold in period t�1. For CPI purposes,the price increase between the two qualities is not theobserved difference Pt � pt�1 but p*

t � pt�1. It is impor-tant to note that p*

t , the hypothetical price for the oldquality in period t, is directly comparable with the actualprice of the old quality in period t�1 because both referto the same identical product. The difference betweenthem is a pure price change. The difference between Ptand p*

t is not a price change but an evaluation of thedifference in the quality of the two items in period t. Theactual price of the new quality in period t needs to bemultiplied by the ratio p*

t /Pt in order to make thecomparison between the prices in periods t�1 and t acomparison between products of equal quality in theeyes of the consumer. The ratio p*

t =Pt is the requiredquality adjustment.

1.234 Of course, it is difficult to estimate the qualityadjustment in practice, but the first step has to be toclarify conceptually the nature of the adjustment that isrequired in principle. In practice, producers often treatthe introduction of a new quality, or new model, as aconvenient opportunity at which to make a significantprice change. They may deliberately make it difficult forconsumers to disentangle how much of the observeddifference in price between the old and the new qualitiesrepresents a price change.

1.235 Chapter 7 explains the two possibilities opento statistical offices. One possibility is to make an explicitadjustment to the observed price change on the basis ofthe different characteristics of the old and new qualities.The other alternative is to make an implicit adjustmentby making an assumption about the pure price change;for example, on the basis of price movements observedfor other products. It is convenient to take the implicitmethods first.

Implicit methods for adjusting forquality changes

1.236 Overlapping qualities. Suppose that the twoqualities overlap, both being available on the market attime t. If consumers are well informed, have a freechoice and are collectively willing to buy some of both atthe same time, economic theory suggests that the ratioof the prices of the new to the old quality should reflecttheir relative utilities to consumers. This implies that thedifference in price between the old and the new qualitiesdoes not indicate any change in price. The price changesup to period t can be measured by the prices for the oldquality, while the price changes from period t onwardscan be measured by the prices for the new quality. Thetwo series of price changes are linked in period t, thedifference in price between the two qualities not havingany impact on the linked series.

1.237 When there is an overlap, simple linking of thiskind may provide an acceptable solution to the problemof dealing with quality change. In practice, however, thismethod is not used very extensively because the requisitedata are seldom available. Moreover, the conditions maynot be consistent with those assumed in the theory. Evenwhen there is an overlap, consumers may not have had

AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY

27

time to acquire sufficient knowledge of the character-istics to be able to evaluate the relative qualities properly,especially when there is a substantial change in quality.Not all consumers may have access to both qualities.When the new quality first appears, the market is liableto remain in disequilibrium for some time, as it takestime for consumers to adjust their consumption patterns.

1.238 There may be a succession of periods in whichthe two qualities overlap before the old quality finallydisappears from the market. If the market is temporarilyout of equilibrium, the relative prices of the two qualitiesmay change significantly over time so that the marketoffers alternative evaluations of the relative qualitiesdepending on which period is chosen. When new qual-ities that embody major new improvements appear onthe market for the first time, there is often a tendency fortheir prices to fall relatively to older qualities before thelatter eventually disappear. In this situation, if the priceseries for the old and new qualities are linked in a singleperiod, the choice of period can have a substantial effecton the overall change in the linked series.

1.239 The statistician has then to make a deliberatejudgement about the period in which the relative pricesappear to give the best representation of the relativequalities. In this situation, it may be preferable to use amore complex linking procedure which uses the pricesfor both the new and the old qualities in several periodsin which they overlap. However, the information neededfor this more complex procedure will never be availableif price collectors are instructed only to introduce a newquality when an old one is dropped. In this case, thetiming of the switch from the old to the new can have asignificant effect on the long-term change in the linkedseries. This factor must be explicitly recognized andtaken into consideration.

1.240 If there is no overlap between the new and theold qualities, the problems just discussed do not arise asno choice has to be made about when to make the link.Other and more difficult problems nevertheless taketheir place.

1.241 Non-overlapping qualities. In the followingsections, it is assumed that the overlap method cannotbe used because there is a discontinuity between theseries of price observations for the old and new qualities.Again, using lower case p for the old quality andupper case P for the new, it is assumed that the pricedata available to the index compiler take the follow-ing form:

. . . ; pt�3; pt�2; pt�1;Pt;Pt+1;Pt+2; . . .

The problem is to estimate the pure price changebetween t�1 and t in order to have a continuous seriesof price observations for inclusion in the index. Usingthe same notation as above:

– price changes up to period t�1 are measured by theseries for the old quality;

– the change between t�1 and t is measured by theratio p�t =pt�1 where p�t is equal to Pt after adjustmentfor the change in quality;

– price changes from period t onwards are measured bythe series for the new quality.

1.242 The problem is to estimate p�t . This may bedone explicitly by one of the methods described later.Otherwise, one of the implicit methods has to be used.These may be grouped into three categories:

� The first solution is to assume that p�t =pt�1=Pt=pt�1

or p�t=Pt. No change in quality is assumed to haveoccurred, so the whole of the observed price increaseis treated as a pure price increase. In effect, this con-tradicts the assumption that there has been a changein quality.

� The second is to assume that p�t =pt�1=1, or p�t=pt�1.No price change is assumed to have occurred, thewhole of the observed difference between pt�1 and Ptbeing attributed to the difference in their quality.

� The third is to assume that p�t =pt�1=I , where I is anindex of the price change for a group of similar pro-ducts, or possibly a more general price index.

1.243 The first two possibilities cannot be recom-mended as default options to be used automatically inthe absence of any adequate information. The use of thefirst option can only be justified if the evidence suggeststhat the extent of the quality change is negligible, eventhough it cannot be quantified more precisely. ‘‘Doingnothing’’, in other words ignoring the quality changecompletely, is equivalent to adopting the first solution.Conversely, the second can only be justified if the evi-dence suggests that the extent of any price changebetween the two periods is negligible. The third option islikely to be much more acceptable than the other two. Itis the kind of solution that is often used in economicstatistics when data are missing.

1.244 Elementary indices are typically based on anumber of series relating to different sampled products.The particular linked price series relating to the twoqualities is therefore usually just one out of a number ofparallel price series. What may happen in practice is thatthe price observations for the old quality are used up toperiod t�1 and the prices for the new quality from tonwards, the price change between t�1 and t beingomitted from the calculations. In effect, this amounts tousing the third option: that is, estimating the missingprice change on the assumption that it is equal to theaverage change for the other sampled products withinthe elementary aggregate.

1.245 It may be possible to improve on this estimateby making a careful selection of the other sampledproducts whose average price change is believed to bemore similar to the item in question than the average forthe group of sampled products as a whole. This proce-dure is described in some detail in Chapter 7, where it isillustrated with a numerical example and described as‘‘targeting’’ the imputation or estimation.

1.246 The general method of estimating the priceon the basis of the average change for the remaininggroup of products is widely used. It is sometimesdescribed as the ‘‘overall’’ class mean method. The morerefined targeted version is the ‘‘targeted’’ mean method.In general, one or other method seems likely to be prefer-able to either of the first two options listed above,although each case must be considered on its individualmerits.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

28

1.247 While the class mean method seems a sensiblepractical solution, it may nevertheless give biasedresults, as explained in Chapter 7. The introduction of anew quality is precisely the occasion on which a pro-ducer may choose to make a significant price change.Many of the most important price changes may bemissed if, in effect, they are assumed to be equal to theaverage price changes for products not subject to qualitychange.1.248 It is necessary, therefore, to try to make an

explicit adjustment for the change in quality, at least whena significant quality change is believed to have occurred.Again there are several methods that may be used.

Explicit quality adjustments1.249 Quantity adjustments. The quality change may

take the form of a change in the physical characteristicsof the product that can easily be quantified, such aschange in weight, dimensions, purity, or chemical com-position of a product. It is generally a considerableoversimplification to assume that the quality of a pro-duct changes in proportion to the size of some singlephysical characteristic. For example, most consumers arevery unlikely to rate a refrigerator that has three timesthe capacity of a smaller one as being worth three timesthe price of the latter. Nevertheless it is clearly possible tomake some adjustment to the price of a new quality ofdifferent size to make it more comparable with the priceof an old quality. There is considerable scope for thejudicious, or common sense, application of relativelystraightforward quality adjustments of this kind. Athorough discussion of quality adjustments based on‘‘size’’ is given in Chapter 7.1.250 Differences in production or option costs. An

alternative procedure may be to try to measure thechange in quality by the estimated change in the costs ofproducing the two qualities. The estimates can be madein consultation with the producers of the goods or ser-vices, if appropriate. This method, like the first, is onlylikely to be satisfactory when the changes take the formof relatively simple changes in the physical character-istics of the good, such as the addition of some newfeature, or option, to an automobile. It is not satisfac-tory when a more fundamental change in the natureof the product occurs as a result of a new discovery ortechnological innovation. It is clearly inapplicable, forexample, when a drug is replaced by another moreeffective variant of the same drug that also happens tocost less to produce.1.251 Another possibility for dealing with a quality

change that is more complex or subtle is to seek theadvice of technical experts. This method is especiallyrelevant when the general consumer may not have theknowledge or expertise to be able to assess or evaluatethe significance of all of the changes that may haveoccurred, at least when they are first made.1.252 The hedonic approach. Finally, it may be

possible to systematize the approach based on produc-tion or option costs by using econometric methods toestimate the impact of observed changes in the char-acteristics of a product on its price. In this approach,

the market prices of a set of different qualities or modelsare regressed on what are considered to be the mostimportant physical or economic characteristics of thedifferent models. This approach to the evaluation ofquality change is known as hedonic analysis. When thecharacteristics are attributes that cannot be quantified,they are represented by dummy variables. The regres-sion coefficients measure the estimated marginal effectsof the various characteristics on the prices of the modelsand can therefore be used to evaluate the effects ofchanges in those characteristics, i.e., changes in quality,over time.

1.253 The hedonic approach to quality adjustmentcan provide a powerful, objective and scientific methodof evaluating changes in quality for certain kinds ofproducts. It has been particularly successful in dealingwith computers. The economic theory underlying thehedonic approach is examined in more detail in Chapter21. The application of the method is explained in somedetail in Chapter 7. Products can be viewed as bundlesof characteristics that are not individually priced, asthe consumer buys the bundle as a single package. Theobjective is to try to ‘‘unbundle’’ the characteristicsto estimate how much they contribute to the total price.In the case of computers, for example, three basic char-acteristics are the processor speed, the size of the RAMand the hard drive capacity. An example of a hedonic re-gression using these characteristics is given in Chapter 7.

1.254 The results obtained by applying hedonics tocomputer prices have had a considerable impact onattitudes towards the treatment of quality change inCPIs. They have demonstrated that for goods wherethere are rapid technological changes and improvementsin quality, the size of the adjustments made to the marketprices of the products to offset the changes in the qualitycan largely determine the movements of the elementaryprice index. For this reason, the manual contains athorough treatment of the use of hedonics. Chapter 7provides further analysis, including a comparison show-ing that the results obtained by using hedonics andmatched models can differ significantly when there is ahigh turnover of models.

1.255 It may be concluded that statistical officesmust pay close attention to the treatment of qualitychange and try to make explicit adjustments wheneverpossible. The importance of this topic can scarcely beover-emphasized. The need to recognize and adjust forchanges in quality has to be impressed on price collec-tors. Failure to pay proper attention to quality changescan introduce serious biases into a CPI.

Item substitution and new goods1.256 As noted above, ideally price indices would

seek to measure pure price changes between matchedproducts that are identical in the two periods compared.However, as explained in Chapter 8, the universe ofproducts that a CPI has to cover is a dynamic universethat is gradually changing over time. Pricing matchedproducts constrains the selection of products to a staticuniverse of products given by the intersection of the twosets of products existing in the two periods compared.

AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY

29

This static universe, by definition, excludes both newproducts and disappearing products, whose price behav-iour is likely to diverge from that of the matched prod-ucts. Price indices have to try to take account of theprice behaviour of new and disappearing products as faras possible.

1.257 A formal consideration and analysis of theseproblems are given in Appendix 8.1 to Chapter 8. Areplacement universe is defined as one that starts with thebase period universe but allows new products to enter asreplacements as some products disappear. Of course,quality adjustments of the kind discussed above areneeded when comparing the prices of the replacementproducts with those of the products that they replace.

1.258 One way in which to address the underlyingproblem of the changing universe is by sample rotation.This requires a completely new sample of products to bedrawn to replace the existing one. The two samples mustoverlap in one period that acts as the link period. Thisprocedure can be viewed as a systematic exploitation ofthe overlap method of adjusting for quality change. Itmay not therefore deal satisfactorily with all changes inquality that occur, because the relative prices of differentgoods and services at a single point of time may notprovide satisfactory measures of the relative qualities ofall the goods and services concerned. Nevertheless, fre-quent sample rotation helps by keeping the sample up todate and may reduce the extent to which explicit qualityadjustments are required. Sample rotation is expensive,however.

New goods and services1.259 The difference in quality between the original

product and the one that it replaces may become sogreat that the new quality is better treated as a newgood, although the distinction between a new qualityand a new good is inevitably somewhat arbitrary. Asnoted in Chapter 8, a distinction is also drawn in theeconomics literature between evolutionary and revolu-tionary new goods. An evolutionary new good or serviceis one that meets existing needs in much more efficient ornew ways, whereas a revolutionary new good or serviceprovides completely new kinds of services or benefits. Inpractice, an evolutionary new good can be fitted intosome sub-class of the product or expenditure classifica-tion, whereas a revolutionary new good will requiresome modification to the classification in order toaccommodate it.

1.260 There are two main concerns with new goodsor services. The first relates to the timing of the intro-duction of the new product into the index. The secondrelates to the fact that the mere availability of the newproduct on the market may bring a welfare gain toconsumers, whatever the price at which it is sold initial-ly. Consider, for example, the introduction of the firstantibiotic drug, penicillin. The drug provided cures forconditions that previously might have been fatal. Thebenefit might be virtually priceless to some individuals.One way of gauging how much benefit is gained by theintroduction of a new good is to ask how high its pricewould have to be to reduce the demand for the product

to zero. Such a price is called the ‘‘demand reservationprice’’. It could be very high indeed in the case of a newlife-saving drug. If the demand reservation price couldbe estimated, it could be treated as the price in theperiod just before the new product appeared. The fallbetween the demand reservation price and the price atwhich the product actually makes its first appearancecould be included in the CPI.

1.261 In practice, of course, statistical offices cannotbe expected to estimate demand reservation prices withsufficient reliability for them to be included in a CPI.The concept is nevertheless useful because it highlightsthe fact that the mere introduction of a new good maybring a significant welfare gain that could be reflected inthe CPI, especially if it is intended to be a COLI. Ingeneral, any enlargement of the set of consumptionpossibilities open to consumers has the potential tomake them better off, other things being equal.

1.262 It is often the case that new goods enter themarket at a higher price than can be sustained in thelonger term, so their prices typically tend to fall relativelyover the course of time. Conversely, the quantities pur-chased may be very small initially but increase sig-nificantly. These complications make the treatment ofnew products particularly difficult, especially if they arerevolutionary new goods. Because of both the welfaregain from the introduction of a new product and thetendency for the price of a new good to fall after it hasbeen introduced, it is possible that important price re-ductions may fail to be captured by CPIs because of thetechnical difficulties created by new products. Chapter 8concludes by expressing concern about the capacity ofCPIs to deal satisfactorily with the dynamics of modernmarkets. In any case, it is essential that statistical officesare alert to these issues and adopt procedures that takeaccount of them to the maximum extent possible, giventhe data and resources available to them.

Calculation of consumer priceindices in practice

1.263 Chapter 9 provides a general overview of theways in which CPIs are calculated in practice. Themethods used in different countries are by no means allthe same, but they have much in common. There isclearly interest from users as well as compilers inknowing how most statistical offices set about calculat-ing their CPIs. The various stages in the calculationprocess are illustrated by numerical examples. Thechapter is descriptive and not prescriptive, although itdoes try to evaluate the strengths and weaknesses ofexisting methods. It makes the point that because of thegreater insights into the properties and behaviour ofindices gained in recent years, it is now recognized thatnot all existing practices are necessarily optimal.

1.264 As the various stages involved in the calcula-tion process have, in effect, already been summarized inthe preceding sections of this chapter, it is not proposedto repeat them all again in this section. It may be useful,however, to give an indication of the nature of the con-tents of Chapter 9.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

30

Elementary price indices

1.265 Chapter 9 starts by describing how the ele-mentary aggregates are constructed by working downfrom groups, classes and sub-classes of COICOP, orsome equivalent expenditure classification. It reviews theprinciples underlying the delineation of the elementaryaggregates themselves. Elementary aggregates are in-tended to be as homogeneous as possible, not merely interms of the physical and economic characteristics of theproducts covered but also in terms of their pricemovements.1.266 Chapter 9 then considers the consequences of

using alternative elementary index formulae to calculatethe elementary indices. It proceeds by means of a seriesof numerical examples that use simulated price data forfour different products within an elementary aggregate.The elementary indices themselves, and their properties,have already been explained above. An elementary priceindex may be calculated either as a chain index or as adirect index; that is, either by comparing the price eachmonth, or quarter, with that in the immediately pre-ceding period or with the price in the fixed price refer-ence period. Table 9.1 of Chapter 9 uses both approa-ches to illustrate the calculation of three basic types ofelementary index, Carli, Dutot and Jevons. It is designedto highlight a number of their properties. For example,it shows the effects of ‘‘price bouncing’’ in which thesame four prices are recorded for two consecutivemonths, but the prices are switched between the fourproducts. The Dutot and Jevons indices record noincrease but the Carli index registers an increase. Table9.1 also illustrates the differences between the direct andthe chain indices. After six months, each of the fourprices is 10 per cent higher than at the start. Each of thethree direct indices records a 10 per cent increase, as alsodo the chained Dutot and Jevons indices because theyare transitive. The chained Carli, however, records anincrease of 29 per cent, which is interpreted as illus-trating the systematic upward bias in the Carli formularesulting from its failure to satisfy the time reversal test.1.267 It is noted in Chapter 9 that the chaining and

direct approaches have different implications when thereare missing price observations, quality changes and re-placements. The conclusion is that the use of a chainindex can make the estimation of missing prices and theintroduction of replacement items easier from a com-putational point of view.1.268 Chapter 9 also examines the effects of missing

price observations, distinguishing between those that aretemporarily missing and those that have become per-manently unavailable. Table 9.2 contains a numericalexample of the treatment of the temporarily missingprices. One possibility is simply to omit the productwhose price is missing for one month from the calcula-tion of indices that compare that month with the pre-ceding and following months, and also with the baseperiod. Another possibility is to impute a price change onthe basis of the average price for the remaining products,using one or other of the three types of average. Theexample is a simplified version of the kind of examplesthat are used in Chapter 7 to deal with the same problem.

1.269 Tables 9.3 and 9.4 illustrate the case in whichone product disappears permanently to be replaced byanother product. In Table 9.3 there is no overlapbetween the two products and the options consideredare again to omit the products or to impute pricechanges for them based on averages for the other prod-ucts. Table 9.4 illustrates the situation in which theproducts overlap in one month.

1.270 Chapter 9 also considers the possibility thatthere may be some expenditure weights available withinan elementary aggregate, in which case it may be pos-sible to calculate a Laspeyres or a geometric Laspeyresindex, these being the weighted versions of the Carli andthe Jevons.

Higher-level indices1.271 Later sections of Chapter 9 illustrate the cal-

culation of the higher-level indices using the elementaryprice indices and the weights provided by the elementaryexpenditure aggregates. It is at this stage that the tra-ditional index number theory that was summarizedearlier in this chapter and is explained in detail inChapters 15 to 19 comes into play.

1.272 At the time the monthly CPI is first calculated,the only expenditure weights available must inevitablyrefer to some earlier period or periods of time. Asexplained earlier in this chapter, this predisposes the CPIto some form of Lowe or Young index in which thequantities, or expenditures, refer to some weight refer-ence period b which precedes the price reference period 0.Such indices are often loosely described as Laspeyrestype indices, but this description is inappropriate. Atsome later date, however, estimates may become avail-able of the expenditures in both the price referenceperiod 0 and the current period t, so that retrospectivelythe number of options open is greatly increased. It thenbecomes possible to calculate both Laspeyres andPaasche type indices, and also superlative indices such asFisher or Tornqvist. There is some interest in calculatingsuch indices later, if only to see how the original indicescompare with the superlative indices. Some countriesmay wish to calculate retrospective superlative indicesfor that reason. Although most of the discussion inChapter 9 focuses on some type of Lowe index becausethe official index first published will inevitably be of thattype, this should not be interpreted as implying that suchan index is the only possibility in the longer term.

1.273 Production and maintenance of higher-levelindices. In practice, the higher-level indices up to andincluding the overall CPI are usually calculated asYoung indices; that is, as weighted averages of the ele-mentary price indices using weights derived from expen-ditures in some earlier weight reference period. This is arelatively straightforward operation, and a numericalexample is given in Table 9.5 of Chapter 9 in which, forsimplicity, the weight and price reference periods areassumed to be the same. Table 9.6 illustrates the case inwhich weight and price reference periods are not thesame, and the weights are price updated between weightreference period b and the price reference period 0. Itillustrates the point that statistical offices have two

AN INTRODUCTION TO CONSUMER PRICE INDEX METHODOLOGY

31

options when a new price reference period is introduced:they can either preserve the relative quantities of theweight reference period or they can preserve the relativeexpenditures, but they cannot do both. Price updatingpreserves the quantities.

1.274 The introduction of new weights is a necessaryand integral part of the compilation of a CPI over thelong run. Weights have to be updated sooner or later,some countries preferring to update their weights eachyear. Whenever the weights are changed, the indexbased on the new weights has to be linked to the indexbased on the old weights. Thus, the CPI inevitablybecomes a chain index over the long term. An exampleof the linking is given in Table 9.7. Apart from thetechnicalities of the linking process, the introduction ofnew weights, especially if carried out at intervals of fiveyears or so, provides an opportunity to undertake amajor review of the whole methodology. New productsmay be introduced into the index, classifications may berevised and updated, while even the index number for-mula might be changed. Annual chaining facilitates theintroduction of new products and other changes on amore regular basis, but in any case some ongoingmaintenance of the index is needed whether it isannually chained or not.

1.275 Chapter 9 concludes with a section on dataediting, a process that is very closely linked to the actualcalculation of the elementary prices indices. Data editingcomprises two steps: the detection of possible errors andoutliers, and the verifying and correction of the data.Effective monitoring and quality control are needed toensure the reliability of the basic price data fed into thecalculation of the elementary prices indices, on whichthe quality of the overall index depends.

Organization and management1.276 The collection of price data is a complex

operation involving extensive fieldwork by a large num-ber of individual collectors. The whole process requirescareful planning and management to ensure that datacollected conform to the requirements laid down by thecentral office with overall responsibility for the CPI.Appropriate management procedures are described inChapter 12 of this manual.

1.277 Price collectors should be well trained toensure that they understand the importance of selectingthe right products for pricing. Inevitably, price collectorsare bound to use their own discretion to a considerableextent. As already explained, one issue of crucialimportance to the quality and reliability of a CPI is howto deal with the slowly evolving set of products withwhich a price collector is confronted. Products maydisappear and have to be replaced by others, but it mayalso be appropriate to drop some products before theydisappear altogether, if they have become unrepre-sentative. Price collectors need to be provided withappropriate training and very clear instructions anddocumentation about how to proceed. Clear instructions

are also needed to ensure that price collectors collect theright prices when there are sales, special offers or otherexceptional circumstances.

1.278 As just noted, the price data collected havealso to be subjected to careful checking and editing.Many checks can be carried out by computer, usingstandard statistical control methods. It may also beuseful to send out auditors to accompany price collectorsand monitor their work. The various possible checks andcontrols are explained in detail in Chapter 12.

1.279 Improvements in information technologyshould obviously be exploited to the fullest extent pos-sible. For example, collectors may use hand-held com-puters and transmit their results electronically to thecentral office.

Publication and dissemination1.280 As noted above and in Chapter 2, the CPI is

an extremely important statistic whose movements caninfluence the central bank’s monetary policy, affectstock markets, influence wage rates and social securitypayments, and so on. There must be public confidence inits reliability, and in the competence and integrity ofthose responsible for its compilation. The methods usedto compile it must therefore be fully documented, trans-parent and open to public scrutiny. Many countrieshave an official CPI advisory group consisting of bothexperts and users. The role of such a group is not just toadvise the statistical office on technical matters but alsoto promote public confidence in the index.

1.281 Users of the index also attach great impor-tance to having the index published as soon as possibleafter the end of each month or quarter, preferablywithin two or three weeks. There are also many userswho do not wish the index to be revised once it has beenpublished. Thus there is likely to be some trade-offbetween the timeliness and the quality of the index.

1.282 Publication should be understood to mean thedissemination of the results in any form. In addition topublication in print, or hard copy, the results should bereleased electronically and be available through theInternet on the web site of the statistical office.

1.283 As explained in Chapter 13, good publicationpolicy goes beyond timeliness, confidence and transpar-ency. The results should be made available to all users, inboth the public and the private sectors, at the same timeand according to a publication schedule announced inadvance. There should be no discrimination among usersin the timing of the release of the results. The resultsshould not be subject to governmental scrutiny as acondition for their release, and should be seen to be freefrom political or other pressures.

1.284 There are many decisions to be taken aboutthe degree of detail in the published data and the dif-ferent ways in which the results may be presented. Usersneed to be consulted about these questions. These issuesare discussed in Chapter 13. As they do not affect theactual calculation of the index, they need not be pursuedfurther at this point.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

32

2USES OF CONSUMER PRICE INDICES

2.1 The consumer price index (CPI) is treated as akey indicator of economic performance in most coun-tries. The purpose of this chapter is to explain why CPIsare compiled and what they are used for.

A range of possible consumerprice indices2.2 As noted in Chapter 1, compilers have to take

into account the needs of users in deciding on the groupof households and range of consumption goods andservices covered by a CPI. As the prices of differentgoods and services do not all change at the same rate, oreven all move in the same direction, changing the cov-erage of the index will change the value of the index.Thus, there can be no unique CPI and a range of pos-sible CPIs could be defined.2.3 While there may be interest in a CPI which is as

broadly defined as possible, covering all the goods andservices consumed by all households, there are manyother options for defining CPIs covering particular setsof goods and services, which may be more useful forparticular analytic or policy purposes. There is no ne-cessity to have only a single CPI. When only a single CPIis compiled and published, there is a risk that it may beused for purposes for which it is not appropriate. Morethan one CPI could be published in order to meet dif-ferent analytic or policy needs. It is important to recog-nize, however, that the publication of more than one CPIcan be confusing to users who view consumer inflationas a pervasive phenomenon affecting all householdsequally. The coexistence of alternative measures couldundermine their credibility for many users.2.4 This chapter is intended not only to describe the

most important uses for CPIs, but also to indicate howthe coverage of a CPI can be affected by the use forwhich it is intended. The question of what is the mostappropriate coverage of a CPI must be addressed beforeconsidering what is the most appropriate methodologyto be used. Whether or not the CPI is intended to be acost of living index (COLI), it is still necessary todetermine exactly what kinds of good and services andwhat types of households are meant to be covered. Thiscan only be decided on the basis of the main uses of theindex.

Indexation2.5 Indexation is a procedure whereby the monetary

values of certain payments, or stocks, are increased ordecreased in proportion to the change in the value of

some price index. Indexation is most commonly appliedto monetary flows such as wages, rents, interest or taxes,but it may also be applied to the capital values of certainmonetary assets and liabilities. Under conditions of highinflation, the use of indexation may become widespreadthroughout the economy.

2.6 The objective of indexation of money incomesmay be either to maintain the purchasing power of thoseincomes in respect of certain kinds of goods and ser-vices, or to preserve the standard of living or welfare ofthe recipients of the incomes. These two objectives arenot quite the same, especially over the longer term.Maintaining purchasing power may be interpreted aschanging money income in proportion to the change inthe monetary value of a fixed basket of goods and ser-vices purchased out of that income. As explained furtherbelow and in detail in Chapter 3, maintaining the pur-chasing power of income over a fixed set of goods andservices does not imply that the standard of living of therecipients of the income is necessarily unchanged.

2.7 When the indexation applies to monetary assetsor liabilities, it may be designed to preserve the realvalue of the asset or liability relative to other assets orrelative to the values of specified flows of goods andservices.

Indexation of wages2.8 As noted in Chapters 1 and 15, the indexation of

wages seems to have been the main objective behind theoriginal compilation of CPIs as the practice goes backover two centuries, although there has always beengeneral interest in measuring inflation. If the indexationof wages is the main justification for a CPI, it has directimplications for the coverage of the index. First, it sug-gests that the index should be confined to expendituresmade by households whose principal source of income iswages. Second, it may suggest excluding expenditures oncertain types of goods and services which are consideredto be luxurious or frivolous. If so, value judgementsor political judgements may enter into the selection ofgoods and services covered. This point is elaboratedfurther below.

Indexation of social security benefits2.9 It has become common practice in many coun-

tries to index-link the rates at which social securitybenefits are payable. There are many kinds of benefits,such as retirement pensions, unemployment benefits,sickness benefits, child allowances, and so on. As in thecase of wages, when index-linking to benefits of this kindis the main reason for compiling the CPI, it may suggest

33

restricting the coverage of the index to certain types ofhouseholds and goods and services. Many kinds of goodsand services may then be excluded by political decisionon the grounds that they are unnecessary or inappro-priate. This type of thinking may lead to pressure toexclude expenditures on items such as holidays, gam-bling, tobacco or alcoholic drink.

2.10 An alternative procedure is to compile separateCPIs for different categories of households. For exam-ple, an index may be compiled covering the basket ofgoods and services purchased by households whoseprincipal source of income is a social security pension.When this is done, it may be superfluous to decide toexclude certain types of luxury or inappropriate expen-ditures, as the actual expenditures on such items may benegligible anyway.

2.11 As already noted, publishing more than oneCPI may be confusing if inflation is viewed as affectingeveryone in the same way. Such confusion can be avoi-ded by suitable publicity; it is not difficult to explain thefact that price changes are not the same for differentcategories of expenditures. In practice, some countriesdo publish more than one index.

2.12 The main reason why it may not be justifiableto publish more than one index is that the movements inthe different indices may be virtually the same, especiallyin the short term. In such cases, the costs of compilingand publishing separate indices may not be worthwhile.In practice, it may need much bigger differences inpatterns of expenditure than actually exist between dif-ferent groups of households to yield significantly dif-ferent CPIs.

2.13 Finally, it should be noted that the deliberateexclusion of certain types of goods and services by po-litical decision on the grounds that the householdstowards whom the index is targeted ought not to bepurchasing such goods, or ought not to be compensatedfor increases in the prices of such goods, cannot berecommended because it exposes the index to politicalmanipulation. For example, suppose it is decided thatcertain products such as tobacco or alcoholic drinkshould be excluded from a CPI. There is then a possibilitythat when taxes on products have to be increased, theseproducts may be deliberately selected in the knowledgethat the resulting price increases do not increase the CPI.Such practices are not unknown.

The type of index used for indexation2.14 When income flows such as wages or social

security benefits are index-linked, it is necessary toconsider the implications of choosing between a cost ofliving index and a price index that measures the changesin the cost of purchasing a fixed basket of goods andservices, a type of index described here as a Lowe index.The widely used Laspeyres and Paasche indices areexamples of Lowe indices. The Laspeyres index uses atypical basket purchased in the earlier of the two periodscompared, while the Paasche uses a basket typical of thelater period. This ‘‘fixed basket’’ method has a longhistory, as explained in Chapter 15. In contrast, a cost ofliving index (COLI) compares the cost of two baskets

that may not be exactly the same but which bring thesame satisfaction or utility to the consumer.

2.15 Indexation using a Laspeyres price index willtend to over-compensate the income recipients forchanges in their cost of living. Increasing incomes inproportion to the change in the cost of purchasing abasket purchased in the past ensures that the incomerecipients have the opportunity to continue purchasingthat same basket if they wish to do so. They would thenbe at least as well off as before. However, by adjustingtheir pattern of expenditures to take account of changesin the relative prices of the goods and services they buy,they will be able to improve their standard of living orwelfare because they can substitute goods that havebecome relatively cheaper for ones that have becomerelatively dearer. In addition, they may be able to startto purchase completely new kinds of goods which pro-vide new kinds of benefits that were not available in theearlier period. Such new goods tend to lower a cost ofliving index when they first appear even though no pricecan actually be observed to fall, as there was no previousprice.

Indexation of interest, rents andother contractual payments

2.16 It is common for payments of both rents andinterest to be index-linked. Governments may issuebonds with an interest rate specifically linked to the CPI.The interest payable in any given period may be equal toa fixed real rate of interest plus the percentage increasein the CPI. Payments of housing rents may also belinked to the CPI or possibly to some other index, suchas an index of house prices.

2.17 Creditors receiving interest payments do notconsist only of households, of course. In any case, thepurpose of index-linking interest is not to maintain thestandard of living of the creditors but rather to maintaintheir real wealth by compensating them for the real hold-ing, or capital, losses on their loans incurred as a result ofgeneral inflation. A CPI may not be the ideal index forthis purpose but may be used by default in the absence ofany other convenient index, a point discussed furtherbelow.

2.18 Many other forms of contractual paymentsmay be linked to the CPI. For example, legal obligationsto pay alimony or for the support of children may belinked to the CPI. Payments of insurance premiums maybe linked either to the index as a whole or to a sub-indexrelating to some specific types of expenditures, such thecosts of repairs.

Taxation2.19 Movements in a CPI may be used to affect the

amounts payable in taxation in several ways. For exam-ple, liability for income tax may be affected by linkingpersonal allowances that are deductible from taxableincome to changes in the CPI. Under a system of pro-gressive taxation, the various thresholds at which higherrates of personal income tax become operative may bechanged in proportion to changes in the CPI. Liabilityfor capital gains tax may be reduced by basing it on real

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

34

rather than nominal capital gains through reducing thepercentage increase in the value of the asset by the per-centage change in the CPI over the same period, fortaxation purposes. In general, there are various ways inwhich some form of indexation may be introduced intotax legislation.

Real consumption and real income2.20 Price indices can be used to deflate expenditures

at current prices or money incomes in order to derivemeasures of real consumption and real income. Realmeasures involve volume comparisons over time (orspace). There are two different approaches to such com-parisons which are analogous to the distinction betweena Lowe, or basket, index and a cost of living index.2.21 The first defines the change in real consumption

as the change in the total value of the goods and servicesactually consumed measured at the fixed prices of somechosen period. This is equivalent to deflating the changein the current value of the goods and services consumedby an appropriately weighted Lowe price index. Thechange in real income can be measured by deflating thechange in total money income by the same price index.2.22 The alternative approach defines the change in

real consumption as the change in welfare derived fromthe goods and services actually consumed. This may beestimated by deflating the change in the current value ofconsumption by using a COLI. Real income may besimilarly obtained by deflating money income by thesame COLI.2.23 The two approaches cannot lead to the same

results if the pure price index and the COLI diverge. Thechoice between the two approaches to the measurementof real consumption and real income will not pursuedfurther here, as the issues involved are essentially thesame as those already considered above in the paralleldiscussion of the choice between a Lowe, or basket, priceindex and a cost of living index.

Consistency between price indices andexpenditure series2.24 The data collected on prices and the data col-

lected on household expenditures must be mutually con-sistent when measuring real consumption. This requiresthat both sets of data should cover the same set of goodsand services and use the same concepts and classifica-tions. Problems may arise in practice because the priceindices and the expenditure series are often compiledindependently of each other by different departments ofa statistical agency or even by different agencies.2.25 The coverage of a CPI need not be the same as

that of total household consumption expenditures in thenational accounts. The CPI may be targeted at selectedhouseholds and expenditures for reasons given above.However, the difference in coverage between the CPIand the national accounts expenditures must be pre-cisely identified so that it is possible to account for thedifferences between them. The price index used to deflatethe expenditures ought to cover the additional goodsand services not covered by the CPI. This may not be

easy to achieve in practice because the relevant pricedata may not be easily available if the price collectionprocedures are geared to the CPI. Moreover, even if allthe basic price data are available, the price index neededfor deflation purposes is likely to be of a different type orformula from the CPI itself.

2.26 In principle, the deflation of national accountsestimates will normally require the compilation ofappropriately defined price indices that differ from theCPI but may draw on the same price database. They maydiffer from the CPI not only in the range of the price andexpenditure data they cover and the weighting and indexnumber formula employed, but also in the frequencywith which they are compiled and the length of the timeperiods they cover. The movements of the resultingindices will tend to differ somewhat from the CPI pre-cisely because they measure different things. Althoughdesigned to be used to deflate expenditure data, they alsoprovide useful additional information about movementsin consumer prices. This information complements andsupplements that provided by the CPI. The CPI itself isnot designed to serve as a deflator. Its coverage andmethodology should be designed to meet the needs of theCPI as described in other sections of this chapter.

2.27 When other types of consumer price indices areneeded in addition to the CPI, this should be recognizedat the data collection stage as it may be more efficient andcost effective to use a single collection process to meet theneeds of more than one kind of price index. This mayimply collecting rather more price data than are neededfor the CPI itself if the coverage of the CPI has beendeliberately restricted in some way.

Purchasing power parities2.28 Many countries throughout the world, includ-

ing all the member countries of the European Union(EU), participate in regular international programmesenabling purchasing power parities (PPPs) to be calcu-lated for household consumption expenditures. Thecalculation of PPPs requires the prices of individual con-sumer goods and services to be compared directly be-tween different countries. In effect, PPP programmesinvolve the compilation of international consumer priceindices. Real expenditures and real incomes can then becompared between countries in much the same way asbetween different time periods in the same country.

2.29 It is not proposed to examine PPP methodol-ogy here but simply to note that PPPs create yet anotherdemand for basic price data. When such data are beingcollected, therefore, it is important to recognize thatthey can be used for PPPs as well as CPIs. PPPs areessentially international deflators which are analogousto the inter-temporal deflators needed for the nationalaccounts of a single country. Thus, while the processingand aggregation of the basic data for CPI purposesshould be determined by the needs of the CPI itself, it isappropriate to take account of the requirements of theseother kinds of price indices at the data collection stage.There may be important economies of scale to be re-alized by using a single collection process to meet theneeds of several different types of indices.

USES OF CONSUMER PRICE INDICES

35

2.30 Thus, operationally as well as conceptually, theCPI needs to be placed in the context of a wider set ofinterrelated price indices. The compilation of CPIs pre-dates the compilation of national accounts by manyyears in some countries, so the CPI may have originatedas a free-standing index. The CPI can, however, nolonger be treated as an isolated index whose compilationand methodology can proceed quite independently ofother interrelated statistics.

Use of the consumer price indexfor accounting under inflation

2.31 When there is inflation, both business andnational accounts have to introduce adjustments whichare not needed when the price level is stable. This is acomplex subject which cannot be pursued in any depthhere. Two methods of accounting are commonly used,and they are summarized below. Both require priceindices for their implementation.

Current purchasing power accounts2.32 Current purchasing power accounts are

accounts in which the monetary values of the flows inearlier time periods are scaled up in proportion to theincrease in some general index of inflation betweenthe earlier period and the current period. In principle,the index used should be a general price index coveringother flows in addition to household consumptionexpenditures, but in practice the CPI is often used bydefault in the absence of a suitable general index.

Current cost accounting2.33 Current cost accounting is a method of

accounting for the use of assets in which the cost of usingthe assets in production is calculated at the current pricesof those assets as distinct from the prices at which theassets were purchased or otherwise acquired in the past(historic costs). The current cost of using an asset takesaccount not only of changes in the general price level butalso of changes in the relative price of that type of assetsince it was acquired. In principle, the price indices thatare used to adjust the original prices paid for the assetsshould be specific price indices relating to that particulartype of asset, and such indices are calculated and used inthis way in some countries. However, when there are nosuch indices available there remains the possibility ofusing the CPI, or some sub-index of the CPI, by default,and CPIs have been used for this purpose.

Consumer price indices andgeneral inflation

2.34 As already noted, measures of the general rateof inflation in the economy as a whole are needed forvarious purposes:

� Controlling inflation is usually one of the mainobjectives of government economic policy, althoughresponsibility for controlling inflation may be dele-gated to the central bank. A measure of general infla-

tion is needed in order to set targets and also to judgethe degree of success achieved by the government orcentral bank in meeting anti-inflationary targets.

� As noted above, a measure of general inflation is alsoneeded for both business and national accountingpurposes, particularly for current purchasing poweraccounting.

� The concept of a relative price change is important ineconomics. It is convenient therefore to be able tomeasure the actual changes in the prices of individualgoods or services relative to some measure of generalinflation. There is also a need to be able to measurereal holding (or capital) gains and losses on assets,including monetary assets and liabilities.

2.35 Suitable measures of general inflation are con-sidered in Chapter 14, in which it shown that a hierarchyof price indices exists that includes the CPI. Clearly, aCPI is not a measure of general inflation, as it onlymeasures changes in the prices of consumer goods andservices purchased by households. A CPI does not covercapital goods, such as houses, or the goods and servicesconsumed by enterprises or the government. Any attemptto analyse inflationary pressures in the economy mustalso take account of other price movements, such aschanges in the prices of imports and exports, the prices ofindustrial inputs and outputs, and also asset prices.

Consumer price indices andinflation targets

2.36 Despite the obvious limitations of a CPI as ameasure of general inflation, it is commonly used bygovernments and central banks to set inflation targets.Similarly, it is interpreted by the press and the public asthe ultimate measure of inflation. Although govern-ments and central banks are obviously well aware of thefact that the CPI is not a measure of general inflation, anumber of factors help to explain the popularity of theCPI, and these are discussed below.

2.37 It may be noted, however, that even though theCPI does not measure general inflation its movementsmay be expected to be highly correlated with those ofa more general measure, if only because consumptionexpenditures account for a large proportion of total finalexpenditures. In particular, the CPI should provide areliable indicator of whether inflation is accelerating ordecelerating and also of any turning points in the rate ofinflation. This information is highly valuable even if theCPI may be systematically understating or overstatingthe general rate of inflation.

Consumer price indices andinternational comparisons of inflation

2.38 CPIs are also commonly used to make inter-national comparisons of inflation rates. An importantexample of their use for this purpose is provided by theEU. In order to judge the extent to which rates ofinflation in the different member countries were conver-ging in the mid-1990s prior to the formation of theEuropean Monetary Union, the member countries

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

36

decided in the Maastricht Treaty that CPIs should beused. Although CPIs measure consumer inflation ratherthan general inflation, their use to evaluate the extent ofconvergence of inflation may be justified on similargrounds to those just mentioned. Presumably, the con-vergence in CPIs will be highly correlated with that ingeneral inflation, so the use of a specific rather than ageneral measure of inflation may lead to the same con-clusions about the extent of convergence and whichcountries diverge the most from the average.

Popularity of consumer priceindices as economic statistics2.39 CPIs seem to have acquired a unique status

among economic statistics in most countries. There areseveral factors which help to explain this:

� First, all households have their own personal experi-ence of the phenomenon the CPI is supposed to bemeasuring. The general public are very conscious ofchanges in the prices of consumer goods and services,and the direct impact those changes have on theirliving standards. Interest in CPIs is not confined to thepress and politicians.

� Changes in the CPI tend to receive a lot of publicity.Their publication can make headline news. The CPI isa high-profile statistic.

� The CPI is published frequently, usually each month,so that the rate of consumer inflation can be closelymonitored. The CPI is also a timely statistic that isreleased very soon after the end of the period to whichit refers.

� The CPI is a statistic with a long history, as noted inChapters 1 and 15. People have been familiar with itfor a long time.

� Although price changes for certain kinds of consumergoods are difficult to measure because of qualitychanges, price changes for other kinds of goods andservices such as capital goods and government ser-vices, especially public services, tend to be even moredifficult to measure. The CPI may be a relativelyreliable price index compared with the price indicesfor some other flows.

� The CPI is widely respected. Its accuracy and reli-ability are seldom seriously questioned.

� Most countries have deliberately adopted a policy ofnot revising the index once it has been published. Thismakes it more attractive for many purposes, especiallythose with financial consequences such as indexation.The lack of revisions may perhaps create a somewhatspurious impression of certainty, but it also seems toenhance the credibility and acceptability of the index.

2.40 The widespread use of the CPI for more pur-poses than it is designed for can be explained by thevarious factors listed above, together with the fact thatno satisfactory alternative or more comprehensive mea-sures of inflation are available monthly in most coun-tries. For example, the CPI may be used as a proxy for amore general measure of inflation in business account-ing, even though it may be clear that, conceptually, theCPI is not the ideal index for the purpose. Similarly, thefact that the CPI is not subject to revision, together withits frequency and timeliness, may explain its popularityfor indexation purposes in business or legal contracts incontexts where it also may not be very appropriateconceptually. These practices may be defended on thegrounds that the alternative to using the CPI may be tomake no adjustment for inflation. Although the CPI maynot be the ideal measure, it is much better to use it thanto make no adjustment whatsoever.

2.41 Although the CPI is often used as a proxy for ageneral measure of inflation, this does not justify extend-ing its coverage to include elements that go beyondhousehold consumption. If broader indices of inflationare needed, they should be developed in addition to theCPI, leaving the CPI itself intact. Some countries are infact developing additional and more comprehensivemeasures of inflation within the kind of conceptualframework outlined in Chapter 14 below.

The need for independenceand integrity of consumerprice indices

2.42 Because of the widespread use of CPIs for allkinds of indexation, movements in the CPI can havemajor financial ramifications throughout the economy.The implications for the government alone can be con-siderable, given that the CPI can affect interest pay-ments and taxation receipts as well as the government’swage and social security outlays.

2.43 When financial interests are involved, there isalways a risk that both political and non-political pres-sure groups may try to exert an influence on the meth-odology used to compile the CPI. The CPI, in commonwith other official statistics, must be protected from suchpressures and be seen to be protected. Partly for thisreason, many countries establish an advisory committeeto ensure that the CPI is not subject to outside influence.The advisory committee may include representatives ofa cross-section of interested parties as well as indepen-dent experts able to offer professional advice. Informa-tion about the methodology used to calculate CPIsshould be publicly available.

USES OF CONSUMER PRICE INDICES

37

3CONCEPTS AND SCOPE

Introduction3.1 The purpose of this chapter is to define and

clarify the basic concepts of price and consumption usedin a consumer price index (CPI) and to explain the scopeof the index. While the general purpose of a consumerprice index is to measure changes in the prices of con-sumption goods and services, the concept of ‘‘consump-tion’’ is an imprecise one that can be interpreted inseveral different ways, each of which may lead to a dif-ferent CPI. The governmental agency or statistical officeresponsible for compiling a CPI also has to decide whe-ther the index is meant to cover all consumers, i.e., allhouseholds, or to be restricted to a particular group ofhouseholds. The precise scope of a CPI is inevitablyinfluenced by what is intended, or believed, to be themain use of the index. Statistical offices should, however,bear in mind that CPIs are widely used as measures ofgeneral inflation, even though they may not have beendesigned for this purpose.3.2 Consumption is an activity in which persons,

acting either individually or collectively, use goods orservices to satisfy their needs and wants. In economics,no attempt is made to observe and record such activitiesdirectly. Instead, consumption is measured either by thevalue of the goods and services wholly or partly used upin some period, or by the value of the goods and servicesthat are purchased, or otherwise acquired, for purposesof consumption.3.3 A consumer price index can have two different

meanings, as ‘‘consumer’’ may refer either to a type ofeconomic unit, typically a person or a household, or to acertain type of good or service. To avoid confusion, theterm ‘‘consumer’’ will, so far as possible, be reserved herefor persons or households, while so-called ‘‘consumer’’goods will be described as ‘‘consumption’’ goods. A con-sumption good or service is defined as one that members ofhouseholds use, directly or indirectly, to satisfy their ownpersonal needs and wants. By definition, consumptiongoods or services provide utility. Utility is simply thegeneric, technical term preferred by economists for thesatisfaction, benefit or welfare that people derive fromconsumption goods or services.3.4 A CPI is generally understood to be a price index

that measures changes in the prices of consumptiongoods and services acquired, or used, by households. Asexplained in Chapter 14, more broadly based priceindices can be defined whose scope extends well beyondconsumption goods and services, but a CPI is deliber-ately focused on household consumption. It is, however,possible to define a CPI that includes the prices of phy-sical assets such as land or dwellings purchased by

households. In the case of owner-occupied dwellings, akey issue is whether to include in the CPI the imputedrents for the flows of housing services provided by thedwellings, or alternatively whether to include the pricesof the dwellings themselves in the index (notwithstandingthe fact that they are treated as fixed assets and notconsumption goods in the system of national accounts(SNA)). Views differ on this issue. In any case, purchasesof financial assets, such as bonds or shares, are excludedbecause financial assets are not goods or services of anykind and are not used to satisfy the personal needs orwants of household members. Financial transactions donot change wealth as one type of financial asset is simplyexchanged for another type of financial asset. For exam-ple, when securities are purchased, money is exchangedfor a bond or share; or alternatively, when a debt isincurred, money is received in exchange for the creationof a liability.

3.5 Although, by definition, a CPI is confined to theprices of goods and services consumed by households, itdoes not necessarily follow that CPIs have to cover allhouseholds or all the goods and services they consume.For example, it might be decided to exclude publiclyprovided goods which households do not pay for. Manydecisions have to be taken about the precise scope of aCPI even though the general purpose of the index maybe determined. These issues are explored in this and thefollowing chapter.

Alternative consumptionaggregates

3.6 As already noted, the concept of consumption isnot a precise one and may be interpreted in differentways. In this section, a hierarchy of different consump-tion concepts and aggregates is examined.

3.7 Households may acquire goods and services forpurposes of consumption in four main ways:

– they may purchase them in monetary transactions;

– they may produce them themselves for their ownconsumption;

– they may receive them as payments in kind throughbarter transactions, particularly as remuneration inkind for work done;

– they may receive them as free gifts, or transfers, fromother economic units.

3.8 The broadest concept of consumption for CPIpurposes would be a price index embracing all fourcategories of consumption goods and services listedabove. This set of consumption goods and services may

39

be described as total acquisitions. Total acquisitions areequivalent to the total actual individual consumption ofhouseholds as defined in the SNA (see Chapter 14). Itshould be noted that total acquisitions constitute abroader concept of consumption than total consump-tion expenditures.

Acquisitions and expenditures3.9 Expenditures are made by the economic units

who pay for the goods and services: in other words, whobear the costs. However, many of the goods and servicesconsumed by households are financed and paid for bygovernment units or non-profit institutions. They aremostly services such as education, health, housing andtransport. Individual goods and services provided free ofcharge, or at nominal prices, to individual households bygovernments or non-profit institutions are described associal transfers in kind. They may make a substantialcontribution to the welfare or standard of living of theindividual households that receive them. (Social trans-fers in kind do not include collective services provided bygovernments to the community as whole, such as publicadministration and defence.)

3.10 The expenditures on social transfers in kindare incurred by the governments or non-profit institu-tions that pay for them and not by the households thatconsume them. It could be decided that the CPI shouldbe confined to consumption expenditures incurred byhouseholds, in which case free social transfers in kindwould be excluded from the scope of the index. Even ifthey were to be included, they can be ignored in practicewhen they are provided free, on the grounds that house-holds incur zero expenditures on them. Of course, theirprices are not zero from the perspective of the units thatfinance the social transfers, but the relevant prices for aCPI are those payable by the households.

3.11 Social transfers cannot be ignored, however,when governments and non-profit institutions decide tointroduce charges for them, a practice that has becomeincreasingly common in many countries. For example, ifthe CPI is intended to measure the change in the totalvalue of a basket of consumption goods and servicesthat includes social transfers, increases in their pricesfrom zero to some positive amount increase the cost ofthe basket and ought to be captured by a CPI.

Monetary versus non-monetaryexpenditures

3.12 A distinction may also be drawn betweenmonetary and non-monetary expenditures depending onthe nature of the resources used to pay for the goods andservices. A monetary expenditure occurs when a house-hold pays in cash, by cheque or credit card, or otherwiseincurs a financial liability to pay, in exchange for theacquisition of a good or service. Non-monetary expen-ditures occur when households do not incur a financialliability but bear the costs of acquiring the goods orservices in some other way.

3.13 Non-monetary expenditures. Payments may bemade in kind rather than cash, as in barter transactions.The goods and services offered as payment in barter

transactions are equivalent to negative expenditures andtheir price changes should, in principle, carry negativeweights in a CPI. If the price of goods sold increases, thehousehold is better off. However, as the two sides of abarter transaction should in principle be equal in value,the net expenditure incurred by two households engagedin barter should be zero. Barter transactions betweenhouseholds may therefore be ignored in practice for CPIpurposes.

3.14 Households also incur non-monetary expendi-tures when household members receive goods and ser-vices from their employers as remuneration in kind. Theemployees pay for the goods and services with their ownlabour rather than cash. Consumption goods and ser-vices received as remuneration in kind can, in principle,be included in a CPI using the estimated prices thatwould be payable for them on the market.

3.15 A third important category of non-monetaryexpenditure occurs when households consume goodsand services that they have produced themselves. Thehouseholds incur the costs, while the expenditures aredeemed to occur when the goods and services are con-sumed. Own account expenditures of this kind includeexpenditures on housing services produced for their ownconsumption by owner-occupiers. The treatment ofgoods and services produced for own consumption raisesimportant conceptual issues that are discussed in moredetail below.

3.16 Monetary expenditures. The narrowest conceptof consumption that could be used for CPI purposesis one based on monetary expenditures only. Such anaggregate would exclude many of the goods and servicesactually acquired and used by households for purposes ofconsumption. Only monetary expenditures generate themonetary prices needed for CPI purposes. The prices ofthe goods and services acquired through non-monetaryexpenditures can only be imputed on the basis of theprices observed in monetary transactions. Imputed pricesdo not generate more price information. Instead, theyaffect the weighting attached to monetary prices byincreasing the weight of those monetary prices which areused to value non-monetary expenditures.

3.17 If the main reason for compiling a CPI is themeasurement of inflation, it may be decided to restrictthe scope of the index to monetary expenditures only,especially since non-monetary expenditures do not gen-erate any demand for money. Harmonized Indices ofConsumer Prices (HICPs), used to measure inflationwithin the European Union, are confined to monetaryexpenditures (see Annex 1).

Acquisitions and uses3.18 It has been customary in the literature on CPIs

to draw a distinction between acquisitions of consump-tion goods and services by households and their sub-sequent use to satisfy their households’ needs or wants.Consumption goods are typically acquired at one pointof time and used at some other point of time, oftenmuch later, or they may be used repeatedly, or even con-tinuously, over an extended period of time. The timesof acquisition and use nevertheless coincide for many

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

40

services, although there are other kinds of services thatprovide lasting benefits and are not used up at the timethey are provided.3.19 The time at which a good is acquired is the

moment at which ownership of the good is transferredto the consumer. In a market situation, it is the momentat which the consumer incurs a liability to pay, either incash or in kind. The time at which a service is acquired isnot so easy to determine precisely as the provision of aservice does not involve any exchange of ownership.Instead, it typically leads to some improvement in thecondition of the consumer. A service is acquired by theconsumer at the same time that the producer provides itand the consumer accepts a liability to pay.3.20 In a market situation, therefore, the time of

acquisition for both goods and services is the time atwhich the liability to pay is incurred. When payments arenot made immediately in cash, there may be a significantlapse of time before the consumer’s bank account isdebited for a purchase settled by cheque, by credit cardor similar arrangement. The times at which these debitsare eventually made depend on administrative con-venience and on the particular financial and institutionalarrangements in place. They have no relevance to thetime of recording the transactions or the prices.3.21 The distinction between time of acquisition and

time of use is particularly important for durable goodsand certain kinds of services.

Durables and non-durables3.22 Goods. A ‘‘non-durable’’ good would be better

described as a single use good. For example, food anddrink are used once only to satisfy hunger or thirst.Heating oil, coal or firewood can be burnt once only, butthey are nevertheless extremely durable physically andcan be stored indefinitely. Households may hold sub-stantial stocks of so-called non-durables, such as manyfoodstuffs and fuel, especially in periods of political oreconomic uncertainty.3.23 Conversely, the distinguishing feature of con-

sumer durables, such as furniture, household equipmentor vehicles, is that they are durable under use. They canbe used repeatedly or continuously to satisfy consumers’needs over a long period of time, possibly many years.For this reason, a durable is often described as provid-ing a flow of ‘‘services’’’ to the consumer over the periodit is used (see also Box 14.3 of Chapter 14). There is aclose parallel between the definitions of consumer dur-ables and fixed assets. Fixed assets are goods that areused repeatedly or continuously over long periods oftime in processes of production: for example, buildingsor other structures, machinery and equipment. A list ofthe different kinds of consumer durables distinguished inthe Classification of Individual Consumption accordingto Purpose (COICOP) is given below. Some durableslast much longer than others, the less durable ones beingdescribed as ‘‘semi-durables’’ in COICOP, for exampleclothing. Dwellings are not classified as consumer dur-ables in COICOP. They are treated as fixed assets andnot consumption goods and therefore fall outsidethe scope of COICOP. However, the housing services

produced and consumed by owner-occupiers are inclu-ded in COICOP and classified in the same way as thehousing services consumed by tenants.

3.24 Services. Consumers may continue to benefit,and derive utility, from some services long after theywere provided because they bring about substantial,long-lasting or even permanent improvements in thecondition of the consumers. The quality of life of personsreceiving medical treatments such as hip replacements orcataract surgery, for example, is substantially and per-manently improved. Similarly, consumers of educationalservices can benefit from them over their entire lifetimes.

3.25 For some analytical purposes, it may beappropriate to treat certain kinds of services, such aseducation and health, as the service equivalents of dur-able goods. Expenditures on such services can be viewedas investments that augment the stock of human capital.Another characteristic that education and health ser-vices share with durable goods is that they are often soexpensive that their purchase has to be financed byborrowing or by running down other assets.

Consumer price indices based onacquisitions and uses

3.26 The distinction between the acquisition and theuse of a consumption good or service has led to twodifferent concepts of a CPI being proposed.

� A CPI may be intended to measure the average changebetween two time periods in the prices of the con-sumption goods and services acquired by households.

� Alternatively, a CPI may be intended to measure theaverage change between two time periods in the pricesof the consumption goods and services used by house-holds to satisfy their needs and wants.

3.27 Flows of acquisitions and uses may be verydifferent for durables. Acquisitions of durables, likeproducer capital goods, are liable to fluctuate, dependingon the general state of the economy, whereas the using upof the stock of durables owned by households tends be agradual and smooth process. A CPI based on the usesapproach requires that the index should measure period-to-period changes in the prices of the flows of servicesprovided by the durables. As explained in Chapter 23,the value of the flow of services from a durable may beestimated by its ‘‘user cost’’, which consists essentially ofthe depreciation on the asset (at current prices) plus theinterest cost. The inclusion of the interest cost as well asthe depreciation means that, over the long term, theweight given to durables is greater than when they aremeasured simply by acquisitions. In principle, the flowsof services, or benefits, derived from major educationaland medical expenditures might also be estimated on thebasis of user costs.

3.28 When durables are rented on the market, therentals have to cover not only the values of the serviceflows but additional costs such as administration andmanagement, repairs and maintenance, and overheads.For example, the amount payable to use a washingmachine in a launderette has to cover the costs of theroom space in which the machine is housed, electricity,

CONCEPTS AND SCOPE

41

repairs and maintenance, the wages of supervisory staff,and so on, as well as the services provided by themachine itself. Similarly, the rentals payable for car hiremay significantly exceed the cost of the service flowprovided by the car on its own. In both cases, the cus-tomer is buying a bundle of services that includes morethan just the use of the durable good.

3.29 Estimating the values and the prices of the flowsof services provided by the stock of durables owned byhouseholds is difficult, whereas expenditures on durablesare easily recorded, as are also the prices at which theyare purchased. Partly because of these practical mea-surement difficulties, CPIs have, up to now, been basedlargely or entirely on the acquisitions approach. Simi-larly, national accounts tend to record expenditures on,or acquisitions of, durables rather than the flows ofservices they provide. As already noted, dwellings aretreated as fixed assets and not consumer durables in theSNA. The treatment of owner-occupied housing is con-sidered separately below.

Basket indices andcost of living indices

3.30 A fundamental conceptual distinction may bedrawn between a basket index and a cost of living index.In a CPI context, a basket index is an index that mea-sures the change between two time periods in the totalexpenditure needed to purchase a given set, or basket, ofconsumption goods and services. It is called a ‘‘Loweindex’’ in this manual. A cost of living index (COLI) isan index that measures the change in the minimum costof maintaining a given standard of living. Both indicestherefore have very similar objectives in that they aim tomeasure the change in the total expenditure needed topurchase either the same basket or two baskets whosecomposition may differ somewhat but between whichthe consumer is indifferent.

Lowe indices3.31 CPIs are almost invariably calculated as Lowe

indices in practice. Their properties and behaviour aredescribed in detail in various chapters of this manual.The operational target for most CPIs is to measure thechange over time in the total value of some specifiedbasket of consumption goods and services purchased, oracquired, by some specified group of households in somespecified period of time. The meaning of such an index isclear. It is, of course, necessary to ensure that the selectedbasket is relevant to the needs of users and also kept upto date. The basket may be changed at regular intervalsand does not have to remain fixed over long periods oftime. The determination of the basket is considered inmore detail later in this chapter and in the following one.

Cost of living indices3.32 The economic approach to index number the-

ory treats the quantities consumed as being dependenton the prices. Households are treated as price takerswho are assumed to react to changes in relative pricesby adjusting the relative quantities they consume. A

basket index that works with a fixed set of quantitiesfails to allow for the fact that there is a systematic ten-dency for consumers to substitute items that havebecome relatively cheaper for those that have becomerelatively dearer. A cost of living index based on theeconomic approach does take this substitution effectinto account. It measures the change in the minimumexpenditure needed to maintain a given standard whenutility-maximizing consumers adjust their patterns ofpurchases in response to changes in relative prices. Incontrast to a basket index, the baskets in the two periodsin a cost of living index will generally not be quite thesame in the two periods because of these substitutions.

3.33 The properties and behaviour of cost of livingindices, or COLIs, are explained in some detail inChapter 17. A summary explanation has already beengiven in Chapter 1. The maximum scope of a COLI wouldbe the entire set of consumption goods and servicesconsumed by the designated households from whichthey derive utility. It includes the goods and servicesreceived free as social transfers in kind from govern-ments or non-profit institutions. Because COLIs mea-sure the change in the cost of maintaining a givenstandard of living or level of utility, they lend themselvesto a uses rather than an acquisitions approach, as utilityis derived not by acquiring a consumer good or servicebut by using it to satisfy personal needs and wants.

3.34 Welfare may be interpreted to mean not onlyeconomic welfare, that is the utility that is linked toeconomic activities such as production, consumptionand working, but also general well-being associated withother factors such as security from attack by others. Itmay not be possible to draw a clear distinction betweeneconomic and non-economic factors, but it is clear thattotal welfare is only partly dependent on the amount ofgoods and services consumed.

3.35 Conditional and unconditional cost of livingindices. In principle, the scope of a COLI is influencedby whether or not it is intended to be a conditional andunconditional cost of living index. The total welfare of ahousehold depends on a string of non-economic factorssuch as the climate, the state of the physical, social andpolitical environment, the risk of being attacked eitherby criminals or from abroad, the incidence of diseases,and so on, as well as by the quantities of goods andservices consumed. An unconditional cost of living indexmeasures the change in the cost to a household ofmaintaining a given level of total welfare allowing thenon-economic factors to vary as well as the prices ofconsumption goods and services. If changes in the non-economic factors lower welfare, then some compensat-ing increase in the level of consumption will be needed inorder to maintain the same level of total welfare. Anadverse change in the weather, for example, requiresmore fuel to be consumed to maintain the same level ofcomfort as before. The cost of the increased quantities offuel consumed drives up the unconditional cost of livingindex, irrespective of what has happened to prices. Thereare countless other events that can impact on an un-conditional cost of living index, from natural disasterssuch as earthquakes to man-made disasters such asChernobyl or acts of terrorism.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

42

3.36 While there may be interest in an uncondi-tional cost of living index for certain analytical andpolicy purposes, it is defined in such a way that it isdeliberately intended to measure the effects of manyother factors besides prices. If the objective is to mea-sure the effects of price changes only, the non-pricefactors must be held constant. Given that a cost ofliving index is meant to serve as a consumer price index,its scope must be restricted to exclude the effects ofevents other than price changes. A conditional cost ofliving index is defined as the ratio of the minimumexpenditures needed to maintain a given level of utility,or welfare, in response to price changes, assuming thatall the other factors affecting welfare remain constant.It is conditional not only on a particular standard ofliving and set of preferences, but also on a particularstate of the non-price factors affecting welfare. COLIsin this manual are to be understood as conditional costof living indices.3.37 A conditional COLI should not be viewed as

second best. An unconditional COLI is a more com-prehensive cost of living index than a conditional COLI,but it is not a more comprehensive price index than aconditional index. An unconditional index does notinclude more price information than a conditional indexand it does not give more insight into the impact of pricechanges on households’ welfare. On the contrary, theimpact of the price changes is diluted and obscured asmore variables impacting on welfare are included withinthe scope of the index.3.38 Lowe indices, including Laspeyres and Paasche,

are also conditional, being dependent on the choice ofbasket. The fact that the value of a basket index variesin predictable ways according to the choice of baskethas generated much of the large literature on indexnumber theory. Conceptually, Lowe indices and condi-tional COLIs have much in common. A Lowe indexmeasures the change in the cost of a specified basket ofgoods and services, whereas a conditional COLI mea-sures the change in the cost of maintaining the level ofutility associated with some specified basket of goodsand services, other things being equal.

Expenditures and other paymentsoutside the scope of consumerprice indices3.39 Given that, conceptually, most CPIs are

designed to measure changes in the prices of consump-tion goods and services, it follows that purchases of itemsthat are not goods and services fall outside the intendedscope of a CPI: for example, purchases of bonds, sharesor other financial assets. Similarly, payments that are noteven purchases because nothing is received in exchangefall outside the index: for example, payments of incometaxes or social security contributions.3.40 The implementation of these principles is not

always straightforward, as the distinction between anexpenditure on a good or service and other paymentsmay not always be clear cut in practice. A number ofconceptually difficult cases, including some borderline

cases of a possibly controversial nature, are examinedbelow.

Transfers3.41 The definition of a transfer is a transaction in

which one unit provides a good, service or asset toanother without receiving any good, service or asset inreturn: i.e., transactions in which there is no counter-part. Transfers are unrequited. As no good or service ofany kind is acquired by the household when it makes atransfer, the transfer must be outside the scope of a CPI.The problem is to determine whether or not certainkinds of transactions are in fact transfers, a problemcommon to both CPIs and national accounts.

3.42 Social security contributions and taxes on incomeand wealth. As households do not receive any specific,individual good or service in return for the payment ofsocial security contributions, they are treated as transfersthat are outside the scope of CPIs. Similarly, all pay-ments of taxes based on income or wealth (the ownershipof assets) are outside the scope of a CPI since theyare unrequited compulsory transfers to government.Property taxes on dwellings (commonly levied as localauthority taxes or rates) are outside the scope. It may benoted, however, that unrequited compulsory transferscould be incorporated within an unconditional COLI orwithin a more broadly defined conditional COLI thatallows for changes in some other factors besides changesin the prices of consumption goods and services.

3.43 Licences. Households have to pay to obtainvarious kinds of licences and it is often not clear whetherthey are simply taxes under another name or whetherthe government agency providing the licence providessome kind of service in exchange, for example by exer-cising some supervisory, regulatory or control function.In the latter case, they could be regarded as purchases ofservices. Some cases are so borderline that they havebeen debated for years by taxation experts under theaegis of the International Monetary Fund (IMF) andother international agencies without reaching consensus.The experts therefore agreed to settle on a number ofconventions based on practices followed in the majorityof countries. It is appropriate to make use of theseconventions for CPI, and also national accounts, pur-poses. These conventions are listed in the IMF’s Govern-ment Finance Statistics (IMF, 2001) and have also beenadopted in SNA 1993.

3.44 Payments by households for licences to own oruse certain goods or facilities are, by convention, classi-fied as consumption expenditures, not transfers, and arethus included within the scope of a CPI. For example,licence fees for radios, televisions, driving, firearms, andso on, as well as fees for passports, are included. On theother hand, licences for owning or using vehicles, boatsand aircraft, and for hunting, shooting and fishing areconventionally classified as direct taxes and are thereforeoutside the scope of CPIs. Many countries, however, doinclude taxes for private vehicle use as they regard themas taxes on consumption for CPI purposes. As the actualcircumstances under which licences are issued, and theconditions attaching to them, can vary significantly from

CONCEPTS AND SCOPE

43

country to country, statistical offices may wish to deviatefrom the proposed conventions in some instances. Ingeneral, however, it seems appropriate to make use ofconventions internationally agreed by the relevantexperts.

3.45 Gifts and subscriptions. Gifts are transfers, bydefinition, and thus outside the scope of a CPI. Pay-ments of subscriptions or donations to charitable orga-nizations for which no easily identifiable services arereceived in return are also transfers. On the other hand,payments of subscriptions to clubs and societies, includ-ing charities, which provide their members with somekind of service (e.g., regular meetings, magazines, etc.)can be regarded as final consumption expenditures andincluded in a CPI.

3.46 Tips and gratuities. Non-compulsory tips orgratuities are gifts that are outside the scope of a CPI.There may be cases, however, where, although tips arenot compulsory, it can be very difficult to obtain a goodor service without some form of additional payment,in which case this payment should be included in theexpenditure on, and the price of, the good or service inquestion.

Insurance3.47 There are two main types of insurance, life and

non-life. In both cases the premiums have two compo-nents. One is a payment for the insurance itself, oftendescribed as the net premium, while the other is animplicit service charge payable to the insurance enter-prise for arranging the insurance: i.e., a fee charged forcalculating the risks, determining the premiums, admin-istering the collection and investment of premiums, andthe payment of claims.

3.48 The implicit service charge is not directlyobservable. It is an integral part of the gross premiumthat is not separately identified in practice. As a pay-ment for a service it falls within the scope of a CPI, butit is difficult to estimate.

3.49 In the case of non-life insurance, the net pre-mium is essentially a transfer that goes into a pool cov-ering the collective risks of policy holders as a whole. Asa transfer, it falls outside the scope of a CPI. In the caseof life insurance, the net premium is essentially a form offinancial investment. It constitutes the purchase of afinancial asset, which is also outside the scope of a CPI.

3.50 Finally, it may be noted that when insurance isarranged through a broker or agent separate from theinsurance enterprise, the fees charged by the brokers oragents for their services are included within the scope ofthe CPI, over and above the implicit service chargesmade by the insurers.

Gambling3.51 The amounts paid for lottery tickets or placed in

bets also consist of two elements that are usually notseparately identified – the payment of an implicit servicecharge (part of consumption expenditures) and a currenttransfer that enters the pool out of which the winningsare paid. Only the implicit or explicit service chargespayable to the organizers of the gambling fall within the

scope of a CPI. The service charges are usually calculatedat an aggregate level as the difference between payables(stakes) and receivables (winnings).

Transactions in financial assets3.52 Financial assets are not consumption goods or

services. The creation of financial assets/liabilities, ortheir extinction, e.g., by lending, borrowing and repay-ments, are financial transactions that are quite differentfrom expenditures on goods and services and take placeindependently of them. The purchase of a financial assetis obviously not expenditure on consumption, being aform of financial investment.

3.53 Some financial assets, notably securities in theform of bills, bond and shares, are tradable and havemarket prices. They have their own separate price indi-ces, such as stock market price indices.

3.54 Many of the financial assets owned by house-holds are acquired indirectly through the medium ofpension schemes and life insurance. Excluding the servicecharges, pension contributions by households are similarto payments for life insurance premiums. They areessentially forms of investment made out of saving, andare thus excluded from CPIs. In contrast, the explicit orimplicit fees paid by households for the services renderedby financial auxiliaries such as brokers, banks, insurers(life and non-life), pension fund managers, financialadvisers, accountants, and so on, are within the scope ofa CPI. Payments of such fees are simply purchases ofservices.

Purchases and sales offoreign currency

3.55 Foreign currency is a financial asset. Purchasesand sales of foreign currency are therefore outside thescope of CPIs. Changes in the prices payable, or receiv-able, for foreign currencies resulting from changes inexchange rates are not included in CPIs. In contrast, theservice charges made by foreign-exchange dealers areincluded within the scope of CPIs when householdsacquire foreign currency for personal use. These chargesinclude not only explicit commission charges but alsothe margins between the buying or selling rates offeredby the dealers and the average of the two rates.

Payments, financing and credit3.56 Conceptually, the time at which an expenditure

is incurred is the time at which the purchaser incurs aliability to pay: that is, when the ownership of the goodchanges hands or the service is provided. The time ofpayment is the time at which the liability is extinguished.The two may be simultaneous when payment is madeimmediately in cash, i.e., notes or coin, but the use ofcheques, credit cards and other forms of credit facilitiesmeans that it is increasingly common for the payment totake place some time after the expenditure occurs. Afurther complication is that payments may be made instages, with a deposit payable in advance. Given thetime lags and complexity of financial instruments and

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

44

institutional arrangements, it may be difficult to deter-mine exactly when payment takes place. The time mayeven be different from the standpoint of the purchaserand the seller.3.57 For consistency with the expenditure data used

as weights in CPIs, the prices should be recorded at thetimes at which the expenditures actually take place. Thisis consistent with an acquisitions approach.

Financial transactions and borrowing3.58 Some individual expenditures may be very

large: for example, the purchase of expensive medicaltreatment, a large durable good, or an expensive holi-day. If the household does not have sufficient cash, ordoes not wish to pay the full amount immediately incash, various options are open.

� The purchaser may borrow from a bank, moneylenderor other financial institution.

� The purchaser may use a credit card.

� The seller may extend credit to the purchaser, or theseller may arrange for a third party, some kind offinancial institution, to extend credit to the purchaser.

The creation of a financial asset/liability3.59 When a consumer borrows to purchase a good

or service, two distinct transactions are involved: thepurchase of the good or service, and the borrowing ofthe requisite funds. The latter is a purely financial trans-action between a creditor and a debtor in which a newfinancial asset/liability is created. This financial trans-action is outside the scope of a CPI. As already noted, afinancial transaction does not change wealth and there isno consumption involved. A financial transaction merelyrearranges the individual’s asset portfolio by exchangingone type of asset for another. For example, when a loanis made, the lender exchanges cash for a financial claimover the debtor. Similarly, the borrower acquires cashcounterbalanced by the creation of an equal finan-cial liability. Such transactions are irrelevant for CPIpurposes.3.60 In general, when a household borrows from

financial institutions, including moneylenders, the bor-rowed funds may be used for a variety of purposesincluding the purchase of assets such as dwellings orfinancial assets (for example, bonds or shares), as well asthe purchase of expensive goods and services. Similarly,the credit extended to the holder of a credit card can beused for a variety of purposes. In itself, the creation of afinancial asset and liability by new borrowing has noimpact on a CPI. There is no good or service acquired,no expenditure and no price.3.61 It should be noted that interest payments are

not themselves financial transactions. The payment ofinterest is quite different from the borrowing, lending orother financial transactions that give rise to it. Interest isconsidered separately below.3.62 Hire purchase and mortgage loans must be

treated consistently with other loans. The fact that cer-tain loans are conditional on the borrower using thefunds for a particular purpose does not affect the

treatment of the loan itself. Moreover, conditional loansare by no means confined to the purchase of durablegoods on ‘‘hire purchase’’. Conditional personal loansmay be made for other purposes, such as large expen-ditures on education or health. In each case, the con-tracting of the loan is a separate transaction from theexpenditure on the good or service and must be dis-tinguished from the latter. The two transactions mayinvolve different parties and may take place at quitedifferent times.

3.63 Although the provision of finance is a separatetransaction from the purchase of a good or service forwhich it is used, it may affect the price paid. Each caseneeds to be carefully considered. For example, supposethe seller agrees to defer payment for one year. Theseller appears to make an interest free loan for a year,but this is not the economic reality. The seller makes aloan but it is not interest free. Nor is the amount lentequal to the ‘‘full’’ price. Implicitly, the purchaser issuesa short-term bill to the seller to be redeemed one yearlater and uses the cash received from the seller to pay forthe good. However, the present value of a bill at the timeit is issued is its redemption value discounted by oneyear’s interest. The amount payable by the purchaser atthe time the purchase of the good actually takes place isthe present discounted price of the bill and not the fullredemption price to be paid one year later. It is thisdiscounted price that should be recorded for CPI pur-poses. The difference between the discounted price andthe redemption price is, of course, the interest that thepurchaser implicitly pays on the bill over the course ofthe year. This way of recording corresponds to the wayin which bills and bonds are actually valued on financialmarkets and also to the way in which they are recordedin both business and economic accounts. Deferring pay-ments in the manner just described is equivalent to aprice reduction and should be recognized as such inCPIs. The implicit interest payment is not part of theprice. Instead, it reduces the price. This example showsthat in certain circumstances the market rate of interestcan affect the price payable, but it depends on the exactcircumstances of the credit arrangement agreed betweenthe seller and the purchaser. Each individual case needsto be carefully considered on its merits.

3.64 This case needs to be clearly distinguished fromhire purchase, considered in the next section, when thepurchaser actually pays the full price and borrows anamount equal to the full price while contracting to makeexplicit interest payments in addition to repaying theamount borrowed.

Hire purchase3.65 In the case of a durable good bought on hire

purchase, it is necessary to distinguish the de facto,or economic, ownership of the good from the legalownership. The time of acquisition is the time the hirepurchase contract is signed and the purchaser takespossession of the durable. From then onwards, it is thepurchaser who uses it and derives the benefit from its use.The purchasing household becomes the de facto ownerat the time the good is acquired, even though legal

CONCEPTS AND SCOPE

45

ownership may not pass to the household until the loan isfully repaid.

3.66 By convention, therefore, the purchasinghousehold is treated as buying the good at the timepossession is taken and paying the full amount in cash atthat point. At the same time, the purchaser borrows,either from the seller or some financial institution speci-fied by the seller, a sum sufficient to cover the purchaseprice and the subsequent interest payments. The differ-ence between the cash price and the sum total of all thepayments to be made is equal to the total interest pay-able. The relevant price for CPI purposes is the cash pricepayable at the time the purchase takes place, whether ornot the purchase is facilitated by some form of borrow-ing. The treatment of hire purchase is the same as that of‘‘financial leasing’’ whereby fixed assets, such as aircraft,used for purposes of production are purchased by afinancial institution and leased to the producer for mostor all of the service life of the asset. This is essentially amethod of financing the acquisition of an asset by meansof a loan and needs to be distinguished from operationalleasing such as hiring out cars for short periods of time.The treatment of hire purchase and financial leasingoutlined here is followed in both business and economicaccounting.

Interest payments3.67 The treatment of interest payments on the

various kinds of debt that households may have incur-red raises both conceptual and practical difficulties.Nominal interest is a composite payment covering fourmain elements whose mix may vary considerably:

� The first component is the pure interest charge: i.e.,the interest that would be charged if there were perfectcapital markets and perfect information.

� The second component is a risk premium thatdepends on the creditworthiness of the individualborrower. It can be regarded as a built-in insurancecharge under uncertainty against the risk of the debtordefaulting.

� The third component is a service charge incurredwhen households borrow from financial institutionsthat make a business of lending money.

� Finally, when there is inflation, the real value of a loanfixed in monetary terms (that is, its purchasing powerover real goods and services) declines with the rate ofinflation. However, creditors are able to offset the realholding, or capital, losses they expect to incur bycharging appropriately high rates of nominal interest.For this reason, nominal interest rates vary directlywith the rate of general inflation, a universally fam-iliar phenomenon under inflationary conditions. Inthese circumstances, the main component of nominalinterest may therefore be the built-in payment ofcompensation from the debtor to the creditor to offsetthe latter’s real holding loss. When there is very highinflation it may account for almost all of the nominalinterest charged.

3.68 The treatment of the first component, pureinterest, is somewhat controversial but this componentmay account for only a small part of the nominal

interest charged. The treatment of the second compo-nent, insurance against the risk of default, is alsosomewhat controversial.

3.69 The fourth component, the payment of com-pensation for the creditor’s real holding loss, is clearlyoutside the scope of a CPI. It is essentially a capitaltransaction. It may account for most of nominal interestunder inflationary conditions.

3.70 The third component constitutes the purchaseof a service from financial institutions whose business itis to make funds available to borrowers. It is known asthe implicit service charge and clearly falls within thescope of a CPI. It is included in COICOP. The servicecharge is not confined to loans made by ‘‘financialintermediaries’’, institutions that borrow funds in orderto lend them to others. Financial institutions that lendout of their own resources provide the same kind ofservices to borrowers as financial intermediaries. Whensellers lend out of their own funds, they are treated asimplicitly setting up their own financial institution thatoperates separately from their principal activity. Therates of interest of financial institutions also includeimplicit service charges. Because some capital marketstend to be very imperfect and most households may nothave access to proper capital markets, many lenders areeffectively monopolists who charge very high prices forthe services they provide, for example village money-lenders in many countries.

3.71 It is clear that interest payments should not betreated as if they were just pure interest or even pureinterest plus a risk premium. It is very difficult to dis-entangle the various components of interest. It may bepractically impossible to make realistic and reliableestimates of the implicit service charges embodied inmost interest payments. Moreover, for CPI purposes itis necessary to estimate not only the values of the servicecharges but changes in the prices of the services overtime. Given the complexity of interest flows and the factthat the different flows need to be treated differently,there seems to be little justification for including pay-ments of nominal interest in a CPI, especially in infla-tionary conditions.

Household production3.72 Households can engage in various kinds of

productive activities that may be either aimed at themarket or intended to produce goods or services for ownconsumption.

Business activities3.73 Households may engage in business or commer-

cial activities such as farming, retail trading, construc-tion, the provision of professional or financial services,and so on. Goods and services that are used up in theprocess of producing other goods and services for saleon the market constitute intermediate consumption. Theyare not part of the final consumption of households. Theprices of intermediate goods and services purchased byhouseholds are not to be included in CPIs. In practice, itis often difficult to draw a clear distinction between

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

46

intermediate and final consumption, as the same goodsmay be used for either purpose.

Consumption of own produce3.74 Households do not in fact consume directly all

of the goods and services they acquire for purposes ofconsumption. Instead, they use them as inputs into theproduction of other goods or services which are thenused to satisfy their needs and wants. There are numer-ous examples. For example, basic foodstuffs such asflour, cooking oils, raw meat and vegetables may beprocessed into bread, cakes or meals with the assistanceof other inputs including fuels, the services provided byconsumer durables, such as fridges and cookers, and thelabour services of members of the household. Inputs ofmaterials, equipment and labour are used to clean,maintain and repair dwellings. Inputs of seeds, fertilizers,insecticides, equipment and labour are used to producevegetables or flowers, and so on.3.75 Some of the production activities taking place

within households’ activities, for example gardening orcooking, may perhaps provide satisfaction in themselves.Others, such as cleaning, may be regarded as chores thatreduce utility. In any case, the goods or services used asinputs into these productive activities do not provideutility in themselves. Again, there are numerous exam-ples of such inputs: raw foodstuffs that are unsuitable foreating without being cooked; cleaning materials; fuelssuch as coal, gas, electricity or petrol; fertilizers; theservices of refrigerators and freezers; and so on.3.76 Utility is derived from consuming the outputs

from household production undertaken for own con-sumption. It is necessary, therefore, to decide whether aCPI should try to measure the changes in the prices ofthe outputs, rather than the inputs. In principle, it seemsdesirable to measure the output prices, but there areserious objections to this procedure.3.77 On a conceptual level, it is difficult to decide

what are the real final outputs from many of the morenebulous household production activities. It is particu-larly difficult to specify exactly what are the outputs fromimportant service activities carried out within house-holds, such as child care or care of the sick or elderly.Even if they could be satisfactorily identified, concep-tually they would have to be measured and priced. Thereare no prices to be observed, as there are no salestransactions. Prices would have to be imputed for themand such prices would be not only hypothetical butinevitably very speculative. Their use in CPIs is not arealistic possibility in general and almost certainly wouldnot be acceptable to most users who are primarilyinterested in the market prices paid by households.3.78 The practical alternative is to treat the goods

and services acquired by households on the market foruse as inputs into the various kinds of household pro-duction activities as if they were themselves final con-sumer goods and services. They provide utility indirectly,assuming that they are used exclusively to produce goodsand services that are directly consumed by households.This is the practical solution that is generally adopted notonly in CPIs but also in national accounts, where

household expenditures on such items are classified asfinal consumption. Although this seems a simple andconceptually acceptable solution to an otherwise intract-able problem, exceptions may be made for one or twokinds of household production that are particularlyimportant and whose outputs can readily be identified.

3.79 Subsistence agriculture. In the nationalaccounts, an attempt is made to record the value of theagricultural output produced for own consumption. Insome countries, subsistence agriculture may account fora large part of the production and consumption ofagricultural produce. The national accounts requiresuch outputs to be valued at their market prices. It isdoubtful whether it is appropriate to try to follow thisprocedure for CPI purposes.

3.80 A CPI may record either the actual inputprices or the imputed output prices, but not both. If theimputed output prices for subsistence agriculture areincluded in a CPI, the prices of the purchased inputsshould be excluded. This could remove from the indexmost of the market transactions made by such house-holds. Expenditures on inputs may constitute the prin-cipal contact that the households have with the marketand through which they experience the effects of infla-tion. It therefore seems preferable to record the actualprices of the inputs and not the imputed prices of theoutputs in CPIs.

3.81 Housing services produced for own consumption.The treatment of owner-occupied housing is difficult andsomewhat controversial. There may no consensus onwhat is best practice. This is discussed in several chaptersof this manual, especially in Chapters 10 and 23. Con-ceptually, the production of housing services for ownconsumption by owner-occupiers is no different fromother types of own account production taking placewithin households. The distinctive feature of the pro-duction of housing services for own consumption, ascompared with other kinds of household production, isthat it requires the use of an extremely large fixed asset inthe form of the dwelling itself. In economics, and alsonational accounting, a dwelling is usually regarded as afixed asset so that the purchase of a dwelling is classifiedas gross fixed capital formation and not as the acquisi-tion of a durable consumer good. Fixed assets are usedfor purposes of production, not consumption. The dwel-ling is not consumed directly. The dwelling provides astream of capital services that are consumed as inputsinto the production of housing services. This productionrequires other inputs, such as repairs, maintenance andinsurance. Households consume the housing servicesproduced as outputs from this production.

3.82 It is important to note that there are two quitedistinct service flows involved:

� One consists of the flow of capital services provided bythe dwelling which are consumed as inputs into theproduction of housing services.

� The other consists of the flow of housing servicesproduced as outputs which are consumed by membersof the household.

The two flows are not the same. The value of the outputflow will be greater than that of the input flow. The

CONCEPTS AND SCOPE

47

capital services are defined and measured in exactly thesame way as the capital services provided by other kindsof fixed assets, such as equipment or structures otherthan dwellings. As explained in detail in Chapter 23, thevalue of the capital services is equal to the user cost andconsists primarily of two elements, depreciation and theinterest, or capital, costs. Capital costs are incurredwhether or not the dwelling is purchased by borrowingon a mortgage. When the dwelling is purchased out ofown funds, the interest costs represent the opportunitycost of the capital tied up in the dwelling; that is, theforegone interest that could have been earned byinvesting elsewhere.

3.83 There are two main options for the own-account production and consumption of housing ser-vices in CPIs. One is to price the output of housingservices consumed. The other is to price the inputs,including the inputs of capital services. If housing ser-vices are to be treated consistently with other forms ofproduction for own consumption within households, theinput approach must be adopted. The production andconsumption of housing services by owner-occupiersmay, however, be considered to be so important as tomerit special treatment.

3.84 If it is decided to price the outputs, the pricesmay be estimated using the market rents payable onrented accommodation of the same type. This is des-cribed as the rental equivalence approach. One practicalproblem is that there may be no accommodation of thesame type that is rented on the market. For example,there may be no rental market for rural dwellings indeveloping countries where most of the housing mayactually be constructed by the households themselves.Another problem is to ensure that the market rents donot include other services, such as heating, that areadditional to the housing services proper. A furtherproblem is that market rents, like the rentals chargedwhen durables are leased, have to cover the operatingexpenses of the renting agencies as well as the costs of thehousing services themselves, and also provide someprofit to the owners. Finally, rented accommodation isinherently different from owner-occupied housing in thatit may provide the tenants with more flexibility andmobility. The transaction costs involved in moving housemay be much less for tenants.

3.85 In principle, if the output, or rental equiva-lence, approach is adopted then the prices of the inputsinto the production of housing services for own con-sumption, such as expenditures on repairs, maintenanceand insurance, should not be included as well. Other-wise, there would be double counting.

3.86 The alternative is to price the inputs into theproduction of housing services for own consumption inthe same way that other forms of production for ownconsumption within households are treated. In additionto intermediate expenditures such as repairs, main-tenance and insurance, the costs of the capital servicesmust be estimated and their prices included in the CPI.The technicalities of estimating the values of the flow ofcapital services are dealt with in Chapter 23. As in thecase of other types of production for own consumptionwithin households, it is not appropriate to include the

estimated costs of the labour services provided by theowners themselves.

3.87 Whether the input or the output approach isadopted, it is difficult to estimate the relevant prices. Thepractical difficulties experienced may sometimes be sogreat as to lead compilers and users to query the reli-ability of the results. There is also some reluctance to useimputed prices in CPIs, whether the prices refer to theinputs or the outputs. It has therefore been suggestedthat the attempt to measure the prices of housing serviceflows should be abandoned. Instead, it may be preferredto include the prices of the dwellings themselves in theCPI. In most cases these are observable market prices,although many dwellings, especially in rural areas indeveloping countries, are also built by their owners, inwhich case their prices still have to be estimated on thebasis of their costs of production.

3.88 Including the prices of dwellings in CPIsinvolves a significant change in the scope of the index. Adwelling is clearly an asset and its acquisition is capitalformation and not consumption. While the same argu-ment applies to durables, there is a substantial differenceof degree between a household durable and a dwelling, asreflected by the considerable differences in their pricesand their service lives. In principle, therefore, extendingthe scope of a CPI to include dwellings implies extendingthe scope of the index to include household gross fixedcapital formation.

3.89 The advantage of this solution is that it doesnot require estimates of either the input or output ser-vice flows, but conceptually it deviates significantly fromthe concept of a CPI as traditionally understood. In thecase of both consumer durables and dwellings, theoptions are either to record the acquisitions of the assetsin the CPIs at their market prices or to record the esti-mated prices of the service flows, but not both. Just asno service flows from durables are included in CPIs atpresent because their acquisitions are included, similarlyif the prices of dwellings are included in CPIs the serviceflows would have to be excluded. As explained inChapter 23, the acquisitions approach may give insuffi-cient weight to durables and dwellings over the long runbecause it does not take account of the capital costsincurred by the owners of the assets.

Coverage of households and outlets3.90 The group of households included in the scope

of a CPI is often referred to as the ‘‘reference house-holds’’, or the ‘‘reference population’’.

Definition of household3.91 For CPI purposes, households may be defined

in the same way as in population censuses. The follow-ing definition is recommended for use in populationcensuses (United Nations, 1998a):

A household is classified as either (a) a one personhousehold defined as an arrangement in which one personmakes provision for his or her food or other essentials forliving without combining with any other person to formpart of a multi-person household; or (b) a multi-person

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

48

household, defined as a group of two or more personsliving together who make common provision for food orother essentials for living. The persons in the group maypool their incomes and have a common budget to agreater or lesser extent; they may be related or unrelatedpersons or a combination of persons both related andunrelated.

3.92 This definition is essentially the same as thatused in household budget surveys and in the SNA. Thescope of a CPI is usually confined to private households,and excludes institutional households such as groups ofpersons living together indefinitely in religious institu-tions, residential hospitals, prisons or retirement homes.Nevertheless, convalescent homes, schools and colleges,the military, and so on are not treated as institutionalhouseholds; their members are treated as belonging totheir private households. The HICP coverage of house-holds, however, is consistent with the SNA 1993 defi-nition and thus includes institutional households.

Types of household3.93 In almost all countries, the CPI scope is

designed to include as many private households as pos-sible, and is not confined to those belonging to a specificsocio-economic group. The HICP regulations requirethat coverage should be of households independent oftheir income level.3.94 In some countries, however, extremely wealthy

households are excluded for various reasons. Theirexpenditures may be considered to be very atypical, whiletheir expenditure data, as collected in household budgetsurveys, may be unreliable. The response rates for wealthyhouseholds in household budget surveys are usually quitelow. In addition, it may be too costly to collect prices forsome of the consumer goods and services purchasedexclusively by the wealthy. Some countries may decide toexclude other kinds of households. For example, theUnited Kingdom CPI excludes not only the top 4 per centof households by income but also households mainlydependent on state pensions, with the net result thatroughly 15 per cent of households, and 15 per centof expenditure, is excluded. Japan and the Republic ofKorea exclude households mainly engaged in agriculture,forestry and fishing, and all one-person households. Suchexclusions affect the expenditure weights to the extentthat the patterns of expenditures of the excluded groupsdiffer from those of the rest of the population.3.95 In addition to a single wide-ranging official

(headline) CPI relevant to the country as a whole, manycountries publish a range of subsidiary indices relating tosub-sectors of the population. For example, the CzechRepublic compiles separate indices for:

– all households;

– all employees;

– employees with children;

– low-income employees;

– employees, incomplete families;

– pensioners;

– low-income pensioners;

– households in Prague;

– households in communities with populations of over5,000.

3.96 In India, CPI compilation originated from aneed to maintain the purchasing power of workers’incomes, and so four different CPIs are compiled at thenational level for reference households headed by thefollowing kinds of workers:

– agricultural labourers;

– industrial workers;

– rural labourers;

– urban non-manual employees.

Geographical coverage3.97 Urban and rural. Geographical coverage may

refer either to the geographical coverage of expendituresor the coverage of price collection. Ideally these twoshould coincide, whether the CPI is intended to be anational or a regional index. In most countries, prices arecollected in urban areas only since their movements areconsidered to be representative of the price movements inrural areas. In these cases national weights are applied andthe resulting index can be considered a national CPI. Ifprice movements in urban and rural areas are felt to besufficiently different – although price collection is restric-ted to urban areas because of resource constraints – thenurban weights should be applied and the resulting indexmust be considered as purely an urban and not a nationalCPI. For example, the following countries cover urbanhouseholds only (expenditure weights and prices): Aus-tralia, Mexico, Republic of Korea, Turkey, United States.Most other developed countries tend to use weights cov-ering urban and rural households, although in nearly everycase price collection takes place in urban areas only. Ofcourse, the borderline between urban and rural is inevi-tably arbitrary and may vary from country to country.For example, in France urban price collection is inter-preted to include villages with as few as 2,000 residents.

3.98 Decisions about geographical coverage in termsof urban versus rural coverage will depend on popula-tion distribution and the extent to which expenditurepatterns and the movements of prices tend to differbetween urban and rural areas.

3.99 Foreign purchases of residents and domesticpurchases of non-residents. Problems arise when house-holds make expenditures outside the boundaries of thearea or country in which they are resident. Decisionsabout the treatment of such expenditures depend on themain use of a CPI. For inflation analysis, it is the pricechange within a country which is of interest. An index ofinflation is needed that covers all so-called ‘‘domestic’’consumption expenditures that take place within thegeographical boundaries of the country, whether madeby residents or non-residents. HICPs (see Annex 1) aredefined in this way as indices of domestic inflation. Thusthey exclude consumption expenditures made by resi-dents when they are outside the country (which belongto the inflation indices of the countries where the pur-chases are made), and they include expenditures withinthe country made by residents of other countries. Inpractice, expenditures by visitors from abroad may be

CONCEPTS AND SCOPE

49

difficult to estimate, since household budget surveys donot cover non-resident households, although estimatesmight be possible for some commodities using retailsales data or special surveys of visitors. These issuesbecome more important when there is significant cross-border shopping as well as tourism.

3.100 When CPIs are used for escalating theincomes of residents, it may be appropriate to adopt theso-called ‘‘national’’ concept of expenditure whichcovers all the expenditures of residents, whether insideor outside the country, including remote purchase fromnon-resident outlets, for example by the Internet, tele-phone or mail. Household budget surveys can cover allthese types of expenditure, although it may be difficultto identify the country from which remote (Internet,mail, etc.) purchases are being made. The prices paid forairline tickets and package holidays bought within thedomestic territory should also be covered. It can bedifficult, however, to obtain price data for the goods andservices purchased by residents when abroad, althoughin some cases sub-indices of the partner countries’ CPIsmight be used.

3.101 Regional indices. When compiling regionalindices, the concept of residence applies to the region inwhich a household is resident. It is then possible to drawa distinction between the expenditures within a regionand the expenditures of the residents of that region,analogous to the distinction between the ‘‘domestic’’ and‘‘national’’ concepts of expenditure at the national level.The same issues arise for regional indices as were dis-cussed in paragraph 3.97. The principles applying tocross-border shopping between regions are the same asfor international cross-border shopping, but data avail-ability is generally different. If the scope of the regionalindex is defined to include the purchases by regionalresidents when in other regions (abroad), then, althoughprice data for the other regions should be readily avail-able, it is unlikely that expenditure data will be availablewith the necessary split between expenditure within andexpenditure outside the region of residence.

3.102 Care must be taken to treat cross-bordershopping in the same way in all regions. Otherwise doublecounting, or omission, of expenditures may occur whenregional data are aggregated. Where regional indices areaggregated to give a national index, the weights shouldbe based on regional expenditure data rather than onpopulation data alone.

3.103 Many countries try to satisfy the differingneeds of their many CPI users by deriving a family ofindices with differing coverage, headed by a single wide-ranging official (headline) CPI which is relevant to thecountry as a whole. In some large countries, regionalindices are more widely used than the national CPI,particularly where the indices are used for escalatingincomes. Thus, in addition to the headline CPI, whichhas the widest coverage possible, subsidiary indices arepublished which may relate to:

– sub-sectors of the population;

– geographical regions;

– specific commodity groups; sub-indices of the overall(official all-items) CPI should be published at as

detailed a level as possible, since many users are inter-ested in the price change of specific commodity groups.

3.104 In effect, many statistical offices are movingtowards a situation in which a database of prices andweights is maintained from which a variety of subsidiaryindices is derived.

Outlet coverage3.105 The coverage of outlets is dictated by the

purchasing behaviour of the reference households. Asalready stated, in principle, the prices relevant to CPIsare the prices paid by households. In practice, however, itis usually not feasible to collect price information directlyfrom households, although as more sales are madethrough electronic points of sale which record and printout both the items purchased and their prices, it maybecome increasingly practical to collect information onthe actual transaction prices paid by households. In themeantime, it is necessary to rely mainly on the prices atwhich products are offered for sale in retail shops orother outlets. All the outlets from which the referencepopulation makes purchases are within the scope of theCPI, and should be included in the sampling frame fromwhich the outlets are selected.

3.106 Examples of outlets are:

– retail shops – from very small permanent stalls tomultinational chains of stores;

– market stalls and street vendors;

– establishments providing household services –electricians, plumbers, window cleaners, and so on;

– leisure and entertainment providers;

– health and education services providers;

– mail or telephone order agencies;

– the Internet;

– public utilities;

– government agencies and departments.

3.107 The principles governing the selection of asample of outlets from which to collect prices are dis-cussed in some detail in Chapters 5 and 6.

Price variation3.108 Price variation occurs when exactly the same

good or service is sold at different prices at the samemoment of time. Different outlets may sell exactly thesame product at different prices, or the same productmay be sold from a single outlet to different categoriesof purchasers at different prices.

3.109 If markets were ‘‘perfect’’ in an economicsense, identical products would all sell at the same price.If more than one price were quoted, all purchases wouldbe made at the lowest price. This suggests that productssold at different prices cannot be identical but must bequalitatively different in some way. When the pricedifferences are, in fact, attributable to quality differ-ences, the price differences are only apparent, not genu-ine. In such cases, a change in the average price resultingfrom a shift in the pattern of quantities sold at differentprices would reflect a change in the average quality of

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

50

the products sold. This would affect the volume and notthe price index.3.110 If statistical offices do not have enough

information about the characteristics of goods and ser-vices selling at different prices, they have to decidewhether to assume that the observed price differences aregenuine or only apparent. The default procedure mostcommonly adopted in these circumstances is to assumethat the price differences are apparent. This assumptionis typically made for both CPI and national accountspurposes.3.111 However, markets are seldom perfect. One

reason for the co-existence of different prices for identicalproducts may be that the sellers are able to practise pricediscrimination. Another reason may simply be thatconsumers lack information and may buy at higher pri-ces out of ignorance. Also, markets may be temporarilyout of equilibrium as a result of shocks or the appearanceof new products. It must be recognized, therefore, thatgenuine price differences do occur.

Price discrimination3.112 Economic theory shows that price discrimi-

nation tends to increase profits. It may not be feasible topractise price discrimination for goods because they canbe retraded. Purchasers discriminated against would notbuy directly but would try to persuade those who couldpurchase at the lowest prices to buy on their behalf.Services, however, cannot be retraded, as no exchange ofownership takes place.3.113 Price discrimination appears to be extremely

common, almost the norm, for many kinds of servicesincluding health, education and transport. For example,senior citizens may be charged less than others forexactly the same kinds of health or transportation ser-vices. Universities may charge foreign students higherfees than domestic students. As it is also easy to vary thequalities of the services provided to different consumers,it can be difficult to determine to what extent observedprice differences are a result of quality differences orpure price discrimination. Sellers may even attach trivialor spurious differences in terms or conditions of sale tothe services sold to different categories of purchasers inorder to disguise the price discrimination.3.114 Price discrimination can cause problems with

regard to price indices. Suppose, for example, that aservice supplier discriminates by age by charging seniorcitizens aged 60 years or over price p2 and others pricep1, where p1>p2. Suppose, further, that the supplierthen decides to redefine senior citizens as those aged 70years or over while otherwise keeping prices unchanged.In this case, although neither p1 nor p2 changes, the pricepaid by individuals aged 60 to 70 years changes and theaverage price paid by all households increases.3.115 This example illustrates a point of principle.

Although neither of the stated prices, p1 and p2, at whichthe services are on offer changes, the prices paid bycertain households do change if they are obliged toswitch from p2 to p1. From the perspective of thehouseholds, price changes have occurred and a CPIshould, in principle, record a change. When prices are

collected from sellers and not from households, suchprice changes are unlikely to be recorded.

Price variation between outlets3.116 The existence of different prices in different

outlets raises similar issues. Pure price differences arealmost bound to occur when there are market imper-fections, if only because households are not perfectlyinformed. When new outlets open selling at lower pricesthan existing ones, there may be a time lag during whichexactly the same item sells at different prices in differentoutlets because of consumer ignorance or inertia.

3.117 Households may choose to switch their pur-chases from one outlet to another or even be obliged toswitch because the universe of outlets is continuallychanging, some outlets closing down while new outletsopen up. When households switch, the effect on the CPIdepends on whether the price differences are pure orapparent. When the price differences are genuine, aswitch between outlets changes the average prices paidby households. Such price changes ought to be capturedby CPIs. On the other hand, if the price differencesreflected quality differences, a switch would change theaverage quality of the products purchased, and henceaffect volume, not price.

3.118 Most of the prices collected for CPI purposesare offer prices and not the actual transactions pricespaid by households. In these circumstances, the effectsof switches in the pattern of households’ purchasesbetween outlets may remain unobserved in practice.When the price differences reflect quality differences, thefailure to detect such switches does not introduce anybias into the CPI. Buying at a lower price means buyinga lower-quality product, which does not affect the priceindex. However, when the price differences are genuine,the failure to detect switches will tend to introduce anupward bias in the index, assuming households tend toswitch towards outlets selling at lower prices. Thispotential bias is described as outlet substitution bias.

Outlet rotation3.119 A further complication is that, in practice,

prices are collected from only a sample of outlets and thesamples may change, either because outlets open andclose or because there is a deliberate rotation of thesample periodically. When the prices in the outlets newlyincluded in the sample are different from those in theprevious outlets, it is again necessary to decide whetherthe price differences are apparent or genuine. If they areassumed to be apparent, the difference between the pricerecorded previously in an old outlet and the new pricein the new outlet is not treated as a price change forCPI purposes, the difference being treated as attributableto quality difference. As explained in more detail inChapter 7, if this assumption is correct, the price changesrecorded in the new outlets can simply be linked to thosepreviously recorded in the old outlets without intro-ducing any bias into the index. The switch from the oldto the new outlets does not have any impact on the CPI.

3.120 If the price differences between the old and thenew outlets are deemed to be genuine, however, the

CONCEPTS AND SCOPE

51

simple linking just described can lead to bias. Whenhouseholds change the price they pay for a product bychanging outlets, the price changes should be capturedby the CPI. As explained in more detail in Chapter 7, itseems that most statistical offices tend to assume thatthe price differences are not genuine and simply linkthe new price series on to the old. Given that it is un-realistic to assume that markets are always perfect andthat pure price variation never occurs, this procedure,although widely used, is questionable and may lead toupward bias. Such bias is described as outlet rotationbias. One possible strategy that has been suggested isto assume that half of any observed price differencebetween old and new outlets is genuine and half is a resultof quality difference, on the grounds that, althoughinevitably somewhat arbitrary, it is likely to be closer tothe truth than assuming that the difference is eitherentirely genuine or entirely attributable to quality dif-ferences (see McCracken, Tobin et al., 1999).

Treatment of some specifichousehold expenditures

3.121 Some of the expenditures made by householdsmay not be on goods and services for household con-sumption and may therefore fall outside the scope of aCPI. One major category consists of the business expen-ditures made by households.

Fees of agents and brokers3.122 When a house is purchased for own use by an

owner-occupier, it can be argued that the transfer costsassociated with purchase (and sale) should be treated asconsumption expenditures in the same way as the bro-kers’ fees incurred when financial assets are bought orsold. The fees paid to an agent to buy or sell houses areincluded in many national CPIs, provided that the houseis to be occupied by the owner and not rented to a thirdparty.

Undesirable or illegal goods and services3.123 All the goods and services that households

willingly purchase in order to satisfy their personal needsor wants constitute consumers’ expenditures and there-fore fall within the scope of a CPI, irrespective of whe-ther their production, distribution or consumption isillegal or carried out in the underground economy or onthe black market. Particular kinds of goods or servicesmust not be excluded because they are considered to beundesirable, harmful or objectionable. Such exclusionscould be quite arbitrary and undermine the objectivityand credibility of the CPI:

� First, it should be noted that some goods and servicesmight be deemed to be undesirable at some times anddesirable at others, or vice versa. People’s attitudeschange as they acquire more information, especiallyas a result of scientific advances. Similarly, somegoods or services may be deemed to be undesirable insome countries but not in others at the same point of

time. The concept of an undesirable good is inher-ently subjective and somewhat arbitrary and volatile.

� Second, if it is accepted that some goods and servicesmay be excluded on the grounds that they are un-desirable, the index is thereby exposed to actual orattempted manipulation by pressure groups.

� Third, attempts to exclude certain goods or services bypressure groups may be based on a misunderstandingof the implications of so doing. For example, if the CPIis used for escalating incomes, it may be felt thathouseholds ought not be compensated for increases inthe prices of certain undesirable products. However,excluding them does not imply lowering the index. Apriori, excluding some item is just as likely to increasethe CPI as reduce it, depending on whether the priceincrease for the item in question is below or above theaverage for other goods and services. For example, if itis decided to exclude smoking from a CPI and the priceincrease for smoking products is below average,excluding smoking actually increases the income ofsmokers (just as it does for non-smokers).

3.124 While goods and services that householdswillingly choose to consume should not, in principle, beexcluded from a CPI because they are acquired in theunderground economy or even illegally, it may be impos-sible to obtain the requisite data on the expenditures orthe prices, especially on illegal goods and services. Theymay well be excluded in practice.

Luxury goods and services3.125 When a CPI is used as an index of general

inflation, it ought to include all households regardless oftheir socio-economic group and also all consumer goodsand services regardless of how expensive they are. Simi-larly, the scope of an index used for purposes of esca-lating incomes should include all the goods and servicespurchased by the reference households, irrespective ofwhether any of these goods and services are considered tobe luxuries or otherwise unnecessary or undesirable.

3.126 Of course, if the reference households areconfined to a select group of households, the index willeffectively exclude all those items that are purchasedexclusively by households that are not in the group. Forexample, excluding the wealthiest 5 per cent of house-holds will, in practice, exclude many luxury items fromthe scope of the index. As already noted, such house-holds may be excluded for various reasons, includingthe unreliability of their expenditure data and the factthat collecting prices for some items purchased exclu-sively by a tiny minority of households may not be cost-effective. Once the group of reference households hasbeen decided and defined, however, judgements shouldnot be made about whether to exclude certain of theirexpenditures that are considered to be non-essential oron luxuries.

Second-hand goods3.127 Markets for used or second-hand goods exist

for most durable goods. Household expenditures includeexpenditures on second-hand goods and are therefore

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

52

within the scope of a CPI. Households’ sales of durablesconstitute negative expenditures, however, so that theweights for second-hand goods are based on households’net expenditures: i.e., total purchases less sales. The totalexpenditure on a particular type of second-hand good isa function of the rate at which it is bought and sold, i.e., ahigher turnover rate (number of transactions) gives ahigher total expenditure. A higher turnover does not,however, increase the rate at which any individual goodcan be used for purposes of consumption or the flow ofservices that may be obtained from the good.3.128 Households may buy second-hand goods

through any of the following routes:

� Directly from another household – the selling house-hold will record the proceeds of the sale as receipts.Net expenditures, i.e., expenditures less receipts, arezero so no weight is attached to purchases and salesfrom one household to another.

� From another household via a dealer – in principle,households’ expenditures on the services of the dealersare given by the values of their margins (the differencebetween their buying and selling prices). These inter-mediation services should be included in CPIs. Theyshould be treated in the same way as the fees chargedby agents such as financial auxiliaries. The marginsmay be extremely difficult to estimate in practice. Careshould be taken to include trade-ins either as pur-chases by the dealers or receipts of households.

� Directly from another sector, i.e., from an enterprise orfrom abroad – the weight would be household pur-chases of the second-hand goods from other sectorsless sales to other sectors.

� From an enterprise or from abroad via a dealer – theappropriate weight is given by household purchasesfrom dealers less any household sales to dealers plusthe aggregate of dealers’ margins on the products thatthey buy from and resell to households. Trade-insshould count as part of sales by households (in thecase of cars, the weight given to new cars should notinclude any deduction for the value of trade-ins).

3.129 In some countries, many of the durables pur-chased by households, especially vehicles, may be importsof second-hand goods from other countries. The pricesand expenditures on these goods enter the CPI in thesame way as those for newly produced goods. Similarly,in some countries there may be significant net pur-chases of second-hand vehicles by households from thebusiness sector, these vehicles possibly carrying moreweight in the index than new vehicles purchased byhouseholds.

Imputed expenditures on goodsand services3.130 As explained in earlier sections, many of the

goods and services acquired and used by households forpurposes of their own final consumption are not pur-chased in monetary transactions but are acquiredthrough barter or as remuneration in kind or are pro-duced by households themselves. It is possible to esti-mate what households would have paid if they had

purchased these goods and services in monetary trans-actions or, alternatively, what it cost to produce them.In other words, values may be imputed for these non-monetary expenditures.

3.131 The extent to which it is desirable to includeimputed expenditures within the scope of a CPI dependspartly on the main purpose of the index. If the CPI isintended to be a measure of consumer inflation, it can beargued that only monetary expenditures should beincluded. Inflation is a monetary phenomenon measuredby changes in monetary prices recorded in monetarytransactions. Even when the main use of a CPI is forindexation purposes, it can be argued that it should onlyreflect changes in the monetary prices actually paid bythe reference population. Consistent with the objective ofmonitoring inflation in the European Union, the aim ofthe Harmonized Index of Consumer Prices (HICP)compiled by Eurostat is to measure inflation faced byconsumers. The concept of ‘‘household final monetaryconsumption expenditure’’ (HFMCE) used in the HICPdefines both the goods and services to be covered, andthe price concept to be used, i.e., prices net of reimburse-ments, subsidies and discounts. HFMCE refers only tomonetary transactions and includes neither consumptionof own production (e.g., agricultural goods or owner-occupied housing services) nor consumption of goodsand services received as income in kind.

3.132 When the CPI is intended to be a cost of livingindex, some imputed expenditures would normally beincluded within the scope of the CPI on the grounds thatthe goods and services acquired in non-monetary trans-actions affect households’ living standards. As alreadynoted, most countries include households’ imputedexpenditures on housing services produced by owner-occupiers but not imputed expenditures on goods such asagricultural goods produced for own consumption.

Price coverage3.133 A CPI should reflect the experience of the

consumers to whom it relates, and should thereforerecord what consumers actually pay for the goods andservices which are included in the scope of the index.The expenditures and prices recorded should be thosepaid by consumers, including any taxes on the products,and taking account of all discounts, subsidies and mostrebates, even if discriminatory or conditional. It may bevirtually impossible, however, to take account of alldiscounts and rebates in practice. Sensible practicalcompromises are needed, for which recommendationsand examples are given in Chapter 6.

3.134 When households pay the full market pricesfor products and are then subsequently reimbursed bygovernments or social security schemes for some of theamounts paid, CPIs should record the market prices lessthe amounts reimbursed. This kind of arrangement iscommon for educational and medical expenditures.

Taxes and subsidies3.135 All taxes on products, such as sales taxes,

excise taxes and value added tax (VAT), are part of

CONCEPTS AND SCOPE

53

the purchasers’ prices paid by consumers that should beused for CPI purposes. Similarly, subsidies should betaken into account, being treated as negative taxes onproducts.

3.136 For some analytical and policy purposes, itmay be useful to estimate a CPI that measures pricemovements excluding the effects of changes in taxesand subsidies. For monetary policy-makers, the priceincreases resulting from changes in indirect taxes orsubsidies are not part of an underlying inflationaryprocess but are attributable to their own manipulation ofthese economic levers. Similarly, when a CPI is used forescalation purposes, any increase in a CPI resulting fromincreases in indirect taxes leads to an increase in wagesand benefits linked to the CPI, despite the fact that theaim of the tax increase might have been to reduce con-sumers’ purchasing power. Alternatively, an increase insubsidies might be intended to stimulate consumption,but the resulting lower prices could be offset by a smallerincrease in indexed wages and benefits.

3.137 Net price indices. Net price indices may becompiled in which taxes on consumer goods or servicesare deducted from the purchasers’ prices, and subsidiesare added back on. Such indices do not, however,necessarily show how prices would have moved if therewere no taxes or changes in taxes. It is notoriously dif-ficult to estimate the true incidence of taxes on products:that is, the extent to which taxes or subsidies, or changestherein, are passed on to consumers. It is also difficult totake account of the secondary effects of changes in taxes.In order to estimate the secondary effects, input–outputanalysis can be used to work out the cumulative impactof taxes and subsidies through all the various stages ofproduction. For example, some of the taxes on vehiclefuel will enter the price of transport services which inturn will enter the prices of transported goods, some ofwhich will enter the prices paid for consumer goods byretailers and hence the prices which they charge to con-sumers. To track all these impacts would demand a muchmore detailed and up-to-date input–output table than isavailable in most countries. A more practicable alter-native is therefore simply to confine the taxes and sub-sidies for which correction is made to those levied at thefinal stage of sale at retail; that is, primarily to VAT, salesand excise taxes. Estimating prices less these taxes only,or corrected for changes in these taxes only, is morefeasible. In the case of a percentage sales tax or VAT, thecalculation is simple, but in the case of excise taxes, it isnecessary to ascertain the percentage mark-up by theretailer, since the excise tax will also be marked up by thispercentage.

Discounts, rebates, loyalty schemesand ‘‘free’’ products

3.138 CPIs should take into account the effects ofrebates, loyalty schemes, and money-off vouchers.Given that a CPI is meant to cover all the referencehouseholds, whether in the country as a whole or in aparticular region, discounts should be included even ifthey are available only to certain households or toconsumers satisfying certain payment criteria.

3.139 It may be difficult to record discriminatory orconditional discounts for practical reasons. When onlyone selected group of households can enjoy a certaindiscount on a specific product, the original stratumfor that product is split into two new strata, eachexperiencing different price changes and each requiring aweight. So, unless base period expenditures for allpossible strata are known, it is not possible to recorddiscriminatory discounts correctly. Similarly, with con-ditional discounts, e.g. discounts on utility bills forprompt payment, it can be difficult to record the effect ofthe introduction of such offers unless data are availableon the proportion of customers taking advantage ofthe offer. These kinds of practical problems also arisewhen there is price discrimination and the sellers changethe criteria that define the groups to whom differentprices are charged, thereby obliging some householdsto pay more or less than before without changing theprices themselves. These cases are discussed further inChapter 7.

3.140 Although it is desirable to record all pricechanges, it is also important to ensure that the qualitiesof the goods or services for which prices are collected donot change in the process. While discounted prices maybe collected during general sales seasons, care should betaken to ensure that the quality of the products beingpriced has not deteriorated.

3.141 The borderline between discounts and rebatescan be hazy and is perhaps best drawn according totiming. In other words, a discount takes effect at thetime of purchase, whereas a rebate takes effect some timelater. Under this classification, money-off vouchers arediscounts, and as with the conditional discounts men-tioned above, can only be taken into account in a CPIif they relate to a single product and if the take-up rateis known at the time of CPI compilation. Since this ishighly unlikely, the effect of money-off vouchers isusually excluded from a CPI. It should be noted that thediscount is recorded only when the voucher is used, notwhen the voucher is first made available to the con-sumer.

3.142 Rebates may be made in respect of a singleproduct, e.g. air miles, or may be more general, e.g.supermarket loyalty schemes where a $10 voucher isawarded for every $200 spent. As with discounts dis-cussed above, such rebates can only be recorded as pricefalls if they relate to single products and can be weightedaccording to take-up. Bonus products provided ‘‘free’’to the consumer, either by larger pack sizes or offerssuch as ‘‘two packs for the price of one’’, should betreated as price reductions, although they may beignored in practice when the offers are only temporaryand quickly reversed. When permanent changes to packsizes occur, quality adjustments should be made (seeChapter 7).

3.143 Given the practical difficulties in correctlyrecording all these types of price falls, it is usual toreflect discounts and rebates only if unconditional,whereas loyalty schemes, money-off coupons, and otherincentives are ignored. Discounts during seasonal salesmay be recorded provided that the quality of the goodsdoes not change.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

54

Classification3.144 The classification system upon which any CPI

is built provides the structure essential for many stagesof CPI compilation. Most obviously, it provides theweighting and aggregation structure, but it also providesthe scheme for stratification of products in the samplingframe, at least down to a certain level of detail, and itdictates the range of sub-indices available for publica-tion. Several factors must be taken into account when aCPI classification system is being developed.

� First, the classification must reflect economic reality.For example, it must be possible to accommodate newgoods and services in a manner that minimizes theneed for later restructuring of the higher-level cate-gories. Restructuring is undesirable because manyusers require long time series, and restructuring of theclassification will produce breaks in the series.

� Second, the needs of users for sub-indices should begiven a high priority when constructing aggregategroups, so that if, for example, some users are parti-cularly interested in price change in food products,then the classification should provide sufficient detailin that area.

� Third, it is a requirement of any classification that itscategories are unambiguously mutually exclusive, andat the same time provide complete coverage of allproducts considered to be within its scope. In practicethis means that it should be a straightforward task toassign any particular expenditure, or price, to a singlecategory of the classification system.

3.145 The availability and nature of the data them-selves will also affect the design of a classification system.The availability of expenditure and price data will dictatethe lowest level of detail that might be possible.Obviously it is not possible to produce a separateindex for a product for which either weights or pricesare not available. At the most detailed level, a highvariance of the price changes, or relatives, will suggestwhere additional categories are needed. In line withstandard sampling procedures, the stratification schemeshould minimize the within-stratum variance while at thesame time maximizing the between-stratum variance.The classification should reflect this requirement.

Criteria for classifyingconsumption expenditure3.146 Although a classification may be conceived

according to economic theory or user requirements usinga top-down approach, in practice the statistical compilercollects data about individual products and then aggre-gates them according to the classification scheme (abottom-up application). For example, the units of clas-sification for the Classification of Individual Consump-tion according to Purpose (COICOP) are expendituresfor the acquisition of consumer goods and services, notexpenditures on purposes as such. Divisions 01 to 12 ofCOICOP convert these basic statistics into a purposeclassification by grouping together the various goodsand services which are deemed to fulfil particular pur-

poses, such as nourishing the body, protecting it againstinclement weather, preventing and curing illness,acquiring knowledge, travelling from one place toanother, and so on.

3.147 Classifications of expenditure data are schemesfor aggregating expenditures on products according tocertain theoretical or user-defined criteria, such as:

� Product type – products may be aggregated by:

– physical characteristics of goods and the natureof services; for example, biscuits are divided intothose with and without a chocolate coat. This cri-terion can be meaningfully implemented down tothe most detailed level, and is the basis of theCentral Product Classification 1.0 (United Nations,1998b);

– economic activity from which the productoriginated. The International Standard IndustrialClassification of All Economic Activities (ISIC),Revision 3.1 (United Nations, 2002) is the inter-national standard classification;

– production process from which the product origi-nated;

– retail outlet type from which the product was pur-chased;

– geographical origin of the product.

� Purpose to which the products are put, e.g. to providefood, shelter, transport, etc. COICOP is the interna-tional standard.

� The economic environment, where products could beaggregated according to criteria such as:

– substitutability of products;

– complementarity of products;

– application of sales taxes, consumer subsidies,excise taxes, customs duties, etc.;

– imports from different countries (and in some cases,a classification of exportable products may be ofinterest).

Classification by product type3.148 Where indices of price change for specific

products groups are required, a product-based classifi-cation would be appropriate. Product classifications maycombine several of the criteria listed above; for example,the Classification of Products by Activity (CPA) in theEuropean Economic Community (Eurostat, 1993),which is linked to the CPC at the detailed level and theISIC at the aggregate level.

3.149 Inevitably, price collectors and index compil-ers will encounter products for which no detailed class orsub-class exists, for example, entirely new products, ormixed products which are bundles of existing products.This is a problem frequently encountered with hightechnology goods, telecommunications goods and ser-vices, and food items in the form of ‘‘ready meals’’.Initially, the expenditure on these products may berecorded in an ‘‘other’’ or n.e.c. (not elsewhere classified)class, but once expenditure on these products becomessignificant, a separate class should be created.

CONCEPTS AND SCOPE

55

Classification by purpose3.150 For a CPI compiler aiming to produce a

measure of the change in the cost of satisfying particularneeds, a purpose-based classification is appropriate. TheCOICOP breakdown at the highest level is by purposesuch that the 12 divisions of COICOP are categories ofpurpose, and below this level the groups and classes areproduct types. In other words, products are allocated topurpose headings. The allocation of products is com-plicated by the existence of multi-purpose products(single products that can be used for a variety of pur-poses), such as electricity, and mixed purpose (bundled)products, such as package holidays comprising trans-port, accommodation, meals, and so on.

3.151 Multi-purpose goods and services. The majorityof goods and services can be unambiguously assigned toa single purpose, but some goods and services couldplausibly be assigned to more than one purpose. Exam-ples include motor fuel, which may be used to powervehicles classified as transport as well as vehicles classi-fied as recreational, and snowmobiles and bicycles whichmay be bought for transport or for recreation.

3.152 In drawing up COICOP, the general rule fol-lowed has been to assign multi-purpose goods and ser-vices to the division that represents the predominantpurpose. Hence, motor fuel is shown under ‘‘Transport’’.Where the predominant purpose varies between coun-tries, multi-purpose items have been assigned to thedivision that represents the main purpose in the countrieswhere the item concerned is particularly important. As aresult, snowmobiles and bicycles are both assigned to‘‘Transport’’ because this is their usual function in theregions where most of these devices are purchased – thatis, North America and the Nordic countries in the case ofsnowmobiles, and Africa, South-East Asia, China andthe low countries of Northern Europe in the case ofbicycles.

3.153 Examples of other multi-purpose items inCOICOP include: food consumed outside the home,which is shown under ‘‘Hotels and restaurants’’, not‘‘Food and non-alcoholic beverages’’; camper vans,which are shown under ‘‘Recreation and culture’’, not‘‘Transport’’; and basket-ball shoes and other sportsfootwear suitable for everyday or leisure wear, which areshown under ‘‘Clothing and footwear’’, not ‘‘Recreationand culture’’.

3.154 National statisticians may wish to reclassifymulti-purpose items if they consider that an alternativepurpose is more appropriate in their country. Suchreclassifications should be footnoted.

3.155 Mixed purpose goods and services. Single out-lays may sometimes comprise a bundle of goods andservices which serve two or more different purposes. Forexample, the purchase of an all-inclusive package tourwill include payments for transport, accommodationand catering services, while the purchase of educationalservices may include payments for health care, trans-port, accommodation, board, educational materials, andso on.

3.156 Outlays covering two or more purposes aredealt with case by case with the aim of obtaining a

breakdown by purpose that is as precise as possible andconsistent with practical considerations of data avail-ability. Hence, purchases for package holidays are shownunder ‘‘Package holidays’’ with no attempt to isolateseparate purposes such as transport, accommodation andcatering. Payments for educational services, in contrast,should as far as possible be allocated to ‘‘Education’’,‘‘Health’’, ‘‘Transport’’, ‘‘Hotels and restaurants’’ and‘‘Recreation and culture’’.

3.157 Two other examples of mixed purpose itemsare: the purchase of in-patient hospital services whichinclude payments for medical treatment, accommoda-tion and catering; and the purchase of transport serviceswhich include meals and accommodation in the ticketprice. In both cases, there is no attempt to isolateseparate purposes. Purchases of in-patient hospital ser-vices are shown under ‘‘Hospital services’’ and pur-chases of transport services with accommodation andcatering are shown under ‘‘Transport services’’.

Classifications for consumerprice indices

3.158 In practice, most countries use a hybrid clas-sification system for their CPI in the sense that thebreakdown of expenditure at the highest level is bypurpose, with breakdowns by product at the lowerlevels. In some countries the higher-level purpose clas-sifications were developed many years ago for CPIs thatwere originally devised as measures of the changing costof a basket of goods and services that were, at the time,considered necessary for survival or maintaining some‘‘basic’’ standard of living. Thus, the classifications werebased on consumer needs, where ‘‘need’’ may have had asomewhat subjective interpretation depending on po-litical requirements.

3.159 The recommended practice today is still touse a purpose classification at the highest level, withproduct breakdowns below, but to use the recentlydeveloped international standard classifications as far aspossible, with adaptations to national requirementswhere necessary. In other words, divisions 01 to 12 ofCOICOP, with Central Product Classification (CPC)product classes and sub-classes mapped onto them toprovide the next two levels of detail.

Publication level3.160 As mentioned above, any restructuring of the

classification of published indices will inconvenienceusers and should be avoided so far as possible by carefulplanning and development of the classification schemein the first place. There is a trade-off between providingusers with as much detail as they would like in terms ofproduct indices and weights, and preserving some free-dom to restructure the lower levels (unpublished) with-out apparently affecting the published series.

3.161 Item samples below the level at which weightsare published can be revised between major weightrevisions. As explained in Chapter 9, new and replace-ment items and varieties can also be introduced pro-vided they can be included within an existing publishedweight. A major new product, such as a personal

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

56

computer, could only be introduced at the time of amajor weight revision, whereas it might be possible tointroduce mobile phones at any time if the lowest-levelweight published in the telecommunications category isfor telephone services.

Classification of IndividualConsumption according to Purpose(COICOP)3.162 COICOP structure. The international stan-

dard classification of individual consumption expendi-tures is the Classification of Individual Consumptionaccording to Purpose (COICOP). COICOP is a func-tional classification that is also used in SNA 1993 andcovers the individual consumption expenditures in-curred by three institutional sectors, i.e. households,non-profit institutions serving households (NPISHs),and general government. Individual consumptionexpenditures are those which benefit individual personsor households.3.163 COICOP has 14 divisions:

– divisions 01 to 12 covering the final consumptionexpenditure of households;

– division 13 covering the final consumption expendi-ture of NPISHs;

– division 14 covering the individual consumption ex-penditure of general government.

The classification has three levels of detail:

– division or two-digit level, e.g. 01. Food and non-alco-holic beverages;

– group or three-digit level, e.g. 01.1 Food;

– class or four-digit level, e.g. 01.1.1 Bread and cereals.

3.164 The 12 divisions covering households consistof 47 groups and 117 classes and are listed in Annex 2.Below the level of class, CPI compilers have to createadditional detail by further subdividing the classesaccording to their national needs. Of course, there areclear advantages, in terms of comparability betweencountries, and between the different uses of COICOP(CPIs, household expenditure statistics, national ac-counts aggregates), if the basic, higher-level structure ofCOICOP is maintained.3.165 There are some COICOP classes which may,

or may not, be included in most CPIs, or for whichexpenditure data cannot be collected directly fromhouseholds. For example, COICOP has a class for theimputed rentals of owner-occupiers, which may beoutside the scope of some CPIs. COICOP also has aclass for financial intermediation services indirectlymeasured, which may be outside the scope of some CPIsbecause of practical measurement difficulties. In anycase, the expenditures on these services cannot be col-lected in household budget surveys. Similarly, COICOPhas a group for expenditure on insurance service

charges, which may be within the scope of CPIs butcannot be measured using household surveys.

3.166 Type of product. COICOP classes are dividedinto: services (S), non-durables (ND), semi-durables(SD) and durables (D). This supplementary classificationprovides for other analytical applications. For example,an estimate may be required of the stock of consumerdurables held by households, in which case the goods inCOICOP classes that are identified as ‘‘durables’’ pro-vide the basic elements for such estimates.

3.167 As explained above, the distinction betweennon-durable goods and durable goods is based on whe-ther the goods can be used only once or whether they canbe used repeatedly or continuously over a period ofconsiderably more than one year. Moreover, durables,such as motor cars, refrigerators, washing machines andtelevisions, have a relatively high purchasers’ value.Semi-durable goods differ from durable goods in thattheir expected lifetime of use, though more than oneyear, is often significantly shorter and their purchasers’value is substantially less. Because of the importanceattached to durables, the categories of goods defined asdurables in COICOP are listed below:

– furniture, furnishings, carpets and other floor coverings;

– major household appliances;

– tools and equipment for house and garden;

– therapeutic appliances and equipment;

– vehicles;

– telephone and fax equipment;

– audiovisual, photographic and information pro-cessing equipment (except recording media);

– major durables for recreation;

– electrical appliances for personal care;

– jewellery, clocks and watches.

The following goods are listed as semi-durables:

– clothing and footwear;

– household textiles;

– small electrical household appliances;

– glassware, tableware and household utensils;

– spare parts for vehicles;

– recording media;

– games, toys, hobbies, equipment for sport, camping,etc.

3.168 Some COICOP classes contain both goodsand services because it is difficult for practical reasons tobreak them down into goods and services. Such classesare usually assigned an (S) when the service componentis considered to be predominant. Similarly, there areclasses which contain either both non-durable and semi-durable goods or both semi-durable and durable goods.Again, such classes are assigned a (ND), (SD) or (D)according to which type of good is considered to be themost important.

CONCEPTS AND SCOPE

57

Appendix 3.1 Consumer priceindices and national accountsprice deflators

1. The purpose of this appendix is to explain why and howconsumer price indices (CPIs) differ from the price indices usedto deflate household consumption expenditures in nationalaccounts. The differences between the two kinds of price indexare often not well understood.

Coverage of households2. The sets of households covered by CPIs and the national

accounts are not intended to be the same, CPIs typically cov-ering a smaller set of households. Household consumptionexpenditures in national accounts cover the expenditures madeby all households, including institutional households residentin the country or region, whether those expenditures are madeinside or outside the country or region of residence. CPIs tendto cover the expenditures and prices paid by households withinthe geographical boundaries of a country or region, whether thehouseholds are residents or visitors. More importantly, mostCPIs are purposely defined to cover only selected groups ofnon-residential households. For example, CPIs may excludevery wealthy households or be confined to households in urbanareas or headed by wage-earners.

Coverage of consumption expenditures3. The sets of expenditures covered by CPIs and national

accounts are not intended to be the same, CPIs typicallycovering a smaller set of expenditures. Most CPIs do not covermost of the imputed non-monetary consumption expendituresincluded in national accounts, either on principle or in practicebecause of lack of data. Many CPIs include the imputed rentson owner-occupied housing, but CPIs are not intended tocover the imputed expenditures and prices of agricultural

products or other goods produced for own consumption thatare included in national accounts.

Timing4. Most CPIs measure price changes between two points of

time or very short intervals of time such as a week. The priceindices in national accounts are intended to deflate expendi-tures aggregated over long periods of time, generally a year.The ways in which monthly or quarterly CPIs are averaged toobtain annual CPI indices are unlikely to be conceptuallyconsistent with the annual price indices in national accounts.

Index number formulae5. The index number formulae used by CPIs and national

accounts are not intended to be the same. In practice, mostCPIs tend to use some kind of Lowe price index that uses thequantities of an earlier period, whereas the price indices, orprice deflators, in national accounts are usually meant to bePaasche indices. Paasche indices are used in order to obtainLaspeyres volume indices. These differences, arising from theuse of different index formulae, would tend to be reduced ifboth CPIs and national accounts adopted annual chaining.

Conclusions6. It is clear that, in general, CPIs and the price deflators for

national accounts can differ for a variety of reasons, such asmajor differences in the coverage of households and expendi-tures, differences in timing and differences in the underlyingindex number formulae. These differences are intentional andjustified. Of course, the price data collected for CPI purposesmay also be used to build up the detailed price deflators usedfor national accounts purposes, but at an aggregate level CPIsand national accounts deflators may be quite different for thereasons just given.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

58

4EXPENDITURE WEIGHTS AND THEIR SOURCES

Introduction4.1 A consumer price index (CPI) is usually calcu-

lated as a weighted average of the price changes for theconsumption goods and services covered by the index.The weights are meant to reflect the relative importanceof the goods and services as measured by their sharesin the total consumption of households. The weightattached to each good or service determines the impactthat its price change will have on the overall index. Theweights should be made publicly available in the interestsof transparency, and for the information of the users ofthe index.4.2 The weights depend on the scope of the index

which, in turn, depends on the main use, or uses, for theindex. The uses and scope of a CPI have already beenexplained in some detail in the two previous chapters.This chapter therefore focuses on the derivation andcompilation of the weights and the data sources thatmay be used to estimate them. In practice, the weightsusually refer to expenditures on consumption goods andservices by households, as distinct from the actual use ofthose goods and services to satisfy the needs and wantsof households. Expenditure-based weights are appro-priate for a CPI based on the acquisitions approach. Thedifference between the acquisitions and uses approach toCPIs was explained in the previous chapter.4.3 In the special case of owner-occupied housing,

however, many countries adopt the uses rather than theacquisitions approach. They measure changes in theprices of the flows of housing services consumed byhouseholds as distinct from changes in the prices ofdwellings. It is shown in Chapter 23 of this manual thatone important consequence of adopting the usesapproach to owner-occupied housing is that its weight inthe overall CPI is considerably greater than when theacquisitions approach is used. The reason is that thevalues of the housing services consumed by owner-occupiers have to cover not only the depreciation on thehouses purchased but also the interest costs on thecapital invested in the dwellings. Over a period of years,the uses approach may well give twice as much weight toowner-occupied housing as the acquisitions approach.Reference may be made to Chapter 23 for further detailsand explanation.

The weighting structure of theconsumer price index4.4 As explained in more detail in Chapters 7 and 9,

the calculation of a CPI usually proceeds in two stages.In the first stage, elementary indices are estimated for

each of the elementary aggregates. Elementary indicesare constructed by (a) collecting a sample of repre-sentative prices for each elementary aggregate, and then(b) calculating an average price change for the sample.In the second stage, a weighted average is taken of theelementary indices using the expenditures within theelementary aggregates as weights.

4.5 Elementary aggregates are usually the smallestgroups of goods and services for which expenditure dataare available to be used as weights. They may cover thewhole country or separate regions within the country.Likewise, elementary aggregates may be distinguishedfor different types of outlets. The nature of the ele-mentary aggregates depends on circumstances and theavailability of expenditure data. Elementary aggregatesmay therefore be defined differently in different coun-tries. In general:

� Elementary aggregates should consist of groups ofgoods or services that are as similar as possible.

� They should also consist of goods or services that maybe expected to have similar price movements. Theobjective is to minimize the dispersion of price move-ments within the aggregate.

� The elementary aggregates should be appropriate toserve as strata for sampling purposes in the light ofthe sampling regime planned for the data collection.

4.6 The aggregation structure for a CPI is illustratedin Figure 4.1 using the Classification of IndividualConsumption according to Purpose (COICOP) des-cribed in Chapter 3, although similar national classifi-cations may be used instead:

� First, the entire set of consumption goods and servicescovered by the overall CPI is divided into divisions,such as ‘‘food and non-alcoholic beverages’’.

� Each division is then divided into groups, such as‘‘food’’.

� Each group is further divided into classes, such as‘‘bread and cereals’’.

� Each class may be divided into more homogeneoussub-classes, such as ‘‘rice’’.

� Finally, a sub-class may be further subdivided toobtain the elementary aggregates, by dividing accord-ing to region or type of outlet, as illustrated in Figure4.1. In some cases, a particular sub-class cannot be, ordoes not need to be, further subdivided, in which casethe sub-class becomes the elementary aggregate.

The sub-classes and elementary aggregates are not partof COICOP itself but more detailed breakdowns ofCOICOP classes that are needed for CPI purposes.

59

4.7 Within each elementary aggregate, one or moreproducts are selected to represent the price movementsof all the goods and services in the elementary aggregate.For example, the elementary aggregate consisting of ricesold in supermarkets in the northern region covers alltypes of rice, from which parboiled white rice and brownrice with over 50 per cent broken grains are selected asrepresentative products. Of course, more representativeproducts might be selected in practice. Finally, for eachkind of representative product, a number of individualproducts can be selected for price collection, such as

particular brands of parboiled rice. Again, the numberof sampled products selected may vary depending on thenature of the representative product.

4.8 The methods used to calculate the elementaryprice indices from the individual price observations col-lected within each elementary aggregate are explainedin Chapter 9, and are not of immediate concern here.Working upwards from the elementary price indices, allindices above the elementary aggregate level are des-cribed as higher-level indices that can be calculated fromthe elementary price indices using the elementary

OVERALL CPIproducts

GROUPFood and non-

alcoholic beverages GROUP

Alcoholic beverages and tobacco OTHER GROUPS

CLASSBread and cereals

CLASSMeat OTHER CLASSES

SUB-CLASSRice

SUB-CLASSBread

OTHER SUB-CLASSES

Sold in northernregion Sold in southern region Sold in other

regions

ELEMENTARYAGGREGATE

Rice sold in northernsupermarkets

ELEMENTARY AGGREGATE

Rice sold in other northernoutlets

REPRESENTATIVEPRODUCT

Parboiled long-grainwhite rice

REPRESENTATIVE PRODUCT

Brown rice: over 50 per centbroken rice

SAMPLED PRODUCT

Brand A

SAMPLED PRODUCTBrand B

Figure 4.1 Typical aggregation structure of a consumer price index (CPI)

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

60

expenditure aggregates as weights. The aggregationstructure is consistent, so that the weight at each levelabove the elementary aggregate is always equal to thesum of its components. The price index at each higherlevel of aggregation can be calculated on the basis of theweights and price indices for its components, that is, thelower-level or elementary price indices. The individualelementary price indices are not necessarily sufficientlyreliable to be published separately, but they remain thebasic building blocks of all higher-level indices. Abovethe level of the elementary aggregate, therefore, no newinformation is introduced into the calculation of the CPI.

Group, class and sub-class weights4.9 The weights for the groups, classes and sub-

classes are their shares in the total consumption expen-ditures of the reference population. They are most oftenderived from household expenditure surveys (HESs).These surveys are also described as household budgetsurveys (HBSs). As these surveys are sample surveyssubject to reporting and non-response errors as well assampling errors, the estimated shares for certain sub-classes are often modified or revised on the basis ofsupplementary or additional information from othersources.

Regional weights4.10 Within a given sub-class, the regional weight

shows the consumption expenditure in the region inproportion to the expenditure in the whole country forthat sub-class. For example, if 60 per cent of the totalexpenditure on fresh fruit occurs in the North regionand 40 per cent in the South region, then the regionalweight for fresh fruit is 60 per cent for the North regionand 40 per cent for the South region.4.11 A region may also be a geographical area, a city

or a group of cities, with a particular location or of acertain size. The rationale for introducing regionalweights is to create more homogeneous entities whichare likely to experience similar price movements andhave similar consumption patterns. For example, theremay be quite large differences in consumption patternsand price developments between urban and rural areas.It may be necessary to distinguish different regions infederal countries because CPIs for the provinces or localstates may be required for administrative or politicalpurposes. In addition, in federal countries indirect taxesand hence price development may differ between theprovinces.4.12 Regional weights may typically be obtained

from the HES or they may be estimated from retail salesdata or population data. Regional weights may or maynot be introduced into the CPI, depending on the sizeand structure of the country, data availability, resourcesand the purpose of the index.

Outlet or outlet-type weights4.13 Prices are collected from a variety of outlets

and outlet types. Information about the sale or marketshare of the outlets may be used to form elementary

aggregate weights specific to a given region and outlettype. One advantage from applying outlet weights is thatit may allow prices to be collected centrally from super-markets or other types of chain outlets.

Elementary aggregate weights4.14 The elementary aggregate weights are the

stratum weights according to expenditure class or sub-class, region and type of outlet. For example, expendi-tures within the sub-class ‘‘fresh fruit’’ may be dividedinto four regions, each having its own regional weight,as in Table 4.1. Assume further that it is known orestimated that 60 per cent is sold in supermarkets and40 per cent in independent outlets, and that this samebreakdown holds for all regions. Let the weight of freshfruit in the CPI for the whole country be, say, 5 per cent.If no breakdown by region or outlet is carried out, thenthe sub-class as a whole becomes the elementaryaggregate carrying a weight of 5 per cent in the overallindex.

4.15 If weights are available by region but not bytype of outlet, the 5 per cent is distributed over the fourregions to obtain four separate elementary aggregates,one for each region. For example, the elementaryaggregate for the North region will have a weight of0.20� 0.05=1.0 per cent in the overall CPI for the wholecountry. If a further division can be made according totype of outlet as well as region, then each region com-prises two elementary aggregates: one for supermarketsand one for independent outlets. The weight for, say, theelementary aggregate for fresh fruit in the North regionsold in supermarkets is then 0.12� 0.05=0.6 per cent inthe overall CPI for the whole country.

Data sources4.16 The decision about what source or sources to

use and how they should be used depends on an analysisof their respective advantages and disadvantages, and onthe main purpose of the index. In most countries, thetwo main sources for calculation of the weights are theHES and the national accounts estimates for households’final consumption expenditures. Additional informationmay, however, be obtained from production and tradestatistics, or from government departments, producers,marketing bodies and individual enterprises. Such addi-tional information is particularly useful for estimatingweights at the most detailed level. Although several ofthe sources may have been used to prepare the national

Table 4.1 Example of weights by region and outlet type forthe sub-class ‘‘fresh fruit’’

Regional weights Type of outlet

Supermarkets(60 per cent)

Independent(40 per cent)

North 20 12 8South 40 24 16West 30 18 12East 10 6 4Total 100 60 40

EXPENDITURE WEIGHTS AND THEIR SOURCES

61

accounts estimates, they may be able to provide furtherdetails that were not used by the compilers of thenational accounts.

Household expenditure surveys4.17 As the HES may have been designed to serve

more than one purpose, it is desirable to ensure that thesurvey design also meets the requirements for the CPI.The main requirements are that the survey should berepresentative of all private households in the country,and not exclude any particular group, and should includeall types of consumption expenditures by households.

4.18 The HES may include payments that are out-side the scope of the CPI: for example, payments ofincome taxes, life insurance premiums, remittances, giftsand other transfers, investments, savings and debtrepayments. These should be excluded from the totalused to calculate the expenditure shares that serve as thebasis for estimating the CPI weights. There may alsobe a difference in the population coverage between theintended scope for the CPI and the actual scope of theHES, but the effects on the CPI of any consequent bias inthe resulting weight estimates are likely to be very minorif the HES is designed to provide results for the wholepopulation and not just a particular population group.

4.19 National food surveys are special surveys inwhich the primary emphasis is on collecting informationon family expenditures for food products. These surveysprovide a very detailed breakdown of food expendituresthat can be used to derive the weights for elementaryaggregates for food below the level of a COICOP class.

4.20 The HES may provide the basis for estimatingspecific weights for regions with different consumptionpatterns. These weights should be applied to the respec-tive elementary price indices to calculate indices for theregions concerned.

4.21 In general, HES data for certain types ofexpenditures may not be sufficiently reliable and need tobe checked against data from other sources. Certaintypes of expenditures may not even be covered by anHES so that they have to be estimated using othersources. The reliability of the CPI weights will obviouslydepend to a large extent on the reliability of thehousehold expenditure data. As the HES is a samplesurvey, the estimates are bound to be subject to sam-pling errors, which may be relatively large for small orinfrequent expenditures. The quality of the estimateswill also suffer from non-response and from the under-reporting of some types of consumption. Under-reporting is probably the most serious and commonproblem affecting HESs. Some expenditures are notreported because the purchases are small or exceptional,and therefore easy to forget. Although large, estimatesof expenditures on durable goods may also be proble-matic, since they are only purchased very infrequently.Some expenditures are not reported because the pro-ducts have a social stigma or are illegal (e.g., drugs,alcohol and tobacco). When no adjustments are madefor such under-reporting, the consequence is an under-estimation of the weights for these items and an over-estimation of the weights for the correctly reported

items. For these reasons, to the extent possible, resultsfrom the HES should be compared and/or combinedwith the statistics from other sources when constructingCPI weights, especially when the HES sample is small.

4.22 For the purposes of the CPI, it is desirable forthe HES to be conducted annually. This will allow coun-tries to revise and update their expenditure weights eachyear. One advantage of annual updating of weights isthat the differences between the results obtained fromthe use of different index number formulae tend to bereduced. Any bias which may follow from using a Loweindex that uses a fixed basket of goods and services willnot have time to accumulate to a significant magnitude,as explained in Chapters 1, 9 and 15.

4.23 Some countries conduct continuous HESs withgradually rotating samples. A programme of annualsurveys with samples large enough to provide the type ofestimates required for CPI weights can, however, be verycostly. For this reason, some countries conduct large-scale surveys at ten-year or five-year intervals, perhapssupplemented with a smaller annual sample. Othercountries distribute a large sample over several years.The average of the results over several successive yearsof smaller-scale surveys may provide a set of satisfactoryannual estimates. The weights derived in this way as theaverage rates of expenditure over periods of two or threeyears will also smooth any erratic consumer behaviourover a short period, for example as a result of eventssuch as droughts or floods, civil strife, oil shocks, orexceptionally mild or cold winters.

4.24 It should be noted that in some countries it maybe possible to experiment with new methods of record-ing expenditures in an HES by using scanner data gen-erated by electronic points of sale. For example, bycollecting the printed bar code receipts for cash whichcustomers receive, the Icelandic HES could obtain, atvirtually no cost to the surveyed households, preciseinformation about types and brands of goods purchasedin different outlets.

National accounts4.25 There may be differences in the scope and defi-

nition of consumption between the national accountsand the CPI, and also a difference in the referencepopulation of households between the national accountsand the HES.

4.26 First, in national accounts, the household sec-tor consists of all resident households, including peopleliving in institutional households. HESs, however, donot usually cover persons living permanently in institu-tional households, such as retirement homes or religiousinstitutions. If the CPI is meant to cover all residenthouseholds, therefore, national accounts estimates maybe used to adjust the HES data.

4.27 Second, as already explained in Chapter 3, it ispossible to have two alternative concepts of total finalconsumption, domestic and national. The domestic con-cept refers to consumption on the economic territory,including the consumption of visiting foreign householdsbut excluding the consumption of resident householdswhen abroad. The national concept used in national

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

62

accounts refers to the consumption of all the residents ofthe country, whether at home or abroad, the consump-tion of non-residents being excluded. The HES usuallycovers only resident households, and may or may notcover their expenditures abroad, depending on theinstructions given to the respondents.4.28 National accounts data may be used to improve

HES weights for products that are under-reported in theHES. Note that national accounts figures for house-holds’ final consumption are usually based on statisticsfrom the HES and from several other sources. Thismeans that national accounts estimates are likely to beuseful for estimating weights for consumption categoriesthat tend to be wrongly reported in the HES, and whereresults from the HES suffer from a significant and dis-torting partial or total non-response rate.

Retail sales data4.29 Statistics on retail sales by region and type of

outlet may be available for broad groups of items. Onedisadvantage is that some of the sales may be to groupsoutside the reference population, perhaps to the businesssector or to the government. The corresponding pur-chases do not form part of household private consump-tion. Some sales may also be to non-residents, whomay or may not be part of the reference population.Furthermore, for regional sales data, it needs to be keptin mind that sales may include sales to people living inother regions.

Point-of-purchase surveys4.30 Point-of-purchase surveys may provide statis-

tics that can be used to estimate weights for price data, asthey permit the analysis of shopping patterns for varioussegments of the population. Households are asked, foreach item purchased, about the amounts spent in eachoutlet where purchases have been made, and the nameand addresses of these outlets. On this basis, a list ofoutlets can be established, with the total sales for all thedifferent items to the sample of households. A sample ofoutlets is then drawn from this list, with probabilityproportional to the sales. Given that household surveysare expensive and that there is an overlap between theHES and point-of-purchase survey, it is possible tocombine the two data-collecting activities in an inte-grated survey that elicits expenditure and outlet data atdetailed levels, along with the demographic informationabout the households needed for sub-group indices.4.31 A simpler version of this survey may be con-

ducted to obtain weights for groups of products byoutlet type. In this case, a purposive sample of eachoutlet type should be selected. As an alternative, in theabsence of this type of survey, national retail sales sta-tistics by outlet type from a survey of outlets could beused to estimate a breakdown of sales by outlet type.

Scanner data4.32 In the last few years some countries have star-

ted to use statistics obtained from cash register data toderive CPI weights. These statistics are based on elec-

tronic data records that are stored as scanner data in thedatabases of sellers. Such scanner data sets include thequantities sold and the corresponding value aggregates.(The cash register receipts usually give the followinginformation: name of the outlet, date and time of pur-chase, description of items bought, quantity, price andvalue, form of payment, and VAT amount where rele-vant.) A comparison of the results from the HES withthe corresponding scanner data from the biggest super-market chains indicates that the use of scanner data canadd to the reliability of the weights (Guðnason, 1999).This strengthens the case for using such data to reviseCPI weights more often than otherwise would be pos-sible, and probably at lower cost. The limitations of thissource of information should, however, be borne inmind. The first one is that scanned data cannot be con-nected to a specific type of household, whereas the datafrom theHES can. Another important difference betweenHES data and scanner data from sellers is that the HESdata cover goods bought from outlets that are not usingthis technology, as well as goods and services that do notcarry scanner codes, regardless of where they are sold.Although the use of electronic data records is increasingevery year, significant components of the retail trademarket are not using scanner data even in countries thatare electronically most advanced.

Population censuses4.33 Population censuses provide statistics on the

geographical distribution of the population and house-holds, as well as on the regional differences in householdsize and composition. Combined with estimates of regio-nal levels of household expenditure, these statistics can beused to estimate regional expenditure weights, especiallywhen such estimates are not available from an HES with asatisfactory degree of precision. In the absence of anyexpenditure statistics, population statistics might be usedas the basis for regional weights. Such estimates for theweights usually have to assume that expenditures percapita or per household are the same in all regions, andhave to ignore the fact that there are usually large dif-ferences between the urban and rural populations in thelevel and pattern of items that they consume.

Deriving the weights in practice4.34 Once the reference population and the coverage

of goods and services have been decided, the weightsneed to be derived. In principle, this is a relatively simplematter, as the weights are calculated as the proportionsof the total consumption expenditure of all goods andservices included in the index basket for the referencepopulation during the reference period. In practice,however, the calculation of weights is not so straight-forward and involves a series of steps.

Payments that are notconsumption expenditures

4.35 Only consumption expenditures are relevantfor the construction of CPI weights. As explained in

EXPENDITURE WEIGHTS AND THEIR SOURCES

63

Chapter 3, outlays such as payments of social securitycontributions or income taxes, or repayments of debts,are irrelevant and should be ignored because they arenot consumption expenditures.

Unimportant expenditures4.36 Each elementary aggregate consists of a fairly

homogeneous group of products from which one ormore representative products are selected for pricing.Some products may have a weight which for all practicalpurposes is negligible and for which prices are unlikelyto be collected in practice. The HES, which in most casesis the main source for deriving the detailed weights,usually includes observations on a much larger varietyof goods and services than it is practical to collect pricesfor. The prices of very minor products may not be worthcollecting if they contribute almost nothing to the CPI.

4.37 Even though it may be decided not to collectprices for a certain product, it remains within the scopeof the CPI. Some price change has to be explicitly orimplicitly assumed, or imputed, and weighted by ex-penditures. There are two options:

� The product and the expenditures on it remain withinthe elementary aggregate, even though no prices arecollected for it. The elementary price index for theaggregate as a whole is estimated entirely by the pricesof the representative products for which prices arecollected. This is equivalent to assuming that the priceof the product changes at the same rate as the averagefor the prices of the representative products.

� The alternative is to reduce the weight for the ele-mentary expenditure aggregate by excluding theexpenditures on the product. This is equivalent toassuming that the price of the excluded product wouldhave moved in the same way as the overall CPI for allthe products actually included in the index.

4.38 In principle, the CPI should cover all types ofproducts and expenditures within its scope, even if pricesare not collected for some products. It might be decided,for example, to exclude from the index calculationsgroups with weights lower than, say, 0.1 per cent for foodgroups and 0.2 per cent for non-food groups. A lowerminimum threshold for the food items might be setbecause the prices for these products tend to displaygreater variability and because prices for food productsare normally less expensive to collect. If an expendituregroup is excluded, its weight may be redistributed toanother expenditure group that is similar in terms ofcontent and price development. Alternatively, the expen-ditures may be completely excluded from the calculationof the weights.

Products that are difficult to price4.39 Among the consumption expenditures, there

are likely to be a few products for which the prices, orprice changes, cannot be directly or satisfactorily mea-sured, for example, illicit drugs or payments for cateringand other service charges for private receptions andparties. Even if reliable prices cannot be obtained, theseproducts should be included in the calculation of the

weights if they are within the scope of the index. Fordifficult-to-price products, the options available are thesame as those used for unimportant expenditures.

Use and combination of different sources4.40 In most countries, the main source for deriving

the weights is the HES. As noted above, however, theresults from the HES need to be carefully examined andadjusted to take account of under- or over-reportingof certain types of products. The usual strategy is touse supplementary information from other relevantsources to adjust the HES results in order to derive theweights.

4.41 In countries where national accounts dataprovide reliable estimates of household expenditures,these data can be used to derive the weights at anaggregate level. Detailed HES data can then be used tobreak down or adjust these weights. In this way, it ispossible to reconcile the detailed data from the HESwith the aggregate national accounts data to calculatethe weights. Weights for the main consumption groupscan be obtained from the national accounts down to acertain level of disaggregation, say, 70 consumptiongroups or classes. Each of these weights can then befurther distributed by applying the detailed HES expen-diture groups to the national accounts consumptiongroups or classes. The combination of national accountsand HES data ensures consistency between the CPI andthe national accounts data on consumption expenditureof households at the level of the main consumptiongroups.

Adjusting the weights derived fromhousehold expenditure surveys

4.42 As, in most cases, the information from a house-hold expenditure survey is only available with a lag –often around 18 months or more – the new weights willlag behind the new price reference period for the index,which is the period when the new weights are introduced.

4.43 Adjustments might need to be made to theestimates based on the HES results to take into accountany significant changes in expenditure patterns in theperiod between the time that the survey was carried outand the time that the new weights were introduced.Adjustments will typically be made for products whichare significantly losing or gaining in importance duringthis period. It is also possible that expenditure on someproducts may not be available from the HES becausethe products appeared on the market after the surveyhad been completed. One example is mobile telephonesand the corresponding charges, which emerged as sig-nificant new forms of expenditure in the late 1990s inmany countries. Necessary adjustments then need to bemade to the survey data to take into account thechanges that have occurred. The expenditures on thesenew products should be estimated on the basis ofinformation available from other sources (e.g., importand retail trade statistics), taking into account the needto exclude expenditures by enterprises and for businesspurposes.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

64

Weight reference period4.44 The weight reference period is the time period

to which the estimated weights relate. The choice of theperiod covered by the expenditure statistics used toderive the weights is crucial. Generally speaking, theperiod chosen as the base should be long enough tocover a seasonal cycle. Further, if the index is notannually chained, the year chosen should have economicconditions that can be considered to be reasonablynormal or stable. To achieve this, it may be necessary toadjust some of the values to normalize them and over-come any irregularities in the data for the particularperiod that constitutes the source of the information.The weight reference period should not be too distantfrom the price reference period. The weight referenceperiod is typically a single calendar year. A month orquarter is too short a period to use as a weight baseperiod, since any one month or quarter is likely to beaffected by accidental or seasonal influences. In someinstances, data for a single year may not be adequateeither because of unusual economic conditions orbecause the sample is not large enough. An average ofseveral years of expenditure data may then be used tocalculate the weights. Countries in which this method isapplied include the United States and the UnitedKingdom. In the United States, the expenditure infor-mation from the Consumer Expenditure Survey over athree-year period is used. In the United Kingdom, anaverage of three years of Expenditure and Food Surveydata is used for regional weights, for stratification andfor a limited number of groups of products where pricestend to be particularly volatile.4.45 During periods of high inflation, multiple year

weights may be calculated by averaging value sharesrather than averaging actual value levels. Averaging thevalue levels will give too much weight to the data for themost recent year. Another option is to update the valuesfor each year to a common period and then to computea simple arithmetic average of adjusted yearly data.4.46 As the weight reference period usually precedes

the price reference period, the expenditure weights maybe price updated to take account of the relative pricechanges from the weight reference period to the pricereference period. Price updating of weights is discussedin more detail in Chapter 9, paragraphs 9.95 to 9.104.

Need for revising the weights4.47 Most countries calculate their CPI as the

change in the value of a specified basket of goods andservices. An index of this general kind is described in thismanual as a Lowe index. Its properties and behaviourare explained in Chapters 1, 9 and 15. Although CPIsare often described as Laspeyres indices, they are usuallynot Laspeyres indices in practice. A Laspeyres index isdefined as an index in which the basket of goods andservices is that of the price reference period, but a typicalCPI basket uses the basket of some weight referenceperiod that precedes the price reference period, as justexplained. As many countries continue to use the samefixed basket of goods and services over a period of years,the question arises of how often the basket should be

revised in order to ensure that it does not become out ofdate and irrelevant.

4.48 In the short run, consumers will change con-sumption patterns in response to shifts in relative prices,mostly between products included in the same class orsub-class. Over longer time periods, consumption patternsare also influenced by factors other than price changes.Most importantly, changes in the level and distribution ofhousehold income will cause a shift in demand for goodsand services towards goods and services with higherincome elasticities. Demographic factors such as ageing ofthe population, and technological changes, such as theincrease in the use of computers, are examples of otherfactors that affect spending behaviour in the longer run.Furthermore, new products will be introduced and exist-ing ones may be modified or become obsolete. A fixedbasket will be unresponsive to all these changes.

4.49 As a result of both relative price changes andlong-term effects, the weights may become out of date andless representative of current consumption patterns. Asshown in Chapter 15, the bias in a Lowe index is likely toincrease with the age of the weights. At some point, ittherefore becomes desirable to use the weights of a morerecent period to ensure that the index is weighting appro-priately the price changes currently faced by consumers.

Frequency of updating the weights4.50 The 1987 ICLS resolution concerning consumer

price indices recommended that the weights should beupdated periodically, and at least once every ten years, toguarantee the representativity of the index. However, the2003 ICLS resolution proposes more frequent updates ofthe weights, such as once every five years, to ensure theirrelevance. Countries which are experiencing significanteconomic changes and thus more rapid changes inconsumption patterns should update their weights evenmore frequently, say annually.

4.51 The need to revise the weights generallyincreases as the length of time from the weight referenceperiod increases. The decision when to update theweights depends, for the most part, on the differencesobserved between the current weighting structure andthat for the weight reference year. Changes in the rela-tive importance of each item can be observed throughexpenditure survey results. If these statistics are avail-able only at irregular intervals, the frequency of weightrevision may necessarily be linked to the availability ofresults from the HES.

4.52 The introduction of new weights each yearmight possibly cause an upward drift in the index if thereare big fluctuations in consumption caused by factorssuch as an economic blockade, or extremely favourableor unfavourable weather conditions. In general, the pro-file of the index time series can be sensitive to the selec-tion of the weight reference period. It might be best to usea ‘‘normal’’ consumption period, if possible, as the basisfor weighting information and to avoid periods in whichthere are special factors at work of a temporary nature.All available information concerning the nature of con-sumption in a weight reference period should be takeninto consideration.

EXPENDITURE WEIGHTS AND THEIR SOURCES

65

4.53 When the weights are to be fixed for severalyears, the objective should be to adopt weights that arenot likely to change much in the future rather thanprecisely reflect the activity of a particular period thatmay be abnormal in some way.

4.54 Each year it is desirable to carry out a review ofthe weights in order to ensure that they are sufficientlyreliable and representative. The review, which may beconfined to weights at the level of sub-indices and theirmajor components, should examine whether or not thereare indications that important changes may have takenplace in the consumption pattern since the weightingreference period.

4.55 Whenever the weighting pattern has beenupdated, the new index using updated weights should becalculated for an overlapping period with the old one sothat the two can be linked.

Classification4.56 In deriving the weights, the detailed expenditure

items identified in the HES must be mapped to the CPIexpenditure classes. If HES classes do not match CPIexpenditure classes, the HES results must be transformedto match the CPI categories. This can be done by ag-gregating or disaggregating the relevant HES headingsover the relevant CPI expenditure classes. Such trans-formation is achieved much more easily and more reli-ably if the coding list for expenditure items in the HES iscoordinated with the corresponding list of items used forcollecting price observations for the CPI.

4.57 For the purposes of international comparison,the classification scheme of goods and services should,to the extent practical, be in line with the United NationsClassification of Individual Consumption accordingto Purpose (COICOP) (see Annex 2). To facilitateestimation and application of weights, it is also desir-able that the classification used be consistent with theclassifications used for HESs and other statistics (forexample, retail sales statistics). In the interests ofmaintaining both coordination of the statistical systemand international comparability, the HES should alsouse a classification for types of expenditure that will beconsistent with COICOP, and it should also be possibleto establish a mapping between products in the retailsales price collections and COICOP. Another impor-tant objective is that the aggregation structure employedby the classification system should meet the major needsof users.

4.58 Using COICOP as an example, the classifica-tions have the following hierarchical structure:

– groups: there are 47 of these in COICOP;

– classes: sub-divisions of the groups;

– sub-classes: the lowest-level categories that areweighted and usually the most detailed level of thestructure for which index series are published – theseare the expenditure components and weights thatremain fixed when using a fixed weight index;

– individual products: the lowest level of the CPI basket,that is, the individual goods and services for whichprices are actually collected – this is the level at which

the composition of the CPI basket can be adjustedbetween two major revisions of the weighting struc-ture to reflect changes in product supply and con-sumer behaviour.

4.59 Upper-level indices are formed by weightingtogether lower-level indices through progressive levels ofaggregation, as defined by the classification structure.Weights are fixed for a period (say one, three or fiveyears) between index reweighting.

4.60 The selection of the level in the index hierarchyat which the structure and weights are fixed for a periodis particularly important. The main advantage of settingthe level relatively high is that the actual samples ofproducts and their prices below this level can be adjustedand updated as needed. New products can be introducedinto the samples, and the weights at the lower level re-established on the basis of more recent information.There is thus a greater opportunity to keep the indexrepresentative, through an ongoing review of the sampleof representative products.

4.61 If the level is set relatively low in the indexstructure, there is less freedom to maintain the repre-sentativeness of the index on an ongoing basis, andthere will be a greater dependence on the periodicindex review and reweighting process. In such circum-stances, the arguments for frequent reweighting becomestronger.

Items requiring special treatment4.62 Some products, such as seasonal products,

insurance, second-hand goods, expenditures abroad,etc., may need special treatment when constructing theirweights. Reference may be made to Chapters 3, 10 and22 for further details.

4.63 Seasonal products. Various approaches may beused to deal with seasonal products, for example:

� a fixed weights approach, which assigns the sameweight for the seasonal product in all months, usingan imputed price in the out-of-season months. Sea-sonal products are treated in the same way as otherconsumption products;

� a variable weights approach, in which a changing (ormoving) weight is attached to the product in variousmonths. In this method, the weights of the seasonalproducts change monthly according to changes in thequantities consumed during the different months ofthe weight reference period. The principle of a fixedbasket – i.e. fixed weights – should, however, bemaintained at least at some level of aggregation.

4.64 The advantage of applying the fixed weightsmethod is mainly that it is consistent with the methodsused for other consumption goods and services, andwith the fixed basket index formula. In contrast to themoving weights method, the fixed weights method re-flects monthly changes in prices only, and not in quan-tities. Another disadvantage of the moving weightsmethod is that the weights are based on the monthlyseasonal fluctuations in the weight reference period,whereas the monthly fluctuations in consumption maydiffer every year.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

66

4.65 The fixed weights method may also have dis-advantages, a major one being that during the monthsthat fresh fruits or vegetables disappear, prices andindices have to be estimated or imputed for theseitems (or, as is done in some countries, prices and indiceshave to be frozen throughout the period of dis-appearance). These imputations need not be made whenapplying the moving weights method. In addition, theaverage fixed weight determined for all months of theyear does not actually reflect the monthly consumption.Therefore, if there is a negative correlation betweenprices and quantities, there may be an upward bias inthe index.4.66 The choice of measuring seasonal goods

according to the fixed weights method or the movingweights method should be based on whether the focus ison month-to-month changes or on the long-term indexchanges. The use of an annual basket and the use ofannual expenditure shares are appropriate where themain interest is in the longer-run trend of price changes.On the other hand, if the focus is on month-to-monthchanges, then the annual weights attached to eachmonth-to-month price relative can be unrepresentativeof actual transactions that are taking place in the twoconsecutive months under consideration. In the lattercase, monthly price changes for items that are out ofseason can be greatly magnified by the use of annualweights.1 To satisfy the needs of different users, it maybe appropriate to construct two indexes: one for theshort-term measurement of price changes (with variablemonthly weights) and another longer-term index (withfixed annual weights). The issue of seasonal items isdealt with in detail in Chapter 22.4.67 Insurance. As explained in the section on

insurance in Chapter 3, the weights for non-life insur-ance could be based on either the gross premiums paidor on the implicit service charges. The implicit servicecharges for administering the insurance and providingthe insurance services are estimated by the gross pre-miums plus the income from investment of the insurancereserves less the amounts payable to policy holders insettlement of claims.2 The net premiums are defined asthe gross premiums less the service charges: in otherwords, the net premiums equal the claims. The netpremiums and claims can be regarded as transfers, orredistributions, between policy-holding households. Ingeneral, it seems preferable to base the weights for non-life insurance on the service charges. These are theestimated amounts paid by households for the servicesprovided by insurance firms. However, a case can alsobe made for basing the weights on the gross premiums.This is a difficult area in which there is not yet a con-sensus.

4.68 Second-hand goods, including used cars. Asalready explained in paragraphs 3.127 to 3.129 of Chap-ter 3, the prices of used or second-hand durable goodspurchased by households are included in the CPI in thesame way as the prices of new goods. However, house-holds also sell used durables, such as cars. If the price of asecond-hand good rises, a purchasing household is worseoff, but a selling household is better off. From a weightingperspective, sales constitute negative expenditures, whichimplies that price changes for used goods sold by house-holds implicitly carry a negative weight in the CPI. Ineffect, purchases and sales of second-hand goods betweenhouseholds, whether directly or indirectly through adealer, cancel out (except for the dealers’ margins, seeChapter 3) and carry no weight in the CPI. However,households also buy from, and sell to, other sectors. Forthe reference population as a whole, namely the entire setof households covered by the CPI, the weight to beattached to a particular kind of second-hand good isgiven by households’ total expenditures on it less thevalue of the households’ receipts from sales to/from out-side the household sector. There is no reason why theseshould cancel out on aggregate. For example, many of thesecond-hand cars purchased by households may beimported from abroad. The difference between totalexpenditures and total sales is usually described ashouseholds’ net expenditures. This is the weight to beattached the second-hand good in question.

4.69 Except in the case of used cars, however, it ispractically impossible to estimate the net expenditurebecause most HESs do not collect the data that wouldallow for a comparison between expenditures and receiptsfrom sales of individual kinds of second-hand goods.Usually, only the total amount received from the sale ofsecond-hand goods is collected. This information does,however, give an idea of the volume and significance ofthese transactions in the national economy. In countrieswhere this volume is small, second-hand goods (exceptused cars) may be ignored when calculating the weightsof the index.

4.70 Because the amounts spent on purchasing second-hand cars are usually large, they should be included inthe CPI basket if the data are available. In the absence ofreliable data, however, their weight can be added to theweight of new cars.

4.71 Most countries include expenditure on second-hand goods in the estimation of CPI weights, but second-hand goods are not covered in the price collection(because of the difficulty of pricing the same good eachmonth or, where the goods are different, making anappropriate quality adjustment). It is therefore assumedthat the prices of new and second-hand goods move inthe same way.

4.72 In countries where second-hand purchases areimportant and their prices are believed to change atdifferent rates from those of new goods, separateweights are needed for them. The information could beobtained, at least for some major durables, from HESs,if the surveys ask about expenditure on second-handand new goods.

4.73 Expenditure abroad and expenditures by non-residents. If the objective is to construct an index which

1For example, the impact of change in tomato prices at the beginningof the season would be overstated in the general index. Similarly, itsimpact in the peak months would be understated.2In the national accounts, the gross premiums plus the investmentincome less the estimated service charges are described as ‘‘net pre-miums’’. By definition, ‘‘net premiums’’ equal claims payable, bothflows being treated as transfers, or redistributions, between policy-holding households. The ‘‘net premiums’’ are not regarded as expen-ditures.

EXPENDITURE WEIGHTS AND THEIR SOURCES

67

is representative of price movements within a givencountry or area, the weighting system must reflect pur-chases by both resident and non-resident households. Inpractice, the proportion of total purchases that are madeby visitors from abroad or other areas may be difficult toestimate, except for certain types of purchases in geo-graphical areas where foreign tourism is the dominanteconomic activity. Sources other than HESs must beused in order to ensure that the weights include theexpenditures made by foreign tourists and reflect allpurchases of consumer goods and services made byresident or non-resident households within the country.These sources may be national accounts or commercialsales statistics.

4.74 Where the main purpose of the index is tomeasure price changes experienced by the residentpopulation, the weights should include their expendi-tures abroad. This would require collection, through theHES, of data on expenditures made outside the country(for example, expenditures on hotels and meals duringholidays, durables, health and education). Possible waysof constructing the index to cover expenditure abroadwould be:

– price collection outside the country of residence;

– the use of appropriate sub-indices provided by thestatisticians in other countries for the kinds of pro-ducts purchased there by residents;

– establishing a panel of residents who would reportprices paid for their purchases abroad.

4.75 Given the limitations of HESs to provide reli-able data on expenditures abroad, and the practicaldifficulties of constructing an index for expenditureabroad, the weights may have to be based on expendituresurveys without adjusting for the place of acquisition,

and prices may be collected only for the goods and ser-vices acquired in the economic territory of the country.Such an approach assumes that the price changes of thegoods and services acquired abroad are the same as thosefor the same goods and services acquired at home.

Errors in weighting4.76 If all prices moved in the same way, weights

would not matter. On the other hand, the greater thevariation in price behaviour between products, the greaterthe role of weights in measuring aggregate price change.

4.77 Small changes in the weights usually havevery little effect upon the overall CPI. An error in theweight for a given sub-index only matters to the extentthat the change in the sub-index differs from the averagechange in the overall CPI. In general, the higher a sub-index’s weight, the lower is the tolerable percentageerror in that weight. It follows that the tolerable error inthe weights declines as the rate of relative price changefor the relevant items increases. Finally, it is also clearthat while errors in weighting may not have a largeinfluence on the overall index, the sub group-levelerrors could be significant. Australian experience showsthat even items with relatively large weights can tolerateerrors of 20–30 per cent in the weights (AustralianBureau of Statistics, 2000). According to Eurostat’sstudies, CPIs are fairly insensitive to changes in weights.Eurostat has, however, suggested developing qualitycontrol procedures for monitoring the weights of itemsfor which changes in prices have diverged from themovement of the overall index (Eurostat, 2001). Thequestion of the effects of weighting errors on the sub-index and the overall index is discussed in Rameshwar(1998).

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

68

5SAMPLING

Introduction5.1 The procedure used for price collection by a

national statistical office in the production of a consumerprice index (CPI) is a sample survey. In fact, in manycountries, it might be better viewed as composed of manydifferent surveys, each covering different subsets of theproducts covered by the index. We will therefore beginby outlining some of the general concepts of surveysampling which need to be kept in mind when looking ata particular survey such as price collection for a CPI.5.2 There is a target quantity, for example a CPI,

which is defined with respect to:

– a universe consisting of a finite population of units(e.g. products);

– one or more variables that are defined for each unit inthe universe (e.g. price and quantity);

– a formula which combines the values of one or more ofthese variables for all units in the universe into a singlevalue called a parameter (e.g. the Laspeyres index).

The interest is in the value of this parameter.5.3 The universe usually has three dimensions.

There is a product dimension, consisting of all purchasedproducts and varieties of products. There is a geo-graphical and outlet dimension consisting of all placesand channels where a product is sold. Finally, there is atime dimension consisting of all sub-periods within anindex period. The time dimension will be given lessattention since price variation is usually smaller over ashort time span and since temporal aspects may be dealtwith in product and outlet specifications.5.4 In this chapter, the first two dimensions will be

regarded as being static over the time periods consideredin the index. In other words, it will be assumed that thesame products and outlets are in the universe in bothperiods, or that replacements between old and new prod-ucts or outlets are one to one and without problems. Forthe complications arising from dynamic changes in theuniverse, please refer to Chapter 8, where replacement,resampling and quality adjustment are discussed.5.5 Why take only a sample of units? Apart from the

near physical impossibility and prohibitive cost of tryingto cover all products in all outlets, the data are likely tobe of better quality if there are fewer units to deal withbecause of the use of more specialized and better traineddata collectors. Also, the time required to complete theexercise is shorter.5.6 In probability sampling, the units are selected in

such a way that each unit (an outlet or a product) has aknown non-zero probability of selection. For example,outlets are selected at random from a business register in

which each outlet has an equal chance of being selected.Traditionally, however, non-probability sampling meth-ods have mainly been used in the compilation of a CPIfor choosing outlets or products. The representativeitem method is particularly popular for selecting items.Other methods used are cut-off sampling and quotasampling (see below). There are also instances of a mix-ture of the two methods of sampling; for example, outletsare selected using probability sampling techniques,whilst products are selected using the representative itemmethod.

5.7 Having decided to sample, there are two issues tobe considered: how to select the sample; and how to usethe sample values to estimate the parameter. The formerreflects the choice of sampling design, and the latterconstitutes the estimation procedure. We first take a lookat sampling design.

Probability sampling techniques5.8 This section presents some general concepts and

techniques of survey sampling that have importantapplications for price indices. This brief presentationcovers those concepts of survey sampling that are ofimmediate interest in price index applications. For a fulltreatment of the subject, please refer to one of the manytextbooks available, for example Sarndal, Swensson andWretman (1992) or Cochran (1977).

5.9 Survey sampling theory views the universe ascomposed of a finite number (N) of observational unitsdenoted j=1, . . . ,N. Sampling then amounts to selectingn units out of N by attaching an inclusion probability,pj, to each unit. For price indices there are two samplingdesigns that are of particular interest.

5.10 In simple random sampling and systematic sam-pling each unit is sampled with equal probability and wehave pj= n/N. In simple random sampling, all units areselected using a random mechanism. In systematic sam-pling, the sampling units are selected at equal distancesfrom each other in the frame, with random selection ofonly the first unit. These techniques are usually recom-mended in situations where the units are relativelyhomogeneous.

5.11 In probability proportional to size (pps) samplingthe inclusion probability is proportional to some aux-iliary variable xj and we have pj=nxj=

PNj=1xj . Units for

which initially this quantity is larger than one are selectedwith certainty, whereafter the inclusion probabilities arecalculated for the remainder of the universe.

5.12 The universe may be divided into strata,denoted h=1, . . . ,H. In each stratum, there are then Nh

69

units and we havePHh=1Nh=N. The purpose of strati-

fication is usually to group units together that are eitherhomogeneous in some sense or satisfy some adminis-trative convenience such as being physically close to-gether. Each stratum is a mini-universe with samplingtaking place independently in each one. In a CPI, thepractice is to use elementary aggregates as strata. In theremainder of this chapter, we look at sampling in asingle stratum, corresponding to an elementary aggre-gate, and drop the subscript h.

Implementing probability samplingin consumer price indices

5.13 A sampling frame is a list of all (or most) of theN units in the universe. A frame may have overcoverageto the extent that it includes units that are not in theuniverse or includes duplicates of units. It may haveundercoverage to the extent that some units in the uni-verse are missing from the frame.

5.14 Sampling frames for the outlet dimension couldbe:

� Business registers. These should include locations ofretail trade businesses with addresses and be updatedregularly. If a size measure (turnover or number ofemployees) is included in the register, it is a usefuldevice for performing probability proportional to size(pps) sampling and this size measure would then beincluded in the universe parameter also.

� Telephone directories (‘‘yellow pages’’). These usuallydo not include size measures so simple random sam-pling or systematic sampling would then be necessary.Sometimes informal knowledge of the importance ofdifferent outlets could be used to stratify the universeinto two or more categories and then draw a relativelylarger sample from the more important strata.

� Records of local administrations, organizations ofenterprises, and so on, could be used for local marketsand suchlike, which are especially important in devel-oping countries.

5.15 Sampling frames for the product dimensioncould be:

� product lists provided by major wholesalers showingsales values for varieties in an earlier period. Salesvalues provide an obvious size measure for weightsand pps sampling;

� outlet-specific lists of products. These lists could alsobe drawn up by the price collectors themselves bynoting the products displayed on the shelf. Shelf spacecould then be used as a size measure for pps sampling.

Sampling techniques based onprobability proportional to size

5.16 Several techniques exist for drawing pps sam-ples. They fall into two main categories according towhether the size of the sample is fixed or random. Afixed, predetermined sample size is clearly desirable forCPIs since the sample size in each stratum is often smalland a random size would entail the risk of an empty

sample. We therefore present two techniques here thatprovide fixed size pps samples.

5.17 Systematic pps sampling. The procedure is bestexplained by an example. In Table 5.1 we show how asample of 3 outlets can be drawn from 10. In this casewe have the number of employees as our size measure.We look at the list, where we have included the cumu-lative sizes and the inclusion intervals. We take the totalnumber of our size measure, which is 90 in this case, anddivide it by the sample size, 3. This gives us a samplinginterval of 30. We next choose a random numberbetween 1 and 30 (random number functions are given in,for example, the Excel spreadsheet software). Say thatwe get 25. The sample will then consist of the outletswhose inclusion intervals cover the numbers 25, 25+30and 25+2� 30.

5.18 Systematic sampling is easy to execute. If,however, the frame has some overcoverage, the samplesize will not be the predetermined one. Let us say that,at the first visit to the outlets, we discover that outlet 6does not sell the products in the product sample. Wewould then be left with a sample of only two outlets. Wewould either be content with that, or somehow seek areplacement for the invalid outlet, which is not deter-mined by the basic sampling procedure. Moreover, theselected sample depends on the order in which theoutlets or products are listed. This might be important,especially if the listing order is correlated to the sizemeasure.

5.19 Order pps sampling. This is a relatively newmethod for drawing pps samples. Rosen (1997a, 1997b)gives its theory. In this case, a uniform random numberUi between 0 and 1 and a variable zi=nxi/

Pixi, where xi

is a size variable, are associated with each sampling unitand a ranking variable is constructed as a function ofthese two variables. The units in the universe are thensorted in ascending order and the n units with thesmallest values of the ranking variable are included inthe sample. Two important examples of such rankingvariables Qi are:

� for sequential pps sampling: Qi=Ui/zi;� for Pareto pps sampling: Qi=Ui (1�zi)/zi (1�Ui).

5.20 For the same universe as above and with Paretopps as our example, we show in Table 5.2 how thisworks. We have now ordered the universe in ascendingorder with respect to the ranking variable. Our first

Table 5.1 Systematic sample of 3 out of 10 outlets, basedon probability proportional to size

Outlet Number ofemployees = x

Cumulative x Inclusioninterval

Included whenstarting point is 25

1 13 13 1–132 2 15 14–153 5 20 16–204 9 29 21–29 X5 1 30 306 25 55 31–55 X7 10 65 56–658 6 71 66–719 11 82 72–82

10 8 90 83–90 X

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

70

sample turns out to consist of outlets 6, 1 and 8. Say thatwe now discover, however, that it is inappropriate toinclude outlet 1. We then turn to the fourth unit in order– outlet 9 – and include that one instead. Thus, orderpps sampling is easy to combine with a fixed sample sizeand more flexible than systematic sampling.5.21 Neither of the two order sampling procedures

is, however, exactly pps, because the obtained inclusionprobabilities vary somewhat from the desired ones.Rosen (1997b) shows, however, that for the purpose ofestimating means and variances, they are approximatelypps. In the case of the price index, this still holds whenthere is sample substitution resulting from overcoverage.Pareto pps is marginally better than sequential pps andshould therefore be preferred.5.22 Order pps sampling is at present used in many

areas of the Swedish CPI, for example for sampling:

� outlets from the business register (the size measure isnumber of employees+1);

� products from databases provided by major retailchains (the size measure is historic sales);

� car models from the central car register (the sizemeasure is number of cars registered in the referenceperiod).

5.23 Further details on the application of theseprocedures are given in Statistics Sweden (2001). Rosen(1997b) shows that Pareto pps and systematic pps are thetwo optimal pps sampling methods. Pareto pps permitsan objective assessment of estimation precision. Withregard to final precision, however, Pareto pps is best insome situations whereas systematic pss is best in othersituations. The choice between them is therefore a matterof judgement and practicality in a particular samplingsituation. The great flexibility of order pps sampling withregard to imperfections in the frame, an aspect ofimportance in CPI applications, leads us to make thisprocedure our first recommendation among pps pro-cedures.

Sampling methods used by theUS Bureau of Labor Statistics5.24 The US Bureau of Labor Statistics (BLS) uses

probability methods in all stages of sample selection.In the last stage, individual items in outlets are selectedin a process designed to approximate pps sampling withrespect to the sales of each such item. To this end, the

BLS field representatives are allowed to use any of fourprocedures for determining the sales proportions (U.S.BLS, 1997):

� obtaining the proportions directly from a respondent;

� ranking the subgroups/items by importance of sales asindicated by the respondent and then obtaining theproportions directly or using pre-assigned propor-tions;

� using shelf space to estimate the proportions whereapplicable;

� using equal probability.5.25 The advantages of this procedure, according to

the BLS, are that it ensures an objective and efficientprobability sampling procedure, where no other suchprocedure would be available. It allows broad defini-tions of the item strata so that the same tight specifica-tion need not be priced everywhere. The wide variety ofspecific items greatly reduces the within-item componentof variance, reduces the correlation of price movementbetween areas, and allows a reduction of the sample sizeneeded for a given variance.

5.26 A potential pitfall in this approach is that, if thesales value measure is taken during a very short period, itmay coincide with a special campaign with temporarilyreduced prices. It could then happen that an item witha temporarily reduced price is given a large inclusionprobability. Since this price will tend to increase morethan average, an overestimating bias may result. It isthus essential that the sampling of the item takes placeat an earlier point in time than the first price collectionor that sales values from an earlier period are used.Okamoto (1999) emphasizes this point for Japan, whereprice bouncing seems to be a very common phenomenon.

Non-probability samplingtechniques

5.27 Modern statistical sampling theory focuses onprobability sampling. Use of probability sampling isalso strongly recommended and standard practice for allkinds of statistical surveys, including economic surveys.But price index practice in most countries is still domi-nated by non-probability techniques. It may then befruitful to speculate somewhat about the rational andirrational reasons for this situation. In the followingsection we discuss a number of such possible reasons,one by one. We then go on to consider some non-probability techniques.

Reasons for using non-probabilitysampling

5.28 No sampling frame is available. This is oftentrue for the product dimension but less frequently so forthe outlet dimension, for which business registers ortelephone directories do provide frames, at least in somecountries, notably in Western Europe, North Americaand Oceania. There is also the possibility of constructingtailor-made frames in a limited number of cities or loca-tions, which are sampled as clusters in a first stage. Forproducts, it may be noted that the product assortment

Table 5.2 Pareto sample of 3 out of 10 outlets, based onprobability proportional to size

Outlet xi Ui Qi Sample

6 25 0.755509 0.036943 X1 13 0.198082 0.207721 (X)8 6 0.915131 0.310666 X9 11 0.277131 0.346024 X

10 8 0.834138 0.3804687 10 0.709046 0.4125994 9 0.46373 0.5802643 5 0.500162 1.255 1 0.067941 1.8364352 2 0.297524 2.926051

SAMPLING

71

exhibited in an outlet provides a natural samplingframe, once the outlet is sampled as a kind of cluster, asin the BLS sampling procedure presented above. So theabsence of sampling frames is not a good enough excusefor not applying probability sampling.

5.29 Bias resulting from non-probability sampling isnegligible. There is some empirical evidence to supportthis assertion for highly aggregated indexes. Dalen(1998b) and De Haan, Opperdoes and Schut (1999)both simulated cut-off sampling of products within itemgroups. Dalen looked at about 100 groups of items soldin supermarkets and noted large biases for the sub-indices of many item groups, which however almostcancelled out after aggregation. De Haan, Opperdoesand Schut used scanner data and looked at three cate-gories (coffee, babies’ napkins and toilet paper) and,although the bias for any one of these was large, themean square error (defined as the variance plus thesquared bias) was often smaller than that for ppssampling. Biases were in both directions and so couldbe interpreted to support Dalen’s findings. The largebiases for item groups could, however, still be disturb-ing. Both Dalen and De Haan, Opperdoes and Schutreport biases for single-item groups of many indexpoints.

5.30 We need to ensure that samples can be monitoredfor some time. If we are unlucky with our probabilitysample, we may end up with a product that disappearsimmediately after its inclusion in the sample. We are thenfaced with a replacement problem, with its own biasrisks. Against this, it may happen that short-lived pro-ducts have a different price movement from the pricemovement of long-lived ones and constitute a significantpart of the market, so leaving them out will create bias.

5.31 A probability sample with respect to the baseperiod is not a proper probability sample with respect tothe current period. This argument anticipates some of thediscussion in Chapter 8 below. It is certainly true thatthe bias protection offered by probability sampling is toa large extent destroyed by the need for non-probabil-istic replacements later on.

5.32 Price collection must take place where there areprice collectors. This argument applies to geographicalsampling only. It is, of course, cheaper to collect pricesnear the homes of the price collectors, and it would bedifficult and expensive to recruit and dismiss price col-lectors each time a new sample is drawn. This problemcan be reduced by having good coverage of the countryin terms of price collectors. One way to achieve this isto have a professional and geographically distributedinterviewer organization within the national statisticalagency, which works on many surveys at the same time.Another way of reducing the problem is to have a first-stage sample of regions or cities or locations whichchanges only very slowly.

5.33 The sample size is too small. Stratification issometimes made so fine that there is room for only a verysmall sample in the final stratum. A random selection of1–5 units may sometimes result in a final sample that isfelt to be skewed or otherwise to have poor representa-tivity properties. Unless the index for this small stratumis to be publicly presented, however, the problem is also

small. The skewness of small low-level samples will evenout at higher levels. The argument that sample size is toosmall has a greater validity when it relates to first-stageclusters (geographical areas) that apply to most sub-sequent sampling levels simultaneously.

5.34 Sampling decisions have to be taken at a low levelin the organization. Unless price collectors are well versedin statistics, it may be difficult for them to performprobability sampling on site. Such sampling would benecessary if the product specification that has been pro-vided centrally covers more than one product (price)in an outlet. Nevertheless, in the United States (U.S.BLS, 1997) field representatives do exactly this. In Swe-den, where central product sampling (for daily neces-sities) is carried to the point of specifying well-definedvarieties and package sizes, no sampling in the outlets isneeded. In countries where neither of these possibilities isat hand, full probability sampling for products would bemore difficult.

5.35 In some situations, there are thus valid reasonsfor using non-probability techniques. We discuss twosuch techniques below.

Cut-off sampling5.36 Cut-off sampling refers to the practice of

choosing the n largest sampling units with certainty andgiving the rest a zero chance of inclusion. In this context,the term ‘‘largeness’’ relates to some measure of size thatis highly correlated with the target variable. The word‘‘cut-off ’’ refers to the borderline value between theincluded and the excluded units.

5.37 In general, sampling theory tells us that cut-offsampling does not produce unbiased estimators (seeparagraphs 5.51 to 5.60 below for a discussion of biasand variance), since the small units may display pricemovements which systematically differ from those of thelarger units. Stratification by size or pps sampling alsohas the advantage of including the largest units withcertainty while still giving all units a non-zero prob-ability of inclusion.

5.38 If the error criterion is not minimal bias butminimal mean square error (=variance+squared bias)then, since any estimator from cut-off sampling has zerovariance, cut-off sampling might be a good choice wherethe variance reduction more than offsets the introduc-tion of a small bias. De Haan, Opperdoes and Schut(1999) demonstrate that this may indeed be the case forsome item groups.

5.39 Often, in a multi-stage sampling design there isroom for only a very small number of units at a certainstage. Measurement difficulties that are sometimes asso-ciated with small units may then be a reason, in additionto large variances, for limiting price collection to thelargest units.

5.40 Note that a hybrid design can also be applied inwhich there is a certainty stratum part, some probabilitysampling strata and a low cut-off point below whichno sample at all is drawn. In practice, this design isvery often used where the ‘‘below cut-off section’’ of theuniverse is considered insignificant and perhaps difficultto measure.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

72

5.41 A particular CPI practice that is akin to cut-offsampling is for the price collector to select the most soldproduct in an outlet, within a centrally defined specifi-cation. In this case, the sample size is one (in each outlet)and the cut-off rule is judgemental rather than exact,since exact size measures are only rarely available. In allcases of size-dependent sampling in an outlet, it is cru-cial to take a long-term view of size, so that temporarilylarge sales during a short period of reduced prices arenot taken as a size measure. Such products will tend toincrease in price in the immediate future much morethan the product group which they represent and thuscreate a serious overestimating bias.

Quota sampling5.42 Many product groups, even rather small ones,

are quite heterogeneous in nature, and the price variesaccording to a large number of subgroups or char-acteristics. There may well be different price movementsgoing on within such a product group, and a procedureto represent the group by just one or a few tightly spe-cified product types may then carry an unnecessarilygreat risk of bias.5.43 The definition of quota sampling is that the

selected sample shall have the same proportions of unitsas the universe with respect to a number of knowncharacteristics, such as product subgroup, type of outlet,and location. The actual selection of sampling units isthen done by judgemental procedures in such a mannerthat the composition of the final sample meets the quotacriteria.5.44 The following example illustrates the concept of

quota sampling. A sample of 20 package holidays isdesired. It is known that, in the universe, 60 per cent ofthe holidays are to Spain, 30 per cent to Greece, and 10per cent to Portugal. Of the travel groups, 70 per centcomprise 2 adults, 20 per cent comprise 2 adults+1child, and 10 per cent comprise 2 adults+2 children. Ofthe sample, 20 per cent stay in 2-star hotels, 40 per cent in3-star hotels, 30 per cent in 4-star hotels, and 10 per centin 5-star hotels. With this information, it is possible todesign the sample purposively so that all these propor-tions are retained in the sample, which then becomes self-weighted. Note that these proportions reflect volumes,not values, and may need to be adjusted depending onthe elementary aggregate formula used.5.45 Quota sampling requires central management

of the whole sampling process, which may limit its use-fulness in some situations. It is more difficult, but notimpossible, to manage a quota sampling system wherelocal price collection is used. One would then need todivide the price collectors into subgroups with somewhatdifferent instructions for selecting products. A limitationof quota sampling, as in other non-probability sampling,is that the standard error of the estimate cannot bedetermined.

The representative item method5.46 This is the traditional CPI method. The central

office draws up a list of product types, with product-typespecifications. These specifications may be tight, in that

they narrowly prescribe for the price collectors whatproducts they are permitted to select, or they may beloose, giving the price collector freedom to choose locallypopular varieties.

5.47 The method with tight specifications is in asense diametrically opposite to the quota samplingmethod discussed above. Unless the product groups aredefined so as to include a very large number of producttypes, representativity will suffer in this procedure, sinceno products falling outside the specification will enterthe index. Another disadvantage with the method is thatit may lead to more missing products in the outlets andthus reduce the effective sample. Its main advantage issimplicity. It is easy to maintain a central control overthe sample. If quality adjustments are needed, they canbe decided in the central office, which may or may not bean advantage.

5.48 The method with loose specifications gives pricecollectors the chance to adjust the sample to local con-ditions and will normally lead to greater representativityof the sample as a whole. Where it is combined with the‘‘most sold’’ criterion it will, however, systematicallyunderrepresent the smaller brands and products thatmay be bought by important minorities.

Sampling in time5.49 A CPI usually refers to a month, during which

prices are not constant. The issue of sampling in timethen arises. Often, this problem is ignored, for exampleby using the 15th day of the month, or the days sur-rounding the 15th, as the target date for price measure-ment. In some areas, there is a day-of-the-week effect onprices, for example in cinemas, theatres and restaurants,but this may be taken into account in the product spe-cification rather than in sampling, for example by spe-cifying a weekday evening price.

5.50 As far as is known, random sampling in time isnot used anywhere. The method used in some countriesis to spread price collection over several weeks accordingto some pattern, for example different weeks in differentregions or for different product groups. In some cases,more frequent pricing than monthly is also used, forexample for fresh produce. There is not yet any sys-tematic knowledge about the pros and cons of suchpractices. Chapter 6 discusses the more practical aspectsof distributing price collection over time.

Choice of sampling method5.51 In this section, we discuss how choices in sam-

pling method could depend on specific factors in acountry. But first we consider the matter of sample size.

5.52 Sample size. The final precision of a sampleestimate depends only on the size and allocation of thesample and not on the size of the country, so in thissense there is no need for a larger sample in a largercountry. Larger samples are called for if regional dif-ferences in price change are of interest and if the amountof product disaggregation that is desired in presentingthe indices is very high. Of course, the budget allocated

SAMPLING

73

to CPI work may be larger in large countries, allowingfor larger samples.

5.53 Studies of bias (not the sampling bias describedin paragraphs 5.61 to 5.64) and of sampling error showthat bias in CPIs is generally a much greater problemthan sampling error. This leads to the conclusion that, inmany cases, smaller samples that are better monitoredwith respect to replacements, resampling and qualityadjustment could give a higher quality index for thesame budget. In some countries, local price collection isa fixed resource and it is therefore difficult to moveresources from local price collection to central analyticalwork. Still, it is advisable to try to use local resources forhigher quality price collection rather than just for manyobservations. The quality of price collection is furtherdiscussed in Chapter 6.

5.54 Monthly sample sizes in different countriesseem to vary from several thousand to several hundredthousand. Often, the reasons for these differences liemore in tradition than in a rational analysis of the needsof precision. Countries with very large sample sizeswould probably do well to look at ways of reallocatingtheir total resources.

5.55 Geographical distribution of price collectors.Sampling is more expensive further away from thehomes of the price collectors. If the organization forprice collection is centralized in a few main cities, it willbe difficult to sample outlets elsewhere. It should beborne in mind, however, that rural and urban inflationmay well be different, so failure to collect prices in bothrural and urban areas would be detrimental to efforts toachieve the best measure of average national inflation. Itwould be better to have at least a small sample in therural areas so that this factor can be taken into account.The major part of the saving arising from allocatingoutlets close to price collectors can then still be realized.

5.56 Sophistication of price collectors. If price col-lectors are well educated, they may be instructed tocarry out more complex sampling schemes such as ppssampling in the outlets. Otherwise, simpler methods arecalled for.

5.57 Access to sampling expertise in the central office.Probability sampling requires access to methodologicalexpertise in the central statistical office.

5.58 Homogeneous versus heterogeneous productgroups. The representative item method is more suitablefor homogeneous product groups. In heterogenousgroups, it is more likely that important segments of theproduct universe, with different price movement, will beleft out.

5.59 Access to sampling frames and their quality.Probability sampling requires sampling frames. But theydo not necessarily have to be available at the nationallevel. By applying geographical cluster sampling in thefirst stage (where the sampling frame is just a map), a listof relevant outlets can be constructed in each sampledcluster using telephone directories or local enumeration,as is done in the United Kingdom. This method is alsoused to select urban areas for the United States CPI(Dippo and Jacobs, 1983).

5.60 Scanner data. The discussion in this chapter isbased on the traditional situation, where prices have to

be collected locally and centrally, and entered indivi-dually into a central database. Where prices and possi-bly quantities are collected electronically, as is the casewith sale point scanner data, sampling could be differ-ent. There is then no need for sampling products orvarieties or points in time, since they are completelyenumerated automatically. Nevertheless, not all outletsselling a product will be covered by scanner data in theforeseeable future. Since all kinds of outlets should berepresented in the index, there will be continue to be aneed to combine scanner data samples with traditionalsamples for non-scanner outlets.

Estimation procedures5.61 There is a crucial distinction to be made

between what is to be estimated, the parameter, which isdefined for the whole universe, and the estimator, whichis a formula to be calculated using the sample values asan estimate of the parameter. Now, in survey samplingin general we want to estimate a population total or afunction of several such totals, for example a ratio oftotals. So, if we have two variables y and z defined foreach sampling unit (for example, prices at two differentperiods), we may want to estimate the following pa-rameters:

Y=PNj=1

yj and Z=PNj=1

zj or R=Y=Z

5.62 Several different estimators may be proposedfor the same population parameter, in which case weneed to decide which of these estimators to use. Inassessing the quality of a sample estimator, i.e. how wellit estimates the parameter, two measures are often con-sidered in the probability sampling paradigm. The firstmeasure is the bias of an estimator, which is the differ-ence between the universe parameter and the average ofthe estimator over all possible samples that could bedrawn under the specified sample design (referred to asthe mean of the sampling distribution of the estimator).Note that this bias refers to something different from theindex number bias discussed elsewhere in this manual.An estimator is unbiased if it has zero bias. The secondmeasure is the variance of the estimator with respect tothis sampling distribution. An estimator is consideredgood if both its bias and variance are small; that is, theestimator is on average very close to the parameter anddoes not vary much from its mean.

5.63 The good fortune of finding an estimator thatminimizes both bias and variance at the same time doesnot often happen. An estimator with a small bias mayhave a large variance, and one with a small variance mayhave a large bias. So use is frequently made of a criterioncalled the mean square error, which is the sum of thebias squared and the variance. A ‘‘good’’ estimator isthen one which minimizes this criterion.

5.64 Sampling theory tells us that the followingestimators are unbiased, respectively, for the parameters

Y and Z above: YY=Pj 2 Syj=pj, ZZ=

Pj 2 Szj=pj, where S

is the sample, and that RR=YY=ZZ is approximately

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

74

unbiased for R, subject to a (usually negligible) technicalratio estimator bias.

Implementing estimationprocedures for consumerprice indices5.65 As stated earlier, sampling for CPIs is usually

stratified, with elementary aggregates as strata. Let usassume that the universe parameter is I and that theparameter in stratum h is labelled Ih. Then we have:

I=Ph

whIh

where wh is the weight of stratum h. The issue then is toestimate Ih for each stratum. In the following discussion,we therefore concentrate on estimating for a singlestratum and drop the subscript h.5.66 Depending on the content, degree of homo-

geneity, price elasticity and access to weighting infor-mation within the stratum, different parameters may beappropriate in different strata. The choice of parameteris an index number problem, to be solved by referenceto the underlying economic concepts. As discussed inChapter 20, it could be the unit value index, the Laspeyresindex, the Lowe index, or the geometric Laspeyres index.5.67 Suppose we have a sample of size n and that the

units in the sample are labelled 1,2, . . . , n. Very often,one of the three formulae below is used as an estimatorof the stratum index:The arithmetic mean of price relatives (Carli index):

r=1

n

Pj 2 S

p1j

p0j(5.1)

The ratio of mean prices (Dutot index):

a=

1n

Pj 2 S

p1j

1n

Pj 2 S

p0j(5.2)

The geometric mean (Jevons index):

g=Yj 2 S

p1j

p0j

!1n

(5.3)

For discussion below, we also need to introduce theratio of harmonic mean prices:

h=

1n

Pj 2 S

1=p0j

1n

Pj 2 S

1=p1j(5.4)

5.68 When comparing the above estimators with thefunctional form of the parameters in Chapter 20, werealize that very special conditions are needed to makethem unbiased estimators of those parameters. For onething, unlike the parameters in Chapter 20, there are noquantities involved in the sample estimators.5.69 We state, without proof, some results concern-

ing the statistical properties of the above estimators (see

Balk (2002) for details). Suppose we have N products inthe universe labelled 1,2, . . . , N. Let ptj ; q

tj be respectively

the price and quantity for product j in period t (t=0 forbase period and 1 for current period), and let

w0j=q0j p

0jPN

j=1

q0j p0j

( j=1, . . . ,N)

be the base period expenditure share of product j. Then:

� Under simple random sampling, none of r, a or gestimates any of the population parameters withoutbias. Instead, weights need to be used in the estima-tors also.

� Under pps, if pj / w0j for all j, then r, the average ofrelatives, is unbiased for the Laspeyres index (thesymbol ‘‘/’’ means ‘‘proportional to’’).

� Under pps, if pj / q0j for all j, then a, the ratio ofaverages, is approximately unbiased for the Laspeyresindex.

� Under pps, if pj / w0j for all j, then g is approximatelyunbiased for the geometric Laspeyres index. In thiscase log g is unbiased for the logarithm of the geo-metric Laspeyres index. The remaining bias tends tobe of a similar order to that of a.

5.70 All these results are somewhat theoretical innature since neither w0j nor q

0j are known at the time

when the sample could be drawn. This is a reason forintroducing the Lowe index:

� Under pps, if pj / qbj (where b is some period before 0)for all j, then a is approximately unbiased for theLowe index.

5.71 There is no simple way to relate any of theestimators to the unit value index. In fact, estimatingthat index requires separate samples in the two timeperiods, since its numerator and denominator refer todifferent universes.

� Under two separate sample designs, one for period 0and one for period 1, which are both pps and wherep0j / q0j and p1j / q1j , then a is approximately unbiasedfor the unit value index. In this case, however, theinterpretation of the a formula will be different, sincethe samples in the numerator and the denominator aredifferent.

� Under two separate sample designs, one for period 0and one for period 1, which are both pps and wherep0j / v0j=p0j q0j and p1j / v1j=p1j q1j , then h, the ratio ofharmonic mean prices, is approximately unbiased forthe unit value index. The following algebraic refor-mulation of the unit value index helps to clarify thatfact:

UV=

Pj 2 S

v1j. Pj 2 S

v1j =p1jP

j 2 Sv0j. Pj 2 S

v0j =p0j

:

As for a, however, the interpretation of the h formulawill be different, since the samples in the numerator andthe denominator are different.

SAMPLING

75

5.72 The phrase ‘‘approximately unbiased’’ needssome explanation. It refers to the fact that the estimatoris not exactly unbiased but that the bias is small anddecreases towards zero as the sample size and the size ofthe universe simultaneously go to infinity in a certain,mathematically well-defined manner. In the ratio esti-mator case applicable to a, the sign of this bias is in-determinate and its size after aggregation is probablynegligible. In the case of the geometric mean, however,the bias is always positive, i.e. the sample geometric meantends to overestimate the universe geometric mean onaverage over many sample drawings. In the case of sim-ple random sampling and an unweighted geometric meanin both the universe and the sample, the bias expressionis: b&s2/2n, where s2 is the variance of the price ratios.For small universes, a finite population correction needsto be multiplied to this expression. This result is easilyderived from expression (4.1.4) in Dalen (1999b). Thebias may be significant for small sample sizes, so that acaution against very small samples in a stratum may bewarranted when the geometric mean is applied.

Variance estimation5.73 A CPI is a complex statistic, usually with a

complex design. It is thus not a routine task to estimatethe variance of a CPI. To the extent that samples are notprobability based, variance estimates need to make useof some kind of model in which random sampling isassumed. In the absence of systematic and generallyaccepted knowledge, the approaches to variance esti-mation used in four countries will be briefly described.

Variances of elementary index formulae5.74 As a preliminary, some variance estimators for

elementary aggregate formulae will be provided. In ordernot to overburden the text with formulae, the varianceestimators, not the exact variance, will be given. Thevariance estimators are approximately unbiased undersimple random sampling, where the corresponding uni-verse parameter is unweighted. They are also applicableto the case of pps sampling for a weighted universeparameter, where the size measure is the same as theparameter weight. For definitions of the formulae, seeequations (5.1)–(5.3).

V(r)=s2rn, where s2r=

1

n� 1Pj 2 S

(rj � r)2

and rj=p1j

p0j; (5.5)

V(a)=1

n( �pp 0)2(s21+r

2s20 � 2rs01), (5.6)

where s21=1n�1Pj 2 S ( p

1j � �pp1)2, s20=

1n�1Pj 2 S ( p

0j� �pp 0)2,

s01= 1n�1Pj 2 S ( p

1j � �pp1)( p0j � �pp 0),

�pp1= 1n

Pj 2 S p

1j and �pp 0= 1

n

Pj 2 S p

0j .

This estimate follows from the fact that a, unlike r, is aratio of stochastic variables. See, for example, Cochran(1977) for a derivation of this formula.

5.75 The geometric mean is more complex, since it isnot a linear estimator. However, Dalen (1999b) derivedthe following easily applied variance expression, whichholds with good approximation if price ratios do nothave too extreme variation (sr/r<0.2, say):

V(g)=s2rn

1� s2rr2

� �(5.7)

The United States approach5.76 The United States CPI uses sampling and esti-

mation procedures which are in many ways unique incomparison to those of other countries. The exact designobviously varies somewhat over time. The followingdescription is based on U.S. BLS (1997) and Leaver andValliant (1995).

5.77 The United States CPI is composed of buildingblocks consisting of geographical areas crossed withproduct strata to a total of 8,487 ‘‘basic CPI strata’’corresponding to elementary aggregates. The 88 geo-graphical areas were selected by pps in a controlledselection procedure and 29 of them were included withcertainty (self-representing). Within each basic CPIstratum an estimation procedure is applied in whichindices for a particular time period are based on theoverlapping sample units (outlets and items) between thistime period and the immediately preceding period. Theperiod-to-period indices are then multiplied to obtain anindex from the base period to the current period. Sam-pling within the basic CPI strata is approximately ppsaccording to the description above.

5.78 Variance estimation for this design proves to betoo complex for the use of a direct design-based varianceestimator. Instead a random group replication method,using the so-called VPLX software, is applied. Othermethods have also been tried.

5.79 Leaver and Swanson (1992) provide a detailedaccount of the variance estimation methods used up tothen. They also present the following numerical esti-mates of (median) standard errors for CPI changes forvarious intervals over the 1987–91 period: 1 month�standard error 0.074; 2 months� standard error 0.103; 6months� standard error 0.130; and 12 months�standard error 0.143.

The Swedish approach5.80 The following outline summarizes the descrip-

tion given by Dalen and Ohlsson (1995). The SwedishCPI uses a primary stratification into product groups,which are measured in separate and independent pricesurveys. The first step in the Swedish approach is there-fore to note that the variance of the all items price indexis a weighted sum of the variances of the separate sur-veys:

V(I)=Ph

w2hV(Ih) (5.8)

5.81 The reason that all these surveys can reasonablybe assumed to be independent is that there is no commonregional sampling scheme used in them. Altogether, there

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

76

are about 60 different surveys. Some of them cover manyproduct groups and have a complex design and there isstochastic dependence between them. Other surveyscover only one group and have simple designs. Somecover their universes, without any sampling, so they havezero variance.5.82 In many simple product groups it is fairly rea-

sonable to assume that the price ratios obtained areeffectively random samples. In some cases this may leadto some overestimation of variance since there is in factsome substratification or quota sampling within thegroup. In those product groups, stratum variances couldthen be estimated according to formulae (5.5)–(5.7).When a price survey is stratified, formula (5.8) can beapplied at lower levels above the elementary aggregate.5.83 Some price surveys are more complex, how-

ever. This is especially the case for that large part of theindex where outlets and products are simultaneouslysampled. In the Swedish case these surveys are called thelocal price survey and the daily necessities survey. Inboth these cases, outlets are sampled by probability(pps) from the central business register. Products aresampled by pps in the daily necessities survey but by therepresentative item method in the local price survey. Inthe Swedish variance estimation model, the final sampleis in these cases considered as drawn from a two-dimensional universe of products and outlets. The finalsampling units are thus sampled products sold in sam-pled outlets – a cross-classified sample.5.84 In a cross-classified sample, the total variance

can be decomposed into three parts:

– variance between products (in the same outlet);

– variance between outlets (for the same product);

– outlet and product interaction variance.

Dalen and Ohlsson (1995) provide the exact formulaeused.5.85 In the daily necessities survey, the cross-classi-

fied model comes fairly close to the actual samplingdesign. In the local price survey, it is more of a model,since the products are in fact purposively drawn. It hasnevertheless been considered a useful model for thepurpose of getting a first idea of the sampling error andfor analysing allocation problems.5.86 The total variance of the Swedish CPI, accord-

ing to this model, was estimated to be 0.04, corre-sponding to a 95 per cent confidence interval of ±0.4.This estimate appeared to be fairly stable over the period1991–95 for which the model was tried.

The French approach5.87 In France, variance calculation at present only

takes into consideration items accounting for 65 per centof the total weight of the index.5.88 The smallest element of the calculation is a

product type in an urban area. For these elements one oftwo formulae are applied, depending on whether theproduct is homogeneous (ratio of arithmetic means) orheterogeneous (geometric means). A two-stage randomsample is assumed, first of urban areas and then of aparticular item (variety) in an outlet. The variance

obtained is thus the sum of a ‘‘between urban areas’’ anda ‘‘within urban areas’’ component. Linearization basedon second-degree expansions is done, because of thenon-linear nature of the estimators. Higher-level vari-ances are obtained by weighting the elementary levelvariances.

5.89 After an optimization exercise which took placein 1997, the standard deviation of the all-products index(for 65 per cent of the total weight of the index) wascomputed as 0.03. This value is close to that estimated in1993, although the number of observations was reduced.The precision of a number of sub-indices was alsoimproved.

5.90 Covariance terms are ignored. In fact, thismakes a very small difference in the ‘‘between urbanareas’’ component. In the ‘‘within urban areas’’ compo-nent it has undoubtedly a greater influence. The effect is,however, seen as limited because of a rule which limitsthe number of products observed in the same outlet.

5.91 For the 35 per cent of the weight that is atpresent excluded from the variance calculation (calledthe ‘‘tariffs’’), such calculations will be introducedfor insurance. The necessary elements for variance cal-culation are also present for physicians’ and dentists’services. Variances will soon be calculated for theseproducts, as well as for new cars. For a certain number ofsub-indices (tobacco and pharmaceuticals) the sample isin effect a total count. Their variances are thus zero.

5.92 A 95 per cent confidence interval for a 12-monthcomparison can be expressed as the estimated index±0.06 for the ordinary, non-tariff items. If zero varianceis assumed for the remaining 35 per cent of the index, theconfidence interval for the all-products index wouldbecome±0.04. This assumption is clearly too optimistic,but from the work on variance estimation done so far, itcan be concluded that the confidence interval is certainlysmaller than 0.1.

5.93 More details on the French computations canbe found in Ardilly and Guglielmetti (1993).

The Luxembourg approach5.94 The Luxembourg CPI can be described as a

stratified purposive sample with 258 product strata.There are slightly fewer than 7,000 observations eachmonth, giving an average of 27 observations per stra-tum. In each stratum, observations are taken from sev-eral different outlets; but the same outlet is representedin many product strata. The outlet is here used as theidentifier for the price-setting organization (for rents it isa landlord, for insurance it is the companies, and so on).In each stratum, there are observations from severaloutlets. Since there is good reason to believe that eachoutlet has its own price-setting behaviour, prices andprice changes in the same outlet tend to be correlated,resulting in positive covariances in the general varianceexpression:

V(I)=Pkw

2kV(Ik)+

Pk

PlwkwlCov(Ik; Il) (5.9)

5.95 In the sampling model, each separate outletsample within a product stratum is regarded as a simplerandom sample. Further, a two-stage model was

SAMPLING

77

assumed such that, in the first stage, a simple randomsample of outlets was assumed to have been drawn froma (fictitious) sampling frame of all outlets in Lux-embourg. Then, in each sampled outlet, a second-stagesample of observations was assumed to be drawn inproduct stratum h so that the combined product–outletstratum became the lowest computational level inthe index. All second-stage samples were assumed to bemutually independent and sampling fractions to besmall. This model resulted in three components of totalvariance:

� variance within outlets;� variance between outlets;� covariance between outlets.Covariances are difficult to calculate, even with a com-puter. Luckily, however, it was possible algebraically tocombine the last two components into one, with thenumber of summation levels reduced.

5.96 Numerical estimates were made with this modelfor 22 consecutive 12-month changes starting from theperiod January 1996 to January 1997 and ending withthe period October 1997 to October 1998. The averagevariance estimate was 0.02 (corresponding to a standarderror of 0.14), which is surprisingly small given the smallsample size. The reason for this small value was notexplored in detail but may lie in a combination of thespecial circumstances in the markets in Luxembourg andin procedures used in the index estimation system.

5.97 The full variance estimation model for theLuxembourg CPI and the results from it are presented inDalen and Muelteel (1998).

Other approaches5.98 A number of experimental models have been

tried out and calculations done for the United Kingdom.None of them has so far been acknowledged as anofficial method or estimate. Kenny (1995 and earlierreports) experimented with the Swedish approach onUnited Kingdom data. He found a standard error of theUnited Kingdom Retail Price Index as a whole ofaround 0.1, which was reasonably constant over severalyears, although the detailed composition of the variancevaried quite a lot. Sitter and Balshaw (1998) used apseudo-population approach but did not present anyoverall variance estimates.

5.99 For Finland, Jacobsen (1997) provided partialcalculations according to a similar design as in theSwedish approach. His analysis was used to suggestchanges in the allocation of the sample.

Optimal allocation5.100 Producing a consumer price index is a major

operation in any country and a great deal of resourcesare spent on price collection. It is therefore worthwhileto devote some effort to allocating these resources in themost efficient way.

5.101 The general approach to sample allocationwas established by Neyman and is described in anysampling textbook. It uses a mathematical expression

for the variance of the estimate and another expressionfor the cost. Both variance and cost are functions ofsample size. Optimal allocation then amounts to mini-mizing variance for a given cost or minimizing cost for agiven variance.

5.102 Variance estimation was discussed above. Asfor cost, it is important to note that not all priceobservations are equally costly. It is less expensive tocollect an extra price in an outlet that is already in thesample than to add a price in an outlet that is new to thesample. For example, in the Swedish CPI, the followingcost function was used:

C=C0 þPhnh ah+bh

Pgmgrgh

n o(5.10)

where C refers to total cost and C0 to the fixed part ofthe cost that is independent of sample size,

nh is the number of outlets in outlet stratum h,

mg is the number of product varieties in product stratum g,

ah is the unit cost per outlet and reflects travelling timeto the outlet,

bh is the unit cost per product, which reflects the addi-tional cost for observing a product, when the pricecollector is already in the outlet,

rgh is the average relative frequency of products instratum g sold in outlets of stratum h.

5.103 In formula (5.10), ah is usually much largerthan bh. This fact calls for an allocation with relativelymore products than outlets, i.e. of several products peroutlet. This allocation is further reinforced to the extentthat variances between products in the same outlet andproduct stratum are usually larger than variancesbetween outlets for the same product. This is the case, atleast according to experience in Sweden.

5.104 With a specified variance function and a spe-cified cost function, it is possible, using the mathematicaltechnique of Lagrange multipliers, to derive optimalsample sizes in each stratum. It is usually not possibleto obtain explicit expressions, however, since we run intoa non-linear optimization problem for which it is notpossible to find an explicit solution.

5.105 In a CPI, the all-products index is usually themost important statistic. Therefore, the allocation of thesample should be directed towards the minimization ofits error. It is also important that other published sub-indices are of high quality, but the sub-index quality canoften be taken as the criterion for publication, ratherthan the other way round.

Summary5.106 The above discussion can be summarized in

the form of a small number of specific recommenda-tions.

5.107 Clarity – sampling rules should be welldefined. In many CPIs, there is a wide range of samplingand other solutions for different product groups. Afairly well-defined method is often used for the fieldcollection of prices, but the exact methods used for thecentral price collection of many products are commonlyin the hands of one or a few responsible persons and are

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

78

sometimes poorly documented. It is essential for thebasic credibility of the CPI that rules for sampling andestimation (e.g. the treatment of outliers) are welldefined and described.5.108 Probability sampling should be seriously con-

sidered. The use of probability sampling designs shouldbe increased. In many areas, useful sampling frames doexist or could be constructed without excessive difficul-ties. Stratified, order pps sampling is an important typeof design that ought to be considered in many situations.Size measures used for sampling must have a long-terminterpretation, so that they are uncorrelated with pricemovements.5.109 Representativity – no large part of the universe

should be left out. When sampling designs are planned,the full universe of items and outlets belonging to theitem group in question should be taken into account. Allsignificant parts of that universe should be appropriately

represented, unless there are excessive costs or estima-tion problems involved in doing so.

5.110 Variance or mean square error should be as lowas possible. Samples should be reasonably optimized,based on at least a rudimentary analysis of samplingvariance. As a first-order approximation, sample sizescould be set approximately proportional to the weightsof the commodity groups. A better approximation isobtained by multiplying each weight by a measure ofprice change dispersion in the group. Variance and costconsiderations together call for allocations where rela-tively many products are measured per outlet and rela-tively few outlets are contained in the sample. Sincebiases are generally a greater problem than samplingerrors, smaller but better samples, allowing for morefrequent renewal and careful monitoring of replace-ments and quality adjustments, generally make goodsense.

SAMPLING

79

6PRICE COLLECTION

Introduction6.1 The most appropriate sampling and survey

methods for a price survey will vary depending on theuse of the price index and local circumstances. Forinstance, the diversity of available goods and services,their turnover and the range of prices charged, the fre-quency and size of price changes, consumers’ purchasinghabits (including the use of telephone, catalogue andInternet shopping) and the structure of retailing in termsof the local economy, types of outlets and geographicalspread will all have a bearing.6.2 This chapter gives an overview of some of the

issues, but bearing the above in mind it is clear that thetreatment of these will require different solutions indifferent countries according to local circumstances.Solutions cannot be too prescriptive, and the compilershould always be guided by the fundamental principlesand objectives of a price index as addressed in earlierchapters. The structure of Western economies togetherwith the retailing patterns and associated consumerpurchasing habits lend themselves to more structuredprice collections. In contrast, subsistence economies anddeveloping economies will require more flexible pricecollection techniques.6.3 Consideration has to be given as to how best to

collect prices in terms of efficiency, accuracy andrepresentation of consumers’ purchasing patterns. Insome cases, price collection directly from individualshops around the region or country (local pricecollection) may be considered appropriate. In othercircumstances, prices collected centrally by stafflocated in the headquarters or regional offices of thenational statistical institute (central price collection)may be more appropriate. Many of the issues covered inthis chapter are relevant to both local and central pricecollection.6.4 The advantages and disadvantages of local ver-

sus central price collection for different types of pricesare covered later in this chapter. Briefly, local pricecollection has the advantage of covering a wide range oflocations and item selections, particularly for food,alcohol, tobacco and durable goods (for example,clothing, furniture and electrical goods). Central col-lection is useful for prices that are difficult to observedirectly (for example, costs of housing or utilities),where there are national pricing policies, for goods soldthrough mail order and catalogues, or for items wherethere are limited collection opportunities or difficultiesin making an adjustment for technical and qualitycharges (particularly transport and services).

Frequency and timing of collection6.5 The type of economy can initially govern the

frequency and timing of price collection. Where tran-sient markets are important to a wide spectrum of thepopulation, the timing of these markets will affect thetiming of price collecting because of the need to considerthe availability of goods and services to consumers.

6.6 A fundamental decision about the frequency andtiming of price collection is whether the index shouldrelate to monthly average prices or prices for a specificpoint in time (for example, a single day or week in amonth). This decision is related to a number of factors,including the uses of the index, the practicalities ofcarrying out price collections, the pattern of pricemovements and the timing of index publication. Thesefactors are discussed in turn below.

6.7 It has been argued that the question of whetherthe index should relate to a period or to a point in timegenerally becomes less of an issue the more frequently theprices are collected. But it is far from certain whether thisis true in all circumstances. For instance, prices overcertain holiday periods or at particular times of yearmight be especially volatile. In such cases, the smooth-ness resulting from a period rather than a point estimatemay be considered an advantage over a potentiallymisleading short-term trend shown by a more highlyvolatile point estimate. In answering this question, thereis also a need to take into account the primary use of theindex.

6.8 In principle, if used for deflating income, expen-diture or sales, the index should relate to the periodof time of these money flows. For economic analysis,where the index will be used in conjunction with othereconomic statistics, most of which relate to a periodrather than to a point in time, it seems logical, againin principle, that the consumer price index should dothe same.

6.9 In reality, when making this choice, consider-ations of principle have to be weighed against variouspractical considerations. The first point to note is thatwhen inflation is low and stable there will be little dif-ference between, for example, the annual rate of changein the index from Monday 3 January 2000 to Wednesday1 January 2001 and the corresponding annual rate ofchange between the complete months of January 2000and January 2001. This will not be the case if inflation israpid or the rate changes significantly during the year.The difference between 1 January and 1 February andaverage January to average February inflation rates maybe different – particularly if so-called ‘‘sale’’ periods are

81

limited by laws or local ordinance, as they are in somecountries. For certain products with high index weights,where price changes are sudden and tend to affect thewhole market on about the same day, the choice betweentime period and point is important. Examples are petrol,electricity and telecommunication prices. Here the casefor an average price for the period is strong. Obviously,the weights should relate to the periodicity of the col-lection taking into account the appropriate expenditureand pricing periods (for example, if prices are increasedone-third of the way through the period then two-thirdsof the weight should reflect the higher pricing).

6.10 Not all price observations can be made within asingle day, let alone at one point in time during that day.This is particularly true of local price collections, butmay also be true of central price collections, dependingon the resources available at head office. In practice, thereal issue is whether the observations are spread over afew days to provide an approximation to a point-in-timeestimate (for example, Monday to Wednesday to repre-sent prices on a specific Tuesday) or spread over thewhole month to provide an estimate for the average forthat month.

6.11 It should also be borne in mind that the sam-pling variance will differ according to whether a periodor point-in-time index is compiled and, in the case of thelatter, the frequency of collection. In considering the tim-ing and frequency of price collection, consideration alsoneeds to be given more generally to the trade-off betweenstatistical accuracy and cost. It should be noted thatcollecting prices locally from shops is normally a rela-tively expensive activity. In practice, the budget for pricecollection usually limits the available options.

6.12 The desired frequency of price collection mayvary by commodity, depending on how frequently theprices to be observed change. For example, it is possiblethat prices charged by public utilities, central or localgovernment fees or charges, or mail order catalogueprices will change annually or quarterly according to aknown timetable, and therefore these price collectionscan be carried out according to the timetable of changesrather than every month. In contrast, food prices – whereshopkeepers may review the prices they charge on acontinuous basis to reflect market conditions and theprices charged by their suppliers – need to be collectedmore frequently. Clearly, statisticians will need to beabsolutely sure about the frequency of price changes forany specific good or service before coming to a decisionto collect prices on a less frequent basis. They will alsoneed to keep in touch with current pricing policies tomonitor whether the position changes, so that they canimmediately reflect these changes in their price collectionprocedures. In addition, statisticians need to be awareof any unusual price changes that could be missed byless frequent price collections, for example, changes toindirect taxation rates or one-off timing changes ofincreases (for example, service providers moving annualincreases from April to March or prices of school dinnerschanging each term, with terms starting in differentmonths from one year to the next).

6.13 Another point to note is the timing of pub-lication of the resulting price indices. There may be legal

constraints on the timing of the publication of indices.In such cases, prices must be collected in time to allowquality assurance, processing and aggregation pro-cedures to be completed before the deadline.

6.14 As mentioned above, in cases where inflation isstable and where collection costs are not an issue, col-lection can be spread over a whole month. In these cases,different neighbourhoods should be scheduled for pricecollection at different times of the month according to aregular pattern to be repeated every month. This notonly makes the use of the collectors’ time more efficient,but also has the advantage of providing a spread ofcollection dates for many representative items. It is alsoimportant that individual price observations are carriedout at the same time each month so that the index doesnot move as a result of a change in the length of intervalbetween collection dates. A further important con-sideration, particularly of consequence in Middle East-ern countries, is in cases where prices can vary by day ofthe week (for example, depending when market day is) ortime of the day according to various marketing specialoffers to attract more customers at less busy times or toreflect the freshness of the goods.

6.15 When the aim is to compute a point-in-timeindex, price observations need to be spread over a verysmall number of days each month. The interval betweenprice observations should be uniform for each outlet.Since the length of the month varies, this uniformity hasto be defined carefully.

6.16 Preferably, days of the week and times of themonth should be chosen taking into account whenpurchases are concentrated and where prices and goodsin stock are known to be representative of the month asa whole. In Middle Eastern countries, results of thehousehold expenditure surveys suggest that most house-holds do the shopping on the day of the souk (market).It should also be borne in mind, however, that retailersmay be less prepared to cooperate when they are busy,so a balance needs to be struck between the ideal timingfor collection and the impact on response rates. Itshould be noted that a fixed interval is impossiblebecause of the varying length of a month and the timingof public and religious holidays. One solution is to takesequences of four and five weeks, so maintaining arelatively stable monthly or quarterly period; another isto follow a rule such as collecting on the regular marketday or on Wednesday through to Friday of the first fullweek in the month.

6.17 Price collection days (and sometimes times)need to be set in advance. In some countries or econo-mies, decisions need to be taken in advance about whe-ther and how these days should be kept confidential toavoid key sources, such as major stores or governments,adjusting prices for collection days and thus distortingthe price indices. It is nevertheless important for publicperception about the integrity of the index that a statis-tical office is able to explain the procedures used forsetting collection dates and the underlying objectivity ofits method. Any price collection agencies carrying out thecollection for national statistical institutes need to knowcollection dates a long way in advance for resourceplanning purposes. In addition, any data suppliers who

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

82

supply prices direct to head office staff need to know thecollection date a short time in advance to be able toprepare and supply the necessary price returns.6.18 Regular timing is particularly important when

inflation is rapid. Where there is a specific collection day,it is most important that the most volatile prices arecollected on that specific day rather than the days aroundit. Particular items may include fresh fruit and vege-tables, fresh meat, and items subject to varying indirecttaxation and duties (such as tobacco and petrol). In thecase of foodstuffs sold in marketplaces, the time of day aswell as the day of the week is important. In MiddleEastern countries, at least, these prices are usually higherin the mornings and lower in the evenings.6.19 Price collection days need to be set after con-

sidering a variety of factors that affect prices andshopping patterns. Holidays and weekends should beavoided except for items with large sales during thesetimes, such as petrol, leisure services and entertainment(for example, restaurant meals and tourist attractions).Some countries have limited shop openings on somedays or half-day closing on other days, which can limitthe number of prices that can be collected or bias thelocation sample to certain types of outlets or serviceproviders. On the days approaching long holiday peri-ods when many shops are shut, there can be limitedsupplies of fresh food and many abnormal price reduc-tions to clear stocks before the shops shut for theholiday period. The implications of any sale periodscontrolled by law should also be considered.6.20 Whether collection is continuous or point-in-

time, the interval between successive price observationsat each outlet must be held constant by visiting that outletduring a fixed time period each month (or quarter).6.21 Another issue is raised by the pricing of tariffs

(telephone charges, for example, depend on the time ofday and the destination of the call), variable pricingpolicies dependent on demand (sporting and leisure fees,for example, depend on the time of day – peak demandtimes attracting higher prices) and prices where there ispotentially limited availability (such as air, rail and taxifares). For each of these, price collections should bemade consistently over time and in a way that representsconsumer purchasing patterns. The selection of therepresentative items should represent consumer behav-iour (for example, air fares may be priced 6, 3, 2 and 1month in advance and include last-minute bookingoptions too) and be weighted by consumer spendingpatterns (for example, weighting together prices for peakand off-peak entry to a swimming pool).6.22 A final point to note is that with the point-in-

time approach, major price setters, notably the govern-ment, can influence the index according to whether theirprice changes take effect on a day just before or after theday for which their price information is obtained, or onthe day of collection. Since prices are often collectedcentrally from such price setters, it should be possible toobtain information from them about both the amountand timing of price changes at the end of each month, sothat in applying the period-of-time approach, an aver-age price for the whole month can be calculated. Forexample, if electricity charges are made quarterly and

prices increase part way through the 3-month period,individual customers’ payments could include 0, 1, 2 or 3months at the higher rate.

Taking account of hyperinflation6.23 Special arrangements may need to be put into

place where there is hyperinflation. In these circum-stances it becomes even more important that the prices ofindividual items in individual shops are collected atprecisely the same time each month, otherwise mislead-ing figures may result. Consideration should be given tothe more frequent collection of prices and correspond-ingly a more frequent compilation of the index. Whereprices are normally collected on a quarterly basis, itmay be sensible to collect them more frequently. If this isnot feasible, it may be appropriate to up-rate pricesproportionately by some relevant indicator to providean approximation to a monthly index. If this is done,however, great care needs to be taken in choosingthe appropriate comparator, particularly as relativitiesbetween prices can change dramatically in periods ofhyperinflation.

6.24 In some circumstances, rapid or frequent pricechanges may be associated with certain items only andaction should be taken accordingly. By way of example,food prices may rise disproportionately because of a badharvest and it may be sensible to increase the frequencyof the index for food items only. Alternatively, a simplerway of dealing with this situation may be to monitor asmall number of relevant prices on a regular basiswithout producing a full price index. Such sub-indicescould be published separately or used to up-rate the laterprices collected in the period, as mentioned above. Theseitems may be chosen according to their importance forthe family budget and whether they are particularlysusceptible to big price increases.

Item specification6.25 Specific representative items should be chosen

to be typical of price movements in the consumer priceindex basket. An item consumed by households orindividuals that has a price is a definable good or ser-vice. In some cases, however, such as a la carte restau-rant meals, cars (where the purchaser may be able to buyoptional extras on top of the basic model) and car ren-tals (where insurance may be extra), a decision needs tobe taken about whether to treat the package as a singleitem or whether to price components separately. As ageneral rule, the package should be considered a singleitem when it can be expected that the offer is not tem-porary and where the purchaser typically buys the wholebundle of goods and services on offer. Otherwise, thecomponents should be treated as separate items andindividual cost quotes obtained. Where the purchase isnot normally of the whole bundle, it is usual to be ableto pick up individual quotes for the different parts of thepackage. This provides some indication of whetherpackages or individual items are being purchased.

6.26 Ideally, the selection of items should be basedupon a complete census of relevant transactions relating

PRICE COLLECTION

83

to different items purchased by individuals. In practice,this is not normally universally available, although insome countries useful data may be generated by point-of-sale and scanner data.

6.27 How tightly defined (narrow) or general (broad)the item specification should be is an issue of greattheoretical and practical importance. Whether specifi-cations are broad or narrow can vary according toindividual circumstances and may differ throughout thebasket of goods and services being priced. Narrow itemdescriptions are generally more effective for control-ling sample representativeness (assuming that a reliablesampling frame or set of reference data is available) andfor controlling for quality differences, and can alsoreduce the variance of prices and price relatives, thusoptimizing the performance of some aggregation for-mulae. But they can result in a smaller achieved sample,as there is less flexibility for price collectors to choosean appropriate item in a particular shop. In contrast,broad item descriptions can increase the size of theachieved sample but can be more difficult to controlfor sample representativity and will normally result inhigher variances.

6.28 In some countries, prices for clothing are speci-fied very tightly to ensure that quality differences areminimized. A description might be as detailed as ‘‘knit-ted top; mid-season; with sleeves; no collar; no buttons;made in Morocco; acrylic; mid to light thickness’’. Incomparison, a general description used in another con-sumer price index for an equivalent item might be ‘‘men’sformal shirt; long sleeved’’.

6.29 Whichever of the two approaches is used, rulesshould be established for selecting representative itemsthat fit the item descriptions (for example, best-sellinglines as reported by the individual retailer, or itemsselected by probability proportional to size sampling).It is important that the representative items chosen,whether with tight or general product descriptions, areactually representative of the consumer spending pat-terns. There is no point, for instance, in pricing an itemthat is rarely sold but looks good in the shop window, oris just in a convenient location for the collector to find iteach month. The rules for selection should also takeaccount of the sampling methodology underlying theselection of shops. There is a stronger argument forusing some form of probability sampling for the selec-tion of items using tight descriptions when the selectionof shops is more loosely specified, and vice versa. This isbecause the broader the item descriptions and the moreloosely controlled the item selection in the field, themore the representativity of the sample is reliant on thequality of the initial selection of shops.

6.30 It is also important under either specificationregime that instructions to price collectors give an ade-quate description of the item to be priced. For instance,in the case of a washing machine the informationrequired for a tight specification may include make,model number, capacity, whether automatic, whethertop or front loaded, and spin speed. As well as providingeffective sampling control, this will also be useful infor-mation if a price collector has to choose the nearestequivalent should the particular model cease to be

available. It is important that the number of pricesobtained for tightly specified goods or services are reg-ularly reviewed so that specifications can be updated ifthese items are being phased out or if consumer pur-chasing patterns are changing.

6.31 A loose specification may simply specify awashing machine with a particular range either for thecapacity or the spin speed. In this case it is still impor-tant that the collector records a detailed description ofthe washing machine being priced to enable the selectionof a comparable model if that model is discontinued orso that a future collector can carry out the price col-lection when the original collector is not available.

Collection procedures6.32 An important consideration in the collection of

prices is the scope of the price index being constructed.For example, should black market or contraband goodsbe priced as part of the price collection? In general, ifsuch purchases constitute a significant part of expendi-ture then there is an argument in principle that theyshould be considered for inclusion. This, however, leadsto price collection difficulties such as finding the neces-sary outlets, which may be transient and not advertisethemselves, as well as the actual pricing of goods andservices. Another difficulty regarding scope concernsactivities that are considered illegal in some countriesbut not in others (for example, prostitution, gambling orsales of alcohol).

6.33 The greatest difficulty in collecting prices arisesfor goods and services in economies where barteringplays an important part. Examples range from pricesfor cars, which can be individually negotiated (includingthe possibility of trading in an old car), to market stalls insome communities.Ultimately theprice obtaineddependson the likelihood of a real purchase, and the negotiatingskills of the price collector, as well as factors such as howdesperate the retailer is for a sale. Ideally the price col-lector should obtain the price that a consumer wouldactually pay. In some cases it may be appropriate to lookat alternative price collection methods or indicatorsinstead (such as the advertised price, which could beassumed to move in the same way as the bartered pricedepending on circumstances).

6.34 In some Middle Eastern countries where pricesvary according to the time of day and where prices arenot usually advertised (for example, in the souk), it isnecessary to employ a variety of collection procedures.Prices for fresh meat and vegetables may be collectedthree to six times in a day, including a morning,lunchtime and evening visit. In addition, collectors canbe trained to recognize ‘‘deceptive’’ prices and can beencouraged to linger and listen to transaction prices forgenuine sales.

6.35 Different collection procedures may be applic-able for different outlets. Permanent outlets can some-times be selected on the basis of a sampling frame eitherheld centrally or through local enumeration (see Chapter5). In the souk or marketplace it may be appropriate touse other collection procedures, particularly where the

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

84

opening times and the variety of stalls and goods for salevary at different times. In these cases, the item list may berestricted to items known to be available in the souk andprice collectors may be asked to obtain a fixed number ofprice quotes for each item, the number being determinedby local knowledge of the varieties on sale and the var-iation in price. Some items, such as fruit and vegetables,may warrant more quotes than others and prices mayneed to be collected at intervals throughout the day (forexample, three times in the morning, three times duringlunchtime and three times in the afternoon or evening) toensure that any variability in price according to time ofday is taken into account. Consideration may also needto be given to collecting from farmers (who travel to thesouk to sell their wares) and middlemen (who buy thefood from the farmers and then sell it on).6.36 Another difference between countries may be

where a significant proportion of expenditure occursabroad and then the purchased items are imported byindividuals (for example, car sale markets in Lithuaniaare frequented by the population of other Baltic States).Under such circumstances, price collections need to beconsidered in terms of the scope of the index (forexample, should prices in other countries be considered?)as well as the more complicated matter of pricing thesame or similar quality car each month.6.37 An overview of local price collection for

straightforward outlets is given in Figure 6.1. This dia-gram assumes that outlets have already been enumeratedand selected, that the shopkeeper or head office of achain has agreed that price collectors may visit on aregular basis, and that usual identification formalities onarrival and departure will have been carried out. Inaddition, it assumes that item selections have alreadybeen made in previous months. This is usually best doneon a separate pre-collection visit where the price collec-tors introduce themselves, familiarize themselves withthe shop, and explain the price collection procedure tothe shopkeeper.6.38 The diagram details different decisions and

actions that a price collector must make to price anyindividual item. The diagram starts with arrival of theprice collector at the outlet, at a mutually agreed time,which may or may not coincide with the usual shopopening times. Having gained entry to the outlet (or areplacement outlet), the collector attempts to price thenecessary item or items. In a straightforward situationthe item is immediately available for purchase andpriced. More complicated situations arise when the itemis different from the previous collection in some way(such as in size, description, weight, or quantity), inwhich case the usual procedure is to price the item andreport the facts to head office. Finally, if the item isunavailable, another item has to be selected as a repla-cement comparable item or a replacement new item.Having priced all the items required in that outlet, thecollector can move on to the next outlet.6.39 The choice of a comparable item is made using

the same bundle of key characteristics as that potentiallyaffecting the price. For example, the brand name, washcycles, capacity, energy consumption and spin speedmay affect the price of a washing machine.

6.40 The most complicated situation arises when adifferent item that is not of a comparable quality has tobe priced. How this is treated depends on the proceduresin place for adjusting prices for changes in quality. Forexample, quality changes may be treated implicitly byconsidering the item as a new item with an imputed baseprice. The latter may be calculated by head office staffwho may require supplementary information from theprice collection or by the price collector in the outletwith the assistance of sales staff.

6.41 Seasonal items require special attention. Insome situations, seasonal items such as fruit, vegetablesor clothing may not be available for pricing all yearround. One way of reflecting this in the index is to useseasonal weights, which differ for each month of theyear and reflect expenditure information from house-hold budget surveys or other sources. Alternatively,other seasonal items may be priced at different timesof the year to directly replace the unavailable items(for example, bathing costumes and shorts may bepriced for six months, and gloves and scarves for sixmonths).

6.42 One possibility for data collection is to collectsome items less often than monthly, thereby makingpossible a larger total sample. Many items in the UnitedStates Consumer Price Index (CPI) are collected onlybimonthly in any given area; similarly, the rent samplesare divided into six panels, each priced twice each year.It makes the calculation more complex but may be moreefficient from both a statistical standpoint and also forcollectors.

Price collection techniques6.43 For many items, prices will be collected locally

by price collection agencies employed by the nationalstatistical institute, or their own employees, visiting retailoutlets and recording current prices for an agreed selec-tion of items. But some prices may be collected centrallyfrom catalogues, by retailers providing list prices cover-ing a range of outlets, by telephone, fax, letter, emails orfrom Internet sites. All these methods may be cost-effective or necessary to represent different aspects ofconsumer purchasing behaviour and so it is not sur-prising that many statistical offices use a variety of datacollection techniques. In addition, such price collectionscan allow for the implementation of specific methodo-logical procedures by head office staff (for example,quality changes). Either local collection agents or headoffice staff can use these varying collection methods.Examples of price collection techniques include the fol-lowing:

� Prices may be obtained from mail catalogues torepresent a certain type of retail outlet, or where highstreet catalogue stores have nationwide coverage withuniform pricing policies. Increasingly in some coun-tries mail order suppliers are offering their own Inter-net services. In the case of both mail order and Internetshopping, care has to be taken to treat delivery pricesand sales taxes consistently and correctly.

� Prices may be obtained over the Internet either forconvenience (where major stores offer the same prices

PRICE COLLECTION

85

Is the outlet open? Is it permanently shut?Select anotheroutlet accordingto instructions.Report it.

Is the outlet willing tocooperate?

Is refusal because authorizedperson is not available?

Try again laterif theinstructions sayso, otherwisereport the facts.

Is the collectorauthorized toselect areplacementoutlet?

Is the product type tobe priced currentlyavailable to purchase?

Report thenon-availabilityand the reasonfor it.

If the variety is neitherseasonal nor expected tobe available to purchase again next month, then:

Report thefacts to HQ

Is there a large differencefrom the price recorded last month?

Is this because offinal clearancefor damaged or soiled items?

Disregard the price.

Report the non-availability and the reason for it, providing the fulldescription of the unavailable variety ifcentral office doesnot already have it.

Record the price and, if it is non-standard, alsorecord the weight, size or quantity.

State the reason, e.g. sale,special offer, black market price, replacement outlet,replacement item.

Is the collectorauthorized to find a replacement?

Is the replacementthe same kind as its predecessor?

Is the outlet likelyto be permanently shut next month?

Select another variety likely to remainavailable. Record a description of it insufficient detail both to cover qualitydifferences and to enable exactidentification.

Are both pricesavailable for the same month?

Yes

Yes

No

Yes

Yes

No

Is the varietyexpected to becomepermanentlyunavailable nextmonth?

Estimate and report the amountof price difference reflecting the value of the quality differences revealed by the descriptions of the original variety and itsreplacement.

Is the collectorauthorized to makequality judgements?

Yes

No

No

YesYes

Yes

No No

No No

Yes

Yes

No

No

Yes Yes

No

Yes

No

No

Yes

Price next item.

No

Price next item.

Report thefacts to HQ.

Figure 6.1 Price collection procedures

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

86

on the Internet as in the shops) or through necessity inorder to maintain a representative sample where thistype of retail outlet is increasingly used (for example,for books).

� Some retailers have national pricing policies with noindividual pricing discretion, even for sales and specialoffers. In these cases, a single store can be visited orthe retailer’s head office may agree to supply a singleprice list (covering all items or prices for the specificselected items).

� Prices may be obtained over the telephone or by faxwhere there will be no ambiguity in price because theitem being priced is standard and the contractorwill quote a standard charge (for example, electriciansmay be telephoned for charges for providing a newsingle electricity socket). In addition, obtaining a priceover the telephone will reflect what the consumer willoften do in practice. A further factor is that manyservice providers (such as plumbers or window clean-ers) tend not to work from retail outlets and it wouldbe difficult to visit due to their variable working hoursoff-site at the customer’s own premises.

� Prices may be obtained by letter, fax or email,accompanied by relevant head office forms for com-pletion and return in cases where central office col-lection is deemed more efficient or where local pricecollection is not possible (for example, tariff prices).Examples include prices collected from a sample oflocal authorities, insurance companies, public utilities,and telephone companies.

� Prices may be obtained from other governmentagencies or regulatory authorities, which can act asintermediaries in the price collection process. In somecountries, for example, this would be the case forelectricity prices.

� In some cases, secondary sources can provide data onspecific goods. Two examples, taken from the UnitedStates CPI but by no means unique to that country,are airline fares and used vehicles. A sample ofscheduled airline flights is selected using detailedticket data from the United States Department ofTransportation. Monthly pricing is then carried outby online reference to a private-sector computerizedfare database widely used by travel agents and others.In the case of used cars and trucks, both sampling andpricing employ published data from a dealer tradeassociation. The benefits of using secondary data mayinclude larger sample sizes, faster or less expensiveaccess to data, or the avoidance of particularly diffi-cult collection problems.

6.44 When using other sources such as catalogues orthe Internet for prices, special care must be taken toensure that they are correctly recorded with or withoutsales taxes, or with or without delivery charges. In thesecases, procedures should include a check that the pricesare relevant for the index period.6.45 It is important to remember that all the usual

price collection principles and quality assurance con-cerns remain relevant for prices collected from theInternet (including the need for detailed descriptions,immediate availability of the item for purchase, treat-

ment of special offers, and possibility of substitutingcomparable or new items).

6.46 Where prices are taken over the telephone it isrecommended that the retailer be visited occasionally,where practicable, to maintain personal contact andresponse rates and to ensure that no misunderstandingsof items or pricing are occurring. As far as possible,prices collected by telephone should also be confirmed inwriting to provide confirmation for quality assuranceprocedures (see Chapter 12).

6.47 Many households may be unable to access theInternet, and Internet shopping provides additionalservices, such as home delivery. This means that col-lecting prices from the Internet can be considered eitheras the introduction of a new outlet or a new item. Inboth instances action should be taken as part of theprocedure for maintaining a representative sample at thetime of the regular updating of the item and locationselections, usually at the time of chain linking. It shouldbe noted that consideration will also need to be given towhether a move to Internet shopping involves a qualitychange. For example, in the case of food shopping, freedelivery may be included for payment over a certainlevel, or the average ‘‘use by’’ date may differ from thatfound in traditional outlets.

6.48 The scope for improving the efficiency of datacollection can increase with the arrival of technologicaladvances in the marketplace. New collection methodsare continually becoming available, particularly in thetechnologically advanced countries. Methods for futurecollection include touch-tone dialling facilities and scan-ner data, both of which have the advantage of offeringnew ways for businesses to reduce the burden or incon-venience of supplying data.

6.49 It should be remembered that to keep the indexrepresentative, it might be appropriate to collect pricesfor an item in more than one way. For example, peo-ple may buy books from catalogues, from a variety ofshops (bookshops, newsagents, supermarkets, depart-ment stores, and so on) and through the Internet. Inthese circumstances, it is appropriate to collect pricesthrough all types of outlet where transactions are sig-nificant.

Questionnaire design6.50 Good design of the questionnaire form (or its

electronic equivalent) is essential for the successful col-lection of prices. Not only is it important that the pricecollectors find it easy to use, but the format and layoutshould facilitate the extraction of data (price, itemdescription, comments, and so on) by head office foreffective quality assurance.

6.51 The first step in designing a questionnaire is todefine the information that needs to be gathered andhow it will be collected. Different forms will be appro-priate for each of the collection methods that aredeployed, for instance visiting retailers compared withcollecting by post. There will, however, be a number ofcommon principles. The questionnaire should be prac-tical for the price collector to use in the field and shouldalso facilitate basic quality assurance. It is for the latter

PRICE COLLECTION

87

reason that it has been argued that the price recordedlast time the item was surveyed should be shown on thequestionnaire, as this will prompt the collector to askquestions if the previous price is very different from thecurrent price. The converse to this argument is that therecording of the last price may mistakenly lead pricecollectors to identify the item to be priced by referenceto price rather than item description, or at the extremeto estimate the price or repeat the previous price withoutactually visiting the shop.

6.52 It should be remembered that, at the time anindex is chained, the questionnaire will need to list allitems included in the old as well as the new basket. Forexample, an index chained annually on January pricesrequires both the old sample of locations and items andthe new sample of locations and items for the basemonth.

6.53 An example of a price collection form is givenin Appendix 6.1. This example is of a form used by thecollector for recording prices when visiting an outlet andcould be either a paper or electronic version. It is alsopossible to ask the shopkeepers concerned to completethe form themselves and to send it to the national sta-tistical institute. Such a form may therefore serve forreporting as well as for collecting. If the form has spacefor recording prices over a whole sequence of months,the collector may keep the form and transcribe the pricesfrom it each month onto a separate report form, whichis sent to the national statistical institute. Where theform used for collection is also used for reporting, thereare two main possibilities: either the form has space forrecording prices over a whole sequence of months, andthe form is shuttled backwards and forwards monthlybetween the collector and the office; or new forms forcollection and reporting are printed out by the computereach month. In the latter case, if considered desirable,the form may contain the prices recorded in the previousmonth alongside the spaces for recording the currentmonth’s prices. It should be noted that the transfer ofthe prices to another form or system, whether done bycomputer or manually, may lead to transcription errors.

6.54 Increasingly, the use of an electronic version ofthe questionnaire on a hand-held computer or ‘‘personalassistant’’, with built-in validation checks, is seen asadvantageous for local price collection by price collec-tors. The data can then be transferred electronically fromcollector to head office via a variety of intermediate stepsfor further validation checks by the price collectionagency.

6.55 It is recommended that price collectors berequired to provide full descriptions of the items beingpriced. This enables checks to be put in place to ensurethat collectors are properly following instructions, par-ticularly on the selection of items to be priced. It alsoensures that any changes, including changes in thequality of the items, are being properly identified, withenough detailed information to enable decisions to betaken on quality adjustment. Price collectors shouldbe given a checklist or set of codes to record relevantinformation on changes relating to outlets, items orprices. The information needs to be systematically col-lected. For instance, codes to help with quality adjust-

ment need to reflect those characteristics that mostinfluence price. Prior research, for example, based on thehedonic method, can help to predetermine these (seeChapters 7 and 21).

6.56 Codes for managing the sample of outlets mayinclude:

– closed down: outlet permanently shut or closed down;

– temporarily unavailable: outlet temporarily closed, butlikely to be open next month;

– refusal: owner or staff refuses to cooperate;

– change of details: change of ownership or name, orchange of purpose.

6.57 Continuity is one of the most important prin-ciples of price collection. As the index measures pricechanges, it is vital that the same item is priced everymonth in order to establish a true picture of pricechanges. So if, for example, a jar of a supermarket’s ownbrand strawberry jam has been selected, that particularbrand and flavour should continue to be collected; if it isout of stock, another brand and flavour should not beused without further investigation to establish whetherthis is a temporary situation or likely to be permanent.In the latter case, and if another flavour of the samebrand, size and quality is available, then this item shouldin normal circumstances be chosen as a ‘‘comparable’’item and the item description suitably amended. If adifferent brand, size or quality product is available thenthis should be selected as a ‘‘new’’ item, but only whereno comparable items are available. The same principlesapply to other items, such as clothes, and fresh fruit andvegetables. With clothes, it may be important that col-our, fabric, country of origin, logos and size are specifiedto ensure that the same item is priced each month. Forfresh fruit and vegetables, useful attributes to recordmay be ‘‘country of origin’’, ‘‘class’’ and variety. Forelectrical equipment, it may be the specifications andfeatures given in the manufacturer’s catalogue that areimportant.

6.58 It is not possible to be prescriptive because theconcept of equivalence will vary between different coun-tries; but for practical purposes it is important that adetailed description of the items being priced is recorded.Item descriptions will assist the price collector and headoffice in choosing or confirming the suitability of areplacement for an item that has been withdrawn andwill also help identify changes in quality. The focusshould be on recording price-determining characteristics.

6.59 Should the regular price collector, for whateverreason, be unable to carry out the normal collection, fulland accurate descriptions will enable a relief collector tocarry out the collection without any doubt as to thecorrect items.

6.60 Most of the time, the item will be exactly ascollected the previous month and all that will be recordedis a new price. However, should there be a change oruncertainty in the item, then it will be necessary for pricecollectors to use their own judgement and to informhead office, bearing in mind that head office staff areresponsible for making the final decision. A pre-codedspecification will be less time-consuming and will pro-vide better guidance to the price collector on what

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

88

information should be reported. The codes mightinclude:

Comparable (C): The original item is no longer stockedbut a similar alternative has been collected that does notdiffer in terms of major attributes. The price is likely tobe in a similar range although this may not always bethe case.

New (N): The item has been replaced by something newthat is not really comparable but is equally representa-tive of that commodity group. If possible the collectorshould try to find out the price of the ‘‘new’’ item in thechain link or base period.

Sale or special offer (S): A price decrease because of agenuine sale or special offer, with a sale or discountsticker present. This does not include damaged or out ofdate stock or clearance goods. The latter should neverbe included. A price reduction where there is no noticeof a sale or special offer is not a ‘‘sale’’; the item shouldstill be priced, but without the S indicator code.

Recovery (R): A return to the normal selling price, forexample after a sale or special offer. This need not be areturn to the same price as before the sale or specialoffer.

Temporarily out of stock (T): Guidance will need to begiven to the price collector concerning the meaning of‘‘temporarily’’ (in terms of expected duration, whichmay vary for different items). It may be advisable toreplace items immediately (for example, fashion cloth-ing, if it is unlikely that the identical item will come backinto stock). Typically, T indicators should not be usedfor more than two consecutive months – in the thirdmonth, a replacement should be chosen. In food outlets,it is very unusual for items to go permanently out ofstock. The collector should always try to check futureavailability with the retailer.

Missing (M): Used where the outlet has never stocked orno longer intends to stock an item and there is noappropriate alternative item. In these circumstances it isrecommended that the item is checked at subsequentcollections to ensure that a suitable replacement itemhas not come into stock.

Weight (W): A permanent weight or quantity change tothe product.

Query (Q): Such a code may be used to supply extraretail information to head office (for example, ‘‘10 percent extra free’’, ‘‘3 for the price of 2’’, or a strange pricedifference that is not covered by one of the other indi-cators, such as a bumper issue of a magazine at anincreased price). Arrangements need to be in place forhead office to respond to these comments and to treatthe price quotes accordingly.

6.61 The use of these codes is illustrated in Appen-dix 6.1. Even if the retailer says there have been no pricechanges since the previous month, the price collectorshould confirm prices anyway. This will require somediplomacy, but it is important because it is easy for theshopkeeper to overlook a small number of priceincreases, forget when the last increase occurred or evendeliberately mislead the price collectors. The use ofcodes is important for operational reasons. For exam-

ple, if an item is unlikely to remain available the nextmonth, then a substitute can be selected in advance andan overlap price collected.

6.62 As a general rule, a price should be recordedonly if the exact product being priced is on displayand immediately available for sale. No price should berecorded if a product is temporarily out of stock. Forcertain large items such as furniture, however, where theitem must normally be ordered, the price should berecorded as long as the retailer confirms that it is avail-able for delivery within an ‘‘acceptable’’ time period.

6.63 Some food items, such as meat, fish and cheese,can be sold in variable weights, so it is sensible to collectprices per unit of weight. This should be taken from thepackage labelling or calculated directly by the collector.Roughly the same package size and type should be usedeach month, as the unit price might be lower for largerpack sizes or differ between package types. Other items,such as eggs, are often sold in specified quantities. Forthese, it is essential that collectors record prices for thespecified quantity, as total and unit prices usually dependon the number bought. If X eggs are to be priced and theprice for the number is not quoted directly, then theprice of one egg can be obtained and multiplied by X toget the required price. Care does need to be taken,however, to ensure that unit price does not decrease withquantity. Another example is mint. This herb is oftensold in bunches of variable size, so a number of bunchesshould be weighed and priced to obtain a price perkilogram.

6.64 Certain food items, such as fruit or vegetables,are more difficult to price as some outlets might priceitems per number purchased while others might price byweight. For example, peppers may be priced by weightor by unit no matter what the size. Garlic may be pricedper bulb, clove or by weight. Various types of berriesmay be priced by weight or by punnet, which may differin size or how full they are. In these instances, care hasto be taken with the product descriptions. Collectorsneed to be aware of the importance of collecting thesame thing from one month to another, so that genuineprice changes are recorded and not quantity or qualityprice changes.

6.65 The use of hand-held computers for local pricecollection provides more scope for quality assuranceboth in the field and at head office, without some of thedisadvantages associated with paper forms. Price col-lection using hand-held computers is discussed in moredetail below. Use of electronic forms on floppy disks orby email, for example for central price collection fromthe head offices of large retail chains, may be more cost-effective than sending price collectors into individualoutlets. But in these circumstances, care needs to betaken to check that there are no price variations betweendifferent outlets in a chain and that any special offersgiven locally are covered. Where there are such localfactors, adequate account will need to be taken of them,otherwise the price recorded for the index may be mis-leading.

6.66 A decision needs to be taken over whetherlarge retail chains should be placed in separate strata(treating the chain rather than an individual outlet as the

PRICE COLLECTION

89

sampling unit) or whether a sample of outlets from eachchain is taken (thus taking as the sampling unit an outletfrom a particular chain). As a general rule, a retail chainwith no national pricing policy cannot be treated as asingle sampling unit, but it may be possible to visit onlya few of their outlets if it can be established that eachoutlet visited reflects the chain’s prices over a wide area.In these situations, it is usual to approach the manage-ment of the chain’s head office to confirm their pricingpolicies and obtain permission for the collection. Eachyear, when approached again for permission to continuethe price collection, the management should be askedto confirm that their regional pricing policy remainsunchanged. The prices collected are then given a weightto reflect the market share they represent, in the samefashion as the weights applied to prices collected cen-trally for a chain where there are no price variationsbetween outlets. Issues relating to central collection ofprices from local businesses and outlets (for instance,over the telephone by staff located in the national sta-tistical institute) and the collection of prices for retailchains from their corresponding head office are dis-cussed further below.

Field procedures6.67 Adequate field procedures are required to

ensure that the quality of the price index is not com-promised through errors in price collection. Price col-lection needs to be carefully planned and managed, andeffective instructions and training given to price collec-tors. Most prices are likely to be collected through pricecollectors visiting individual outlets. Guidance on theorganization and management of field proceduresrelating to local price collection is given in Chapter 12.

6.68 In some circumstances it may be more efficientfor prices to be obtained from one source rather thanthrough surveys in the field. These are covered in thesection that follows.

Central and head office collection6.69 One form of central and head office price col-

lection is where price data representing a number ofshops are collected from one single source. This can takeplace when chains of shops have proven national pricingpolicies, with no local variations between stores in termsof either price normally paid or special offers and dis-counts. In these cases, the chains’ outlets should beexcluded from local price collection and the prices col-lected should be weighted according to the market shareof sales.

6.70 The selection of this type of central collectionand calculation is usually dependent on one or more ofthe following considerations: national or local pricingpolicies; available sources of data (including willingnessof chains to assist in this way and forward commitmentby them to provide data centrally); data presentation andformat (advertised prices or average transaction pricesprovided by email, on floppy disk or paper); referencepoint of available data (price lists match the collectionday or period); and frequency of price changes.

6.71 Central price collection may also be appro-priate for some service prices. These could include:

– fees set by a professional or trade association or union;

– charges for public utilities or services provided byderegulated (and regulated) bodies or government(such as: water, gas and electricity tariffs; bus and trainfares; birth, marriage and death registration fees);

– prices centrally determined by government (forexample, fees to be paid for services, such as healthcare and education, that may be partially or fullyfunded by government);

– taxes and licence fees paid to government (for exam-ple, television licences and vehicle excise duties).

In some instances, data may need to be requested fromregional authorities, for example where there are region-al utility providers.

6.72 Data may be requested in writing, or by tele-phone or electronically. Where letters are sent, con-sideration should be given to using office automation forthe generation of data requests (for example, mail mergefacilities), logging responses, monitoring progress andsending reminders to non-respondents. Useful categoriesfor informing progress might include: return received;return being checked; query sent and awaiting resolutionof query; figures finalized.

6.73 The greatest gains from electronic reporting ofcentrally collected prices are likely to be efficiency gainsfrom automation, better work monitoring and fewerproblems arising from transcription errors. The risk –one that is associated with all central price collection – isthat the impact of an undetected error can be com-pounded because of the relatively large weight that maybe placed on one price or set of prices. Clearly, thisfactor should be reflected in the quality assurance pro-cedures as well as in the sampling procedures. It hasbeen observed that national statistical institutes can beslow in reviewing their quality assurance proceduresfollowing a move to greater central price collecting. Thiscan lead to a disproportionate amount of effort at headoffice being focused on the checking of local prices. Thisis particularly so if local prices have been rigorouslyscrutinized in the field; any individual error will not havea noticeable impact on the index unless it is part of asystematic bias, for example arising from inadequateinstructions to collectors.

6.74 Providers of goods and services may send eithera full price list or a tariff from which an appropriatesample of prices and weights can be extracted or justthose prices required for compilation of the index. Insome instances, for example a regional transport author-ity, it may be acceptable for data to be provided in theform of a price index. In these cases it is clearly impor-tant to ensure that the index has been calculated accu-rately and in accordance with the requirements of theconsumer price index, using agreed methodology, andthat the central office exercises strict quality control. Thelatter may be done by, for example, checking the com-putation once a year or more frequently against thebasic data or by setting up automated systems to detectabnormal changes. Agreements on the methodology ofthe computation should include such things as item

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

90

selection, weighting of components and timing of col-lection, as well as the mathematical construction of theindex. The index should also be made available to thecentral office with supplementary briefing and explana-tions for price movements. Any potential problems,such as the need to resample where previously quoteditems are no longer available, should be discussed withthe national statistical institute in advance. A con-tinuous quality control may take the form of a recon-ciliation analysis against other related data (includingannounced price changes) and identifying outliers whencompared to previous values of the index. Data andprices published by another organization or governmentbody may provide a useful comparator. Where pricesare taken over the telephone, it is highly recommendedthat all price quotations are subsequently confirmed inwriting to ensure that any queries may be resolved andthat an audit trail is maintained for future months incase of subsequent discrepancies that cannot beresolved.6.75 In all instances it is important to check on a

regular basis that the item or service being provided hasnot changed in any respect, because if this is the case aquality adjustment may need to be applied. In the caseof supermarkets and other large suppliers of data, con-firmation should be sought from head office that codenumbers have not changed to ensure that the itemsbeing priced do not change unexpectedly between oneprice collection period and the next.6.76 As stated previously, the frequency of collection

depends on both the range of prices being monitored andwhen the prices are known or expected to change. Forinstance, bus and rail fares may change once a year on aprespecified date. In other instances, prices may changethroughout the year as different providers review theirpricing structures, but the expectation may be that priceswill show little volatility. For example, it may be neces-sary only to contact health insurance companies on aquarterly basis or local authorities for prices of schooldinners only at the start of each term. Decisions on theseissues will need to be based on knowledge of local cir-cumstances, with satisfactory procedures in place todetect any change in procedures.6.77 The number of price quotes required at each

collection will depend on individual circumstances andwill need to take account of the weights and homo-geneity of the index as well as the underlying volatility ofprices (see Chapter 5). It is also best to avoid situationswhere a handful of price quotes from, say, an individualretail chain represents a large weight in the index. Thenumber of prices collected centrally should where pos-sible reflect the importance of that item in the shoppingbasket and the range and volatility of the prices.6.78 All the data collection principles above are to

be followed for all central and head office price collec-tion, regardless of whether these forms of collectionhave been introduced for reasons of practicality, cost-effectiveness, or special methodological concerns.6.79 Further examples of items that may be collected

centrally include: some aspects of transport, such astolls for bridges; situations where there may be a varietyof different outlets but where there is uniform pricing

for all consumers; and instances where the datarequirements for quality adjustment are better met byexploiting a single data source. By way of elaboration,if none of the towns or cities selected for local pricecollections have tolls on roads, bridges or tunnels,then these could be unintentionally excluded from theindex; but by selecting a sample of these across thecountry – with prices collected centrally – the indexremains representative of these types of expenditure.Similarly, if the prices of goods and services are thesame all over the country, regardless from whom theyare bought (for example, newspapers and magazines),then these prices are most cost-effectively collected cen-trally. The more complex methodological calculations ofprices, including quality adjustments, may also be bestcollected centrally. Examples of these include somehousing costs, and computers and cars (where informa-tion on technical specifications at the level of detailrequired for quality adjustment may not be availablefrom shopkeepers).

Price reductions6.80 One of the principles relating to consumer price

indices, which is applied with few exceptions (such asowner-occupier housing costs), is that only transactionprices, that is prices actually paid by individuals orhouseholds, should be included in the index. This maydiffer from the advertised price if, for example, a dis-count is offered. In practice, however, discriminatorydiscounts, which are available only to a restricted groupof households (as opposed to non-discriminatory dis-counts that are available to all), are generally excludedon principle. For example, money-off coupons andloyalty rewards for previous expenditure are normallyignored and the non-discounted price is recorded. Also,it may be difficult to obtain the price paid if this issubject to individual bargaining. It may therefore notcome as a surprise that, while the general rule abovemay appear simple, there are a number of instancesrequiring special treatment either because of conceptualissues or because of practical difficulties. The followingguidelines reflect practices followed by a number ofcountries. They do not represent a set of rules becausethe appropriate practice to be followed will be deter-mined by individual circumstances, which might varybetween different countries.

6.81 Discounted prices should only be taken if gen-erally available to anyone with no conditions attached;otherwise the non-discounted or unsubsidized price isrecorded. In particular, the general practice is to ignoremoney-off coupons and loyalty rewards. A judgementneeds to be made, however, relating to the interpretationof ‘‘generally available’’. For instance, reduced pricesfor payment by direct debit may be taken into accountdepending on the extent to which consumers as a wholehave access to and use such a service. A judgement isrequired in the latter case on the threshold to be set foraccess, above which action is taken for inclusion in theindex. Alternatively, different payment methods may allbe priced individually (for example, separate data col-lection for electricity payments by cash, direct debit and

PRICE COLLECTION

91

pre-payment) and weighted together to form a singleprice index for that item.

6.82 Price discrimination. Discounts available onlyto a restricted group of households should be dis-regarded because they are discriminatory, unless theyare significant and are available either to the vastmajority of the population or to identifiable subgroupswho qualify for such discounts on the basis of demo-graphic or other characteristics not requiring action bythe individuals concerned at the time of purchase. In thelatter case, they should be treated as stratification orcoverage issues in item sampling. Some judgement isrequired. Examples of allowable price discriminationmay include lower prices offered to pensioners (forexample, discounted travel or haircuts) and discountsfor people who receive state benefits. Another exampleof a case where prices are not universally available to all,and where judgement is required, is where a nominal ortoken membership fee is required by the retail outlet. Inthese cases, the take-up of such membership – which iswidely available to all – needs to be considered in termsof thresholds and general spending patterns of theconsumers and the conditions placed on membershipwhich may make the latter restrictive (for example,minimum levels of purchase). Ease of access to theoutlets in question may be a relevant factor as well, say,if in practice the customers need to have the use of aprivate car.

6.83 Sale or special offer prices should be recorded ifthese are either temporary reductions on goods that arelikely to be available again at normal prices, or arestock-clearing sales (such as January sales or summersales). Before designating a price as a ‘‘sale’’ price,however, special care should be taken to ascertain thatthere is a genuine sale with price reductions on normalstock. On occasion, stock is continually sold below therecommended retail price or advertised as a special offereven though these prices are available all year. In suchcases, prices should not be considered as sale prices, butcan still be collected. Special purchases of end-of-range,damaged, shop-soiled or defective goods should notnormally be priced, as they are likely not to be the samequality as, or comparable with, goods previously pricedand are unlikely to be available in future. If the specialoffer is limited to the first customers, the item should notbe priced, as the offer is not available to everyone.Introductory special offers may be included if they areavailable to all. In reality, however, given the need toprice the same ‘‘basket’’ each month, such offers will notbe chosen as representative items unless they are intro-duced at the time of an update of the ‘‘basket’’ or whena replacement item needs to be chosen. Discounts ongoods close to expiry dates should be disregarded ortreated as specification or quality changes.

6.84 Bonus offers, extras and free gifts. Prices foritems temporarily bearing extra quantities (for example,30 per cent extra free) should not be adjusted to takeaccount of the increased quantity if it is thought that theextra quantities involved may not be wanted by mostconsumers, will not have influenced the decision topurchase or will not be consumed. Similarly, free itemswith other purchases (such as buy 2 get 1 free or free gift

with every product purchased) should be disregarded.Money-off coupons for future purchases should be dis-regarded, as these may not be used or wanted. Free giftssuch as plastic toys in cereal boxes should be ignoredbecause they are not included in the list for priceobservations; it is the price to be paid to get the cereal inthe box that is relevant. Collectors should be aware thattemporary ‘‘special offer’’ weight changes (X per centextra free) could become a permanent weight change(for example, cans of alcoholic drinks changing sizefrom 440ml to 500ml) and should feed the informationback to head office as they become aware of it. In thisway, head offices can issue new or amended guidance toprice collectors about item specifications.

6.85 Stamps. Sometimes purchasers are given specialstamps, which can be accumulated and subsequentlyexchanged for goods and services. If a discount isavailable as an alternative to such stamps, then the dis-counted price should be recorded. Otherwise, the stampsshould be disregarded.

6.86 Trade-ins. In general the price reductionobtained by trading in an old item (for example, a car)compared with the nominal full price should be ignored.This treatment follows convention, as the transactionessentially relates to a second-hand good and only theservice charge levied by the outlet in buying and sellingthe good comes under the scope of the index. In reality,however, the situation is not so clear-cut. For instance, agarage may well give a discount which is greater than theretail value of the traded-in car and, therefore, in effectgives a genuine discount on the new car. In many cases,discounts from trade-ins are very difficult to evaluate.The trade-in value may be negotiable in each case, andthe full nominal price – which is used as the benchmarkagainst which the discount is measured – may not beknown. It may therefore be best to report the list priceor asking price.

6.87 Sales taxes. When an indirect tax is not includedin the price of individual items in a shop, but is insteadadded on when the customer pays for the item, greatcare must be taken to record the price including tax.To make sure of this, with items for which the priceis normally quoted pre-tax, and in areas where a gen-eral sales tax is added to the bill, the price collectionforms should require the collector to indicate whether ornot the price recorded does include the tax – as a pricecheck – so that it can be added where necessary.

6.88 Tips for services. If a compulsory service chargeis included, for example on a restaurant bill, only thecompulsory amount should be included in the price, butnot any additional discretionary tips. For services whichare free in principle, but which in practice can rarely beobtained without what amounts to a tip, or where tip-ping at a standard rate is the common practice, such tipsshould be added to the specified price.

6.89 Regular rebates or refunds should only be takeninto account when attributable to the purchase of anindividual identifiable product and granted within a timeperiod from the actual purchase such that they areexpected to have a significant influence on the quantitiesbuyers wish to buy. For example, money-back depositson bottles should be deducted from the price if they are a

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

92

sufficient incentive for returning the bottle, while money-back offers on lawn mowers after a five-year periodshould be disregarded. In all cases, a consistent decisionfor each item must be applied over time. Decisions aboutthe treatment of rebates are not easy to recommend, asmany decisions are made on an individual basis. Theymay reflect income rather than expenditure changesand may require different treatment for, say, nationalaccounts uses.6.90 Irregular rebates or refunds should only be taken

into account when they apply to the purchase of anindividual product and are granted within a time periodsuch that they are expected to have a significant influenceon the quantities purchasers are willing to buy. Loyaltyrebates or coupons associated with previous expenditureat the outlet, to be used for similar or other purchases,should generally be disregarded, as they are discrimi-natory. If they are significant factors, they should betreated as stratification or coverage aspects of sampling(see Chapter 5). One-off rebates (for example, associatedwith privatization) should be disregarded as they do notrelate to the specific time period of the consumption andare unlikely to affect levels of consumption. They can beviewed more as a source of additional income.6.91 Credit card and other payment arrangements

involving interest, service charges or extra charges incur-red as a consequence of failing to pay within a specifiedperiod of time from the purchase should be disregarded.For example, zero interest as well as positive interestloans granted to finance a purchase should be dis-regarded when determining the price. Reductions forcash payments may be included but care should betaken to ensure consistent treatment from one period tothe next.

Price bargaining6.92 Bargaining relates to a situation where prices

are individually negotiated between sellers and pur-chasers, and are not predetermined. The process ofnegotiation is a characteristic of, for example, market-places in many African countries where almost every-thing to be purchased must be negotiated to arrive at anagreed price, including a wide range of life’s dailynecessities that can account for a large part of householdconsumption. The system of bargaining is characterizedby its great flexibility in the setting of prices. Final trans-action prices and quantities will vary from one trans-action to another and cannot be determined until thepurchase has been made. Similarly, there will be vari-ations between transactions in the quality of the goodsbeing purchased. Clearly, these special conditions re-quire special methods to determine purchasers’ pricesfor inclusion in the consumer price index.6.93 It can be argued from the viewpoint of the

system of national accounts that bargaining is a form ofprice discrimination. A purchaser is not free to choosethe purchase price because the seller can charge differentcategories of purchasers different prices for identicalgoods and services sold under exactly the same cir-cumstances. It follows that ‘‘identical’’ products sold atdifferent prices should be recognized as having the same

quality, and their prices must be averaged to obtain asingle price relative to calculating price indices. In re-ality, the variation in transaction price can rarely beassociated with identifiable price-related categories ofcustomers. Rather, purchasers may inadvertently buy ata higher price than may be found elsewhere or couldhave been finally negotiated. Notwithstanding this,collectors of prices should guard against the presump-tion that price differences do not relate to quality (orquantity) differences.

6.94 Where prices are determined by bargaining,standard price survey methods – which consist of col-lecting prices directly from sellers – can generate erraticprice indices that do not reflect actual price movementsin a market. For example, prices collected by enumera-tors depend on their ability, willingness and power tobargain, in the same way as actual prices paid by genuinepurchasers. In addition, prices can vary during thecourse of a day as well as from one day to the next,adding an extra dimension to the concept of repre-sentativeness. A number of survey methods and pricecollection techniques have been developed to overcomethe difficulties inherent in measuring prices that havebeen bargained.

6.95 Survey by purchase of products. The principle isthat price collection should be carried out in conditionsthat simulate as closely as possible situations in whichreal transactions actually take place. Price collectorsbehave like regular purchasers by actually purchasingitems to be priced and spreading their purchases overthe day to ensure representativeness. In each case, thefield manager will need to carry out regular checks onquantities and prices obtained by collectors. The fol-lowing approaches may be taken:

� Price collectors buy items to determine the relevantprice through bargaining. They should be trained tobehave as normal purchasers and strive to get thelowest possible price from selected outlets and sellers.Given the high turnover of sellers, the sample ofsellers should be partially renewed on a regular basisto ensure that it remains representative and chained inas appropriate.

� Price collectors buy items and, in addition, are givenan incentive to get the best price. For example, aprice ceiling may be set and the collector may receivea proportion of the difference between the ceilingand the bargained price. This incentive system guardsagainst potential difficulties caused by the collectornot getting the lowest price because, unlike an ordi-nary customer, he or she is not concerned with max-imizing value for money and is not constrained byincome.

6.96 Survey of purchasers. The prices purchasershave paid are collected throughout the day immediatelyafter the purchaser leaves the outlet or market stall,together with a record of the quantity and quality of theproduct purchased. The extent of the haggling should bedetermined (for example, opening and closing prices)together with an indication of the relevant parametersdetermining the price. A form of incentive paymentfor survey participation may be needed where there is

PRICE COLLECTION

93

reluctance among purchasers to submit to such time-consuming questions.

6.97 For the survey by purchase of products and thesurvey of purchasers, all items in the basket of itemsused to calculate the consumer price index that aresubject to bargaining should be covered. The number ofprices collected needs to be sufficient both to cover allrelevant items and to provide a reliable guide to averageprice. This may be difficult to determine beforehand,although previous price collections should provide someguidance. It is suggested that price collectors engaging ina survey of purchasers are given a form on which torecord the number of quotations per stall or shop, asindicated by the various respondents. This can be usedto check the number of quotes obtained against thetarget number set by head office. An example of such aform is given in Table 6.1.

6.98 Survey of trends in wholesale prices. A limitedparallel collection of wholesale prices can be a usefulsupplement for problematic items where the informationobtained from the above survey techniques is only par-tially successful, for example where there is a deficit in thenumber of observations obtained. Ideally, prices shouldbe obtained from the particular wholesalers where therelevant retailers get their goods. All factors should beobserved which might result in increases in the corre-sponding retail prices, such as changes in taxes on retailactivities, licence fees and the rental for the market stall.Assuming that those factors remain constant over time,the evolution of wholesale prices may be used as a proxyfor the retail price index of relevant items. The price of anitem for the current period would be estimated by mul-tiplying the price of the previous period by the corre-sponding evolution in wholesale price.

6.99 Determination of the prices paid by a purchasercan be problematic where the final price is for a bundleof items, for example where a stall holder gives thepurchaser extra quantities as a bonus for buying a num-ber of goods. If the bonus comprises several categories ofitems, including the item on which a transaction pricewas being directly negotiated, then the purchase has to besplit into as many sub-transactions as item categories. Inthese cases, a commonsense approach is needed. There isa fine dividing line between this type of circumstanceand the ‘‘two-for-one’’ offers sometimes found, for exam-ple, in Western-style supermarkets. The latter form ofdiscount is often excluded from price calculations on thebasis that the purchaser does not want or use the addi-

tional amount supplied. Additional perishable goods, forexample, will become out of date and be thrown away.This argument is less relevant in market purchases in adeveloping country, where many consumers will be livingon a subsistence income and therefore will consume allpurchases. In such cases, purchasers will have activelybartered an overall price for the total basket of pur-chases, including any ‘‘free’’ goods thrown in.

6.100 The method for determining the price paid bythe purchaser is illustrated in the following example: apurchaser wants to buy 5 kg of carrots and is offered abonus consisting of 500 grams of carrots, 100 grams oflettuce and 200 grams of baby marrow.

6.101 Three transactions can be identified, involv-ing: 5.5 kg of carrots; 100 grams of lettuce; and 200grams of marrow. The bonus has to be valued at pricesat which the seller would have sold and the purchaserwould have bought the items. The assumption made isthat prices, in local currency units (LCU), would havebeen determined through bargaining on the same con-ditions as the price of the item needed (carrots). If theopening value of 5 kg of carrots is LCU 15,000 and theclosing value LCU 12,000, whereas the opening valuesof other foodstuffs included in the bonus are LCU 990for a bunch of 264 grams of lettuce and LCU 4,620 for aheap of baby marrow of 4.4 kg, the actual closing priceof carrots will be determined as shown in Table 6.2. Theactual purchaser’s price of carrots is found to be LCU2.0967 per gram or LCU 2,096.7 per kilo.

6.102 If the price collector does not know the closingprice at which lettuce and baby marrow would have beensold by the seller of the carrots, then it can be estimated.This is done by collecting opening values and standardquantities from a sample of sellers in the same market orat different outlets in the same area. The average openingprice of an item is equal to the sum of opening values ofthe item divided by the sum of relevant standard quan-tities. For each bonus item (lettuce and baby marrow),the resulting average opening price will be divided by thebargaining ratio calculated on the item needed (carrots)to estimate a closing price for that bonus item. The valueof each bonus item is obtained by multiplying the closingprice by the quantity offered. If the packet of bonus itemscontains an item of the same quality as the requesteditem, that bonus item will be valued on the basis of theclosing value of the requested item.

Forced replacements, productsubstitution and quality adjustment

6.103 A difficulty which confronts both local andcentral price collections occurs when an item that wasbeing priced is no longer available and a substitute needsto be found. This is briefly discussed here because itrelates to real decisions facing price collectors in thefield, but the issues are covered in more depth inChapters 7 and 8. In cases where a replacement has to befound, the price collector should normally take thenearest equivalent product available in the outlet, takinginto account those characteristics which will be mostinfluential in determining price and purchasing habits(for example, one out-of-date or obsolete item should

Table 6.1 Example of a survey form showing the number ofprice quotations by shop or stall

Items Targetednumber ofquotations(set by

Actual number of quotations

head office) Shop/Stall 1 Shop/Stall 2 . . . . Shop/Stall n

Item 1 5 0 3 5Item 2 4 4 5 4Item 3 8 5 8 8. . .Item k 5 7 2 6

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

94

not be replaced with a close item which may also shortlysuffer the same fate). Nevertheless, where it is considereddesirable to take the opportunity provided by productsubstitution to update the sample, a ‘‘most representa-tive’’ replacement may be chosen. In the latter case, caremust be taken to ensure that sufficient controls are inplace to achieve the desirable end.6.104 When a replacement is made, it is important

for the price collector to provide a detailed specificationof the new item so that head office can identify anyassociated quality change. This is to ensure that theconsumer price index continues to reflect the cost ofbuying a fixed, constant quality basket of goods. Headoffice should then use the information collected todecide on any relevant quality adjustment to be applied.6.105 When such a situation occurs, a nominal price

in the base month (which for some indices will be theprevious month) is needed for the new or replacementitem. The latter may be obtained from the shopkeeper orone of three methods can be applied to take account ofquality differences, which can then be used to estimatea new base price. These are direct comparison (that is,when there is no change in quality), direct (explicit)quality adjustment, or indirect (implicit) quality adjust-ment. When a new rather than a comparable replace-ment item is priced, it may be necessary for the new itemto be kept out of the index for a short period until there issufficient evidence of its longer-term availability andprice stability.6.106 In some countries, a table of quality coeffi-

cients is used to adjust prices. In one North African

country, for example, the item ‘‘green tea’’ should berepresented by Minara tea; however, if this is unavail-able an alternative tea may be collected and that pricescaled by the relevant coefficient (for example, Oudayatea�1.20). More detailed guidance on direct and in-direct quality adjustment is given in Chapter 7.

6.107 If an outlet closes or refuses to allow furtherprice collection, then another similar outlet should beselected from the same location and the indirect qualityadjustment approach used to calculate new base prices.See Chapter 5 on sampling for replacing outlets withinlocations.

Related issues

Electronic reporting6.108 Electronic reporting for centrally collected

prices and use of hand-held computers for local pricecollection can introduce greater efficiency into pricecollection and processing, as well as providing morescope for effective auditing, but both are dependent onthe introduction of effective quality control procedures.Electronic reporting through the use of electronic pointof sale (EPOS) or scanner data is also likely to increaseover time.

6.109 Electronic reporting for centrally collected prices.Centrally collected data can be collected electronic-ally in a number of ways. Once initial contact hasbeen made with data suppliers, a mutually convenient

Table 6.2 Example illustrating the method for determining the actual price paid by the purchaser when bargaining takesplace

Requested item Bonus items

Carrots Carrots Lettuce Baby marrow

Opening value of standard/requestedquantity (local currency units)

15 000 15 000 990 4 620

Standard/requested quantity (grams) 5 000 5 000 264 4 400Opening unit price of standard/requestedquantity (local currency units per gram)

3 3 3.75 1.05

Opening unit price of bonus quantity(local currency units per gram)

3 3.75 1.05

Bonus quantity (grams) 500 100 200Opening value of bonus quantity (local currency units) 1 500 375 210

Closing value of items received (local currency units) 12 000 1 200 300 168

New price (local currency units per gram) 2.4 2.4 3 0.8

Bargaining ratio 1.25 1.25 1.25 1.25

Payment (local currency units) 12 000Estimated closing value of bonus (local currency units) 1 668Actual value of requested item (all carrots)(local currency units)

10 332

Quantity received of requested item (grams) 5 500Actual purchaser’s unit price of requested item(local currency units per gram)

2.09671

Improved bargaining ratio 1.522

1(12000�300�168)7 5500= 2.0967. 2372.0967 =1.52.

PRICE COLLECTION

95

electronic data collecting procedure can be initiated.Options include:

– emailing data collection spreadsheets between thenational statistics institute and the retailer;

– emailing of price lists at agreed times by retailers;

– touch-tone dialling facilities for data to be supplied inan agreed format;

– use of the Internet (supplemented if necessary by tele-phone calls to clarify definitions and availability).

6.110 Hand-held computers. The greatest gains fromthe use of hand-held computers for local price collectionare likely to be drawn from efficiency in data transmis-sion, better quality data as a result of the additionalediting facilities available in the field, and the elimina-tion of transcription errors. In addition, hand-heldcomputers can generally speed up timetables.

6.111 The validation checks made during local pricecollection using hand-held computers will generallydiffer very little from those that should be carried out atcentral office when paper forms are received under themore traditional methods of collecting price data. Theadvantage of hand-held computers is that they providethe opportunity to validate prices in the field and inconsequence correct errors at the time of price collec-tion, rather than attempting to do so afterwards. Inpractice, it may be expensive and very difficult to checkprices after collection. For example, prices may havechanged in the intervening period and the price collectormay have to rely on the shopkeeper’s memory.

6.112 The choice of hand-held computer will dependon a number of factors, including price, reliability,maintenance and ease of use. The computing functionsof data transfer, including back-up and downloading ofdata, as well as compatibility with office systems, arealso important. Other considerations of particular con-cern to the price collector include ergonomic aspects,size and weight, editing facilities, and expected batterylife. The risk of theft and other security matters will alsoplay a part.

6.113 The introduction of hand-held computers caninvolve a significant initial outlay associated with pur-chasing the computers, developing the software andtraining price collectors. In addition there will be on-going maintenance costs. These costs can sometimes bereduced or spread either by using the machines for otherdata collection in the national statistical institute, forexample a household budget survey, or by contractingout to another organization that may already deploythese machines for other statistical surveys. These costscan be offset, at least in part, by more efficient workingby price collectors and savings generated by less tran-scribing and inputting of data by hand, and a reductionin data editing by head office staff.

6.114 Careful planning is required in moving from apaper-based collection system to one using computers inorder to avoid the risks inherent in such a change.National statistical institutes planning a move to pricecollection using hand-held computers should embark onextensive pilot testing and should also consider somelimited double-running in parallel with the old papercollection system to ensure the robustness of the new

method and that it is producing the same numericalresults.

6.115 The additional facilities offered by hand-heldcomputers, including local editing of prices, plus theelimination of the need for data transcription, maynecessitate a general reorganization of the process forproducing the consumer price index, and a redefining ofroles and interaction between different members of theproduction team and between head office and pricecollectors.

6.116 It is important that clear rules and proceduresare set out controlling the changes that can be made inthe field by the price collector and the changes thatshould be made centrally. For example, replacementoutlets could be pre-programmed in the event that out-lets close down or refuse entry. Flexibility should allowprice collectors to select and key in the new attributes forreplacement items subject to procedures controlled cen-trally.

6.117 EPOS or scanner data. Electronic point of sale(EPOS) data usually refers to data obtained directly froma retailer’s electronic point of sale, while scanner datausually refers to a commercial database that collatesindividual EPOS data. National statistical institutes areincreasingly looking towards EPOS or scanner data as aconvenient method of obtaining up-to-date and accurateinformation, not only on the quantity and prices ofgoods sold but also on their specification. The latter canbe used to control the representativity of the sample andalso to measure changes in quality. The advantage ofthis is that data are collated electronically without thenecessity of sending price collectors into the field.

6.118 When considering the use of scanner data,account needs to be taken of such matters as the repre-sentativity of outlet and product coverage, and alsowhether the average prices given in scanner data accu-rately reflect actual transaction prices in the outletsthemselves. In addition, it cannot be assumed that thegeographical and population coverage or the treatmentof goods and transactions matches the scope of theindex. Scanner data are also likely to be of little use incollecting prices of services, which in many countriescomprise an increasing share of transactions and thus ofweights in consumer price indices. On a practical front,the unique identification of products can sometimes beproblematical, as one item might be covered by morethan one code number, and code numbers may not beuniquely assigned to one product and may be recycled asitems disappear.

Purchasing power parities6.119 Purchasing power parities are used to deflate

major economic aggregates, such as gross domesticproduct, to enable intercountry comparisons of realincome levels to be made in terms of real volume, that is,adjusted to account for local prices and different con-sumption patterns. Purchasing power parities consist ofintercountry comparisons of prices for a basket of goodsand services that is both representative of and compara-ble between the countries involved. The underlying pricedata therefore differ from those used in consumer price

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

96

indices in so far as the latter basket is designed to berepresentative solely of private household consumptionin the economic territory of an individual country.6.120 It would be attractive in principle to construct

consumer price indices and purchasing power paritiesfrom the same basic set of price data. In practice, thescope for this may be limited because of the differingobjectives of the two exercises. In particular, the addi-tional need for prices collected in the context of purchas-ing power parities to be comparable between countrieswill generally result in a more tightly defined basketcompared with that likely to be available and used for aconsumer price index.6.121 An investigation of the potential overlap

between the two baskets may, however, identify potentialareas where one price collection may suit both purposes.This may be the case particularly for unbranded goodsand locally produced fresh fruit and vegetables, for exam-ple, where a locally produced dessert apple of a standardquality may be compared across countries withoutrecourse to a reference to the variety concerned. In con-trast, branded items – whether food or non-food – maybe more problematical because of differences in avail-ability and specification between countries.6.122 In some cases, scanner data might provide a

useful common source of price data for at least someelements of the purchasing power parities calcula-tion, notwithstanding the drawbacks mentioned above.Annex 4 goes into more detail on issues relating to pur-chasing power parities and the International Compar-ison Program (ICP).

Data quality and quality assurance6.123 Checks should be carried out to ensure the

accuracy of the data on prices and that the index itself

has been compiled according to the proper methodology.Checks to ensure that data are complete and correctshould be carried out as early as possible in the collectionand compilation processes. A return to the shop to re-input prices becomes increasingly less feasible as timegoes on, and there is a greater risk that the prices in theshops will have changed since the initial collection. It isnot possible to prescribe the type and range of checksthat should be carried out. The checks will depend onindividual circumstances, including sample design andthe medium used for the collection of prices. For exam-ple, the use of hand-held computers by price collectorsfacilitates much more detailed checking at the time of theinitial collection of prices in the shop than the equivalentpaper system. Further guidance on quality assurance isgiven in Chapter 12.

Documentation6.124 The importance of good documentation can-

not be overemphasized. Documents are needed toexplain what is to be done, when it should be done, howit should be done and why it should be done. Preparingsuch documents provides a useful opportunity to ensurethe quality of current procedures used to collect pricesand compile the index. It also provides an opportunityto review and improve these procedures. Once in place,documentation serves two purposes in the context ofproducing the index. First, it enables somebody to takeover the work if the person responsible falls ill or leaves.Second, it provides a quality check to ensure that theprocedures that should be carried out are indeed beingcarried out in practice. More generally, documentationcan provide a useful reference for users of consumerprice indices. Documentation is discussed in more detailin Chapter 12.

PRICE COLLECTION

97

Appendix

6.1

Extractfrom

asim

ple

pricecollectionform

Note

s:Theco

llec

torfillsin

thela

stfo

urco

lum

ns,

leavin

g‘‘bra

nd

orm

ake’’bla

nk

when

inapplica

ble.Ther

ewillusu

allybease

para

teques

tionnairefo

rea

chty

peofitem

orfo

rea

choutlet

.

Collec

tion

date

:Collec

tor’snam

e:O

utlet

nam

e:

Item

Retailoutlet

Description:brandormake

Price

Indicator

code1

Furtherexplanation

(complete

asnecessary

for

outlet/item

usingagreedelectronic

pre-codedmessagesasappropriate)

Potatoes–new,looseperkg

GreenFingers

Green

Grocers

JerseyRoyals

59p

CComparableitem.Lastmonth

EgyptianQueens.Seasonalvariation.

Homekilledbeef,bestminceperkg

SuperBuysSuperm

arket

Ownbrand.Premium

cut,lowfat.

Redandbluepackaging.

£3.45

SSpecialOffer.Halfprice.

Frozenpizza,medium

size300–450g

SuperBuysSuperm

arket

OwnBrand.MeatFeast.Redbox

withpizzapictured.

400g

WPrevioussize450g.

Milk,pasteurized,4ptor2l

SuperBuysSuperm

arket

Fullcream

pasteurized.Plastic

bottlewithbluelabel.

89p

Men’sform

alshirt,longsleeved

Form

alForMen

‘‘DickieDirts’’brand.White.75percentcotton,

25percentpolyester.MadeinEngland.

Blueflashonpacket.

£34.99

QIncludesfreetie.

Women’sshoes,fashion

Steps

Blackcourtshoes.Shoename‘‘Sleekie’’.

Leatheruppers

andleathersoles.MadeinChina.Neartill.

£30.00

RRecovery

from

25percentoff.

Restaurantmeal,maincourse,

eveningmeal,specify

FillUp

Cod,chipsandsalad.Mainmenu.

£7.50

CPreviouslyservedas‘‘Plaice,

chipsandsalad’’.

Theatreadmission,evening,

frontstalls,adult

CivicTheatre

JackandtheBeanstalk.Weekday(M

on–Thurs)

eveningperform

ance.

£12.00

NPreviously,‘‘Talkingheads’’.

1C=comparable;S=sale

orspecialoffer;W

=weight;Q=query;R=recovery;N=new.

98

7ADJUSTING FOR QUALITY CHANGE

Introduction7.1 The measurement of changes in the level of con-

sumer prices is complicated by the appearance and dis-appearance of new and old goods and services, as well aschanges in the quality of existing ones. If there were nosuch complications, then a representative sample could betaken of the items households consume in period 0, theirprices recorded and compared with the prices of the samematched items in subsequent periods, say t. In this way theprices of like would be compared with like. However, suchcomplications do exist. For example, an item may nolonger be produced in period t+1, so its price comparisoncannot be undertaken between periods 0 and t+1.7.2 A number of methods are available to remedy

this. A replacement item may exist in period t+1. If it isof the same quality, its price can be compared with the‘‘old’’ item’s price in period t. But the replacement itemmay well be of a different quality. One option is to ignorethe quality difference and continue comparing the priceof the ‘‘new’’ replacement item in t+1with that of the oldone in period t to continue the series. An adjustment forthe difference in quality is still being made; it is just that itis a very poor adjustment, because the change in qualityhas no effect on the price. A second option is to excludefrom the index those items for which quality changes,and to compile the index link between t and t+1 only formatched items having characteristics that are the same.This exclusion amounts to an implicit quality adjust-ment, one that assumes the overall price change ofexisting matched items will be the same as the quality-adjusted price change between the missing old andreplacement new items. In reality, however, price chan-ges generally vary over the stages of a product’s life cycle.Price changes at the time of, say, a model’s upgrade –when an item is missing and replaced – may be quitedifferent from those at other stages. The implicitassumption may therefore be inappropriate. Third, theprice change of a new replacement item may be splicedonto the index if the prices of the disappearing andreplacement items are available in a common overlapperiod, say period t. The old item’s price change betweenperiods 0 and t is multiplied by the replacement item’sprice change between periods t and t+1. Yet again, thereis an implicit quality adjustment, one that requires theprice difference between the old item and its replacementin period t to reflect the effect of the quality difference onprice. Such differences may also be in part the result ofstrategic price-setting behaviour related to the period inthe item’s life cycle.7.3 There are other methods of adjusting the prices

of non-comparable replacements for quality differences,

including ones that use explicit estimates of the effectof the quality change on price. There are a number ofmethods of deriving such explicit estimates, and the suit-ability of explicit quality adjustments depends as muchon the method used as on the availability of appropriatedata to implement the method. In each case, whateverprocedure a statistical office follows, a quality adjust-ment to prices is made in every period when an item isnot available. The purpose of this chapter is to helpensure that these quality adjustments are the appropriateones.

7.4 There are three main reasons for considering howto adjust for quality change. First, the scale and pace ofmethodological innovations are substantial. Second,there is a lack of consistency in the methods chosen bystatistical offices for dealing with quality changes; thuscomparisons of consumer price indices between productareas, across countries, and over time may be misleading.Finally, a number of empirical studies on the effects ofusing different methods found that choice of methoddoes indeed matter (Dulberger, 1989; Armknecht andWeyback, 1989; Moulton andMoses, 1997; Lowe, 1996).

7.5 Against these concerns, it must be recognizedthat statistical agencies do guard against quality changesby using the matched models method. Price collectorsrecord the features of selected items and collect pricesfor the very same models in subsequent periods in orderto compare like with like. If a product group existsin which there are no items whose quality changes andno new or disappearing goods and services, then thematched models method based on representative itemsworks. More generally, three potential sources of errorarise from the matched models approach: missing items,sample space change, and new products.

Why the matched modelsmethod may fail

7.6 The long-run price change for an item is mea-sured by comparing the price of the item in the currentperiod with that in the price reference period, the periodin which it, along with most other items, entered thesample.

Missing items7.7 The first source of error, and the focus of this

chapter, is when an item is no longer available in theoutlet. It may be discontinued or it may not be availableto the same specification – its quality has changed – andit is effectively missing in the current period. The item’s

99

price may be missing for other reasons. It may be aseasonal item or one whose price does not need to berecorded so frequently, or it may be that the item is acustom-made product or service, supplied each time tothe customer’s specification.

7.8 It is necessary to distinguish between items thatare permanently and temporarily missing. Items that aretemporarily missing are items not available and notpriced in the month in question, but that are priced insubsequent months. The items may be missing because,for example, demand is seasonal, as is the case withsome fruits and vegetables, or there are shortages. Somecommodities are priced on a less frequent basis, maybequarterly or biannually, because their price changesare irregular. They are therefore missing when they are‘‘off cycle’’.

7.9 The concern with seasonal items is to imputetheir missing prices until the item reappears. The impu-tation methods used are similar in some cases to thoseused for quality adjustment. The temporary nature of theimputation, however, requires that they be separatelyidentified by the respondent as ‘‘temporarily missing’’ or‘‘seasonal’’. Principles and methods for such imputationsare outlined by Armknecht and Maitland-Smith (1999)and Feenstra and Diewert (2001), and in Chapter 22. Theconcern in this chapter is with permanently missing itemsand with making imputations on a continuing basis orusing a replacement item.

7.10 A number of approaches are available fordealing with missing items:

� The item may be dropped on the assumption that theaggregate price change of a group of other itemsreflects change in the missing item – an implicit qualityadjustment to price.

� A replacement item may be selected and the replace-ment item’s price may be used for the comparisonbecause the replacement is deemed to be comparablein quality to the missing item.

� The replacement may be deemed to be non-comparable with the missing item, but prices on boththe missing and replacement items may be available inan overlap period before the former item was missing.The price difference in this overlap period may beused as an estimate of the quality difference to quality-adjust the replacement item’s price.

� The replacement price of a non-comparable replace-ment may be used, with an explicit estimate of theadjustment for the quality difference to extricate the‘‘pure’’ price and quality change.

7.11 In many cases, therefore, there is a need to makea quality adjustment to the replacement item’s price. Aquality adjustment in this instance is an adjustment tothe price (price change) of the replacement item (com-pared with the missing item) to remove that part of theprice change that results from quality differences. Aquality adjustment can be taken to be a coefficient thatmultiplies the price of, say, the replacement item to makeit commensurate, from the consumer’s point of view,with the price of the original.

7.12 To take a simple example, suppose that the size(or quantity) an item is sold in is a quality feature.

Suppose that the size of the missing item and its replace-ment differ. Assume that a quantity k of the replacementis sold for the same price as a quantity j of the original.Whether the consumer buys one unit of the original orj/k units of the replacement makes no difference – theyare worth the same. In order to make the price of oneunit of the replacement commensurate with the price ofone unit of the original, the replacement must be multi-plied by k/j. This is the required quality adjustment. Forexample, if 2 units of the replacement item wereequivalent to 3 of the original, the required qualityadjustment to be applied to the price of the replacementitem is 2/3. Suppose one unit of the replacement actuallysells at the same price as one unit of the original, then theprice of the replacement, after adjusting for the change inquality, is only 2/3 that of the price of the original. If oneunit of the replacement sells for twice the price of theoriginal, then the quality-adjusted price is 4/3 that of theoriginal: the price increase is 33 per cent, not 100 percent. The consumer price index seeks to record thechange between the price of the original and the quality-adjusted price of the replacement.

7.13 The approaches listed in paragraph 7.10 will bediscussed later in some detail, along with the assump-tions implied by them. By definition, the prices ofthe unavailable items cannot be determined. The veracityof some of the assumptions about their price changes,had they been available, is therefore difficult to estab-lish. What is stressed here is that the matching of pricesof items allows for the measurement of price changesuntainted by quality changes. When items are replacedwith new ones of a different quality, then a quality-adjusted price is required. If the adjustment is inap-propriate, there is an error, and if it is inappropriate in asystematic direction, there is a bias. Careful qualityadjustment practices are required to avoid error andbias. Such adjustments are the subject of this chapter.

Sampling concerns7.14 There are four main concerns with regard to

sampling. First, the matching of prices of identical itemsover time, by its nature, is likely to lead to the monitoringof a sample of items increasingly unrepresentative of thepopulation of transactions. It may be that the prices ofold items being dropped are relatively low and the pricesof new ones relatively high, and such differences in priceremain even after quality differences have been takeninto account (Silver and Heravi, 2002). For strategicreasons, firms may wish to dump old models, perhaps tomake way for the introduction of new models pricedrelatively high. Ignoring such ‘‘unmatched’’ models inmeasuring a consumer price index will bias the indexdownwards (see paragraphs 7.150 to 7.152 below).Therefore, in a curious way, the very method of match-ing, used to ensure constant quality, may itself lead tobias by omitting items whose price changes are unusual(see also Koskimaki and Vartia (2001) for an example).Chapter 8 suggests that the strategy for quality adjust-ment of prices should be linked to one of item selectionand chaining. The strategy is particularly pertinent to

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

100

sectors with dynamic technological innovations (see alsothe discussion of hedonic price indices, below).7.15 Second, because of the additional resources

required for quality adjustments to prices, it may be inthe interests of the price collectors and desk statisticians,and indeed fall within their guidelines, to avoid makingnon-comparable replacements and quality adjustments.Thus items continue to be monitored until they are nolonger produced. This means that old items with limitedsales are monitored. Such items may exhibit unusualprice changes as they near the end of their life cycle,because of the marketing strategies of firms. Firms typi-cally identify gains to be made from different pricingstrategies at different times in the life cycle of products,particularly at the introduction and end of the cycle(Parker, 1992). The (implicit or otherwise) weight of end-of-cycle items in the index would thus remain relativelyhigh, being based on their sales share when they weresampled. Furthermore, new unmatched items with pos-sibly relatively large sales would be ignored. As a con-sequence, undue weight would be given to the unusualprice changes of matched items at the end of their lifecycle.7.16 A third sampling concern relates to the timing

of item substitution: when a replacement item is chosento substitute for an old one. Instructions to pick a com-parable replacement to avoid messy quality adjustmentsto prices compound the problem. Obsolete items areby their nature at the end of their cycles and comparablereplacements, to be comparable, must also be near orat the end of their cycles. Obsolete items with unusualprice changes at the end of their cycles are thus re-placed by obsolete items with, again, unusual pricechanges. This compounds the problem of unrepresenta-tive samples and continues to bias the index againsttechnically superior items delivering cheaper serviceflows.7.17 The final sampling problem with the matching

procedure is when the price collector continues to reportprices of items until replacements are forced, that is, untilthe items are no longer available, and has instructions toreplace those items with typically consumed or popularitems. This improves the coverage and representativity ofthe sample. But it also makes reliable quality adjustmentsof prices between the old obsolete and new popular itemsmore difficult. The differences in quality are likely to bebeyond those that can be attributed to price differences insome overlap period, as one item is in the last stages of itslife cycle and the other in its first. Furthermore, thetechnical differences between the items are likely to be ofan order that makes it more difficult to provide reliable,explicit estimates of the effect of quality differences onprices. Finally, the (quality-adjusted) price changes ofvery old and very new items are unlikely to meetassumptions of ‘‘similar price changes to existing itemsor classes of items’’, as required by the imputationmethods. Many of the methods of dealing with qualityadjustment for unavailable items may be better served ifthe switch to a replacement item is made earlier ratherthan later. Sampling concerns can be seen to be inex-tricably linked to quality adjustment methods. This willbe taken up in Chapter 8 on item selection and the need

for an integrated approach to dealing with both repre-sentativity and quality-adjusted prices.

New products7.18 A third potential source of error arises when

something new is introduced into the marketplace. It isdifficult to distinguish between new items and qualitychanges in old ones; this difficulty will be discussed inChapter 8. When a really new item is introduced, there isan immediate gain in welfare or utility as demandswitches from the previous technology and other goods.For example, the introduction of the zip fastener forclothing, instead of buttons, was a completely new goodthat led to an initial gain in utility or welfare to con-sumers as they switched from the old to the new tech-nology. This gain from its introduction would not beproperly brought into the index by waiting until theindex was rebased, or by waiting for at least two suc-cessive periods of prices for zip fasteners and linking thenew price comparison to the old index. Subsequent pricesmight be constant or even fall. The initial welfare gainwould be calculated from a comparison between theprice in the period of introduction and the hypotheticalprice in the preceding period, during which supply wouldbe zero. The practical tools for estimating such a hypo-thetical price are not well developed, though this subjectis discussed in more detail in Chapter 21. For a consumerprice index built on the concept of a base period and afixed basket, there is, strictly speaking, no problem. Thenew good was not in the old basket and should beexcluded. Although an index properly measuring an oldfixed basket would be appropriate in a definitional sense,it would not be representative of what we buy. Such anindex would thus be inappropriate. For a cost of livingindex concerned with measuring the change in expendi-ture necessary to maintain a constant level of utility (seeChapter 17), there is no doubt that it would be con-ceptually appropriate to include the new good.

The nature of quality change7.19 This section considers what is meant by quality

change and then outlines the methods available fordealing with unavailable price quotes. To understand the‘‘meaning’’ of quality change requires a conceptual andtheoretical platform, so that adjustments to prices forquality differences are made against a well-consideredframework.

7.20 A starting point is to appreciate that over timethe quality of what is produced changes. The example ofnew cars is used here. Bode and van Dalen (2001)undertook an extensive study of the price measurementof new cars in the Netherlands between 1990 and 1999.The average nominal price increase over this periodwas found to be around 20 per cent, but the mix ofaverage quality characteristics changed over this period.For example, the horsepower (HP) increased on averagefrom 79 to 92HP; the average efficiency of fuel con-sumption improved from 9.3 to 8.4 litres/100 km; theshare of cars with fuel injection rose from 51 per cent to91 per cent; the proportion of cars with power steering

ADJUSTING FOR QUALITY CHANGE

101

increased from 27 per cent to 94 per cent; airbags from 6per cent to 91 per cent, and similarly for central locking,tinted glass and much more. This churning in the qualitymix of what is purchased is one aspect of quality change.In matching the prices of a sample of models in, forexample, January with the self-same models in sub-sequent months, the quality mix is kept constant in anattempt to avoid contaminating the price measurementthrough quality differences. As will be seen later, how-ever, the resulting sample of models is one that gives lessemphasis to models subsequently introduced which mayhave benefited from more recent technological changeand have different price changes given the quality ofservices they provide. One approach, which corrects forsuch quality changes but uses the whole sample, is that ofthe dummy variable hedonic regressions (see below).Bode and van Dalen (2001), using a variety of formula-tions of hedonic regressions, found quality-correctedprices of these new automobiles to be about constantover this period, while their average nominal priceincrease was around 20 per cent.

7.21 It will be argued in Chapter 21 that observedchanges in prices arise in theory fromanumber of sources,including quality changes, changes in tastes and prefer-ences, and changes in the technology of producers. Moreformally, the observed data on prices are the locus ofintersection of the demand curves of different consumerswith varying tastes and the supply curves of differentproducers with possibly varying technologies of pro-duction. The separation of the effects of changes in tastesand preferences from quality changes is only possiblein highly restrictive circumstances. Chapter 8 suggestschaining or regular rebasing, so that weights – whichreflect tastes and preferences – are not unduly out ofdate.

7.22 The changing mix of the observed character-istics of items is not the only concern. There is also thepractical problem of not always being able to observeor quantify characteristics such as the style, reliability,ease of use and safety of what is produced. Chapter 16of the System of National Accounts, 1993 (SNA 1993) onprice and volume measurement notes factors other thanchanges in physical characteristics that give rise toimproved quality. These include ‘‘transporting a good toa location in which it is in greater demand is a process ofproduction in its own right in which the good is trans-formed into a higher quality good’’. The same goodprovided at a different and more convenient locationmay command a higher price and be of a higher quality.Furthermore, different times of the day or periods of theyear may also give rise to quality differences: ‘‘Forexample, electricity or transport provided at peak timesmust be treated as being of higher quality than the sameamount of electricity or transport provided at off-peaktimes. The fact that peaks exist shows that purchasers orusers attach greater utility to the services at these times,while the marginal costs of production are usuallyhigher at peak times . . . .’’ Other differences, includingthe conditions of sale and circumstances or environmentin which the goods or services are supplied or delivered,can make an important contribution to differences inquality. A retailer, for example, may attract customers

by providing free delivery, credit opportunity or bettervariety, by being more accessible, by offering shorterorder times, smaller tailor-made orders, clearer label-ling, better support and advice, more convenient carparking or a wider range of brands, or simply by oper-ating in a more pleasant or fashionable environment.These sorts of benefits are not always specified in theitem description because, first, the services are providedwithout specific charge – they are incorporated in theprices of the goods sold. Second, by matching the pricesof models in specific outlets the level of such services isassumed to remain constant. This does not mean,however, that conceptually such quality improvementsshould be outside the scope of the index. If any suchbenefits change, a price adjustment for the estimatedvalue of the benefits should be made.

7.23 To ask how to adjust prices for quality changes,it is first necessary to ask what is meant by quality. Whilethere may be an intuition as to whether an item con-sumed in one period is better than its counterpart in thenext, a theoretical framework will help in establishing thebasis for such comparisons. For example, an item ofclothing is sampled and, after a few months, it is missing.One option is to replace it with a similar item. Thenearest comparable option may have more cloth in it, orhave a lining, be a different colour, have different but-tons, have better stitching or be considered to be betterstyled in some fashionable sense. There is a need to put aprice estimate on the difference in quality between theold and new items so that like can be compared with like.To propose or criticize a quality adjustment procedurerequires some concept of what is ideally required andhow the procedure stands up to this. Although such adiscussion takes us away from the practicalities of theprocedures for a while, its use will become apparent insubsequent sections.

A utility-based approach7.24 In Chapter 17 a cost of living index (COLI) is

defined as the ratio of the minimum expenditures in thebase and current period required to achieve a givenstandard of living or ‘‘utility’’. Quality adjustments toprices involve trying to measure the price change for aproduct which has exhibited some change in its char-acteristics from an earlier period that provides a differentlevel of utility to the consumer. The equating of the valueof a quality change with the change in utility derivedby the consumer, while falling naturally under a COLIframework, is not exclusive to it. A cost of a fixed basketof goods index (COGI) can also benefit from regardingquality in this way. While a COGI requires the pricing ofa fixed basket of products, some items will becomeunavailable and the replacement items selected to main-tain the sample may not be of the same quality. The aimis to determine what proportion of the total price changeresults from a change in quality and what results frompure price change. The concept of utility will be used tohelp with the former.

7.25 Note that the definition of a quality change isbased on equating some change in characteristics to adifferent level of utility provided. Consider an example

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

102

in which a new, improved quality item is substituted foran old one in period t, the consumer having to choosebetween the two. Suppose that after the new quality itemappeared, both qualities were offered to a consumer atthe same price, say pt=100. The consumer was thenasked to choose between them and naturally preferredthe new quality. Say the price of the old quality was thenprogressively reduced until it reached a point, say pt*=75, at which the consumer was indifferent as regards thechoice between purchasing the old quality at pt*=75and the new quality at pt=100. The consumer mightthen select the old quality at 75 or the new one at 100.Either way, the consumer would obtain the same utility,because of being indifferent as to which to choose. Anyfurther decrease below pt*=75 would cause the con-sumer to switch back to the old quality.7.26 The difference between pt and pt* would be a

measure of the additional utility that the consumerplaced on the new quality as compared with the oldquality. It would measure the maximum amount that theconsumer was prepared to pay for the new quality overand above the price of the old quality. In economictheory, as will be outlined in Chapter 21, if consumers (orhouseholds) are indifferent between two purchases, theutility derived from them is the same. The differencebetween 75 and 100 must therefore arise from the con-sumers’ valuation of the utility they derive from the twoitems: their quality difference. The definition is sensibleas a conceptual framework. It naturally has problemsrelating to implementation, but this is not our concernhere. Our initial concern is with the provision of ananalytical framework on which to ground our thinkingand analysis.7.27 The utility-based framework is concerned with

the question of how consumers choose between items ofdifferent qualities. The answer, in part, is because moreutility is derived from an item of higher quality than froman item of lower quality, and thus consumers prefer it.But this does not explain why one item is bought ratherthan the other. For this it is also necessary to know therelative price of one item with respect to the other, sinceif the lower-quality item is cheaper, it may still be pur-chased. The above thought experiment to determine theprice below which the old quality would be purchased,pt*� 75, serves this purpose.7.28 Defining quality change in terms of its effect

on utility is of obvious benefit to the economic approachto index numbers (Chapter 21). Fixler and Zieschang(1992), Feenstra (1995), Triplett (1987) and Diewert(2003a) have developed theoretical frameworks forCOLIs akin to those defined in Chapter 21, but whichalso incorporate goods and services whose qualitychanges. Silver and Heravi (2001a and 2003) andKokoski et al. (1999) have undertaken empirical studiesbased on these frameworks for comparisons over timeand between geographical areas, respectively. The use ofutility as a guide towards understanding quality adjust-ments to prices is not, however, confined to the economictheory of cost of living indices (Chapter 21). Consumerprice indices based on a fixed basket concept have thepragmatic need to adjust for quality differences when anitem is unavailable, and there is nothing in the definition

of a fixed basket index that precludes differences in utilitybeing used as a guideline. If item A is better than its oldversion, item B, it is because it delivers somethingmore tothe consumer who is willing to pay more. That ‘‘thing’’ iscalled utility.

7.29 It is as well to distinguish between two conceptsof value used in the analysis of quality adjustment:resource cost and user value. The value users derive fromtheir consumption is their utility. Triplett (1990, pp. 222–223) considers how a consumer price index differs from aproducer price index:

Fisher and Shell (1972) were the first to show that dif-ferent index number measurements (they considered out-put price indexes and consumer price indexes) implyalternative treatments of quality change, and that thetheoretically appropriate treatments of quality change forthese two indexes correspond respectively, to ‘‘resource-cost’’ and ‘‘user-value’’ measures. Triplett (1983) derivesthis same result for cases where ‘‘quality change’’ is iden-tified with characteristics of goods – and therefore withempirical hedonic methods; the conclusions are that theresource cost of a characteristic is the appropriate qualityadjustment for the output price index, and its user value isthe quality adjustment for the COLI index or input index.

7.30 This position is not without difficulties. Diewert(2002d) has advocated a user cost approach for theproducer price output index. This in part arises from theneed to consolidate the inputs and outputs at constantprices in national accounts. If different quality adjust-ments are used for the same items in the producer priceinput index and the producer price output index, then thedeflated constant price value added series, as their dif-ference, will not balance. The issue arises generally in thefield of producer price indices, since it concerns thequestion of whether the producer price output indexshould use a user value concept. It does not dispute theuse of this concept in consumer price indices.

Conditional indices7.31 The domain of a COGI is its fixed basket of

goods and services. The use of a COLI frameworkrequires consideration of wider issues concerning ourquality of life. There are changes in the social, physicaland economic environment that require more or lessexpenditure to maintain a given level of utility. Manyfactors affect our welfare, and in practice not all can beincluded in a consumer price index. It is thus appro-priate to consider indices that are conditional on exclu-ded factors remaining constant. These generally includehealth status, the environment and the quantity andquality of government-provided goods and services. Theminimum expenditure necessary for achieving a givenlevel of utility will increase as, for example, the policebecome less effective. Expenditure would then be nec-essary for better household security. It would cost moreto maintain a given level of utility than in the previousperiod. Similarly, an outbreak of illness would lead toincreased expenditure on medicines to maintain a givenlevel of utility. Bad winter weather increases heating billsto maintain the same utility as before. In each casethere is a very real sense in which the cost of living willhave changed. Yet it is not generally accepted that

ADJUSTING FOR QUALITY CHANGE

103

the consumer price index should directly reflect suchchanges. What should be reflected are changes in theprices of locks, medicines and fuel that arise because thedemand for such items changes. In addition, as more orless is spent on such items, the index should eventuallyincorporate such changes in the weighting as and whenthe weights are updated – and the more frequent theupdate, the better are such effects incorporated. But theindex should not normally reflect short-run changes inthe quantities used of security, medicine, heat and thelike as a result of such external factors. Gordon andGriliches (1997, p. 87) comment in a similar vein:

It is not clear, moreover, whether events such as acolder winter, the appearance of AIDS, or a rise in thecrime rate should be included in the definition of a priceindex. A change in expenditures due to an unanticipatedchange in the weather should raise the price index only tothe extent that energy prices go up, not quantities con-sumed. If the event persists, ultimately it will affect thecommodity weights in the index, but that is a differentmatter. (Authors’ emphasis)

7.32 It may be inappropriate to disregard environ-mental factors if they seriously affect a given group ofpeople. In such cases, indexing for special factors some-times takesplaceoutside the index.Forexample, agovern-ment may provide cold-weather payments to pensionersif the temperature falls below a threshold condition. If aspecific factor has a substantial effect for a significantgroup of households, an additional index might becompiled which includes the effect.

An overview of methods ofquality adjustment when matcheditems are unavailable

7.33 It is apparent from the above that qualityadjustments to prices are not a simple matter of applyingroutine methods to prices in specified product areas. Anumber of alternative approaches are suggested below.Some will be more appropriate than others for specificproduct areas. An understanding of the consumer mar-ket, technological features of the producing industry,and alternative data sources will all be required for thesuccessful implementation of quality adjustments. Spe-cific attention will need to be devoted to product areaswith relatively high weights, where large proportions ofitems are turned over. Some of the methods are notstraightforward and require a level of expertise. Qualityadjustment needs to be implemented by developing agradual approach on a product-by-product basis. Suchconcerns should not be used as excuses for failing toattempt to estimate quality-adjusted prices. The practiceof statistical agencies in dealing with missing items, evenif it is to ignore them, implicitly involves a quality adjust-ment. Such an implicit approach may not be the mostappropriate method, and may even be misleading. Theextent of quality changes and the pace of technologicalchange require that appropriate methods be used.

7.34 To measure aggregate price changes, a repre-sentative sample of items is selected from a sample ofoutlets, along with a host of details that define each

price. The items are repriced each month. The detailedspecifications are included on the repricing form eachmonth as a prompt to help ensure that the same items arebeing priced. Merkel (2000) has proposed that de-tailed checklists of item descriptions should be used, asany lack of clarity in the specifications may lead toerrors. It should be borne in mind that price collectorsmay have no incentive to report changes in specifica-tions, since this will invariably involve additional work.Attention should also be devoted to ensuring that thespecifications used contain all pertinent, price-deter-mining elements, otherwise there may be cases in whichthe quality change would become invisible in the pricemeasurement process.

7.35 When an item is missing in a month for reasonsother than being off season or off cycle, the replacementmay be of a different quality – like may no longer becompared with like. A number of approaches exist fordealing with such situations and are well documented forthe consumer price index (CPI), as outlined in Turveyet al. (1989), Moulton and Moses (1997), Armknechtet al. (1997), Moulton et al. (1999) and Triplett (2002).Though the terminology differs between authors andstatistical agencies, they include:

� imputation – where no information is available toallow reasonable estimates to be made of the effect onprice of a quality change. The price changes of allitems, or of more or less similar items, are assumed tobe the same as that for the missing item;

� overlap – used where no information is available toallow reasonable estimates to be made of the effect onprice of a quality change, but where a replacementitem exists in the same period as the old item. Theprice difference between the old item and its replace-ment in the overlap period is then used as a measureof the quality difference;

� direct comparison – if another item is directly com-parable, that is, it is so similar that it can be assumedto have had more or less the same quality character-istics as the missing one, its price replaces the una-vailable price. Any difference in price level betweenthe new and old is assumed to arise from pricechanges and not quality differences;

� explicit quality adjustment – where there is a sub-stantial difference between the quality of the old andreplacement items, estimates of the effect of qualitydifferences on prices are made to enable quality-adjusted price comparisons to be made.

7.36 Before outlining and evaluating these methodsit is as well to say something about the extent of theproblem. This arises when the item is unavailable. It isnot just a problem when comparable items are unavail-able, for the judgement as to what is and what is notcomparable itself requires an estimate of quality differ-ences. Part of a statistical meta-information system forstatistical offices (outlined in Chapter 8) is to identify andmonitor sectors that are prone to such replacements andwhether the replacements used really are comparable.Seminal studies in Canada and the United States throwsome light on the extent of such replacements. Moultonet al. (1999) examined the extent to which items became

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

104

unavailable for televisions in the compilation of theUnited States CPI. Between 1993 and 1997, a total of10,553 prices on televisions were used, of which 1,614(15 per cent) were replacements, of which, in turn, 934(57 per cent) were judged to be directly comparable. Thusa typical television remained in the sample less than ayear. The Canadian experience for televisions over analmost identical period (1993 to November 1997) found750 of the 10,050 prices (7.5 per cent) to be replacements.Of these, 178 (24 per cent) were directly comparable, 162(22 per cent) required a judgement and 410 (55 per cent)were ‘‘spliced’’ – the price difference between thereplacement and the unavailable model in the two peri-ods being attributed to quality differences (Lowe, 1999).Thus, there was wide variation in the frequency oftotal replacements, although the frequency of non-comparable replacements was roughly similar (6.4 percent in the United States sample and 5.7 per cent inCanada). Liegey (2000) found that of the 215 average(August 1999 to April 2000) monthly prices collected formajor appliances for the United States CPI, 22 itemreplacements were required because of missing prices, ofwhich comparable replacements were found for 16 andnon-comparable replacements for the remaining six.7.37 Information across a wider range of items is

available for the United States. Armknecht (1996) foundthat, over the three years 1993 to 1995, the annualaverage number of price observations collected for theUnited States CPI was 835,443, of which 59,385 (7.1 percent) were substitutions (as opposed to imputations formissing values). Of these substitutes, about half werecarried out using comparable replacements, under aquarter using overall mean imputation, about 12 per centusing direct quality adjustment, and 10 per cent usingclass mean imputation. It should be borne in mind thatthese figures ignore the implicit quality adjustments thattake place when the Bureau of Labor Statistics rotates itssample between rebasing. The overlap method is effec-tively applied on sample rotation, the outlet and itemsamples being reselected for about one-fifth of the geo-graphical areas, with prices of old and new items sampledin the same month. All price-level differences between theold and new items are treated as quality differences as thenew sample is spliced onto the old.7.38 Methods of quality adjustment for prices are

generally classified into implicit/imputed (or indirect)quality adjustment methods – the differences in termi-nology are notorious in this area – and explicit (or direct)methods. Implicit and explicit methods are discussedbelow. Both decompose the price change between the olditem and its replacement into quality and pure pricechanges. For explicit adjustments, however, an explicitestimate is made of the quality difference, usually on thebasis of external information, and the pure price effect isidentified as a remainder. For implicit adjustments, ameasurement technique is used to compare the old itemto the replacement item, in which the extent of thequality and pure price change is implicitly determined bythe assumptions of the method. The accuracy of themethod relies on the veracity of the assumptions asopposed to the quality of the explicit estimate. Explicitadjustments make use of separate estimates of the por-

tion of prices ascribed to quality differences, so that theprice of the original item can be compared with that of areplacement of the same quality. The suitability of theexplicit methods thus depends to a large extent on howgood such estimates are, on average. Implicit adjust-ments involve assumptions about price movements, andfor these informed intuition or theory is relied upon –though in some cases national statistical offices maymake use of more specific empirical market knowledge.

Additive versus multiplicativeadjustment

7.39 The quality adjustments to prices may beundertaken either by adding a fixed amount or multi-plication by a ratio. For example, wherem is the old itemand n its replacement for a comparison over periods t,t+1, t+2, the use of the overlap method in period t+1requires the ratio pt+1

n =pt+1m to be used as a measure of

the relative quality difference between the old item andits replacement. This ratio could then bemultiplied by theprice of the old item in period t, ptm, to obtain the quality-adjusted prices p*tm as follows:

t t+1 t+2

old item m pt+1m

replacement n p� tm pt+1

n pt+2n

7.40 Such multiplicative formulations are generallyadvised, as the adjustment is invariant to the absolutevalue of the price. It would otherwise be possible for theabsolute value of the change in specifications to exceedthe value of the item in some earlier or (with techno-logical advances) later period. Yet there may be someitems for which the worth of the constituent parts is notconsidered to be in proportion to the price. In otherwords, the constituent parts have their own, intrinsic,absolute, additive worth, which remains constant overtime. Producers selling over the World Wide Web may,for example, include postage, which in some instancesmay remain the same irrespective of what is happening toprice. If postage is subsequently excluded from the price,this fall in quality should be valued as a fixed sum.

Base versus current period adjustment7.41 Two variants of the approaches to quality

adjustment are to make the adjustment either to the pricein the base period or to the price in the current period.For example, in the overlap method, described above,the implicit quality adjustment coefficient was used toadjust ptm. An alternative procedure would have been tomultiply the ratio pt+1

m =pt+1n by the price of the replace-

ment item pt+2n to obtain the quality-adjusted price p*t+2

n ,etc. The first approach is easier since, once the baseperiod price has been adjusted, no subsequent adjust-ments are required. Each new replacement price can becompared with the adjusted base period price. Formultiplicative adjustments, the end result is the samewhichever approach is used. For additive adjustments,

ADJUSTING FOR QUALITY CHANGE

105

the results differ and it is more appropriate to make theadjustment to prices near to the overlap period.

Long-run versus short-run comparisons7.42 Much of the analysis of quality adjustments in

this manual has been undertaken by comparing pricesbetween two periods, say, period 0 prices with those in asubsequent period 1. For long-run comparisons the baseperiod is taken as, say, period t and the index is compiledby comparing prices in period t first with t+1; then twitht+2; then t with t+3, etc. The short-run frameworkallows long-run comparisons, say, between periods t andt+3, to be built up as a sequence of links joined togetherby successive multiplication, say period t with t+2 andperiod t+2 with t+3; or with chaining, period t witht+1, t+1 with t+2 and t+2 with t+3. The advantagesof the short-run framework for imputations are dis-cussed in paragraphs 7.165 to 7.173.

7.43 Following a discussion of implicit and explicitmethods of quality adjustment, issues relating to choiceof method are considered. The implicit and explicitadjustment methods are outlined under a standard long-run Laspeyres framework, in which prices in a base (orreference) period are compared with those in eachsubsequent period. Where products are experiencingrapid technological change, however, these methodsmay be unsuitable. The matching and repricing of likeitems, and ‘‘patching in’’ of quality-adjusted replace-ment prices when the matching fails, are appropriatewhen failures are the exception. But in high-technologyproduct markets likely to experience rapid turnover ofmodels, they are the rule. Alternative methods usingchained or hedonic frameworks are therefore also con-sidered. These are quite radical approaches to meet theneeds of rapidly changing production portfolios.Finally, the use of short-run comparisons as an alter-native to long-run ones is considered as an intermediary– and for imputation a more appropriate – approach.Chapter 22 discusses issues relating to seasonal items inmore detail.

Implicit methods of qualityadjustment

7.44 This section discusses the following implicitmethods of quality adjustment: overlap; overall mean ortargeted mean imputation; class mean imputation; com-parable replacement; linked to show no price change;and carry-forward.

Overlap7.45 Consider for illustration the case where the

items are sampled in, say, January and prices are com-pared over the remaining months of the year. Matchedcomparisons are undertaken between the January pricesand their counterparts in successive months. Five itemsare assumed to exist in January sold by two outlet typeswith prices p11; p21; p51; p61 and p81 (Table 7.1(a)). At thislevel of aggregation, the weights can be ignoredassuming only one quote is taken on each item. A price

index for February compared with January=100.0 isstraightforward, in that prices of items 1, 2, 5, 6 and 8only are used and compared by way of the geometricmean of price ratios, the Jevons index (which isequivalent to the ratio of the geometric mean in Feb-ruary over the geometric mean in January – see Chapter20). In March the prices for items 2 and 6 are missing,one from specialized chain stores and one from depart-ment stores.

7.46 Table 7.1(b) is a numerical counterpart to Table7.1(a) to further illustrate the calculations. The overlapmethod requires prices of the old and replacement itemsto be available in the same period. In Table 7.1(a), inMarch item 2 has no price quote. Its new replacement is,say, item 4. The overlap method simply measures theratio, in a common overlap period (February), of theprices of the old and replacement items (items 2 and 4,respectively). This is taken to be an indicator of theirquality differences. The two approaches outlined aboveare apparent: either to insert a quality-adjusted price inJanuary for item 4 and continue to use the replacementitem 4 series, or to continue the item 2 series by patchingin quality-adjusted item 4 prices. Both yield the sameanswer. Consider the former. For a Jevons geometricmean for January to March for specialized chain storesonly, assuming equal weights of unity:

PJ( p1, p3)=[ p13=p11 � p43=(( p42=p22)� p21Þ]1=2

=[6=4� 8=((7:5=6)� 5)]1=2=1:386: (7.1)

Table 7.1 Example of the implicit methods of qualityadjustment(a) General illustration

Outlet Item January February March April

Specialized chain stores 1 p11 p12 p13 p14

2 p21 p22

3 p33 p34

4 p42 p43 p44

Department stores 5 p51 p52 p53 p54

6 p61 p62

7 p73 p74

8 p81 p82 p83 p84

(b) Numerical illustration

Outlet Item January February March

Specializedchain stores

1 4 5 6

2 5 62. overlap 6.9– imputation 6.56– targeted imputation 7.2– comparable replacement 6.53 6.54 7.5 8

Departmentstores

5 10 11 12

6 12 12– imputation 13.13– targeted imputation 12.5337 148 10 10 10

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

106

Note that the comparisons are long-run ones. That is,they are between January and the month in question.The short-run modified Laspeyres framework provides abasis for short-run changes based on data in each currentmonth and the immediately preceding one. In Tables7.1(a) and (b) the comparison for specialized chain storesonly would first be undertaken between January andFebruary using items 1 and 2, and this would be multi-plied by the comparison between February and Marchusing items 1 and 4. This still implicitly uses the differ-ences in prices in the overlap in February between items 2and 4 as a measure of this quality difference. It yields thesame result as before:

5

4� 6

5

� �12� 6

5� 8

7:5

� �12=1:386

The advantage of recording price changes for, say, Jan-uary to October in terms of January to September, andSeptember to October, is that it allows the compiler tocompare immediate month-on-month price changes fordata editing purposes. Moreover, it has quite specificadvantages for the use of imputations (as discussed inparagraphs 7.53 to 7.68 below) for which different resultsarise for the long- and short-run methods. The long-runand short-run frameworks are discussed more fully inparagraphs 7.159 to 7.173.7.47 The method is only as good as the validity of its

underlying assumptions. Consider i=1 . . .m items whereptm is the price of item m in period t, pt+1

n is the price of areplacement item n in period t+1, and there are overlapprices for both items in period t. Now item n replaces m,but is of a different quality. So let A(z) be the qualityadjustment to pt+1

n which equates its quality to pt+1m such

that the quality-adjusted price p*t+1m =A(zt+1)pt+1

n . Verysimply, the index for the item in question over the periodt�1 to t+1 is:

( ptm=pt�1m )� ( pt+1

n =ptn)=I t�1,t+1

=pt+1n

pt�1m

� ptmptn

(7.2)

7.48 Now the quality adjustment to prices in periodt+1 is defined as previously, p*t+1

m =A(zt+1)pt+1n , which

is the adjustment to pn in period t+1 which equates itsutility to pm in period t+1 (had it existed then). Adesired measure of price changes between periods t�1and t+1 is thus:

( p*t+1m =pt�1m ) (7.3)

The overlap formulation equals this when:

p*t+1

pt�1m

=A(zt+1)pt+1n

pt�1m

=pt+1n

ptn� ptm

pt�1m

A(zt+1)=ptmptnand similarly for future periods of the series

A(zt+1)=ptmptn

forp*t+im

pt�1m

for i=2; . . .T (7.4)

The assumption is that the quality difference in anyperiod equates to the price difference at the time of thesplice. The timing of the switch from m to n is thuscrucial. Unfortunately, price collectors usually hangonto an item so that the switch may take place at anunusual period of pricing, near the end of item m’s lifecycle and the start of item n’s life cycle.

7.49 But what if the assumption does not hold?Whatif the relative prices in period t, Rt=ptm=p

tn, do not equal

A(z) in some future period, say A(zt+i)=aiRt? If ai=a,the comparisons of prices between future successiveperiods, say between t+3 and t+4, are unaffected, aswould be expected, since item n is effectively beingcompared with itself,

p*t+4m

pt�1m

.p*t+3m

pt�1m

=aRt

aRt

pt+4n

pt+3n

=pt+4n

pt+3n

(7.5)

However, if differences in the relative prices of the oldand replacement items vary over time, then:

p*t+4m

pt�1m

.p*t+3m

pt�1m

=a4a3

pt+4n

pt+3n

(7.6)

Note that the quality difference here is not related to thetechnical specifications or resource costs, but to therelative prices consumers pay.

7.50 Relative prices may also reflect unusual pricingpolicies aimed at minority segments of the market. Inthe example of pharmaceutical drugs (Berndt et al.,2003), the overlap in prices of a generic with a brandedproduct was argued to reflect the needs of two differentmarket segments. The overlap method can be usedwith a judicious choice of the overlap period. It shouldif possible be a period before the use of the replace-ment since in such periods the pricing may reflect astrategy to dump the old model to make way for thenew one.

7.51 The overlap method is implicitly employedwhen samples of items are rotated. That is, the oldsample of items is used to compute the category indexprice change between periods t�1 and t, and the newsample is used between t and t+1. The ‘‘splicing’’together of these index movements is justified by theassumption that – on a group-to-group rather thanitem-to-item level – differences in price levels at acommon point in time accurately reflect differences inqualities.

7.52 The overlap method has at its roots a basis inthe law of one price: that when a price difference isobserved it must arise from some difference in physicalquality or some such factors for which consumers arewilling to pay a premium, such as the timing of the sale,location, convenience or conditions. Economic theorywould dictate that such price differences would not per-sist, given markets made up of rational producers andconsumers. However, Chapter 16 of SNA 1993 notesthree reasons why this might fail:

First, purchasers may not be properly informed aboutexisting price differences and may therefore inadvertentlybuy at higher prices. While they may be expected tosearch out for the lowest prices, costs are incurred in theprocess.

ADJUSTING FOR QUALITY CHANGE

107

Secondly, purchasers may not be free to choose theprice at which they purchase because the seller may be in aposition to charge different prices to different categories ofpurchasers for identical goods and services sold underexactly the same circumstances – in other words, topractise price discrimination.

Thirdly, buyers may be unable to buy as much as theywould like at a lower price because there is insufficientsupply available at that price. This situation typicallyoccurs when there are two parallel markets. There maybe a primary, or official, market in which the quantitiessold, and the prices at which they are sold, are subjectto government or official control, while there maybe a secondary market – a free market or unofficialmarket – whose existence may or may not be recognizedofficially.

Overall mean or targetedmean imputation

7.53 This method uses the price changes of otheritems as estimates of the price changes of the missingitems. Consider a Jevons elementary price index, i.e.,a geometric mean of price relatives (Chapter 20).The prices of the missing items in the current period,say t+1, are imputed by multiplying their prices inthe immediately preceding period t by the geometricmean of the price relatives of the remaining matcheditems between these two periods. The comparison isthen linked by multiplication to the price changes forprevious periods. It is the computationally moststraightforward of methods since the estimate can beundertaken by simply dropping the items that aremissing from both periods from the calculation. Inpractice, the series is continued by including in thedatabase the imputed prices. It is based on theassumption of similar price movements. A targetedform of the method would use similar price move-ments of a cell or elementary aggregate of similar items,or be based on price changes at a higher level ofaggregation if either the lower level had an insufficientsample size or price changes at the higher level werejudged to be more representative of the price changes ofthe missing item.

7.54 In the example in Table 7.1, the January toFebruary comparison for both outlet types is based onitems 1, 2, 5, 6 and 8. For March compared with Jan-uary – weights all equal to unity – the item 2 and item 6prices are imputed using the short-run price change forFebruary ( p2) compared with March ( p3) based onitems 1, 5 and 8. Since different formulae are used forelementary aggregation, the calculations for the mainthree formulae are illustrated here (but see Chapter 20on choice of formulae). The geometric mean of the priceratios – the Jevons index – is:

PJ( p2, p3)=

QNi=1

[ p3i =p2i ]1=N ð7:7Þ

=[( p13=p12)� ( p53=p52)� ( p83=p82)]1=3

=[(6=5)� (12=11)� (10=10)]1=3=1:0939,

or a 9:39 per cent increase:

The ratio of the average (mean) prices – the Dutot index– is:

PD( p2, p3)=

PNi=1

p3i =N

� �. PNi=1

p2i =N

� �ð7:8Þ

=[( p13+p53+p83)=3� ( p12+p52+p82)=3]

=(6+12+10)=(5+11+10)=1:0769,

or a 7:69 per cent increase:

The average (mean) of the price ratios – the Carli index– is:

PC( p3, p2)=

PNi=1

( p3i =p2i )=N ð7:9Þ

=[( p13=p12)+( p53=p52)+( p83=p82)]=3

=[6=5+12=11+10=10]=3=1:09697,

or a 9:697 per cent increase:

In practice, the imputed figure would be entered on thedata sheet. In Table 7.1(b) the overall mean imputationsin March for items 2 and 6, using the Jevons index, are1:0939� 6=6:563 and 1:0939� 12=13:127, respec-tively: these are shown in bold. It should be noted thatthe Dutot index is in this instance lower that the Jevonsindex, a result not expected from the relationshipestablished in Chapter 20. The relationship in Chapter 20assumed that the variance of prices would increase overtime, while in Table 7.1(b) it decreases for the three items.The arithmetic mean of relatives, the Carli index, equallyweights each price change while the ratio of arithmeticmeans, the Dutot index, weights price changes accordingto the prices of the item in the base period relative to thesum of the base period prices. Item 1 has a relatively lowprice (4), and thus weight, in the base period, but thisitem has the highest price increase (6/5). The Dutot indexis thus lower than the Carli index.

7.55 As noted above, it is also possible to refinethe imputation method by ‘‘targeting’’ the imputation: byincluding the weight for the unavailable items in group-ings likely to experience similar price changes, say byoutlet type, specific product area or geographical region.Any stratification system used in the selection of outletswould facilitate this. For example, in Table 7.1 assumethat the price change of the missing item 2 in March ismore likely to follow price changes of item 1 in specializedchain stores, and item 6 is more likely to experiencesimilar price changes to those of items 5 and 8 indepartment stores. For March compared with February,and weights all equal to unity, the geometric mean of theprice ratios – the Jevons index – is:

PJ( p2, p3)=

QNi=1

( p3i =p2i )1=N (7.10)

=[( p13=p12)2 � ( p53=p52 � p83=p82)3=2]1=5

=[(6=5)2 � (12=11� 10=10)3=2]1=5=1:1041:

Note the weights used: for specialized chain stores theone price represents two prices, while for departmentstores the two prices represent three prices, or 3/2=1.5each.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

108

The ratio of the average (mean) prices – the Dutot index– is:

PD( p2, p3)=

PNi=1

p3i

.N

� �. PNi=1

p2i

.N

� �ð7:11Þ

=[(2 p13+1:5 p53+1:5 p83)=5

� (2 p12+1:5 p52+1:5 p82)=5]

=[(2� 6+1:5� 12+1:5� 10)� (2� 5+1:5� 11+1:5� 10)]=1:0843

The average (mean) of the price ratios – the Carli index– is:

PC( p2, p3)=

PNi=1

( p3i =p2i ).N ð7:12Þ

=2

5( p13=p12)+

3

5[( p53=p52+p83=p82)=2]

=2

5(6=5)+

3

5[(12=11+10=10)=2]=1:1073

7.56 Alternatively, and more simply, imputed fig-ures could be entered in Table 7.1(b) for items 2 and 6 inMarch, just using specialized chain stores and depart-ment store price movements for items 2 and 6 respec-tively, and indices calculated accordingly. Using a Jevonsindex, for item 2 the imputed value in March would be6=5� 6=7:2 and for item 6 it would be [(12=11)�(10=10)]1=2=12:533. It is thus apparent that not onlydoes the choice of formula matter, as discussed inChapter 20, but so too may the targeting of the impu-tation. In practice, the sample of items in a targetedsubgroup may be too small. An appropriate stratumis required with a sufficiently large sample size, butthere may be a trade-off between the efficiency gainsfrom the larger sample and the representativity of pricechanges achieved by that sample. Stratification by pro-duct area and region may be preferred to stratificationjust by product area, if regional differences in pricechanges are expected, but the resulting sample size maybe too small. In general, the stratum used for the targetshould be based on the analyst’s knowledge of themarket, as well as an understanding of similarities ofprice changes between and within strata, and the relia-bility of the sample available to be representative ofprice changes.7.57 The underlying assumptions of these methods

require some analysis since, as discussed by Triplett (1999and 2002), they are often misunderstood. Consideri=1 . . .m items where, as before, ptm is the price of itemmin period t, pt+1

n is the price of a replacement item n inperiod t+1. Now n replaces m, but is of a differentquality. So, as before, let A(z) be the quality adjustmentto pt+1

n which equates its quality services or utility to pt+1m

such that the quality-adjusted price p*t+1m =A(z)pt+1

n . Forthe imputation method to work, the average price chan-ges of the i=1 . . .m items, including the quality-adjustedprice p*t+1

m , given on the left-hand side of equation (7.13),must equal the average price change from just using theoverall mean of the rest of the i=1 . . .m� 1 items, on the

right-hand side of equation (7.13). The discrepancy orbias from the method is the balancing term Q. It is theimplicit adjustment that allows the method to work. Thearithmetic formulation is given here, though a similargeometric one can be readily formulated. The equationfor one unavailable item is given by:

1

m

p*t+1m

ptm+Pm�1i=1

pt+1i

pti

� �=

1

(m� 1)Pm�1i=1

pt+1i

pti

� �+Q (7.13)

Q=1

m

p*t+1m

ptm� 1

m(m� 1)Pm�1i=1

pt+1i

pti(7.14)

and for x unavailable items by:

Q=1

m

Pxi=1

p*t+1m

ptm� x

m(m� x)

Pm�xi=1

pt+1i

pti(7.15)

7.58 The relationships are readily visualized if r1 isdefined as the arithmetic mean of price changes of itemsthat continue to be recorded and r2 of quality-adjustedunavailable items. For the arithmetic case,

where r1=Pm�xi=1

pt+1i =pti

� �� (m� x) and

r2=Pxi=1

p*t+1i =pti

� �� x

(7.16)

then the ratio of arithmetic mean biases from sub-stituting equation (7.16) in (7.15) is:

Q=x

m(r2 � r1) (7.17)

which equals zero when r1=r2. The bias depends on theratio of unavailable values and the difference between themean of price changes for existing items and the mean ofquality-adjusted replacement price changes. The biasdecreases as either (x=m) or the difference between r1 andr2 decreases. Furthermore, the method is reliant on acomparison between price changes for existing items andquality-adjusted price changes for the replacement orunavailable comparison. This is more likely to be justi-fied than a comparison without the quality adjustment toprices. For example, suppose there were m=3 items,each with a price of 100 in period t. Let the t+1 prices be120 for two items, but assume the third is unavailable,i.e., x=1 and is replaced by an item with a price of 140,of which 20 is attributable to quality differences. Thenthe arithmetic bias as given in equations (7.16) and(7.17), where x=1 and m=3, is

1

3(� 20+140)=100� 120

100+120

100

� �=2

� �=0

Had the bias depended on the unadjusted price of 140compared with 100, the imputation would be prone toserious error. In this calculation, the direction of the biasis given by (r2 � r1) and does not depend on whetherquality is improving or deteriorating, in other wordswhether A(z)> pt+1

n or A(z)< pt+1n . If A(z)> pt+1

n , aquality improvement, it is still possible that r2 < r1 andfor the bias to be negative, a point stressed by Triplett(2002).

ADJUSTING FOR QUALITY CHANGE

109

7.59 The analysis here is framed in terms of a short-run price change framework. That is, the short-run pricechanges between the prices in a period and those in thepreceding period are used for the imputation. This isdifferent from the long-run imputation where a baseperiod price is compared with prices in subsequentmonths, and where the implicit assumptions are morerestrictive.

7.60 Table 7.2 provides an illustration in which the(mean) price change of items that continue to exist, r1,is allowed to vary for values between 1.00 and 1.5 –corresponding to a variation between no price changeand a 50 per cent increase. The (mean) price change ofthe quality-adjusted new items compared with the itemsthey are replacing is assumed not to change, i.e., r2=1.00. The bias is given for ratios of missing values of0.01, 0.05, 0.1, 0.25 and 0.5, both for arithmetic meansand geometric means. For example, if 50 per cent ofprice quotes are missing and the missing quality-adjus-ted prices do not change, but the prices of existing itemsincrease by 5 per cent (r1=1.05), then the bias for thegeometric mean is represented by the proportional fac-tor 0.9759; i.e., instead of 1.05, the index should be0.9759� 1.05=1.0247. For an arithmetic mean, the biasis � 0.025; instead of 1.05 it should be 1.025.

7.61 Equation (7.17) shows that the ratio x/m andthe difference between r1 and r2 determine the bias. Table7.2 shows that the bias can be quite substantial when x/mis relatively large. For example, for x/m=0.25, an infla-tion rate of 5 per cent for existing items translates to anindex change of 3.73 per cent and 3.75 per centfor the geometric and arithmetic formulations, respec-tively, when r2=1.00, i.e., when quality-adjusted pricesof unavailable items are constant. Instead of being1.0373 or 1.0375, ignoring the unavailable items wouldgive a result of 1.05. Even with 10 per cent missing (x/m=0.1), an inflation rate of 5 per cent for existing itemstranslates to 4.45 per cent and 4.5 per cent for the geo-metric and arithmetic formulations, respectively, whenr2=1.00. Considering a fairly low ratio of x/m, say 0.05,then even when r2=1.00 and r1=1.20, Table 7.2 showsthat the corrected rates of inflation should be 18.9 per

cent and 19 per cent for the geometric and arithmeticformulations, respectively. In competitive markets, r1and r2 are unlikely to differ by substantial amounts sincer2 is a price comparison between the new item and the olditem after adjusting for quality differences. If r1 and r2are the same, then there would be no bias from themethod even if x/m=0.9. There may, however, be moresampling error. It should be borne in mind that it is notappropriate to compare bias between the arithmetic andgeometric means, at least in the form they take in Table7.2. The latter would have a lower mean, renderingcomparisons of bias meaningless.

7.62 An awareness of the market conditions relatingto the commodities concerned is instructive in under-standing likely differences between r1 and r2. The con-cern here is when prices vary over the life cycle of theitems. Thus, for example, at the introduction of a newmodel, the price change may be quite different fromprice changes of other existing items. Thus assumptionsof similar price changes, even with quality adjustment,might be inappropriate. Greenlees (2000) gives the exam-ple of personal computers: new computers enter the mar-ket at prices equal to, or lower than, prices of previousmodels, but with greater speed and capability. An as-sumption that r1=r2 could not be justified. He continueswith the example of apparel, in which new clothingenters the market at relatively high quality-adjustedprices, while old, end-of-season or out-of-style clothesare being discounted. Again there will be bias, as r1differs from r2.

7.63 Some of these differences arise because marketsare composed of different segments of consumers.Indeed, the very training of consumer marketers involvesconsideration of developing different market segmentsand ascribing to each appropriate pricing, productquality, promotion and place (method of distribution) –the 4Ps of the marketing mix (Kotler, 1991). In addition,consumer marketers are taught to plan the marketingmix for the life cycle of items. Such planning allows fordifferent inputs of each of these marketing mix variablesat different points in the life cycle. This includes ‘‘priceskimming’’ during the period of introduction, when

Table 7.2 Example of the bias from implicit quality adjustment when the (mean) price change of quality-adjusted new itemscompared with the items they are replacing is assumed not to change (r2=1.00)

Geometric mean Arithmetic mean

Ratio of missing items, x/m Ratio of missing items, x/m0.01 0.05 0.1 0.25 0.5 0.01 0.05 0.1 0.25 0.5

r11 1 1 1 1 1 0 0 0 0 01.01 0.999901 0.999503 0.999005 0.997516 0.995037 �0.0001 �0.0005 �0.001 � 0.0025 � 0.0051.02 0.999802 0.99901 0.998022 0.995062 0.990148 �0.0002 �0.001 �0.002 � 0.005 � 0.011.03 0.999704 0.998523 0.997048 0.992638 0.985329 �0.0003 �0.0015 �0.003 � 0.0075 � 0.0151.04 0.999608 0.998041 0.996086 0.990243 0.980581 �0.0004 �0.002 �0.004 � 0.01 � 0.021.05 0.999512 0.997563 0.995133 0.987877 0.9759 �0.0005 �0.0025 �0.005 � 0.0125 � 0.0251.1 0.999047 0.995246 0.990514 0.976454 0.953463 �0.001 �0.005 �0.01 � 0.025 � 0.051.15 0.998603 0.993036 0.986121 0.965663 0.932505 �0.0015 �0.0075 �0.015 � 0.0375 � 0.0751.2 0.998178 0.990925 0.981933 0.955443 0.912871 �0.002 �0.01 �0.02 � 0.05 � 0.11.3 0.99738 0.986967 0.974105 0.936514 0.877058 �0.003 �0.015 �0.03 � 0.075 � 0.151.5 0.995954 0.979931 0.960265 0.903602 0.816497 �0.005 �0.025 �0.05 � 0.125 � 0.25r1=(mean) price change for items that continue to exist.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

110

higher prices are charged to skim off the surplus fromsegments of consumers willing to pay more. The eco-nomic theory of price discrimination would also predictsuch behaviour. Thus the quality-adjusted price changeof an old item compared with a new replacement itemmay be higher than price changes of other items in theproduct group. After the introduction of the new item itsprices may fall relative to others in the group. There maybe no law of one price change for differentiated itemswithin a market. Berndt et al. (2003) clearly show how,after patents expire, the price of branded prescriptionpharmaceuticals can increase with the entry of newgeneric pharmaceuticals at a lower price, as particularlyloyal, less price-sensitive customers maintain their al-legiance to the branded pharmaceuticals.7.64 There is thus little in economic or marketing

theory to support any expectation of similar (quality-adjusted) price changes for new and replacement items,as compared to other items in the product group. Someknowledge of the realities of the particular market understudy would be helpful when considering the suitabilityof this approach. Two aspects need to be considered inany decision to use the imputation approach. The firstis the proportion of replacements; Table 7.2 providesguidance here. The second is the expected differencebetween r1 and r2. It is clear from the above discussionthat there are markets in which they are unlikely to besimilar. This is not to say the method should not be used.It is a simple and expedient approach. What arguablyshould not happen is that it is used by default, withoutany prior evaluation of expected price changes and thetiming of the switch. Furthermore, its use should betargeted, by selecting items expected to have similar pricechanges. The selection of such items, however, shouldtake account of the need to include a sufficiently largesample so that the estimate is not subject to unduesampling error.7.65 The manner in which these calculations are

undertaken is also worth considering. In its simplestform, the pro forma setting for the calculations, say on aspreadsheet, would usually have each item descriptionand its prices recorded on a monthly basis. The imputedprices of the missing items are inserted into the spread-sheet, and are highlighted to show that they are imputed.The need to highlight such prices is, first, because theyshould not be used in subsequent imputations as if theywere actual prices. Second, the inclusion of imputedvalues may give a false impression of a larger sample sizethan actually exists. Care should be taken in any audit ofthe number of prices used in the compilation of the indexto code such observations as ‘‘imputed’’.7.66 The method described above is an illustration

of a short-run imputation. As is discussed in paragraphs7.165 to 7.173 below, there is a strong case for usingshort-run imputations as against long-run ones.

Class mean imputation7.67 The class mean (or substitution relative)

method of implicit quality adjustment to prices as used inthe United States CPI is discussed by Schultz (1996),Reinsdorf, Liegey and Stewart (1996), Armknecht, Lane

and Stewart (1997), and Armknecht andMaitland-Smith(1999). It arose from concerns similar to those con-sidered in the previous section, that unusual pricechanges were found in the early introductory period,when new models were being introduced, particularly forconsumer durables. Moulton and Moses (1997), usingUnited States CPI data for 1995 in their study of selectedproducts, found the average pure price change to be only0.12 per cent for identical items being repriced (on amonthly or bimonthly basis), compared to an average2.51 per cent for comparable substitutes – items judgedequivalent to the items they replaced. The correspondingaverage price change for directly substituted quality-adjusted price changes was 2.66 per cent. Thus, the pricemovement of continuing items appears to be a flawedproxy for the pure price component of the differencebetween old and replacement items.

7.68 The class mean method was adopted in theUnited States CPI for cars in 1989 and was phased in formost other non-food commodities, beginning in 1992. Itdiffered from the overall mean imputation method onlyin the source for the imputed rate of price change for theold item in period t+1. Rather than using the categoryindex change, obtained using all the non-missing items inthe category, the imputed rate of price change was basedon constant quality replacement items – those items thatwere judged comparable or that were quality-adjusteddirectly. The classmeanapproachwas seen as an improve-ment on the overall mean imputation approach becausethe imputed price changes were based on items that hadnot just been replaced, but whose replacement price hadbenefited from a quality adjustment or the new replace-ment item had been judged to be directly comparable. Itmay be the case, however, that sufficiently large samplesof comparable substitutes or directly quality-adjusteditems are unavailable. Or it may be that the qualityadjustments and selection of comparable items are notdeemed sufficiently reliable. In that case, a targetedimputation might be considered. The targeted mean isless ambitious in that it seeks only to capture pricechanges of similar items, irrespective of their point in thelife cycle. Yet it is an improvement on the overall meanimputation, as long as sufficiently large sample sizes areused.

Comparable replacement7.69 The comparable replacement method requires

the respondent to make a judgement that the replace-ment is of a similar quality to the old item and any pricechanges are untainted by quality changes. For specia-lized chain stores in Table 7.1(b), item 3 might be judgedto be comparable to item 2 and its prices in subsequentmonths might be used to continue the series. The price ofitem 3 (6.5) inMarch would be used as the price inMarchof item 2, whose January to March price change wouldbe 6.5/6�100=1.0833 or 8.33 per cent. Lowe (1999)notes the common practice of manufacturers of televi-sion sets to change model numbers with a new produc-tion run, though nothing physically has changed, orwhen small changes take place in specifications, such asthe type of remote controls, or the number or placement

ADJUSTING FOR QUALITY CHANGE

111

of jacks. The method of comparable replacement relieson the efficacy of the price collectors and, in turn, on theadequacy of the specifications used as a description of theitems. Statistical agencies may rightly be wary of samplesizes being reduced by dropping items for which pricesneed to be imputed, and also wary of the intensive use ofresources to make explicit estimates as outlined below.The use of repriced items of a comparable specificationhas much to commend it. If the quality of items isimproving, however, the preceding item will be inferiorto the current one. Continually ignoring small changes inthe quality of replacements can lead to an upward bias inthe index. The extent of the problem will depend on theproportion of such occurrences, the extent to whichcomparable items are accepted as being so despite qualitydifferences, and the weight attached to those items.Proposals in Chapter 8 to monitor types of qualityadjustment methods by product area provide a basis fora strategy for applying explicit adjustments where theyare most needed.

Linked to show no price change7.70 Linking attributes any price change between the

replacement item in the current period and the old itemin the preceding period to the change in quality. Forexample, in Table 7.1(b), a replacement item 7 is selectedfrom a department store for the missing March item 6.Items 6 and 7 may be of different quality, the price dif-ference being quite large. The change in price is assumedto be attributable to a change in quality. An estimateis made for p72 by equating it to p73, to show no change,i.e., the assumed price of item 7 in February is 14 inTable 7.1(b). There is thus assumed to be no price changeover the period February to March for item 7. TheJanuary to March result for item 6 is (12=12)�(14=14)=1:00, indicating no change. For the periodMarch to April, however, the price of item 7 in Marchcan be compared with the imputed p72 for February andlinked to the preceding results. So the January to Aprilcomparison is composed of the January to Februarycomparison for item 6, linked to (multiplied by) theFebruary to April comparison for item 7. This linking isanalogous to the procedures used for the chained andshort-run framework discussed in paragraphs 7.153 to7.158 and 7.171 to 7.173 below. The method is born outof circumstances where comparable replacement itemsare not available and there are relatively large price dif-ferences between the old and replacement items, thesebeing from different price bases and of different qualities.It is not possible to separate out how much of this dif-ference is attributable to price changes and how much toquality changes, so the method attributes it all to qualityand holds price constant. The method introduces adegree of undue price stability into the index. It may wellbe the case that the period of replacement is when sub-stantial price changes are taking place and that these arewrongly attributed to quality changes by this method.Article 5 of the European Commission (EC) RegulationNo. 1749/96 requires Member States to avoid ‘‘auto-matic linking’’. Such linking is equivalent to theassumption that the difference in price between two

successive models is wholly attributable to a difference inquality (Eurostat, 2001a, p. 125).

Carry-forward7.71 With the carry-forward method, when an item

becomes unavailable, say in period t, the price changecalculation uses the old t�1 price, simply carried for-ward as if there were no change. Thus from Table 7.1(a)for specialized chain stores for the period January toMarch, the Jevons and Dutot indices (Chapter 20) are:

PJ( p1, p3)=[( p13=p11 � p22=p21)]1=2 and

PD( p1, p3)=[( p13+p22)=( p11+p21)] (7.18)

with p22 filling in for the missing p23. This introducesundue stability into the index, which is aggravated if theold price, p22, continues to be used to fill in the unob-served prices in subsequent periods. It induces an inap-propriate amount of stability into the index and may givea misleading impression of the active sample size. Thepractice of the carry-forward method is banned underArticle (6) of the EC Regulation No. 1749/96 for Har-monized Indices of Consumer Prices (Eurostat, 2001a,p. 126). To use this method an assumption is made thatthe price from this outlet would not change. This methodshould only be used if it is fairly certain that there wouldbe no price change.

Explicit methods ofquality adjustment

7.72 The aforementioned methods do not rely onexplicit information on the value of the change in quality,A(z). This section discusses the following methods thatrely on obtaining an explicit valuation of the qualitydifference: expert judgement; quantity adjustment; dif-ferences in production or option costs; and the hedonicapproach.

Expert judgement7.73 Hoven (1999) describes comparable replace-

ment as a special case of subjective quality adjustment,because the determination of product equivalence isbased on the judgement of the commodity specialist. Oneobjection to subjective methods is the inability to provideresults that can be independently replicated. Yet in com-parable replacement, and for the selection of repre-sentative items, a subjective element is part of normalprocedure. This is not, of course, an argument for expand-ing the use of subjective methods.

7.74 Hoffman (1999) describes a possibly uniquealternative for quality adjustment of replacement itemsin the German CPI. When a new product is moreexpensive than the item it replaces, a flexible adjustmentfactor can be employed, attributing none, some, or all ofthe price difference to improved quality. In particular,when no precise information is available on which tomake a quality determination, it is permissible for anadjustment to be made of 50 per cent of the price dif-ference. The guidelines used in Germany since 1997

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

112

replaced flawed procedures in which the particularmethods chosen for individual quality adjustmentsdepended on the difference in price alone. As Hoffmannnotes, however, even in the current approach no qualityadjustment is made if the new item is less expensivethan the old. Consequently, problems could arise if anincrease in quality were accompanied by a decrease inprice (or vice versa). The methods used in the GermanCPI are needed because quality adjustments for mostgoods are made not in the central CPI office but by pricecollectors in the field. Wide use of the hedonic and pro-duction cost approaches is precluded under these condi-tions. Thus, the organizational structure of the statisticalagency, as well as its funding level, will necessarily influ-ence its choice of quality adjustment methods.7.75 Reports by consumer associations and product

evaluations in consumer magazines are not advised byTurvey (1998), who cites a study which correlated qual-ity ratings and prices for 135 products categories usingConsumer Reports. The average correlation was 0.26,with over half having a positive association, just over athird no association and the rest a negative one. He alsoargues against ‘‘best buy’’ estimates, which are expertviews as to what a sensible consumer should pay, asopposed to what the market price will be (see alsoCombris, Lecocqs and Visser, 1997).7.76 The use of expert views as to consumer calcu-

lations may be appropriate for highly complex itemswhere alternative methods are not feasible. Expertsshould be guided with regard to the nature of the estimaterequired. More than one expert should be chosen and,where possible, the experts should be from differentbackgrounds. It is also advisable to give the experts someindication of the interval in which their estimate shouldlie. The well-known Delphi method (for example, seeCzinkota and Ronkainen, 1997) may be applicable. Inthis approach a panel of experts never meet, to avoid any‘‘bandwagon’’ effect regarding their estimates. They areasked to provide an estimate of the average response andthe range of likely responses. The median is taken of theseestimates and any estimate that is considered extreme issent back to the expert concerned, who is asked toaccount for possible reasons behind the difference. It maybe that the particular expert has a useful perspective onthe problem, which the other experts had not considered.If the expert argues the case convincingly, the response isfed back to the panel who are asked if they wish to changetheir views. A new median is taken, and further iterationsare possible. The Delphi method is time-consuming andexpensive, but it reflects the care needed in such matters.If an adjustment is required for a product area with alarge weighting in the CPI, and no other techniques areavailable, it is a possible alternative.

Quantity adjustment7.77 Quantity adjustment is one of the most straight-

forward explicit adjustments to undertake. It is appli-cable when the site of the replacement item differs fromthat of the available item. In some situations there is areadily available quantity metric that can be used tocompare the items. Examples are the number of units in a

package (e.g., paper plates or vitamin pills), the size orweight of a container (e.g., kilogram of flour, litre ofcooking oil), or the size of sheets or towels. Qualityadjustment to prices can be accomplished by scalingthe price of the old or new item by the ratio of quanti-ties. The index production system may do this scalingadjustment automatically, by converting all prices in thecategory to a price per unit of size, weight or number.Scaling is important. For example, if cooking oil is nowsold in 5 litre containers instead of 2.5 litre ones, it shouldnot be the case that prices have doubled.

7.78 There is, however, a second aspect. In the phar-maceutical context, for example, prices of bottles of pillsof different sizes differ. A bottle of 100 pills, each having50 milligrams of a drug, is not the same as a bottle of 50pills of 100 milligrams, even though both bottles contain5,000 milligrams of the same drug. If there is a change,say, to a larger size container, and a unit price decrease of2 per cent accompanies this change, then it should not beregarded as a price fall of 2 per cent if consumers gain lessutility from the larger and more inconvenient containers.In practice it will be difficult to determine what propor-tion of the price fall is attributable to quality and whatproportion to price. A general policy is not to auto-matically interpret unit price changes arising frompackaging size changes as pure price changes, if contraryinformation is available.

7.79 Consider a further example: a branded bag offlour previously available in a 0.5 kilogram bag priced at1.5 is replaced by a 0.75 kilogram bag priced at 2.25. Themain concern here is with rescaling the quantities. Themethod would use the relative quantities of flour in eachbag for the adjustment. The prices may have increasedby [(2:25=1:5)� 100=150] 50 per cent but the quality-adjusted prices (i.e. prices adjusted by size) have rem-ained constant [(2:25=1:5)� (0:5=0:75)� 100=100]: Theapproach can be outlined in a more elaborate manner byrecourse to Figure 7.1. The concern here is with the partof the unbroken line between the (price, quantity) coor-dinates (1.5, 0.5) and (2.25, 0.75), both of which haveunit prices of 3 (price=1.5/0.5 and 2.25/0.75). Thereshould be no change in quality-adjusted prices. Thesymbol D denotes a change. The slope of the line is b

0

0.5

1

1.5

2

2.5

0 0.25 0.5 0.75

Size in kilograms

Pric

e

5

3

3

∆price

∆size

β = ∆price/∆size

Figure 7.1 Quality adjustment for different-sized items

ADJUSTING FOR QUALITY CHANGE

113

which is Dprice=Dsize=(2:25�1:5)=(0:75� 0:50)=3,i.e., the change in price arising from a unit (kilogram)change in size. The quality- (size-) adjusted price inperiod t�1 of the old m bag is:

ppt�1m =pt�1m +bDsize=1:5+3(0:75� 0:5)=2:25 (7.19)

The quality-adjusted price change shows no change, asbefore:

ptn=ppt�1m =2:25=2:25=1:00

The approach is outlined in this form so that it can beseen as a special case of the hedonic approach (discussedbelow), where price is related to a number of qualitycharacteristics of which size may be only one.

7.80 The method can be seen to be successful onintuitive grounds as long as the unit price of different-sized bags remains constant. If the switch was from thereplacement of the 0.5 kilogram bag to a 0.25 kilogramone priced at 0.75, as shown by the continuation tocoordinate (0.75, 0.25) of the unbroken line in Figure 7.1,the quality-adjusted prices would again not change,assuming, however, that the unit (kilogram) priceswere 5, 3 and 3 for the 0.25, 0.5 and 0.75 kilogram bags,respectively, as shown in Table 7.3 and in Figure 7.1(including the broken line). Then the measure of quality-adjusted price change would depend on whether the 0.5kilogram bag was replaced by the 0.25 kilogram one (a67 per cent increase) or the 0.75 kilogram one (nochange). This is not satisfactory because the choice ofreplacement size is arbitrary. The rationale behind thequality adjustment process is to ask: does the differencein unit price in each case reflect different levels of utility?If so, adjustments should be made to the unit prices tobring them into line. If not, adjustments should bemade to the unit prices for that proportion attributableto differences in utility gained from, say, more con-venient packaging or the availability of smaller lots. Itmay be obvious from the nature of the product that anitem packaged in a very small size with a dis-proportionately high unit price carries an unusually highprofit margin, and that an appropriate replacement for alarge-sized item would not be this very small one.

Differences in production oroption costs

7.81 A natural approach to quality adjustment is toadjust the price of an old item by an amount equal to theresource costs of the additional features of the new item;i.e., to compare relative prices using:

ptn=ppt�1m where ppt�1m =pt�1m +x (7.20)

and x is the value of the additional features in period t�1prices. This value should be a consumer’s valuation,reflecting the additional flow of services or utility. Onesource of data is the manufacturers. They would beasked to provide data on production costs, to whichretail mark-ups and associated indirect taxes would beadded. This approach is most practicable in marketswhere there is a relatively small number of manu-facturers, and where updates of models are infrequent

and predictable. It only works if there is good commu-nication between manufacturers and the statisticalagency staff. It is particularly suitable when the qualityadjustments are also being undertaken to calculate theproducer price index (PPI) or other price programmes.Greenlees (2000) provides an example for new trucks andmotor vehicles in the United States in 1999. Just prior tothe introduction of the annual models, BLS staff visitselected manufacturers to collect cost information. Thedata are used in the PPI and International ComparisonProgrammes as well as in the CPI, and the information-gathering activity is a joint operation of the three pro-grammes. Allowable product changes for the purpose ofquality adjustments include occupant safety enhance-ments, mechanical and electrical improvements to over-all vehicle operation or efficiency, changes that affectlength of service or need for repair, and changes affectingcomfort or convenience.

7.82 Bearing in mind the caveat in paragraph 7.30,the producer orientation of the PPI implies that resourcecost is the appropriate criterion for quality adjustmentto prices (Triplett, 1983). One distinction, then, betweenthe use of producer cost estimates in the CPI and PPI isthat only the former programme will add retail mark-ups and indirect taxes. Another important differencemay occur in situations where product improvementsare mandated by government. Some of these mandatedimprovements provide no direct benefit to the pur-chaser. In these cases it is appropriate to make a qualityadjustment to prices for the associated resource cost inthe PPI, but not in the CPI, where the appropriate cri-terion is user value. If only production cost data areavailable, then estimates of the retail mark-up must takeinto account the (average) age of the models underconsideration. Mark-ups will decrease as models cometo the end of their life cycles. Therefore, mark-ups basedon models at the end of their life cycle should not beapplied to the production costs of models at the start oftheir life cycle.

7.83 Because of these difficulties in using the pro-duction cost approach, the option cost method is gen-erally preferred. Often it is the retail price of an optionthat is available and this of course includes the mark-upfor profit. Consider an example of the price of an optionbeing used to adjust for quality. Let the prices for an itemin periods t�1 and t be 10,000 and 10,500, respectively,but assume the price in period t is for the item with a newfeature or ‘‘option’’, and let the price of the additionalfeature in period t be known to be 300. Then the pricechange would be 10,200/10,000=1.02 or 2.0 per cent.The adjustment may take a multiplicative form (seeparagraphs 7.39–7.40 above): the additional option isworth 300/10,500=0.028571 of the period t price. The

Table 7.3 Example of size, price and unit price of bags offlour

Size(kilograms)

First price First unitprice

Secondprice

Second unitprice

0.25 0.75 3 1.25 50.5 1.50 3 1.50 30.75 2.25 3 2.25 3

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

114

adjusted price in period t�1 is therefore 10,000�1:028571=10; 285:71 and the price change 10,500/10,285.71=1.020833 or about 2.08 per cent. If in sub-sequent periods either of these elements changes, then sotoo must ppn;t�1 for those comparisons. The option costmethod is thus a method for use in stable markets withstable technologies. Alternatively, it may be preferable toestimate a one-off adjustment to the preceding baseperiod price and then compare all subsequent prices withthe new option to this estimate; i.e. 10,500/10,300=1.019417 or approximately 2 per cent.7.84 Option costs are thus useful in situations in

which the old and new items differ by quantifiablecharacteristics that can be valued in monetary terms byreference to market prices. For example, nuts may beavailable roasted or unroasted, and food items maybe available cooked or uncooked. Consider the additionof a feature to a car model. The feature may havebeen available as an option either in the prior period orcurrently for other models, providing an absolute orproportional consumer valuation. Armknecht andMaitland-Smith (1999) note that when radial tyresbecame a standard feature on new cars, the price ofadding optional radial tyres was used to determine thequality adjustments in the United States CPI. Thevaluation of a quantifiable product feature may bereadily available from the comparison of different pro-duct prices. Turvey et al. (1989) give the example ofwhiskies of different proofs (percentage alcohol content).The quality adjustment for a change in the alcoholcontent of one product may be inferred from the marketrelationship between proof and price.7.85 Consider the addition of a feature to a product

– say an installed automatic ice-maker in a refrigerator(Shepler, 2000). Refrigerators can be sold as standard orwith an installed automatic ice-maker. The price col-lector may always have collected prices on the standardmodel, but this may no longer be in production, beingreplaced by a model with an installed automatic ice-maker. The cost of the option is thus known from beforeand a continuing series can be developed by usingequation (7.20) and simply adjusting the old price in thebase period for the option cost. Even this process mayhave its problems. First, the cost of producing some-thing as standard may be lower than when it was anoption, all new refrigerators now having the installedautomatic ice-maker. This saving may be passed on, atleast in part, to the consumer. The option cost methodwould thus understate a price increase. Triplett (2002)cites a study by Levy et al. (1999) in which a car theftsystem was installed as standard but disabled when theoption was not required. It was seemingly cheaper toproduce this way. Second, by including something asstandard the consumer’s valuation of the option mayfall since buyers cannot refuse it. Some consumers mayattribute little value to the option. The overall effectwould be that the estimate of the option cost, priced forthose who choose it, is likely to be higher than theimplicit average price consumers would pay for it asstandard. Estimates of the effect on price of this dis-crepancy should in principle be made, though in practiceare quite difficult.

7.86 Option cost adjustments can be seen to besimilar to quantity adjustments, except that instead ofsize being the additional quality feature of the replace-ment, the added quality can be any other individualfeature. The comparison is: ptn=pp

t�1m where ppt�1m =pt�1m +

bDz for an individual z characteristic where Dz=(ztn�zt�1m ). The characteristics may be the size of the randomaccess memory (RAM) of a personal computer (PC)when a specific model of PC is replaced by a model that isidentical except for the amount of RAM it possesses. Ifthe relationship between price and RAM is linear, theabove formulation is appropriate. Many web pages givethe price of additional RAM as being independent ofother features of PCs, and a linear adjustment is appro-priate. Bear in mind that a linear formulation values theworth of a fixed additional amount of RAM to be thesame, irrespective of the amount of RAM the machinepossesses.

7.87 The relationship may, of course, be non-linear.Say, for example, for every additional 1 per cent of x, yincreases by 1.5 per cent (b=1:015). In this case,

ppt�1m =pt�1m bz (7.21)

for ptn=ppt�1m as a measure of quality-adjusted price

changes. Again the z change may reflect the service flow,but the non-linearity in the price–z relationship mayreflect the increasing or decreasing utility to the scale ofthe provision. Possession of the characteristic in up-market models of the item may be priced at a higher ratethan in a lower-priced one, i.e. b 1 in equation (7.21).

7.88 Consider Figure 7.1 with the z characteristicbeing the option on the horizontal axis. The similaritybetween the quantity adjustment and the option costapproaches is apparent since both relate price to somedimension of quality: the size or the option. The optioncost approach can be extended to more than one qualitydimension. Both approaches rely on the acquisition ofestimates of the change in price resulting from a changein the option or size: the b slope estimates. In the case ofthe quantity adjustment, this was taken from an itemidentical to the one being replaced, aside from the factthat it was of a different size. The b slope estimate in thiscase was perfectly identified from the two pieces ofinformation. It is as if the nature of the experimentcontrolled for changes in the other quality factors bycomparing prices of what is essentially the same thingexcept for the quantity (size) change.

7.89 The same reasoning applies to option costs.There may be, for example, two items, identical but forthe possession of a feature. This allows the value of thefeature to be determined. Yet sometimes the value of afeature or option has to be extracted from a much largerdata set. This may be because the quality dimensiontakes a relatively large range of possible numerical valueswithout an immediately obvious consistent valuation.Consider the simple example of only one feature varyingfor a product, the speed of processing of a PC. It is nota straightforward matter to determine the value of anadditional unit of speed. To complicate matters, theremay be several quality dimensions to the items and notall combinations of these may exist as items in the market

ADJUSTING FOR QUALITY CHANGE

115

in any one period. Furthermore, the combinationsexisting in the second period being compared may bequite different to those in the first. Considering theseaspects leads to a more general framework, known as thehedonic approach.

Hedonic approach7.90 The hedonic approach is an extension of the two

preceding approaches in that, first, the change in pricearising from a unit change in quality – the slope of theline in Figure 7.1 – is now estimated from a data setcomprising prices and quality characteristic values of alarger number of varieties. Second, the quality charac-teristic set is extended to cover, in principle, all majorcharacteristics that might influence price, rather than justthe quantity or option adjustment. The theoretical basisfor hedonic regressions will be covered in Chapter 21 andis briefly reviewed below, following an example based onpersonal computers.

7.91 It should be noted that the method requires anextension of the data set to include values of the price-determining quality characteristics for each item. Underthe matched models method each price collector neededonly to collect sufficient data on each model to allow themodel to be identified for subsequent repricing. Theextension required in the hedonic approach is that allprice-determining characteristics should be collected foreach model. Checklists for the characteristics of a pro-duct have been found by Merkel (2000) to improve thequality of data collected, as well as serving the needs ofhedonic adjustments (see also Chapter 6 on price col-lection and Liegey, 1994). If an item goes missing, anydifference in the characteristics of its replacement can beidentified and, as will be shown, a valuation can beascribed to such differences using the hedonic approach.

7.92 Appendix 7.1 to this chapter provides datataken from the United Kingdom Compaq and Dell websites in July 2000 on the prices and characteristics of 64desktop personal computers (PCs). Figure 7.2 is a scatterdiagram constructed from this information, relating theprice (£ sterling) to the processing speed (MHz). It isapparent that PCs with higher speeds command higherprices – a positive relationship. Under the option costframework above, a switch from a 733MHz to a933MHz PC would involve a measure of the slope of the

line between two unique points. The approach requiresthat there are 733MHz and 933MHz PCs that areidentical except for their processing speed. From Figure7.2 and Appendix 7.1 it is apparent that there are severalPCs with the same speed but different prices, resultingfrom the fact that other things differ. To estimate thevalue given to additional units of speed, an estimate ofthe slope of the line that best fits the data is required. InFigure 7.1 the actual slope was used; for the data inFigure 7.2 an estimate of the slope needs to be derivedfrom an estimate of the equation of the line that best fitsthe data, using ordinary least squares regression. Facil-ities for regression are available on standard statisticaland econometric software, as well as spreadsheets. Theestimated (linear) equation in this instance is:

PPrice=� 658:436+3:261 Speed �RR2=0:820 (7.22)

The coefficient of speed is the estimated slope of theline: the change in price (£3,261) resulting from a 1MHzchange in speed. This can be used to estimate quality-adjusted price changes for PCs of different speeds.The value of �RR2 indicates that 82 per cent of price vari-ation is explained by variation in processing speed. At-statistic to test the null hypothesis of the coefficientbeing zero was found to be 18.83: recourse to standardtables on t-statistics found the null hypothesis wasrejected at a 1 per cent level. The fact that the estimatedcoefficient differs from zero cannot be attributed tosampling errors at this level of significance. There is aprobability of 1 per cent that the test has wronglyrejected the null hypothesis.

7.93 The range of prices for a given speed – forexample for 933MHz – can, however, be seen fromAppendix 7.1 to be substantial. There is a price range ofabout £1,000, which suggests that other quality char-acteristics may be involved. Table 7.4 provides the resultsof a regression equation that relates price to a number ofquality characteristics using the data in Appendix 7.1.Such estimates can be provided by standard statisticaland econometric software, as well as spreadsheets.

7.94 The second column provides the results from alinear regression model, the dependent variable beingprice. The first variable is (processor) speed, with acoefficient of 2.731 – a unit MHz increase in processingspeed leads to an estimated £2.731 increase (positivesign) in price. A change from 733MHz to 933MHzwould be valued at an estimated 200�2.731=£546.20.The coefficient is statistically significant – its differencefrom zero (no effect) not being attributable to samplingerrors at a 0.1 per cent level of significance. This esti-mated coefficient is based on a multivariate model: it isthe effect of a unit change in processing speed on price,having controlled for the effect of other variables in theequation. The preceding result of 3.261 in equation (7.22)was based on only one variable, and is different from thisimproved result.

7.95 The brand variables are dummy interceptstaking values of 1 for, say, a Dell computer, and zerootherwise. While brands are not in themselves qualitycharacteristics, they may be proxy variables for otherfactors such as reliability or after-sales service. The

0

500

1000

1500

2000

2500

3000

3500

0 200 400 600 800 1000 1200

Pric

e, £

ste

rling

Speed, MHz

Figure 7.2 Scatter diagram showing prices and processingspeeds of personal computers

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

116

inclusion of such brand dummies also goes some waytowards reflecting segmented markets as communitiesof buyers, as discussed in Chapter 21. Similar dummyvariables were used for the other makes or brands(Compaq Presario and Compaq Presignia), except forone brand (Compaq Deskpro) which, in this case, wastaken to form the benchmark against which other modelsare compared. The coefficient of the Dell brand is anestimate of the difference between a Dell brand’s worthand that of a Compaq Deskpro, other variables beingconstant, i.e. £1,330.78 cheaper. Similarly, an Intel Pen-tium III commands an estimated £282.78 premium on anAMD Athlon.7.96 The estimate for processor speed was based on

data for Dell and Compaq PCs. If the adjustment forquality is between two Dell PCs, it might be argued thatdata on Compaq PCs should be ignored. Separateregressions could be estimated for each make, but thiswould severely restrict the sample size. Alternatively, aninteraction term or slope dummy can be used for vari-ables which are believed to have a distinctive brand-interaction effect. An example of such a dummy wouldbe, say, Dell * Speed, which takes the value of ‘‘speed’’when the PC is a Dell and zero otherwise. The coefficientof this variable (see Table 7.4) is 1.714; it is an estimateof the additional (positive sign) price arising for a DellPC over and above that already arising from the stan-dard valuation of a 1MHz increase in speed. For DellPCs it is £2.731+£1.714=£4.445. Thus if the replace-ment Dell PC is 200MHz faster than the unavailablePC, the price adjustment to the unavailable PC is to add200� £4:465=£893. Interactive terms for other vari-ables can similarly be defined and used. The estimationof regression equations is easily undertaken usingeconometric or statistical software, or data analysisfacilities in spreadsheets. An explanation of the techni-ques is given in many texts, including Kennedy (1998)and Maddala (1988). In Chapter 21, econometric con-cerns particular to the estimation of hedonic regressionsare discussed.

7.97 The value �RR2 is the proportion of variation inprice explained by the estimated equation. More for-mally, it is 1 minus the ratio of the variance ofthe residuals,

PNi=1( p

ti � ppti)

2=N, of the equation to the

variance of prices,PN

i=1( pti � �ppti)

2=N. The bar on theterm R2 denotes that an appropriate adjustment fordegrees of freedom is made to this expression, which isnecessary when comparing equations with differentnumbers of explanatory variables. At 0.934 (see Table7.4), the value �RR2 is very high. A high value of �RR2 can,however, be misleading for the purpose of qualityadjustment. First, such values indicate that the expla-natory variables account for much of the price variation.This may be over a relatively large number of varietiesof goods in the period concerned. This, of course, is notthe same as implying a high degree of prediction for anadjustment to a replacement item of a single brand in asubsequent time period. Predicted values depend fortheir accuracy not just on the fit of the equation, butalso on how far the characteristics of the item whoseprice is to be predicted are from the means of the sam-ple. The more unusual the item, the higher the predic-tion probability interval. Second, the value �RR2 indicatesthe proportion of variation in prices explained by theestimated equation. It may be that 0.90 is explainedwhile 0.10 is not explained. If the dispersion in prices isvery large, this still leaves a large absolute margin ofprices unexplained. Nonetheless, a high �RR2 is a necessarycondition for the use of hedonic adjustments.

7.98 Hedonic regressions should generally be con-ducted using a semi-logarithmic formulation (Chapter21). The dependent variable is the (natural) logarithmof the price, but the variables on the right-hand side ofthe equation are kept in their normal units, hence thesemi-logarithmic formulation. A double-logarithmic for-mulation would also take logarithms of the right-handside z variables. However, if any of these z variables aredummy variables which take the value of zero in someinstances, the double-logarithmic formulation would

Table 7.4 Hedonic regression results for Dell and Compaq personal computers

Dependent variable Price ln price

Constant �725.996 (2.71)** 6.213 (41.95)***Speed (processor, MHz) 2.731 (9.98)*** 0.001364 (9.02)***RAM (random access memory, MB) 1.213 (5.61) *** 0.000598 (5.00) ***HD (hard drive capacity, MB) 4.517 (1.96)* 0.003524 (2.76)**Brand (benchmark: Compaq Deskpro)Compaq Presario �199.506 (1.89)* � 0.152 (2.60)**Compaq Prosignia �180.512 (1.38)* � 0.167 (2.32)*Dell �1330.784 (3.74)*** � 0.691 (3.52)***Processor (benchmark: AMD Athlon)Intel Celeron 393.325 (4.38)*** 0.121 (2.43)**Intel Pentium III 282.783 (4.28)*** 0.134 (3.66)***

ROM-drive (benchmark: CD-ROM){

CD-RW (compact disk, re-writable) 122.478 (56.07)*** 0.08916 (2.88)**DVD (digital video drive) 85.539 (1.54) 0.06092 (1.99)*Dell * Speed (MHz) 1.714 (4.038)*** 0.000820 (3.49)***N 63 63�RR2 0.934 0.934{ Read only memory.Figures in parentheses are t-statistics testing a null hypothesis of the coefficient being zero.***,** and * denote statistically significant at 0.1 per cent, 1 per cent and 5 per cent levels, respectively, tests being one-tailed.

ADJUSTING FOR QUALITY CHANGE

117

break down because logarithms of zero cannot be taken.The focus is thus on the semi-logarithmic form. Thisconcern with linear and semi-logarithmic formulationsis equivalent to the consideration of additive and mul-tiplicative formulations discussed in paragraphs 7.39 to7.40 above. A linear model would, for example, ascribean extra £282.78 to a PC with an Intel Pentium III asopposed to an AMD Athlon, irrespective of the price ofthe PC. This is common in pricing strategies using theWorld Wide Web. More often than not, however, thesame options are valued at a higher price for up-marketgoods and services. In this case, equation (7.22) for amultivariate model becomes:

Price=b0bz11 b

z22 b

z33 . . . bznn e

or ln Price= ln b0+z1 ln b1+z2 ln b2+z3 ln b3+ . . . . . . zn ln bn+ln e: (7.23)

Note that this is a semi-logarithmic form; logarithms aretaken of only the left-hand-side variable, i.e., price. Eachof the z characteristics enters the regression withouthaving logarithms taken. This has the advantage ofallowing dummy variables for the possession or other-wise of a feature to be included on the right-hand side.Such dummy variables take the value of one if the itempossesses the feature and zero otherwise. Matters relat-ing to the choice of functional form are discussed in moredetail in Chapter 21.

7.99 The taking of logarithms of the first equation(7.23) allows it to be transformed in the second equationto a linear form. This allows the use of a conventionalordinary least squares (OLS) estimator to yield estimatesof the logarithms of the coefficients. These are given in thethird column of Table 7.4 and have a useful directinterpretation: if these coefficients are multiplied by 100,they are the percentage change in price arising from a1 unit change in the explanatory variable. For (processor)speed there is an estimated 0.1364 per cent change in pricefor each additional MHz the replacement item has overand above the unavailable item. When dummy variablesare used, the coefficients, when multiplied by 100, areestimates of the percentage change in price, given by(eb�1)100. For example, for a rewritable CD-RWcompared to a read-only CD-ROM the change in price is8.916 per cent. There is some bias in these coefficients;and in the (semi-) logarithmic equation, half the varianceof each coefficient should be added to the coefficientbefore using it (Teekens and Koerts, 1972). For a read-only CD-ROM, the t-statistic is 2.88; this is equal to thecoefficient divided by its standard error, the standarderror being 0.08916/2.88=0.03096 and the variance:0.030962=0.000958. The adjustment is to add 0.000958/2to 0.08916, giving 0.089639 or 8.9639 per cent.

7.100 The approach is particularly useful when themarket does not reveal the price of the quality char-acteristics required for the adjustment. Markets revealprices of items, not quality characteristics, so it is usefulto consider items as tied bundles of characteristics. A suf-ficiently large data set of items with their characteristicsand sufficient variability in the mix of characteristicsbetween the items allows the hedonic regression to pro-vide estimates of the implicit prices of the characteristics.

The theory behind such estimates is discussed in Chapter21. A number of ways of implementing the method areoutlined below.

7.101 Some mention should first be made of theinterpretation of the coefficients from hedonic regres-sions. The matter is discussed in detail in Chapter 21;only the conclusions are summarized here. There used tobe an erroneous perception that the coefficients fromhedonic methods represented estimates of user value asopposed to resource cost. The former is the relevantconcept in constructing a consumer price index, whilefor PPI construction it is the latter. Rosen (1974) foundthat hedonic coefficients may reflect both user valueand resource cost – both supply and demand influences.There is what is referred to in econometrics as an iden-tification problem; in other words, the observed data donot permit the estimation of the underlying demand andsupply parameters. Suppose that the production tech-nology of sellers is the same, but that buyers differ. Thenthe hedonic function describes the prices of character-istics that the firm will supply with the given rulingtechnology to the current mixture of tastes. There aredifferent tastes on the consumer side, so what appears inthe market is the result of firms trying to satisfy con-sumer preferences for a constant technology and profitlevel. The structure of supply is revealed by the hedonicprice function. Now suppose that sellers differ, but thatbuyers’ tastes are the same. Here the hedonic functionp(z) identifies the structure of demand. Of these twopossible assumptions, uniformity of tastes is unlikely,while uniformity of technologies is more likely, especiallywhen access to technology is unrestricted in the long run.Griliches (1988, p. 120) has argued in the context of aconsumer price index:

My own view is that what the hedonic approach triesto do is to estimate aspects of the budget constraint facingconsumers, allowing thereby the estimation of ‘‘missing’’prices when quality changes. It is not in the business ofestimating utility functions per se, though it can also beuseful for these purposes . . . what is being estimated isthe actual locus of intersection of the demand curves ofdifferent consumers with varying tastes and the supplycurves of different producers with possible varying tech-nologies of production. One is unlikely, therefore, to beable to recover the underlying utility and cost functionsfrom such data alone, except in very special circum-stances.

It is thus necessary to take a pragmatic stance. In manycases the implicit quality adjustment to prices outlined inparagraphs 7.44 to 7.71 may be inappropriate becausethe implicit assumptions are unlikely to be valid. In suchinstances, the practical needs of economic statisticsrequire explicit quality adjustments. Not to do anythingon the grounds that the measures are not conceptuallyappropriate would be to ignore quality change andprovide wrong results. Hedonic techniques provide animportant tool, making effective use of data on the price–quality relationship derived from other items in themarket to adjust for changes in one or more character-istics.

7.102 The proper use of hedonic regression requiresan examination of the coefficients of the estimated

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

118

equations to see if they make sense. It might be arguedthat the very multitude of distributions of tastes andtechnologies, along with the interplay of supply and de-mand, that determine the estimated coefficients (Chapter21) make it unlikely that ‘‘reasonable’’ estimates willarise from such regressions. A firm may, for example, cuta profit margin relating to a characteristic for reasonsrelated to long-run strategic plans; this may yield acoefficient on a desirable characteristic that may even benegative (Pakes, 2001). This does not negate the useful-ness of examining hedonic coefficients as part of a strat-egy for evaluating estimated hedonic equations. First,there has been extensive empirical work in this field andthe results for individual coefficients are, for the mostpart, quite reasonable. Even over time individual coeffi-cients can show quite sensible patterns of decline (vanMulligen, 2003). Unreasonable coefficients on estimatedequations are the exception and should be treated withsome caution. Second, one can have more faith in anestimated equation whose coefficients make sense andwhich predicts well, than one which may also predict wellbut whose coefficients do not make sense. Third, if acoefficient for a characteristic does not make sense, itmay be due to multicollinearity, a data problem, andshould be examined to see if this is the case (see Appendix21.1 to Chapter 21).7.103 The implementation of hedonic methods to

estimate quality adjustments for matched items whichare no longer available is considered below. Consideritems l, m and n where item l is available in periods t andt+2, the ‘‘old’’ item m is only available in period t andthe replacement item n only in period t+2. The items aredefined by their z quality characteristics, item m forexample being ztm and the price of itemm in period t is ptm,as depicted below. There is no problem with comparingthe prices ptl and pt+2

l of matched items with character-istics ztl with zt+2

l , for they have the same l qualitycharacteristics. But there is a problem with item m. Ahedonic imputation approach would predict the price ofitem m’s characteristics in period t+2 at the character-istic prices taken from a hedonic regression estimated inperiod t+2, i.e. ppt+2

m :

In this case, item m’s characteristics are held constant inthe comparison ppt+2

m =ptm. A similar exercise can be con-

ducted for the replacement item n using pt+2n =pptn. In this

comparison, item n’s characteristics are held constantand compared at period t+2 and period t prices. Theseimputation approaches are outlined below. Yet there is asecond approach, an adjustment one. Here the char-acteristics of the replacement item n are identified andcompared with those of the old item m, (zt+2

n � ztm), andestimated coefficients from hedonic equations used toestimate the value of the changes. These two approaches,hedonic imputations and hedonic adjustments, are con-

sidered below in further detail. This ‘‘patching’’ ofmissing prices is quite different from the use of hedonicprice indices discussed in paragraphs 7.132 to 7.149 andChapter 21. These use hedonic regressions to providehedonic price indices of overall quality-adjusted pricesusing a sample of all of the data in each period with nopatching. The ‘‘patching’’ of missing prices is a partialapplication of the hedonic approach, used in imputationsfor missing items or on non-comparable replacementsfor missing items when the matched models approach isbeing used and an item’s price is missing.

7.104 Hedonic imputation: Predicted vs. actual. Inthis approach a hedonic regression of the natural loga-rithm of the price of model i in period t on its char-acteristics set ztki is estimated for each month, using theequation:

ln pti=bt0+PKk=1

btk ztki+eti (7.24)

Say the price of an item m available in January (period t)is unavailable in March (period t+2). The price of itemm can be predicted for March by inserting the char-acteristics of the old unavailable item m into the esti-mated regression equation for March, and similarly forsuccessive months. The predicted price for this old itemin March and price comparison with January (period t)are given, respectively, by:

ppt+2m =exp [bt+2

0 +P

kbt+2k ztk,m] and ppt+2

m =ptm

(7.25a)

That is, the old model’s price is predicted for period t+2and patched in. In the example in Table 7.1(a), pp23, pp24,etc. and pp63, pp64, etc. would be estimated and comparedwith p21 and p61 respectively. The blanks for items 2 and6 in Table 7.1(a) would be effectively filled in by theestimated price from the regression equation.

7.105 An alternative procedure is to select for eachunavailable m item a replacement item n. In this case theprice of n in period t+2 is known, and a predicted pricefor n in period t is required. The predicted price for thenew item and the required price comparison are:

pptn=exp [bt0+P

kbtk z

t+2k,m ] and pt+2

n =pptn (7.25b)

That is, the new model’s price is adjusted. In this casethe characteristics of item n are inserted into the right-hand side of an estimated regression for period t. Theprice comparisons of equation (7.25a) may be weightedby wt

m, as would those of its replaced price comparisonin equation (7.25b).

7.106 Another option is to take the geometric meanof the formulations in equations (7.25a) and (7.25b) ongrounds analogous to those discussed in Chapter 15 andby Diewert (1997) with regard to similar index numbers.

7.107 Hedonic imputation: Predicted vs. predicted.This approach uses predicted values for, say, item n inboth periods, e.g., ppt+2

n =pptn. Consider a misspecificationproblem in the hedonic equation. For example, theremay be an interaction effect between a brand dummyand a characteristic – say for Dell and speed in theexample in Table 7.4. Possession of both characteristics

Item/period t t+2

l ptl pt+2

l

m ptm ppt+2

m

n pptn pt+2

n

ADJUSTING FOR QUALITY CHANGE

119

may be worth more in terms of price (in a semi-loga-rithmic form) than their separate individual components(for evidence of interactive effects see Curry et al., 2001).The use of pt+2

n =pptn would be misleading since the actualprice in the numerator would incorporate the 5 per centpremium, while the one predicted from a straight-forward semi-logarithmic form would not. It is stressedthat, in adopting this approach, a recorded actual priceis being replaced by an imputation. This is not desirable,but neither is the form of bias discussed above. Diewert(2002e) considers a similar problem and suggests anadjustment to bring the actual price back in line with thehedonic one. The comparisons using predicted values inboth periods are given as:

ppt+2n =pptn for the new item

ppt+2m =pptm for the disappearing or old item; or

[( ppt+2n =pptn )( pp

t+2m =pptm )]

1=2 (7.26)

as a geometric mean of the two.7.108 Hedonic adjustments. In this approach a

replacement item is used and any differences between thek characteristics of the replacement n in, for example,period t+2 and m in period t are ascertained. A pre-dicted price for m adjusted to be compatible with n isestimated for period t, i.e., ppt+2

m and is compared with theactual price, ptm, where

ppt+2m pt+2

n exp [�P

kbt+2k (zt+2

nk � ztmk)] (7.27a)

or alternatively, a predicted price for n adjusted to becompatible with m is estimated for period t, i.e. pptn iscompared with the actual price, pt+2

n , where

pptn ptm exp [P

kbtk(z

t+2nk � ztmk)] (7.27b)

The adjustments here are undertaken using predictedvalues. However, unlike the formulations in equation(7.27b), for example, pptn may be estimated by applyingthe subset of the k characteristics that distinguished mfrom n to their respective implicit prices in period testimated from the hedonic regression, and adjusting theprice of ptm. For example, if the nearest replacement foritem 2 is item 3, then the characteristics that differ-entiated item 3 from item 2 are identified and the price inthe base period, p31, is estimated by adjusting p21 usingthe appropriate coefficients from the hedonic regressionin that month. For example, for washing machines, ifitem 2 had a spin speed of 800 rpm and item 3 a spinspeed of 1,100 rpm, other things being equal, the shadowprice of the 300 rpm differential would be estimatedfrom the hedonic regression and p21 would be adjustedfor comparison with p33. Note that if the z variablesin the characteristic set are perfectly independent ofeach other, the results from this approach will be similarto those from equation (7.25). This is because inter-dependence between the variables on the right-hand sideof the hedonic equation – multicollinearity – leads toimprecise estimates of the coefficients (see Chapter 21).Hedonic imputations and adjustments of the form(7.25b) and (7.27b) have an advantage over their coun-terparts (7.25a) and (7.27a) since the regression equation

does not have to be updated in each period. However,(7.25b) and (7.27b) effectively compare a constant fixedbasket of current period characteristics while (7.25a) and(7.27a) compare a fixed basket of price reference periodcharacteristics. There is no reason to prefer one to theother and if the difference or spread between the twoindices is large, this is reason for caution over the use ofone against a geometric mean of the two. Regularupdating of hedonic regressions would be likely tominimize spread.

7.109 Hedonic: Indirect adjustment. An indirectadjustment may be made for the current period, whichonly requires the hedonic regression to be estimated inthe base period t, using:

pt+2n

ptm

�pptnpptm

(7.28)

The first term is the change in price between the old andreplacement items in periods t and t+2 respectively. Butthe quality of the item has also changed, so this pricechange needs to be divided by a measure of the change inquality. The second term uses the hedonic regression inperiod t in both the numerator and denominator. Thecoefficients – the shadow prices of each characteristic –are held constant. The predicted prices nevertheless differbecause different quantities of the characteristics arebeing inserted into the numerator and denominator: thecharacteristics of the replacement item n in the formerand the old item m in the latter. The measure is thechange in price after removing (by division) the change inquantity of characteristics for each item at a constantperiod t price. Of course, conceptually, the constantvaluation by a period t+2 regression would be equallyvalid and a geometric mean of the two ideal. However, ifhedonic regressions cannot be run in real time this is acompromise. As the spread between the current and baseperiod results increases, its validity decreases. As such,the regression estimates should be updated regularlyusing old and current period estimates and results com-pared retrospectively as a check on the validity of theresults.

Limitations of the hedonic approach7.110 The limitations of the hedonic approach

should be borne in mind. Some points are summarizedbelow (see also Chapter 21). First, the approach requiresstatistical expertise for the estimation of the equations.The availability of user-friendly software with regressionfacilities makes this less problematic. Statistical andeconometric software carries a range of diagnostic teststo help judge if the final formulation of the model issatisfactory. These include �RR2 as a measure of the overallexplanatory power of the equation, and F-test and t-teststatistics to enable tests to be conducted as to whether thedifferences between the coefficients of the explanatoryvariables are jointly and individually different from zeroat specified levels of statistical significance. Most of thesestatistics make use of the errors from the estimatedequation. The regression equation can be used to predictprices for each item by inserting the values of the

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

120

characteristics of the items into the explanatory vari-ables. The differences between the actual prices and thesepredicted results are the residual errors. Bias or impre-cise, and thus misleading, results may arise from a rangeof factors including heteroscedasticity (non-constantvariances in the residuals suggesting non-linearitiesor omission of relevant explanatory variables), a non-normal distribution for the errors, and multicollinearity,where two or more explanatory variables are related. Thelatter in particular has been described as the ‘‘bane ofhedonic regressions’’ (Triplett, 1990). Such econometricissues have been discussed in the context of hedonicregressions (Berndt, 1991; Berndt et al., 1995; Triplett,1990; Gordon, 1990; Silver, 1999; and in Chapter 21) andmore generally by Kennedy (1998) and Maddala (1988).For the reasons discussed above, when multicollinearityis suspected, the use of predicted values rather thanindividual coefficients is advised.7.111 Second, the estimated coefficients should be

updated regularly. If the adjustment is to the old model,then the price comparison is between the price of the oldmodel in some reference period adjusted for the qualitydifference between the old and new models, using coef-ficients from an estimated hedonic equation in the pricereference period as estimates of the value of such dif-ferences, as in (7.27b). There is, at first sight, no need toupdate the estimated coefficients each month. Yet thevaluation of a characteristic in the price reference periodmay be quite out of line with its valuation in the newperiod. For example, a feature may be worth an addi-tional 5 per cent in the reference period instead of 10 percent in the current period, because it might have beenintroduced at a discount at that point in its life cycle toencourage usage. Continuing to use the coefficients fromsome far-off period to make adjustments to prices in thecurrent period is akin to using out-of-date base periodweights. The comparison may be well defined, but havelittle meaning. If price adjustments for quality differ-ences are being made to the old item in the price refer-ence period using hedonic estimates from that period,then there is a need to update the estimates if they areconsidered to be out of date, say because of changingtastes or technology, and splice the new estimatedcomparisons onto the old. The regular updating ofhedonic estimates when using imputations or adjust-ments is thus recommended, especially when there isevidence of parameter instability over time. Ideally ageometric mean of either (7.25a) and (7.25b) or of(7.27a) and (7.27b) should be used, but this requires anupdating of hedonic regressions in real time.7.112 Third, the sample of prices and characteristics

used for the hedonic adjustments should be suitable forthe purpose. If they are taken from a particular outlet oroutlet type, trade source or web page and then used toadjust non-comparable prices for items sold in quitedifferent outlets, then there must at least be an intu-ition that the marginal utilities for characteristics aresimilar between the outlets. A similar principle appliesfor the brands of items used in the sample for thehedonic regression. It should be borne in mind that high�RR2 statistics do not alone ensure reliable results. Suchhigh values arise from regressions in periods prior to

their application and indicate the proportion of vari-ation in prices across many items and brands. They arenot in themselves a measure of the prediction error for aparticular item, sold in a specific outlet, of a given brandin a subsequent period, though they can be an importantconstituent of this.

7.113 Fourth, there is the issue of functional formand the choice of variables to include in the model.Simple functional forms generally work well. Theseinclude linear, semi-logarithmic (logarithm of the left-hand side) and double-logarithmic (logarithms of bothsides) forms. Such issues are discussed in Chapter 21. Thespecification of a model should include all price-deter-mining characteristics. Some authors advise quite simpleforms with only the minimum number of variables, aslong as the predictive capacity is high (Koskimaki andVartia, 2001). Shepler (2000) included 33 variables in herhedonic regressions of refrigerators – a fairly homo-geneous product. These included nine dummy variablesfor brand and four for colour, five types of outlets, threeregions as control variables, and 11 characteristics includ-ing capacity, types of ice-maker, energy-saving control,extra drawers, sound insulation, humidifier and fil-tration device. Typically, a study would start with a largenumber of explanatory variables and a general econo-metric model of the relationship, while the final modelwould be more specific, having dropped a number ofvariables. The dropping of variables would depend onthe result of experimenting with different formulations,and seeing their effects on diagnostic test statistics,including the overall fit of the model and the accordanceof signs and magnitudes of coefficients with priorexpectations. Reese (2000), for example, started with ahedonic regression for United States college textbookswhich included about 50 explanatory variables, subse-quently reduced to 14 such variables with little loss ofexplanatory power.

7.114 Finally, Bascher and Lacroix (1999) list sev-eral requirements for successful design and use ofhedonic quality adjustment in the consumer price index,noting that these require heavy investments over a longperiod involving:

� intellectual competencies and sufficient time todevelop and re-estimate the model, and to employ itwhen products are replaced;

� access to detailed, reliable information on productcharacteristics;

� a suitable organization of the infrastructure for col-lecting, checking and processing information.

7.115 Hedonic methods may also improve qualityadjustment in the consumer price index by indicatingwhich product attributes do not appear to have materialimpacts on price. That is, if a replacement item differsfrom the old item only in characteristics that have beenrejected as price-determining variables in a hedonic study,this would support a decision to treat the items as com-parable or equivalent to and include the entire price dif-ference, if any, as pure price change. Care has to beexercised in such analysis because a feature of multi-collinearity in regression estimates is that the imprecisionof the parameter estimates may give rise to statistical tests

ADJUSTING FOR QUALITY CHANGE

121

that do not reject null hypotheses that are false, i.e., theydo not find significant parameter estimates that are sig-nificant. The results from such regressions can none-theless provide valuable information on the extent towhich different characteristics influence price variation,and this in turn can help in the selection of replacementitems. Enhanced confidence in item substitution and thequality adjustment of prices that arises from using thehedonic approach, and the parallel reduction in relianceon ‘‘linking’’, has been cited as a significant benefit interms of the reliability of the measurement of pricechanges for apparel in the United States consumer priceindex (Reinsdorf, Liegey and Stewart, 1996). The resultsfrom hedonic regressions have a role to play in identifyingprice-determining characteristics and may be useful in thedesign of quality checklists in price collection (Chapter 6).

Choice between qualityadjustment methods

7.116 Choice of method for quality adjustments toprices is not straightforward. The analyst must considerthe technology and market for each commodity anddevise appropriate methods. This is not to say themethods selected for one product area will be indepen-dent of those selected for other areas. Expertise built upusing one method may encourage its use elsewhere, andintensive use of resources for one commodity may lead toless resource-intensive methods for others. The methodsadopted for individual product areas may vary betweencountries as access to data, relationships with the outletmanagers, resources, expertise and features of the pro-duction, and market for the product vary. Guidelines onchoice of method arise directly from the features of themethods outlined above. A good understanding of themethods, and their implicit and explicit assumptions, isessential to the choice of an appropriate method.

7.117 Figure 7.3 provides a guide to the decision-making process. Assume that the matched modelsmethod is being used. If the item is matched for re-pricing, there being no change in the specification, noquality adjustment is required. This is the simplest ofprocedures. However, a caveat applies. If the itembelongs to a high-technology industry where modelreplacement is rapid, the matched sample may becomeunrepresentative of the universe of transactions. Alter-natively, matching may be under a chained framework,where prices of items in a period are matched to those inthe preceding period to form a link. A series of successivelinks of matched comparisons combined by successivemultiplication makes up the chained matched index. Orhedonic indices may be used which require no matching.The use of such methods is discussed in paragraphs 7.132to 7.149. At the very least, attention should be directed tomore regular item re-sampling. Continued long-runmatching would deplete the sample and an alternativeframework to long-run matching would be required.

7.118 Consider a change in the quality of an item andassume that a replacement item is available. The selec-tion of a comparable item to the same specification andthe use of its price as a comparable replacement require

that none of the price difference is attributable to quality.It also requires confidence that all price-determiningfactors are included in the specification. The replacementitem should also be representative and account for areasonable proportion of sales. Caution is required whenreplacing near obsolete items with unusual pricing at theend of their life cycles with similar ones that account forrelatively low sales, or with ones that have quite sub-stantial sales but are at different points in their cycle.Strategies for ameliorating such effects are discussedbelow and in Chapter 8, including early substitutionsbefore pricing strategies become dissimilar.

7.119 Figure 7.3 illustrates the case where qualitydifferences can be quantified. Explicit estimates are gen-erally considered to be more reliable, although they arealso more resource intensive, at least initially. Once anappropriate methodology has been developed, they canoften be easily replicated. General guidelines are moredifficult here as the choice depends on the host of factorsdiscussed above, which are likely to make the estimatesmore reliable in each situation. Central to all of this is thequality of the data upon which the estimates are based. Ifreliable data are unavailable, subjective judgements maybe used. Product differences are often quite technical andvery difficult to specify and quantify. The reliability of themethod depends on the expertise of the experts and thevariance in opinions. Estimates based on objective dataare thus preferred. Good production cost estimates inindustries with stable technologies and identifiable con-stant retail mark-ups and where differences between theold and replacement items are well specified and exhaus-tive are, by definition, reliable. Estimates of the retailmark-up are, however, prone to error and the option costapproach is generally preferable. This requires that theold and new items differ by easily identifiable character-istics which are or have been separately priced as options.

7.120 The use of hedonic regressions for partialpatching is most appropriate where data on price andcharacteristics are available for a range of models andwhere the characteristics are found to predict and explainprice variability well in terms of a priori reasoning andeconometric terms. Their use is appropriate where thecost of an option or change in characteristics cannotbe separately identified and has to be gleaned from theprices of items sold with different specifications inthe market. The estimated regression coefficients are theestimate of the contribution to price of a unit change in acharacteristic, having controlled for the effects of vari-ations in the quantities of other characteristics. Theestimates are particularly suited to valuing changes in thequality of an item when only a given set of characteristicschanges and the valuation is required for changes in thesecharacteristics only. The results from hedonic regressionsmay be used to target the salient characteristics for itemselection. The synergy between the selection of pricesaccording to characteristics defined as price determin-ing by the hedonic regression, and their subsequent usefor quality adjustment, should reap rewards. Themethod should be applied where there are high ratios ofnon-comparable replacements and where the differencesbetween the old and new items can be well defined by alarge number of characteristics.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

122

7.121 If explicit estimates of quality are unavailable,and no replacement items are deemed appropriate, thenimputations may be used. The use of imputations hasmuch to commend it resource-wise. It is relatively easy toemploy – though some verification of the validity of theimplicit assumptions might be appropriate. It requires no

judgement (unless targeted) and is therefore objective.Targeted mean imputation is preferred to overall meanimputation as long as the sample size upon which thetarget is based is adequate. Class mean imputation ispreferred when models at the start of their life cycles arereplacing those around the end of their life cycle,

Has the specification changed?

Continue to use matched models

Has the quality changed?

Yes

Yes

Yes

Yes

Can the quality difference be explicitly quantified?

Expert judgement

Replacement available: assume none of price difference

No replacement Replacement

Is a replacement

Source: Chart developed from a version by Fenella Maitland-Smith and Rachel Bevan, OECD; see also a version in Triplett (2002).

No

No

No

No

Quantity adjustment

Production cost

Option costs

Hedonic techniques

Comparable replacement

Overall/targeted/class mean imputation

Carry-forward

Linked to show no change Overlap pricing

available?

Yes − use an explicit adjustment

available: assume all price difference is attributableto quality

is attributableto quality

Are the old and new varieties available simultaneously?

Figure 7.3 Flowchart for making decisions on quality change

ADJUSTING FOR QUALITY CHANGE

123

although the approach requires faith in the adequacy ofthe explicit and comparable replacements being made.

7.122 Bias from using imputation is directly relatedto the proportion of missing items and the differencebetween quality-adjusted prices of available matcheditems and the quality-adjusted prices of unavailable ones(see Table 7.2 on page 110). The nature and extent of thebias depends on whether short-run or long-run imputa-tions are being used (the former being preferred) and onmarket conditions (see paragraphs 7.159 to 7.173).Imputation, in practical terms, produces the same resultas deletion of the item. The inclusion of imputed pricesmay give the illusion of larger sample sizes. Imputationis less likely to introduce bias where the proportion ofmissing prices is low. Table 7.2 can be used to estimatelikely error margins arising from its use and a judgementcan be made as to whether they are acceptable. The useof imputation across many products need not necessarilycompound the errors since, as noted in the above dis-cussion of this method, the direction of bias need not besystematic. It is cost-effective for product areas with alarge number of missing items because of its ease of use.But the underlying assumptions required by imputationmust be very carefully considered if it is widely used.Imputation should by no means be the overall catch-all strategy, and statistical agencies are advised againstits use as a default device without due considerationof the nature of the markets, the possibility of target-ing the imputation and the viability of estimates fromthe sample sizes involved if such targeting is employed.

7.123 If the old and replacement items are availablesimultaneously, and if the quality difference cannot bequantified, an implicit approach can be used wherebythe price difference between the old and replacementitems in a period in which they both exist is assumed tobe attributable to quality. This overlap method, in repla-cing the old item by a new one, takes the ratio of prices ina period to be a measure of their quality difference. It isimplicitly used when new samples of items are taken. Theassumption of relative prices equating to quality differ-ences at the time of the splice is unlikely to hold if the oldand replacement items are at different stages in their lifecycles and different pricing strategies are used at thesestages. For example, there may be deep discounting of theold item to clear inventories, and price skimming ofmarket segments that will purchase new models at rela-tively high prices. As with comparable replacements, earlysubstitutions are advised so that the overlap is at a timewhen items are at similar stages in their life cycles.

7.124 For the reasons discussed, the use of the linkedto show no change method and the carry-forward methodis not generally advised for making quality adjustmentimputations, unless the implicit assumptions are deemedto be valid.

High-technology and other sectorswith a rapid turnover of models

7.125 The measurement of price changes of itemsunaffected by quality changes is primarily achieved bymatching models, the above techniques being applicable

when the matching breaks down. But what of industrieswhere the matching breaks down on a regular basisbecause of the high turnover in new models of differentqualities to the old ones? The matching of prices ofidentical models over time, by its nature, is likely to leadto a seriously depleted sample. There is both a dynamicuniverse of all items consumed and a static universe ofthe items selected for repricing (Dalen, 1998a). If, forexample, the sample is initiated in December, by thesubsequent May the static universe will be matchingprices of those items available in the static universe inboth December and May, but will omit the unmatchednew items introduced in January, February, March,April and May, and the unmatched old ones availablein December but unavailable in May. Two empiricalquestions show whether there will be any significantbias. First, is sample depletion substantial? Substantialdepletion of the sample is a necessary condition for suchbias. Second, are the unmatched new and unmatched olditems likely to have quality-adjusted prices that sub-stantially differ from those of the matched items in thecurrent and the base periods?

7.126 The matching of prices of identical modelsover time may lead to the monitoring of a sample ofmodels that is increasingly unrepresentative of thepopulation of transactions. Some of the old models thatexisted when the sample was drawn are not available inthe current period; and new models that enter the sampleare not available in the base period. It may be that themodels that are going out have relatively low prices,while the entrants have relatively high ones. By ignoringthese prices, a bias is being introduced. Using old low-priced items and ignoring new high-priced ones has theeffect of biasing the index downwards. In some indus-tries, the new item may be introduced at a relatively lowprice and the old one may become obsolete at a relativelyhigh price, serving a minority segment of the market(Berndt et al., 2003). In this case, the bias would take theopposite direction. The nature of the bias will depend onthe pricing strategies of firms for new and old items.

7.127 This sampling bias exists for most products.Our concern here, however, is with product marketswhere the statistical agencies are finding the frequency ofnew item introductions and old item obsolescence suffi-ciently high that they may have little confidence in theirresults. First, some examples of such product marketswill be given and then two procedures will be considered:the use of hedonic price indices (as opposed to the par-tial, hedonic patching discussed above) and chaining.

Some examples7.128 Koskimaki and Vartia (2001) attempted to

match prices of models of personal computers (PCs) overthree two-month periods (spring, summer and autumn)using a sample of prices collected as part of standardprice collection for the Finnish consumer price index. Ofthe 83 spring prices, only 55 matched comparisons couldbe made with the summer prices, and then only 16 con-tinued through to the autumn. The sample of matchedpairs became increasingly rapidly biased: of the 79models in the autumn, the 16 matched ones had a mean

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

124

processor speed of 518MHz compared with 628MHz forthe remaining 63 unmatched ones; the hard disk sizeswere, respectively, 10.2 and 15.0 Gigabytes, and thepercentages of high-end processors (Pentium III andAMD Atl.) were 25 per cent and 49.2 per cent, respec-tively. Hardly any change in matched prices was foundover this six-month period, while a hedonic regressionanalysis using all of the data found quality-adjusted pricefalls of around 10 per cent. Instructions to price collec-tors to hold onto models until forced replacements arerequired may thus lead to a sample that is increasinglyunrepresentative of the population and is biased towardstechnically inferior variants. In this instance, the hedonicprice changes fell faster since the newer models becamecheaper for the services supplied.7.129 Kokoski et al. (1999) used hedonic regressions

in an empirical study of inter-area price comparisons offood products across urban areas in the United Statesusing the United States consumer price index data. Theyfound a negative sign on the coefficients of dummyvariables for whether or not the sample items were fromnewly rotated samples (dummy variable=1) or samplesprior to rotation (dummy variable=0). This indicatedthat quality-adjusted prices were lower for the newlyincluded items compared with the quality-adjusted pri-ces of the old items.7.130 Silver and Heravi (2002) found evidence of

sample degradation when matching prices of UnitedKingdom washing machines over a year. By December,only 53 per cent of the January basket of model varietieswas used for the December/January index, although thisaccounted for 81.6 per cent of January expenditure.Models of washing machines with lower sales valuesdropped out faster. However, the remaining models inDecember accounted for only 48.2 per cent of the value oftransactions in December. The active sample relating tothe universe of transactions in December had substantiallydeteriorated. The prices of unmatched and matchedmodels were found to differ, as were their vintage andquality. Even when prices were adjusted for quality usinghedonic regressions, prices of unmatched old models werefound to be lower than matched ones, there also beingevidence of higher prices for unmatched new models.Quality-adjusted prices fell faster for the matched samplethan for the full sample: about 10 per cent compared withabout 7 per cent. Residuals from a common hedonic sur-face and their leverage were also examined. The residualsfrom unmatched new models were higher than matchedones, while residuals from unmatched old models weremuch lower. Unmatched observations had nearly twice the(unweighted) leverage as matched ones – their influence inthe estimation of the parameters of the regression equationwas much greater, and their exclusion more serious.7.131 The above studies demonstrate how serious

sample degradation can occur and how unmatchedexcluded items may be quite different from includedones. Two procedures for dealing with such situationswill be considered: the use of hedonic price indices (asopposed to the partial, hedonic patching discussedabove) and chaining. Both rely on a data set of a repre-sentative sample of items and their characteristics in eachperiod. Price collectors might use a checklist of char-

acteristics in gathering the data (Merkel, 2000). They willbe asked to collect prices and characteristics of morethan one item in each store, the items being the major ortypical ones sold. If a new item is introduced which hasor is likely to have substantial sales, then it is included asa replacement or even addition, and its characteristics aremarked off against a checklist of salient characteristics.The list will be developed at the time of initiating thesample, and updated as required. Alternatively, marketresearch agencies, web pages and trade associations mayalso be able to provide lists of models and their prices.Nevertheless, there is a need to collect transaction prices,as opposed to list prices.

Hedonic price indices7.132 It is important to distinguish between the use

of hedonic regressions to make adjustments for qualitydifferences when a non-comparable substitute is used, asin paragraphs 7.90 to 7.115, and their use in their ownright as hedonic price indices, which are measures ofquality-adjusted price changes. Hedonic price indices aresuitable when the pace and scale of replacements ofitems are substantial because, first, an extensive use ofquality adjustments may lead to errors and, second, thesampling will be from a matched/replacement universelikely to be biased. With new models being continuallyintroduced and old ones disappearing, the coverage of amatched sample may deteriorate and bias may beintroduced as the price changes of new/old models differfrom those of the matched ones. What is required is asample to be drawn in each month and price indicesconstructed; but instead of controlling for quality dif-ferences by matching, they will be controlled for, or‘‘partialled out’’, in the hedonic regression. Note that allthe indices described below use a fresh sample of thedata available in each period. If there is a new item in aperiod, it is included in the data set and its quality dif-ferences controlled for by the regression. Similarly, if olditems drop out, they are still included in the data for theindices in the periods in which they exist. Paragraphs7.110 to 7.115 stress the need for caution in the use ofhedonic regressions for quality adjustments; some of thetheoretical and econometric aspects are considered inChapter 21. This need for caution extends to the use ofthe results from hedonic indices, and the discussion isnot repeated here for the sake of brevity.

7.133 In Chapter 17, theoretical price indices aredefined and practical index number formulae are con-sidered as boundsor estimatesof these indices. Theoreticalindex numbers are also defined in Chapter 21 to includegoods made up of tied characteristics, so something canbe said about how such theoretical indices relate to dif-ferent forms of hedonic indices. A number of forms areconsidered in Chapter 21; they are summarized below.

7.134 Hedonic functions with dummy variables fortime. The sample covers the two time periods beingcompared, say t and t+2, and does not have to bematched. The hedonic formulation regresses the price ofitem i, pi, on the k=2; . . .K characteristics of the items zki.A single regression is estimated on the data in the two timeperiods compared, the equation also including a dummy

ADJUSTING FOR QUALITY CHANGE

125

variable Dt+2 being 1 in period t+2, zero otherwise:

ln pi=b0+b1Dt+2+

PKk=2

bkzki+ei (7.29)

The coefficient b1 is an estimate of the quality-adjustedprice change between period t and period t+2. It is anestimate of the change in the logarithm of price, havingcontrolled for the effects of variation in quality viaPK

k=2bkzki. Note that an adjustment is required for b1:the addition of ½ (standard error)2 of the estimate, asdiscussed in Goldberger (1968) and Teekens and Koerts(1972). Two variants of equation (7.28) are considered.The first is the direct fixed base version, that comparesperiod t with t+2 as outlined: January–February,January–March, etc. The second is a rolling chainedversion evaluated for period t with t+1; then again fort+1 with t+2, the links in the chain being combined bysuccessive multiplication. A January–March compar-ison, for example, would be the January–February indexmultiplied by the February–March one. There is, ofcourse, a fully constrained version: a single constrainedregression for, say, January to December with dummyvariables for each month, but this is impractical in realtime since it requires data on future observations.

7.135 The above approach uses the dummy vari-ables on time to compare prices in period 1 with pricesin each subsequent period. In doing so, the b parametersare constrained to be constant over the period beingcompared. A fixed base, bilateral comparison usingequation (7.29) makes use of the constrained parameterestimates over the two periods and, given an equalnumber of observations in each period, is a form of asymmetric average. A chained formulation would esti-mate I1,4, for example, as: I1,4=I1,2� I2,3� I3,4. In eachbinary comparison for matched data, equal weight isalso given to the data in each period.

7.136 There is no explicit weighting in these for-mulations and this is a serious disadvantage. In practice,‘‘cut-off’’ sampling might be employed to include only themost important items. If sales data are available, aweighted least squares (WLS) estimator should be used,as opposed to an ordinary least squares (OLS) estimator.It is axiomatic in normal index number construction thatthe same weight should not be given to each price com-parison, since some items may account for much largersales revenues than others. The same considerationapplies to these hedonic indices. Diewert (2002e) hasargued for a preference for sales value weights overquantity weights. Two items may have sales equal to thesame quantity, but if one is priced higher than another, itsprice changes should be accordingly weighted higher forthe result to be meaningful in an economic sense. Addi-tionally, Diewert (2002e) has shown that value sharesshould form the weights, since values will increase, in sayperiod t+2, with prices, the residuals and their variancethus being higher in period t+2 than in t. This hetero-scedasticity is an undesirable feature of a regressionmodel, resulting in increased standard errors. Silver(2002) has further shown that a WLS estimator does notpurely weight the observations by their designatedweights, the actual influence given being also the result of

a combination of the residuals and the leverage effect. Thelatter is higher as the characteristics of the observationsdiverge from the average characteristics of the data. Silversuggests that observations with relatively high leverageand low weights be deleted and the regression re-run.

7.137 Period-to-period hedonic indices. An alter-native approach for a comparison between periods t andt+2 is to estimate a hedonic regression for period t+2,and insert the values of the characteristics of each modelexisting in period t into the period t+2 regression topredict, for each item, its price. This would generatepredictions of the prices of items existing in period tbased on their zti characteristics, at period t+2 shadowprices, ppt+2

i (zti). These prices (or an average) can becompared with the actual prices (or the average of pri-ces) of models in period t, pti(z

ti) as, for example, a

Jevons hedonic base period index:

PJHB=

QNt

i=1

ppt+2i (zti)

� �1=Nt

QNt

i=1

pti(zti)

� �1=Nt =

QNt

i=1

ppt+2i (zti)

� �1=Nt

QNt

i=1

ppti

� �1=Nt

=

QNt

i=1

ppt+2i (zti)

� �1=Nt

QNt

i=1

pti

� �1=Nt (7.30a)

7.138 Alternatively, the characteristics of modelsexisting in period t+2 can be inserted into a regressionfor period t. Predicted prices of period t+2 items gen-erated at period t shadow prices, pti(z

t+2i ), are the prices

of items existing in period t+2 estimated at period tprices and these prices (or an average) can be comparedwith the actual prices (or the average of prices) in periodt+2, pt+2

i (zt+2i ); a Jevons hedonic current period index is:

PJHC=

QNt+2

i=1

pt+2i (zt+2

i )

" #1=Nt+2

QNt+2

i=1

pti(zt+2i )

� �1=Nt+2 =

QNt+2

i=1

ppt+2i

" #1=Nt+2

QNt+2

i=1

pti (zt+2i )

� �1=Nt+2

=

QN+2t

i=1

pt+2i

" #1=Nt+2

QNt+2

i=1

pti(zt+2i )

� �1=Nt+2 (7.30b)

7.139 For a fixed base bilateral comparison usingeither equation (7.30a) or equation (7.30b), the hedonicequation is only estimated for one period, the currentperiod t+2 in equation (7.30a) and the base period t inequation (7.30b). For reasons analogous to thoseexplained in Chapters 15, 16 and 17, a symmetric averageof these indices would have some theoretical support.

7.140 Note that a geometric mean of (7.30) uses allthe data available in each period, as does the hedonicindex using a time dummy variable in (7.29). If in (7.29)there is a new item in, say, period t+2, it is included in the

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

126

data set and its quality differences controlled for by theregression. Similarly, if old items drop out, they are stillincluded in the indices in the periods in which they exist.This is part of the natural estimation procedure, unlikeusing matched data and hedonic adjustments on non-comparable replacements when items are no longeravailable.7.141 With the dummy variable approach, there is

no explicit weighting in its formulation in (7.29), andthis is a serious disadvantage. In practice, cut-off sam-pling might be employed to include only the mostimportant items; or if expenditure data are available, aWLS as opposed to an OLS estimator might be used,with expenditure value shares as weights, as discussed inAppendix 21.1 to Chapter 21.7.142 Superlative and exact hedonic indices (SEHI).

In Chapter 17, Laspeyres and Paasche bounds are definedon a theoretical basis, as are superlative indices, whichtreat both periods’ data symmetrically. These superlativeformulae, in particular the Fisher index, are also seen inChapter 16 to have desirable axiomatic properties. Fur-thermore, the Fisher index is supported by economictheory as a symmetric average of the Laspeyres andPaasche bounds, being found to be the most suitable suchaverage on axiomatic grounds. The Tornqvist index isseen to be best from the stochastic viewpoint, and alsodoes not require strong assumptions for its derivationfrom the economic approach as a superlative index. TheLaspeyres and Paasche indices are found to correspondto (be exact for) underlying Leontief aggregator functionswith no substitution possibilities, while superlative indi-ces are exact for flexible functional forms, including thequadratic and translogarithmetic forms for the Fisherand Tornqvist indices, respectively. If data on prices,characteristics and quantities are available, analogousapproaches and findings arise for hedonic indices (Fixlerand Zieschang, 1992 and Feenstra, 1995). Exact theore-tical bounds on a hedonic index have been defined byFeenstra (1995). Consider the theoretical index inChapter 17, equation (17.3), but now defined only overitems in terms of their characteristics zi. The prices (andquantities) are still of items, but they are wholly definedthrough their characteristics pi (zi). An arithmetic aggre-gation for a linear hedonic equation finds a Laspeyresupper bound (as quantities demanded decrease withincreasing relative prices) given by:

PNi=1

qti ppt+2i

PNi=1

qtipti

=PNi=1

stippt+2i

pti

� � C(ut, p(z)t+2)

C(ut, p(z)t)(7.31a)

where the right-hand-side expression is the ratio of thecost of achieving a period t level of utility (ut), whereutility is a function of the vector of quantities, i.e.,ut=f (qt). The price comparison is evaluated at a fixedlevel of period t quantities, and sti are the shares in totalvalue of expenditure on product i in period t,sti=qtip

ti=PN

i=1qtip

ti and

ppt+2i pt+2

i �PNi=1

bt+2k (zt+2

ik � ztik) (7.31b)

are prices in period t+2 adjusted for the sum of thechanges in each quality characteristic weighted by theircoefficients derived from a linear hedonic regression.Note that the summation is over the same i in bothperiods, since replacements are included when an item ismissing and equation (7.31b) adjusts their prices forquality differences.

7.143 A Paasche lower bound is estimated as:

PNi=1

qt+2i pt+2

i

PNi=1

qt+2i ppti

=PNi=1

st+2i

pt+2i

ppti

� �� ��1� C(ut+2, p(z)t+2)

C(ut+2, p(z)t)

(7.32a)

where st+2i =qt+2

i pt+2i =

PNi=1

qt+2i pt+2

i and

ppti pti+PNi=1

btk(zt+2ik � ztik) (7.32b)

which are prices in periods t adjusted for the sum of thechanges in each quality characteristic weighted by itsrespective coefficients derived from a linear hedonicregression.

7.144 In Chapter 17 it is shown that Laspeyres PL

and Paasche PP price indices form bounds on theirrespective ‘‘true’’ economic theoretic indexes. Usingsimilar reasoning to that in Chapter 17 applied toequations (7.31a) and (7.32a), it can be shown that underhomothetic preferences these true economic indices col-lapse into a single theoretical index c( pt+2)/c( pt), and:

PL c( pt+2)=c( pt) PP (7.33)

7.145 The approach is akin to that used for adjust-ments to non-comparable replacement items in equa-tions (7.27a) and (7.27b), above. However, the SEHIapproach first uses all the data in each period, not justthe matched sample and selected replacements. Second,it uses the coefficients from hedonic regressions onchanges in the characteristics to adjust observed pricesfor quality changes. Third, it incorporates a weightingsystem using data on the expenditure shares of eachmodel and their characteristics, rather than treating eachmodel as equally important. Finally, it has a directcorrespondence to formulations defined from economictheory.

7.146 Semi-logarithmic hedonic regressions wouldsupply a set of b coefficients suitable for use with thesebase and current period geometric bounds:

QNi=1

pt+2i

ppti

� �s t+2i

� C(u; p(z)t+2)

C(u; p(z)t)�QNi=1

ppt+2i

pti

� �s ti(7.34a)

ppti pti expPNi=1

btk(zt+2ik � ztik)

� �

ppt+2i pt+2

i exp �PNi=1

bt+2k (zt+2

ik � ztik)

� �(7.34b)

7.147 In equation (7.34a) the two bounds on therespective theoretical indices have been shown to bebrought together under an assumption of homothetic

ADJUSTING FOR QUALITY CHANGE

127

preference (see Chapter 17). The calculation of suchindices is no small task. For examples of their applica-tion, see Silver and Heravi (2001a and 2003) for com-parisons over time and Kokoski et al. (1999) for pricecomparisons across areas of a country. Kokoski et al.(1999) used a sample from a replacement universe ofotherwise matched data from the United States Bureauof Labor Statistics consumer price index, though thesample benefited from rotation. Silver and Heravi (2001aand 2003) used scanner data for the universe of trans-actions via a two-stage procedure in which cells weredefined according to major price-determining featuressuch as all combinations of brand, outlet type and (fortelevision sets) screen size – much like strata. There maybe a gain in the efficiency of the final estimate since theadjustment is for within-strata variation, much in theway that stratified random sampling improves on simplerandom sampling. The average price in each matched cellcould then be used for the price comparisons usingequations (7.32a) and (7.34a), except that – to ensure thatthe quality differences in each cell from characteristicsother than these major ones did not influence the pricecomparison – adjustments were made for quality changesusing equations (7.32b) and (7.34b). This allowed allmatched, old unmatched and new unmatched data to beincluded since, if the average price in, say, a cell ofequation (7.32a) was increased because of the inclusionof a new improved item, equation (7.32b) would be usedto remove such improvements, on average. Consider, forexample, a brand X, 14-inch television set with stereosound sold to multiple outlets. There might be matchedcells for brand X television sets sold in multiples, but notmatched cells also including stereo. The new model mayhave to be grouped in a cell with the brand X, 14-inchtelevision sets sold in multiples, and the average price ofthe cells compared in equation (7.32a) or (7.34a), andmaking a quality adjustment for the stereo in the form ofequation (7.32b) or (7.34b). The estimated coefficient forstereo would be derived from a hedonic equation esti-mated from data of other television sets, some of whichpossess stereo.

7.148 The above description illustrates how weightedindex number formulae such as Laspeyres, Paasche,Fisher and Tornqvist might be constructed using data onprices, quantities and characteristics of an item. Silverand Heravi (2003) show that as the number of char-acteristics over which the summation takes place inequations (7.32a) and (7.34a) increases, the moreredundant the adjustment in equations (7.32b) and(7.34b) becomes, until, when all combinations of char-acteristics are used in equations (7.32a) and (7.34a) asstrata, the calculation extends to a matched modelsproblem in which each cell uniquely identifies an item.For matched data, equations (7.32b) and (7.34b) serveno purpose; the aggregation in equations (7.32a) and(7.34a) would be over all items, and would reduce to theusual index number problem. Diewert (2003a), com-menting on the method, explains why, when matching isrelatively large, the results given are similar to those fromsuperlative hedonic index numbers.

7.149 Weighted index number formulae might thus beconstructed using data on prices, quantities and char-

acteristics of an item when the data are not matched. Thisis because continuing withmatched datamay lead to errorsfrom two sources: multiple quality adjustments from itemsno longer available and their non-comparable replace-ments; and sample selectivity bias from sampling from areplacement universe as opposed to a double universe.

The difference between hedonicindices and matched indices

7.150 In previous sections, the advantages of hedo-nic indices over matched comparisons are referred toin terms of the inclusion by the former of unmatcheddata. This relationship is discussed more formally here.Triplett (2002) argued and Diewert (2002e) showed thatan unweighted geometric mean (Jevons) index formatched data gives the same result as a logarithmichedonic index run on the same data. Consider the mat-ched sample m and Zt+2 and Zt as overall qualityadjustments to the dummy variables for time in equation(7.29), that is,

PKk=2bkzki. The very first line in equation

(7.35) below is shown by Aizcorbe et al. (2001) to equalthe difference between two geometric means of quality-adjusted prices. The sample space m=Mt=Mt+2 is thesame model in each period. Consider the introduction ofa new model n introduced in period t+2 with no coun-terpart in t and the demise of an old model o so it has nocounterpart in t+2. SoMt+2 is composed ofm and n, andMt is composed of m and o, whileM consists only of thematched models m. Silver and Heravi (2002) have shownthe dummy variable hedonic comparison to now be:

ln pt+2=pt= m=(m+n)Pm

ln ( pt+2m � Zm)=m

�

+n=(m+n)Pn

ln ( pt+2n � Zn)=n

�

� m=(m+o)Pm

ln ( ptm � Zm)=m

�

+o=(m+o)Po

ln ( pto � Zo)=o

�

= m=(m+n)Pm

ln ( pt+2m � Zm)=m

�

�m=(m+o)Pm

ln ( ptm � Zm)=m

�

� n=(m+n)Pn

ln ( pt+2n � Zn)=n

�

�o=(m+o)Po

ln ( pto � Zo)=o

�(7.35)

7.151 Consider the second expression in equation(7.35). First, there is the change for the m matchedobservations. This is the change inmeanprices ofmatchedmodels m in period t+2 and t, adjusted for quality. Notethat the weight in period t+2 for this matched compo-nent is the proportion of matched to all observations inperiod t+2. Similarly, for period t, the matched weightdepends on how many unmatched old observations arein the sample. In the last line of equation (7.35), thechange is between the unmatched new and the un-matched old mean (quality-adjusted) prices in periods

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

128

t+2 and t. Thus matched methods can be seen to ignorethe last line in equation (7.35) and will thus differ fromthe hedonic dummy variable approach in at least thisrespect. The hedonic dummy variable approach, in itsinclusion of unmatched old and new observations, can beseen from equation (7.35) possibly to differ from a geo-metric mean of matched prices changes, the extent of anydifference depending, in this unweighted formulation, onthe proportions of old and new items leaving andentering the sample and on the price changes of old andnew items relative to those of matched ones. If themarket for products is one in which old quality-adjustedprices are unusually low while new quality-adjustedprices are unusually high, then the matched index willunderstate price changes (see Silver and Heravi, 2002 andBerndt et al., 2003 for examples). Different marketbehaviour will lead to different forms of bias.7.152 If sales weights replace the number of obser-

vations in equation (7.35), then different forms ofweighted hedonic indices can be derived, as explainedin Chapter 21. Silver (2002) has also shown that thehedonic approach will differ from a correspondingweighted or unweighted hedonic regression in respect ofthe leverage and influence that the hedonic regressiongives to observations.

Chaining7.153 An alternative approach to dealing with pro-

ducts with a high turnover of items is to use a chained,say monthly, index instead of the long-term fixed basecomparison. A chained index compares prices of items inperiod t with period t+1 (Indext,t+1) and then, as a newexercise, studies the universe of items in period t+1 andmatches them with items in period t+2. These links(Indext,t+1 and Indext,t+2) are combined by successivemultiplication, continuing to, say, Indext+5,t+6 to formIndext+1,t+6. Only items available in both period t andperiod t+6 would be used in a fixed base consumer priceindex. Consider the five items 1, 2, 5, 6 and 8 over thefour months January–April, as shown in Table 7.1. Theprice index for January compared with February (J:F)involves price comparisons for all five items. ForFebruary–March (F:M) it involves items 1, 4, 5 and 8and for March–April (M:A) six items: 1, 3, 4, 5, 7 and 8.The sample composition changes for each comparison asold items disappear and new items come in. Price indicescan be calculated for each of these successive pricecomparisons using any of the unweighted formulaedescribed in Chapter 21. The sample will grow in sizewhen new products appear and shrink when old productsdisappear, changing in composition through time(Turvey, 1999).7.154 Sample depletion may be reduced in long-run

comparisons by the judicious use of replacement items.As discussed in Chapter 8, however, the replacementsample would only include a new item as and when areplacement was needed, irrespective of the number ofnew items entering the market. Furthermore, the re-placement item is likely to be either of a similar quality,to facilitate quality adjustment, and thus have relativelylow sales, or of a different quality with relatively high

sales, but requiring an extensive quality adjustment. Ineither case this is unsatisfactory.

7.155 Chaining, unlike hedonic indices, does not useall the price information in the comparison for each link.Items 2 and 6, for example, may be missing in March.The index makes use of the price information on items 2and 6 when they exist, for the January–February com-parison, but does not allow their absence to disrupt theindex for the February–March comparison. It may bethat item 4 is a replacement for item 2. Note how easily itis included as soon as two price quotes become available.There is no need to wait for rebasing or sample rotation.It may be that item 7 is a replacement for item 6. Aquality adjustment to prices may be required for theFebruary–March comparison between items 6 and 7, butthis is a short-run one-off adjustment, the compilation ofthe index continuing in March–April using item 7instead of item 6. SNA 1993 (Chapter 16, para. 54) onprice and volume measurement picks up on the point:

In a time series context, the overlap between the pro-ducts available in the two periods is almost bound to begreatest for consecutive time periods (except for sub-annual data subject to seasonal fluctuations). The amountof price and quantity information that can be utilizeddirectly for the construction of the price or volume indicesis, therefore, likely to be maximized by compiling chainindices linking adjacent time periods. Conversely, thefurther apart the two time periods are, the smaller theoverlap between the ranges of products available in thetwo periods is likely to be, and the more necessary itbecomes to resort to implicit methods of price compar-isons based on assumptions. Thus, the difficulties createdby the large spread between the direct Laspeyres andPaasche indices for time periods that are far apart arecompounded by the practical difficulties created by thepoor overlap between the sets of products available in thetwo periods.

7.156 The chained approach has been justified as thenatural discrete approximation to a theoretical Divisiaindex (Forsyth and Fowler, 1981 and Chapter 16).Reinsdorf (1998) has formally determined the theoreticalunderpinnings of the index, concluding that in generalchained indices will be good approximations to the the-oretical ideal – though they are prone to bias when pricechanges ‘‘swerve and loop’’, as Szulc (1983) hasdemonstrated (see also Forsyth and Fowler, 1981 and deHaan and Opperdoes, 1997).

7.157 The dummy variable hedonic index uses all thedata in January and March for a price comparisonbetween the two months. Yet the chained index ignoresunmatched successive pairs, as outlined above; but this ispreferable to its fixed base equivalent. The hedonicapproach, in predicting from a regression equation,naturally has a confidence interval attached to suchpredictions. The width of the interval is dictated by the fitof the equation, the distance of the characteristics fromtheir mean and the number of observations. Matching,chained or otherwise, does not suffer from any predictionerror. Aizcorbe et al. (2001) undertook an extensive andmeticulous study of high-technology goods (personalcomputers and semiconductors) using quarterly data forthe period 1993 to 1999. The results from comparablehedonic and chained indices were remarkably similar

ADJUSTING FOR QUALITY CHANGE

129

over the seven years of the study. For, example, fordesktop central processing units (CPUs) the index fellbetween the seven years from 1993:Q1 to 1999:Q4 by60.0 per cent (dummy variable hedonic), 59.9 per cent(chained Fisher) and 57.8 per cent (chained geometricmean). The results differed only for quarters when therewas a high turnover of items, and in these cases suchdifferences could be substantial. For example, for desk-top CPUs in 1996:Q4 the 38.2 per cent annual fall mea-sured by the dummy variable hedonic method differedfrom the chained geometric mean index by 17 percentagepoints. Thus with little model turnover there is littlediscrepancy between hedonic and chained matchedmodels methods and, for that matter, fixed base matchedindices. It is only when binary comparisons or links havea high model turnover that differences arise (see alsoSilver and Heravi, 2001a and 2003).

7.158 Of course it is possible to make up for missingprices by using partial, patched hedonic estimates, asdiscussed above. Dulberger (1989) computed hedonicindices for computer processors and compared theresults to those from a matched models approach. Thehedonic dummy variable index fell by about 90 per centover the period 1972 to 1984, about the same as for thematched models approach where missing prices for newor discontinued items were derived from a hedonicregression. However, when using a chained matchedmodels approach with no estimates or imputations formissing prices, the index fell by 67 per cent. It is alsopossible to combine methods; de Haan (2003) usedmatched data when available and the time dummy onlyfor unmatched data – his double imputation method.

Long-run and short-runcomparisons

7.159 This section describes a useful formulation toaid quality adjustment. Its innovation arises from a

possible concern with the long-run nature of the quality-adjusted price comparisons being undertaken. In theexample in Table 7.1, prices in March were comparedwith those in January. Assumptions of similar pricechanges are required by the imputation method to holdover this period for long-run imputations – somethingthat gives rise to increasing concern when price com-parisons continue over longer periods, between Januaryand October, January and November, January andDecember, and even subsequently. To help alleviate suchconcerns, this section considers a short-run formulation,mentioned in paragraph 7.42. Consider Table 7.5, which,for simplicity, has a single item A that exists throughoutthe period, an item B which is permanently missing inApril, and a possible replacement item C in April.

Quality adjustment methodsin short-run comparisons

7.160 A comparable replacement C may be found. Inthe previous example the focus was on the use of theJevons index at the elementary level, since it is shown inChapter 20 that this has much to commend it. Theexample here uses the Dutot index, the ratio of arith-metic means. This is not to advocate it, but only toprovide an example using a different formulation. TheDutot index also has much to commend it on axiomaticgrounds, but fails the commensurability (units of mea-surement) test and should only be used for relativelyhomogeneous items. The long-run Dutot index for Aprilcompared with January is:

PD

PNi=1

pApri =N

PNi=1

pJani =N

which is 8/5=1.30, a 30 per cent increase.

Table 7.5 Example of long-run and short-run comparisons

Item January February March April May June

Comparable replacementA 2 2 2 2 2 2B 3 3 4C 6 7 8Total 5 5 6 8 9 10

Explicit adjustmentA 2 2 2 2 2 2B 3 3 4 (5/6)�6=5 (5/6)�7=5.8 (5/6)�8=6.67C (6/5)�3=3.60 6 7 8Total 5 5 6 8 9 10

OverlapA 2 2 2 2 2 2B 3 3 4 6�(4/5)=4.8C 5 6 7 8Total 5 5 6 6.8

ImputationA 2 2 2.5 3.5 4 5B 3 3 4 (3.5/2.5)�4=5.6 (4/3.5)�5.6=6.4 (5/4)�6.4=8Total 5 5 6.5 9.1 8.4 13

Figures in bold are estimated quality-adjusted prices described in the text.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

130

The short-run equivalent is the product of a long-runindex up to the immediately preceding period, and anindex for the preceding to the current period, i.e., forperiod t+4 compared with period t:

PD

PNi=1

pt+3i =N

PNi=1

pti=N

26664

37775�

PNi=1

pt+4i =N

PNi=1

pt+3i =N

26664

37775

or for January with April:

PD

PNi=1

pMari =N

PNi=1

p jani =N

26664

37775�

PNi=1

pApri =N

PNi=1

pMari =N

26664

37775 (7.36)

which is, of course,6

5� 8

6=1:30 as before.

7.161 Consider a non-comparable replacement withan explicit quality adjustment. Say, for example, that C’svalue of 6 in April is quality-adjusted to be consideredto be worth only 5 when compared to the quality of B.The quality adjustment to prices may have arisen froman option cost estimate, a quantity adjustment, a sub-jective estimate or a hedonic coefficient, as outlinedabove. Say that the long-run comparison uses anadjusted January price for C, which is B’s price of 3multiplied by 6/5 to upgrade it to the quality of C, i.e.,(6/5)� 3=3.6. From April onwards, the prices of thereplacement item C can be readily compared to itsJanuary reference period price. Alternatively, the pricesof C in April onwards might have been adjusted bymultiplying them by 5/6 to downgrade them to thequality of B and enable comparisons to take place withitem B’s price in January: for April the adjusted price is(5/6)� 6=5; for May the adjusted price is 5.8 and forJune it is 6.67 (see Table 7.5). Both procedures yield thesame results for long-run price comparisons. The resultsfrom both methods (rounding errors aside) are the samefor item B.7.162 For the overall Dutot index, however, the

results will differ, since the Dutot index weights pricechanges by their price in the initial period as a proportionof total price (see Chapter 20, footnote 27). The twoquality adjustment methods will have the same pricechanges, but different implicit weights. The Dutot indexin May is 9/5.6=1.607 if an adjustment is made to theinitial (January) price or 7.8/5=1.56 if an adjustment ismade to the current period (May) price. The short-runindices give the same results for each adjustment:

8

5:6� 9

8=1:607 using an adjustment to the initial

(January) price, and

7

5� 7:8

7=1:56 using an adjustment to the

current period (May) price.

7.163 The overlap method may also take the short-run form. In Table 7.5 there is a price for C in March of5 that overlaps with B in March. The ratio of theseprices is an estimate of their quality difference. A long-run comparison between January and April would be

6� 45+2

� �=5=1:36. The short-run comparison would

be based on the product of the January to March and

March to April link:6:8

6� 6

5=1:36.

7.164 At this unweighted level of aggregation it canbe seen that there is no difference between the long-runand short-run results when items do not go missing,when comparable replacements are available, whenexplicit adjustments are made for quality or when theoverlap method is used. The separation of short-run(most recent month-on-month) and long-run changesmay have advantages for quality assurance to help spotunusual short-run price changes. But this is not theconcern of this chapter. The short-run approach does,however, have advantages when imputations are made.

Implicit short-run comparisonsusing imputations

7.165 The use of the short-run framework has beenmainly considered for temporarily missing values, asoutlined by Armknecht and Maitland-Smith (1999) andFeenstra and Diewert (2001). Similar issues neverthelessarise in the context of quality adjustment. Consider againTable 7.5, but this time there is no replacement item Cand item A’s prices have been changed to show anupward trend. Item B is again missing in April. A long-run imputation for item B in April is given by (3.5/2)�3=5.25. The price change is thus (5.25+3.5)/5=1.75 or75 per cent. This is, of course, the same result as thatobtained by simply using item A (3.5/2=1.75), since theimplicit assumption is that price movements of item B,had it continued to exist, would have followed those ofA. The assumption of similar long-run price movementsmay, in some instances, be difficult to support over verylong periods. An alternative approach would be to use ashort-run framework in which the imputed price forApril is based on the (say, overall) mean price changebetween the preceding and current period, i.e. (3.5/2.5)� 4=5.6 in the above example. In this case, theprice change between March and April is (5.6+3.5)/(2.5+4)=1.40. This is combined with the price changebetween January and March 6.5/5=1.30, to give theJanuary to April change of 1.30 � 1.40=1.82, an 82 percent increase.

7.166 Consider why the short-run result of 82 percent is larger than the long-run result of 75 per cent. Theprice change for A between March and April of 40 percent, upon which the short-run imputation is based, islarger than the average annual change of A, which is justover 20 per cent. The extent of any bias from thisapproach was found, above, to depend on the ratio ofmissing values, and the difference between the averageprice change of the matched sample and the quality-adjusted price change of the item that went missing, hadit continued to exist. The short-run comparison is to be

ADJUSTING FOR QUALITY CHANGE

131

favoured if the assumption of similar price changes isconsidered more likely to hold than the long-runassumption.

7.167 There are data on price changes of the itemthat are no longer available, item B in Table 7.5, up to theperiod preceding the period in which it is missing. InTable 7.5, item B has price data for January, Februaryand March. The long-run imputation makes no use ofsuch data, simply assuming that price changes over theperiod of January to April, for example, are the samefor B as for A. Let the data for B’s prices in Table 7.5(penultimate row) now be 3, 4 and 6 in January, Feb-ruary and March, respectively, instead of 3, 3 and 4. Thelong-run estimate for B in April is 5.25, as before. Theestimated price change between March and April for B isnow a fall from 6 to 5.25. A short-run imputation basedon the price movements of A between March and Aprilwould more correctly show an increase from 6 to (3.5/2.5)� 6=8.4.

7.168 There may, however, be a problem with thecontinued use of short-run imputations. Returning to thedata for A and B in Table 7.5, consider what happensin May. Adopting the same short-run procedure, theimputed price change is given in Table 7.5 as (4/3.5)�5.6=6.4 and for June as (5/4)� 6.4=8. In the formercase, the January to May price change is:

(6:4+4)

(5:6+3:5)

� �� (5:6+3:5)

(3+2)

� �=2:08

and in the latter, for June:

(8+5)

(6:4+4)

� �� (6:4+4)

(3+2)

� �=2:60

compared with long-run comparisons for May and June,respectively, of:

((4=2)� 3+4)

(3+2)

� �=2:00

((5=2)� 3+5)

(3+2)

� �=2:50

7.169 A note of caution is required. The compar-isons here use an imputed value for item B in April andalso an imputed value for May. The price comparisonfor the second term in equation (7.36) above, for thecurrent versus immediately preceding period, usesimputed values for item B. Similarly, for the January toJune results, the May to June comparison uses imputedvalues for item B for both May and June. The pragmaticneeds of quality adjustment may of course demand this.If comparable replacements, overlap links and resourcesfor explicit quality adjustment are unavailable, animputation must be considered. However, using impu-ted values as lagged values in short-run comparisonsintroduces a level of error into the index which willbe compounded with their continued use. Long-runimputations are likely to be preferable to short-runchanges based on lagged imputed values, unless thereis something in the nature of the industry that cau-tions against such long-run imputations. There are

circumstances in which the price collector may believethe missing item to be temporarily missing, and theimputation is conducted in the expectation that pro-duction will subsequently continue; a wait-and-see pol-icy is adopted under some rule, say that the item ismissing for a maximum of three months, after which theitem is deemed to be permanently missing. Such prag-matic situations require imputations to extend valuesover consecutive periods and call for the use of laggedimputed values to be compared with current imputedvalues, despite the fact that this is cautioned against,especially over several months. There is an intuitivefeeling that the period over which this is undertakenshould not be extensive. First, the effective sample sizedecreases as the use of imputation increases. Second, theimplicit assumptions of similar price movements inher-ent in imputations are less likely to hold over the longerrun. Finally, there is some empirical evidence, albeitfrom a different context, against the use of imputedvalues as if they were lagged actual values (see Feenstraand Diewert’s study (2001) using data from the UnitedStates Bureau of Labor Statistics for their InternationalPrice Program).

7.170 The above short-run approach will be devel-oped in the next section, where weighted indices areconsidered. The practice of estimating quality-adjustedprices is usually carried out at the elementary itemlevel. At this lower level, the prices of items may sub-sequently be missing, and replacements with or withoutadjustments and imputations are used to allow theseries to continue. New items are also being introduced,as are newer varieties and switching of sales betweensections of the index. The turmoil of changing qualityis not just about maintaining similar price compari-sons, but also about the accurate reweighting of the mixof what is consumed. Under a Laspeyres framework,the bundle is held constant in the base period, so anychange in the relative importance of items consumed isheld to be of no concern until the next re-basing of theindex. Yet procedures for updating the weights arerequired to capture something of the very real changesin the mix of what is consumed. This is considered inChapter 9. The concern here is with an equivalenthigher-level procedure to the short-run adjustmentsdiscussed above. It is one particularly suited to coun-tries where resource constraints prohibit the regularupdating of weights through regular household surveys.

Single-stage and two-stage indices7.171 Consider aggregation at the elementary level.

This is the level at which prices are collected from arepresentative selection of outlets across regions in aperiod and compared with the matched prices of thesame items in a subsequent period to form an index for,say, lamb. Each price comparison is equally weightedunless the sample design gives proportionately morechance of selection to items with more sales. The ele-mentary price index for lamb is then weighted, andcombined with the weighted elementary indices for otherproducts to form the consumer price index. A Jevons

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

132

elementary aggregate index for period t+6 comparedwith period t, for example, is given as:

PJ QN

i 2N(t+6)\N(t)( pt+6

i =pTi ): (7.37)

Compare this with a two-stage procedure:

PJ QN

i 2N(t+5)\N(t)( pt+5

i =pTi )QN

i 2N(t+6)\N(t+5)( pt+6

i =pT+5i ) (7.38)

7.172 If an item is missing in period t+6, an impu-tation may be undertaken. If equation (7.37) is used, therequisite assumption is that the price change of the miss-ing item, had it continued, is equal to that of the averageof the remaining items over the period t to t+6. Inequation (7.38), the missing item in period t+6 may beincluded in the first stage of the calculation, betweenperiods t and t+5, but excluded in the second stage,between periods t+5 and t+6. The requisite assumptionis that price changes between t+5 and t+6 are equal.Assumptions of short-run price changes are generallyconsidered to be more valid than their long-run counter-parts. The two-stage framework also has the advantage ofincluding in the worksheet prices for the current periodand the immediately preceding one which, as is shown inChapter 9, promotes good data validity checks.7.173 Feenstra and Diewert (2001) applied a num-

ber of mainly short-run imputation procedures to

price comparisons for the United States Bureau ofLabor Statistics International Price Program. Althoughsuch price indices are not the direct interest of thismanual, the fact that about one-quarter of the indivi-dual items tracked did not have price quotations in anygiven month makes it an interesting area in which toexplore the results from different imputation pro-cedures. When using the two-stage procedure, Feenstraand Diewert (2001) advise against carrying forwardimputed period prices as if they were actual values, forthe subsequent price comparison. The resulting pricerelatives for the subsequent period based on priorimputations had a standard deviation about twice thatof price relatives where no imputation was required,leading the authors to conclude that such a practiceintroduced a significant amount of error into the cal-culation. Feenstra and Diewert (2001) found that highervariances of price changes arose from long-run impu-tation compared with the short-run imputation method.They also found, from both theory and empirical work,that when actual prices become available in a futuredata set and were used to interpolate back on a linearbasis the missing prices, then such estimates lead tomuch lower variances than the short-run imputationapproach. Such linear interpolations, however, requirethe statistical agency to store past information until aprice quote becomes available, interpolate back themissing price, and then publish a revised consumer priceindex.

ADJUSTING FOR QUALITY CHANGE

133

Appendix 7.1 Data on personal computers, obtained from United KingdomCompaq and Dell web sites, July 2000, to illustrate hedonic regression

PRICE (£) SPEED (MHz) RAM, MB. HD, MB. DELL PRESARIO PROSIGNIA CELERON PENTIUM III CD-RW DVD DELL*SPEED (MHz)

2123 1000 128 40 0 1 0 0 0 0 0 0

1642 700 128 40 0 1 0 0 0 0 0 0

2473 1000 384 40 0 1 0 0 0 0 0 0

2170 1000 128 60 0 1 0 0 0 0 0 0

2182 1000 128 40 0 1 0 0 0 0 1 0

2232 1000 128 40 0 1 0 0 0 1 0 0

2232 1000 128 40 0 1 0 0 0 0 0 0

1192 700 384 40 0 1 0 0 0 0 0 0

1689 700 384 60 0 1 0 0 0 0 0 0

1701 700 384 40 0 1 0 0 0 0 1 0

1751 700 384 40 0 1 0 0 0 1 0 0

1851 700 384 40 0 1 0 0 0 0 0 0

2319 933 128 15 0 0 0 0 1 0 0 0

2512 933 256 15 0 0 0 0 1 0 0 0

2451 933 128 30 0 0 0 0 1 0 0 0

2270 933 128 10 0 0 0 0 1 0 0 0

2463 933 256 10 0 0 0 0 1 0 0 0

2183 933 64 10 0 0 0 0 1 0 0 0

1039 533 64 8 0 0 1 1 0 0 0 0

1139 533 128 8 0 0 1 1 0 0 0 0

1109 533 64 17 0 0 1 1 0 0 0 0

1180 533 64 8 0 0 1 1 0 1 0 0

1350 533 128 17 0 0 1 1 0 1 0 0

1089 600 64 8 0 0 1 0 1 0 0 0

1189 600 128 8 0 0 1 0 1 0 0 0

1159 600 64 17 0 0 1 0 1 0 0 0

1230 600 64 8 0 0 1 0 1 1 0 0

1259 600 128 17 0 0 1 0 1 0 0 0

1400 600 128 17 0 0 1 0 1 1 0 0

2389 933 256 40 0 1 0 0 1 0 0 0

1833 733 256 40 0 1 0 0 1 0 0 0

2189 933 128 40 0 1 0 0 1 0 0 0

2436 933 256 60 0 1 0 0 1 0 0 0

2397 933 256 40 0 1 0 0 1 0 1 0

2447 933 256 40 0 1 0 0 1 1 0 0

2547 933 256 40 0 1 0 0 1 0 0 0

2845 933 384 60 0 1 0 0 1 0 0 0

2636 933 384 60 0 1 0 0 1 0 0 0

1507 733 64 30 0 1 0 0 1 0 0 0

1279 667 64 10 1 0 0 0 1 0 0 667

1379 667 128 10 1 0 0 0 1 0 0 667

1399 667 64 30 1 0 0 0 1 0 0 667

1499 667 128 30 1 0 0 0 1 0 0 667

1598 667 128 30 1 0 0 0 1 1 0 667

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

134

1609 667 128 30 1 0 0 0 1 0 1 667

1389 667 64 10 1 0 0 0 1 0 1 667

999 667 64 10 1 0 0 1 0 0 0 667

1119 566 64 30 1 0 0 1 0 0 0 566

1099 566 128 10 1 0 0 1 0 0 0 566

1097 566 64 10 1 0 0 1 0 1 0 566

1108 566 64 10 1 0 0 1 0 0 1 566

1219 566 128 30 1 0 0 1 0 0 0 566

1318 566 128 30 1 0 0 1 0 1 0 566

1328 566 128 30 1 0 0 1 0 0 1 566

1409 566 128 10 1 0 0 0 1 0 0 733

1809 733 384 10 1 0 0 0 1 0 0 733

1529 733 128 30 1 0 0 0 1 0 0 733

1519 733 128 10 1 0 0 0 1 0 1 733

1929 733 384 30 1 0 0 0 1 0 0 733

2039 733 384 30 1 0 0 0 1 0 1 933

2679 933 128 30 1 0 0 0 1 0 0 933

3079 933 384 10 1 0 0 0 1 0 0 933

2789 933 128 10 1 0 0 0 1 0 1 933

3189 933 384 10 1 0 0 0 1 0 1 933

ADJUSTING FOR QUALITY CHANGE

135

8ITEM SUBSTITUTION, SAMPLE SPACEAND NEW PRODUCTS

Introduction8.1 As new items are introduced and old items no

longer sold, the universe of items from which prices aresampled changes. Yet index number methodology mayconstrain the sampling to subsets of the universe. Thesamples selected from such subsets are referred to hereas the ‘‘sample space’’ of the index. A focus of thischapter is the limitations of such sample spaces. InChapter 7 the use of the matched models method wasrecognized as the accepted approach to ensuring that themeasurement of price changes was untainted by changesin quality. It was noted, however, that the approachmight fail in three respects: missing items, the limitedsample space, and new goods and services (in theremainder of this chapter ‘‘goods’’ is taken to includeservices). In Chapter 7, several implicit and explicitmethods of quality adjustment to prices, and the choicebetween them, are discussed as ways of dealing withmissing items. In this chapter, attention is turned to thetwo other reasons why the matched models method mayfail: sampling concerns (the limited sample space) andnew products. The three sources of potential error arefirst briefly outlined below.8.2 Missing items. A problem arises when an item is

no longer produced. An implicit quality adjustment maybe made using the overlap or imputation method, or therespondent may choose a replacement item of a com-parable quality and its price may be directly comparedwith the missing item’s price. If the replacement is ofa non-comparable quality, an explicit price adjustmentis required. This was the subject of Chapter 7, para-graphs 7.72 to 7.115. In paragraphs 7.125 to 7.158 acaveat was added. It was recognized that for items inindustries where model replacements are rapid, con-tinued long-runmatching depletes the sample and qualityadjustment becomes unfeasible on the scale required.Chained matching or hedonic indices are deemed pre-ferable.8.3 Sampling concerns. The matching of prices of

identical items over time, by its very nature, is likely tolead to the monitoring of a sample of items that is in-creasingly unrepresentative of the population of trans-actions. Price collectors may keep following thoseselected items until they are no longer available. Thus,price collectors may continue to monitor old items, withunusual price changes and limited sales. With regard toitem replacement, price collectors may select unpopularcomparable items in order to avoid explicit quality adjust-ments. Thus obsolete items with unusual price changesmay be replaced by near obsolete items, again withunusual price changes. That the replacement items are

near obsolete will mean that their expenditure shares willbe relatively small. This will compound the problem ofunrepresentative samples. The substitution of an itemwith relatively high sales for an obsolete one has its ownproblems, since the difference in quality is likely to besubstantial and substantive, beyond that which can beattributed to, say, the price difference in some overlapperiod. One item might be in the last stage of its life cycleand the other in the first stage of its life cycle. The prob-lem has implications for sample rotation and item sub-stitution.

8.4 New products. A third potential difficulty ariseswhen something ‘‘new’’ is produced. There is a difficultyin distinguishing between new items and quality changesin old ones, and this is discussed below. When a quitenew good is produced, there is a need for it to beincluded in the index as soon as possible, especially if theproduct is expected to be responsible for relatively highsales. New goods might have quite different pricechanges from those of existing ones, especially at thestart of the life cycle. Furthermore, in the initial periodof introduction there is often a welfare gain to theconsumer. The new good is not a perfect substitutefor the old good and this uniqueness gives economicvalue to the consumer which would have otherwisenot been obtained, had the new good not been avail-able (Trajtenberg, 1989). But by definition, there is noprice for the new product in the period preceding itsintroduction. So even if prices of new products areobtained and included in the index from the initialintroduction date, there is still something missing – theinitial gain in welfare that consumers experience in theperiod of introduction. The difficulties in capturing sucheffects are discussed in paragraphs 8.59–8.60 andAppendix 8.2.

8.5 The problem of missing items was the subject ofChapter 7. This chapter considers sampling concernsarising out of the matched models approach and theproblem of introducing new products into the index.

Matched samples8.6 The matching procedure has at its roots a

conundrum. Matching is designed to avoid pricechanges being contaminated by quality changes. Yetits adoption constrains the sampling to a static universeof items that exist in both the reference and baseperiods. Outside this matched sample, there are ofcourse the items that exist in the reference period butnot in the current period, and are therefore notmatched, and similarly those new items existing in

137

the current period but not in the reference period – thedynamic universe (Dalen, 1998a; Sellwood, 2001). Theconundrum is that the items not in the matched uni-verse – the new items appearing after the referenceperiod and the old items that disappeared from thecurrent period – may experience price changes thatdiffer substantially from the price changes of existingmatched items. This is because these products willembody different technologies and be subject to dif-ferent (quality-adjusted) strategic price changes. Thevery device used to maintain a sample of constantquality, i.e., matching, may itself give rise to a samplethat is biased away from technological developments.Furthermore, when this matched sample is used toimpute the price changes of missing items (see Chapter7, paragraphs 7.53 to 7.68), it will reflect the technologyof a sample that is not representative of current tech-nological changes.

8.7 A formal consideration of matching and thedynamic universe is provided in Appendix 8.1 to thischapter. Three universes are considered:

� an intersection universe, which includes only matcheditems;

� a dynamic double universe, which includes all items inthe base comparison period and all in the currentperiod, although they may be of different qualities;

� a replacement universe, which starts with the baseperiod universe, but also includes one-to-one replace-ments when an item from the sample in the baseperiod is missing in the current period.

8.8 It is, of course, difficult to ascertain the extentto which matching from the intersection universe con-strains the penetration of the sample into the dynamic,double universe, since statistical agencies generallydo not collect data for the latter. Its extent will in anyevent vary between products. Sellwood (2001) advo-cated simulations using the universe of scanner data.Silver and Heravi (2002) undertook such an experimentusing scanner data on the consumer prices of washingmachines in the United Kingdom in 1998. A matchedLaspeyres index, based on price comparisons withmatched models existing in both January and Decem-ber, covered only 48 per cent of December expenditureon washing machines, as a result of new models intro-duced after January not being included in the matchedindex. Furthermore, the January to December matchedcomparison covered slightly more than 80 per cent ofJanuary expenditure, resulting from the exclusion ofmodels available in January but not in December. Abiannual sample rotation (rebasing) increased theDecember expenditure coverage to just over 70 per cent,while a monthly (chained) rotation increased that cov-erage to 98 per cent (see also Chapter 7, paragraphs7.128 to 7.131 for further examples). Two implicationsarise from this. First, the selection of item substitutes(replacements) puts the coverage of the sample to someextent under the control of the price collectors. Guide-lines on directed replacements in particular productareas have some merit. Second, chaining, hedonic indi-ces (as considered in Chapter 7, paragraphs 7.125 to7.158) and regular sample rotation have merit in some

product areas as devices to refresh the sample. These areconsidered in turn.

Sample space and itemreplacement or substitution

8.9 When an item goes missing, one possibility is forthe price collector to select a replacement item. Thesample space of the index is thus the matched itemsinitially selected and replacement items selected whenmatched items are missing. Price collectors are often bestplaced to select replacement items. The price collectorsare often physically present in the same store as themissing item and thus any replacement price selected islikely to be unaffected by price differences which may beattributed to differences in the services (ease of location,parking, warranties, service) provided by different stores.It may also be the case that an obvious replacement isprovided by a store which wishes to cater to the samemarket segment, and this will be conspicuous to the pricecollector. Sometimes the replacement may have a dif-ferent code or model number which a desk officer maytake to indicate a different item, but which the pricecollector can identify as simply being a difference of, say,a colour or packaging. Price collectors can also identifywhether a new (replacement) model of an item has stylingand other qualitative factors so different from the oldmodel that in themselves they would account for sub-stantial price differences. In such instances, a desk officermay only focus on the technical specifications and beunaware of these other differences. Against this, deskofficers have additional information. This might includeinformation from a similar store in a different locationon the price of the missing item, which might be tem-porarily out of stock.

8.10 The price collector takes on the task of identi-fying whether an item is of comparable quality or not. Ifthe price collector judges the item to be comparable whenin fact it is not, the quality difference will be taken to be aprice difference, resulting in bias where the unrecognizedquality changes are in a consistent direction. Informedcomparable substitution requires general guidelines onwhat makes a good substitute, as well as product-specificinformation on characteristics likely to determine price.It also requires timely substitution, to maximize theprobability of an appropriate substitute being available.

8.11 Guidelines for selection of comparable itemsand monitoring of the nature of the selections are goodpractice. Liegey (1994) notes how useful the results fromhedonic regressions are in the selection of items. Theresults provide an indication of the major quality factorsthat explain price variation in the product or service.Price collectors can thus be given guidelines as to whichcharacteristics are important – in the sense that they areprice determining – in the selection of the sample andreplacement items.

8.12 The matter of sample space requires considera-tion regarding the selection of replacement items/sub-stitutes for missing items. The initial selection of itemswhose prices are matched may best be made at random,though such items are more often selected as those

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

138

‘‘typically’’ purchased. Similarly items ‘‘typically’’ pur-chased should be included as replacements. Not all pricecollectors should aim to sample the same ‘‘most typical’’item. It is desirable to sample a distribution of itemswhich broadly represents the distribution of purchases.For example, a particular brand – one that accounts for,say, 40 per cent of sales revenue –may be known to be themarket leader. This common knowledge should not leadall price collectors to select that brand on rebasing. Arepresentative sample is required.8.13 Replacement items should intrude into the

universe of transactions so that the sample is broadlyrepresentative of the dynamic universe. The inclusion ofa popular replacement item to refresh the sample – one atthe same point in its life cycle as the original popular oneselected in the base period – allows for a useful andaccurate price comparison, assuming that an appropriatequality adjustment is made. Substitute or replacementitems should, where possible, not merely be comparablein quality, but should also be likely to account for arelatively substantial amount of sales value. It is of littlemerit to substitute a new item with limited sales for amissing item, again with limited sales, just because theyhave similar features, both being ‘‘old’’; the index wouldbecome more unrepresentative. The replacement of anitem only when the item is no longer available may beineffectual with regard to the representativity of theindex. In that case, items with relatively low sales wouldcontinue to be monitored until they died. And evenreplacement might not remedy the situation. If thereplacement guidelines indicate that the price collectorshould select a similar item sold in the outlet, then thereplacement selected will be almost as obsolete (Lane,2001, p. 21).8.14 Guidelines to select ‘‘similar’’ items are given to

ease quality adjustment between the old and new items;at best the items are ‘‘comparable’’ and require noquality adjustment. The institutional mechanism devisedto help in making quality adjustments to prices can leadto bias because of its adherence to a sample of itemswhich do not enjoy the benefits of recent technologicalinnovations and are unrepresentative of what is pro-duced. Bear in mind that an index number methodologybased on an initially selected matched sample and asample of substitute replacement items, when items gomissing, may not be representative of the universe of allitems being consumed. In particular, if the index numbermethodology is biased to the selection of replacementitems with relatively low sales, so that they are compar-able with obsolete items, then the sampling from newitems and the sample space of the index are biased.Quality adjustment and representativity are interrelated,since the former affects the sample space of the index.8.15 The importance of, and care required in, the use

of replacements to militate against sample depletion isworth reiterating. Consider the case where there is onlyone model of a product available in the market at thestart of the price comparison in period t. A price collectorincludes it in the sample in period t and then monitors itsprice in subsequent periods. A new (replacement) modelenters the market in, say, period t+2, but it is ignoredsince the original model continues to exist for several

months. However, in, say, period t+9 the old item is nolonger on the market and is replaced, with a qualityadjustment, by the new item. The long-run price com-parison between the newmodel’s price in period t+9 andthe old model’s price in period t has no sampling bias.Both account for 100 per cent of the market in theirrespective periods, being the only items available. Bothare near the start of their life cycles, so the price com-parison is a fair one. If the new and old items have dif-ferent price changes, sampling bias will occur betweenperiods t+2 and t+8, when only one of the two items isbeing sampled, but sampling will be unbiased once themodel is replaced in period t+9.

8.16 There is thus a case for managing the replace-ment strategy to minimize sample depletion. In thatrespect, the following points should be borne in mind:

� Replacements offer an opportunity to cut back on, andpossibly remove, sample bias in the period of replace-ment, though not prior to it.

� The more frequent is the replacement, the less thesample bias.

� If there is more than one new (replacement) item in themarket, there may still be bias as only the most pop-ular one will be selected and it may well be at a dif-ferent stage in its life cycle and thus be experiencingdifferent price changes in comparison with other new(replacement) models.

� The analysis assumes that perfect quality adjustmentsaremade on replacement. The less frequent the replace-ment, the more difficult this might be to achieve, as thevery latest replacement item on the market may havemore substantial differences in quality than earlierones.

� If the best-selling replacement item is of comparablequality and at the same stage in its life cycle as themissing item, then its selection will minimize samplebias.

� If there is more than one replacement item and themost comparable one – having the old technology – isselected, it will have low market share and unusualprice changes.

� Given advance information on market conditions,replacements that are included in the sample wellbefore the old item becomes obsolescent are likely toincrease the sample’s share of the market, includeitems that are more representative of the market, andfacilitate quality adjustment.

8.17 The problem of item substitution is analogousto the problems that arise when an outlet closes. It maybe possible to find a comparable outlet not already in thesample, or a non-comparable one for which, in principle,an adjustment can be made for the better quality ofservice provided. It is not unusual for an outlet to closefollowing the introduction of a new, more competitiveoutlet. Where the matching of prices between theseoutlets broadly follows the consumption patterns of theclients of the original outlet, there is an obvious replace-ment outlet. If, however, the new outlet has comparableprices but, say, a better range of items, parking andservice, there is a gain to consumers from substituting

ITEM SUBSTITUTION, SAMPLE SPACE AND NEW PRODUCTS

139

one outlet for the other. Yet since such facilities have nodirect price, it is difficult to provide estimates of theirvalue in order to make an adjustment for the betterquality of service of the new outlet. The index would thushave an upward bias, which would be lost on rebasing. Insuch cases, replacing the old outlet by a new one thatprovides a similar standard of service may be preferableto replacing it by one that has a different standard butserves the same catchment area. In their regressionanalyses for consumer durables, Liegey (2000), Shepler(2000) and Silver and Heravi (2001b) found ‘‘outlet type’’to be a substantial and statistically significant explana-tory variable for price variation, while for a particularoutlet type – grocery outlets, for food and petrol prices inthe United States – Reinsdorf (1993) found much smallerdifferences.

Sample rotation, chaining andhedonic indices

8.18 It is important to recognize the interrelation-ships among the methods for handling item rotation,item replacement and quality adjustment. When con-sumer price index (CPI) item samples are rotated, this isa form of item substitution, except that it is not ‘‘forced’’by a missing item, but is undertaken for a general groupof items to update the sample of items. It has the effectof making future forced replacements less likely. Yet theassumption implicit in its use is equivalent to that for theoverlap adjustment technique: that price differences arean adequate proxy for the change in price per unit ofquality between items disappearing from the sample andreplacement items.

8.19 Consider the initiation of a new sample ofitems. This may be by probability or judgementalmethods, or a combination of the two. Prices for the oldand new samples are returned in the same month, andthe new index is compiled on the basis of the newsample, the results being linked to the old. This is animplicit use of the overlap method, in which all pricedifferences between the new and old items in that monthare taken to be quality changes. Assume that the newsample is initiated, say, in January. Assume also that theprices of an old item in December and January are $10and $11 respectively, a 10 per cent increase, while thosefor the new replacement item in January and Februaryare $16 and $18 respectively, an increase of 12.5 percent. The new item in January is of a better quality thanthe old, and this difference in quality may be worth$16�11=5 to the consumer. That is, the price differenceis assumed to be equal to the quality difference, which isthe assumption implicit in the overlap method. Had theprice of the old item in December been compared withthe quality-adjusted price of the new item in Januaryunder this assumption, the price change would in thiscase be the same, 10 per cent (i.e. (16�5)/10=1.10). Inpractice, the need to simultaneously replace and updatea large number of items requires the assumptions of theoverlap method, the point being that this process shouldnot be regarded as error free. In cases where theassumptions are considered likely to be particularly

untenable (discussed in Chapter 7, paragraphs 7.44 to7.52), explicit adjustments of the form discussed inparagraphs 7.72 to 7.115 should be used.

8.20 It was noted above that when samples areupdated, any difference in the average quality of itemsbetween the samples is dealt with in a way that isequivalent to the overlap adjustment technique. Samplerotation to refresh the sample between rebasing is anexpensive exercise. If rebasing is infrequent, however,and if there is a substantial loss of items in particularproduct areas, then sample rotation might be appro-priate for those areas. A metadata system (describedbelow) will aid such decision-making. More frequentsample rotation aids the process of quality adjustment intwo ways. First, the new sample will include newer vari-eties. Comparable replacements with substantial saleswill be more likely to be available and non-comparableones will be of a similar quality, which facilitates goodexplicit adjustments. Second, because the sample hasbeen rotated, there will be fewer missing items thanotherwise and thus less need for quality adjustments.

8.21 A natural extension of more frequent samplerotation is to use a chained formulation in which thesample is reselected each period. In Chapter 7, para-graphs 7.153 to 7.158, the principles and methods wereoutlined in the context of sectors in which there was arapid turnover of items. These principles are echoedhere. Similarly, the use of hedonic indices (as outlined inparagraphs 7.132 to 7.152) or the use of short-runcomparisons (discussed in paragraphs 7.159 to 7.173)might be useful in this context.

Information requirements for aquality adjustment strategy

8.22 It should be apparent from the above that astrategy for quality adjustment must not only be linkedto one relating to sample representativity, but must alsorequire the building of a statistical metadata system. Thisis not an area where the approach for the index as awhole can be simply described, but one that requiresthe continual development of market information andthe recording and evaluation of methods on a product-by-product basis.

Statistical metadata system8.23 The methods used for estimating quality-

adjusted prices should be well documented as part of astatistical metadata system. Metadata are systematicdescriptive information about data content and organi-zation that help those who operate the systems thatproduce statistics to remember what tasks they shouldperform and how they should perform them. A relatedpurpose is to train new staff and introduce them to theproduction routines (Sundgren, 1993). Metadata systemsalso help to identify where current methods of qualityadjustment require reconsideration, and prompt the useof alternative methods. They may also serve user needs,the oldest and most extensive form being footnotes.

8.24 The dramatic increase in volume of statisticaldata in machine-readable form, with a concomitant

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

140

increase in metadata, argues for keeping the metadata insuch a form. This is to enhance transparency in themethods used and help ensure that the methods areunderstood and continued, as staff leave the CPI teamand new staff join it. Changes in quality adjustmentmethodology can in themselves lead to changes in theindex. Indices produced using new procedures shouldbe spliced onto existing indices. The metadata systemshould also be used as a tool to help with quality adjust-ment. Because so much of the rationale for the employ-ment of different methods is specific to the features ofthe products concerned, data should be held on suchfeatures.8.25 Statistical agencies should monitor the incidence

of missing items against each Classification of IndividualConsumption according to Purpose (COICOP) group. Ifthat incidence is high, then the monitoring should becarried out by class within each group. Again, if thatincidence is high, the monitoring should be done by ele-mentary aggregate or selected representative items withineach group, or at the most detailed level of the system.Where the incidence is high, the ratios of temporarymissing prices, comparable replacements and non-com-parable replacements to the overall number of prices, andthe methods for dealing with each of these three cir-cumstances, should also be monitored to provide thebasis of a statistical metadata system. The advantage of atop-down approach is that resources are saved by moni-toring only at the detailed level product areas which areproblematic.8.26 Product-specific information, such as the

timing of the introduction of new models, pricing poli-cies (especially with regard to months in which nochanges were made) and the popularity of models andbrands according to different data sources, should beincluded in the metadata as the system develops. Anestimate, if available, of the weight of the productconcerned should be given, so that a disproportionateeffort is not given to relatively low-weighted items. Allthis will lead to increased transparency in the proceduresused and allow effort to be directed where it is mostneeded.8.27 For items for which replacement levels are high,

the metadata system would benefit from contacts be-tween statistical agencies and market research organiza-tions, retailers, manufacturers and trade associations.Such links will allow staff to better judge the validity ofthe assumptions underlying implicit quality adjustments.Where possible, staff should be encouraged to be respon-sible for learning more about specific industries whoseweights are relatively high and where item replacement iscommon.8.28 Statistical staff should identify price-determining

characteristics for product areas using hedonic regres-sions, information from market research, store man-agers, trade associations and other such bodies, and theexperience of price collectors. This information shouldcontribute to the statistical metadata system and beparticularly useful in providing subsequent guidelines onitem selection.8.29 When hedonic regressions are used either for

partial patching of missing prices or as indices in their

own right, information on the specification, estimatedparameters and diagnostic tests of the regression equa-tions should be kept, along with the data and with notesas to why the final formulation was chosen and used.This will allow the methodology for subsequent updatedequations to be benchmarked and tested against theprevious versions.

8.30 The metadata system should help statisticalstaff to:

– identify product areas likely to be undergoing regulartechnological change;

– ascertain the pace at which models change and, pos-sibly, the timing of changes;

– undertake an analysis of what have in the past beenjudged to be ‘‘comparable’’ replacements in terms ofthe factors that distinguish the replacement and olditems;

– identify whether different price collectors are makingsimilar judgements regarding comparable replace-ments, and whether such judgements are reasonable.

8.31 Price statisticians may have more faith in theuse of some quality adjustment procedures than others.When such procedures are used extensively it might beuseful to note, as part of the metadata system, the degreeof faith the statistician has in the procedures. Follow-ing Shapiro and Wilcox (1997b), this may be envisagedas a traditional confidence interval: the statistician maybelieve at, say, a 90 per cent level of confidence that thequality-adjusted price change is 2 per cent (0.02) with aninterval of plus or minus 0.5 per cent (0.005). There maybe an indication as to whether the interval is symmetric,or positively or negatively one-sided. Alternatively, sta-tisticians may use a simple subjective coding on a scaleof, say, 1 to 5.

New products and how they differfrom products with quality changes

8.32 The question arises of how to define new prod-ucts (goods and services) and to distinguish them fromexisting products whose quality has changed. A newmodel of a good may provide more of a currentlyavailable set of service flows. For example, a new carmodel may differ from existing ones in that it may havea bigger engine. There is a continuation of a service andproduction flow, and this may be linked to the serviceflow and production technology of the existing models.A practical definition of a new good, as against qualitychanges in an updated existing model, is that, first, thenew good cannot easily be linked to an existing item as acontinuation of an existing resource base and serviceflow, because of the very nature of its ‘‘newness’’. Forexample, frozen foods, microwave ovens and mobilephones, while extensions of existing flows of services tothe consumer, have a dimension of service that is quitenew. Second, as discussed below, new goods can gen-erate a welfare gain to consumers by their very intro-duction. The simple introduction of the new good intothe index, once two successive price quotes are available,misses this gain.

ITEM SUBSTITUTION, SAMPLE SPACE AND NEW PRODUCTS

141

8.33 Oi (1997) directs the problem of defining ‘‘new’’goods to that of defining a monopoly. If there is noclose substitute, the good is new. He argues that indi-vidual new books, new videos and new television serialsmay have quite small cross-price elasticities in somecases; their shared service is to provide entertainmentand they are similar in this respect. Hausman (1997),however, found cross-elasticities for substitution to bequite substantial for new television serials (though seeBresnahan (1977)). There are many new forms ofexisting products, such as fashionable toys and clothes,which are not easily substitutable for similar items andfor which consumers would be willing to pay a premium.

8.34 Bresnahan (1997, p. 237) notes that Brandweekcounted over 22,000 new-product introductions for theUnited States for 1994 – the purpose of their introduc-tion being, as differentiated products, not to be exactsubstitutes for existing ones, but to be distinct. Theirdistinctiveness is in many cases the rationale behind theirlaunch. The extent of differentiated markets neverthelessmakes the definition and treatment of such things as‘‘new’’ impractical. Oi (1997, p. 110) sets out the prag-matic case: ‘‘Our theory and statistics would be undulycluttered if separate product codes had to be set aside forClear Coke and Special K.’’ Furthermore, the techniquesfor including such products are not, as indicated below,readily applicable. The sound practical advice given byOi (1997) to keep matters ‘‘uncluttered’’ is therefore notunreasonable.

8.35 The terminology adopted here is that used byMerkel (2000) for producer price index (PPI) measure-ment, but considered in a CPI context. It distinguishesbetween evolutionary goods and revolutionary goods.Evolutionary goods are replacement or supplementarymodels that continue to provide a similar service flow,but maybe in new ways or to different degrees. Theseare distinguished from revolutionary goods, which areentirely new goods not closely tied to a previously avail-able product. Although revolutionary goods may satisfya long-standing consumer need in a novel way, theydo not fit into any established CPI item category (Arm-knecht et al., 1997). Problems are associated withincorporating distinctly new revolutionary goods. This isbecause a good, which by its nature is unique, is unlikelyto be incorporated into the sample as a replacement foran existing item. It would neither be comparable nor beamenable to explicit adjustments to its price for qualitydifferences with existing goods. Since a distinctly newitem is not replacing an item, it does not have an existingweight; its introduction therefore implies a need to re-weight the index.

Incorporation of new products8.36 There are three major concerns regarding the

incorporation of new goods into the CPI. The first con-cern is the detection and identification of the new good;these are facilitated by close links with market research,and producer and trade associations. The second con-cern, which is related to the first, is the decision on theneed and timing for their inclusion. This refers to both

the weight and price changes of the new good. The thirdconcern relates to the incorporation of the initial welfareto the consumer arising from the switch from the oldtechnology.

8.37 Consider some examples on the timing ofthe introduction of new goods. The sales of mobilephones were at such a significant level in some countriesthat their early inclusion in the CPI became a matter ofpriority. They simply rose from nothing to relativelyquickly account for quite a large proportion of sales intheir product classification. Furthermore, their pricechanges were atypical of other goods in their productclassification. Being new, they might be produced usinginputs and technologies quite different from those usedfor existing telephones. Because of substantial marketingcampaigns, many new goods command substantial salesand are the subject of distinct pricing strategies at launch.For radical innovations, however, there may be a delayin their incorporation into the index since they cannot bedefined within existing classification systems.

8.38 Armknecht et al. (1997) cite the example of theincorporation of video cassette recorders (VCRs) intothe United States CPI. VCRs were launched in 1978 witha sales value of US$299 million and estimated averageretail price of US$1,240. Because the CPI was rebasedevery ten years, VCRs were introduced into the CPI onlyin 1987 when their sales value was US$3,442 million andaverage price had fallen to US$486. All the extraordinaryprice movements between 1978 and 1987 were thusmissed by the index.

8.39 Dulberger (1993) provides some estimates forUnited States PPIs for dynamic random access memory(DRAM) computer memory chips. She calculated priceindices for the period 1982 to 1988 with varying amountsof delay in introducing new chips into the index. Theindices were chained so that new chips could be intro-duced, or not, as soon as they had been available for twosuccessive years. Using a Laspeyres chained index, therewas a fall of 27 per cent if there was no delay in intro-ducing the new goods, as compared with falls of 26.2 percent, 24.7 per cent, 19.9 per cent, 7.1 per cent and 1.8 percent if the introductions were delayed by, respectively, 1,2, 3, 4 or 5 years. In all cases the index was biaseddownwards because of the delay. Berndt et al. (1997)provide a detailed study of a new anti-ulcer drug,Tagamet. They found that the effects of pre-introductionmarketing of the drug on its price and market share atintroduction were substantial. Not unexpectedly, therewere price falls for the generic form of a pharmaceuticalon the expiry of the patent, but there were increases forthe branded form; loyal customers were found to bewilling to pay a premium over the price prior to thepatent expiry (Berndt et al., 2003).

8.40 Waiting for a new good to be established orwaiting for the rebasing of an index before incorporatingnew products may lead to errors in the measurement ofprice changes if the unusual price movements at criticalstages in the product life cycles are ignored. Strategies arerequired for the early identification of new products, andmechanisms are needed for their incorporation either atlaunch (if preceded by major marketing strategies) orsoon after (if there is evidence of market acceptance).

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

142

These strategies and mechanisms should form part ofthe metadata system. Waiting for new products toachieve market maturity may result in an implicit policyof ignoring the quite disparate price movements thataccompany their introduction (Tellis, 1988, and Parker,1992). This is not to say that new goods will always havedifferent price changes. Merkel (2000) gives the exampleof ‘‘light’’ varieties of foods and beverages, which aresimilar to the original varieties but with fewer calories.The prices of ‘‘light’’ products are very close to the pricesof the original products. The introduction of ‘‘light’’varieties simply serves to expand the market. While thereis a need to capture such expansion when the weights arerevised, the price changes for the existing items can beused to capture those of the ‘‘light’’ items.8.41 The second measurement concern with respect

to new products is the incorporation of the effect ofthose products at launch. The preceding discussion isconcerned with the incorporation of price changes intothe index once two successive quotations are available.Yet there is a gain to the consumer when comparing theprice in the first of these periods with the price in theperiod that preceded the introduction of the product.This latter price is a hypothetical price. It is the pricewhich would make the demand from the communityfor the product equal to zero; that is, it is the reservationprice which, when inserted into the demand function, setsdemand to zero. If a demand system can be estimated,so too can the reservation price. The virtual reservationprice is compared with the actual price in the period ofintroduction, and this is used to estimate the surplusfrom the introduction of the good. If the reservationprice is relatively high, then the introduction of the newgood is clearly of some benefit to the consumer. Toignore this benefit, and the change from the shadowprice to the actual price in its period of launch, is toignore something of the price movements that give riseto improvements in the standard of living. Of course, ifa ‘‘new’’ good is a close substitute – at the price it isbrought into the index – for goods already in existence,then no additional consumer surplus is generated.8.42 It should be noted that a consumer may be in a

geographical area in which a new good or service, say,cable television, a video rental outlet or health facility, isnot present. The benefits of the new good on its intro-duction to different geographical areas will thereforedevelop over time as the new good becomes more gen-erally available. The benefits will emerge again and againfor each sector of the population that benefits fromaccess to the new product. In practice, such items gainincreasing weight as the index is rebased or the samplerotated.8.43 The methods outlined below for the inclusion of

substitute and new goods include both normal CPIprocedures and exceptional treatments. In regard to theformer, consideration is given in paragraphs 8.44 to 8.58to the rebasing of the index, rotating of items, intro-duction of new goods as replacements for discontinuedones on rotating, and a strategy for dealing with newitem bias. In regard to the latter, techniques that requiredifferent sets of data are outlined. The use of chainedmatched models and hedonic indices was discussed in

Chapter 7 in the context of products experiencing rapidturnover in models. Analytic frameworks that considernew goods bias by way of reservation prices and sub-stitution effects are considered in paragraphs 8.59 and8.60 and Appendix 8.2. The data requirements andeconometric expertise are much more demanding forthese approaches.

Sample rebasing and rotation8.44 A new good may be readily incorporated in the

index at the time of rebasing the index, or when thewhole or the pertinent part of the sample is rotated. Ifthe new good has, or is likely to have, substantial sales,and is not a replacement for a pre-existing one, or islikely to command a much higher or lower market sharethan the pre-existing one it is replacing, then new weightsare necessary to reflect this. New weights are only fullyavailable on rebasing, not on sample rotation. There willthus be a delay in the new item’s inclusion in the index.The extent of the delay will depend on how close theintroduction of the item is to the next rebasing and, moregenerally, the frequency with which the index is rebased.This discussion of rebasing is effectively concerned withthe use of new weights for the index. Even if the index isrebased annually and chained, there will be a delay untilthe annual rebasing before weights can be assigned, andthere may even be a further six-month delay for thesampling and collating of the survey results for theweights. Such frequent rebasing allows for the earlyintroduction of a new good and is to be advised when theweights are not keeping pace with product innovations.

8.45 At the elementary level of aggregation, animplicit weight equal to the expenditure share is given bythe Jevons index, for example, to each price relative. TheDutot index gives each price change the weight of itsprice relative to the sum of the prices in the initial baseperiod of the comparison (see Chapter 7). If a productarea is expected to be subject to dynamic innovations,then the sample may be increased on rotation, withoutany changes to the weight for the group. There wouldsimply be more items selected to form the arithmetic orgeometric average price change. As new varieties becomeavailable, they could be substituted for some of theexisting ones, there being a wider range from which todraw a comparable item, or less effort involved in thequality adjustment procedure for a non-comparable one.

8.46 Some statistical agencies rotate (resample) itemswithin product groups. Opportunities exist to introducenew items within a weighted group under such circum-stances. The resource practicalities of such schemesrequire items to be rotated on a staggered basis for dif-ferent product groups. Product groups experiencingrapid change should be rotated more frequently. Theincorporation of new goods using sample rotation allowssome of the existing weight of the product group to be re-allocated to the new good. Yet it implicitly uses theoverlap method for the introduction of the new good ofa different quality. The difference in prices in the overlapperiod of the new and obsolete items is assumed tobe equal to their quality difference. The assumptionsimplicit in such procedures have been outlined above and

ITEM SUBSTITUTION, SAMPLE SPACE AND NEW PRODUCTS

143

their likely veracity needs to be considered. Since evo-lutionary items are defined as continuations of the ser-vice flow of existing (and possibly exiting) ones, thehedonic framework may be more suitable in some casesto the use of the overlap method. These and othermethods and the choice between methods are discussedin Chapter 7.

8.47 In many countries, rebasing is infrequent andsample rotation is not undertaken, despite their advan-tages. The rotating of samples on a frequent basis shouldnot, however, be considered a panacea. Sample rotationis an arduous task, especially when performed over arange of product groups experiencing rapid change. Evenfrequent rotation, say, every four years, may miss manynew goods. Yet it is not necessary for statistical agenciesto wait until an item is obsolete before a new one isintroduced. It is quite feasible for statistical agencies topre-empt the obsolescence of an old item and decide onits early substitution by a new one. In some productareas, the arrival of a new good is well advertised inadvance of the launch. In other areas, it is feasible for astatistical agency to have general procedures for sub-stitutions, as outlined below. Without such a strategy,and where rotation or rebasing is infrequent, a countrywould be open to serious new product bias.

8.48 In summary:

� The treatment of a new good as a replacement for anexisting one can be undertaken if the old item’s weightsuitably reflects the new good’s sales and if a suitablequality adjustment can be made to its price to link itto the existing, old, price series.

� If the new good does not fit into the pre-existingweighting structure, it can be included on rebasing,though this may be infrequent in some countries.

� Regular sample rotation provides a means by whichthe inclusion of such items can be formally recon-sidered, though since this is undertaken on a staggeredbasis, only the weights within the product group arereallocated, not those between the groups.

� Directed sample substitution, as opposed to waitingfor sample rotation, may be used to pre-empt thearrival of new goods.

� Revolutionary items will not fit into existing weight-ing structures and alternative means are required.

� The modified short-run or chained framework out-lined in Chapter 7, paragraphs 7.153 to 7.173, may bemore appropriate for product areas with high turn-over of items.

Directed replacements for evolutionary items and di-rected augmentation of the sample for revolutionarygoods are considered below.

Directed replacements andsample augmentation

8.49 For evolutionary goods in product areas wherethere is a rapid replacement and introduction of suchgoods, apolicyofdirectedreplacementsmightbeadopted.Judgement, experience, discussions with store managers,market research companies and a statistical metadatasystem should help identify such products. The selection

of replacements is directed to evolutionary items in orderto ensure that the index maintains its representativity. Ifthe new version of a product is designed as a replacementfor an existing one, then substitution might be auto-matic. Once a substitute has been made, the pricesrequire an adjustment for the quality difference using,perhaps, the overlap method, imputation, or an explicitestimate based on production or option costs, or ahedonic regression as discussed in Chapter 7.

8.50 The management of the directed substitutioncan take a number of forms. It can comprise instructionsto price collectors who are informed of defined config-urations of a product, such as ‘‘high end’’, ‘‘main-stream’’, ‘‘economy’’, ‘‘entry level’’ and ‘‘other’’ (Lane,2001). Directions might also be given as to the propor-tions expected of items at these levels, say, 20 per centof the market should be ‘‘high end’’. Such informa-tion should be based on actual data or judgement ofspecialists. The configurations are revised, say, every sixmonths. What was ‘‘high end’’ at the start of the periodmay now be ‘‘entry level’’ and the price collectors willhave new configurations indicating what the desiredreplacements should look like. They are directed toparticular replacements. Alternatively, the price collectormight be responsible for the selection of replacements,either after discussion with store managers or, if anindication of the market share of the popular makes isgiven, with probability proportionate to size. There are,of course, other variants. In such markets, the desiredend effect is that replacement items likely to be repre-sentative of substantial sales are selected and that thisselection is made earlier rather than later. The point isnot to miss the birth of such items and to facilitatequality adjustment.

8.51 It is important to emphasize that, on the intro-duction of new versions of these evolutionary goods, aparticularly high price may be charged to take advantageof segments of the market willing to pay a premium forthe ‘‘newness’’ of the item. Alternatively, a particularlylow price may be charged to introduce the good to themarket in order to help gain acceptance. After a while,prices may be changed as the novelty of the item wearsoff or as it gains acceptance, or as competitors bring outimproved products. Directed substitution is importantin ensuring that the CPI captures the unusual priceincreases at the launch. It is also necessary in ensuringthat the coverage of items becomes more representa-tive. Although directed substitution allows for both, acaveat applies. If the overlap method is used, the item isintroduced on the assumption that the price differencebetween the old and new items equates to their qualitydifference. For example, if a new type of detergent isintroduced with a new, biological cleaning action, it maybe that the typical consumer is willing to pay a price of 10against the existing standard detergent’s price of 8. Withno explicit estimate of the additional usefulness or utilityto be gained from the biological action, the overlapmethod implicitly assumes it is worth 2. Yet there may bean introductory launch price of 8 and the price may laterincrease to 10. At the time of the overlap, the two priceswould be the same, there being no adjudged qualitydifference. In fact, the quality-adjusted price would

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

144

be falling; there is a quality difference of 2, but thiscannot be deduced by the statistical office. In general,therefore, when there is evidence of items being launchedat unusual prices and the overlap method is used, it isbetter to make the replacement later when the market hassettled.8.52 For revolutionary goods, substitution may not

be appropriate. First, they may not be able to be definedwithin the existing classification systems. Second, amajorpart of their uniqueness may be the manner in which theyare sold, which will require extending the sample to suchnew sales channels. Third, there will be no previous itemsto match such goods against in order to make a qualityadjustment to prices since, by definition, they are sub-stantially different from pre-existing goods. Finally,there is no weight to attach to the new outlets or items.8.53 The first need is to identify new goods. The

suggested contacts with market research companies,outlet managers and manufacturers, mentioned above inrelation to producing a supportive metadata system, arealso pertinent here. Once the new goods are identified,sample augmentation is appropriate for the introductionof revolutionary goods. It is necessary to bring the newrevolutionary goods into the sample, in addition to whatalready exists in the sample. This may involve extendingthe classification, the sample of outlets, and the item listwithin new or existing outlets. The choice of means bywhich the new goods are introduced is more problematic.8.54 Once two price quotes are available, it should be

possible to splice the new good onto an existing orobsolete one. This of course misses the impact of the newitem in its initial period. As discussed below, however,including such effects is not a trivial exercise. Considerthe linking of a good that is likely to be replaced in themarket by the new good. For example, a relatively newelectrical kitchen appliance might follow the price indexfor existing kitchen appliances up to the period of thelink, and then the price changes for the new good insubsequent periods. This would create a separate andadditional price series for a new good that augments thesample, as illustrated in Table 8.1. Item C is new inperiod 2 and has no base period weight. Its price changebetween periods 1 and 2, had it existed, is assumed tofollow the overall index for products A and B. For period3 onwards, a new linked price series is formed for C,which for period 3 is 101.40� 0.985=99.88 and forperiod 4 is 101.40� 0.98=99.37. New revised weights inperiod 2 show C’s weight to be 20 per cent of all theitems. The new index for period 3 is:

101:40� [0:8� (101:9=101:4)+0:2� (99:88=101:4)]

=0:8� 101:9+0:2� 99:88=101:50

and for period 4:

101:40� [0:8� (102:7=101:4)+0:2� (99:37=101:4)]

=0:8� 102:7+0:2� 99:37=102:05

8.55 If C were an evolutionary good replacing B,then there would be no need to introduce new weightsand no need to augment the sample. The revolutionarygood C has no weight in the base period; the splicing thus

requires a revision of the weights at the same time. Boththe selection of the series onto which the new item isspliced, and the product groups selected for the weightrevision, require some judgement. Items whose marketshare is likely to be affected by the introduction of thenew good should be selected. If the new good is likely tobe responsible for a significant share of expenditure, suchthat it will affect the weights of a broad class of productgroups, then there may be a case for a realignment of theoverall weighting procedure. Such seismic shifts can ofcourse occur, especially in the communications indus-tries, and for a wider range of markets when trade bar-riers are relaxed in less-developed economies or whenregulations are removed. The change in weights may alsobe required for disappearing goods no longer sold in aneconomy. In that case, the weights of these goods need tobe reassigned. As noted in Chapter 7, paragraphs 7.132to 7.158, chaining and hedonic indices may well beappropriate when there is a rapid turnover in such newand obsolete goods. Chaining is an extension of theabove procedure and can be used to introduce a newgood as soon as it is available for two successive periods.

8.56 Item augmentation may also be used for evo-lutionary goods that are likely to be responsible for asubstantial share of the market, while not displacing theexisting goods. Say, for example, that a country has alocal brewery and that a licensing agreement with aforeign brewery has led to the joint production of twobeers, under different brand names. Say the market sharefor beer from the brewery remains the same, but onesegment of the market now drinks foreign as opposed todomestic beer. Price collectors may be directed to aforced substitution of some of the sample of domesticbeers for foreign ones, the weight remaining the same.This would be similar to a quality adjustment using anon-comparable replacement, as discussed in Chapter 7,paragraphs 7.72 to 7.115. Alternatively, the sample maybe augmented since there is concern that a smaller sam-ple of domestic beers may now not be sufficientlyrepresentative. The augmentation process may be similarto that outlined in Table 8.1, with the new foreign beer Caccounting for 20 per cent of the market. If the advent offoreign beers displaced some of the alcoholic spiritsmarket, say, then the revision of weights would extendinto that product group. As noted in Chapter 7, para-graphs 7.125 to 7.158, chaining and hedonic indices maywell be appropriate when there is a rapid turnover insuch new and obsolete goods. With chaining, the goodneeds to be available for only two successive periods toallow for its introduction.

Table 8.1 Example of sample augmentation

Products Baseweight

Revisedweight

Period 1 Period 2 Period 3 Period 4

A 0.6 0.5 100.00 101.00 101.50 102.50B 0.4 0.3 100.00 102.00 102.50 103.00All items 0.8 100.00 101.40 101.90 102.70C 100.00 98.50 98.00Spliced C 0.2 100.00 101.40 99.88 99.37All items(revised)

100.00 101.40 101.50 102.05

ITEM SUBSTITUTION, SAMPLE SPACE AND NEW PRODUCTS

145

8.57 In some instances a directed replacement isrequired for evolutionary and revolutionary outlets.Forced augmentation of the sample of outlets may beimplemented so that new goods available only in specificoutlets are included. This is especially likely in the servicesector, where a new service is particular to specific out-lets, for example cyber cafes or online retailers. Theprocedures are similar to those described for items. Forexample, in the above example, instead of products A, Band C, consider C as a new outlet in addition to outlets Aand B. Some estimate would be required of its expectedsales share to form the revised weights.

8.58 The effect of a new outlet on the index dependson how it is included, as well as the nature of the marketand its reaction to the new outlet. First, if a new outletoffers some innovation which induces some consumersto shop there, there is an increase in usefulness or uti-lity. Because of imperfect knowledge about the newoutlet or different preferences of different segments ofthe market, the old outlet may not close down. There isno natural prompt for the new outlet to be introducedinto the CPI, as there is with the closure of an oldoutlet. The start-up of the new outlet may have beenapparent to the statistical office. If the new outlet isexpected to have substantial sales, it may augment thesample. It may be spliced onto the index in the mannerof item C above. Such a methodology would notinclude the welfare gain to consumers arising from theuniqueness of the outlet (Trajtenberg, 1989), since pricecomparisons are only being undertaken once it has beenintroduced. The initial welfare effect is between theperiod prior to its existence and the period of itsintroduction. Second, all other outlets might lower theirquality-adjusted prices to match those from the newoutlet. The fall in price and gain in usefulness or utilityarising from the new outlet’s technology would then becaptured by the CPI. Finally, outlets may appear thatoffer a wider range of options in terms of goods andservice, which is valued by consumers and is thereforean improvement in the standard of living via the gain inutility. There is nothing in current CPI methodologythat allows for the valuation of such gains (Shapiro andWilcox, 1997a).

Reservation prices8.59 Shapiro and Wilcox (1997a, p. 144) expressed

concerns over:

. . . the rare new item that delivers services radicallydifferent from anything previously available. For exam-ple, even the earliest generation of personal computersallowed consumers to undertake tasks that previouslywould have been prohibitively expensive. This problemcan be solved only by estimating the consumer surpluscreated by the introduction of each new item. Hausman(1994) [republished as Hausman (1997)] argues that thismust involve explicit modeling of the demand for eachnew item. Although explicit modeling of demand may beof dubious practicality for widespread implementation inthe CPI, strategic application in a few selected casesmight be worthwhile.

8.60 The technical means for such estimates isrecognized as being beyond the practical capabilities of a

statistical agency. More disturbing is that the argumentfor the inclusion of such effects extends from revolu-tionary new goods to the clutter of evolutionary itemssuch as new breakfast cereals. Appendix 8.2 providessome details of a generalized Laspeyres approach whichtakes account of substitution between new and oldmodels. Given the complexity of the estimation systemsinvolved, however, this manual envisages a pragmaticapproach which would initially exclude such effects.

Summary8.61 The need to consider the sample space of the

items selected by the index number methodology andnew goods arises out of a very real concern with thedynamic nature of modern markets. New goods andquality changes are far from being a new phenomenon.As Triplett (1999) has argued, it has not been demon-strated that the rate of new product development andintroduction is much higher now than in the past. It iscertainly accepted, however, that the number of newproducts and varieties is substantially greater thanbefore. Computer technology provides cost-effectivemeans for collecting and analysing very large sets ofdata. Chapter 6 considers the use of hand-held com-puters for data capture, and the availability of bar-codescanner data. The proper handling of such data requiresconsideration of aspects beyond those normally takeninto account in regard to the static intersection universewhich underscores matched samples. Appendix 8.1 tothis chapter provides an outline of these sampling issues.

8.62 The following important points should beborne in mind:

� Where nothing much in the quality and range of goodsavailable changes, use of the matched models meth-od presents many advantages. The matched modelsmethod compares like with like, from like outlets.

� Statistical metadata systems are needed to help iden-tify the product areas in which matching provides fewproblems, and to focus attention on those areas thatare problematic. They show how to collect and providethe information that will facilitate quality adjustment.They also allow for transparency in methods and theyfacilitate retraining.

� Where there is a very rapid turnover in items such thatserious sample depletion takes place quickly, replace-ments cannot be relied upon to make up the sample.Alternative mechanisms, which sample from or use thedouble universe of items in each period, are required.These include chained formulations and hedonicindices, as discussed in Chapter 7, paragraphs 7.125 to7.158.

� Some new goods can be treated as evolutionary andincorporated using non-comparable replacementswith associated quality adjustments. The timing ofthe replacement is critical for both the efficacy of thequality adjustment and the representativity of theindex.

� Instructions to price collectors on the selection of re-placements are important, for they too have a bearing

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

146

on the representativity of the index. The replacementof obsolete items with newly introduced items, in turn,leads to difficulties in undertaking quality adjustments,while their replacement with similar items leads toproblems with representativity.

� Sample rotation is an extreme form of the use ofreplacements, and is one mechanism for refreshingthe sample and thus increasing its representativity.Against this, however, is the possibility of biasarising from the implicit assumptions underlyingthe overlap procedure for quality adjustment notbeing met.

� Revolutionary goods may require the augmentation ofthe sample to make room for new price series and newweighting procedures. The classification of new goodsinto evolutionary goods and revolutionary goods has abearing on the strategy for their introduction, directedreplacement (substitution) and sample augmentation.

� The initial gain in consumer welfare arising from newitems and loss in welfare because items disappear arenot captured by either of these procedures. Econo-metric estimates of reservation prices provide anapproach that is theoretically appropriate, althoughproblematic in practice.

ITEM SUBSTITUTION, SAMPLE SPACE AND NEW PRODUCTS

147

Appendix 8.1 Appearance ordisappearance of productsor outlets

1. In previous chapters, it was generally assumed that thetarget quantity for estimation could be defined in terms of afixed set of products. Here we consider the complicationsarising from the fact that the products and outlets are con-tinually changing. The rate of change is rapid in many indus-tries. Sampling to estimate price changes is thus a dynamicrather than a static problem. Somehow, the prices of newproducts and the prices in new outlets have to be compared toold ones. Whatever methods and procedures are used in a priceindex to handle these dynamic changes, the effects of theseprocedures will always amount to an explicit or implicit esti-mation approach for this dynamic universe.

The representation of change ina price index

2. From a sample selection perspective, there are three waysof handling dynamic changes in an elementary aggregate universe(Dalen, 1998a), where varieties and outlets move in and out:

– by resampling the whole elementary aggregate at certainpoints in time;

– by a one-to-one replacement of one variety or outlet foranother;

– by adding and deleting single observation points (items inoutlets) within an index link.

Sample rotation3. Byresampling it ismeant that theoldsample is reconsidered

as a whole so as to make it representative of the universe in alater period. This does not necessarily mean that all or even mostsampling units have to be changed, only that a fresh look istaken at the representativity of the whole sample and changes areundertaken, as appropriate. The methods used for resamplingcould be any of those used for the initial sampling. In the case ofprobability sampling, every unit belonging to the universe in thelater period needs to have a non-zero probability, equal to itsrelative market share, of being included in the sample.

4. Resampling (or sample rotation) is traditionally combinedwith the overlap method outlined in Chapter 7, paragraphs 7.45to 7.52. It is a similar procedure to that used when combining twolinks in chain indices. The first period for which the new sample isused is also the last period for which the old sample is used.Thereby, price change estimation is always based on one sampleonly – the old sample up to the overlap period and the new samplefrom the overlap period onwards (see below). Resampling is theonly method that is fully able to maintain the representativity ofthe sample. Resources permitting, resampling should be under-taken frequently. The appropriate frequency, of course, dependson the rate of change in a particular product group. It also relieson the assumption that the price differences between the old andnew items are appropriate estimates of quality differences. At itsextreme, resampling amounts to drawing a new sample in eachperiod and comparing the average price between the samples,instead of the usual procedure of averaging price changes formatched samples. Although logical from the point of view ofrepresentativity, resampling in each period would aggravate theproblem of quality adjustment by its implicit procedure of qualityadjustment, and is thus not recommended.

Replacements5. A replacement can be defined as an individual successor

to a sampled product that has either disappeared completely

from the market or lost market share in either the market as awhole or a specific outlet. Criteria for selecting replacementsmay differ considerably. First, there is the question of when tomake the replacement. The usual practice is to do it eitherwhen an item disappears completely or when its share of thesales is reduced significantly. Another possible, but less-usedrule, would be to replace an item when another variety withinthe same group, or representative item definition, has becomelarger with regard to sales, even if the old variety is still sold insignificant quantities.

6. The second question is how to select the replacementitem. If the rule for initial selection was ‘‘most sold’’ or withprobability proportionate to (sales) size, then the replacementrule could follow the same selection rule. Alternatively, thereplacement could be that item which is ‘‘most like’’ the oldone. The advantage of the former rule is that it produces betterrepresentativity. The advantage of the ‘‘most like’’ rule is that,at least superficially, it might reduce the quality adjustmentproblem.

7. It is important to realize that, under current conditions,replacements cannot adequately represent new items that arecoming onto the market. This is because what triggers areplacement is not the appearance of something new, but thedisappearance or reduced importance of something old. Forexample, if the range of varieties in a certain group is increas-ing, sampling can only represent this increase directly from theset of new varieties, say by sample rotation.

Adding and deleting8. It is possible to add a new observation point into an

elementary aggregate within an index link. For example, if anew brand or model of a durable is introduced withoutreplacing any particular old model, it is desirable to add it tothe sample, starting from the time of its introduction. In orderto accommodate this new observation in the index system, areference price needs to be imputed. A practical way to do thisis to use the ratio of the price of the new item in the month ofits introduction to the average of all other items in the ele-mentary aggregate from the reference period to the month ofintroduction. In this way, the effect of the new item on theindex for months up to the introduction month will be neutral.

9. Similarly, an item that disappears could just be deletedfrom the sample without replacement. Price change can then becomputed over the remaining items. If no further action istaken, this means that the price change for the deleted item,which was measured up to the month prior to deletion, will bedisregarded from the month of deletion. This may or may notbe desirable, depending on the circumstances in the particularproduct group.

Formulating an operational targetin a dynamic universe

10. A rigorous approach to statistical estimation requiresan index estimation strategy, including both the operationaltarget of measurement and the sampling strategy (design andestimator) needed for estimating this target. This strategywould have to consist of the following components:

– a definition of the universe of transactions or observationpoints (usually a product variety in an outlet) in each of thetwo time periods between which we want to estimate pricechange;

– a list of all variables defined for these units. These variablesshould include prices and quantities (number of units sold ateach price), but also all relevant price-determining char-acteristics of the products (and possibly also of the outlets).This forms the price basis;

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

148

– the target algorithm (index formula) that combines thevalues of the defined variables for the observation points inthe defined universe into a single value;

– procedures used for initial sampling of items and outletsfrom the defined universe;

– procedures within the time span for replacing, samplerotation, adding or deleting observations;

– the estimation algorithm (index formula) applied to thesample with the purpose of minimizing the expected error ofthe sample estimate compared with the target algorithm. Inprinciple, the estimation needs to consider all the proceduresundertaken in replacement and sample rotation situations,including procedures for quality adjustment.

11. Because of its complexity, the rigorous strategy out-lined above is generally not used in practical index construc-tion, although the associated information (statistical metadata)system is discussed in paragraphs 8.23 to 8.31 above. A fewcomments on such possible strategies are made below.

A two-level aggregation system12. A starting point for discussing an objective of estimating

a price index from a sample drawn from a dynamic universe is atwo-level structuring of the universe of items and outlets thatare considered in the scope of a price index. These levels are:

� the aggregate level: at this level there is a fixed structure ofitem groups h ¼ 1, . . . ,H (or perhaps a fixed cross-structureof item groups by regions and outlet types) within an indexlink. In terms of updating the universe of items, new goodsand services would be defined as new groups at the aggre-gate level and moved into the index only in connection witha new index link;

� the elementary level: at this level the aim is to capture theproperties of a changing universe in the index by comparingnew and old items. The micro-comparison from period s toperiod t must be defined so that new products or outletsenter the market and old products or outlets disappear fromthe market.

13. The common starting point for the three alternativeapproaches at the elementary level presented here is a basketindex from period s to period t at the aggregate level:

Ist=

Ph

QhPthP

h

QhPsh

=P

hWshIst

h ,

where Wsh=

QhPshP

h

QhPsh

and Isth =

Pth

Psh

: (A8.1)

The quantities, Qh, are for h=1 . . . H item groups from anyperiod, or functions of quantities from several periods, forexample, a symmetric average of the base and current periods sand t. Special cases of such a basket index are the Laspeyres(Qh=Qs

h), Paasche (Qh=Qth), Edgeworth (Qh=Qs

h+Qth ) and

Walsh (Qh=[QshQt

h]1=2) price indices outlined in Chapters 15 to

17. Alternative formulations for an elementary-level estimationstrategy now enter in the definition of Ist

h . As a further commonstarting point, the set of items or outlets belonging to h in periodu (= s or t) is defined as Ou

h. The concept of an observation pointis introduced, usually a tightly specified item in a specific outlet.For each observation point j 2Ou

h, there is a price puj and

a quantity sold quj . There are now three possibilities for defining

the operational target.

The intersection universe14. The elementary index is defined over the intersection

universe, that is, only over observation points existing in both s

and t. This index may also be called the identical units index. Itis equivalent to starting out with the observation pointsexisting in s and then dropping (deleting) missing or dis-appearing points. An example of such an index is:

Isth =

Pj 2 Os

h\Oth

qjptjP

j 2Osh\O

th

qjpsj

(A8.2)

The intersection universe decreases successively over time, asfewer matches are found for each long-run comparison betweens and t, s and t+1, s and t+2 etc., until it eventually becomesempty. An attraction of the intersection universe is that thereare, by definition, no replacements involved, and thus, nor-mally, no quality adjustments. If the identical units index iscombined with a short index link, followed by resampling fromthe universe in a later period, sampling from the intersectionuniverse is a perfectly reasonable strategy, as long as theassumption implicit in the overlap procedure, that the pricedifferences at that point in time reflect the quality differences, isvalid.

The double universe15. The polar opposite approach to the intersection uni-

verse is to consider Psh and Pt

h as average prices defined overtwo separately defined universes in the two periods. A doubleuniverse could then be considered as the operational target ofmeasurement: one universe in period s and another in period t.This seems to be a natural way of defining the target, sinceboth time periods should be of equal status and all productsexisting in either of them should be taken into account. Thedifficulty with this approach is that the two universes arerarely comparable in terms of quality. Some kind of adjust-ment for average quality change would need to be broughtinto the index. The natural definition of the average pricesinvolved in this approach is based on unit values. This wouldlead to the following definition of a quality-adjusted unit valueindex:

Isth =

�PPth

�PPshgst

h

,

where �PPth=

Pj 2Ot

h

qtjp

tjP

j 2Oth

qtj

and �PPsh=

Pj 2Os

h

qsj p

sjP

j 2Osh

qsj

(A8:3Þ

where gsth is the average quality change in h (also interpretable

as a quality index), which of course needs further definition. Forexample, g st

h could be thought of as a hedonic adjustmentprocedure, where characteristics are held constant. Equation(A8.3) was discussed in Chapter 7, paragraphs 7.142–7.149, aspart of Laspeyres, Paasche, Fisher and Tornqvist indices (asopposed to unit value ones), in a form which includes explicithedonic quality adjustments, g st

h . This operational target isattractive for products where the rate of turnover of varieties isvery fast, but where average quality changes only slowly orwhere reliable estimates of quality changes can be made. Thecommonly used representative-item method is not really com-patible with a double universe target. It implicitly focuses onpre-selected primary sampling units that are used for bothperiod s and t.

The replacement universe16. Neither sampling from the intersection universe nor

from the double universe bears a close resemblance to usualpractices for constructing price indices. The most common

ITEM SUBSTITUTION, SAMPLE SPACE AND NEW PRODUCTS

149

sampling method used in practice – the representative-itemmethod combined with one-to-one replacements – needs arationalization in terms of operational targets which differsfrom these alternatives. Such a rationalization of samplingfrom a replacement universe is considered below.

17. For each j 2Osh and j 2=Ot

h we define replacement itemsaj 2Ot

h whose price replaces that of j in the formula. Obviously,for j 2Os

h and j 2Oth, aj=j. In addition to a replacement, a

quality change from j to aj is included. This gives rise to aquality adjustment factor gj, interpreted as the factor withwhich p s

j must be multiplied for the consumer to be indifferentbetween consuming items j and aj at prices ps

j and ptaj .

Isth =

Pj 2Os

h

qjptajP

j 2Osh

qjpsj gj: (A8.4)

18. This step towards an operational use of the formularequires, first, a definition of gj, which is possible using ahedonic regression as described in Chapter 7, paragraphs7.132 to 7.152. Second, there is a need to define aj. A naturalprocedure is to use a dissimilarity function from j to aj. Thenotation d( j, aj) is introduced for this function. The commonprocedure of choosing the most similar item in cases ofreplacement now corresponds to minimizing the dissimilarityfunction. Some further specifications nevertheless need to bemade. When is the replacement defined to take place? Inpractice, this ought to be done when the first chosen variety is

no longer representative. Mathematically, this could bedefined as follows: observation point j should be replaced inthe first period in which qt

j < cqsj , where c is a suitably chosen

constant between 0 and 1 (a modification being requiredfor seasonal items). The choice of replacement point wouldthen be governed by a rule such as: aj should be chosen sothat d( j, aj) is minimized for j. Since some priority should begiven to observation points that are important in terms ofquantities or values, that definition can be modified tobecome, for example: aj should be chosen so that d(j; aj)=qt

aj isminimized for j. Other rules for the choice of replacementpoint or function to be minimized can of course be chosen.

19. The dissimilarity function needs to be specified; it maydepend on the item group h. In general, it must be some kindof metric defined on the set of characteristics of the productand outlet in question. For example, priority could be given todissimilarity either to ‘‘same outlet’’ or ‘‘same product’’, con-cepts which could easily be worked into such a metric. A moretroublesome concern is the inclusion of as many new points inOt

h as possible in the index definition, in order to ensure thatthe sample is representative. As the above definitions nowstand, the same new point could replace many predecessors,whereas there might be many new points which will not besampled unless there is a need for a replacement. This short-coming of the replacement universe is an inherent trait in thereplacement method as such. The replacement method isdesigned to maintain only the representativity of the oldsample, not that of the new sample.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

150

Appendix 8.2 New goodsand substitution

1. An alternative approach to estimating the effect ofintroducing new goods is to see new goods as a special case ofsubstitution. In each period a consumer, faced with a set ofprices, decides what to consume. The relative sales of the dif-ferent items sold may change over time. Consumers may decideto consume less of one existing item and more of anotherexisting one, or substitute consumption of an existing old itemby a new one not previously available, or discontinue con-sumption of an existing item and substitute it by consumptionof an existing or new one. Such changes are generally promptedby changes in relative prices. In many cases the ‘‘decision’’ ofthe consumer is tied to that of the producer or retailer, as itemsare no longer produced or sold so as to make way for new ones.Such substitutions between items apply as much to radicallynew goods as to new models of existing goods. In economictheory, the elasticity of substitution, denoted as s, is a measureof the change in the quantity of, say, item i relative to item j,that would arise from a unit change in the price of item i relativeto item j. A value of zero would imply that a change in pricewould lead to no substitution between the consumption ofitems and s>1 implies that the change in expenditure arising asa result of substituting items is positive: it is worth switching.

2. There is an intuition here that, if s is known, and theextent to which substitutions occur in terms of their expendi-ture shares is also known, then estimates of the underlyingprice change that prompted the substitution can be derived.This applies as much to substitution between existing items asto substitution between existing, discontinued and new ones.The framework for operationalizing this institution for CPIuse is proposed by Shapiro and Wilcox (1997b) – see alsoLloyd (1975) and Moulton (1996a) – whereby the usual Las-peyres formulation is generalized to include the (demand)elasticity of substitution:

Pn 2 0,t

w0pit

pi0

� �1�s" #1=ð1�sÞ(A8.5)

where w0 are expenditure shares in the base period and thesummation is over matched items available in both periods. Thecorrection, using s, incorporates a substitution effect into thebasic Laspeyres formula. If s=0, the formula is the traditionalLaspeyres one. As s! 1, the formula tends towards a base-period weighted geometric mean. To use this formulation togeneralize across the items in the summation, the restrictionmust apply that for any pair of items, the elasticity of sub-stitution must be the same. The elasticity of substitution mustalso be the same over time. Such forms are referred to as con-stant elasticity of substitution (CES) functional relationships.

3. Feenstra (1994), Feenstra and Shiells (1997) and Balk(2000b) have extended the substitution to discontinued and newitems. The advantage of equation (A8.5) is that, given an esti-mate of s, a cost of living index which includes an estimate ofsubstitution effects can be measured in real time. The incor-poration of the effects of new and discontinued items followsdirectly from this. Alternative frameworks for including sub-stitution effects (discussed in Chapter 17) require expendituredata for the base and current periods.

4. To extend the framework to new items, it is necessary toknow how expenditures shift between new, existing and dis-

continued items. Let lt be the expenditure share of matchedexisting items out of the total in period t. The total includesexisting and new items, so 1� lt is the share of new items inperiod t. Similarly, 1� l0 is the expenditure share of old, dis-continued items in period 0. The generalized Laspeyres index,which includes substitution between existing and old and newitems, is given by:

lt

l0

� �1=ðs�1Þ Pn 2 0,t

w0pit

pi0

� �1�s" #1=ð1�sÞ(A8.6)

Like the usual Laspeyres index, it requires only the pricerelatives, the base period weights, the ratio of expenditureshares and an estimate of the elasticity of substitution. It canbe derived in a number of alternative forms, including gen-eralized, Paasche, Fisher or Sato–Vartia indices.

5. While there is an intuition behind the above formula, itsformal correspondence to an index of consumer prices definedin economic theory is given by Balk (2000b). De Haan (2001)shows how the Fisher equivalent could be derived from adecomposition of a Fisher index when there are new and dis-appearing goods. The derivations show how the frameworkrequires that s>1, a factor prompting Balk (2000b) to arguefor its use for lower-level index aggregation, where this is morelikely. The remaining problems are the estimation of s, theavailability of data on current expenditure shares, and thevalidity of the implied constant s. There are also some con-ceptual issues. Increases in utility are regarded as havingresulted from increases in the desirability of the items includedin the above aggregation. If such items improve, then utilityincreases. Yet there are other goods outside the aggregation orsystem of demand equations. Deterioration in such goods willlead to increases in the desirability of the included items anddecreases in utility. For example, if a consumer switches toprivate transport as a result of a deterioration in public trans-port, this should not be measured as a welfare gain resultingfrom better private transport, even though the expenditureflows in equation (A8.6) shift that way (Nevo, 2001).

6. The direct estimation of s requires considerable econo-metric expertise. This puts it outside the routine constructionof index numbers (see Hausman, 1997). Balk (2000b) showshow an alternative numerical routine might work. De Haan(2001) used scanner data to apply the methodology to a gen-eralized Fisher index. He applied Balk’s routine to nine pro-duct groups, using data from the Netherlands CPI, and foundvalues of s that exceeded unity. He advised the use of chainedindices to maximize the matching of ongoing items, a principlediscussed in Chapter 7, paragraphs 7.153 to 7.158. De Haan(2001) found major discrepancies between a generalized andordinary Fisher index for at least six of the products, arguingfor the need to incorporate the effects of new goods (see alsoOpperdoes, 2001). He also demonstrates how sensitive theprocedure is to the selection of s: for a share in currentexpenditure for new items of 4.8 per cent, and s=1.2, aPaasche-type index which includes new goods would be 93 percent below the Paasche price change for ongoing goods only.For s=5.0 and the same expenditure share, the discrepancyfalls to 34.1 per cent. For very large values, say s>100, the twoindices would be relatively close. In such cases, the goods arealmost identical, being near-perfectly substitutable; a switch toa new good would have little effect, the new and existing goodshaving similar prices.

ITEM SUBSTITUTION, SAMPLE SPACE AND NEW PRODUCTS

151

9CALCULATING CONSUMER PRICEINDICES IN PRACTICE

Introduction9.1 The purpose of this chapter is to provide a general

description of the ways in which consumer price indices(CPIs) are calculated in practice. The methods used indifferent countries are not exactly the same, but they havemuch in common. There is clearly interest from bothcompilers and users of CPIs in knowing how most sta-tistical offices actually set about calculating their CPIs.9.2 As a result of the greater insights into the prop-

erties and behaviour of price indices that have beenachieved in recent years, it is now recognized that sometraditional methods may not necessarily be optimal froma conceptual and theoretical viewpoint. Concerns havealso been voiced in a number of countries about possiblebiases that may be affecting CPIs. These issues andconcerns need to be considered in this manual. Of course,the methods used to compile CPIs are inevitably con-strained by the resources available, not merely for col-lecting and processing prices, but also for gathering theexpenditure data needed for weighting purposes. In somecountries, the methods used may be severely constrainedby lack of resources.9.3 The calculation of CPIs usually proceeds in two

stages. First, price indices are estimated for the elemen-tary expenditure aggregates, or simply elementary aggre-gates. Then these elementary price indices are averagedto obtain higher-level indices using the relative values ofthe elementary expenditure aggregates as weights. Thischapter starts by explaining how the elementary aggre-gates are constructed, and what economic and statisticalcriteria need to be taken into consideration in definingthe aggregates. The index number formulae most com-monly used to calculate the elementary indices are thenpresented, and their properties and behaviour illustratedusing numerical examples. The pros and cons of thevarious formulae are considered, together with somealternative formulae that might be used instead. Theproblems created by disappearing and new items are alsoexplained, as well as the different ways of imputingvalues for missing prices.9.4 The second part of the chapter is concerned with

the calculation of higher-level indices. The focus is on theongoing production of a monthly price index in whichthe elementary price indices are averaged, or aggregated,to obtain higher-level indices. Price-updating of weights,chain linking and reweighting are discussed, with ex-amples being provided. The problems associated withintroduction of new elementary price indices and newhigher-level indices into the CPI are also dealt with. It isexplained how it is possible to decompose the change inthe overall index into its component parts. Finally, the

possibility of using some alternative and rather morecomplex index formulae is considered.

9.5 The chapter concludes with a section on dataediting procedures, as these are an integral part of theprocess of compiling CPIs. It is essential to ensure thatthe right data are entered into the various formulae.There may be errors resulting from the inclusion ofincorrect data or from entering correct data inappro-priately, and errors resulting from the exclusion of cor-rect data that are mistakenly believed to be wrong. Thesection examines data editing procedures which try tominimize both types of errors.

The calculation of price indicesfor elementary aggregates

9.6 CPIs are typically calculated in two steps. In thefirst step, the elementary price indices for the elementaryaggregates are calculated. In the second step, higher-level indices are calculated by averaging the elementaryprice indices. The elementary aggregates and their priceindices are the basic building blocks of the CPI.

Construction of elementary aggregates9.7 Elementary aggregates are groups of relatively

homogeneous goods and services. They may cover thewhole country or separate regions within the country.Likewise, elementary aggregates may be distinguishedfor different types of outlets. The nature of the elemen-tary aggregates depends on circumstances and the avail-ability of information. Elementary aggregates maytherefore be defined differently in different countries.Some key points, however, should be noted:

� Elementary aggregates should consist of groups ofgoods or services that are as similar as possible, andpreferably fairly homogeneous.

� They should also consist of items that may beexpected to have similar price movements. Theobjective should be to try to minimize the dispersionof price movements within the aggregate.

� The elementary aggregates should be appropriate toserve as strata for sampling purposes in the light ofthe sampling regime planned for the data collection.

9.8 Each elementary aggregate, whether relatingto the whole country or an individual region or groupof outlets, will typically contain a very large number ofindividual goods or services, or items. In practice, only asmall number can be selected for pricing. When selectingthe items, the following considerations need to be taken

153

into account:

� The items selected should be ones for which pricemovements are believed to be representative of all theproducts within the elementary aggregate.

� The number of items within each elementary aggregatefor which prices are collected should be large enoughfor the estimated price index to be statistically reliable.The minimum number required will vary betweenelementary aggregates depending on the nature of theproducts and their price behaviour.

� The object is to try to track the price of the same itemover time for as long as possible, or as long as the itemcontinues to be representative. The items selectedshould therefore be ones that are expected to remainon the market for some time, so that like can becompared with like.

9.9 The aggregation structure. The aggregationstructure for a CPI is illustrated in Figure 9.1. Using aclassification of consumers’ expenditures such as theClassification of Individual Consumption according toPurpose (COICOP), the entire set of consumption goodsand services covered by the overall CPI can be dividedinto groups, such as ‘‘food and non-alcoholic beverages’’.Each group is further divided into classes, such as ‘‘food’’.For CPI purposes, each class can then be further dividedinto more homogeneous sub-classes, such as ‘‘rice’’. Thesub-classes are the equivalent of the basic headings used inthe International Comparison Program (ICP), whichcalculates purchasing power parities (PPPs) betweencountries. Finally, the sub-class may be further sub-divided to obtain the elementary aggregates, by dividingaccording to region or type of outlet, as in Figure 9.1. Insome cases, a particular sub-class cannot be, or does notneed to be, further subdivided, in which case the sub-classbecomes the elementary aggregate. Within each elemen-tary aggregate, one or more items are selected to representall the items in the elementary aggregate. For example, theelementary aggregate consisting of rice sold in super-markets in the northern region covers all types of rice,from which parboiled white rice and brown rice with over50 per cent broken grains are selected as representativeitems. Of course, more representative items might beselected in practice. Finally, for each representative item,a number of specific products can be selected for pricecollection, such as particular brands of parboiled rice.Again, the number of sampled products selectedmay varydepending on the nature of the representative product.

9.10 Methods used to calculate the elementary indi-ces from the individual price observations are discussedbelow. Working upwards from the elementary priceindices, all indices above the elementary aggregate levelare higher-level indices that can be calculated from theelementary price indices using the elementary expendi-ture aggregates as weights. The aggregation structure isconsistent, so that the weight at each level above theelementary aggregate is always equal to the sum of itscomponents. The price index at each higher level ofaggregation can be calculated on the basis of the weightsand price indices for its components, that is, the lower-level or elementary indices. The individual elementaryprice indices are not necessarily sufficiently reliable to be

published separately, but they remain the basic buildingblocks of all higher-level indices.

9.11 Weights within elementary aggregates. In mostcases, the price indices for elementary aggregates arecalculatedwithout the use of explicit expenditure weights.Whenever possible, however, weights should be used thatreflect the relative importance of the sampled items, evenif the weights are only approximate. Often, the elemen-tary aggregate is simply the lowest level at which reliableweighting information is available. In this case, the ele-mentary index has to be calculated as an unweightedaverage of the prices of which it consists. Even in thiscase, however, it should be noted that when the items areselected with probabilities proportional to the size ofsome relevant variable such as sales, weights are impli-citly introduced by the sampling selection procedure.

9.12 For certain elementary aggregates, informationabout sales of particular items, market shares and re-gional weights may be used as explicit weights within anelementary aggregate. Weights within elementary aggre-gates may be updated independently and possibly moreoften than the elementary aggregates themselves (whichserve as weights for the higher-level indices).

9.13 For example, assume that the number of sup-pliers of a certain product such as fuel for cars is limited.The market shares of the suppliers may be known frombusiness survey statistics and can be used as weights inthe calculation of an elementary aggregate price indexfor car fuel. Alternatively, prices for water may be col-lected from a number of local water supply serviceswhere the population in each local region is known. Therelative size of the population in each region may then beused as a proxy for the relative consumption expendi-tures to weight the price in each region to obtain theelementary aggregate price index for water.

9.14 A special situation occurs in the case of tariffprices. A tariff is a list of prices for the purchase of aparticular kind of good or service under different termsand conditions. One example is electricity, where one priceis charged during daytime while a lower price is charged atnight. Similarly, a telephone company may charge a lowerprice for a call at the weekend than in the rest of the week.Another example may be bus tickets sold at one price toordinary passengers and at lower prices to children or oldage pensioners. In such cases, it is appropriate to assignweights to the different tariffs or prices in order to calcu-late the price index for the elementary aggregate.

9.15 The increasing use of electronic points of sale inmany countries, in which both prices and quantities arescanned as the purchases are made, means that valuablenew sources of information may become increasinglyavailable to statistical offices. This could lead to sig-nificant changes in the ways in which price data arecollected and processed for CPI purposes. The treatmentof scanner data is examined in Chapters 7, 8 and 21.

Construction of elementary price indices9.16 An elementary price index is the price index for

an elementary aggregate. Various different methods andformulae may be used to calculate elementary priceindices. The methods that have been most commonly

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

154

used are illustrated by means of a numerical example inTable 9.1. In the example, it is assumed that prices arecollected for four items within an elementary aggregate.The quality of each item remains unchanged over time,so the month-to-month changes compare like with like.It is assumed initially that prices are collected for all fouritems in every month covered, so that there is a completeset of prices. There are no disappearing items, no missingprices and no replacement items. This is quite a strongassumption because many of the problems encounteredin practice are attributable to breaks in the continuity of

the price series for the individual items for one reason oranother. The treatment of disappearing and replacementitems is taken up later. It is also assumed that there areno explicit weights available.

9.17 Three widely used formulae that have been, orstill are, in use by statistical offices to calculate elemen-tary price indices are illustrated in Table 9.1. It shouldbe noted, however, that these are not the only possibili-ties and some alternative formulae are considered later.

� The first is the Carli index for i=1,...., n items. It isdefined as the simple, or unweighted, arithmetic mean

OVERALL CPIproducts

GROUPFood and non-

alcoholic beverages GROUP

Alcoholic beverages and tobacco OTHER GROUPS

CLASSBread and cereals

CLASSMeat OTHER CLASSES

SUB-CLASSRice

SUB-CLASSBread

OTHER SUB-CLASSES

Sold in northernregion Sold in southern region Sold in other

regions

ELEMENTARYAGGREGATE

Rice sold in northernsupermarkets

ELEMENTARY AGGREGATE

Rice sold in other northernoutlets

REPRESENTATIVEPRODUCT

Parboiled long-grainwhite rice

REPRESENTATIVE PRODUCT

Brown rice: over 50 per centbroken rice

SAMPLED PRODUCT

Brand A

SAMPLED PRODUCTBrand B

Figure 9.1 Typical aggregation structure of a consumer price index (CPI)

CALCULATING CONSUMER PRICE INDICES IN PRACTICE

155

of the price relatives, or price ratios, for the twoperiods, 0 and t, to be compared:

I0:tC =

1

n

P ptip0i

� �(9.1)

� The second is the Dutot index, defined as the ratio ofthe unweighted arithmetic mean prices:

I0:tD =

1n

Ppti

1n

Pp0i

(9.2)

� The third is the Jevons index, defined as theunweighted geometric mean of the price relative orratio which is identical to the ratio of the unweightedgeometric mean prices:

I0:tJ =

Q ptip0i

� �1=n=

Q( pti)

1=nQ( p0i )

1=n(9.3)

9.18 The properties of the three indices are examinedand explained in some detail in Chapter 20. Here, thepurpose is to illustrate how they perform in practice, tocompare the results obtained by using the different for-mulae and to summarize their strengths and weaknesses.

9.19 Each month-to-month index shows the changein the index from one month to the next. The chainedmonthly indices link together these month-to-monthchanges by successive multiplication. The direct indicescompare the prices in each successive month directlywith those of the reference month, January. By simpleinspection of the various indices, it is clear that the

choice of formula and method can make a substantialdifference to the results obtained. Some results arestriking, in particular the large difference between thechained Carli index for July and each of the directindices for July, including the direct Carli.

9.20 The properties and behaviour of the differentindices are summarized in the following paragraphs (seealso Chapter 20). First, the differences between theresults obtained by using the different formulae tend toincrease as the variance of the price relatives, or ratios,increases. The greater the dispersion of the price move-ments, the more critical the choice of index formula,and method, becomes. If the elementary aggregates aredefined in such a way that the price movements withinthe aggregate are minimized, the results obtained becomeless sensitive to the choice of formula and method.

9.21 Certain features displayed by the data in Table9.1 are systematic and predictable; they follow from themathematical properties of the indices. For example, it iswell known that an arithmetic mean is always greaterthan, or equal to, the corresponding geometric mean, theequality holding only in the trivial case in which thenumbers being averaged are all the same. The direct Carliindices are therefore all greater than the Jevons indices,except in May and July when the four price relativesbased on January are all equal. In general, the Dutot maybe greater or less than the Jevons, but tends to be lessthan the Carli.

9.22 One general property of geometric meansshould be noted when using the Jevons index. If any oneobservation out of a set of observations is zero, theirgeometric mean is zero, whatever the values of the other

Table 9.1 Calculation of price indices for an elementary aggregate1

January February March April May June July

PricesItem A 6.00 6.00 7.00 6.00 6.00 6.00 6.60Item B 7.00 7.00 6.00 7.00 7.00 7.20 7.70Item C 2.00 3.00 4.00 5.00 2.00 3.00 2.20Item D 5.00 5.00 5.00 4.00 5.00 5.00 5.50Arithmetic mean prices 5.00 5.25 5.50 5.50 5.00 5.30 5.50Geometric mean prices 4.53 5.01 5.38 5.38 4.53 5.05 4.98

Month-to-month price ratiosItem A 1.00 1.00 1.17 0.86 1.00 1.00 1.10Item B 1.00 1.00 0.86 1.17 1.00 1.03 1.07Item C 1.00 1.50 1.33 1.25 0.40 1.50 0.73Item D 1.00 1.00 1.00 0.80 1.25 1.00 1.10

Current-to-reference-month (January) price ratiosItem A 1.00 1.00 1.17 1.00 1.00 1.00 1.10Item B 1.00 1.00 0.86 1.00 1.00 1.03 1.10Item C 1.00 1.50 2.00 2.50 1.00 1.50 1.10Item D 1.00 1.00 1.00 0.80 1.00 1.00 1.10Carli index – the arithmetic mean of price ratiosMonth-to-month index 100.00 112.50 108.93 101.85 91.25 113.21 100.07Chained month-to-month index 100.00 112.50 122.54 124.81 113.89 128.93 129.02Direct index on January 100.00 112.50 125.60 132.50 100.00 113.21 110.00Dutot index – the ratio of arithmetic mean pricesMonth-to-month index 100.00 105.00 104.76 100.00 90.91 106.00 103.77Chained month-to-month index 100.00 105.00 110.00 110.00 100.00 106.00 110.00Direct index on January 100.00 105.00 110.00 110.00 100.00 106.00 110.00Jevons index – the ratio of geometric mean prices = geometric mean of price ratiosMonth-to-month index 100.00 110.67 107.46 100.00 84.09 111.45 98.70Chained month-to-month index 100.00 110.67 118.92 118.92 100.00 111.45 110.00Direct index on January 100.00 110.67 118.92 118.92 100.00 111.45 110.00

1 All price indices have been calculated using unrounded figures.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

156

observations. The Jevons index is sensitive to extremefalls in prices and it may be necessary to impose upperand lower bounds on the individual price ratios of say 10and 0.1, respectively, when using the Jevons. Of course,extreme observations often result from errors of one kindor another, so extreme price movements should becarefully checked anyway.9.23 Another important property of the indices

illustrated in Table 9.1 is that the Dutot and the Jevonsindices are transitive, whereas the Carli is not. Transi-tivity means that the chained monthly indices are iden-tical to the corresponding direct indices. This property isimportant in practice, because many elementary priceindices are in fact calculated as chain indices which linktogether the month-on-month indices. The intransitivityof the Carli index is illustrated dramatically in Table 9.1when each of the four individual prices in May returnsto the same level as it was in January, but the chain Carliregisters an increase of almost 14 per cent over January.Similarly, in July, although each individual price isexactly 10 per cent higher than in January, the chainCarli registers an increase of 29 per cent. These resultswould be regarded as perverse and unacceptable in thecase of a direct index, but even in the case of a chainindex the results seems so intuitively unreasonable as toundermine the credibility of the chain Carli. The pricechanges between March and April illustrate the effectsof ‘‘price bouncing’’ in which the same four prices areobserved in both periods but they are switched betweenthe different items. The monthly Carli index fromMarchto April increases whereas both the Dutot and theJevons indices are unchanged.9.24 The message emerging from this brief illustra-

tion of the behaviour of just three possible formulae isthat different index numbers and methods can deliververy different results. Index compilers have to familiarizethemselves with the interrelationships between the vari-ous formulae at their disposal for the calculation ofthe elementary price indices so that they are aware ofthe implications of choosing one formula rather thananother. Knowledge of these interrelationships is never-theless not sufficient to determine which formula shouldbe used, even though it makes it possible to make a moreinformed and reasoned choice. It is necessary to appealto other criteria in order to settle the choice of formula.There are two main approaches that may be used, theaxiomatic and the economic approaches.

Axiomatic approach to elementaryprice indices9.25 As explained in Chapters 16 and 20, one way in

which to decide upon an appropriate index formula is torequire it to satisfy certain specified axioms or tests. Thetests throw light on the properties possessed by differentkinds of indices, some of which may not be intuitivelyobvious. Four basic tests will be cited here to illustratethe axiomatic approach:

� Proportionality test – if all prices are l times the pricesin the price reference period (January in the example),the index should equal l. The data for July, whenevery price is 10 per cent higher than in January, show

that all three direct indices satisfy this test. A specialcase of this test is the identity test, which requires thatif the price of every item is the same as in the referenceperiod, the index should be equal to unity (as in Mayin the example).

� Changes in the units of measurement test (commen-surability test) – the price index should not change ifthe quantity units in which the products are measuredare changed (for example, if the prices are expressedper litre rather than per pint). The Dutot index failsthis test, as explained below, but the Carli and Jevonsindices satisfy the test.

� Time reversal test – if all the data for the two periodsare interchanged, then the resulting price index shouldequal the reciprocal of the original price index. TheCarli index fails this test, but the Dutot and the Jevonsindices both satisfy the test. The failure of the Carli tosatisfy the test is not immediately obvious from theexample, but can easily be verified by interchangingthe prices in January and April, for example, in whichcase the backwards Carli for January based on Aprilis equal to 91.3 whereas the reciprocal of the forwardsCarli is 1/132.5 or 75.5.

� Transitivity test – the chain index between two periodsshould equal the direct index between the same twoperiods. It can be seen from the example that theJevons and the Dutot indices both satisfy this test,whereas the Carli index does not. For example,although the prices in May have returned to the samelevels as in January, the chain Carli registers 113.9.This illustrates the fact that the Carli may have a sig-nificant built-in upward bias.

9.26 Many other axioms or tests can be devised, butthe above are sufficient to illustrate the approach andalso to throw light on some important features of theelementary indices under consideration here.

9.27 The sets of products covered by elementaryaggregates are meant to be as homogeneous as possible.If they are not fairly homogeneous, the failure of theDutot index to satisfy the units of measurement orcommensurablity test can be a serious disadvantage.Although defined as the ratio of the unweighted arith-metic average prices, the Dutot index may also beinterpreted as a weighted arithmetic average of the priceratios in which each ratio is weighted by its price in thebase period. This can be seen by rewriting formula (9.2)above as

I0:tD =

1n

Pp0i ( p

ti=p

0i )

1n

Pp0i

However, if the products are not homogeneous, therelative prices of the different items may depend quitearbitrarily on the quantity units in which they are mea-sured.

9.28 Consider, for example, salt and pepper, whichare found within the same sub-class of COICOP. Sup-pose the unit of measurement for pepper is changedfrom grams to ounces, while leaving the units in whichsalt is measured (say kilos) unchanged. As an ounce ofpepper is equal to 28.35 grams, the ‘‘price’’ of pepper

CALCULATING CONSUMER PRICE INDICES IN PRACTICE

157

increases by over 28 times, which effectively increasesthe weight given to pepper in the Dutot index by over 28times. The price of pepper relative to salt is inherentlyarbitrary, depending entirely on the choice of units inwhich to measure the two goods. In general, when thereare different kinds of products within the elementaryaggregate, the Dutot index is unacceptable conceptually.

9.29 The Dutot index is acceptable only when the setof items covered is homogeneous, or at least nearlyhomogeneous. For example, it may be acceptable for aset of apple prices even though the apples may be ofdifferent varieties, but not for the prices of a number ofdifferent kinds of fruits, such as apples, pineapples andbananas, some of which may be much more expen-sive per item or per kilo than others. Even when theitems are fairly homogeneous and measured in the sameunits, the Dutot’s implicit weights may still not besatisfactory. More weight is given to the price changesfor the more expensive items, but in practice they maywell account for only small shares of the total expendi-ture within the aggregate. Consumers are unlikely tobuy items at high prices if the same items are available atlower prices.

9.30 It may be concluded that from an axiomaicviewpoint, both the Carli and the Dutot indices, althoughthey have been, and still are, widely used by statisticaloffices, have serious disadvantages. The Carli indexfails the time reversal and transitivity tests. In principle,it should not matter whether we choose to measure pricechanges forwards or backwards in time. We wouldexpect the same answer, but this is not the case for theCarli. Chained Carli indices may be subject to a sig-nificant upward bias. The Dutot index is meaningful fora set of homogeneous items but becomes increasinglyarbitrary as the set of products becomes more diverse.On the other hand, the Jevons index satisfies all the testslisted above and also emerges as the preferred indexwhen the set of tests is enlarged, as shown in Chapter 20.From an axiomatic point of view, the Jevons index isclearly the index with the best properties, even though itmay not have been used much until recently. Thereseems to be an increasing tendency for statistical officesto switch from using Carli or Dutot indices to the Jevonsindex.

Economic approach to elementaryprice indices

9.31 In the economic approach, the objective is toestimate an economic index – that is, a cost of living indexfor the elementary aggregate (see Chapter 20). The itemsfor which prices are collected are treated as if they con-stituted a basket of goods and services purchased byconsumers, from which the consumers derive utility. Acost of living index measures the minimum amount bywhich consumers would have to change their expendi-tures in order to keep their utility level unchanged,allowing consumers to make substitutions between theitems in response to changes in the relative prices ofitems. In the absence of information about quantities orexpenditures within an elementary aggregate, the indexcan only be estimated when certain special conditions areassumed to prevail.

9.32 There are two special cases of some interest. Thefirst case is when consumers continue to consumethe same relative quantities whatever the relativeprices. Consumers prefer not to make any substitutionsin reponse to changes in relative prices. The cross-elasticities of demand are zero. The underlying pre-ferences are described in the economics literature as‘‘Leontief ’’. With these preferences, a Laspeyres indexwould provide an exact measure of the cost of livingindex. In this first case, the Carli index calculated for arandom sample would provide an estimate of the cost ofliving index provided that the items are selected withprobabilities proportional to the population expenditureshares. It might appear that if the items were selectedwith probabilities proportional to the population quan-tity shares, the sample Dutot would provide an estimateof the population Laspeyres. If the basket for the Las-peyres index contains different kinds of products whosequantities are not additive, however, the quantity shares,and hence the probabilities, are undefined.

9.33 The second case occurs when consumers areassumed to vary the quantities they consume in inverseproportion to the changes in relative prices. The cross-elasticities of demand between the different items are allunity, the expenditure shares being the same in bothperiods. The underlying preferences are described as‘‘Cobb–Douglas’’. With these preferences, the geometricLaspeyres would provide an exact measure of the cost ofliving index. The geometric Laspeyres is a weightedgeometric average of the price relatives, using the expen-diture shares in the earlier period as weights (the expen-diture shares in the second period would be the same inthe particular case under consideration). In this secondcase, the Jevons index calculated for a random samplewould provide an unbiased estimate of the cost of livingindex, provided that the items are selected with prob-abilities proportional to the population expenditureshares.

9.34 On the basis of the economic approach, thechoice between the sample Jevons and the sample Carlirests on which is likely to approximate the more closelyto the underlying cost of living index: in other words, onwhether the (unknown) cross-elasticities are likely to becloser to unity or zero, on average. In practice, the cross-elasticities could take on any value ranging up to plusinfinity for an elementary aggregate consisting of a set ofstrictly homogeneous items, i.e., perfect substitutes. Itshould be noted that in the limit when the productsreally are homogeneous, there is no index number prob-lem, and the price ‘‘index’’ is given by the ratio of theunit values in the two periods, as explained later. It maybe conjectured that the average cross-elasticity is likelyto be closer to unity than zero for most elementaryaggregates so that, in general, the Jevons index is likelyto provide a closer approximation to the cost of livingindex than the Carli. In this case, the Carli index must beviewed as having an upward bias.

9.35 The insight provided by the economic approachis that the Jevons index is likely to provide a closerapproximation to the cost of living index for the ele-mentary aggregate than the Carli because, in mostcases, a significant amount of substitution is more likely

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

158

than no substitution, especially as elementary aggregatesshould be deliberately constructed in such a way as togroup together similar items that are close substitutes foreach other.9.36 The Jevons index does not imply, or assume,

that expenditure shares remain constant. Obviously, theJevons can be calculated whatever changes do, or do notoccur in the expenditure shares in practice. What theeconomic approach shows is that if the expenditureshares remain constant (or roughly constant), then theJevons index can be expected to provide a good estimateof the underlying cost of living index. Similarly, if therelative quantities remain constant, then the Carli indexcan be expected to provide a good estimate, but the Carlidoes not actually imply that quantities remain fixed.9.37 It may be concluded that, on the basis of the

economic approach as well as the axiomatic approach,the Jevons emerges as the preferred index in general,although there may be cases in which little or no sub-stitution takes place within the elementary aggregateand the Carli might be preferred. The index compilermust make a judgement on the basis of the nature of theproducts actually included in the elementary aggregate.9.38 Before leaving this topic, it should be noted

that it has thrown light on some of the sampling prop-erties of the elementary indices. If the products in thesample are selected with probabilities proportional toexpenditures in the price reference period:

– the sample (unweighted) Carli index provides anunbiased estimate of the population Laspeyres;

– the sample (unweighted) Jevons index providesan unbiased estimate of the population geometricLaspeyres.

These results hold irrespective of the underlying cost ofliving index.

Chain versus direct indices forelementary aggregates9.39 In a direct elementary index, the prices of the

current period are compared directly with those of theprice reference period. In a chain index, prices in eachperiod are compared with those in the previous period,the resulting short-term indices being chained together toobtain the long-term index, as illustrated in Table 9.1.9.40 Provided that prices are recorded for the same

set of items in every period, as in Table 9.1, any indexformula defined as the ratio of the average prices will betransitive: that is, the same result is obtained whether theindex is calculated as a direct index or as a chain index. Ina chain index, successive numerators and denominatorswill cancel out, leaving only the average price in the lastperiod divided by the average price in the referenceperiod, which is the same as the direct index. Both theDutot and the Jevons indices are therefore transitive. Asalready noted, however, a chain Carli index is not tran-sitive and should not be used because of its upward bias.Nevertheless, the direct Carli remains an option.9.41 Although the chain and direct versions of the

Dutot and Jevons indices are identical when there are nobreaks in the series for the individual items, they offerdifferent ways of dealing with new and disappearing

items, missing prices and quality adjustments. In prac-tice, products continually have to be dropped from theindex and new ones included, in which case the direct andthe chain indices may differ if the imputations for missingprices are made differently.

9.42 When a replacement item has to be included in adirect index, it will often be necessary to estimate theprice of the new item in the price reference period, whichmay be some time in the past. The same happens if, as aresult of an update of the sample, new items have to belinked into the index. Assuming that no informationexists on the price of the replacement item in the pricereference period, it will be necessary to estimate it usingprice ratios calculated for the items that remain in theelementary aggregate, a subset of these items or someother indicator. However, the direct approach shouldonly be used for a limited period of time. Otherwise, mostof the reference prices would end up being imputed,which would be an undesirable outcome. This effectivelyrules out the use of the Carli index over a long period oftime, as the Carli can only be used in its direct formanyway, being unacceptable when chained. This impliesthat, in practice, the direct Carli may be used only if theoverall index is chain linked annually, or at intervals oftwo or three years.

9.43 In a chain index, if an item becomes perma-nently missing, a replacement item can be linked into theindex as part of the ongoing index calculation byincluding the item in the monthly index as soon as pricesfor two successive months are obtained. Similarly, if thesample is updated and new products have to be linkedinto the index, this will require successive old and newprices for the present and the preceding months. For achain index, however, the missing observation will havean impact on the index for two months, since the missingobservation is part of two links in the chain. This is notthe case for a direct index, where a single, non-estimatedmissing observation will only have an impact on theindex in the current period. For example, for a com-parison between periods 0 and 3, a missing price of anitem in period 2 means that the chain index excludes theitem for the last link of the index in periods 2 and 3, whilethe direct index includes it in period 3 since a direct indexwill be based on items whose prices are available inperiods 0 and 3. In general, however, the use of a chainindex can make the estimation of missing prices and theintroduction of replacements easier from a computa-tional point of view, whereas it may be inferred that adirect index will limit the usefulness of overlap methodsfor dealing with missing observations.

9.44 The direct and the chain approaches also pro-duce different by-products that may be used for moni-toring price data. For each elementary aggregate, achain index approach gives the latest monthly pricechange, which can be useful for both data editing andimputation of missing prices. By the same token, how-ever, a direct index derives average price levels for eachelementary aggregate in each period, and this informa-tion may be a useful by-product. Nevertheless, becausethe availability of cheap computing power and of spread-sheets allows such by-products to be calculated whethera direct or a chained approach is applied, the choice of

CALCULATING CONSUMER PRICE INDICES IN PRACTICE

159

formula should not be dictated by considerationsregarding by-products.

Consistency in aggregation9.45 Consistency in aggregation means that if an

index is calculated stepwise by aggregating lower-levelindices to obtain indices at progressively higher levels ofaggregation, the same overall result should be obtainedas if the calculation had been made in one step. Forpresentational purposes this is an advantage. If the ele-mentary aggregates are calculated using one formula andthe elementary aggregates are averaged to obtain thehigher-level indices using another formula, the resultingCPI is not consistent in aggregation. It may be argued,however, that consistency in aggregation is not necessar-ily an important or even appropriate criterion, or that itis unachievable when the amount of information avail-able on quantities and expenditures is not the same atthe different levels of aggregation. In addition, theremay be different degrees of substitution within elemen-tary aggregates as compared to the degree of substitutionbetween products in different elementary aggregates.

9.46 As noted earlier, the Carli index would beconsistent in aggregation with the Laspeyres index if theitems were to be selected with probabilities proportionalto expenditures in the reference period. This is typicallynot the case. The Dutot and the Jevons indices are alsonot consistent in aggregation with a higher-level Las-peyres. As explained below, however, the CPIs actuallycalculated by statistical offices are usually not true Las-peyres indices anyway, even though they may be basedon fixed baskets of goods and services. As also notedearlier, if the higher-level index were to be defined as ageometric Laspeyres, consistency in aggregation could beachieved by using the Jevons index for the elementaryindices at the lower level, provided that the individualitems are sampled with probabilities proportional toexpenditures. Although unfamiliar, a geometric Las-peyres has desirable properties from an economic pointof view and is considered again later.

Missing price observations9.47 The price of an item may fail to be collected in

some period either because the item is missing tem-porarily or because it has permanently disappeared. Thetwo classes of missing prices require different treatment.Temporary unavailability may occur for seasonal items(particularly for fruit, vegetables and clothing), becauseof supply shortages or possibly because of some collec-tion difficulty (say, an outlet was closed or a price col-lector was ill). The treatment of seasonal items raises anumber of particular problems. These are dealt with inChapter 22 and will not be discussed here.

9.48 The treatment of temporarily missing prices. Inthe case of temporarily missing observations for non-seasonal items, one of four actions may be taken:

– omit the item for which the price is missing so that amatched sample is maintained (like is compared withlike) even though the sample is depleted;

– carry forward the last observed price;

– impute the missing price by the average price changefor the prices that are available in the elementaryaggregate;

– impute the missing price by the price change for aparticular comparable item from another similaroutlet.

9.49 Omitting an observation from the calculationof an elementary index is equivalent to assuming thatthe price would have moved in the same way as theaverage of the prices of the items that remain included inthe index. Omitting an observation changes the implicitweights attached to the other prices in the elementaryaggregate.

9.50 Carrying forward the last observed price shouldbe avoided wherever possible and is acceptable only for avery limited number of periods. Special care needs to betaken in periods of high inflation or when markets arechanging rapidly as a result of a high rate of innovationand product turnover. While simple to apply, carryingforward the last observed price biases the resulting indextowards zero change. In addition, there is likely to be acompensating step-change in the index when the price ofthe missing item is recorded again, which will be wronglymissed by a chain index, but will be included in a directindex to return the index to its proper value. The adverseeffect on the index will be increasingly severe if the itemremains unpriced for some length of time. In general, tocarry forward is not an acceptable procedure or solutionto the problem.

9.51 Imputation of the missing price by the averagechange of the available prices may be applied for ele-mentary aggregates where the prices can be expected tomove in the same direction. The imputation can be madeusing all of the remaining prices in the elementaryaggregate. As already noted, this is numerically equiva-lent to omitting the item for the immediate period, but itis useful to make the imputation so that if the pricebecomes available again in a later period the sample sizeis not reduced in that period. In some cases, dependingon the homogeneity of the elementary aggregate, it maybe preferable to use only a subset of items from the ele-mentary aggregate to estimate the missing price. In someinstances, this may even be a single comparable itemfrom a similar type of outlet whose price change can beexpected to be similar to the missing one.

9.52 Table 9.2 illustrates the calculation of the priceindex for an elementary aggregate consisting of threeitems where one of the prices is missing inMarch. Section(a) of Table 9.2 shows the indices where the missing pricehas been omitted from the calculation. The direct indicesare therefore calculated on the basis of items A, B and Cfor all months except March, where they are calculatedon the basis of items B and C only. The chained indicesare calculated on the basis of all three prices from Jan-uary to February and from April to May. From Feb-ruary to March and from March to April the monthlyindices are calculated on the basis of items B and C only.

9.53 For both the Dutot and the Jevons indices, thedirect and chain indices now differ fromMarch onwards.The first link in the chain index (January to February) isthe same as the direct index, so the two indices are

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

160

identical numerically. The direct index for March com-pletely ignores the price decrease of item A betweenJanuary and February, while this is taken into account inthe chain index. As a result, the direct index is higherthan the chain index for March. On the other hand, inApril and May, when all prices are again available, thedirect index captures the price development, whereas thechain index fails to track the development in the prices.9.54 In section (b) of Table 9.2 the missing price for

item A in March is imputed by the average price changeof the remaining items from February to March. Whilethe index may be calculated as a direct index, comparingthe prices of the present period with the reference periodprices, the imputation of missing prices should be madeon the basis of the average price change from the pre-ceding to the present period, as shown in the table.Imputation on the basis of the average price change fromthe base period to the present period should not be usedas it ignores the information about the price change ofthe missing item that has already been included in theindex. The treatment of imputations is discussed in moredetail in Chapter 7.9.55 Treatment of items that have permanently dis-

appeared and their replacements. Items may disappearpermanently for a variety of reasons. The item maydisappear from the market because new items have beenintroduced or the outlets from which the price has beencollected have stopped selling the product. Where prod-ucts disappear permanently, a replacement product hasto be sampled and included in the index. The replace-ment product should ideally be one that accounts for asignificant proportion of sales, is likely to continue to be

sold for some time, and is likely to be representative ofthe sampled price changes of the market that the oldproduct covered.

9.56 The timing of the introduction of replacementitems is important. Many new products are initially soldat high prices which then gradually drop over time,especially as the volume of sales increases. Alternatively,some products may be introduced at artificially lowprices to stimulate demand. In such cases, delaying theintroduction of a new or replacement item until a largevolume of sales is achieved may miss some systematicprice changes that ought to be captured by CPIs. It maybe desirable to try to avoid forced replacements causedwhen products disappear completely from the market,and to try to introduce replacements when sales of theitems they replace are falling away, but before they ceasealtogether.

9.57 Table 9.3 shows an example where item A dis-appears after March and item D is included as a replace-ment from April onwards. Items A and D are notavailable on the market at the same time and their priceseries do not overlap.

9.58 To include the new item in the index from Aprilonwards, an imputed price needs to be calculated eitherfor the base period (January) if a direct index is beingcalculated, or for the preceding period (March) if a chainindex is calculated. In both cases, the imputation methodensures that the inclusion of the new item does not, initself, affect the index. In the case of a chain index,imputing the missing price by the average change of theavailable prices gives the same result as if the item issimply omitted from the index calculation until it has

Table 9.2 Imputation of temporarily missing prices

January February March April May

PricesItem A 6.00 5.00 7.00 6.60Item B 7.00 8.00 9.00 8.00 7.70Item C 2.00 3.00 4.00 3.00 2.20

(a) Omit missing prices from the index calculationCarli index – the arithmetic mean of price ratiosDirect index 100.00 115.87 164.29 126.98 110.00Dutot index – the ratio of arithmetic mean pricesMonth-to-month index 100.00 106.67 118.18 84.62 91.67Chained month-to-month index 100.00 106.67 126.06 106.67 97.78Direct index 100.00 106.67 144.44 120.00 110.00Jevons index – the ratio of geometric mean prices = geometric mean of price ratiosMonth-to-month index 100.00 112.62 122.47 81.65 87.31Chained month-to-month index 100.00 112.62 137.94 112.62 98.33Direct index 100.00 112.62 160.36 125.99 110.00

(b) ImputationCarli index – the arithmetic mean of price ratiosImpute price for item A in March as 5 �(9/8+4/3)/2=6.15Direct index 100.00 115.87 143.67 126.98 110.00Dutot index – the ratio of arithmetic mean pricesImpute price for item A in March as 5 �((9+4)/(8+3))=5.91Month-to-month index 100.00 106.67 118.18 95.19 91.67Chained month-to-month index 100.00 106.67 126.06 120.00 110.00Direct index 100.00 106.67 126.06 120.00 110.00Jevons index – the ratio of geometric mean prices = geometric mean of price ratiosImpute price for item A in March as 5 �(9/8) �(4/3)0.5=6.12Month-to-month index 100.00 112.62 122.47 91.34 87.31Chained month-to-month index 100.00 112.62 137.94 125.99 110.00Direct index 100.00 112.62 137.94 125.99 110.00

CALCULATING CONSUMER PRICE INDICES IN PRACTICE

161

been priced in two successive periods. This allows thechain index to be compiled by simply chaining themonth-to-month index between periods t�1 and t, basedon the matched set of prices in those two periods, ontothe value of the chain index for period t�1. In theexample, no further imputation is required after April,and the subsequent movement of the index is unaffectedby the imputed price change between March and April.

9.59 In the case of a direct index, however, an impu-ted price is always required for the reference period inorder to include a new item. In the example, the price ofthe new item in each month after April still has to becompared with the imputed price for January. As alreadynoted, to prevent a situation in which most of the refer-ence period prices end up being imputed, the directapproach should only be used for a limited period of time.

9.60 The situation is somewhat simpler when there isan overlap month in which prices are collected for boththe disappearing and the replacement item. In this case,it is possible to link the price series for the new item tothe price series for the old item that it replaces. Linkingwith overlapping prices involves making an implicitadjustment for the difference in quality between the twoitems, as it assumes that the relative prices of the newand old item reflect their relative qualities. For perfect ornearly perfect markets this may be a valid assumption,but for certain markets and products it may not be soreasonable. The question of when to use overlappingprices is dealt with in detail in Chapter 7. The overlapmethod is illustrated in Table 9.4.

9.61 In the example in Table 9.4, overlapping pricesare obtained for items A and D in March. Their relative

prices suggest that one unit of item D is worth two unitsof item A. If the index is calculated as a direct Carli, theJanuary base period price for item D can be imputed bydividing the price of item A in January by the price ratioof items A and D in March.

9.62 A monthly chain index of arithmetic mean pri-ces will be based on the prices of items A, B and C untilMarch, and from April onwards on the prices of items B,C and D. The replacement item is not included untilprices for two successive periods are obtained. Thus, themonthly chain index has the advantage that it is notnecessary to carry out any explicit imputation of areference price for the new item.

9.63 If a direct index is calculated defined as the ratioof the arithmetic mean prices, the price of the new itemneeds to be adjusted by the price ratio of A and D inMarch in every subsequent month, which complicatescomputation. Alternatively, a reference period price ofitem D for January may be imputed. This, however,results in a different index because the price ratios areimplicitly weighted by the relative base period prices inthe Dutot index, which is not the case for the Carli or theJevons indices. For the Jevons index, all three methodsgive the same result, which is an additional advantage ofthis approach.

Other formulae for elementaryprice indices

9.64 A number of other formulae have been sug-gested for the price indices for elementary aggregates.

Table 9.3 Disappearing items and their replacements with no overlapping prices

January February March April May

PricesItem A 6.00 7.00 5.00Item B 3.00 2.00 4.00 5.00 6.00Item C 7.00 8.00 9.00 10.00 9.00Item D 9.00 8.00

(a) ImputationCarli index – the arithmetic mean of price ratiosImpute price for item D in January as 9/((5/3+10/7) � 0.5)= 5.82Direct index 100.00 99.21 115.08 154.76 155.38Dutot index – the ratio of arithmetic mean pricesImpute price for item D in March as 9/((5+10)/(4+9))=7.80Month-to-month index 100.00 106.25 105.88 115.38 95.83Chained month-to-month index 100.00 106.25 112.50 129.81 124.40Impute price for item D in January as 9/((5+10)/(3+7))=6.00Direct index 100.00 106.25 112.50 150.00 143.75Jevons index – the ratio of geometric mean prices = geometric mean of price ratiosImpute price for item D in March as 9/((5/4 �10/9)0.5)=7.64Month-to-month index 100.00 96.15 117.13 117.85 98.65Chained month-to-month index 100.00 96.15 112.62 132.73 130.94Impute price for item D in January as 9/((5/3 �10/7)0.5)=5.83Direct index 100.00 96.15 112.62 154.30 152.22

(b) Omit missing pricesDutot index – the ratio of arithmetic mean pricesMonth-to-month index 100.00 106.25 105.88 115.38 95.83Chained month-to-month index 100.00 106.25 112.50 129.81 124.40Jevons index – the ratio of geometric mean prices = geometric mean of price ratiosMonthly index 100.00 96.15 117.13 117.85 98.65Chained month-to-month index 100.00 96.15 112.62 132.73 130.94

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

162

The most important are presented below and discussedfurther in Chapter 20.9.65 Laspeyres and geometric Laspeyres indices. The

Carli, Dutot and Jevons indices are all calculatedwithout the use of explicit weights. However, as alreadymentioned, in certain cases there may be weight-ing information that could be exploited in the calcula-tion of the elementary price indices. If the referenceperiod expenditures for all the individual items withinan elementary aggregate, or estimates thereof, wereto be available, the elementary price index could itselfbe calculated as a Laspeyres price index, or as a geo-metric Laspeyres. The Laspeyres price index is definedas:

I0:tLa=

Pw0i

ptip0i

� �,P

w0i=1 (9.4)

where the weights, wi0, are the expenditure shares for the

individual items in the reference period. If all the weightswere equal, the formula (9.4) would reduce to the Carliindex. If the weights were proportional to the prices inthe reference period, the formula (9.4) would reduce tothe Dutot index.9.66 The geometric version of the Laspeyres index is

defined as:

I0:tJW=

Q ptip0i

� �wi0

=

Q( pti)

w0iQ

( p0i Þw0i

,P

w0i=1 (9.5)

where the weights, wi0, are again the expenditure shares

in the reference period. When the weights are all equal,the formula (9.5) reduces to the Jevons index.9.67 Some alternative index formulae. Another

widely used type of average is the harmonic mean. Inthe present context, there are two possible versions:either the harmonic mean of price ratios or the ratio of

harmonic mean prices. The harmonic mean of pricerelatives, or ratios, is defined as:

I0:tHR=

1

1

n

P p0ipti

(9.6)

The ratio of harmonic mean prices is defined as:

I0:tRH=

P n

p0iP n

pti

(9.7)

Formula (9.7), like the Dutot index, fails the commen-surability test and would only be an acceptable possi-bility when the items are all fairly homogeneous. Neitherformula appears to be used much in practice, perhapsbecause the harmonic mean is not a familiar conceptand would not be easy to explain to users. Nevertheless,at an aggregate level, the widely used Paasche index is aweighted harmonic average.

9.68 The ranking of the three common types of meanis always arithmetic� geometric� harmonic. It is shownin Chapter 20 that, in practice, the Carli index (thearithmetic mean of the relatives) is likely to exceed theJevons index (the geometric mean) by roughly the sameamount that the Jevons exceeds the harmonic meangiven by the formula (9.6). The harmonic mean of theprice relatives has the same kinds of axiomatic propertiesas the Carli index, but with opposite tendencies andbiases. It fails the transitivity, time reversal and pricebouncing tests. As it can be viewed conceptually as thecomplement, or rough mirror image, of the Carli index, ithas been argued that a suitable elementary index wouldbe provided by a geometric mean of the two, in the sameway that, at an aggregate level, a geometric mean is takenof the Laspeyres and Paasche indices to obtain the Fisher

Table 9.4 Disappearing and replacement items with overlapping prices

January February March April May

PricesItem A 6.00 7.00 5.00Item B 3.00 2.00 4.00 5.00 6.00Item C 7.00 8.00 9.00 10.00 9.00Item D 10.00 9.00 8.00Carli index – the arithmetic mean of price ratiosImpute price for item D in January as 6/(5/10)=12.00Direct index 100.00 99.21 115.08 128.17 131.75Dutot index – the ratio of arithmetic mean pricesChain the monthly indices based on matched pricesMonth-to-month index 100.00 106.25 105.88 104.35 95.83Chained month-to-month index 100.00 106.25 112.50 117.39 112.50Divide item D’s price in April and May by 10/5=2 and use item A’s price in January as base priceDirect index 100.00 106.25 112.50 121.88 118.75Impute price for item D in January as 6/(5/10)=12.00Direct index 100.00 106.25 112.50 109.09 104.55Jevons index – the ratio of geometric mean prices = geometric mean of price ratiosChain the monthly indices based on matched pricesMonth-to-month index 100.00 96.15 117.13 107.72 98.65Chained month-to-month index 100.00 96.15 112.62 121.32 119.68Divide item D’s price in April and May with 10/5=2 and use item A’s price in January as base priceDirect index 100.00 96.15 112.62 121.32 119.68Impute price for item D in January as 6/(5/10)=12.00Direct index 100.00 96.15 112.62 121.32 119.68

CALCULATING CONSUMER PRICE INDICES IN PRACTICE

163

index. Such an index has been proposed by Carruthers,Sellwood and Ward (1980) and Dalen (1992), namely:

I0:tCSWD=(I0:t

C I0:tHR)

1=2 (9.8)

ICSWD is shown in Chapter 20 to have very good axiom-atic properties, although not quite as good as the Jevonsindex, which is transitive, whereas the ICSWD is not. Itcan, however, be shown to be approximately transitive,and it has been observed empirically to be very close tothe Jevons index.

9.69 In recent years, attention has focused on for-mulae that can take account of the substitution that maytake place within an elementary aggregate. As alreadyexplained, the Carli and the Jevons indices may beexpected to approximate to the cost of living index if thecross-elasticities of substitution are close to 0 and 1,respectively, on average. A more flexible formula thatallows for different elasticities of substitution is theunweighted Lloyd–Moulton (LM) index:

I0:tLM=

P 1

n

Pti

P0i

� �1�s" # 11�s

(9.9)

where s is the elasticity of substitution. The Carli andthe Jevons indices can be viewed as special cases of theLM in which s=0 and s=1. The advantage of the LMformula is that s is unrestricted. Provided a satisfactoryestimate can be made of s, the resulting elementary priceindex is likely to approximate to the underlying cost ofliving index. The LM index reduces ‘‘substitution bias’’when the objective is to estimate the cost of living index.The difficulty is the need to estimate elasticities of sub-stitution, a task that will require substantial develop-ment and maintenance work. The formula is describedin more detail in Chapter 17.

Unit value indices9.70 The unit value index is simple in form. The unit

value in each period is calculated by dividing totalexpenditure on some product by the related total quan-tity. It is clear that the quantities must be strictly additivein an economic sense, which implies that they shouldrelate to a single homogeneous product. The unit valueindex is then defined as the ratio of unit values in thecurrent period to that in the reference period. It is not aprice index as normally understood, as it is essentially ameasure of the change in the average price of a singleproduct when that product is sold at different prices todifferent consumers, perhaps at different times within thesame period. Unit values, and unit value indices, shouldnot be calculated for sets of heterogeneous products.

9.71 Unit values do play an important part in theprocess of calculating an elementary price index, as theyare the appropriate average prices that need to be enteredinto an elementary price index. Usually, prices are sam-pled at a particular time or period each month, and eachprice is assumed to be representative of the average priceof that item in that period. In practice, this assumptionmay not hold. In this case, it is necessary to estimate theunit value for each item, even though this will inevitablybe more costly. Thus, having specified the item to be

priced in a particular store, data should be collected onboth the value of the total sales in a particular month andthe total quantities sold in order to derive a unit value tobe used as the price input into an elementary aggregateformula. It is particularly important to do this if the itemis sold at a sale price for part of the period and at theregular price in the rest of the period. Under these con-ditions, neither the sale price nor the regular price islikely to be representative of the average price at whichthe item has been sold or the price change between peri-ods. The unit value over the whole month should beused. With the possibility of collecting more and moredata from electronic points of sale, such procedures maybe increasingly used. It should be stressed, however, thatthe item specifications must remain constant throughtime. Changes in the item specifications could lead to unitvalue changes that reflect quantity or quality changes,and should not be part of price changes.

Formulae applicable to scanner data9.72 Scanner data obtained from electronic points of

sale are becoming an increasingly important source ofdata for CPI compilation. Their main advantage is thatthe number of price observations can be enormouslyincreased and that both price and quantity information isavailable in real time. There are, however, many practicalconsiderations to be taken into consideration which arediscussed in other chapters of this manual.

9.73 Access to detailed and comprehensive quantityand expenditure information within an elementaryaggregate means that there are no constraints on thetype of index number that may be employed. Not onlyLaspeyres and Paasche but superlative indices such asFisher and Tornqvist may be envisaged. As noted atthe beginning of this chapter, it is preferable tointroduce weighting information as it becomes availablerather than continuing to rely on simple unweightedindices such as Carli and Jevons. Advances in technol-ogy, both in the retail outlets themselves and in thecomputing power available to statistical offices, suggestthat traditional elementary price indices may eventuallybe replaced by superlative indices, at least for some ele-mentary aggregates in some countries. The methodologymust be kept under review in the light of the resourcesavailable.

The calculation ofhigher-level indices

9.74 A statistical office must have some target indexat which to aim. Statistical offices have to consider whatkind of index they would choose to calculate in the idealhypothetical situation in which they had completeinformation about prices and quantities in both timeperiods compared. If the CPI is meant to be a cost ofliving index, then a superlative index such as a Fisher,Walsh or Tornqvist–Theil would have to serve as thetheoretical target, as a superlative index may be expectedto approximate to the underlying cost of living index.

9.75 Many countries do not aim to calculate a cost ofliving index and prefer the concept of a basket index. A

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

164

basket index is one that measures the change in the totalvalue of a given basket of goods and services betweentwo time periods. This general category of index isdescribed here as a Lowe index (see Chapters 1 and 15).The meaning of a Lowe index is clear and can be easilyexplained to users, these being important considerationsfor many statistical offices. It should be noted that, ingeneral, there is no necessity for the basket to be theactual basket in one or other of the two periods com-pared. If the theoretical target index is to be a basket orLowe index, the preferred basket might be one thatattaches equal importance to the baskets in both periods;for example, the Walsh index. The quantities that makeup the basket in theWalsh index are the geometric meansof the quantities in the two periods. Thus, the same kindof index may emerge as the theoretical target in both thebasket and the cost of living approaches. In practice, astatistical office may prefer to designate a basket indexthat uses the actual basket in the earlier of the two periodsas its target index on grounds of simplicity and practi-cality. In other words, the Laspeyres index may be thetarget index.9.76 The theoretical target index is a matter of

choice. In practice, it is likely to be either a Laspeyres orsome superlative index. Even when the target index is theLaspeyres, there may a considerable gap between what isactually calculated and what the statistical office con-siders to be its target. It is now necessary to considerwhat statistical offices tend to do in practice.

Consumer price indices asweighted averages ofelementary indices9.77 A higher-level index is an index for some

expenditure aggregate above the level of an elementaryaggregate, including the overall CPI itself. The inputsinto the calculation of the higher-level indices are:

– the elementary price indices;

– weights derived from the values of elementary aggre-gates in some earlier year or years.

9.78 The higher-level indices are calculated simply asweighted arithmetic averages of the elementary priceindices. This general category of index is described hereas a Young index after another nineteenth-century indexnumber pioneer who advocated this type of index.9.79 The weights typically remain fixed for a

sequence of at least 12 months. Some countries revisetheir weights at the beginning of each year in order to tryto approximate as closely as possible to current consump-tion patterns, but many countries continue to use thesame weights for several years. The weights may bechanged only every five years or so. The use of fixedweights has the considerable practical advantage that theindex can make repeated use of the same weights. Thissaves both time and money. Revising the weights can beboth time-consuming and costly, especially if it requiresnew household expenditure surveys to be carried out.9.80 The second stage of calculating a CPI does not

involve individual prices or quantities. Instead, a higher-level index is a Young index in which the elementary

price indices are averaged using a set of pre-determinedweights. The formula can be written as follows:

I0:t=P

wbi I

0:ti ,

Pwbi=1 (9.10)

where I 0:t denotes the overall CPI, or any higher-levelindex, from period 0 to t; wi

b is the weight attached toeach of the elementary price indices; and Ii

0:t is thecorresponding elementary price index. The elementaryindices are identified by the subscript i, whereas thehigher-level index carries no subscript. As already noted,a higher-level index is any index, including the overallCPI, above the elementary aggregate level. The weightsare derived from expenditures in period b, which inpractice has to precede period 0, the price referenceperiod.

9.81 It is useful to recall that three kinds of referenceperiod may be distinguished for CPI purposes:

� Weight reference period. The period covered by theexpenditure statistics used to calculate the weights.Usually, the weight reference period is a year.

� Price reference period. The period for which prices areused as denominators in the index calculation.

� Index reference period. The period for which the indexis set to 100.

9.82 The three periods are generally different. Forexample, a CPI might have 1998 as the weight referenceyear, December 2002 as the price reference month andthe year 2000 as the index reference period. The weightstypically refer to a whole year, or even two or threeyears, whereas the periods for which prices are com-pared are typically months or quarters. The weights areusually estimated on the basis of an expenditure surveythat was conducted some time before the price referenceperiod. For these reasons, the weight reference periodand the price reference period are invariably separateperiods in practice.

9.83 The index reference period is often a year; but itcould be a month or some other period. An index seriesmay also be re-referenced to another period by simplydividing the series by the value of the index in thatperiod, without changing the rate of change of the index.The expression ‘‘base period’’ can mean any of the threereference periods and is ambiguous. The expression‘‘base period’’ should only be used when it is absolutelyclear in context exactly which period is referred to.

9.84 Provided the elementary aggregate indices arecalculated using a transitive formula such as the Jevonsor Dutot, but not the Carli, and provided that there areno new or disappearing items from period 0 to t,equation (9.10) is equivalent to:

I0:t=P

wbi I

0:t�1i I t�1:t

i ,P

wbi=1, (9.11)

The advantage of this version of the index is that it allowsthe sampled products within the elementary price indexfrom t�1 to t to differ from the sampled products in theperiods from 0 to t�1. Hence, it allows replacementitems and new items to be linked into the index fromperiod t�1 without the need to estimate a price for per-iod 0. For example, if one of the sampled items in periods0 and t�1 is no longer available in period t, and the price

CALCULATING CONSUMER PRICE INDICES IN PRACTICE

165

of a replacement product is available for t�1 at t, the newreplacement product can be included in the index usingthe overlap method.

A numerical example9.85 Equation (9.10) applies at each level of aggre-

gation. The index is additive; that is, the overall index isthe same whether calculated on the basis of the originalelementary price indices or on the basis of the inter-mediate higher-level indices. This facilitates the presen-tation of the index.

9.86 Table 9.5 illustrates the calculation of higher-level indices in the special case where the weight and theprice reference period are identical, i.e. b=0. The indexconsists of five elementary aggregate indices and twointermediate higher-level indices, G and H. The overallindex and the higher-level indices are all calculated usingequation (9.10). Thus, for example, the overall index forApril can be calculated from the two intermediatehigher-level indices for April as:

IJan:Apr=0:6� 103:92+0:4� 101:79=103:06

or directly from the five elementary indices as:

IJan:Apr=0:2� 108:75+0:25� 100+0:15� 104+0:1

� 107:14+0:3� 100=103:06

Note from equation (9.11) that:

I0:t=P

wbi I

0:t�1i I t�1:t

i 6¼ I0:t�1Pwbi I

t�1:ti )

I0:t

I0:t�1 6¼P

wbi I

t�1:ti

(9.12)

This shows that if the month-to-month indices areaveraged using the fixed weights wi

b, the resulting indexis not equal to the month-to-month higher-level index.As explained later, in order to be able to obtain themonth-to-month higher-level index, the weights appliedto the month-to-month indices need to be updated toreflect the effects of the price changes that have takenplace since January.

Young and Lowe indices9.87 It is useful to clarify the relationship between

Lowe and Young indices. As already noted, when sta-tistical offices explain their CPIs to users they oftendescribe them as Lowe indices, which measure the changeover time in the value of a fixed basket of goods andservices. But when they calculate their CPIs, the formulathey actually use is that of a Young index. The rela-tionship between the two indices is given in equation(9.13), where ILo is the Lowe index and IYo is the Youngindex:

ILo=

Pptjq

bjP

p0j qbj

=

Pptjq

bjP

pbj qbj

,Pp0j q

bjP

pbj qbj

=P

wj

ptj

p0j

!=IYo

where wj=poj q

bjP

p0j qbj

(9.13)

The values qjb, the individual quantities in the weight

reference period b, make up the basket. Assume initiallythat the weight reference period b has the same dura-tion as that of the two periods 0 and t that are beingcompared. It can be seen from the relationship (9.13)that:

� the Lowe index is equal to a Young index in which theweights are hybrid value shares obtained by revaluingthe values qb, the quantities in the weight referenceperiod b, at the prices of the price reference month 0;

� the Lowe index can be expressed as the ratio of thetwo Laspeyres indices for periods t and 0, respectively,based on month b;

� the Lowe index reduces to the Laspeyres index whenb=0, and to the Paasche index when b=t.

9.88 In practice, the situation is more complicatedfor actual CPIs because the duration of the referenceperiod b is typically much longer than periods 0 and t.The weights wj usually refer to the expenditures over aperiod of a year, or longer, while the price referenceperiod is usually a month in some later year. For exam-ple, a monthly index may be compiled from January2003 onwards with December 2002 as the price reference

Table 9.5 The aggregation of elementary price indices

Weight January February March April May June

Month-to-month elementary price indicesA 0.20 100.00 102.50 104.88 101.16 101.15 100.00B 0.25 100.00 100.00 91.67 109.09 101.67 108.20C 0.15 100.00 104.00 96.15 104.00 101.92 103.77D 0.10 100.00 92.86 107.69 107.14 100.00 102.67E 0.30 100.00 101.67 100.00 98.36 103.33 106.45Direct or chained monthly elementary price indices with January=100A 0.20 100.00 102.50 107.50 108.75 110.00 110.00B 0.25 100.00 100.00 91.67 100.00 101.67 110.00C 0.15 100.00 104.00 100.00 104.00 106.00 110.00D 0.10 100.00 92.86 100.00 107.14 107.14 110.00E 0.30 100.00 101.67 101.67 100.00 103.33 110.00Total 100.00 100.89 99.92 103.06 105.03 110.00Higher-level indicesG=A+B+C 0.60 100.00 101.83 99.03 103.92 105.53 110.00H=D+E 0.40 100.00 99.46 101.25 101.79 104.29 110.00Total 100.00 100.89 99.92 103.06 105.03 110.00

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

166

month, but the latest available weights during the year2003 may refer to the year 2000 or even some earlieryear.9.89 Conceptually, a typical CPI may be viewed as a

Lowe index that measures the change from month tomonth in the total cost of an annual basket of goods andservices that may date back several years before the pricereference period. Because it uses the fixed basket of anearlier period, it is sometimes loosely described as a‘‘Laspeyres-type index’’, but this description is unwar-ranted. A true Laspeyres index would require the basketto be that consumed in the price reference month,whereas in most CPIs the basket not only refers to adifferent period from the price reference month but to aperiod of a year or more. When the weights are annualand the prices are monthly, it is not possible, even retro-spectively, to calculate a monthly Laspeyres priceindex.9.90 As shown in Chapter 15, a Lowe index that

uses quantities derived from an earlier period than theprice reference period is likely to exceed the Laspeyres,and by a progressively larger amount, the further back intime the weight reference period is. The Lowe index islikely to have an even greater upward bias than theLaspeyres as compared with some target superlativeindex or underlying cost of living index. Inevitably, thequantities in any basket index become increasingly outof date and irrelevant the further back in time theperiod to which they relate. To minimize the result-ing bias, the weights should be updated as often aspossible.9.91 A statistical office may not wish to estimate a

cost of living index and may prefer to choose somebasket index as its target index. In that case, if the the-oretically attractive Walsh index were to be selected asthe target index, a Lowe index would have the same biasas just described, given that the Walsh index is also asuperlative index.

Factoring the Young index9.92 It is possible to calculate the change in a higher-

level Young index between two consecutive periods,such as t�1 and t, as a weighted average of the indivi-dual price indices between t�1 and t provided that theweights are updated to take account of the price changesbetween the price reference period 0 and the previousperiod, t�1. This makes it possible to factor the formula(9.10) into the product of two component indices in thefollowing way:

I0:t=I0:t�1Pwb(t�1)i I t�1:1

i

where wb(t�1)i =wb

i I0:t�1i

.Pwbi I

0:t�1i : (9.14)

I 0:t�1 is the Young index for period t�1. The weightwi

b(t�1) is the original weight for elementary aggregate iprice-updated by multiplying it by the elementary priceindex for i between 0 and t�1, the adjusted weightsbeing rescaled to sum to unity. The price-updatedweights are hybrid weights because they implicitlyrevalue the quantities of b at the prices of t�1 instead

of at the average prices of b. Such hybrid weights do notmeasure the actual expenditure shares of any period.

9.93 The index for period t can thus be calculated bymultiplying the already calculated index for t�1 by aseparate Young index between t�1 and t with hybridprice-updated weights. In effect, the higher-level index iscalculated as a chain index in which the index is movedforward period by period. This method gives more flex-ibility to introduce replacement items and makes it easierto monitor the movements of the recorded prices forerrors, as month-to-month movements are smaller andless variable than the total changes since the base period.

9.94 Price-updating may also occur between theweight reference period and the price reference period,as explained in the next section.

Price-updating from the weight referenceperiod to the price reference period

9.95 When the weight reference period b and theprice reference period 0 are different, as is normally thecase, the statistical office has to decide whether or not toprice-update the weights from b to 0. In practice, theprice-updated weights can be calculated by multiplyingthe original weights for period b by elementary indicesmeasuring the price changes between periods b and 0and rescaling to sum to unity.

9.96 The issues involved are best explained with thehelp of a numerical example. In Table 9.6, the baseperiod b is assumed to be the year 2000, so the weightsare the expenditure shares in 2000. In section (a) of Table9.6, 2000 is also used as the price reference period. Inpractice, however, weights based on 2000 cannot beintroduced until some time after 2000 because of the timeneeded to collect and process the expenditure data. Insection (b) of Table 9.6, it is assumed that the 2000weights are introduced in December 2002 and that this isalso chosen as the new price reference base.

9.97 Note that it would be possible in December2002 to calculate the indices based on 2000 shown insection (a) of the table, but it is decided to makeDecember 2002 the price reference base. This does notprevent the index with the December 2002 price referenceperiod from being calculated backwards a few monthsinto 2002, if desired.

9.98 The statistical office compiling the index hastwo options at the time the new index is introduced. It hasto decide whether the weights in the new index shouldpreserve the quantities in 2000 or the expenditures in2000. It cannot do both.

9.99 If it is decided to preserve the quantities, theresulting index is a basket index, or Lowe index, in whichthe quantities are those of the year 2000. This impliesthat the movements of the index must be identical withthose of the index based on 2000 shown in section (a) ofthe table. In this case, if the index is to be presented as aweighted average of the elementary price indices withDecember 2002 as the price reference period, theexpenditure weights for 2000 have to be price-updated toDecember 2002. This is illustrated in section (b) of Table9.6, where the updated weights are obtained by multi-plying the original weights for 2000 in section (a) of the

CALCULATING CONSUMER PRICE INDICES IN PRACTICE

167

table by the price indices for the elementary aggregatesbetween 2000 and December 2002, and then rescaling theresults to sum to unity. These are the weights labelledw00(Dec02) in the table.

9.100 The indices with price-updated weights insection (b) of Table 9.6 are Lowe indices in which b=2000 and 0=December 2002. These indices can beexpressed as ratios of the indices in the upper part of thetable. For example, the overall Lowe index for March2003 with December 2002 as its price reference base,namely 101.97, is the ratio of the index for March 2003based on 2000 shown in section (a) of the table, namely106.05, divided by the index for December 2002 based on2000, namely 104.00. Thus, the price-updating preservesthe movements of the indices in section (a) of the tablewhile shifting the price reference period to December2002.

9.101 Alternatively, it could be decided to calculate aseries of Young indices using the expenditure weightsfrom 2000 as they stand without updating. If theexpenditure shares were actually to remain constant,the quantities would have had to move inversely with theprices between 2000 and December 2002. The quantitiesthat make up the basket for the new Young index couldnot be the same as those of 2000. The movements of thisindex would have to be slightly different from those ofthe price-updated index.

9.102 The issue is whether to stick with the knownquantities of the weight reference period 2000, which arethe latest for which firm data have been collected, or tostick with the known expenditure shares of the weightreference period. If the official objective is to measure aLowe index that uses a fixed basket, the issue is decidedand the statistical office is obliged to price-update. On theother hand, some statistical offices may have to decidefor themselves which option to adopt.

9.103 Updating the prices without updating thequantities does not imply that the resulting expenditureweights are necessarily more up to date. When there is astrong inverse relation between movements of price andquantities, price-updating on its own could produceperverse results. For example, the prices of computershave been declining rapidly in recent years. If the quan-tities are held fixed while the prices are updated, theresulting expenditures on computers would also declinerapidly. In practice, however, the share of expendi-tures on computers might be actually be rising becauseof a very rapid increase in quantities of computerspurchased.

9.104 When there are rapid changes taking place inrelative quantities as well as relative prices, statisticaloffices are effectively obliged to change their expenditureweights more frequently, even if this means conductingmore frequent expenditure surveys. Price-updating onits own cannot cope with this situation. The expenditureweights have to be updated with respect to their quan-tities as well as their prices, which, in effect, impliescollecting new expenditure data.

The introduction of new weightsand chain linking

9.105 From time to time, the weights for the ele-mentary aggregates have to be revised to ensure thatthey reflect current expenditure patterns and consumerbehaviour. When new weights are introduced, the pricereference period for the new index can be the last periodof the old index, the old and the new indices being linkedtogether at this point. The old and the new indices makea chain index.

9.106 The introduction of new weights is often acomplex operation because it provides the opportunity

Table 9.6 Price-updating of weights between the weight and price reference periods

Weight 2000 Nov. 02 Dec. 02 Jan. 03 Feb. 03 Mar. 03

(a) Index with 2000 as weight and price reference periodElementary price indices

w00

A 0.20 100.00 98.00 99.00 102.00 101.00 104.00B 0.25 100.00 106.00 108.00 107.00 109.00 110.00C 0.15 100.00 104.00 106.00 98.00 100.00 97.00D 0.10 100.00 101.00 104.00 108.00 112.00 114.00E 0.30 100.00 102.00 103.00 106.00 105.00 106.00

Higher-level indicesG=A+B+C 0.60 100.00 102.83 104.50 103.08 104.08 104.75H=D+E 0.40 100.00 101.75 103.25 106.50 106.75 108.00Total 100.00 102.40 104.00 104.45 105.15 106.05(b) Index re-referenced to December 2002 and weights price-updated to December 2002

Elementary price indicesw00(Dec02)

A 0.190 101.01 98.99 100.00 103.03 102.02 105.05B 0.260 92.59 98.15 100.00 99.07 100.93 101.85C 0.153 94.34 98.11 100.00 92.45 94.34 91.51D 0.100 96.15 97.12 100.00 103.85 107.69 109.62E 0.297 97.09 99.03 100.00 102.91 101.94 102.91

Higher-level indicesG=A+B+C 0.603 95.69 98.41 100.00 98.64 99.60 100.24H=D+E 0.397 96.85 98.55 100.00 103.15 103.39 104.60Total 96.15 98.46 100.00 100.43 101.11 101.97Rescaled to 2000= 100 100.00 102.40 104.00 104.45 105.15 106.05

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

168

to introduce new items, new samples, new data sources,new compilation practices, new elementary aggregates,new higher-level indices or new classifications. Thesetasks are often undertaken simultaneously at the time ofreweighting to minimize overall disruption to the timeseries and any resulting inconvenience to users of theindices.9.107 In many countries, reweighting and chaining

are carried out about every five years, but some countriesintroduce new weights each year. Chain indices do nothave to be linked annually; the linking may be done lessfrequently. The real issue is not whether to chain or notbut how frequently to chain. Reweighting is inevitablesooner or later, as the same weights cannot continueto be used for ever. Whatever the time frame, sta-tistical offices have to address the issue of chain linkingsooner or later. It is inevitable and a major task for indexcompilers.9.108 Frequency of reweighting. It is reasonable to

continue to use the same set of elementary aggregateweights so long as consumption patterns at the elemen-tary aggregate level remain fairly stable. Over time,consumers will tend to substitute away from products ofwhich the prices have increased relatively. Thus, in gen-eral, movements in prices and quantities tend to beinversely related. This kind of substitution behaviour onthe part of consumers implies that a Lowe index basedon the fixed basket of an earlier period will tend to havean upward bias compared with a basket index usingup-to-date weights.9.109 Another reason why consumption patterns

change is that new products are continually beingintroduced on the market while others drop out. Over thelonger term, consumption patterns are also influenced byseveral other factors. These include rising incomes andstandards of living, demographic changes in the structureof the population, changes in technology, and changes intastes and preferences.9.110 There is wide consensus that regular updating

of weights – at least every five years, and more often ifthere is evidence of rapid changes in consumption pat-terns – is a sensible and necessary practice. The questionof how often to change the weights and chain link theindex is nevertheless not straightforward, as frequentlinking can also have some disadvantages. It can becostly to obtain new weights, especially if they requiremore frequent expenditure surveys. Annual chaining hasthe advantage that changes (such as the inclusion of newgoods) can be introduced on a regular basis, althoughevery index needs some ongoing maintenance, whetherannually chained or not.9.111 Expenditures on certain types of products

are strongly influenced by short-term fluctuations inthe economy. For example, expenditures on cars, majordurables, expensive luxuries, and so on, may change dras-tically from year to year. In such cases, it may be pre-ferable to base the weight on an average of two or moreyears of expenditure.9.112 The calculation of a chain index. Assume that a

series of fixed weight Young indices has been calculatedwith period 0 as the price reference period and that in asubsequent period, k, a new set of weights has to be

introduced in the index. The new set of weights may, ormay not, have been price-updated from the new weightreference period to period k. A chain index is then cal-culated as:

I0:t=I0:kPwki I

k:t�1i I t�1:t

i

=I0:kPwki I

k:ti

=I0:kIk:t (9.15)

9.113 There are several important features of a chainindex:

� The chain index formula allows weights to be up-dated, and facilitates the introduction of new itemsand sub-indices and the removal of obsolete ones.

� In order to be able to link the old and the new series,an overlapping period (k) is needed in which the indexhas to be calculated using both the old and the new setof weights.

� A chain index may have two or more links. Betweeneach link period, the index may be calculated as afixed weight index using the formula (9.10), or indeedusing any other index formula. The link period maybe a month or a year, provided the weights and indicesrefer to the same period.

� Chaining is intended to ensure that the individualindices on all levels show the correct developmentthrough time.

� Chaining leads to non-additivity. When the newseries is chained onto the old, as in equation (9.15),the higher-level indices after the link cannot beobtained as weighted arithmetic averages of indivi-dual indices using the new weights. If, on the otherhand, the index reference period is changed and theindex series prior to the link period is rescaled to thenew index reference period, this series cannot beaggregated to higher-level indices by use of the newweights. Such results need to be carefully explainedand presented.

9.114 An example of the calculation of a chain indexis presented in Table 9.7. From 1998 to December 2002the index is calculated with the year 1998 as weight andprice reference period. From December 2002 onwards, anew set of weights is introduced. The weights may referto the year 2000, for example, and may or may not havebeen price-updated to December 2002. A new fixedweight index series is then calculated with December2002 as the price reference month. Finally, the new indexseries is linked onto the old index with 1998=100 bymultiplication to get a continuous index from 1998 toMarch 2003. The chained higher-level indices in Table9.7 are calculated as:

I00:t=I98:Dec02Pw00(Dec 02)i IDec 02:t

i (9.16)

Because of the lack of additivity, the overall chain indexfor March 2003 (129.07), for example, cannot be cal-culated as the weighted arithmetic mean of the chainedhigher-level indices G and H using the weights fromDecember 2002.

CALCULATING CONSUMER PRICE INDICES IN PRACTICE

169

9.115 The introduction of new elementary aggregates.First, consider the situation in which new weights areintroduced and the index is chain linked in December2002.Theoverall coverageof theCPI isassumedto remainthe same, but certain items have increased sufficiently inimportance to merit recognition as new elementaryaggregates. Possible examples are the introduction ofnew elementary aggregates for mobile telephones orInternet access.

9.116 Consider the calculation of the new index fromDecember 2002 onwards, the new price reference period.The calculation of the new index presents no specialproblems and can be carried out using formula (9.10).However, if the weights are price-updated from, say,2000 to December 2002, difficulties may arise because theelementary aggregate for mobile telephones did not existprior to December 2002, so there is no price index withwhich to price-update the weight for mobile telephones.Prices for mobile telephones may have been recordedprior to December 2002, possibly within another ele-mentary aggregate (communications equipment), so itmay be possible to construct a price series which can beused for price-updating. Otherwise, price informationfrom other sources, such as purchasing power parity(PPP) surveys, business statistics, or industry sources,may have to be used. If no information is available, thenmovements in the price indices for similar elementaryaggregates may be used as proxies for price-updating.

9.117 The inclusion of a new elementary aggregatemeans that the next higher-level index contains a differ-ent number of elementary aggregates before and after thelinking. Therefore, the rate of change of the higher-levelindex whose composition has changed may be difficult tointerpret. However, failing to introduce new goods orservices for this reason would result in an index that doesnot reflect the actual dynamic changes taking place in theeconomy. If it is customary to revise the CPI backwards,then the prices of the new product and their weightsmight be introduced retrospectively. If the CPI is notrevised backwards, which is usually the case, there is littlethat can be done to improve the quality of the chainindex. In many cases, the addition of a single elementaryaggregate is unlikely to have a significant effect on the

next higher-level index into which it enters. If the addi-tion of an elementary aggregate is believed to have asignificant impact on the time series of the higher-levelindex, it may be necessary to discontinue the old seriesand commence a new higher-level index. These decisionscan only be made on a case-by-case basis.

9.118 The introduction of new higher-level indices. Itmay be necessary to introduce a new higher-level index inthe overall CPI. This situation may occur if the coverageof the CPI is enlarged or the grouping of elementaryaggregates is changed. It then needs to be decided whatthe initial value of the new higher-level index should bewhen it is included in the calculation of the overall CPI.Take as an example the situation in Table 9.7 and assumethat a new higher-level index from January 2003 has tobe included in the index. The question is then whatshould be the December 2002 value to which the newhigher-level index is to be linked. There are two options:

� Estimate the value in December 2002 that the newhigher-level index would have had with 1998 as theprice reference period, and link the new series fromJanuary 2003 onwards on to this value. This proce-dure will prevent any break in the index series.

� Use 100 in December 2002 as the starting point for thenew higher-level index. This simplifies the problemfrom a calculation perspective, although there remainsthe difficulty of explaining the index break to users.

In any case, major changes such as those just describedshould, so far as possible, be made in connection withthe regular reweighting and chaining in order to mini-mize disruptions to the index series.

9.119 A final case to consider concerns classificationchange. For example, a country may decide to changefrom a national classification to an international one,such as the Classification of Individual Consumptionaccording to Purpose (COICOP). The changes in thecomposition of the aggregates within the CPI may thenbe so large that it is not meaningful to link them. In suchcases, it is recommended that the CPI with the newclassification should be calculated backwards for at leastone year so that consistent annual rates of change can becalculated.

Table 9.7 Calculation of a chain index

Weight 1998 1998 Nov. 2002 Dec. 2002 Weight 2000 Dec. 2002 Jan. 2003 Feb. 2003 Mar. 2003

1998=100 December 2002=100Elementary price indicesA 0.20 100.00 120.00 121.00 0.25 100.00 100.00 100.00 102.00B 0.25 100.00 115.00 117.00 0.20 100.00 102.00 103.00 104.00C 0.15 100.00 132.00 133.00 0.10 100.00 98.00 98.00 97.00D 0.10 100.00 142.00 143.00 0.18 100.00 101.00 104.00 104.00E 0.30 100.00 110.00 124.00 0.27 100.00 103.00 105.00 106.00Total 100.00 119.75 124.90 100.00 101.19 102.47 103.34Higher-level indicesG=A+B+C 0.60 100.00 120.92 122.33 0.55 100.00 100.36 100.73 101.82H=D+E 0.40 100.00 118.00 128.75 0.45 100.00 102.20 104.60 105.20Total 100.00 119.75 124.90 100.00 101.19 102.47 103.34Chaining of higher-level indices to 1998=100G=A+B+C 0.60 100.00 120.92 122.33 0.55 122.33 122.78 123.22 124.56H=D+E 0.40 100.00 118.00 128.75 0.45 128.75 131.58 134.67 135.45Total 100.00 119.75 124.90 124.90 126.39 127.99 129.07

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

170

9.120 Partial reweighting. The weights for the ele-mentary aggregates may be obtained from a variety ofsources over a number of different periods. Conse-quently, it may not be possible to introduce all the newweighting information at the same time. In some cases, itmay be preferable to introduce new weights for someelementary aggregates as soon as possible after theinformation is received. The introduction of new weightsfor a subset of the overall index is known as partialreweighting.9.121 Partial reweighting has particular implications

for the practice of price-updating the weights. Weightinginformation may not be available for some elementaryaggregates at the time of rebasing. Thus, it may benecessary to consider price-updating not only the newweights, but also the old weights for those elementaryaggregates for which no new weights are available. Theweights for the latter may have to be price-updated overa long period, which, for reasons given earlier, may giverise to serious problems if relative quantities havechanged inversely to the relative price changes. Data onboth quantity and price changes should be sought beforeundertaking such updates. The disadvantage of partialreweighting is that the implicit quantities belong todifferent periods, so that the composition of the basketis obscure and not well defined.9.122 It may be concluded that the introduction of

new weights and the linking of a new series to an oldseries is not difficult in principle. The difficulties arise inpractice when trying to align weight and price referenceperiods and when deciding whether higher-level indicescomprising different elementary aggregates should bechained over time. It is not possible for this manual toprovide specific guidance on decisions such as these, butcompilers should consider carefully the economic logicand statistical reliability of the resulting chained seriesand also the needs of users. In order to facilitate thedecision-making process, careful thought should begiven to these issues in advance during the planning of areweighting exercise, paying particular attention towhich of the indices are to be published.9.123 Long-term and short-term links. Consider a

long-term chain index in which the weights are changedannually. In any given year, the current monthly indicesare first calculated using the latest set of availableweights, which cannot be those of the current year.However, when the weights for the year in questionbecome available subsequently, the monthly indices canthen be recalculated on the basis of the weights forthat same year. The resulting series can then be used inthe long-term chain index, rather than the originalindices first published. Thus, the movements of thelong-term chain index from, say, any one December tothe following December are based on weights of thatsame year, the weights being changed each December.This method has been developed by the Central Statis-tical Office of Sweden, where it is applied in the calcu-lation of the CPI. It is described in Swedish ConsumerPrice Index: A Handbook of Methods (Statistics Sweden,2001).9.124 Assume that each link runs from December to

December. The long-term index for month m of year Y

with December of year 0 as index reference period isthen calculated using the formula:

IDec0:mY=YY�1y=1

IDecy�1:Decy

!IDecY�1:mY ð9:17Þ

=IDec 0:Dec1IDec1:Dec2 . . . IDecY�2:DecY�1IDecY�1:mY

In the actual Swedish practice, a factor scaling the indexfrom December year 0 to the average of year 0 is mul-tiplied onto the right-hand side of the formula (9.19) tohave a full year as the reference period. The long-termmovement of the index depends on the long-term linksonly, as the short-term links are successively replaced bytheir long-term counterparts. For example, let the short-term indices for January to December 2001 be calculatedas:

IDec 00:m01=P

w00(Dec 00)i IDec 00:m01

i (9.18)

where Wi00(Dec00) are the weights from 2000 price-

updated to December 2000. When weights for 2001become available, this is replaced by the long-term link:

IDec00:Dec01=P

w01(Dec 00)i IDec 00:Dec 01

i (9.19)

where Wi01(Dec00) are the weights from 2001 price-

backdated to December 2000. The same set of weightsfrom 2001 price-updated to December 2001 is used inthe new short-term link for 2002:

IDec01:m02=P

w01(Dec01)i IDec01:m02

i : (9.20)

9.125 Using this method, the movement of the long-term index is determined by contemporaneous weights.The method is conceptually attractive because the weightsthat are most relevant for most users are those based onconsumption patterns at the time the price changesactually take place. The method takes the process ofchaining to its logical conclusion, at least assuming theindices are not chained more frequently than once a year.As the method uses weights that are continually revised toensure that they are representative of current consumerbehaviour, the resulting index also largely avoids thesubstitution bias that occurs when the weights are basedon the consumption patterns of some period in the past.The method may therefore appeal to statistical officeswhose objective is to estimate a cost of living index.

9.126 Finally, it may be noted that the methodinvolves some revision of the index first published. Insome countries, there is opposition to revising a CPI onceit has been first published, although it is standard prac-tice for other economic statistics, including the nationalaccounts, to be revised as more information and moreup-to-date information become available. This point isconsidered further below.

Decomposition of index changes9.127 Users of the index are often interested in how

much of the change in the overall index is attributable tothe change in the price of some particular good or groupof products, such as oil or food. Alternatively, there may

CALCULATING CONSUMER PRICE INDICES IN PRACTICE

171

be interest in what the index would be if housing orenergy were left out. Questions of this kind can beanswered by decomposing the change in the overall indexinto its constituent parts.

9.128 Assume that the index is calculated as inequation (9.10) or (9.11). The relative change of theindex from t�m to t can then be written as:

I0:t

I0:t�m � 1=

Pwbi I

0:t�mi I t�m:t

iPwbi I

0:t�mi

� 1 (9.21)

Hence, a sub-index from t�m to 0 enters the higher-levelindex with a weight of:

wbi I

0:t�miP

wbi I

0:t�mi

=wbi I

0:t�mi

I0:t�m (9.22)

The effect on the higher-level index of a change in asub-index can then be calculated as:

Effect=wbi I

0:t�mi

I0:t�mI0:ti

I0:t�mi

� 1

� �

=wbi

I0:t�m (I0:ti � I0:t�m

i ) (9.23)

With m=1, the formula (9.23) gives the effect of amonthly change; with m=12, it gives the effect of thechange over the past 12 months.

9.129 If the index is calculated as a chain index, as inequation (9.15), then a sub-index from t�m enters thehigher-level index with a weight of:

w0i I

k:t�mi

Ik:t�m =w0i (I

0:t�mi

�I0:ki )

(I0:t�m�I0:k)

(9.24)

The effect on the higher-level index of a change in a sub-index can then be calculated as:

Effect=w0i

Ik:t�m (Ik:ti � Ik:t�m

i )

=w0i

(I0:t�m�I0:k)

I0:ti � I0:t�m

i

I0:ki

� �(9.25)

It is assumed that t�m lies in the same link (i.e. t�mrefers to a period later than k). If the effect of a sub-index on a higher-level index is to be calculated across achain, the calculation needs to be carried out in twosteps: one with the old series up to the link period, andone from the link period to period t.

9.130 The calculation of the effect of a change in asub-index on a higher-level index is illustrated in Table9.8. The index is calculated in one link so that equation(9.25) may be applied for the decomposition. Forinstance, the effect in percentage points of the increasefor housing from January 2002 to January 2003 can becalculated as 0.25/118.6� (120.0� 110.0)=2.11 percen-tage points. This means that, of the increase of 10.03 percent in the all-items index, 2.11 percentage points can beattributed to the increase in the index for housing.

Some alternatives tofixed weight indices

9.131 Monthly CPIs are, typically, arithmeticweighted averages of the price indices for the elementaryaggregates, in which the weights are kept fixed over anumber of periods – which may range from 12 months tomany years. The repeated use of the same weights relat-ing to some past period b simplifies calculation proce-dures and reduces data collection requirements. It is alsocheaper to keep using the results from an old expendituresurvey than to conduct an expensive new one. Moreover,when the weights are known in advance of the pricecollection, the index can be calculated immediately afterthe prices have been collected and processed.

9.132 The longer the same weights are used, how-ever, the less representative they become of currentconsumption patterns, especially in periods of rapidtechnical change when new kinds of goods and servicesare continually appearing on the market and old onesdisappearing. This may undermine the credibility of anindex that purports to measure the rate of change in thetotal cost of a basket of goods and services typicallyconsumed by households. Such a basket needs to berepresentative not only of the households covered by theindex, but also of expenditure patterns at the time theprice changes occur.

9.133 Similarly, if the objective is to compile a cost ofliving index, the continuing use of the same fixed basketis likely to become increasingly unsatisfactory the longerthe same basket is used. The longer the same basket isused, the greater the upward bias in the index is likely tobecome. It is well known that the Laspeyres index has anupward bias compared with a cost of living index.However, a Lowe index between periods 0 and t withweights from an earlier period b will tend to exceed theLaspeyres between 0 and t by an amount that increasesthe further back in time period b is (see Chapter 15).

Table 9.8 Decomposition of index changes

Weight Index Change in %from Jan. 02to Jan. 03

Effect(contribution)

2000 Jan. 02 Jan. 03 % points oftotal change

% of totalchange

1 Food 0.30 100.0 120.0 130.0 8.33 2.53 25.212 Clothing 0.10 100.0 130.0 145.0 11.54 1.26 12.613 Housing 0.25 100.0 110.0 120.0 9.09 2.11 21.014 Transport 0.20 100.0 125.0 130.0 4.00 0.84 8.405 Miscellaneous 0.15 100.0 114.0 140.0 22.81 3.29 32.77All items 1.00 100.0 118.6 130.5 10.03 10.03 100.00

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

172

9.134 There are several possible ways of minimizingor avoiding the potential biases from the use of fixedweight indices. These are outlined below.9.135 Annual chaining. One way in which to mini-

mize the potential biases from the use of fixed-weightindices is obviously to keep the weights and the baseperiod as up to date as possible by frequent rebasing andchaining. Quite a number of countries have adopted thisstrategy and revise their weights annually. In any case, asnoted earlier, it would be impossible to deal with thechanging universe of products without some chaining ofthe price series within the elementary aggregates, even ifthe weights attached to the elementary aggregates remainfixed. Annual chaining eliminates the need to choose abase period, as the weight reference period is always theprevious year, or possibly the preceding year.9.136 Annual chaining with current weights. When the

weights are changed annually, it is possible to replace theoriginal weights based on the previous year, or years, bythose of the current year, if the index is revised retro-spectively as soon as information on the current year’sexpenditures becomes available. The long-term move-ments in the CPI are then based on the revised series.This is the method adopted by the Swedish StatisticalOffice, as explained above. This method could provideunbiased results.9.137 Other index formulae. When the weights are

revised less frequently, say every five years, anotherpossibility would be to use a different index formula forthe higher-level indices instead of an arithmetic averageof the elementary price indices. One possibility wouldbe a weighted geometric average. This is not subjectto the same potential upward bias as the arithmeticaverage. More generally, a weighted version of theLloyd–Moulton formula might be considered. This for-mula takes account of the substitutions that consumersmake in response to changes in relative prices, andshould be less subject to bias for this reason. It reduces tothe geometric average when the elasticity of substitutionis unity, on average. It is unlikely that such a formulacould replace the arithmetic average in the foreseeablefuture and gain general acceptance, if only because itcannot be interpreted as measuring changes in the valueof a fixed basket. It could, however, be compiled on anexperimental basis and might well provide a useful sup-plement to the main index. It could at least flag the extentto which the main index is liable to be biased and throwlight on its properties.9.138 Retrospective superlative indices. Finally, it is

possible to calculate a superlative index retrospectively.Superlative indices, such as Fisher and Tornqvist indices,treat both periods compared symmetrically and requireexpenditure data for both periods. Although the CPImay have to be some kind of Lowe index when it is firstpublished, it may be possible to estimate a superlativeindex later when much more information becomesavailable about consumers’ expenditures period by pe-riod. At least one office, the United States Bureau ofLabor Statistics, publishes such an index. The publica-tion of revised or supplementary CPIs raises mattersof statistical policy, although users readily accept revi-sions in other fields of economic statistics. Moreover,

users are already confronted with more than one CPI inthe European Union (EU) where the harmonized indexfor EU purposes may differ from the national CPI. Thusthe publication of supplementary indices which throwlight on the properties of the main index and which maybe of considerable interest to some users seems justifiedand acceptable.

Data editing9.139 This chapter has been concerned with the

methods used by statistical offices to calculate their CPIs.This concluding section considers the data editing car-ried out by statistical offices, a process that is very closelylinked to the calculation of the price indices for the ele-mentary aggregates. Data collection, recording andcoding – the data capture processes – are dealt with inChapters 5 to 7. The next step in the production of priceindices is data editing. Data editing is here meant tocomprise two steps:

– detection of possible errors and outliers;

– verifying and correction of data.

9.140 Logically, the purpose of detecting errors andoutliers is to exclude errors or outliers from the indexcalculation. Errors may be falsely reported prices, or theymay be caused by recording or coding mistakes. Also,missing prices because of non-response may be dealt withas errors. Possible errors and outliers are usually iden-tified as observations that fall outside some pre-specifiedacceptance interval or are judged to be unrealistic by theanalyst on some other ground. It may also be the case,however, that even if an observation is not identified as apotential error, it may actually show up to be false. Suchobservations are sometimes referred to as inliers. Some-times, by chance, the sampling may have captured anexceptional price change, which falls outside the accep-tance interval but has been verified as correct. In somediscussions of survey data, any extreme value is des-cribed as an outlier. The term is reserved here for extremevalues that have been verified as being correct.

9.141 When a possible error has been identified, itneeds to be verified whether it is in fact an error or not.This clarification can usually be made by asking therespondent to verify the price, or by comparison with theprice change of comparable items. If the value is in factan error, it needs to be corrected. This can be done easilyif the respondent can provide the correct price or, wherethis is not possible, by imputation or omitting the pricefrom the index calculation. If the value proves to becorrect, it should be included in the index. If it proves tobe an outlier, it can be accepted or corrected according toa pre-defined practice, e.g. omitting or imputation.

9.142 Although the power of computers providesobvious benefits, not all of these activities have to becomputerized. There should be a complete set of proce-dures and records that controls the processing of data,even though some or all of it may be undertaken withoutthe use of computers. It is not always necessary for all ofone step to be completed before the next is started. Ifthe process uses spreadsheets, for example, with defaultimputations predefined for any missing data, the index

CALCULATING CONSUMER PRICE INDICES IN PRACTICE

173

can be estimated and re-estimated whenever a newobservation is added or modified. The ability to examinethe impact of individual price observations on elemen-tary aggregate indices and the impact of elementaryindices on various higher-level aggregates is a usefulaid in all aspects of the computation and analyticalprocesses.

9.143 It is neither necessary nor desirable to applythe same degree of scrutiny to all reported prices. Theprice changes recorded by some respondents carry moreweight than others, and statistical analysts should beaware of this. For example, one elementary aggregatewith a weight of 2 per cent, say, may contain 10 prices,while another elementary aggregate of equal weight maycontain 100 prices. Obviously, an error in a reportedprice will have a much smaller effect in the latter, where itmay be negligible, while in the former it may cause asignificant error in the elementary aggregate index andeven influence higher-level indices.

9.144 There may be an interest in the individual ele-mentary indices, as well as in the aggregates built fromthem. Since the sample sizes used at the elementary levelmay often be small, any price collected, and error in it,may have a significant impact on the results for individualproducts or industries. The verification of reported datausually has to be done on an index-by-index basis, usingthe statistical analysts’ experience. Analysts will also needthe cooperation and support of the respondents to thesurvey to help explain unusual price movements.

9.145 Obviously, the design of the survey and ques-tionnaires also influences the occurrence of errors. Hence,price reports and questionnaires should be as clear andunambiguous as possible to prevent misunderstandingsand errors. Whatever the design of the survey, it isimportant to verify that the data collected are those thatwere requested initially. The survey questionnaire shouldprompt the respondent to indicate if the requested datacould not be provided. If, for example, a product is notproduced any more and thus is not priced in the currentmonth, a possible replacement would be requested alongwith details as to the extent of its comparability with theold one. In the event that a respondent cannot supply areplacement, there are a number of procedures for deal-ing with missing data (also discussed in Chapter 7).

Identifying possible errors and outliers9.146 One of the ways in which price surveys are

different from other economic surveys is that, althoughprices are recorded, the measurement concern is withprice changes. As the index calculations consist of com-paring the prices of matching observations from oneperiod to another, editing checks should focus on theprice changes calculated from pairs of observations,rather than on the reported prices themselves.

9.147 Identification of unusual price changes can beaccomplished by:

– non-statistical checking of input data;

– statistical checking of input data;

– output checking.

These will be described in turn.

9.148 Non-statistical checking of input data. Non-statistical checking can be undertaken by manuallychecking the input data, by inspection of the data pre-sented in comparable tables, or by setting filters.

9.149 When the price reports or questionnaires arereceived in the statistical office, the reported prices can bechecked manually by comparing these to the previouslyreported prices of the same items or by comparing themto prices of similar items from other outlets. While thisprocedure may detect obvious unusual price changes, it isfar from certain that all possible errors will be detected.It is also extremely time-consuming and, of course, itdoes not identify coding errors.

9.150 After the price data have been coded, the sta-tistical system can be programmed to present the data in acomparable tabular form. For example, a table showingthe percentage change for all reported prices from theprevious to the current month may be produced and usedfor detection of possible errors. Such tables may alsoinclude, for comparison, the percentage changes of pre-vious periods and 12-month changes. Most computerprograms and spreadsheets can easily sort the observa-tions according to, say, the size of the latest monthly rateof change, so that extreme values can easily be identified.It is also possible to group the observations by elementaryaggregates.

9.151 The advantage of grouping observations isthat it highlights potential errors so that the analyst doesnot have to look through all observations. A hierarchicalstrategy whereby all extreme price changes are firstidentified and then examined in context may save time,though the price changes underlying elementary aggre-gate indices, which have relatively high weights, shouldalso be examined in context.

9.152 Filtering is a method by which possible errors oroutliers are identified according to whether the pricechanges fall outside some predefined limits, such as plusor minus 20 per cent or even 50 per cent. This test shouldcapture any serious errors of data coding, as well as someof the cases where a respondent has erroneously reportedon a different product. It is usually possible to identifythese errors without reference to any other observations inthe survey, so this check can be carried out at the datacapture stage. The advantage of filtering is that it avoidsthe analyst having to look through a lot of individualobservations. The upper and lower limits may be set forthe latest monthly change, or change over some otherperiod. Again, they should take account of the contextof the price change, in that theymay be specified by item orelementary aggregates or higher-level indices. Largerchanges for items with prices that are known to be volatilemight be accepted without question. For example, formonthly changes, limits of plus or minus 10 per cent mightbe set for oil prices, while for professional services thelimits might be zero per cent to plus 5 per cent (as any pricethat falls is suspect), and for computers it might be �5 percent to zero per cent (as any price that rises is suspect).The limits can also be changed over time. If it is knownthat oil prices are rising, the limits could be 10 per cent to20 per cent, while if they are falling, the limits might be�10 per cent to �20 per cent. The count of failuresshould be monitored regularly to examine the limits. If too

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

174

many observations are being identified for review, thelimits will need to be adjusted, or the scope refined.9.153 The use of automatic deletion systems is not

advised, however. It is a well-recorded phenomenon inpricing that price changes for many products, especiallydurables, are not undertaken smoothly over time, butsaved up to avoid what are termed ‘‘menu costs’’ asso-ciated with making a price change. These relatively sub-stantial increases may take place at different times fordifferent models of products and have the appearance ofextreme, incorrect values. To delete a price change foreach model of the product as being ‘‘extreme’’ at the timeit occurs is to ignore all price changes for the industry.9.154 Statistical checking of input data. Statistical

checking of input data compares, for some time period,each price change with the change in prices in the sameor a similar sample. Two examples of such filtering aregiven here, the first based on non-parametric summarymeasures and the second on the log normal distributionof price changes.9.155 The first method involves tests based on the

median and quartiles of price changes, so they are unaf-fected by the impact of any single ‘‘extreme’’ observa-tion. Define the median, first quartile and third quartileprice ratios as RM, RQ1, and RQ3, respectively. Then anyobservation with a price ratio that is more than a certainmultiple C of the distance between the median and thequartile is identified as a potential error. The basicapproach assumes that price changes are normally dis-tributed. Under this assumption, it is possible to estimatethe proportion of price changes that are likely to falloutside given bounds expressed as multiples of C. Undera normal distribution, RQ1 and RQ3 are equidistantfrom RM. Thus, if C is measured as RM� (RQ1+RQ3)/2,then 50 per cent of observations would be expected to liewithin plus or minus C from the median. From the tablesof the standardized normal distribution this is equivalentto about 0.7 times the standard deviation (s). If, forexample, C was set to 6, the distance implied is about 4sof the sample, so about 0.17 per cent of observationswould be identified this way. With C=4, the corre-sponding figures are 2.7s, or about 0.7 per cent ofobservations. If C=3, the distance is 2.02s, so about 4per cent of observations would be identified.9.156 In practice, most prices may not change each

month and the share of observations identified as possibleerrors as a percentage of all changes would be undulyhigh. Some experimentation with alternative values of Cfor different industries or sectors may be appropriate. Ifthis test is to be used to identify possible errors for furtherinvestigation, a relatively low value of C should be used.9.157 To use this approach in practice, three mod-

ifications should be made:

� First, to make the calculation of the distance from thecentre the same for extreme changes on the low side aswell as on the high side, a transformation of the ratiosshould be made. The transformed distance for theratio of one price observation i, Si, should be:

Si=1� RM=Ri if 0<Ri <RM and

Si=Ri=RM � 1 if Ri �RM:

� Second, if the price changes are grouped closely to-gether, the distances between the median and quartilesmay be very small, so that many observations wouldbe identified that had quite small price changes. Toavoid this, some minimum distance, say 5 per cent formonthly changes, should also be set.

� Third, with small samples the impact of one obser-vation on the distances between the median andquartiles may be too great. Because sample sizes forsome elementary indices are small, samples for similarelementary indices may need to be grouped together.

9.158 For a detailed presentation of this method, seeHidiroglou and Berthelot (1986). The method can beexpanded to also take into account the level of the prices.Thus, for example, a price increase from 100 to 110 willbe attributed a different weight from the weight attrib-uted to a price increase from 10 to 11.

9.159 An alternative method can be used if it isthought that the price changes may be distributed lognormally. To apply this method, the standard deviationof the log of all price changes in the sample (excludingunchanged observations) is calculated and a goodness offit test (w2) is undertaken to identify whether the dis-tribution is log normal. If the distribution satisfies thetest, all price changes outside two times the exponentialof the standard deviation are highlighted for furtherchecking. If the test rejects the log normal hypothesis,all the price changes outside three times the exponentialof the standard deviation are highlighted. The samecaveats mentioned before about clustered changes andsmall samples apply.

9.160 The second example is based on the Tukeyalgorithm. The set of price ratios is sorted and thehighest and lowest 5 per cent flagged for further atten-tion. In addition, having excluded the top and bottom 5per cent, exclude the price ratios that are equal to 1 (nochange). The arithmetic (trimmed) mean (AM) of theremaining price ratios is calculated. This mean is used toseparate the price ratios into two sets, an upper and alower one. The upper and lower ‘‘mid-means’’, that is,the means of each of these sets (AML, AMU), are thencalculated. Upper and lower Tukey limits (TL, TU) arethen established as the mean plus (minus) 2.5 times thedifference between the mean and the mid-means:

TU=AM+2:5 (AMU �AM)

TL=AM� 2:5 (AM�AML)

Then all those observations that fall above TU andbelow TL are flagged for attention.

9.161 This is a simpler method similar to that basedon the normal distribution. Since it excludes all cases ofno change from the calculation of the mean, it is unlikelyto produce limits that are very close to the mean, so thereis no need to set a minimum difference. Its success willalso depend on there being a large number of observa-tions on the set of changes being analysed. Again, it willoften be necessary to group observations from similarelementary indices. For any of these algorithms, thecomparisons can be made for any time periods, includingthe latest month’s changes or longer periods, in partic-ular, 12-month changes.

CALCULATING CONSUMER PRICE INDICES IN PRACTICE

175

9.162 The advantage of these two models of filteringcompared to the simple method of filtering is that foreach period the upper and lower limits are determined bythe data and hence are allowed to vary over the year,given that the analyst has decided on the value of theparameters entering the models. A disadvantage is that,unless the analyst is prepared to use approximationsfrom earlier experience, all the data have to be collectedbefore the filtering can be undertaken. Filters should beset tightly enough so that the percentage of potentialerrors that turn out to be real errors is high. As with allautomatic methods, the flagging of an unusual observa-tion is for further investigation, as opposed to automaticdeletion.

9.163 Checking by impact, or data output checking.Filtering by impact, or output editing, is based on cal-culating the impact that an individual price change hason an index to which it contributes. This index can be anelementary aggregate index, the total index, or someother aggregate index. The impact that a price changehas on an index is its percentage change times its effec-tive weight. In the absence of sample changes, the cal-culation is straightforward: it is the nominal (referenceperiod) weight, multiplied by the price relative, anddivided by the level of the index to which it is con-tributing. So the impact on the index I of the change ofthe price of product i from time t to t+1 is ±wi ( pt+1 /pt)/It where wi is the nominal weight in the base period.A minimum value for this impact can be set, so that allprice changes that cause an impact greater than thischange can be flagged for review. If index I is an ele-mentary index, then all elementary indices may bereviewed, but if I is an aggregative index, prices thatchange by a given percentage will be flagged or notdepending on how important the elementary index towhich they contribute is in the aggregate.

9.164 At the lowest level, the appearance and dis-appearance of products in the sample cause the effectiveweight of an individual price to change substantially.The effective weight is also affected if a price observationis used as an imputation for other missing observations.The evaluation of effective weights in each period ispossible, though complicated. As an aid to highlightingpotential errors, the nominal weights, as a percentage oftheir sum, will usually provide a reasonable approx-imation. If the impact of 12-month changes is requiredto highlight potential errors, approximations are theonly feasible filters to use, as the effective weights willvary over the period.

9.165 One advantage of identifying potential errorsin this way is that it focuses on the results. Anotheradvantage is that this form of filtering also helps theanalyst to describe the contributions to change in theprice indices. In fact, much of this kind of analysis isdone after the indices have been calculated, as the analystoften wishes to highlight those indices that have con-tributed the most to overall index changes. Sometimesthe analysis results in a finding that particular industrieshave a relatively high contribution to the overall pricechange, and that is considered unrealistic. The change istraced back to an error, but it may be late in the pro-duction cycle and jeopardize the schedule release date.

There is thus a case for identifying such unusual con-tributions as part of the data editing procedures. Thedisadvantage of this method is that an elementary index’schange may be rejected at that stage. It may be necessaryto over-ride the calculated index, though this shouldbe only a stopgap measure until the index sample isredesigned.

Verifying and correcting data9.166 Some errors, such as data coding errors, can be

identified and corrected easily. Ideally, these errors arecaught at the first stage of checking, before they need tobe viewed in the context of other price changes. Dealingwith other potential errors is more difficult. Many resultsthat fail a data check may be judged by the analyst to bequite plausible, especially if the data checking limits arebroad. Some potential failures may only be resolved bychecking the data with the respondent.

9.167 If a satisfactory explanation can be obtainedfrom the respondent, the data can be verified or cor-rected. If not, procedures may differ. Rules may be estab-lished that if a satisfactory explanation is not obtained,then the reported price is omitted from the index calcu-lation. Alternatively, it may be left to the analyst to makethe best judgement as to the price change. If an analystmakes a correction to some reported data without veri-fying it with the respondent, the change may subse-quently cause problems with the respondent. If therespondent is not told of the correction, the same errormay persist in the future. The correct action depends ona combination of confidence in the analysts, the revisionpolicy of the survey, and the degree of communicationwith respondents. Most statistical organizations do notwant to burden respondents unduly.

9.168 In many organizations, a disproportionateshare of activity is devoted to identifying and followingup potential errors. If this practice leads to little changein the results, as a result of most reports finally beingaccepted, then the ‘‘bounds’’ on what are consideredto be extreme values should be relaxed. More errorsare likely to be introduced by respondents failing toreport changes that occur than from wrongly reportingchanges, and the good will of respondents should not beunduly undermined.

9.169 Generally, the effort spent on identifyingpotential errors should not be excessive. Obvious mis-takes should be caught at the data capture stage. Thetime spent in identifying observations to query, unlessthey are highly weighted and excessive, is often betterspent treating those cases in the production cycle wherethings have changed – quality changes or unavailableprices – and reorganizing activities towards maintainingthe relevance of the sample, and checking for errors ofomission.

9.170 If the price observations are collected in a waythat prompts the respondent with the previously repor-ted price, the respondent may report the same price as amatter of convenience. This can happen even though theprice may have changed, or even when the particularproduct being surveyed is no longer available. As pricesfor many items do not change frequently, this kind of

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

176

error is unlikely to be spotted by normal checks. Oftenthe situation comes to light when the contact at theresponding outlet changes and the new contact has dif-ficulty in finding something that corresponds to the pricepreviously reported. It is advisable, therefore, to keep arecord of the last time a particular respondent reporteda price change. When that time has become suspiciouslylong, the analyst should verify with the respondent thatthe price observation is still valid. What constitutes toolong will vary from product to product and the level ofoverall price inflation, but, in general, any price that hasremained constant for more than a year is suspect.9.171 Treatment of outliers. Detection and treatment

of outliers (extreme values that have been verified asbeing correct) is an insurance policy. It is based on thefear that a particular data point collected is exceptionalby chance, and that if there were a larger survey, or evena different one, the results would be less extreme. Thetreatment, therefore, is to reduce the impact of theexceptional observation, though not to ignore it as, afterall, it did occur. The methods to test for outliers are thesame as those used to identify potential errors by sta-tistical filtering, described above. For example, upperand lower bounds of distances from the median pricechange are determined. In this case, however, whenobservations are found outside those bounds, they maybe changed to be at the bounds or imputed by the rateof change of a comparable set of prices. This outlieradjustment is sometimes made automatically, on thegrounds that the analyst by definition has no additionalinformation on which to base a better estimate. Whilesuch automatic adjustment methods are employed, thismanual proposes caution in their use. If an elementaryaggregate is relatively highly weighted and has a rela-tively small sample, an adjustment may be made. Thegeneral prescription should be to include verified prices;the exception should be to dampen them.9.172 Treatment of missing price observations. It is

likely that not all the requested data will have beenreceived by the time the index needs to be calculated. Itis generally the case that missing data turn out to bedelayed. Sometimes, the respondent may report that aprice cannot be reported because neither the product,nor any similar substitute is being made any more.Sometimes, of course, what started apparently as a latereport becomes a permanent loss to the sample. Differ-ent actions need to be taken depending on whether thesituation is temporary or permanent.9.173 For temporarily missing prices, the most

appropriate strategy is to minimize the occurrence ofmissing observations. Survey reports are likely to come in

over a period of time before the indices need to be cal-culated. In many cases, they follow a steady routine; somerespondents will tend to file quickly, others typically willbe later in the processing cycle. An analyst should becomefamiliar with these patterns. A computerized data capturesystem can flag those reports that appear to be later thanusual, well before the processing deadline. Also, somedata are more important than others. Depending on theweighting system, some respondents may be particularlyimportant, and important products should be flagged asrequiring particular scrutiny.

9.174 For those reports for which no estimate can bemade, two basic alternatives are considered here (seeChapter 7 for a full range of approaches): imputation,preferably targeted, in which the missing price changeis assumed to be the same as some other set of pricechanges; or an assumption of no change, as the precedingperiod’s price is used. This latter procedure ignores thefact that some prices will prove to have changed, and ifprices are generally moving in one direction, this willmean that the change in the index will be understated.It is not advised. However, if the index is periodicallyrevised, this approach will lead to fewer subsequentrevisions than imputations, since for most products,prices do not generally change in any given period. Stand-ard imputation is to base the estimate of the missing priceobservation on the change of some similar group ofobservations.

9.175 There will be situations where the price ispermanently missing because the product no longerexists. As there is no replacement for the missing price,an imputation will have to be made for each period untileither the sample is redesigned or until a replacementcan be found. It is, therefore, more important than inthe case of temporarily missing reports, and requirescloser attention.

9.176 The missing price can be imputed by using thechange in the remaining price observations in the ele-mentary aggregate, which has the same effect as remov-ing the missing observation from the sample, or by thechange in a subset of other price observations for com-parable items. The series should be flagged as beingbased on imputed values.

9.177 Samples are designed on the basis that theproducts chosen for observation are representative of awider range of products. Imputations for permanentlymissing prices are indications of weakness in the sample,and their accumulation is a signal that the sample shouldbe redesigned. For indices where there are known to be alarge number of disappearances in the sample, the needfor replacements should be anticipated.

CALCULATING CONSUMER PRICE INDICES IN PRACTICE

177

10SOME SPECIAL CASES

Introduction10.1 This chapter focuses on a number of expendi-

ture areas that pose particular problems for price indexcompilers, both in terms of identifying an agreed con-ceptual approach and also overcoming practical mea-surement difficulties. Six areas have been selected fordiscussion, mainly from the service sector. They are:

– owner-occupied housing;

– clothing;

– telecommunication services;

– financial services;

– real estate agency services;

– property insurance services.

10.2 This chapter is therefore structured into sixsections, in turn dealing with the problem areas listedabove. Under each section, any necessary theoreticalconsiderations are discussed and relevant measurementissues explored. Where appropriate, illustrative exam-ples of alternative approaches to the measurement ofweights or price changes are provided, and the advan-tages and disadvantages are outlined.10.3 It is important to note that the examples shown

are neither definitive nor prescriptive, but rather providebroad guidance as to how the problem areas can be ap-proached. User requirements, data availability and thestatistical resources available are important factors thatneed to be taken into consideration in choosing anappropriate methodology. Market conditions and pro-duct market regulations, which can differ widely betweencountries, also have a critical impact on the choice ofmethod.

Owner-occupied housing10.4 The treatment of owner-occupied housing in

consumer price indices (CPIs) is arguably the most dif-ficult issue faced by CPI compilers. Depending on theproportion of the reference population that are owner-occupiers, the alternative conceptual treatments can havea significant impact on the CPI, affecting both weightsand, at least, short-term measures of price change.10.5 Ideally, the approach chosen should align with

the conceptual basis that best satisfies the principalpurpose of the CPI. However, the data requirements forsome (or even all) of these options may be such that it isnot feasible to adopt the preferred treatment. Equallyimportant, it may be difficult to identify a single principalpurpose for the CPI. In particular, the dual use of CPIsas both macroeconomic indicators and also for indexa-

tion purposes can lead to clear tensions in designingan appropriate treatment for owner-occupied housingcosts. In these circumstances, itmay be necessary to adopta treatment that is not entirely consistent with theapproach adopted for other items in the CPI. In somecountries, the difficulties in resolving such tensions haveled to the omission of owner-occupied housing from theCPI altogether or the publication of more than oneindex.

10.6 The remainder of this section discusses theconceptual basis and data requirements for the use,payments and acquisitions approaches in turn.

Use10.7 The general objective of this approach is to

measure the change over time in the value of the flow ofshelter services consumed by owner-occupiers. Detailedapproaches fall under one of two broader headings: usercost or rental equivalence.

10.8 The user cost approach attempts to measure thechanges in the cost to owner-occupiers of using thedwelling. In the weighting base period, these costscomprise two elements: recurring actual costs, such asthose for repairs and maintenance, and property taxes;and the opportunity cost of having money tied up in thedwelling rather than being used for some other purpose.At its simplest, and where houses are purchased out-right, this latter element is represented by the rate ofreturn available on alternative assets. More usually,house purchase will be at least part financed throughmortgage borrowing. In this case, opportunity cost canbe viewed as an average of interest rates on mortgagesand alternative assets, weighted by the proportion of thepurchase price borrowed and paid outright, respectively.

10.9 Estimation of the base period weight for recur-ring actual costs such as expenditures on repairs andmaintenance is relatively straightforward and generallyobtainable from household expenditure surveys. Simi-larly, the construction of price measures for these itemspresents few difficulties.

10.10 Estimation of the base period weight foropportunity costs is more complicated and will requiremodelling. One approach is to assume that all owner-occupiers purchased their dwellings outright at thebeginning of the period and sold them at the end. Duringthe period their opportunity costs comprise the amountof interest forgone (i.e. the amount of interest they mighthave earned by investing this money elsewhere) anddepreciation. Offsetting these costs would be any capitalgains earned on the sale of the dwellings. Construction ofthe required measures of price change is likewise quite

179

complicated (see Chapter 23 for a more complete dis-cussion) and, particularly for the depreciation element, agood deal of imputation is required. Allowing for housepurchases part financed by mortgage borrowing, a typi-cal formula for user cost (UC) is:

UC=rM+iE+D+RC � K

where M and E represent mortgage debt and equity inthe home, and r and i represent mortgage interest ratesand the rate of return available on alternative assets,respectively. D is depreciation, RC other recurring costsand K capital gains.

10.11 No national statistical office is currently usingthe full user cost approach. This partly reflects theconceptual and methodological complexity of the mea-sure, which may also make it difficult to obtain wide-spread public support for the approach. For this reason,the methodology is not discussed in detail here. It is,however, worth noting that both the weights and theongoing measures of price change are significantlyinfluenced by the relative rate of change in house prices.Since the user cost formula is typically dominated bycapital gains and interest rates, where house priceinflation exceeds nominal interest rates the user costweight is likely to be negative (implying a negative pricefor user cost).

10.12 In practice, it is possible to avoid some of thesedifficulties by adopting a variant or a narrower definitionof user cost. For example, some countries have adopted avariant of the user cost approach focusing on grossmortgage interest payments and depreciation, in partbecause these items are readily recognizable as key costsby home owners. The former may be viewed as the costof retaining housing shelter today, while the depreciationelement represents current expenditure that would berequired to offset the deterioration and obsolescence indwellings that would otherwise occur over time. Meth-odologies for calculating actual average mortgage inter-est payments for index households are described in thesection on the payments approach to owner-occupiedhousing costs, below.

10.13 Depreciation is a gradual process and so is bestrepresented by the amount that needs to be put aside yearby year as opposed to actual expenditures (which willtypically be large but infrequent). The base period weightfor depreciation may be estimated from the currentmarket value of the owner-occupied housing stockexcluding land values, multiplied by an average rate ofdepreciation. The latter may be derived from nationalaccounts estimates of housing capital consumption.Imputed this way, the appropriate price indicator shouldideally be an index of house prices excluding land ratherthan an index of the costs of renovation work.

10.14 The rental equivalence approach attempts tomeasure the change in the price of the housing serviceconsumed by owner-occupiers by estimating the marketvalue of those services. In other words, it is based onestimating how much owner-occupiers would have topay to rent their dwelling. Under this approach, it wouldbe inappropriate also to include those input costs nor-mally borne by landlords such as dwelling insurance,major repair and maintenance, and property taxes as

this would involve an element of double counting. Therental equivalence approach is recommended in SNA1993 for measuring household consumption and is alsoused in constructing international comparisons of livingstandards.

10.15 Deriving the weight for rental equivalencerequires estimating how much owner-occupiers wouldhave paid in the weighting base period to rent theirdwellings. This is not something that owner-occupierscan normally be expected to estimate reliably in a house-hold expenditure survey. In principle, however, it can beestimated by matching the dwellings of owner-occupierswith comparable dwellings that are being rented andapplying those rents to the owner-occupied dwellings.

10.16 In practice, this raises a number of problems,particularly in countries where the overall size of theprivate rental market is small or if rented housing is of adifferent type from owner-occupied housing in terms ofgeneral quality, age, size and location. Direct imputa-tion from actual rents may also be inappropriate ifthe rental market is subject to price control. In addition,owner-occupiers may be considered to derive significantadditional utility from features such as security of tenureand the ability to modify the dwelling, implying a needto make additional adjustments to the initial imputa-tions.

10.17 In those countries where the reference popu-lation for the CPI corresponds to all resident households,the estimation problem is identical to that faced by thenational accountants and a collaborative approachwouldbe beneficial.

10.18 The corresponding price series for owner-occupiers’ rent can be derived from an actual rent index,except where such rents are subject to price control.Depending on both the relative significance of owner-occupiers to renters and the composition of the twomarkets in terms of dwelling characteristics, any existingrent surveys may need to be modified to meet the par-ticular requirements of an owners’ equivalent rent series.If the total value of owners’ equivalent rent is signifi-cantly larger than actual rents, the absolute size of theexisting price sample may be deemed insufficient. If thecharacteristics of owner-occupied dwellings differ sig-nificantly from the overall rental market, the existingrent survey may also require stratifying more finely (e.g.by type and size of dwelling, and by location). The pricemeasures for the different strata can then be given dif-ferent weights when calculating the actual rents and theowners’ equivalent rent series, respectively.

10.19 While it may be acceptable to include sub-sidized and controlled prices in the actual rent series,these should not be used in calculating the owners’equivalent rent series. Given the increased significance ofrent prices in the overall index, it may also be necessaryto pay greater attention to the measurement of pricechange for individual properties when tenancies change.As this often presents landlords with an opportunity torefurbish properties and increase rents, the practice ofregarding the whole of all such price changes as arisingfrom quality change should be avoided. Furthermore,the rent series may need to be quality-adjusted to takeaccount of ongoing depreciation to housing structures.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

180

This question is discussed in Chapter 23, paragraphs23.69 to 23.78.

Payments10.20 The item domain for a payments index is

defined by reference to actual outlays made by house-holds to gain access to consumer goods and services.The set of outlays peculiar to owner-occupiers in theweighting base period includes:

– down payments or deposits on newly purchaseddwellings;

– legal and real estate agency fees payable on propertytransfers;

– repayments of mortgage principal;

– mortgage interest payments;

– alterations and additions to the dwelling;

– insurance of the dwelling;

– repair and maintenance of the dwelling;

– property rates and taxes.

10.21 While it is conceivable to include all of theseitems in the index, it is generally agreed that at least somerepresent capital transactions that ought to be excludedfrom a CPI. For example, while down payments andrepayments of mortgage principal result in a runningdown of household cash reserves, they also result in thecreation of a real asset (at least part of a dwelling) or inthe reduction of a liability (the amount of mortgage debtoutstanding). Similarly, any cash expenditures on alter-ations and additions result in a running down of cashreserves offset by increases in dwelling values. In otherwords, those transactions which result in no net changeto household balance sheets should be excluded.10.22 The remaining items can be regarded as cur-

rent expenditures which do not result in any offsettingadjustments to household balance sheets. It is thereforeconsidered appropriate that these items be included in apayments-based CPI. By defining a payments index inthis way, it is clear that the aggregate payments equal ahousehold’s source of funds which comprise incomeafter tax (wages, transfers, property income, insuranceclaims, etc.) and net savings (as a balancing item). It isfor this reason that a payments-based CPI is commonlyconsidered to be the best construct for assessing changesin net money incomes over time.10.23 Estimation of gross expenditures on these items

in the weighting base period is readily achievable via ahousehold expenditure survey, as the items are generallyreportable by households. The construction of priceindices for real estate agency fees and insurance is dis-cussed later in this chapter. Indices for repair and main-tenance, and property rates and taxes are not consideredparticularly problematic so are not discussed here. Theremainder of this section is therefore devoted to the con-struction of price measures for mortgage interest charges.10.24 The construction of price indices for mortgage

interest charges is not altogether straightforward. Thedegree of complexity will vary from country to countrydepending on the operation of domestic financial mar-kets and the existence (or otherwise) of any income tax

provisions applying to mortgage interest payments.What follows therefore is a description of an overallobjective and an illustrative methodology for producingthe required index in the most straightforward of cases.The methodology will require modifying to account foradditional complexities that may be encountered insome countries.

10.25 The general approach may be summarizedbriefly as follows. Under a fixed basket approach, theobjective of the index is to measure the change over timein the interest that would be payable on a set of mort-gages equivalent to those existing in the weighting baseperiod. This base stock of mortgages will, of course,vary widely in age, from those taken up in the baseperiod itself to those taken up many years previously. Incompiling a fixed base index, the distribution of mort-gages by age is required to be held constant.

10.26 The amount of interest payable on a mortgageis determined by applying some rate of interest, ex-pressed as a percentage, to the monetary value of debt.Changes in mortgage interest charges over time thereforecan, in principle, be measured by periodically collectinginformation on a representative selection of mortgageinterest rates, using these to derive an average interestrate, and then applying this to an appropriate debt fig-ure. At least for standard variable rate mortgages,interest due on the revalued stock of base period mort-gages may be derived simply with reference to currentmortgage interest rates.

10.27 The main problem then is in determining theappropriate debt figure in each of the comparison peri-ods. Since the real value of any monetary amount ofdebt varies over time according to changes in the pur-chasing power of money, it is not appropriate to use theactual base period monetary value of debt in calcula-tions for subsequent periods. Rather, it is necessary firstto update that monetary value in each comparisonperiod so that it remains constant in real terms (i.e. sothat the quantities underpinning the base period amountare held constant).

10.28 In order to do this, it is necessary to form atleast a theoretical view of the quantities underpinning theamount of debt in the base period. The amount ofmortgage debt outstanding for a single household in thebase period depends on the original house purchase priceand loan-to-value ratio, and also the rate of repaymentof principal since the house was purchased. An equiva-lent value of debt can be calculated in subsequent com-parison periods by holding constant the age of the debt,the original value of the debt (as some fixed proportionof the total value of the dwelling when the mortgage wasinitially entered into) and the rate of repayment of theprincipal (as some proportion of the original debt), andapplying these factors to house prices for periods cor-responding to the age of the debt.

10.29 To illustrate, suppose a base period householdpurchased a dwelling five years earlier for $100,000 andfinanced 50 per cent by mortgage. If, between the timeof purchase and the base period, the household repaid20 per cent of this debt, then the outstanding debt onwhich base period interest charges were calculated wouldhave been $40,000. Now move to some subsequent

SOME SPECIAL CASES

181

comparison period and suppose that it is known thathouse prices doubled between the period when thehousehold originally purchased and the period five yearsprior to the comparison period. The equivalent amountof outstanding debt in the comparison period would becalculated by first taking 50 per cent of the revaluedhouse price (of $200,000) to give $100,000, and thenreducing this by the principal repayment rate (of 20 percent) to give $80,000.

10.30 Under these assumptions, it is clear that thecomparison period value of outstanding debt may beestimated directly from the base period value of out-standing debt solely on the basis of house price move-ments between five years prior to the base period and fiveyears prior to the comparison period. In other words,while preservation of original debt/equity ratios andrates of repayment of principal help in understanding theapproach, estimates of these variables are not strictlyrequired to calculate the required comparison perioddebt. All that is required is the value of the outstandingdebt in the base period, the age of that debt and a suit-able measure of changes in dwelling prices.

10.31 Now suppose that all mortgages are of thevariable rate type, and that average nominal interestrates rose from 5 per cent in the base period to 7.5 percent in the comparison period. Interest payments in thetwo periods can be calculated as $2,000 and $6,000respectively, and so the mortgage interest paymentsindex for the comparison period is 300.0. An identical re-sult may of course be found directly from index numberseries for debt and nominal interest rates. The mortgageinterest charges index equals the debt index multiplied bythe nominal interest rate index divided by 100. In thisexample, the debt index equals 200.0 and the nominalinterest rate index equals 150.0. Therefore the mortgageinterest rate index equals (200.0�150.0)/100 or 300.0.This simple example also serves to illustrate the veryimportant point that percentages (interest rates, taxes,etc.) are not prices and cannot be used as if they were.Percentages must be applied to some monetary value inorder to determine a monetary price.

10.32 While the single-household example shownabove is useful in explaining the basic concepts, it isnecessary to devise a methodology that can be employedto calculate a mortgage interest charges index for thereference population as a whole. The main complicationwhen moving from the single-household to the many-household case is the fact that the age of the debt willvary across households. Given the importance of re-valuing base period debt to maintain a constant age, thisis no trivial matter. While it is conceivable that infor-mation on the age of mortgage debt could be collected inhousehold expenditure surveys, the additional respon-dent burden and the generally small number of house-holds reporting mortgages often serve to make estimatesfrom this source unreliable. Another option is toapproach a sample of providers of mortgages (banks,building societies, etc.) for an age profile of their currentmortgage portfolio. This type of data is normallyavailable and is generally reliable.

10.33 Table 10.1 illustrates how an aggregate debtprice index can be constructed. For the purpose of

illustrating the methodology, some simplifying assump-tions have been made:

� The index is assumed to be quarterly rather thanmonthly.

� The oldest age of mortgage debt is assumed to bebetween three and four years (in practice, it is nor-mally the case that debt older than eight years isinsignificant).

� Each annual cohort of debt is assumed to be dis-tributed evenly across the year.

� A quarterly index of dwelling prices (new and second-hand dwellings, including land) is available.

10.34 Column (1) of Table 10.1(a) contains indexnumbers for dwelling prices extending back four yearsprior to the base period for the debt series (quarter 1 ofyear 0). Column (2) contains a four-quarter movingaverage of the first series – this is required to reflect‘‘yearly’’ prices to correspond with the debt cohorts,which are only available in yearly age groups in thisexample (if quarterly cohorts were available it would notbe necessary to calculate the moving average series).

10.35 Columns (1) to (4) of Table 10.1(b) containthe calculated debt indices for each cohort re-referencedto Y0 Q1=100. These series are simple transformationsof the series in column (2) of Table 10.1(a), each with adifferent starting point. For example, the debt series forthat cohort contracted for between three and four yearsago has as a starting point the index number from Y�4Q4 (i.e. 113.9) in column (2), and the series for debt agedbetween two and three years starts from Y�3 Q4 (i.e.118.7) and so on. Column (5) of Table 10.1(b) containsthe aggregate debt index which is derived by weight-ing together the indices for the four age cohorts. Theweights are derived from data from financial institutionson debt outstanding by age, revalued to period Y0 Q1prices.

10.36 A nominal mortgage interest rate index num-ber series is obtained by calculating average quarterlyinterest rates on variable rate mortgages from a sampleof lending institutions (starting in period Y0 Q1) andpresenting them in index number form. The nominalinterest rate series can then be combined with the debtseries to calculate the final mortgage interest rate char-ges series, as illustrated in Table 10.2.

10.37 The construction of equivalent indices forfixed interest mortgages is more complicated in so far asan interest charges index has to be calculated separatelyfor each age cohort of debt to reflect the fact that interestpayable today, on a loan four years old, depends onthe interest rate prevailing four years ago. This requiresthe compilation of a nominal fixed interest rate indexextending back as far as the dwelling price series. To theextent that the interest rates charged on fixed interestloans also depend on the duration of the loan, calcula-tion of the nominal fixed interest rate series is also morecomplex. The additional complexity of these indices maymake the construction of a mortgage interest chargesindex impractical for countries where fixed interest ratemortgages predominate.

10.38 The construction of the index for mortgageinterest payments is predicated on the assumption that

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

182

the purpose of the mortgage is to finance the purchase ofthe dwelling (hence revaluation of debt by changes indwelling prices). However, it is increasingly common,particularly in developed countries, for households todraw down on the equity they have in their home. Thatis, households may take new or additional mortgages,or redraw part of the principal already paid to financeother activities, for example to purchase a large con-sumer durable such as a car or a boat, to go on holidayor even to purchase stocks and bonds. If these alter-native uses of the funds made available by way ofmortgages are significant, it may be appropriate toregard at least some proportion of mortgage interestcharges as the cost of a general financial service rather

than a housing cost. For that proportion of the debtdeemed to be used for other purposes, it would be moreappropriate to use a general index of price inflation fordebt revaluation purposes.

Acquisitions10.39 The item domain for an acquisitions index is

defined as all those consumer goods and servicesacquired by households. Those countries which compiletheir CPIs on an acquisitions basis have generally con-cluded that the principal purpose of their CPI is toprovide a measure of price inflation for the householdsector as a whole. Based on the view that price inflation isa phenomenon peculiar to the operation of markets, thedomain is also normally restricted to those consumergoods and services acquired in monetary transactions.That is, consumer goods and services provided at no costto households by governments and non-profit institu-tions serving households are excluded.

10.40 The expenditures of owner-occupiers thatcould be included in an acquisitions index are:

– net purchases of dwellings (i.e. purchases less sales bythe reference population);

– direct construction of new dwellings;

– alterations and additions to existing dwellings;

Table 10.1 Calculation of a mortgage debt series(a) Dwelling price index

Year Quarter Original houseprice index(1)

Four-quarter movingaverage of (1)(2)

Y�4 Q1 111.9Q2 112.8Q3 114.7Q4 116.2 113.9

Y�3 Q1 117.6 115.3Q2 118.5 116.8Q3 119.0 117.8Q4 119.8 118.7

Y�2 Q1 120.1 119.4Q2 120.3 119.8Q3 120.5 120.2Q4 122.0 120.7

Y�1 Q1 122.3 121.3Q2 123.8 122.2Q3 124.5 123.2Q4 125.2 124.0

Y0 Q1 125.9 124.9Q2 126.1 125.4Q3 127.3 126.1Q4 129.2 127.1

(b) Debt index

Year Quarter Age of debt

3–4 years 2–3 years 1–2 years 0–1 yearWt = 10% Wt = 20% Wt = 30% Wt = 40% Weighted average(1) (2) (3) (4) (5)

Y0 Q1 100.0 100.0 100.0 100.0 100.0Q2 101.2 100.6 100.7 100.7 100.7Q3 102.5 100.9 101.6 101.1 101.4Q4 103.4 101.3 102.2 101.7 101.9

Table 10.2 Calculation of a mortgage interest chargesseries

Year Quarter Debtindex

Nominalinterestratesindex

Mortgageinterestcharges index(1)� (2)/100

(1) (2) (3)

Y0 Q1 100.0 100.0 100.0Q2 100.7 98.5 99.2Q3 101.4 100.8 102.2Q4 101.9 101.5 103.4

SOME SPECIAL CASES

183

– legal and real estate agency fees payable on propertytransfers;

– repair and maintenance of dwellings;

– insurance of dwellings;

– property rates and taxes.

10.41 The construction of price indices for real estateagency fees and insurance is discussed later in thischapter. Indices for repair and maintenance, and prop-erty rates and taxes are not considered particularlyproblematic so are not discussed here. The remainder ofthis section is therefore devoted to a discussion of theissues involved in constructing measures for dwellingpurchase, construction, and alterations and additions.An advantage of the acquisitions approach is that, con-sistent with the treatment of most other goods and ser-vices in the CPI, the owner-occupied housing index willreflect the full price paid for housing. Moreover, it is notaffected by methods of financing for house purchase.

10.42 As CPIs are constructed to measure pricechange for a group of households in aggregate (thereference or target population), the index should notinclude any transactions that take place between thosehouseholds. In the case of an index covering all privatehouseholds, the weight should only reflect net additionsto the household sector owner-occupied housing stock.In practice, net additions will mainly comprise thosedwellings purchased from businesses (newly constructeddwellings, companyhouses, or rental dwellings) and thosepurchased from or transferred from the governmentsector plus any purchases, for owner-occupation, ofrental dwellings from reference population households.If the CPI is constructed for some subgroup of thepopulation (e.g. wage and salary earners), the weightshould also include purchases from other householdtypes.

10.43 Economists regard all housing as fixed capitaland hence would exclude purchases of dwellings fromhousehold consumption. While this is unambiguouslythe case for housing purchased for rental, the case is lessclear-cut when it comes to housing for owner-occupa-tion. Although households recognize the likelihood ofmaking capital gains when they purchase housing andinvariably regard their dwelling as an asset, they alsocommonly cite the primary motivation for the purchaseof a dwelling as being to gain access to a service (i.e.shelter and security of tenure). From the households’perspective, therefore, the costs borne by owner-occu-piers in respect of their principal dwelling represent a mixof investment and consumption expenditure, and thetotal exclusion of these costs from an acquisitions-basedCPI can lead to a loss of confidence in the CPI by thepopulation at large. Particularly in those countries whererental sectors are relatively small, with limited opportu-nities for substitution between owner-occupation andrenting, it might be argued that the consumption elementdominates.

10.44 The problem confronting compilers of CPIs ishow to separate the two elements so as to include onlythe consumption element in the CPI. Although there isno single agreed technique, one approach is to regardthe cost of the land as representing the investment

element and the cost of the structure as representingthe consumption element. The rationale for this is thatwhile the structure may deteriorate over time and hencebe ‘‘consumed’’, the land remains at constant qualityfor all time (except under extremely unusual circum-stances). As the land (or location element) accounts formost of the variation in observable prices for otherwiseidentical dwellings sold at the same point in time, theexclusion of land values may also be seen as an attemptto exclude asset price inflation from the CPI. (Measuresof asset price inflation are, of course, useful in their ownright.)

10.45 Derivation of weighting base period expendi-tures on the net acquisition of dwellings (excludingland), the construction of new dwellings, and alterationsand additions to existing dwellings poses some pro-blems. Although household expenditure surveys mayyield reliable estimates of the amounts householdsspend on alterations and additions, and construction ofdwellings, it is unlikely that they will provide reliableestimates of net expenditures on existing dwellingsexclusive of the value of the land.

10.46 An alternative approach is to combine datafrom censuses of population and housing and buildingactivity surveys. Population censuses normally collectinformation on housing tenure, from which averageannual growth in the number of owner-occupier house-holds represents a good proxy for net additions to thehousing stock. Building activity surveys are also con-ducted in most countries, providing data on the totalvalue of dwellings constructed. These data can be used toestimate the average value of new dwellings, which canthen be applied to the estimated volumes derived fromthe population census. Of course, the suitability of thisapproach would need to be assessed by each country andmay be complicated if the CPI relates only to some sub-set of the total population.

10.47 The price index is required to measure thechange over time in existing dwelling structures, newlyconstructed dwellings, and alterations and additions. Asthe appropriate price for existing dwelling structures iscurrent replacement cost, an index measuring changes inprices of newly constructed dwellings is also appropriatefor this purpose. Given that the prices for both newlyconstructed dwellings and alterations and additions are,in principle, determined by costs of building materials,labour costs and producers’ profits, it may also be sat-isfactory to construct a single price sample for all ele-ments. The requirement for a separate price sample foralterations and additions will depend on the relativesignificance of this activity and whether the material andlabour components differ significantly from those for acomplete dwelling (e.g. if alterations and additions arepredominantly to kitchens and bathrooms). In all cases,it is important that the price indices are mix-adjusted toeliminate price variations that reflect changes in thecharacteristics of newly constructed dwellings.

10.48 The type of dwelling constructed in individualcountries will significantly influence the complexity andcost of constructing appropriate price measures. If eachnewly constructed dwelling is essentially unique (i.e.designed to meet site or other requirements) it will be

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

184

necessary to adopt ‘‘model pricing’’. This requires selec-tion of a sample of building firms, identifying samples ofrecently constructed dwellings and collecting prices forconstructing identical dwellings in subsequent periods(exclusive of site preparation costs, which will vary fromsite to site). This approach is likely to entail significantcosts for the respondents. Moreover, care needs to betaken to ensure that the supplied prices truly reflect allprevailing market conditions. That is, prices need toreflect the amount builders could realistically expect tobe able to charge in the current market rather than theprices they would like to be able to charge based onconditions prevailing in some prior period.10.49 In a number of countries, a significant pro-

portion of newly constructed dwellings are of the typereferred to as ‘‘project homes’’. These are homes thatbuilders construct on a regular basis from a suite ofstandard designs maintained for this purpose. Thispractice is most feasible in countries where a significantproportion of new dwelling construction takes place innew developments (i.e. land recently developed or re-developed specifically for residential housing). Whereproject home construction is significant in scale, then it ispossible to select a sample of these project homes forpricing over time, safe in the knowledge that the pricesprovided will be actual transaction prices (again, pricednet of any site preparation costs). Even if project homesdo not account for the majority of new dwellings con-structed, they may still provide a representative measureof overall price change.10.50 In pricing project homes, it is necessary to

monitor the selected sample to ensure that the selectedplans remain representative and to detect changes inquality arising from modifications in design and changesto basic inclusions. Whenever a change is made to theplans, the change in overall quality has to be estimated.For physically measurable characteristics, such as a smallincrease in the overall size of the dwelling, it may beassumed that the change in quality is proportional to thechange in the relevant quantity. Other changes, such asthe addition of insulation, inclusion of a free drivewayand so on, will need to be valued, preferably in terms ofcurrent value to the consumer. These could be estimatedby obtaining information on the amounts that con-sumers would have to pay if they were to have the itemsprovided separately (the option cost method). An alter-native is to ask the builder if a cash rebate is available inlieu of the additional features. Where plans are modifiedto meet changed legal requirements, the consumer has nochoice in purchase and so it is acceptable to classify thefull change in price as pure price movement (even thoughthere may be some discernible change in quality).

Clothing10.51 Clothing is a semi-durable good and its treat-

ment is not affected by the conceptual basis chosen forthe CPI (acquisitions, use or payments). Particular fea-tures of the clothing market do, however, create pro-blems for price index compilers. Although clothing ispurchased throughout the year, many types of clothing

are only available in particular seasons and, unlike sea-sonal fruit and vegetables, the specific items on sale inone season (say summer) may not return the followingyear. In addition to seasonal availability, the physicalcharacteristics of some items of clothing can also changeas a result of changing fashions.

10.52 The remainder of this section seeks to providea general description of the clothing market applicableto most countries, discusses the most significant pro-blems faced by index compilers and looks at some op-tions for overcoming or at least minimizing these.

The clothing market10.53 Most countries experience at least some cli-

matic variation throughout the year. The number ofdiscrete ‘‘seasons’’ may range from two (‘‘wet’’ and‘‘dry’’, summer and winter) up to the four experiencedin most regions (winter, spring, summer and autumn).Items of clothing tend to fall into two categories: thosethat are available in one season only, and those that areavailable all year round.

10.54 Clothing (whether seasonal or not) is alsosubject to changes in fashion. The fashion for trouserscan change from straight legged to flared; jackets fromsingle-breasted to double-breasted; shirts from button-down collar to not; skirts from long length to shortlength, and so on.

10.55 Even within categories of garments which arenot unduly affected by seasonal influences or generalchanges in fashions, the garments that are available forpricing from one period to the next can vary greatly.Retailers change suppliers in order to seek the best pricesor to maintain an image of a constantly changing rangein order to attract shoppers. Many producers will alsofrequently change product lines in order to maintainbuyer appeal. The practice of single producers usingdifferent and changing brands as a marketing tool isalso common. Isolated countries that rely predominantlyon imported clothing also face the additional problem ofdiscontinuities in supply because of shipping failures oreven the whim of importers.

10.56 The often short life cycles of specific items,and whole categories of items in the case of seasonalitems, mean that retailers have to pay particular atten-tion to inventory control, since they cannot afford to beleft with large volumes of stock that they cannot sell.This is most commonly handled by progressively dis-counting or marking down prices throughout the esti-mated life cycle of an item.

10.57 The fragmented and changing nature of theclothing market invariably means that price index com-pilers have to strike a balance between the ideal require-ments for index purposes and the cost of data collection(of both prices and characteristics that may be requiredto make quality adjustments).

Approaches to constructing indicesfor non-seasonal clothing

10.58 Even where seasonality is not a problem, theconstruction of a price index for clothing is not a simpletask. The range of available items can differ significantly

SOME SPECIAL CASES

185

across outlets, making central determination and detailedspecification of items to be priced ineffective. The brandsand styles of particular garment types can also varysignificantly over time in individual outlets, requiringclose attention to procedures for replacing items andmaking quality adjustments.

10.59 Although it is virtually impossible to set outspecific procedures that will be applicable in all coun-tries, it is possible to develop a set of guidelines to helpavoid the most significant pitfalls. In developing theseguidelines, the key objective is to maximize the numberof usable price quotations (for a given collection cost) inany month, and to minimize the incidence of measuresof price change being affected by changes in quality.

10.60 In some circumstances, it may be possible toidentify ‘‘national’’ specifications to be priced at eachoutlet (e.g. brand X, model Y jeans). The use of thesetypes of specifications can help minimize the effort thatneeds to be put into quality adjustment, and movementsin prices of these items can provide a useful benchmarkagainst which to assess the movements of other items.Reliable identification of such items necessitates ongoingrelationships with the buyers for large chains, or largedomestic producers or importers. These sources need tobe contacted on a regular basis to identify the currentrange of items, the extent of their availability across thecountry and any planned changes (including changes instyle and quality as well as deletions from and additionsto the range). This information may be used proactivelyto update specifications or descriptions of items to bepriced in the field, so minimizing the incidence of pricecollectors attempting to price items that are no longeravailable. It can also be used to assist in the quantifica-tion of any quality changes.

10.61 For some items where availability by brandvaries, it may be possible to identify a number of brandswhich are assessed as being of equal quality (e.g. dif-ferent brands of T-shirts). In these cases, price collectorscould be provided with the list of equivalent brands andinstructed to price the cheapest one of these available ateach outlet without having to ensure that the samebrand is priced this time as on the last visit. The argu-ment for this practice is that, if the brands are trulyequivalent, discerning shoppers will purchase thecheapest at the time of purchase, and to reflect this inthe CPI will result in an index that more closely followsthe experience of households. Clearly, the success orotherwise of this technique depends vitally on theassessment of the ‘‘equality’’ of brands which, whilelargely a matter of judgement, may be assisted by ananalysis of past price behaviour. In general, brandequality might be indicated by narrow longer-term pricedispersion and a tendency for brands to swap prices overtime or outlets.

10.62 In other cases it might be appropriate torestrict sampled items to a subset of brands withoutregarding the brands as equivalent. For example, anumber of brands of jeans might together dominate themarket but with the availability of the individual brandsvarying by outlet. In these cases, price collectors could beprovided with a list of acceptable brands and instructedto price the most representative of these brands at each

outlet. Once the initial selection has been made, pricecollectors should be instructed to record the specificbrand and model priced at each outlet, and shouldcontinue to price that specification on subsequent visitsuntil such time as it ceases to be stocked (or it becomesclear that it is no longer representative of the sales of thatparticular outlet).

10.63 The clothing market has become so diversethat it is not always possible to specify centrally eitherthe item to be priced or even the brand (or brands). Inthese cases, it is necessary to give price collectors muchgreater discretion when it comes to selecting the indivi-dual items for pricing. To avoid the selection of inap-propriate items, it is important for price collectors to beprovided with guidelines to assist in this process. At thevery least, they should be instructed to select the brandand model that the retailer advises is both representativeand is expected to be stocked for some time (littleadvantage is to be gained from selecting an item which,while popular, has been purchased by the retailer on aone-off basis and is thus unlikely to be available forpricing in subsequent periods).

10.64 More sophisticated guidelines can incorporatea checklist of features that the selected item shouldmatch as closely as possible. These features should beranked from most to least important, and it should beclear which features the selected item possesses andwhich it does not (either from the detailed descriptionrecorded by the price collector or through the comple-tion of a separate feature pro-forma). In addition tobrand (or acceptable brands), where possible, the listmight include features such as:

– fabric type (e.g. cotton, wool, linen);

– weight of the fabric (e.g. heavy, medium, light);

– existence of a lining;

– number of buttons;

– type of stitching (e.g. single, double).

10.65 It is recognized that high fashion items poseparticular difficulties in terms of quality adjustment.There is certainly clear potential for such items to biasthe CPI towards the end of their life cycle when pricesmay be heavily discounted and sales volumes are low.For example, compilers need to guard against the dan-ger that items leave the index at a heavily discountedprice to be replaced by items that are on sale at the fullprice (which for a highly fashionable item may be at apremium). More generally, any decision on the inclusionof high fashion items ought certainly to reflect theintended reference population of the index, for examplewhere this excludes households at the upper end of theincome distribution.

Replacement of itemsand quality change

10.66 Even for garment types that are available allyear round, there remains a strong need to replace itemsor to otherwise recognize changes in item characteristics.It is therefore important to ensure that procedures areestablished to minimize any bias resulting from changesin the quality of items priced.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

186

10.67 The appropriate conceptual basis for assessingchanges in the quality of garments is from the perspectiveof value to the consumer. In other words, a garment canbe said to be of different quality to another garment if itis valued differently by the consumer. The difficulty con-fronted by index compilers is that quality differences areonly observable in terms of changes in the physicalcharacteristics of garments (including brand), some ofwhich will have an impact on customer value and someof which will not. The problem is how to distinguishbetween them.10.68 To assist in this task it is important to develop

guidelines for selecting replacement items, with the generalobjective of minimizing the quality difference between theold and new items. For most items, research has shownthat brand is an important price- and quality-determiningcharacteristic (particularly for items that have a significantfashion element) and so, in the first instance, an effortshould be made to select a replacement from the samebrand (but noting the danger that as brands go out offashion they become less representative). As this will notalways be possible, it is useful to enlist experts in the tradeto assist in drawing up a list that classifies brands intoquality groups along the following lines:

– exclusive brands, usually international brands, mostlysold in exclusive stores;

– higher-quality brands, well-known brands at thenational level (which may also include internationalbrands);

– average quality brands;

– other or unknown brands.

10.69 If it is not possible to select a replacementfrom the same brand, the fallback should be to select areplacement from a brand in the same quality group.Similarity of price should never be the guiding objectivewhen a substitute variety has to be chosen.10.70 Once a replacement item has been selected, a

detailed description of the new item needs to be recorded.The physical differences between the old and new itemsshould be described in asmuch detail as possible to enablethe index compiler to assess whether the replacementitem is comparable (i.e. of equal quality) to the old itemor not. As a general guide, changes such as single rows ofstitching replacing double rows, of lighter-weight fabricsreplacing heavier-weight ones, reductions in the numberof buttons on shirts, reductions in the length of shirt tails,disappearance of linings and so on should be regarded aschanges in quality. Changes in physical characteristicsattributable solely to changes in fashion (e.g. straight legto flared leg trousers) should not be regarded as qualitychanges.10.71 Where an item is assessed as not being com-

parable, action will need to be taken to remove theimpact of the quality change from the index. There are anumber of approaches that may be taken to value thequality difference:

� Industry experts may be asked to place a cash valueon the differences.

� The statistical office may arrange for some indexcompilers to receive additional training to become

commodity experts able to estimate the value of suchchanges themselves.

� Hedonic methods may be employed if resources per-mit. Descriptions of hedonic techniques for clothingcan be found in Liegey (1992) and Norberg (1999).

10.72 Eachof thesemethods requires that the changesin the quality-determining characteristics (such as qualityof material and standard of manufacture) are quantifi-able. If such information is not available, implicit qualityadjustment methods may have to be used. In this case, itis important that the price for the outgoing specificationis returned to a normal price before it is removed fromindex calculation.

Approaches to including seasonalclothing in the consumer price index

10.73 The practices adopted by statistical agenciesfor handling seasonal clothing in CPIs vary widely,ranging from complete exclusion of such items to var-ious methods of imputation of prices of items that areunavailable at a particular time of year, or to systems ofweights that vary throughout the year. In some respects,the treatment of seasonal clothing raises similar issues tothose found in dealing with fashion items, in particularreflecting the short life cycles of products and the like-lihood of price-discounting during those cycles.

10.74 This section describes some practical alter-natives for indices constructed using the traditionalannual basket approach to produce a monthly CPI (i.e.systems of explicitly changing weights are not explored,nor is the use of year-on-year changes as proposedin Chapter 22). Further, the examples will be restrictedto the so-called multiple basket approach because of theinherent difficulty of making quality adjustments be-tween seasons in the so-called single basket approach.(The single basket approach takes the view that, say,summer and winter seasonal items are different varietiesof the same article, whereas the multiple basket approachtakes the view that they are completely different articles.)

10.75 CPI compilers may choose to exclude seasonalclothing from the CPI altogether. While this might sim-plify the job of compiling the index, it clearly reduces therepresentativeness of the basket. This might be con-sidered as the option of last resort and will cause pre-sentational difficulties from the point of view of externalusers, particularly where relative expenditure on seasonalclothing is high. Including seasonal items makes thebasket more representative of consumption patterns butcomplicates the process of compiling the index. In reach-ing a decision, it will be necessary to strike a balancebetween representativeness and complexity (cost). Whereseasonal items are excluded, their expenditure weightshould be distributed among non-seasonal counterparts.

10.76 Six possible approaches to constructing aggre-gate clothing price indices in the presence of seasonalitems are described below. A synthetic set of prices isused (see Table 10.3) to illustrate the various options.For simplicity, it is assumed that there are only threecategories of clothing: those available all year (non-seasonal); and two seasonal categories (labelled summerand winter here). The two seasons are assumed to be

SOME SPECIAL CASES

187

non-overlapping and the prices of the seasonal varietiesare contrived to show progressive discounting over thecourse of each season. The prices of the non-seasonalitems show a steady rate of growth. Within each cate-gory, prices are assumed to be for items of identicalphysical characteristics (or alternatively, to have beenadjusted to remove the effects of changes in physicalcharacteristics).

10.77 The price indices have been compiled with abase period of month 1 in year 0 and extend for 24months (prices are provided for year Y�1 in order toimpute base period prices for the winter seasonal item).For the purpose of weighting, it is assumed that each ofthe seasonal categories accounts for 25 per centof expenditure, while non-seasonal items account forthe remaining 50 per cent. For ease of computation,imputation is based on the simple arithmetic average ofthe price movements of the available series (includingmovements from imputed to real prices), though inpractice these imputations would be based on weightedaverages. Tables 10.4 to 10.6 present the calculatedindices and monthly percentage changes for summerseasonal, winter seasonal and total clothing, respectively,based on the alternative methodologies described below.

10.78 Exclude seasonal items. This is the simplestoption from an index construction point of view, butsuffers from a lack of representativeness, which may be acause of concern to some users. In this example, only 50per cent of expenditures would be directly represented inthe index. Clearly, the greater the relative expenditureon seasonal items, the more users are likely to be con-cerned about the lack of representativeness of the index.The results for this index are shown in column (1) ofTable 10.6 and may be used as a benchmark againstwhich the following options can be assessed.

10.79 Impute only on items available all year. Thisapproach is one of the targeted imputation approaches.In this case, the out of season prices for both summerand winter items are imputed based only on the move-ment in the prices of those items available all yearround. The results for the summer and winter items areshown in column (1) of Tables 10.4 and 10.5, respec-tively, while the total clothing index is shown in column(2) of Table 10.6.

10.80 Impute on all available items. This approachimputes all missing prices based on the movements inall available prices of related or similar items. Thisapproach is similar in principle to the approach thatwould be taken in the case of a missing price observa-tion. Prices for seasonal items are collected while theyare observable, and when out of season are imputedbased on items available all year round together withother seasonal items if available. The results are shownin column (2) in Tables 10.4 and 10.5, and in column (3)of Table 10.6.

10.81 Carry forward of last observed price. Thissimpler variant of the methods described above involvesthe carry forward of the last observed prices for sea-sonal items during the months when such prices areunavailable.This approachwouldnotnormallybe recom-mended in the general case where prices are not availablefor non-seasonal items, on the grounds that the likelydownward bias imparted could easily be avoided byobserving the price of some similar item that is available.But where a whole class of goods is unavailable andhence unobservable, and particularly where price move-ments are not strongly correlated with other items, carryforward of prices may be seen as an acceptable approach.The results are shown in column (3) in Tables 10.4 and10.5, and in column (4) of Table 10.6.

10.82 Under this approach, it is preferable todetermine in advance during which months seasonalprices will be collected. This helps prevent distortion ofthe index through collection of possibly atypical pricesfor seasonal items unexpectedly available outside thoseperiods when they would normally be available. Suchdecisions should be subject to regular review on the basisof market developments.

10.83 Return to normal, then impute. This approachrequires the index compiler to estimate the ‘‘normal’’price for the item during the first month when it isunavailable (out of season). This estimated normal priceis then imputed forward until such time as the itembecomes available again. Compared to the methodsdiscussed so far, this approach is designed to avoidartificial depression of the aggregate index beyond theend of season, following progressive discounts over theitem’s short life cycle.

Table 10.3 Synthetic price data to illustrate approaches to constructing clothing price indices

Month Year Y�1 Year Y Year Y + 1

Non-seasonal

Summerseasonal

Winterseasonal

Non-seasonal

Summerseasonal

Winterseasonal

Non-seasonal

Summerseasonal

Winterseasonal

1 100 100 113 110 127 1252 101 80 114 90 128 1003 102 60 115 70 130 804 103 116 1315 104 117 1326 105 118 1337 106 100 120 110 135 1258 107 80 121 90 136 1009 108 60 122 70 137 80

10 109 123 13911 110 124 14012 112 126 142

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

188

10.84 There are some problems with this procedure.Particularly during periods of high inflation, it will bedifficult to determine what the normal price is. Moregenerally, it can be argued that the procedure reducesthe objectivity of the index. In the illustrative examplespresented here, the normal price to which the item isreturned is the price observed at the start of the season.Compared with the previous three approaches, it can beseen that this has the effect of shifting the price increasefrom the commencement of the next season to imme-

diately after the current season, i.e. the index records asharp price change when none is observable. The resultsare shown in column (4) in Tables 10.4 and 10.5, and incolumn (5) of Table 10.6.

10.85 Include only the first seasonal observation, thenimpute. This approach requires that seasonal items bepriced only once per season, when they first appear in themarketplace. This first observed price is then imputedforward until the item is priced again at the commence-ment of the next season. The rationale for this technique

Table 10.4 Alternative price indices for summer seasonalclothing

Month Imputeonly onitemsavailableall year

Imputeon allavailableitems

Carryforwardof lastobserved price

Return tonormal,thenimpute

Include firstseasonalobservation,then impute

(1) (2) (3) (4) (5)

Index numbers

1 100.0 100.0 100.0 100.0 100.02 81.8 81.8 81.8 81.8 100.93 63.6 63.6 63.6 63.6 101.84 64.2 64.2 63.6 100.0 102.75 64.7 64.7 63.6 100.9 103.56 65.3 65.3 63.6 101.7 104.47 66.4 77.0 63.6 102.9 105.48 67.0 70.3 63.6 94.0 106.39 67.5 62.8 63.6 83.9 107.1

10 68.1 63.3 63.6 108.3 108.011 68.6 63.8 63.6 109.2 108.912 69.7 64.9 63.6 110.9 110.713 113.6 113.6 113.6 113.6 113.614 90.9 90.9 90.9 90.9 114.515 72.7 72.7 72.7 72.7 116.316 73.3 73.3 72.7 113.6 117.217 73.8 73.8 72.7 114.5 118.118 74.4 74.4 72.7 115.4 119.019 75.5 93.3 72.7 117.4 120.820 76.1 84.3 72.7 106.1 121.721 76.6 76.2 72.7 95.8 122.622 77.8 77.3 72.7 123.5 124.423 78.3 77.9 72.7 124.4 125.324 79.4 79.0 72.7 126.2 127.1

Monthly percentage changes

2 �18.2 �18.2 �18.2 �18.2 0.93 �22.2 �22.2 �22.2 �22.2 0.94 0.9 0.9 0.0 57.2 0.95 0.8 0.8 0.0 0.9 0.86 0.9 0.9 0.0 0.8 0.97 1.7 17.9 0.0 1.2 1.08 0.9 �8.7 0.0 �8.6 0.99 0.7 �10.7 0.0 �10.7 0.8

10 0.9 0.8 0.0 29.1 0.811 0.7 0.8 0.0 0.8 0.812 1.6 1.7 0.0 1.6 1.713 63.0 75.0 78.6 2.4 2.614 �20.0 �20.0 �20.0 �20.0 0.815 �20.0 �20.0 �20.0 �20.0 1.616 0.8 0.8 0.0 56.3 0.817 0.7 0.7 0.0 0.8 0.818 0.8 0.8 0.0 0.8 0.819 1.5 25.4 0.0 1.7 1.520 0.8 �9.6 0.0 �9.6 0.721 0.7 �9.6 0.0 �9.7 0.722 1.6 1.4 0.0 28.9 1.523 0.6 0.8 0.0 0.7 0.724 1.4 1.4 0.0 1.4 1.4

Table 10.5 Alternative price indices for winter seasonalclothing

Month Imputeonly onitemsavailableall year

Imputeon allavailableitems

Carryforwardof lastobserved price

Return tonormal,thenimpute

Include firstseasonalobservation,then impute

(1) (2) (3) (4) (5)

Index numbers

1 100.0 100.0 100.0 100.0 100.02 100.9 91.4 100.0 91.4 100.93 101.8 81.6 100.0 81.6 101.84 102.7 82.3 100.0 105.3 102.75 103.5 83.0 100.0 106.2 103.56 104.4 83.7 100.0 107.1 104.47 175.2 112.4 183.3 107.8 104.68 143.4 91.9 150.0 88.2 105.49 111.5 71.5 116.7 68.6 106.3

10 112.4 72.1 116.7 107.8 107.211 113.3 72.7 116.7 108.7 108.112 115.2 73.9 116.7 110.4 109.813 116.1 101.9 116.7 112.2 111.714 117.0 92.1 116.7 101.5 112.615 118.8 83.6 116.7 92.1 114.416 119.7 84.3 116.7 118.4 115.217 120.6 84.9 116.7 119.3 116.118 121.6 85.6 116.7 120.2 117.019 199.1 127.7 208.3 122.5 118.820 159.3 102.2 166.7 98.0 119.721 127.4 81.7 133.3 78.4 120.622 129.3 82.9 133.3 122.5 122.423 130.2 83.5 133.3 123.4 123.224 132.1 84.7 133.3 125.2 125.0

Monthly percentage changes

2 0.9 �8.6 0.0 �8.6 0.93 0.9 �10.7 0.0 �10.7 0.94 0.9 0.9 0.0 29.0 0.95 0.8 0.9 0.0 0.9 0.86 0.9 0.8 0.0 0.8 0.97 67.8 34.3 83.3 0.7 0.28 �18.2 �18.2 �18.2 �18.2 0.89 �22.2 �22.2 �22.2 �22.2 0.9

10 0.8 0.8 0.0 57.1 0.811 0.8 0.8 0.0 0.8 0.812 1.7 1.7 0.0 1.6 1.613 0.8 37.9 0.0 1.6 1.714 0.8 �9.6 0.0 �9.5 0.815 1.5 �9.2 0.0 �9.3 1.616 0.8 0.8 0.0 28.6 0.717 0.8 0.7 0.0 0.8 0.818 0.8 0.8 0.0 0.8 0.819 63.7 49.2 78.6 1.9 1.520 �20.0 �20.0 �20.0 �20.0 0.821 �20.0 �20.1 �20.0 �20.0 0.822 1.5 1.5 0.0 56.3 1.523 0.7 0.7 0.0 0.7 0.724 1.5 1.4 0.0 1.5 1.5

SOME SPECIAL CASES

189

is that it is a means of adjusting for the quality degra-dation of seasonal items associated with the commonlyobserved feature of falling prices throughout the season.Further, if it is desirable that the index behave as if itwere constructed as a moving year index (see Chapter22), then this approach provides a cost-effective alter-native that also accommodates changing seasons (e.g.when the items that were in season last March do notappear until April this year).

10.86 On the downside, in fully discounting obser-vable price movements through a seasonal item’s lifecycle, an implicit assumption is made that all such

movements reflect quality changes with no change inunderlying price. This is not likely to fully accord withuser perceptions of price evolution and, unless similartechniques are employed for fashion items, it can beargued that the approach is inconsistent. The results areshown in column (5) in Tables 10.4 and 10.5, and col-umn (6) in Table 10.6.

Summary comments10.87 First, it is worth noting that the consequences of

imputing price changes for baskets of seasonal items based

Table 10.6 Alternative price indices for total clothing

Month Only itemsavailable allyear round

Impute onlyon itemsavailable allyear

Impute onall availableitems

Carryforward oflast observedprice

Return tonormal, thenimpute

Include firstseasonalobservation, thenimput

(1) (2) (3) (4) (5) (6)

Index numbers

1 100.0 100.0 100.0 100.0 100.0 100.02 100.9 96.1 93.8 95.9 93.8 100.93 101.8 92.3 87.2 91.8 87.2 101.84 102.7 93.1 88.0 92.2 102.7 102.75 103.5 93.8 88.7 92.7 103.5 103.56 104.4 94.6 89.5 93.1 104.4 104.47 106.2 113.5 100.5 114.8 105.8 105.68 107.1 106.2 94.1 106.9 99.1 106.59 108.0 98.8 87.6 99.1 92.1 107.4

10 108.8 99.5 88.3 99.5 108.4 108.211 109.7 100.3 89.0 99.9 109.3 109.112 111.5 102.0 90.5 100.8 111.1 110.913 112.4 113.6 110.1 113.8 112.7 112.514 113.3 108.6 102.4 108.5 104.8 113.415 115.0 105.4 96.6 104.9 98.7 115.216 115.9 106.2 97.4 105.3 116.0 116.117 116.8 107.0 98.1 105.8 116.9 117.018 117.7 107.9 98.9 106.2 117.8 117.919 119.5 128.4 115.0 130.0 119.7 119.720 120.4 119.1 106.8 120.0 111.2 120.621 121.2 111.6 100.1 112.1 104.2 121.422 123.0 113.3 101.6 113.0 123.0 123.223 123.9 114.1 102.3 113.5 123.9 124.124 125.7 115.7 103.8 114.3 125.7 125.9

Monthly percentage changes

2 0.9 �3.9 �6.2 �4.1 �6.2 0.93 0.9 �4.0 �7.0 �4.3 �7.0 0.94 0.9 0.9 0.9 0.5 17.8 0.95 0.8 0.8 0.8 0.5 0.8 0.86 0.9 0.9 0.9 0.5 0.9 0.97 1.7 20.0 12.3 23.3 1.3 1.18 0.8 �6.4 �6.4 �6.9 �6.3 0.99 0.8 �7.0 �6.9 �7.4 �7.1 0.8

10 0.7 0.7 0.8 0.4 17.7 0.711 0.8 0.8 0.8 0.4 0.8 0.812 1.6 1.7 1.7 0.9 1.6 1.613 0.8 11.4 21.7 12.8 1.4 1.414 0.8 �4.4 �7.0 �4.6 �7.0 0.815 1.5 �2.9 �5.7 �3.4 �5.8 1.616 0.8 0.8 0.8 0.4 17.5 0.817 0.8 0.8 0.7 0.4 0.8 0.818 0.8 0.8 0.8 0.4 0.8 0.819 1.5 19.0 16.3 22.4 1.6 1.520 0.8 �7.2 �7.1 �7.7 �7.1 0.821 0.7 �6.3 �6.3 �6.6 �6.3 0.722 1.5 1.5 1.5 0.8 18.0 1.523 0.7 0.7 0.7 0.4 0.7 0.724 1.5 1.4 1.5 0.8 1.5 1.5

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

190

on the price movements for other items of clothing isequivalent to allocating the weight for seasonal items toother items when they are out of season, so avoiding thecomplexity involved in systems of explicitly changingweights. In these circumstances, some care needs to betaken in the presentation of estimates of the contributionof both seasonal and non-seasonal items to the change inthe aggregate CPI. The standard practice of determiningan item’s contribution to the total change in the CPI is tomultiply the item’s previous period (price-updated) weightby its percentage change. Only those seasonal items forwhich prices are actually measured in the current periodwill contribute to the change in the aggregate index.Similarly, though only non-seasonal items will contributeto the change in the aggregate index when seasonal itemsare out of season, the standard measure of their con-tribution will be understated. This is mainly an issue ofpresentation, although some compilers might prefer topresent assessments of contributions only down to the levelthat includes both the seasonal and non-seasonal baskets.10.88 There is likely to be a range of views across

countries, and indeed users, concerning the appropriatetreatment of seasonal items within a CPI. There is likelyto be a particular diversity of views about whether thequality of seasonal items should be regarded as dimin-ishing over the life of the season or not and, if so, whethera similar approach should (or can) be taken in respect offashion items. The example data set was contrived so thateach category displayed broadly constant growth in priceson a year-on-year basis. Those users primarily interestedin measures that best capture persistent or underlyingprice pressures in the economy are likely to prefer thoseapproaches which do not yield significant variations inthe rate of price change that are solely attributable tohow the statistical agency treats seasonal items. Suchusers may prefer that seasonal items be excluded al-together or that only the first seasonal observation beincluded with prices for other months being imputed.10.89 What is clear is that national statistical offices

need to carefully weigh up user requirements, theoreticalissues, costs and the implications of alternative ap-proaches before settling on the methodology to beadopted.

Telecommunication services10.90 The global telecommunications sector has

undergone rapid change in recent years. Technologicalinnovation has resulted in a proliferation of new serviceswhile deregulation has led to sharp growth in the numberof providers in many countries. Taken together, thesefactors have resulted in suppliers adopting a range ofnew strategies to differentiate their services in order toattract and retain customers.10.91 Characteristics of particular significance to

compilers of price indices are:

� fewer linear pricing schedules and the adoption ofdifferent pricing structures across providers;

� the increasing tendency to offer contracts that bundleservices together in different ways to appeal to dif-ferent types of consumers;

� rapid changes in the contracts offered to consumers asan effective means of encouraging the take-up of theever-increasing range of services.

10.92 Increasingly, telecommunication companiesoffer services via plans that require customers to enterinto longer-term contractual arrangements with theproviders. This also poses problems for index compila-tion. Two broad types of plan are typically offered. Thefirst has no fixed duration and makes allowance for theprovider to change pricing structures with advancenotice to the consumer. The second and increasinglymore popular type provides a fixed term contract (gen-erally of one to two years) with prices fixed for theduration of the contract. These plans are differentiatedby charging different prices for different services. Forexample, a simple plan may be differentiated by chargingmore for monthly line rental but less for local calls, soappealing to users who make a higher volume of localcalls. The emergence of new tailored plans designed tomaximize customer demand overall is continuous.

10.93 If statistical agencies follow traditional sam-pling approaches and select price schedules according tosome base period set of plans, and follow them untilthey expire, no price changes will be observed (likewise ifplans expire and replacements are linked to show nochange). The marketplace reality, by contrast, is thatunit values for telecommunication services have beendeclining significantly in many countries.

10.94 All statistical agencies are struggling todevelop methodologies capable of coping with the com-plexities of this sector. In particular, it is recognized thatcurrent best practice approaches have difficulty in ac-counting for substitutions across providers and in ade-quately accounting for changes in the quality of theservices provided.

10.95 With the telecommunications sector undercontinual change, statistical practices need to be keptunder constant review. Statistical agencies that are con-sidering the construction of telecommunications indicesfor the first time, or considering reviewing their currentpractices, are advised to seek out the most recent re-search in this field. Nevertheless, this section seeks toprovide a general description of four approaches that arecurrently used by national statistical agencies to measurechanges in the prices of telecommunication services. Theapproaches, in increasing order of cost, are:

– representative items – matched samples;

– representative items – unit values;

– customer profiles;

– sample of bills.

10.96 Each approach is briefly described and poten-tial deficiencies noted. There is no firm recommendationon the best approach as the choice will depend largely onthe market conditions prevailing in individual countries,the sophistication of the index compilation system in use,and the extent of access to accurate and timely tele-communication services data. Depending on these fac-tors, it may be appropriate to use different approachesfor different telecommunication services, or even for thedifferent services of specific providers.

SOME SPECIAL CASES

191

Representative items –matched samples

10.97 This approach mirrors traditional techniquesadopted elsewhere in the CPI. Total expenditure ofreference group households on telecommunication ser-vices in the weighting base period is derived from sourcessuch as household expenditure surveys. A sample ofservice providers is approached to obtain information onrevenue by types of services (such as line rental, localcalls, international calls, handset sales or rentals, con-nection fees, voicemail services, Internet charges and soon) and a number of these are selected as representativeitems with weights derived from the revenue data.

10.98 For each representative item, a sample ofdetailed specifications (such as a telephone call fromlocation A to location B, at time X, of duration Y min-utes) is drawn up sufficient to represent the range ofspecific services purchased by consumers within eachrepresentative item. This sample of specifications isheld constant from period to period, and movementsin the indices for representative items are computed,based on the movements in the prices of this matchedsample of specifications. Table 10.7 illustrates theapproach.

10.99 The list of representative items (the lowestlevel in the structure) generally does not need to coverall telecommunication services, but those selected shouldbe sufficient to be representative of price behaviour as awhole, in particular taking account of published tariffs.Expenditures on those services not selected for pricingshould be distributed over the other services within thatgeneral class for the purpose of deriving weights. Forexample, the expenditures on any fixed line services notselected for pricing should be distributed over thosefixed line services selected.

10.100 Compared to suppliers of goods, serviceproviders have an almost infinite capacity to tailor boththe services and the prices they charge, for example basedon the time at which the service is provided. A telephonecall of five minutes’ duration at 8 a.m. can be regarded asa different product to an equivalent call made at 8 p.m.,

and service providers are able to charge different pricesfor these calls. Representative items therefore need to bedescribed in sufficient detail to capture all the price-determining characteristics.

10.101 Furthermore, given the ease with whichproviders can adjust the differential aspects of theirpricing schedules (such as the time span designated aspeak and the duration of a call before a different rateapplies), it is necessary to use a sufficient number ofvaried specifications to capture these aspects reliably.It is not sufficient to simply describe a call as peak oroff-peak, or from zone 1 to zone 2. Illustrative examplesof the types of specifications that may be applicablefor two representative items – international calls (fixedline) and usage fees (Internet services) – are provided inTable 10.8.

10.102 It is assumed that the origin of both the tele-phone calls and Internet access is also identified. Alltimes are domestic. It should also be noted that thenature of Internet access generally precludes pricing onthe basis of access, and hence the timing of access can-not be as tightly defined as for international telephonecalls; instead, all specifications are for total monthly use.

10.103 The most costly aspect of this approachtherefore is obtaining the data required to establish therepresentative items and to identify suitable specifica-tions, as this will require detailed information fromservice providers. Once implemented, most price infor-mation should be readily available from published feeschedules, so minimizing the burden on respondentsbetween reviews of the specifications.

10.104 The dynamic nature of the telecommunica-tion sector and the common use of the pricing mecha-nism to change consumer behaviour are likely to requirethat the specifications be updated relatively frequently.When a specification disappears (i.e. a particular plan isno longer offered), all efforts must be made to find asuitable comparison specification. Where specificationsare replaced, it is possible to argue that because differentplans involve different conditions of sale they are fun-damentally different products. It is equally reasonable

Table 10.7 An illustrative index structure for telecommuni-cation services (representative item approach)

Fixed line servicesTelephone connection costsTelephone line rentalLocal callsLong-distance national callsInternational calls

Mobile telephonesConnection costsHandset purchase or rentalNational callsInternational calls

PayphonesLocal calls

Internet servicesConnection feesUsage fees

Table 10.8 Examples of specifications of telecommunica-tion services

Representative item Examples of specifications

International calls(fixed line)

Plan A: Call to Athens at 8 a.m. on aFriday, duration 10 minutesPlan B: Call to London at 9 p.m. on aSaturday, duration 5 minutesPlan A: Call to New York at 11 a.m.on a Wednesday, duration 20 minutesPlan B: Call to Paris at 7 p.m. on aSunday, duration 15 minutesPlan A: Call to Durban at 8 p.m. on aMonday, duration 30 minutes

Usage fees(Internet)

Plan A: 10 hours dial-up connect timebetween 4 p.m. and 7 p.m. weekends,total download 20 MbPlan B: 20 hours dial-up connect timebetween 6 p.m. and midnight week-days, total download 50 MbPlan C: Permanent broadband con-nection, total download 100 Mb

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

192

to question whether all of the price difference betweenplans is due to quality differences, particularly in light ofthe evidence of ever-increasing volumes and reductionsin unit values. The difficulty lies in quantifying thequality differences. Although hedonic techniques offersome prospects for resolving this dilemma, they arecostly to implement.

Representative items – unit values10.105 The unit value approach is similar to the

previous approach, with the exception that specificationsare not priced. The price for each representative item iscalculated from revenue and quantity data collectedfrom the service provider. For example, the price fornational long-distance calls can be derived as the totalrevenue received from such calls divided by the numberof call-minutes. Similarly, in the case of monthly linerental fees, the price can be calculated as the total rev-enue from line rental divided by the total number ofsubscribers.10.106 Compared to the matched sample approach,

the unit value approach attributes all of the differencebetween plans, and time and duration of calls to price(i.e. the quality difference is assumed to be zero). The unitvalue approach is also seen as providing a method foraccounting for price change when the items are subject toa proliferation of discount schemes or promotions (e.g.$2 to call anywhere for as long as you like for the nextweek). While the approach avoids some of the customersampling choices inherent in other methodologies, com-pilation does rest on analysis of aggregate company dataand so is likely to be less timely than methodologiesbased on pre-published prices. Moreover, care needs tobe exercised with this approach to ensure that the mea-sure is not affected by undesirable compositional changes(see Chapter 9, where unit value indices are discussed inmore detail). A unit value index should only be con-structed for truly homogeneous items. This points to arequirement for defining the representative items at arelatively fine level of disaggregation. For example,international calls may need to be further subdivided bydestination to avoid changes in unit values arising purelyfrom shifts in the numbers of calls made to differentdestinations.10.107 Although this approach appears to address at

least some of the known deficiencies of the matchedsample approach, it is likely to have a medium- to long-term downward bias and, unless implemented carefully,it is likely to exhibit period-to-period volatility becauseof compositional shifts, if only as a result of seasonalvariations in usage patterns. There are also a number ofrespondent and data quality aspects that need to beconsidered. The unit value approach imposes a greaterdata burden on service providers, who often regardrevenue and quantity data as highly commercially sen-sitive. To be effective, the service providers also need tobe able to furnish data relating only to households (i.e.they have to be able to separate out revenue and quan-tities relating to businesses) and the revenue informationneeds to conform to the requirements of the index. Forexample, some service providers may record certain

discounts as a marketing expense, rather than a reduc-tion in revenue as is required for the unit value index.

Customer profiles10.108 For marketing purposes, telecommunication

companies often classify their customers according totheir volume of service use. Although the number ofcategories can vary, a common approach is to use a three-way classification: low-volume, medium-volume andhigh-volume customers. Service providers analyse cus-tomer usage patterns by category when developing newplans targeted specifically at each group. National reg-ulatory authorities may also be in a position to providedetailed customer use profiles on a confidential basis.

10.109 Statistical agencies can take a similar ap-proach for the construction of price indices by devisingprofiles which reflect the average usage patterns for eachcategory of consumer. Costs faced by these averageconsumers in each period can then be estimated byreference to the rates set out in that plan that is currentlymost commonly applicable to each customer category.Variations on this general theme include estimation ofcosts based on the plan that would deliver the cheapestoverall cost to the consumer (based on the simplifyingassumption of cost-minimizing consumer behaviour withperfect knowledge). This has the advantage of providinga clear basis for choosing a comparable replacementshould an existing package cease to be available. Alter-natively, costs to each customer group may be estimatedwith reference to several plans, where sales informationindicates that this is a closer approximation to reality.The overall index is derived by weighting together theresults from these user profiles according to informationabout the relative importance of each category of con-sumer.

10.110 In constructing the aggregate index, thesecalculations are likely to be made for a representativesample of service providers, exploiting information ontheir overall market share for sampling or weightingpurposes if available. This opens up the possibility offully exploiting all the possible relevant permutations ofprofiles and companies. Information on the distribu-tion of customer profiles by service provider may, how-ever, not be available or at least very costly to obtain.Table 10.9 gives an example of a profile for mobile tele-phone services, taken from Beuerlein (2001), whichdescribes the current approach used in the German CPI.

10.111 Consistent with the fixed basket approach,the activity of consumers (in terms of numbers and typesof calls) is held constant between comparison periods.Prices may, of course, change when not fixed by contractor when plans are replaced. Index compilers may alsoallow rates to change in response to a changing mix ofplans within customer categories. This approach assumesthat plan changes, as such, fundamentally represent pricechange rather than quality change, but it eliminates thecruder compositional effects associated with the unitvalue approach, which does not take account of cus-tomer profiles.

10.112 The success of this approach is determinedby the degree to which the profiles truly reflect consumer

SOME SPECIAL CASES

193

behaviour and therefore a great deal of thought needs tobe put into their development. The construction of thecustomer profiles will require a high degree of co-operation from service providers and, given the knownvolume changes, they will require updating at reason-ably regular intervals, possibly more frequently thanother items in the CPI basket. Data on plan usage bycustomer category for each index compilation period(month or quarter) may also be required if compilersdecide to allow for such effects.

Sample of bills10.113 This method can be seen as a more refined

application of the customer profile approach. A fixedlevel of service activity from an actual sample of cus-tomers is priced each month rather than defining profilesrepresentative of the average monthly activities of cus-tomers. A sample of customers should be selected fromeach category of customer (low-, medium- and high-volume customers) and, ideally, the bills (or activitystatements) should cover a full year’s activity.

10.114 The advantages of this approach comparedto the customer profile approach are:

� It is able to take account of any within-year variationsin customer behaviour (e.g. a higher incidence ofinternational calls associated with religious or culturalevents of significance).

� It better reflects the diversity of consumer behaviourby identifying actual activities (i.e. calls actually madeby a sample of consumers).

� It accommodates within each bill any instances ofannual charges.

� It allows for the detection and recording of othersources of price change associated with customers’overall relationship with the service provider (e.g.where overall discounts are provided when aggregatemonthly spending exceeds certain values, or where anaggregate discount is provided if customers acquirebundles of services from a single provider, such asfixed line phone plus Internet).

10.115 Calculation of the index still requiresmonthly information on the relative significance of var-ious plans by customer category (which can then berandomly allocated across the sampled bills). With the

bill sample repriced each period, the resulting indexmeasures the cost of a full year’s consumption at theprices prevailing in each index period compared to thesame cost at base prices. This assumes that the qualitydifference between old and new plans is zero for house-holds’ changing plans. Because of the generally largernumber of bills (compared with the number of availableprofiles), price changes can be reflected more gradually,as the proportion of bills priced using each plan canbetter mirror the changing population distribution.

10.116 As with the profile approach, it is importantthat the sample of bills is updated regularly to reflectchanges in consumption patterns and the take-up of newservices such as call-waiting, voicemail and text mes-saging. Although, with adequate sampling, the bill ap-proach is likely to provide a better measure of theaggregate rate of price change for telecommunicationservices as a whole, it may not be best suited to the cal-culation of separate indices for the components of thoseservices (depending on whether overall or bottom-linediscounts are offered). The approach is also dataintensive, requiring a large number of calculations eachperiod and thus a sophisticated data processing system.

Financial services10.117 The construction of reliable, comprehensive

price indices for financial services in CPIs is in its infancy.Given the increasing use of financial services by house-holds, however, national statistical agencies are comingunder pressure to account for at least some financialservices in their CPIs. There is a particularly strongdemand for CPIs to include those fees and charges facedby households in respect of deposit and loan accountsheld with financial institutions.

10.118 The construction of price indices for financialservices is inherently difficult, as there is no unanimousview about which financial services ought to be includedin the CPI, or indeed about precisely how they shouldbe measured. The discussion in this section attempts topresent what might be regarded as the majority viewbased on what is practically feasible. Much of the ma-terial is based on Fixler and Zieshang (2001), Frost(2001) and Woolford (2001).

10.119 Common examples of financial servicesacquired by households include financial advice, cur-rency exchange, services associated with deposit and loanfacilities, services provided by fund managers, life insur-ance offices and superannuation funds, stockbrokingservices, and real estate agency services. The range ofitems explicitly regarded as financial services for inclu-sion in a CPI, and also the way in which they are mea-sured, will depend on the principal purpose of the CPIand hence on whether an acquisitions, use or paymentsapproach is employed.

10.120 Where a payments approach is used, thegross interest payable on mortgages is often included asa cost of owner-occupied housing (see paragraphs 10.4to 10.50 above). In the interests of strict consistency, thismight imply that the CPI should also include consumercredit charges (measured in a similar way to mortgageinterest charges), as well as gross outlays on direct fees

Table 10.9 Example of a user profile for mobile phoneservices

Specification Unit Rarecallers

Low-volumecallers

Averagecallers

Total length of calls Minutes 16 42 96Length of individual call

Type A Seconds 35 45 45Type B Seconds 65 95 115

Calls1 Number 20 36 72Within the same network Number 8 12 24Beyond the network Number 12 24 48

1The calls are distributed over times of the day and days of the week so that itis possible to take account of changes in the delimitation of between peak andoff-peak, weekday and weekend tariffs.Source: Beuerlein (2001).

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

194

and charges paid in respect of other financial services. Inpractice, and as noted in the earlier section on housingcosts, the treatment of housing sometimes differs inconcept from other interest charges in national CPIs,partly reflecting mixed objectives for the overall indexcombined with public perceptions of the importance ofthis item within overall budgets. The specific require-ments for a payments approach will not be discussedfurther here as the principles are either described else-where (e.g. under owner-occupied housing) or are rela-tively straightforward.10.121 Assuming that households acquire all of their

financial services from the private sector (i.e. they are notgenerally subsidized by governments or provided by non-profit institutions serving households), the acquisitions anduse approaches take an identical view of the measurementof financial services. In terms of coverage, however, someproponents of the use approach take a more restrictiveview of which services should be included by limiting thescope to only those financial services which are acquiredto directly facilitate current household consumption.10.122 Under the more restrictive view of coverage, it

is argued that the use of some financial services is inex-tricably linked with capital (or investment) activity. Thissuggests that such activities should be considered outsidethe scope of CPIs intended to provide measures of chan-ges in consumption prices. Proponents of this view oftendraw upon national accounts practices as the startingpoint. For example, SNA 1993 classifies expenses asso-ciated with the transfer of real estate (real estate agents’commissions, legal fees, and government taxes and char-ges) as part of gross fixed capital formation. It is impor-tant to note, however, that the CPI is not constrained tofollow the practices adopted for national accounting.Rather, individual countries will need to make decisionson the item coverage of the CPI which best meets thedomestic requirements of the price index itself.10.123 One broad definition that could be adopted

for the coverage of financial services within the CPI is: allthose services acquired by households in relation to theacquisition, holding and disposal of financial and realassets, including advisory services, except those acquiredfor business purposes. This definition serves two purposes.First, it distinguishes between the services facilitating thetransfer and holding of assets and the assets themselves.Second, it makes no distinction between whether theunderlying asset is a real asset or a financial asset.10.124 The degree of complexity involved in placing

a value on financial services acquired by households andconstructing the companion price indices varies mark-edly by service. Three specific examples reflecting currentAustralian research are used to illustrate the issues:currency exchange, stockbroking, and deposit and loanfacilities. Real estate agency services are discussed sepa-rately in this chapter (see paragraphs 10.149 to 10.155)because they may be classified as either a housingexpense or a financial service.

Currency exchange10.125 For weighting purposes, the estimation of

the base period expenditures incurred by households

in exchanging domestic currency for currencies ofother countries is, in principle, relatively straightforwardand should be reportable in household expendituresurveys.

10.126 Construction of the companion price index ismore complex. The service for which a price is requiredis that of facilitating the exchange of domestic currencyfor that of another country (the acquisition of an asset –foreign currency). The price for the service is usuallyspecified in terms of some percentage of the domesticcurrency value of the transaction. These percentagemargins may change only rarely, with service providersrelying on the nominal value of the transactionsincreasing over time to deliver increases in fee receipts.The price required for index construction purposes isthe monetary value of the margin (i.e. the amountdetermined by applying the percentage rate to thevalue of the currency transaction). To measure pricechange over time, the index compiler has to form aview about the quantity underpinning the originaltransaction.

10.127 The purchase of foreign currency can be seenas facilitating the purchase of some desired quantity offoreign goods and services (e.g. expenditure on foreigntravel, or direct import of a commodity). The serviceprice in comparison periods would be expressed as theamount payable on the conversion of a sum of domesticcurrency corresponding to that sum of foreign currencyrequired to purchase the same quantum of foreign goodsand services purchased in the base period.

10.128 A practical translation implies that the ori-ginal foreign currency amount is indexed forward usingchanges in foreign prices, and then converted to domesticcurrency at the prevailing exchange rate, with the pre-vailing percentage margin applied to this new amountto deliver the current price. This current price would becompared to the base price to derive the measure ofprice change. Although the ideal measure for indexingforward the foreign currency amount would be an indexspecifically targeting those foreign goods and servicespurchased by resident households, this is unlikely to befeasible. A practical alternative is to use the publishedaggregate CPI for the foreign countries.

10.129 If a single margin (percentage rate) does notapply to all transactions (e.g. different rates apply todifferent size transactions), then the price measureshould be constructed by reference to a representativesample of base period transactions. The value margin foreach transaction in the current period in the domesticcurrency would be determined by the current domesticcurrency value of each transaction and the current periodpercentage margin applying to each. This captures anyprice change resulting from the value of an underlyingtransaction moving from one price band to another.

Stockbroking services10.130 Consider the case of the purchase of a parcel

of shares in a publicly listed company. In most countries,the purchase has to be arranged through a licensedbroker (stockbroker). The total amount paid by thepurchaser generally comprises three elements: an amount

SOME SPECIAL CASES

195

for the shares (the asset); a fee for the brokerage service;and some form of transaction tax (stamp duty).

10.131 The tax should be considered part of the costof acquiring the shares, as opposed to being part of theprice of the security. The tax should be included alongwith the brokerage cost in the CPI. This is consistentwith both the intention of the tax and the more com-monly accepted basis for the valuation of the shares. (Italso proves convenient to adopt this principle here, as itallows for the – perhaps less contentious – comparabletreatment of taxes on banking services.) Allowing forcurrent tax schedules poses no difficulty in that they willbe widely available in all countries.

10.132 Working from the premise that stockbrokers’fees are more likely to follow a step function than a linearfunction, a price measure would be constructed as fol-lows. First, select a representative sample of transactions(domestic currency values) and calculate the tax payableand the fees payable by reference to the respective sched-ules. The taxes and fees payable in subsequent periods arecalculated by first indexing forward the values of thesample transactions and then applying current fee and taxschedules to the revalued transactions. This methodologyraises twomain issues. First, what is the most appropriateindex for revaluing the transactions and, second, howshould the current schedule of fees be determined?

10.133 The quantum underlying share transactionscan be regarded as forgone consumption, i.e. the quan-tity of goods and services that could have been purchasedinstead. The value of a constant quantum of consump-tion forgone in successive comparison periods thereforewill vary with consumer prices. In this case, the obviouschoice for an escalator would be the CPI itself, based oncurrent period preliminary estimates, or the previousperiod’s result. However, the use of a single period’smovement in the CPI (either previous or current) has thepotential to result in the prices of stockbroking servicesmoving in a way that is unlikely to reflect reality. Thiswould be particularly evident where, for example, thecurrent or previous period’s CPI was influenced sig-nificantly by some one-off, temporary or unusual pricechange (e.g. an oil price shock, or change to health carearrangements). Any ‘‘echoing’’ of abnormal shorter-term price changes through the precise treatment ofstockbrokers’ or similar fees is likely to stretch publiccredibility in the CPI. As an alternative, a 12-monthmoving average CPI might be employed, itself consistentwith a base period comprising a full year’s activity.

10.134 Alternatively, it might be argued that thequantum of shares could be revalued in subsequentperiods in line with movements in equity prices them-selves. According to this view, the price of equities maybe seen as an important influence on the actual costs ofstoring forgone consumption in much the same way astax and fee schedules specific to equity purchases areallowed to enter the calculations described above. Thestrong argument against this treatment is that it assumesthat households have a desire to own equities per se,rather than using them simply as an appropriate vehicleto store forgone consumption. Moreover, the introduc-tion of equity prices within the price indicator is likely toimpart additional short-term volatility to the CPI.

10.135 Competition in the stockbroking industrymeans that there is unlikely to be a common fee schedule.If individual brokers adhere reasonably closely to an in-house fee schedule, obtaining copies of these schedulesshould be a relatively simple matter. On the other hand,if no such fee schedules exist, then a survey of stock-brokers may be required to collect information on a sam-ple of trades (value of trade and fee charged), and thisinformation used to derive a current period fee schedule.

10.136 In the case of sales of shares, the underlyingtransaction represents the exchange of one asset foranother (shares for cash). Quantities underlying sales canbe viewed similarly to share purchases (i.e. some currentperiod basket of consumption goods and services). Inreality, households review their investment strategiesregularly in order to ‘‘store’’ their deferred consumptionin whatever asset class they believe offers the greatestsecurity or prospect for growth. A symmetrical treat-ment of the purchase and sale of shares is particularlyappealing. Unless different fees or taxes apply to sales,there is no need to distinguish between the two in con-structing the index.

Deposit and loan facilities10.137 Accounting for the costs of services provided

by financial intermediaries represents a significant stepup in complexity. Even where a prior decision has beenmade to include such facilities within the scope of theCPI, the service being provided is difficult to visualizecomprehensively, and the prices comprise significantelements that are not directly observable.

10.138 SNA 1993 recommends (6.125 and AnnexIII) that the value of financial intermediation servicesoutput produced by an enterprise should be valued asthe following sum:

� for financial assets involved in financial intermedia-tion, such as loans, the value of services provided bythe enterprise to the borrower per monetary unit onaccount is the margin between the rate payable by theborrower and a reference rate; plus

� for financial liabilities involved in financial inter-mediation, such as deposits, the value of servicesprovided by the enterprise to the lender or depositorper monetary unit on account is the margin betweenthe reference rate and the rate payable by the enter-prise to the lender; plus

� the value of actual or explicit financial intermediationservice charges levied.

10.139 For a summary of the developments innational accounts treatment in this area, and a discussionof the notion of a reference rate, see OECD (1998). Inconcept, SNA 1993 describes the reference rate as therisk-free or pure interest rate. The value of the serviceprovided to a borrower is the difference between theactual amount of interest paid by the borrower andthe lower amount that would have been paid had the ref-erence rate applied. The converse applies for depositors.In practice, it is very difficult to identify the referencerate, and in particular to avoid either volatility in or evennegative measures of the value of such services (as would

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

196

occur if the reference rate lay above the lending rateor below the deposit rate). As a matter of practicalexpediency, an average of borrowing and lending ratesmay be used (with the mid-point being favoured).1 Giventhe complexities involved, expenditures on financialintermediation required for index weighting purposescannot be collected from households in expendituresurveys and so must be estimated by collecting data fromfinancial institutions.10.140 In thinking about the construction of the

index number, it is useful to start by considering the caseof a traditional bank providing a single loan productand a single deposit product; the example will then beextended to a typical bank. In some countries, the tra-ditional bank does not charge direct fees, but all incomeis derived through an interest margin on lending ratesover deposit rates.10.141 The base period weighting value of the finan-

cial service (and so household consumption of such ser-vices) therefore is estimated by applying a margin (theabsolute difference between the reference rate and therate of interest charged to borrowers or paid to deposi-tors) to an aggregate balance (loan or deposit). In linewith the suggested treatment of other financial transac-tions, the construction of accompanying price measuresshould allow for the indexation forward of base periodbalances, applying comparison period margins to cal-culate a money value. The price index is then calculatedas the ratio of comparison period and base period moneyvalues.10.142 Again, the issue of an appropriate escalator

needs to be addressed. While the base period flows ofdeposits and withdrawals can readily be conceptualizedas forgone consumption at base period prices, howshould the balances (stocks) reflecting an accumulationof flows over a number of years be viewed? If an ageprofile for balances were available, accumulated con-sumption forgone could be computed as a movingaverage of the CPI. The more practical alternative is toview base period balances as representing some quan-tum of consumption goods and services at base periodprices, in which case the 12-month moving average CPIcan be used. This is consistent with the idea that house-holds review temporal consumption or investmentdecisions (and so accumulated financial balances) on aregular basis, in this case annually.10.143 The traditional bank has all but disappeared

in some countries and most financial institutions nowderive income from a combination of indirect fees(margins) and direct fees and charges, with the trendbeing for a move from margins towards direct fees. In

this case, the challenge is to construct measures of pricechange that reflect the total price of the service andtherefore capture any offsets between margins and directfees. As with stockbroking services, there may also betaxes levied on financial transactions or balances andthese should also be included in the ‘‘price’’. Frost(2001), for example, provides a description of the morepractical aspects of constructing price indices for depositand loan facilities based on recent Australian experience.

10.144 Given the clear scope for financial inter-mediaries to shift charges between the direct (fee) andindirect (margin) elements, there are clear dangers inconstructing broad measures of margins – known bynational accountants as financial intermediation servicesindirectly measured (FISIM) – independent of direct feesand taxes. Rather, the approach should be to constructprice measures for specific (relatively homogeneous)products that can then be weighted together to provide ameasure for deposit and loan facilities in aggregate, andtaking account of both the direct and indirect elements intotal price. This represents a similar strategy to thatadopted throughout the CPI. For example, the index formotor vehicles is constructed by pricing a sample ofindividual vehicles and weighting these price measures toderive an aggregate, instead of, for example, attemptingto directly construct an index for the supplier or pro-ducer of a range of vehicles.

10.145 The basic process is: first, to select a sample ofrepresentative products from each sampled institution;second, to select a sample of customers for each product,and third, to estimate the total base period value of theservice associated with each product by element (margin,direct fees and taxes). These value aggregates can beviewed as being equivalent to prices for some quantum.Comparison period prices are derived by moving for-ward the base period value aggregates as follows:

� Margin – index forward the base period balance andapply the comparison period margin (the differencebetween the comparison period reference rate and theproduct yield). In practice, the ‘‘price’’ movement isgiven as the product of the indexation factor and theratio of margins.

� Fees – index forward the transaction values for eachsampled account (or profile) and apply the compar-ison period fee structure. The ratio of new aggregatefees to base fees is used to move the fee value aggre-gate. The aggregate fees in the base and comparisonperiods can be constructed as either arithmetic orgeometric averages of the fees calculated for theindividual customers.

� Taxes – as for fees, but use tax schedules instead of feeschedules.

10.146 Appendix 10.1 contains a worked example ofthe calculation of a price index for a single depositproduct.

10.147 Since step function pricing and taxing sche-dules (for example, fees that are only payable after somenumber of transactions or if balances fall below somelevel) are prevalent in financial services, samples of de-tailed customer accounts with all the necessary chargingvariables identified will be required. These samples

1OECD (1998) expresses some concerns about the use of a mid-pointreference rate as a measure of the risk-free rate of interest. There are,however, some doubts about whether the conceptual ideal is for some‘‘risk-free’’ interest rate, or whether a more appropriate concept mightbe the interest rate that would have been struck in the absence offinancial intermediaries (i.e. the rate that would have been struck bydepositors dealing directly with borrowers). Such a rate would haveincorporated the lenders’ knowledge of risk. Taking the mid-point ofthe borrowing and lending rates would appear to be a good means ofestimating this market-clearing rate.

SOME SPECIAL CASES

197

should cover a full year’s activity. If it is not possibleto sample actual accounts, customer profiles may bedeveloped as a fallback option.

10.148 To minimize problems associated with non-response and changing industry structures, a separatereference rate should be constructed for each sampledservice provider. The reference rate should be calculatedin respect of all loans and deposits (including those tobusinesses). Further, to avoid problems that may arisein the timing of accounting entries (e.g. revisions, orinterest income on credit cards), monthly yields, refer-ence rates and margins should be constructed by refer-ence to three-month moving averages of the reportedunderlying balances and interest flows.

Real estate agencyservices

10.149 The services provided by real estate agenciesin the acquisition and disposal of properties can betreated in a number of ways. If the CPI is constructed asan economic cost of use index, these services are out ofscope as they form part of the input costs of the notionallandlords (SNA 1993 also assigns all transfer costs ondwellings to gross fixed capital formation). The transfercosts associated with the acquisition of a dwelling (legalfees, real estate agency fees and taxes) can be included inboth a payments and an acquisitions CPI. They can beclassified as either a cost of home ownership or as adistinctly separate financial service. Although all transfercosts should be included in such measures, the discussionbelow focuses on real estate agents’ fees for simplicity.Price measures for the other elements are calculatedusing similar procedures. In all cases, the general ap-proach is to estimate the current cost of the variousservices relative to, and as they would apply to, somefixed basket of activity in the base period. Consistentwith some of the areas already discussed, this involvesindexing forward the base period expenditures on whichthe fees are charged (to preserve the underlying quan-tum) via some appropriate price index, and then esti-mating the fees payable in the comparison period.

10.150 Real estate agents typically quote their fees assome percentage of the price received for the dwelling. Incommon with other items where charges are determinedas a margin, this needs to be converted to a domesticcurrency price. If the percentage margin is known, theagents’ price for any given transaction (sale/purchase ofa dwelling for a known price) can be computed by mul-tiplying the value of the dwelling by the percentagemargin, and the index can be constructed on the basis ofestimates of both components.

10.151 The methodology chosen for estimating thepercentage margin will depend upon an assessment ofthe variation in margins across and within individualagencies. In the most straightforward case, firms mayoperate with a single percentage margin applicable to alltransactions regardless of value. In other words, at anypoint in time the percentage margins charged may varyby agency, but not by value of transaction within agency.In this case, what is required is an estimate, in each

comparison period, of the average percentage margincharged by agencies. This can be achieved by collectingthe percentage margins, exclusive of any taxes levied onagents’ fees such as value added tax (VAT) or goodsand services tax (GST), from a sample of agencies andderiving an average.

10.152 Percentage margins charged by individualagencies sometimes vary with transaction price (typi-cally declining with increasing prices of dwellings).Where tariffs do vary within agencies, a more sophisti-cated estimation procedure may be required. Using datafrom a sample of transactions from a sample of agents,the relationship between the value of transaction and thepercentage margin can be derived through econometricanalysis. Empirical analysis will be required to deter-mine the precise functional form for this relationship.For example, in the Australian case research has shownthat ordinary least squares regression can be used toestimate this relationship and that the following func-tional form is adequate:

R=a+b1ð1=pÞ+b2ð1=pÞ2

where:R=the commission rate, p=the house price, a=aconstant, and b1 and b2 are parameters to be estimated.

10.153 Estimation of the current period value oftransactions to which the percentage margin appliesdepends on whether real estate agency fees are classified asa cost of housing or as a separate financial service. If theformer, the value of the current period transaction, relativeto the value of the base period transaction, would reflectchanges in house prices. If the latter, where the purchase ofa dwelling is regarded as forgone consumption, the currentperiod value would reflect changes in the CPI itself.

10.154 If a single percentage margin is assumed tooperate, then only a single current period transaction isrequired, i.e. an estimate of the average value of baseperiod transactions at comparison period prices. Forexample, if real estate agency fees are classified as ahousing cost, then the base period price is calculated byapplying the average base period percentage margin tothe average house price in the base period, with anyVAT or GST then added. The comparison period priceis calculated by indexing forward the average base per-iod house price, applying the average comparison periodpercentage margin and adding GST or VAT.

10.155 If a single percentage margin is not assumedto operate, then a sample of representative base periodtransactions is required. The monetary value of themargin on each representative transaction is then cal-culated from published tariffs or from an estimatedfunctional relationship, such as that described above.Comparison period prices are likewise estimated by firstindexing forward each of the base period representativetransactions and then applying the same model. Notethat, in this case, there is no need to exclude any GST orVAT from the initial margins data.

Property insurance services10.156 The construction of reliable price indices for

insurance can be difficult to achieve in practice. This

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

198

section is restricted to a discussion of property insur-ance, as this type of insurance can be assumed to operatein similar ways across countries. It nevertheless providesonly an illustration of the issues that index compilersface, with each sector raising specific conceptual andmeasurement difficulties. For example, in the case of lifeinsurance, insurance policies are often bundled with along-term investment service yielding a financial payoutwhen insured persons survive the policy term. Separa-tion of the service charges relating to the insurance andinvestment elements within a single premium poses sig-nificant problems for index compilers.10.157 For the purposes of the discussion below,

property insurance is defined to include:

– dwelling insurance;

– household contents insurance;

– motor vehicle insurance.

10.158 The common feature of these policies is thatfor a fee (premium), households receive financial com-pensation if a nominated event results in the loss of, ordamage to, designated property. The alternative to pur-chasing insurance is for the household to self-insure. Forhouseholds as a group, the service received is representedby the elimination of the risk of a financial loss. Theappropriate treatment of property insurance in the CPIdepends on whether the CPI is constructed using theacquisitions, use or payments approach.

Payments10.159 Under the payments approach, each of the

above policy types is in scope. In thinking about how thisproperty insurance should be included in the CPI, it isnecessary to consider both the gross premiums payableand the claims receivable by households. The definitionsof gross premiums payable and claims receivable arestraightforward. It is possible, however, to treat claimsreceivable in a number of ways, which will have animpact on either the weight assigned to insurance or theweight assigned to the items insured. Spending oninsurance can be weighted on either a gross basis (i.e.valued using gross premiums payable) or on a net basis(i.e. valued using gross premiums payable less claimsreceivable). Likewise, items which are insured againstloss may also be weighted gross or net (in the latter case,excluding purchases explicitly financed by insuranceclaims receivable). Taken together, this suggests threeplausible alternative treatments:

– gross premiums, net expenditures;

– net premiums, gross expenditures;

– gross premiums, gross expenditures.

10.160 Gross premiums, net expenditures. It may beargued that calculating expenditures net of purchasesfinanced by insurance claims avoids double counting ofthat portion of gross premiums which funds the claims.There are some problems with this approach. First, it isnecessary to assume that all proceeds from insuranceclaims are used to purchase replacement items or torepair damaged items. In some cases, claims receivablemay be to compensate for damage or destruction to the

property of agents beyond the scope of the index (e.g.businesses, government or even other households wherethe CPI reference group covers only some subset ofhouseholds). Households may also choose to use theproceeds for entirely different purposes. Thus the esti-mation of the net expenditure weights is likely to involvesome arbitrary choices. More generally, because moneyis fungible, attempts to restrict coverage only to thoseexpenditures made from selected sources of funds arequestionable. Finally, the potential distortion of weightsfor these items may reduce the usefulness of sub-indicesfor other purposes.

10.161 Net premiums, gross expenditures. Within apayments index, the ‘‘net premiums, gross expenditures’’approach is based on the view that claims receivableshould be regarded as negative expenditure on insurance.This may be seen as an attempt to avoid the doublecounting of expenditures on items financed by claimsreceivable and already included in gross expenditures onother items elsewhere in the index. The net premiumsapproach is much less problematic than the net expen-ditures approach (as at least the impact is restricted tothe weights for insurance). It may, however, be arguedthat the net premiums approach is inconsistent withapproaches adopted for other items in a payments index,in particular mortgage interest and consumer creditcharges, where weights are based on gross payments.Any allowance for interest receipts would be likely toyield negative weights since households are generally netsavers overall.

10.162 The fact that the net premiums approacheffectively measures the value of the insurance service asrequired for indices constructed according to both theacquisitions and use approaches is incidental. The taskhere is to determine the appropriate treatment for apayments-based index.

10.163 Gross premiums, gross expenditures. The‘‘gross premiums, gross expenditures’’ approach is basedon the view that the claims receivable by householdssimply represent one of the sources of funds from whichexpenditures are made. This is the most appealing ap-proach for a payments index, as it recognizes the fungiblenature of money and provides a consistent means ofidentifying both the item coverage of the index and therelative weights by reference only to the actual outlays ofhouseholds.

Use10.164 Under the use approach, dwelling insurance

is out of scope as an input cost of the notional landlord.The weights should relate to the value of the insuranceservice consumed by households. This is defined as beingequal to: gross insurance premiums payable by house-holds, plus premium supplements, less provisions forclaims, less changes in actuarial reserves.

10.165 It is not possible to estimate the nominalvalue of the net insurance service from household expen-diture surveys alone. For weighting purposes, the mostappealing approach is to obtain data from a sample ofinsurance providers, permitting estimation of the ratio ofnet insurance services to gross premiums, and to apply

SOME SPECIAL CASES

199

this ratio to the estimated value of gross premiumsobtained from household expenditure surveys. However,it has not been possible to devise a corresponding pricemeasure that is conceptually sound. For this reason,those countries that have adopted the net concept forweighting purposes are using movements in grossinsurance premiums as a proxy price measure.

Acquisitions10.166 Under the acquisitions approach, all three

items are in scope. Because the objective is to measureprice inflation for the household sector, the expendituresrequired for weighting purposes should reflect the insur-ance companies’ contribution to the inflationary process,which equates to the value of the insurance service as perthe use approach.

Pricing gross insurancepremiums

10.167 The gross insurance premium payable byhouseholds in any one period is determined by the con-ditions of the policy, the administration costs and profitobjectives of the insurance provider, the risk of a claimbeing made and any relevant taxes. For any single policy,the principal quality-determining characteristics (gen-erally specified in the conditions of the policy) can besummarized as being:

– the type of property being covered (dwellings, motorvehicles, etc.);

– the type of cover provided (physical damage, liability,etc.);

– the nature of the compensation (replacement cost,current market value, etc.);

– any limits on the amount claimable;

– the location of the property;

– amount of any excess payable by the insured;

– risks (or events) covered.

10.168 While it is clear that pricing to constantquality requires these conditions to be held fixed, there isalso a question about whether the risk of a claim beingmade should be held constant. In other words, if theincidence of, say, vehicle theft increases, should this beregarded as a quality improvement or simply a pricechange? If, on the one hand, it is argued that as theconsumers’ decision to insure is based on their assess-ment of the likelihood of suffering a loss compared tothe premium charged, the risk factors should be heldconstant. On the other hand, it may be argued that, onceinsured, the consumer simply expects to be compensatedfor any loss. From the perspective of the consumer, anyincrease in risk simply represents an increase in theinsurer’s cost base (which may or may not be passed onto the consumer by way of a price change). Obtainingdata of sufficient reliability to make quality adjustmentsin response to changes in risk is problematic, so inpractice most indices reflect changes in risk as a pricechange.

10.169 In pricing insurance policies, the approachshould be to select a sample of policies representative of

those policies held in the base period and to reprice thesein subsequent periods. Taking dwelling insurance as anexample, base period insurance policies would be takenout to insure dwellings of various values and types (e.g.timber or brick) in different locations. The price samplesshould therefore consist of specifications that aim tocover, in aggregate, as many combinations of thesevariables as is reasonable. While the conditions of thepolicy, the dwelling type and location should be heldconstant over time, the value of the dwelling should beupdated each period to reflect changes in house prices(i.e. the underlying real quantity needs to be preserved).It is important to note that, as the premiums will berelated in some way to the value of the insured property,the price index for insurance can change without therebeing any change in premium schedules.

10.170 Every effort should be made to identify anychanges in the conditions applying to selected policies inorder to facilitate appropriate quality adjustments.Examples would include cessation of coverage for spe-cific conditions and changing the excess (or deductible)paid by the consumer when a claim is made. Estimatesof the value of such changes may be based on theinsurance company’s own assessments of their likelyimpact on the value of total claims payable. If it isassumed that the change in the aggregate value of claimscan be equated to the change in service to the consumer(compared to the service that would have been providedprior to policy renewal), then an appropriate adjustmentcan be made to the premium to provide a (quality-adjusted) movement in price. For example, considerthe case where the excess on a policy is doubled andadvice from the company is that this will result in a 3 percent drop in the aggregate value of claims payable. Thiscould be considered as equivalent to a 3 per centincrease in price.

Using gross premiums as a proxyfor the net insurance service

10.171 The net insurance service charge captures theadministration costs and profits of the insurance provi-der along with any taxes. The problem is that taxes oninsurance are normally levied on the gross premiums.Therefore, if the gross insurance premiums are subject toa high rate of tax, then the taxes will account for an evenhigher proportion of the net insurance service charge.Simply using the gross insurance premium inclusive oftaxes as the price measure understates the real effect ofany increase in the tax rates. This is best illustrated byway of an example.

10.172 For the sake of simplicity, assume that thereare no premium supplements and no actuarial reserves.Then the insurance service charge is given by grosspremiums less provisions for claims. Suppose the onlychange between two periods is a change in the tax rate –from 5 per cent of gross premiums to 20 per cent. Thenthe values in Table 10.10 are likely to be observed.Under this scenario it is clear that the insurance servicecharge has increased from $45 to $60 (an increase of33.3 per cent), yet gross premiums have only increasedby 14.3 per cent.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

200

10.173 Given that changes in the tax rates on grossinsurance premiums are often subject to significant vari-

ation, this is a non-trivial problem. A practical solution isto decompose insurance service into two components –insurance service before tax (or net of tax) and tax oninsurance services. The price measure for the first isconstructed by reference to movements in gross pre-miums net of tax, and the price measure for the second isgiven by changes in taxes on gross premiums. Furtherresearch is required to develop a workable methodologyfor directly measuring changes in prices of insuranceservices before tax.

Table 10.10 Illustration of the impact of taxes on measuresof insurance services ($)

Period Premiumsbefore tax

Tax Grosspremiums

Claims Insuranceservice

1 100 5 105 60 452 100 20 120 60 60

SOME SPECIAL CASES

201

Appendix 10.1 Calculation of a price index for a deposit product(a) Base period sample account. Only a single month’s data is used in this example. In practice, many accounts would be sampledwith each account containing data for a full year.

Taxes

Date Debit (D)or Credit (C)

Transaction Transactionvalue ($)

Tax ($) Balance ($)

456.232 Jan D Over the counter withdrawal 107.05 0.70 348.48

12 Jan C Deposit 4 000.00 2.40 4 346.0813 Jan D EFTPOS1 transaction 50.62 0.30 4 295.1613 Jan D Over the counter withdrawal 371.00 0.70 3 923.4614 Jan D Own ATM2 cash 300.00 0.70 3 622.7614 Jan D Own ATM cash 100.00 0.70 3 522.0616 Jan D Own ATM cash 100.00 0.70 3 421.3616 Jan D Over the counter withdrawal 371.00 0.70 3 049.6616 Jan D Cheque 90.00 0.30 2 959.3619 Jan D Own ATM cash 100.00 0.70 2 858.6619 Jan D Own ATM cash 100.00 0.70 2 757.9619 Jan C Deposit 4 000.00 2.40 6 755.5619 Jan D Cheque 740.00 1.50 6 014.0620 Jan D EFTPOS transaction 76.42 0.30 5 937.3421 Jan D Other ATM cash 20.00 0.30 5 917.0421 Jan D Cheque 100.00 0.70 5 816.3422 Jan D Cheque 43.40 0.30 5 772.6422 Jan D Cheque 302.00 0.70 5 469.9422 Jan D Cheque 37.00 0.30 5 432.6423 Jan D Over the counter withdrawal 371.00 0.70 5 060.9423 Jan D Cheque 72.00 0.30 4 988.6427 Jan D Own ATM cash 150.00 0.70 4 837.9427 Jan D Cheque 73.50 0.30 4 764.1427 Jan D Cheque 260.00 0.70 4 503.4427 Jan D EFTPOS transaction 51.45 0.30 4 451.6928 Jan D Over the counter withdrawal 19.95 0.30 4 431.4428 Jan D Cheque 150.00 0.70 4 280.7429 Jan D Cheque 140.00 0.70 4 140.0430 Jan D Over the counter withdrawal 371.00 0.70 3 768.3430 Jan D Cheque 8.00 0.30 3 760.0430 Jan D Cheque 60.00 0.30 3 699.74Total taxes 21.10

1EFTPOS (Electronic Funds Transfer Point Of Sale).2ATM (Automatic Teller Machine).

Fees

Activity Total no. No. charged Amount($)

Over the counter withdrawal 6 2 6.00EFTPOS transaction 3 0 0.00Own ATM cash 6 0 0.00Own ATM cash 1 1 1.20Cheque 13 3 3.00Deposit 2 2 0.00Total fees 10.20

Fees and taxes are calculated using data in tables (b) and (c), respectively.Source: Woolford (2001)

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

202

(b) Fee schedule. This is a summary of the information typically available from financial institutions. For each period, the tableincludes the number of free transactions and the per transaction charge for additional transactions. A zero number free indicatesthat no transactions are free and a zero charge indicates that all transactions are free.

Description Base period Current period

No. free Charge ($) No. free Charge ($)

Over the counter withdrawal 4 3.00 4 3.00EFTPOS transaction 10 0.50 9 0.50Own ATM cash 10 0.50 9 0.50Other ATM cash 0 1.20 0 1.20Cheque 10 1.00 9 1.00Deposit 0 0.00 0 0.00

Source: Woolford (2001).

(c) Tax schedule. This is a table of tax rates of the type that used to be employed in Australia. The debits tax is levied on all debittransactions to eligible accounts, with the amount charged being set for ranges of transaction values (i.e. using a step function).Financial institutions duty is levied on all deposits, the amount being determined as a percentage of the value of the deposit.

Bank accounts debit tax

Transaction value ($) Tax ($)

Min. Max. Base period Current period

0 1 0.00 0.001 100 0.30 0.30

100 500 0.70 0.70500 5 000 1.50 1.50

5 000 10 000 3.00 3.0010 000+ 4.00 4.00

Financial institutions duty (%)

Base period Current period

0.06 0.06

Source: Woolford (2001).

(d) Interest data. The table presents, in summary form, the balances and annualized interest flows derived by taking movingaverages of data reported by financial institutions. Interest rates and margins are calculated from the balances and flows.

Base period Current period

Balance($ million)

Interest($ million)

Interest rate(%)

Margin(%)

Balance($ million)

Interest($ million)

Interest rate(%)

Margin(%)

Deposit productsPersonal accounts 22 000 740 3.3636 2.4937 23 600 775 3.2839 2.3971

Current accounts 6 000 68 1.1333 4.7241 6 600 75 1.1364 4.5446Other accounts 16 000 672 4.2000 1.6574 17 000 700 4.1176 1.5634

Business accounts 25 000 920 3.6800 2.1774 28 000 1 000 3.5714 2.1096Total deposit accounts 47 000 1 660 3.5319 2.3255 51 600 1 775 3.4399 2.2411Loan productsPersonal accounts 42 000 3 188 7.5905 1.7331 46 000 3 400 7.3913 1.7103Business accounts 28 000 2 540 9.0714 3.2140 31 000 2 700 8.7097 3.0287Total loan accounts 70 000 5 728 8.1829 2.3255 77 000 6 100 7.9221 2.2411Reference rate 5.8574 5.6810

Source: Woolford (2001).

SOME SPECIAL CASES

203

(e) CPI data. The table presents data required to derive the indexation factor. This example follows the Australian practice of aquarterly CPI. If a monthly CPI is produced, 12-term moving averages would be required.

t�5 t�4 t�3 t�2 t�1

All groups 117.5 121.2 123.4 127.6 129.14-term moving average 122.4 125.3Indexation factor (movement) 1.0237

Source: Woolford (2001).

(f) Projected current period sample account. The opening balance and transaction values are derived by applying the indexationfactor to the base period amounts. The tax payable is determined by reference to the data in table (c). Fees payable aredetermined by reference to the data in table (b).

Taxes

Date Debit (D) orCredit (C)

Transaction Transaction value ($) Tax ($) Balance ($)

467.042 Jan D Over the counter withdrawal 109.59 0.70 356.75

12 Jan C Deposit 4 094.75 2.46 4 449.0513 Jan D EFTPOS transaction 51.82 0.30 4 396.9313 Jan D Over the counter withdrawal 379.79 0.70 4 016.4414 Jan D Own ATM cash 307.11 0.70 3 708.6314 Jan D Own ATM cash 102.37 0.70 3 605.5616 Jan D Own ATM cash 102.37 0.70 3 502.5016 Jan D Over the counter withdrawal 379.79 0.70 3 122.0116 Jan D Cheque 92.13 0.30 3 029.5719 Jan D Own ATM cash 102.37 0.70 2 926.5119 Jan D Own ATM cash 102.37 0.70 2 823.4419 Jan C Deposit 4 094.75 2.46 6 915.7319 Jan D Cheque 757.53 1.50 6 156.7020 Jan D EFTPOS transaction 78.23 0.30 6 078.1721 Jan D Other ATM cash 20.47 0.30 6 057.4021 Jan D Cheque 102.37 0.70 5 954.3322 Jan D Cheque 44.43 0.30 5 909.6022 Jan D Cheque 309.15 0.70 5 599.7522 Jan D Cheque 37.88 0.30 5 561.5723 Jan D Over the counter withdrawal 379.79 0.70 5 181.0823 Jan D Cheque 73.71 0.30 5 107.0827 Jan D Own ATM cash 153.55 0.70 4 952.8327 Jan D Cheque 75.24 0.30 4 877.2827 Jan D Cheque 266.16 0.70 4 610.4327 Jan D EFTPOS transaction 52.67 0.30 4 557.4628 Jan D Over the counter withdrawal 20.42 0.30 4 536.7328 Jan D Cheque 153.55 0.70 4 382.4829 Jan D Cheque 143.32 0.70 4 238.4630 Jan D Over the counter withdrawal 379.79 0.70 3 857.9830 Jan D Cheque 8.19 0.30 3 849.4930 Jan D Cheque 61.42 0.30 3 787.77Total taxes 21.21

Fees

Activity Total No. No. charged Amount ($)

Over the counter withdrawal 6 2 6.00EFTPOS transaction 3 0 0.00Own ATM cash 6 0 0.00Own ATM cash 1 1 1.20Cheque 13 4 4.00Deposit 2 2 0.00Total fees 11.20

Source: Woolford (2001).

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

204

(g) Indices for current accounts. This table brings the results together. The current period value aggregates are derived as follows.For margins – the base period aggregate is multiplied by the product of the indexation factor (e) and the ratio of the current andbase period margins for current accounts (d). For fees – the base period aggregate is multiplied by the ratio of total fees payableon the sample account in the current period (f ) and the base period (a). For taxes – the same procedure is followed as for fees.

Component Base period Current period

Valueaggregate ($)

Index Valueaggregate ($)

Index

Margins 28 344 100.0 27 913 98.5Fees 11 904 100.0 13 071 109.8Taxes 14 739 100.0 14 818 100.5Total 54 987 100.0 55 803 101.5

Source: Woolford (2001).

SOME SPECIAL CASES

205

11ERRORS AND BIAS

Introduction11.1 This chapter discusses the general types of

potential error to which all price indices are subject. Theliterature on consumer price indices (CPIs) discussesthese errors from two perspectives, and this chapterpresents the two perspectives in turn. First, the chapterdescribes the sources of sampling and non-samplingerror that arise in estimating a population CPI from asample of observed prices. Second, the chapterreviews the arguments made in numerous recent studiesthat attribute bias to CPIs as a result of insufficientlyaccurate treatment of quality change, consumer sub-stitution and other factors. It should be emphasized thatmany of the underlying issues discussed here are dealtwith in much greater detail elsewhere in the manual.

Types of error11.2 One of the main objectives of a sample survey

is to compute estimates of population characteristics.Such estimates will never be exactly equal to the popu-lation characteristics. There will always be some error.Table 11.1 gives a taxonomy of the different types oferror. See also Balk and Kersten (1986) and Dalen (1995)for overviews of the various sources of stochastic andnon-stochastic errors experienced in calculating a CPI.Two broad categories can be distinguished: samplingerrors and non-sampling errors.

Sampling error11.3 Sampling errors are due to the fact that an

estimated CPI is based on samples and not on a completeenumeration of the populations involved. Samplingerrors vanish if observations cover the complete popula-tion. As mentioned in previous chapters, statistical offi-ces usually adopt a fixed weight price index as the object

of estimation. A fixed weight index can be seen as aweighted average of partial indices of commodity groups,with weights being expenditure shares. The estimationprocedures that most statistical offices apply to a CPIinvolve different kinds of samples. The most importantkinds are:

� for each commodity group, a sample of commoditiesto calculate the partial price index of the commoditygroup;

� for each commodity, a sample of outlets to calculatethe elementary price index of the commodity fromindividual price observations;

� a sample of households needed for the estimation ofthe average expenditure shares of the commoditygroups. (Some countries use data from nationalaccounts instead of a household expenditure survey toobtain the expenditure shares.)

11.4 The sampling error can be split into a selectionerror and an estimation error. A selection error occurswhen the actual selection probabilities deviate from theselection probabilities as specified in the sample design.The estimation error denotes the effect caused by usinga sample based on a random selection procedure. Everynew selection of a sample will result in different elements,and thus in a possibly different value of the estimator.

Non-sampling error11.5 Non-sampling errors may occur even when the

whole population is observed. They can be subdividedinto observation errors and non-observation errors.Observation errors are the errors made during the processof obtaining and recording the basic observations orresponses.

11.6 Overcoverage means that some elements areincluded in the survey which do not belong to the targetpopulation. For outlets, statistical offices usually haveinadequate sampling frames. In some countries, forinstance, a business register is used as the sampling framefor outlets. In such a register, outlets are classified accord-ing to major activity. The register thus usually exhibitsextensive overcoverage, because it contains numerousoutlets which are out of scope from the CPI perspective(e.g. firms that sell to businesses rather than to house-holds). In addition, there is usually no detailed infor-mation on all the commodities sold by an outlet, so it ispossible that a sampled outlet may turn out not to sell aparticular commodity at all.

11.7 Response errors in a household expendituresurvey or price survey occur when the respondent doesnot understand the question, or does not want to give the

Table 11.1. A taxonomy of errors in a consumer price index

Total error:Sampling errorSelection errorEstimation error

Non-sampling errorObservation errorOvercoverageResponse errorProcessing error

Non-observation errorUndercoverageNon-response

207

right answer, or when the interviewer or price collectormakes an error in recording the answer. In householdexpenditure surveys, for example, households appear tosystematically underreport expenditures on commoditygroups such as tobacco and alcoholic beverages. In mostcountries, the main price collection method is by personswho regularly visit outlets. They may return with pricesof unwanted commodities.

11.8 The price data are processed in different stages,such as coding, entry, transfer and editing (control andcorrection). At each step mistakes, so-called processingerrors, may occur. For example, at the outlets the pricecollectors write down the prices on paper forms. Afterthe collectors have returned home, a computer is usedas the input and transmission medium for the priceinformation. It is clear that this way of processing pricesis susceptible to errors.

11.9 Non-observation errors are made when theintended measurements cannot be carried out. Under-coverage occurs when elements in the target populationdo not appear in the sampling frame. The sampling frameof outlets can have undercoverage, which means thatsome outlets where relevant commodities are purchasedcannot be contacted. Some statistical offices appear toexclude mail order firms and non-food market stalls fromtheir outlet sampling frame.

11.10 Another non-observation error is non-response.Non-response errors may arise from the failure to obtainthe required information in a timely manner from all theunits selected in the sample. A distinction can be drawnbetween total and partial (or item) non-response. Totalnon-response occurs when selected outlets cannot becontacted or refuse to participate in the price survey.Another instance of total non-response occurs when mailquestionnaires and collection forms are returned by therespondent and the price collector, respectively, after thedeadline for processing has passed. Mail questionnairesand collection forms that are only partially filled in areexamples of partial non-response. If the price changesof the non-responding outlets differ from those of theresponding outlets, the results of the price survey will bebiased.

11.11 Total and partial non-response may also beencountered in a household expenditure survey. Totalnon-response occurs when households drawn in the sam-ple refuse to cooperate. Partial non-response occurs, forinstance, when certain households refuse to give infor-mation about their expenditure on certain commoditygroups.

Measuring error and bias

Estimation of variance11.12 The variance estimator depends on both the

chosen estimator of a CPI and the sampling design. Boon(1998) gives an overview of the sampling methods thatare applied in the compilation of CPIs by various Euro-pean statistical institutes. It appeared that only four ofthem use some sort of probability techniques for outletselection, and only one uses probability sampling foritem selection. In the absence of probability techniques,

so-called judgemental and cut-off selection methods areapplied.

11.13 In view of the complexity of the (partiallyconnected) sample designs in compiling a CPI, an inte-grated approach to variance estimation appears to beproblematic. That is, it appears to be difficult to presenta single formula for measuring the variance of a CPI,which captures all sources of sampling error. It is, how-ever, feasible to develop partial (or conditional) mea-sures, in which only the effect of a single source ofvariability is quantified. For instance, Balk and Kersten(1986) calculated the variance of a CPI resulting fromthe sampling variability of the household expendituresurvey, conditional on the assumption that the partialprice indices are known with certainty. Ideally, all theconditional sampling errors should be put together in aunifying framework in order to assess the relativeimportance of the various sources of error. Under ratherrestrictive assumptions, Balk (1989a) derived an inte-grated framework for the overall sampling error ofa CPI.

11.14 There are various procedures for trying toestimate the sampling variance of a CPI. Design-basedvariance estimators (that is, variances of Horvitz–Thompson estimators) can be used, in combination withTaylor linearization procedures, for sampling errorsarising from a probability sampling design. For instance,assuming a cross-classified sampling design, in whichsamples of commodities and outlets are drawn inde-pendently from a two-dimensional population, withprobabilities proportional to size (PPS) in both dimen-sions, a design-based variance formula can be derived.In this way Dalen and Ohlsson (1995) found thatthe sampling error for a 12-month change of the all-commodity Swedish CPI was of the order of 0.1–0.2per cent.

11.15 The main problem with non-probabilitysampling is that there is no theoretically acceptableway of knowing whether the dispersion in the sampledata accurately reflects the dispersion in the population.It is then necessary to fall back on approximationtechniques for variance estimation. One such techniqueis quasi-randomization (see Sarndal, Swensson andWretman (1992, p. 574)), in which assumptions aremade about the probabilities of sampling commodi-ties and outlets. The problem with this method is thatit is difficult to find a probability model that ade-quately approximates the method actually used foroutlet and item selection. Another possibility is to use areplication method, such as the method of randomgroups, balanced half-samples, jackknife, or bootstrap.This is a completely non-parametric class of methodsto estimate sampling distributions and standard errors.Each replication method works by drawing a largenumber of sub-samples from the given sample. Fromeach sub-sample the parameter of interest can beestimated. Under rather weak conditions, it can beshown that the distribution of the resulting esti-mates approximates the sampling distribution of theoriginal estimator. For more details on the replicationmethods see Sarndal, Swensson and Wretman (1992,pp. 418–445).

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

208

Qualitative descriptions ofnon-sampling errors11.16 It is still more difficult to obtain quantitative

measures of the non-sampling errors. Thus the useof qualitative indications is the only possibility. Forinstance, the coverage of the sampling frames as a proxyof the target populations can be addressed (includinggaps, duplications and definitional problems). The per-centage of the target outlet samples from which re-sponses or usable price data were obtained (i.e. theresponse rates) can be provided. Any known differencein the prices of responding outlets and non-respond-ing outlets can be described, as can an indication ofthe method of imputation or estimation used to com-pensate for non-response. Several categories of non-sampling errors provide the bulk of the bias issuesdiscussed below.

Procedures to minimize errors11.17 The estimation error can be controlled by

means of the sampling design. For example, by increas-ing the sample size, or by taking selection probabilitiesproportional to some well-chosen auxiliary variable, theerror in the estimated CPI can be reduced. The choice ofan adequate sampling design for the CPI is an extremelycomplex matter. The target population is the set of allgoods and services that are acquired, used or paid for byhouseholds from outlets in a particular time period. Aproper probability sampling procedure selects a sampleby a random mechanism in which each good or service inthe population has a known probability of selection. Incombination with a Horvitz–Thompson estimator, sucha probability sampling design will produce an index thatis (approximately) unbiased and precise.11.18 The following three probability sampling

designs are used extensively in survey practice: simplerandom (SI) sampling, probability proportional to size(PPS) sampling, and stratified sampling with SI or PPSsampling per stratum. The advantage of SI sampling isits simplicity; it gives each population element the sameprobability of being included in the sample. PPS sam-pling has the advantage that the more important ele-ments have a larger chance of being sampled than theless important ones. For instance, at Statistics Swedenthe outlets are selected with probabilities proportional tosome proxy for size, namely their number of employees.Unequal probability designs can lead to a substantialvariance reduction in comparison with equal probabilitydesigns. In stratified sampling, the population is dividedinto non-overlapping sub-populations called strata. Forinstance, at the United Kingdom Office for NationalStatistics the population of outlets is split by outlet type(multiple, independent or specialist) to form differentstrata. In each stratum a sample is selected according toa certain design. One of the reasons why stratifiedsampling is so popular is that most of the potentialgain in precision of PPS sampling can be capturedthrough stratified selection with SI sampling within well-constructed strata. Stratified sampling is in severalaspects simpler than PPS sampling.

11.19 Because appropriate sampling frames arelacking, samples are frequently obtained by non-probability methods. Judgemental (or expert choice)sampling is one form of non-random selection. In thiscase an expert selects certain ‘‘typical’’ elements wheredata are to be collected. With skill on the part of theexpert a fairly good sample might result, but there is noway to be sure. A more sophisticated non-probabilitymethod is quota sampling. In quota sampling the pop-ulation is firstly divided into certain strata. For eachstratum, the number (quota) of elements to be includedin the sample is fixed. Next the interviewer in the fieldsimply fills the quotas, which means in the case of out-let sampling that the selection of the outlets is ulti-mately based on the judgement of the price collectors.Another non-probability method is cut-off sampling,which means that a part of the target population is delib-erately excluded from the sample selection process. Inparticular, this procedure is used when the distribution ofthe value of some auxiliary variable is highly skewed. Forinstance, a large part of the population may consist ofsmall outlets whose contribution to total sales is modest.A decision may then be taken to exclude from the sam-pling frame the outlets with the lowest sales. Becausethe selection is non-random, non-probability methodsusually lead to more or less biased estimates. Empiricalresults of research undertaken by Statistics Netherlandsnevertheless show that non-probability selection meth-ods do not necessarily perform worse, in terms of themean square error, than probability sampling techniques(De Haan, Opperdoes and Schut, 1997).

11.20 Provided that the sampling design is given, thesampling variance of an estimated (all-commodities) CPIcan in general be lowered by:

– enlarging the samples of households, commoditiesand outlets;

– the application of suitable stratifications to the var-ious populations (e.g. grouping commodities withrespect to similarity of price changes).

11.21 It is important to allocate optimally the avail-able resources both between and within the different CPIsamples, since badly allocated samples may lead tounnecessarily high sampling errors. The Swedish varianceestimation results, presented in Dalen and Ohlsson(1995), show that the error resulting from commoditysampling is relatively high compared with the errorresulting from outlet sampling. In this case, it is worth-while increasing the sample size of commodities andreducing the sample size of outlets.

11.22 A systematic analysis of sampling errors offerspossibilities for improving or reducing cost. The pro-blem of optimum sample allocation is usually for-mulated as the determination of the sizes of the samplesof commodities and outlets, and their distribution overthe strata that minimizes the sampling error of an all-commodities CPI, subject to the available budget.

11.23 As already mentioned, a business register isusually not an adequate sampling frame for outlets,because it provides extensive overcoverage. It is recom-mended to set up an appropriate sampling frame byenumeration of the main outlets within each sampled

ERRORS AND BIAS

209

municipality. Such enumeration yields a list of all outletsin a municipality together with the commodity groupsthat belong to their assortments. A less expensive way toorganize an outlet sampling frame is to ask the pricecollectors – who may be assumed to know the localsituation well – to make a list of outlets where purchasesare made by households.

11.24 The populations of commodities (and vari-eties) and outlets are continually changing through time.The composition of most commodity groups is notconstant over time, because commodities disappear fromthe market and new ones appear. The passage of timealso plays a disturbing role with respect to the outlet pop-ulation: outlets close, temporarily or permanently; newoutlets emerge; the importance of some outlets dimin-ishes or increases. The samples of commodities (andvarieties) and outlets should be reviewed and updatedperiodically to maintain their representativity with re-spect to the current buying habits of the households.

11.25 Response errors caused by the underreportingof certain categories of household expenditure can beadjusted by using producer-based estimates from thenational accounts (see Linder (1996) for an example).Measurement errors by price collectors can be reducedby providing them with hand-held computers for dataentry. In this way the validation of observed prices can beexecuted at the point of price collection (i.e. in the out-let), by means of an automatic comparison of the cur-rently observed price quote with the previously observedone (by setting a limit on the percentage price change)and with the price quotes obtained from other outlets (bysetting suitable upper and lower limits). Details areprovided by Haworth, Fenwick and Beaven (1997).

11.26 It is useful to appoint data collection super-visors to conduct quality assurance checks on the datacollectors. It is also a good idea to organize regularlymeetings where price collectors and statisticians fromthe head office can share their experiences. In this way,the statisticians will keep in touch with the conditionsin the field, and may take the opportunity to providemore information about frequently made price collectionerrors and new representative goods.

11.27 It is important to check the collected price datafor processing errors and, where possible, to correct theseerrors. This activity is called data editing. When editing iscarried out on individual observations, it is called micro-editing. When the resources to spend on data editingmust be minimized, while at the same time maintaining ahigh level of data quality, selective editing and macro-editing are possibilities. Selective editing is a form oftraditional micro-editing, in which the number of editsis kept to a minimum. Only those edits which have animpact on the survey results are carried out. Macro-editing offers a top-down approach. The edits are carriedout on aggregated data (for instance, the price indexnumbers of a commodity group) instead of individualrecords (for example, price observations). Micro-editingof individual records is then carried out only if macro-edits raise suspicion. In particular, attention should bepaid to outliers among the observations.

11.28 Non-response usually introduces selection bias.There are three methods for the treatment of missing

price observations. First, the corresponding price can beexcluded from the data set of previous prices, so that theset of previous prices is ‘‘matched’’ with the set of currentprices. Second, this matching can be achieved by using animputed (or artificial) price for the missing one. Theimputed price can be calculated by either carrying for-ward the previous price observation or by extrapolatingthe previous price observation using the change of otherprice observations for the same commodity. Third, thereis the possibility to reweight the sample. The objective ofreweighting is to inflate the weight given to the prices ofthe responding outlets. This compensates for those pricesthat are lost by non-response.

11.29 In a household expenditure survey, missingdata are usually imputed with the help of information onthe same household from a previous observation periodor other households from the same observation period.To reduce bias in the average expenditure pattern arisingfrom selective non-response, a household expendituresurvey sample of households is generally post-stratifiedby a number of household characteristics, such asincome, composition and size.

Types of bias11.30 This section reviews several categories of error,

either in pricing or in index construction, that potentiallycan lead to bias in the overall CPI. The emphasis here ison the categorization of errors, along with some con-sideration of their likely size, rather than on methods toreduce or eliminate the errors. The question might ariseof why such a discussion is necessary, since such issues asquality change, and the appropriate methods for hand-ling them in the CPI, are dealt with at both a conceptualand operational level in other chapters.

11.31 The reason this chapter addresses the topic ofCPI bias per se is the great surge in interest in pricemeasurement problems during the mid-1990s. Especiallyin the United States, the view became widespread thatthe CPI was subject to systematic upward biases becauseof the failure to deal adequately with consumer sub-stitution, product quality improvements, and the intro-duction of new items and services. Moreover, it wasrecognized, first, that the existence of such upward biaswould have fundamental implications for the measure-ment of recent trends in output and productivity, andsecond, that the elimination of upward bias could sub-stantially improve the government budget situationthrough reduced government expenditures and increasedtax revenues (see, for example, Eldridge (1999) andDuggan and Gillingham (1999)). These discoveries led toa series of papers and reports on CPI measurementproblems, often accompanied by point estimates ofaggregate bias.

11.32 Prominent examples of these quantitative stu-dies of bias are those by the Advisory Commissionto Study the CPI (United States Senate, 1996), Con-gressional Budget Office (1994), Crawford (1998),Cunningham (1996), Dalen (1999a), Diewert (1996c),Lebow, Roberts and Stockton (1994), Lebow and Rudd(2003), Shapiro and Wilcox (1997b), Shiratsuka (1999),

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

210

White (1999), and Wynne and Sigalla (1994). Responsesand estimates by statistical agencies include those pro-vided by Abraham et al. (1998), US Bureau of LaborStatistics (1998), Ducharme (1997), Edwards (1997),Fenwick (1997), Lequiller (1997), Moulton (1996b), andMoulton and Moses (1997). Among the many otherdiscussions of the CPI bias issue are those reported byBaker (1998), Boskin et al. (1998), Deaton (1998), Die-wert (1998a), Krueger and Siskind (1998), Nordhaus(1998), Obst (2000), OECD (1997), Pollak (1998), Pop-kin (1997), and Triplett (1997).11.33 Two points are worth making at the outset

with respect to measuring bias in CPIs. First, the issuehas usually been addressed in the context of the cost ofliving index (COLI). That is, the CPI bias has beendefined as the difference between the rate of increase inthe CPI and the rate of increase in a true COLI. Manyauthors on bias have taken as given that the COLIshould be the CPI’s measurement objective. Somewhatdifferent conclusions might be reached if the indexobjective were taken to be a pure price index. Notably,the gains in consumer welfare from a widening array ofnew goods, or the ability of consumers to substituteaway from items with increasing relative prices, might bedeemed irrelevant and an index that ignored those fac-tors might not be judged biased on that account.11.34 The second point is that CPI bias is not

amenable to estimation with the same level of rigour asthat used in CPI variance estimation. Since the COLI orother ideal target index is unobserved, analysts have beenforced to rely in part on conjectures and on general-izations from fragmentary empirical evidence in orderto quantify the extent of bias. The notable exceptionsare with respect to substitution bias, when traditionalLaspeyres indices and indices using superlative formulaecan be computed using the same underlying price andexpenditure data, and the differences construed as ameasure of the upward bias from use of the Laspeyresformula.11.35 Several different taxonomies of bias have

appeared in the literature mentioned above. It is suffi-cient, however, to employ four categories roughly cor-responding to those set forth in the best-known study,namely the Final report of the Advisory Commission toStudy the CPI (the Boskin Commission), established bythe United States Senate Finance Committee in 1995.These categories are: upper-level substitution bias; ele-mentary aggregate bias; quality change and new goodsbias; and new outlet bias.11.36 These categories can be further broken down

into two subgroups according to whether they refer toerrors in individual price measurements or errors in com-puting index series. Quality change bias and new goodsbias arise because of failures to measure adequately thevalue to consumers of individual goods and services thatappear in (or disappear from) the marketplace. It shouldbe recognized that discussions of ‘‘new goods’’ problemsapply equally to all products, whether goods or services.At a conceptual level, it can be difficult to distinguishthese two biases from each other. Operationally, how-ever, quality change bias pertains to the procedures forcomparing new products or models with the older pro-

ducts they replace in the CPI samples. In general, newgoods bias can be thought of as applying to wholly newtypes of products, or products that would not entersamples routinely through forced replacement. Newoutlet bias, sometimes referred to as outlet substitutionbias, is similar to new goods bias but is focused on theappearance of new types of stores or marketing methodsthat offer goods at lower prices or higher quality.

11.37 The other categories of bias refer to the pro-cedures for constructing index values from componentseries. As noted throughout this manual, CPI construc-tion can be thought of as taking place in two steps, or attwo levels. At the lower level, individual price quotationsare combined; at the upper level, these basic indices areaggregated together. Corresponding to these two levelsare two forms of potential bias. Elementary aggregatebias involves the averaging formulae used to combineprice quotations into basic indices. Upper-level substi-tution bias applies to the formulae used to combine thoseelementary aggregates into higher-level indices. Thesecomponents of potential bias, and the means used tomeasure them, are discussed in more detail below.

Components of bias

Upper-level substitution bias11.38 Upper-level substitution bias is perhaps the

most widely accepted source of CPI bias, and the kindwith which economists are most familiar from textbookexpositions of price index theory and practice. Simplystated, it arises when CPIs employ the Laspeyres for-mula (see Chapter 17), which is well known to providean upper bound on a cost of living index under certainassumptions about consumer behaviour. As noted inparagraph 11.34 above, quantitative measures of upper-level substitution bias can be generated by comparingLaspeyres price indices to Fisher ideal, Tornqvist orother superlative indices. Under certain assumptionsabout, for example, constant preferences, these will standas relatively precise bias estimates.

11.39 Genereux (1983) and Aizcorbe and Jackman(1993) provide such index comparisons and estimates ofupper-level substitution bias using actual CPI indexseries for Canada and the United States, respectively.Other early studies by Braithwait (1980) and Manser andMcDonald (1988) estimate the substitution bias in Uni-ted States national account indices. In lieu of superlativeindices, the Braithwait study uses estimated exact costof living indices based on demand system estimation.A similar estimate for the Netherlands is provided byBalk (1990). In these studies, the existence of an upwardbias from the Laspeyres formula is demonstrated con-sistently. The biases in the annual index changes in indi-vidual years are relatively small, averaging 0.1 to 0.3percentage points, and depend empirically on such fac-tors as the distance from the Laspeyres base period, thelevel of index detail at which the alternative formulae areapplied, and whether the superlative index is of the fixedbase or chained variety.

11.40 The major differences between Laspeyres andsuperlative indices derive from the variation in relative

ERRORS AND BIAS

211

prices over the period being compared, and from the shiftin quantities consumed towards those index categoriesthat have fallen in relative price. This leads to severalconclusions:

� If index movements are characterized by continuing,uniform drift in relative prices over time, with accom-panying drifts in consumption, the size of the annualLaspeyres bias will tend to increase with the distancefrom the base period. (Greenlees (1997) notes, how-ever, that there is little evidence for this phenomenonin the United States; see also Szulc (1983).)

� Under the same circumstances, reducing the expendi-ture weight chaining interval will work to reduce theupper-level substitution bias in the Laspeyres CPI. Themore frequent chaining will increase the weight givento indices that are falling in relative price, therebyreducing the rate of CPI growth. Conversely, if thereis ‘‘bouncing’’ in relative index movements, frequentchaining can lead to an upward ‘‘chain drift’’ in aLaspeyres index.

� Upper-level substitution bias will tend to be largerduring periods of higher inflation, if these periods alsohave greater relative price variation. Little empiricalevidence exists on this point, however.

11.41 The concept of upper-level substitution biashas been derived and discussed in the context of cost ofliving index theory, but an equivalent bias may be definedfrom the perspective of the pure price index. If the Fisherideal or other superlative index is judged preferable onthe basis of its symmetric treatment of base period andcurrent period expenditure patterns, then the differencebetween that index and a Laspeyres could be interpretedas a measure of representativity bias. A similar argumentcould be applied with respect to lower-level substitutionbias within elementary index cells.

11.42 Recently, Lebow andRudd (2003) have definedand estimated another category of bias related to upper-level aggregation. They concluded that the consumerexpenditure survey weights used in the United States CPIwere subject to error because of, for example, under-reporting of alcohol and tobacco expenditures. This willlead to a weighting bias if the errors in relative weight arecorrelated with component index changes. (Sources for,and problems in, expenditure weight estimation are dis-cussed in detail in Chapter 4.)

Elementary aggregate bias11.43 Elementary aggregate bias can be divided into

two components: formula bias and lower-level sub-stitution bias. An elementary index in the CPI is biasedif its expectation differs from its measurement objective.The term formula bias (or functional form bias) is usedhere to denote a situation in which the elementary indexformula has an upward bias relative to the pure priceindex. When the measurement objective is a cost ofliving index, the elementary index formula suffers fromlower-level substitution bias (or within-stratum sub-stitution bias) if it does not reflect consumer substitutionamong the items contained in that index cell. Thus,given any elementary index formula, the two forms of

bias can be distinguished according to the objective ofthe elementary index.

11.44 Chapters 9 and 20 of this manual discuss thecharacteristics of alternative elementary index formulae.A key result is that the Carli formula for the arithmeticaverage of ratios has an upward bias relative to thetrend in average item prices. Consequently, Eurostat hasprohibited use of this formula in computations for theHarmonized Indices of Consumer Prices (HICPs). Theweighted formulae used in basic indices of the UnitedStates CPI had some characteristics of the Carli formulaprior to procedural and computational changes made in1995 and 1996. The problems and the methods chosento address them are discussed, for example, by Reins-dorf and Moulton (1997) and Moulton (1996b).

11.45 The ratio of arithmetic averages (Dutot) andgeometric mean (Jevons) formulae eliminate formulabias as defined here, and both are permitted by Eurostat.Their expectations differ, however, when item prices donot change at a uniform rate. The differences provide oneway of evaluating the potential importance of lower-levelsubstitution bias. The geometric mean formula is exactfor a cost of living index if consumers follow the Cobb–Douglas behavioural model, whereas the formula basedon the ratio of arithmetic averages corresponds to zero-substitution behaviour. Thus, if the goal is to approx-imate a cost of living index, the geometric mean formulais likely to be judged preferable.

11.46 In the future, scanner data may make it pos-sible to record item-level consumption data at a daily,weekly or monthly frequency and to use those data insuperlative index calculations. Currently, however, it isimpossible to employ superlative formulae to computeelementary CPI indices. Some assumption, such as theCobb–Douglas, must be made in order to approximate acost of living index. Note that the substitution that theindex ideally should reflect involves consumer choiceamong all the items in the cell: different products, prod-ucts in different outlets, different package sizes of thesame product, or the same product offered for sale atdifferent times of the period to which the index applies(see Dalton, Greenlees and Stewart (1998)). Thus, theappropriate degree of assumed substitution behaviourshould depend, in principle, on the dimensions of varietywithin the item category.

11.47 The method used by the statistical agency forsampling items within a category will determine the effec-tiveness of formula choice in dealing with lower-levelsubstitution bias. For example, if only a single repre-sentative item is chosen to represent the category, theindex formula will fail to reflect the consumer response toany relative price change in the universe of items. Moregenerally, the geometric mean formula index suffers froman upward bias in small samples, so lower-level sub-stitution bias may be underestimated in empirical com-parisons of the geometric mean to other index formulae.White (1999) discusses the relationship between samplingerror and bias estimates. See also McClelland andReinsdorf (1999) on the small sample bias in the geo-metric mean.

11.48 The impact of formula choice can be estimatedwith some degree of precision over a given historical

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

212

period. Any corresponding bias, however, can be esti-mated only by assuming that the geometric mean orother functional form successfully approximates theindex’s measurement objective.11.49 As implied by the above discussion, the impor-

tance of elementary aggregate bias will vary by country,depending on the particular index formulae used, thedegree of heterogeneity within index strata, and thesampling methods employed. Also, as with upper-levelsubstitution bias, elementary aggregate bias will varywith the overall level of inflation in the economy ifabsolute and relative price changes are correlated.11.50 The performance of any formula for elemen-

tary aggregate calculation will also be affected by themethods used by the statistical agency to handle specialsituations, such as seasonal goods and other productsthat are temporarily unavailable. Armknecht andMaitland-Smith (1999) discuss how the failure to imputemissing prices can lead to bias in the modified Laspeyresand other index formulae.

Quality change and new products bias11.51 Discussion of potential CPI biases arising from

inadequate quality adjustment has a long history. Forexample, the Stigler Committee report on United Statesprice statistics (Price Statistics Review Committee, 1961)indicated that ‘‘if a poll were taken of professionaleconomists and statisticians, in all probability they woulddesignate (and by a wide majority) the failure of the priceindices to take full account of quality changes as the mostimportant defect of these indices’’. In most studies of bias,unmeasured or mismeasured quality change is also thelargest contributor to the total estimated bias. Just asquality adjustment is widely recognized as an extremelydifficult process, however, it is correspondingly difficult tomeasure any quality change bias.11.52 Unlike substitution bias, which can be esti-

mated by comparison of alternative formulae, qualitychange bias must be analysed on a product-by-productbasis. Products and their associated index componentswill experience widely varying rates of quality changeover time. Moreover, the methods used for qualityadjustment will also vary. Whereas the linking methodmay dominate in terms of frequency of use, importantindex components may employ production cost, hedo-nic adjustment, or the other methods described inChapters 7 and 21.11.53 A crucial point to recognize is that the direc-

tion of overall quality change does not imply the direc-tion of any quality change bias. Non-experts sometimesassume that the CPI does little or no quality adjustment,and that it therefore must overestimate price change inview of the many demonstrable improvements overtime in the quality of goods and services. Rather, forany component index, the issue is whether the direct orindirect method chosen for quality adjustment over-estimates or underestimates the relative quality ofreplacement items in the CPI sample. The resulting biascan be either positive or negative.11.54 Empirical evidence on quality change bias has

been based largely on extrapolation from individual stu-

dies of particular products. These individual studies mayinvolve, for example, comparisons of hedonic regressionindices to the corresponding CPI series or estimates of thevalue of some product improvement that is ignored inCPI calculations. Although the majority of such studieshave suggested upward rather than downward bias, thereliance on fragmentary evidence has led to criticism byobservers who point to evidence of quality declines thathave not been subjected to systematic analysis.

11.55 Especially for services, overall quality trendscan also be a matter of subjective valuation. New tech-nology has led to unambiguous improvements in thequality of many consumer durables and other goods. Bycontrast, in service sectors such as mail delivery, publictransport and medical care, it can be difficult to evaluatechanges in quality. Airline travel, for example, has be-come safer and faster but perhaps less comfortable andreliable in recent decades, and the lack of cross-sectionalvariation in these characteristics makes the use ofhedonic quality adjustment problematic.

11.56 New product bias, like elementary aggregatebias, can be divided conceptually into two components.The first concerns the failure to bring new productsinto the CPI sample with sufficient speed. This can leadto upward bias if those new products later experiencelarge price reductions that are not reflected in the index.The second component is the welfare gain that con-sumers experience when a new product appears. Thismay not be viewed as a bias, however, when the cost ofliving index is not accepted as the CPI’s measurementobjective.

11.57 As discussed in Chapter 8, ‘‘new goods’’ canbe: products that replace predecessor items, for exampleCDs replacing vinyl records and tapes; product varietiesthat widen the range of consumer choice, such asimported beers and ethnic restaurants; or products thatrepresent wholly new categories of consumption, such asmicrowave ovens or mobile telephones.

11.58 Like quality change bias, new product bias hassometimes been estimated primarily by generalizationfrom individual product evidence. A frequent approachhas been to measure the price change for a product orcategory during a period prior to its entry into the CPIsample. Studies by Hausman (1997, 1999) of breakfastcereals and cellular telephones provided quantitativemeasures of the consumer surplus gain from the newproducts, but this complex econometric approach hasnot been applied widely. Some of the Boskin Commis-sion’s estimates of new product bias, notably those forfood, were necessarily based on conjecture.

11.59 Also, like quality change bias, new productbias could be negative if the range of products decreases,if valuable consumer goods disappear from the market,or if the index fails to capture phases of rapid priceincrease for items. Most observers, however, seem toagree on the direction of bias as upward, and that theuncertainty concerns the magnitude.

New outlet bias11.60 Conceptually, new outlet bias is identical to

new product bias. It arises because of the failure to reflect

ERRORS AND BIAS

213

either price changes in new outlets not yet sampled, orthe welfare gain to consumers when the new outletsappear. The explanation for its existence as a separatebias category is twofold. The first reason is historical:new outlet bias was identified by Reinsdorf (1993) as apotentially major explanation for anomalous movementsin the United States CPI. Second, the methods used tosample and compare outlets differ from those used withproducts, and the problems in controlling new outlet biasare somewhat different.

11.61 A failure to maintain a current outlet samplecan introduce bias because the new outlets are distinctivein their pricing or service policy. Reinsdorf (1993), forexample, focused on the growth of discount stores. Itshould be noted, however, that the problem could also begeographical in nature; it is important to employ outletsampling frames that reflect new as well as traditionalshopping locations.

11.62 One way that new products enter the CPIsample is through forced replacement, when exiting orless successful products disappear from shelves. Outletdisappearance is less frequent, and agency proceduresmay not provide for automatic replacement. Moreover,when a new outlet enters the sample there are no stan-dard procedures for comparing data at the new and oldoutlets. Thus, the index will not incorporate any effectsof, for example, lower price or inferior service quality atthe new outlet.

11.63 Reinsdorf (1993) estimated the degree ofnew outlet bias by comparing average prices at outletsentering and disappearing from United States CPIsamples. There has been little or no empirical work,however, on the measurement or consumer valuation ofoutlet quality. As a consequence, there is little evidenceon which to evaluate the accuracy of new outlet biasestimates.

Summary of bias estimates11.64 The 1996 Boskin Commission report gave a

range of estimates for the total upward United StatesCPI bias of 0.8 to 1.6 percentage points, with the pointestimate being 1.1 percentage point. This total reflectsthe straightforward summation of the component biasestimates. As reported by the United States in UnitedStates General Accounting Office (2000), however,changes in CPI methods subsequent to 1996 led the

Boskin Commission members to reduce their estimatesof total bias. Lacking evidence to the contrary, additivityof biases has been assumed in most such studies. Shapiroand Wilcox (1997b) provide probability distributionsand correlations of their component bias estimates,yielding an overall confidence interval for the total bias.Most detailed studies of bias also conclude that the CPIbias is in an upward direction, although there have beennumerous criticisms of that conclusion.

11.65 It is apparent that statistical agencies cannotcompute or publish CPI bias estimates on a regular basis.Many of the same obstacles that prevent the eliminationof bias also stand in the way of estimating bias. Theseinclude the lack of complete data on product-level con-sumer preferences and spending behaviour, and theinability to observe and value all differences in qualityamong items in the marketplace. Without such infor-mation it is impossible to calculate a true cost of livingindex, and similarly impossible to measure the diver-gence between its rate of growth and the growth rate ofthe CPI.

11.66 Statistical agencies have been reluctant toprovide their own estimates of CPI bias. In some cases,they have accepted the existence of substitution bias,recognizing that the use of a Laspeyres formula impliesthat the CPI usually will overstate price change relativeto a cost of living index. Statistical agencies have, how-ever, been reluctant to draw even qualitative conclusionsfrom the fragmentary and speculative evidence onquality change, new products and new outlet bias.

Conclusion11.67 In order to ensure public confidence in a CPI, a

detailed and up-to-date description of the methods anddata sources should be published. The document shouldinclude, among other things, the objectives and scope ofthe index, details of the weights, and last but not least, adiscussion of the accuracy of the index. A description ofthe sources and magnitude of the sampling and non-sampling errors (coverage, non-response rates, etc.) in aCPI provides users with valuable information on thelimitations that might apply to their uses of the index.One example of a handbook of CPI methods is thatpublished by the United States Bureau of Labor Statis-tics (1997), which devotes a section to the varieties andsources of possible error in the index.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

214

12ORGANIZATION AND MANAGEMENT

Introduction12.1 Consumer price indices (CPI) are one of themost

important and widely used of macroeconomic indicators.As well as informing economic policy, they are used forindexation of welfare benefits, pensions, gilts and sec-urities, and also for escalation clauses in private con-tracts. Accuracy and reliability are paramount for astatistic as important as a CPI.12.2 The process of producing a consumer price

index needs to be carefully planned. Individual circum-stances vary to such an extent that this manual cannot betoo prescriptive about timetables or critical path analysisof all the steps involved. Figure 12.1 nevertheless pro-vides an outline of the kind of schedule of activities thatshould result from a detailed examination of the logisticsof the whole periodic operation of data collection and thecomputation of the index.12.3 The guidance given in this chapter, which is

based on the experiences of a number of national sta-tistical institutes, presents a range of organizationaloptions. As individual circumstances can vary, the exam-ples given of good practice may be for some officesaspirational.12.4 In reviewing these options, this chapter covers

the relationships between the field and central office(which kind of work is carried out in central office, theflow of information between each part of the organiz-ation, etc.). The size, frequency, cost or complexity ofthe collection of prices as the basis of the index maymean that in some countries not all these operations andrelationships will be appropriate. The use of both acentral and a local collection, or outsourcing of certainelements of the collection, may not always be effective. Ifthe index is compiled infrequently, from a relativelysmall number of outlets, or concentrates on only specificlocation types, different circumstances will demand dif-ferent solutions.

Local collection12.5 A local price collection involves collectors

visiting individual outlets to collect prices for a varietyof goods and services. This is the predominant methodof price collection in most countries. The range andnumber of outlets visited and the types of goods andservices priced will vary between countries.12.6 Although the precise method of local price

collection will vary, each price collector will usually beresponsible for collection from a certain location or fromcertain types of outlet. Collectors will visit the sameoutlets in each collection period to attempt to price the

same items. Through this type of arrangement, pricecollectors are able to build up effective relationships withretailers and specialist knowledge.

12.7 There are a number of important criteriarelating to the conduct of the collection, whether thenational statistical institute uses its own staff or contractsout the collection (as discussed below). These criteriainclude:

� Collectors should always be smartly dressed andpolite – whoever employs them, they are representingthe national statistical institute.

� They should carry identification to confirm their roleand status.

� They should make themselves known to the retailer orstore manager when they arrive, and before they begincollecting prices.

� They should comply with any request from the shop-keeper whenever possible, for instance, if the store isvery busy and the shopkeeper asks the collector toreturn later in the day.

� The collection should be carried out as quickly aspossible, causing minimal disruption to store business.

12.8 Collectors should also follow rules of commonsense in preparing for a collection. These may includemaking sure that they have: spare pens; the appropriateforms; a clipboard; a local map; spare batteries (if thecollection is computerized); money for shopping centrecar-parks; and wet weather clothing, if appropriate. Insome circumstances a mobile telephone will also be useful.

Contracting out12.9 One of the decisions facing any statistical

agency carrying out a price collection is whether to usein-house staff or to tender the collection to an externalorganization, such as a private market research com-pany, another part of the agency, or another govern-mental department that specializes in surveys.

12.10 The nature of the price collection and thedistribution and profile of statistical staff may help todetermine whether the collection is suitable for con-tracting out. Where price collection is continuous, orinvolves complicated decision-making (such as qualityadjustment), or where prices are collected from a smallnumber of locations, it may be advantageous to keepthe collection in-house. However, if the collection takesplace over a small number of days per month, from alarge number of locations, is relatively straightforwardand involves routine or simple decision-making (perhapsselecting from a prespecified list of codes), then con-tracting out may be considered if there are enough

215

Is the outlet open? Is it permanently shut?Select anotheroutlet accordingto instructions.Report it.

Is the outlet willing tocooperate?

Is refusal because authorizedperson is not available?

Try again laterif theinstructions sayso, otherwisereport the facts.

Is the collectorauthorized toselect areplacementoutlet?

Is the product type tobe priced currentlyavailable to purchase?

Report thenon-availabilityand the reasonfor it.

If the variety is neitherseasonal nor expected tobe available to purchase again next month, then:

Report thefacts to HQ

Is there a large differencefrom the price recorded last month?

Is this because offinal clearancefor damaged or soiled items?

Disregard the price.

Report the non-availability and the reason for it, providing the fulldescription of the unavailable variety ifcentral office doesnot already have it.

Record the price and, if it is non-standard, alsorecord the weight, size or quantity.

State the reason, e.g. sale,special offer, black market price, replacement outlet,replacement item.

Is the collectorauthorized to find a replacement?

Is the replacementthe same kind as its predecessor?

Is the outlet likelyto be permanently shut next month?

Select another variety likely to remainavailable. Record a description of it insufficient detail both to cover qualitydifferences and to enable exactidentification.

Are both pricesavailable for the same month?

Yes

Yes

No

Yes

Yes

No

Is the varietyexpected to becomepermanentlyunavailable nextmonth?

Estimate and report the amountof price difference reflecting the value of the quality differences revealed by the descriptions of the original variety and itsreplacement.

Is the collectorauthorized to makequality judgements?

Yes

No

No

YesYes

Yes

No No

No No

Yes

Yes

No

No

Yes Yes

No

Yes

No

No

Yes

Price next item.

No

Price next item.

Report thefacts to HQ.

Figure 12.1 Price collection procedures

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

216

market research companies with suitable skills existing inthe country.12.11 Contracting out local collection can lead to

reduced costs. Where price collection is carried outelectronically, the responsibility for purchase and main-tenance of data capture devices may be transferred to thecontractor.12.12 Contracting out may also allow statistical staff

to spend more time analysing data rather than collectingthem. By divorcing the role of data collector and datachecker, statistical staff may feel more comfortablequestioning the validity of price data. Accuracy of datacollected can be directly linked to the performance ofthe contractor through performance measures whichdrive incentive payments (and penalties if targets are notachieved).

Central collection12.13 Central shop-collected prices are prices

obtained from the central offices of major retailing chainswith national pricing policies. Branches of these chainsmay be excluded from the local collection if data can becollected more effectively centrally. Data suppliers mayprovide information on paper forms, or by entering pricedata on spreadsheets and forwarding them to thenational statistical institute by email, CD-ROM or onfloppy disks. Mail order catalogues can also be treated ascentral shops: prices are recorded as and when thecatalogues are issued. These prices are then combinedwith those for the same items from the local collection.12.14 Price data for services or fees may be collected

centrally from organizations such as trade associations,national or local governmental departments and so on.Whenever possible, these prices are obtained from onecentral source, although there will have to be contactwith regional or competing companies if there are localvariations. Data may be requested in writing or by tele-phone, or may come automatically because the nationalstatistical institute is on a provider’s mailing list. Provi-ders may send either a full price list or tariff sheet, fromwhich the relevant prices will be extracted by the CPIstaff, or just the prices of those items specified in the datarequest. All price quotations should be confirmed bysome form of written documentation. Frequency ofenquiry varies across the range of items and depends onwhen prices are known or expected to change. The mostcommon frequencies are monthly or quarterly, but thereare also instances of collecting as and when necessary,although in these cases checks must be in place to ensurethat all price data are reported. For instance, this may bethe case where tariffs for gas, electricity and water changeonce a year on a predetermined date.

Quality in the field12.15 Quality is an important aspect of price collec-

tion. A high-quality price collection enables a statisticalagency to have confidence in the index it produces, andensure that observed price changes are genuine and notthe result of collector error. It is important that pro-cedures are developed to ensure that the standard of

collection is maintained at a high level for every collec-tion period. These procedures will form the basis ofcollector training and should be included in any trainingmaterial developed for price collectors. Guidance tocollectors should cover price index principles, organiz-ational issues and validation procedures.

Descriptions12.16 Accurate item descriptions are vitally impor-

tant in ensuring item continuity. Collectors’ descriptionsshould be comprehensive enough to ensure that collec-tors are able to price the same item in each collectionperiod. It is therefore important that contributors recordthe attributes which uniquely define the item they arepricing. For example, for clothes it will be important thatcolour, size and fabric composition are specified toensure that the same item is priced each month. For freshfruit and vegetables, useful attributes to record may becountry of origin, class and variety.

12.17 Accurate item descriptions will assist the pricecollector and head office in choosing a replacement foran item that has been withdrawn and will also help toidentify changes in quality. Head office staff should beencouraged to spend some time, each collection period,going through collectors’ descriptions to ensure that thecorrect items are being priced. Collectors should also beencouraged to review their descriptions to ensure thatthey contain all of the relevant information. It may beuseful to ask collectors occasionally to switch collectionswith another collector so that they understand theimportance of good descriptions.

Continuity12.18 Continuity is one of the most important prin-

ciples of price collection. Because a price index measuresprice changes, it is vital that the same item is priced everymonth in order to establish a true picture of pricechanges. So if, for example, a jar of a supermarket’s ownbrand of strawberry jam has been selected, that partic-ular brand and flavour should continue to be collected. Ifit is out of stock in the collection period, another brandand flavour should not be used. If, however, in sub-sequent collection periods, the selected jam continues tobe out of stock, but another flavour of the same brandand price is available, then this item should be chosen asa comparable item and the item description suitablyamended. If no comparable item exists, then a new itemmust be chosen, and the description amended. Thus anew price chain will begin. It is not possible to be pre-scriptive because the concept of equivalence will varybetween different countries; but for practical purposes itis important that a detailed description of the items beingpriced is kept.

12.19 As continuity is so important in the compila-tion of an accurate price index, collectors should beencouraged to check with the retailer that an item is outof stock before replacing it. Some guidelines may bedrawn up by the head office of the national statisticalinstitute to cover different items. Food items, for exam-ple, will usually come back into stock in the follow-ing collection period, and so should not be replaced

ORGANIZATION AND MANAGEMENT

217

immediately, whereas fashion clothing will rarely comeback into stock once a ‘‘season’’ has ended or the stockhas been exhausted, and so should be replaced imme-diately in the collection.

12.20 Collectors should also be encouraged to plantheir route for price collection to take account of outletopening and closing times and any special retailerrequests. Collectors may find it useful to compile a routemap, listing the order in which outlets should be visited.This is particularly useful when the collection in a loca-tion has to be undertaken by a different collector, forexample to cover for sick leave. Collectors should beencouraged to try to collect prices at similar times withineach collection period. This is particularly importantwhen pricing volatile items, such as petrol and oil, wherethere can be sharp fluctuations.

Data entry queries12.21 Once the price data are correct and complete, a

series of validation checks may be run. In deciding onwhat checks should be carried out, account should betaken of the validation checks carried out in the field. Forexample, the use of hand-held computers will increasethe potential for validation at time of price collection andreduce the need for detailed scrutiny at head office. Inaddition, it would clearly not be productive or cost-effective to repeat tests already carried out.

12.22 The range of tests carried out may include:

� Price change: The price entered is compared with theprice for the same product in the same shop in theprevious month, and triggers a query where the pricedifference is outside preset percentage limits. Theselimits vary, depending on the item or group of items,and may be determined by looking at historical evi-dence of price variation. If there is no valid price forthe previous month, for example because the item wasout of stock, the check can be made against the pricetwo months or three months ago.

� Maximum/minimum prices: A query is raised if theprice entered exceeds a maximum or is below aminimum price for the item of which the particularproduct is representative. The range may be derivedfrom the validated maximum and minimum valuesobserved for that item in the previous month expan-ded by a standard scaling factor. This factor may varybetween items, again based on previous experience.

12.23 If a hand-held computer is used, both of thesetests can be implemented easily to take place at the timeof collection, otherwise they will need to be conducted inthe head office as soon as possible after collection andbefore prices are processed on the main system. A failurein either test should not result in the collector beingunable to price the item, but should prompt the collectorto check and confirm the entry, and prompt for anexplanatory comment.

12.24 Queries raised may be either dealt with at headoffice or sent to the price collector for resolution. Forexample, scrutiny of a form might show that a big pricedifference has arisen because the item priced was a newproduct replacing another that has been discontinued. Inthis case there may be no need to raise a query with the

price collector unless there is evidence to suggest that tolabel the item ‘‘new product’’ is incorrect.

12.25 Where an error is discovered too late in theprocess to resolve, head office will need to reject it andexclude that item from that month’s index. Care shouldbe taken that the item is also excluded from the basemonth so that the basket is kept constant.

Feedback12.26 Collectors should be encouraged to give feed-

back to head office on their experiences of price collect-ing. Collectors are a valuable source of information andoften give good early feedback on changes in the market-place. Collectors can often warn of size or productchanges before the head office is able to derive thisinformation from other sources such as trade magazines.Collectors’ feedback can be used to support observedprice movements and to provide supplementary briefingmaterial. It can also form the basis of a newsletter forcollectors.

Quality checks in local collection:The role of auditors

12.27 The whole periodic routine of collecting pricesin the field needs to be carefully planned and monitored,with arrangements in place to reflect local conditions.Circumstances vary, so it is not appropriate to be tooprescriptive. It is, however, important to ensure thatprice collectors send in information when it is due. If theydo not do so, it is necessary to find out the reason and totake appropriate action. It is also important to checkthat the information sent in is accurate and complete.

12.28 One way of monitoring the work being carriedout by price collectors is to employ auditors to occa-sionally accompany collectors during the field collection,or to carry out a retrospective check on the data thathave been collected.

Monitoring12.29 If an auditor intends to accompany a price

collector, he or she will need to inform the collector inadvance in order to arrange meeting details. In general,the auditor will not accompany the whole price collectionbut will spend a few hours observing the price collectionin a specific location. For example, it may be desirable toobserve the collection of certain items or in particularoutlets where collection might be problematical, and theprice collector may need to rearrange his or her routeaccordingly.

12.30 Prior to monitoring, the auditor will need tocarry out preparation work – a pre-monitoring check.Sucha checkmight involve looking at descriptions, prices,price history and indicator codes of the items collected inthe chosen location. This type of check will enable theauditor to have a good idea of the standard of collectionprior to going into the field and may suggest on whichareas of the collection the auditor should concentrate hisor her efforts.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

218

12.31 An auditor’s main duty is to ensure that theprice collector is following the procedures and instruc-tions laid down for price collection and that the collec-tion is being performed competently. While the auditormay not have the role of a trainer, the opportunity maybe taken to give some coaching when errors are noted.There should also be the opportunity for the collector toask the auditor relevant questions during the monitoringexercise.12.32 Auditors may undertake other duties at a loca-

tion besides accompanying the collector. For instance,they may enumerate outlets or carry out an item review.Following a monitoring visit, the auditor should compilea report detailing the observations made while accom-panying the collector. This report should include asummary of findings, a list of points for action, and arecommended course of action. Auditors may advise thata collector receive extra training on certain aspects ofthe price collection; head office (or the contractor, if thecollection has been outsourced) should act on this. Thisreport will then be used as a starting point on the audi-tor’s next visit. In other instances, general problems mayarise where solutions need to be disseminated to all pricecollectors, perhaps by issuing revised instructions orthrough a newsletter.

Backchecking12.33 Another approach to monitoring the standard

of price collection is to carry out a backcheck, a retro-spective check of a proportion of the prices recordedduring the collection.12.34 Backchecks can be used to:

– assess the standard of competence of individual pricecollectors;

– audit the overall standard of price collection;

– identify general training needs or the specific needs ofan individual;

– highlight any key issues including, for example, pro-blems with documentation or instructions issued byhead office;

– identify areas where collection is problematical; forexample all collectors may have problems in certaintypes of outlets, prompting the need for more detailedhead office instructions.

12.35 Backchecking should be done by an expertindependent of the process (preferably employed by thenational statistical institute). Backchecking is carried outby visiting the selected outlet and re-collecting theprices and other relevant information, such as attributeor description codes. This activity should be carried outclose to the original collection period to avoid problemsof price changes occurring in the interim. It is importantthat backcheckers seek permission from the shopkeeperbeforehand and follow the general criteria of conduct forlocal collection, as described in paragraphs 12.5 to 12.12.12.36 For a backcheck to be a useful exercise it is

important that performance criteria are determined towhich all backcheck results can be compared. Thesecriteria should set, for example, the acceptable number ofprice errors per number of items checked. Well-defined

criteria will enable easy identification of a poorly per-forming collector or location following a backcheck.

12.37 A backcheck may include a range of tests toidentify the following:

� price difference – if the price is different, the auditorshould check with shopkeepers to see if there has beena price change since the original collection took place;

� insufficient item description – each item should beuniquely defined so that another collector can step into cover the absence of the usual collector, for examplein the event of illness;

� wrong item priced – such as incorrect size beingchosen;

� items wrongly recorded as missing or temporarily outof stock.

12.38 A report should be sent to head office forscrutiny once the backcheck has been completed. Headoffice will then need to take appropriate action, whichmay include, for example, retraining or sending outsupplementary instructions.

Other auditor functions12.39 The range of tasks that an auditor carries out

will vary from one statistical agency to another. Mon-itoring the standard of price collection will always be themain focus of the auditor. There are a number of otherareas, however, in which auditors can be called upon tocontribute.

12.40 Auditors may be required to help with thesampling of locations and items. Auditors can check thatproposed collection locations contain an adequate rangeof shops. They can also advise on economic conditionsin these locations and on any dangerous areas. Auditorscan carry out commodity work. For example, if a par-ticular item seems to be causing difficulty for price col-lectors, auditors can speak to collectors and retailerswith a view to determining reasons for these difficulties.Auditors can also advise on changes to basket compo-sition. They can ensure that products suggested by headoffice are available across the country, and can suggestitem descriptions and weight bands. Furthermore,auditors can provide reports on collection in existinglocations. For example, head office may raise a queryabout a particular outlet in a particular location; audi-tors can visit this outlet to find the answer to the questionor to persuade a retailer to continue with the survey.

Quality checks in head office12.41 Four kinds of regular checking are necessary

in head office:

� to ensure that the price collectors’ reports are sent inwhen they are due. If this is not done, it is necessary tofind out the reason and to take appropriate action toobtain the reports;

� to confirm that the reports contain what they aresupposed to contain, i.e. that fields which must befilled in have not been left blank, that numeric fieldscontain numbers and that non-numeric fields do not;

ORGANIZATION AND MANAGEMENT

219

� to review and edit each return. Substitutions may haveto be made centrally or those made by the collectorsmay have to be approved. Unusual (or simply large)price changes may need to be queried. Items priced inmultiple units or varying weights may have to beconverted to price per standard unit. Missing pricesmust be dealt with according to standard rules relat-ing to the cause;

� errors introduced when keying the numbers into thecomputer or transcribing them onto worksheets mustbe found and corrected, and preferably avoided in thefirst place by eliminating the need to transcribe.

12.42 It should be noted that the way the data areorganized in worksheets or in the computer may differfrom the way they are organized on receipt, since theywill arrive at the central office organized by collector,outlet and item. Their origin should, however, berecorded so that reference to it can be made shouldprocessing disclose any problems with the data. Fur-thermore, even if codes provided to the collectors to listitems and to describe or qualify the prices are usedunchanged in the processing, other codes may have to beused for information which comes in from the collectorsin non-coded form.

12.43 How the checking is organized will vary fromcountry to country. In some cases, local or regionalsupervisors will do some of it; in other cases, it will bemore appropriate for it all to be done centrally. Some ofthese tasks can be done by computer, others manually.Therefore, no general suggestion can be made about thesequence of the work or about its division into differentparts.

12.44 Procedures should be in place to check that alldocuments, messages or files are returned from the fieldso that price collectors can be contacted about missingreturns. Initial checks should then be carried out toensure that data are complete and correct. For instance,checks should be run to ensure that unexpected duplicateprices (i.e. for the same item, in the same shops, in thesame location) are not taken on, and that the location,outlet and item identifier codes which accompany eachprice exist and are valid. If any prices fail these checks, aquery should be raised with the price collector for clar-ification. Since some of the checking may require refer-ence back to the price collectors (or to their supervisorsor respondents when direct mail questionnaires areused), the timetable for producing the index must allowfor this communication to take place.

12.45 Following the checks that the price data arecorrect and complete, a series of validation checks maybe run. In deciding on what checks should be done,account should be taken of the validation checks carriedout in the field. The use of hand-held computers willincrease the potential for validation at the time of pricecollection and reduce the need for detailed scrutiny athead office. It would clearly not be productive or cost-effective to repeat all the tests already carried out locally,except as a secondary audit or random check that thosechecks have been completed.

12.46 The range of checks that might be carried outis covered in paragraphs 12.21 to 12.25. In addition, it is

possible for the head office to use the price data receivedthat month to identify outliers.

Reports12.47 Reports should routinely be generated for

most representative items, to help the analyst pick outparticular prices for which the level or change stands outas different from that reported for similar varieties else-where, or simply where the change lies outside certainspecified limits. Thus, a computer printout can list allprices which either fall well outside the range of pricesobtained last time for that representative item, or forwhich the percentage change from last time for the sameitem in the same outlet falls outside a specified range.The limits used will vary from item to item and can beamended in the light of experience. The analyst canthen work through the printout, first ascertaining whe-ther there has been a keying-in error, and then examin-ing whether any explanation furnished by the collectoradequately explains the divergent price behaviour orwhether a query should be sent back to the supervisoror collector. The timetable should allow for this, andanomalous observations should be discarded where anacceptable explanation or correction cannot be obtainedin time.

12.48 Other reports may be produced regularly onthe basis of reports for several periods (e.g. severalmonths) to detect accumulated patterns, thus enablingbroader problems to be detected. For example:

� One collector’s reports might show many more ‘‘outletclosed’’ remarks than those of other collectors, per-haps indicating either a motivational or training needon the part of that collector, or a change in retail tradepatterns in a particular area.

� Variety substitution for a particular representativeitem might become more numerous than hitherto,suggesting a possible need for revision of the specifi-cation or the choice of another representative item.

� Where tight specifications list a number of brands andmodels of which one is to be chosen, but a largenumber of prices are for items not specified in theoriginal list, this will suggest that the specified brandsand models are no longer appropriate and a review ofthe list is required.

� The dispersion of price changes for a particularrepresentative item might be much larger than it usedto be, raising the question of whether it has beenappropriately specified.

12.49 Routine computer-generated reports shouldenable those in charge of the index to detect the exis-tence of all such problems. Two types of reports areparticularly useful: index dispersion reports, and pricequote reports.

12.50 Index dispersion report. This is a list of itemsindicating the current index for each item, the number ofvalid quotes for each item, and the number of pricerelatives (the ratio of current price to previous validprice) in each of a series of pre-selected ranges (forexample, less than 40, 40–49, . . . 190–199, greater than199). The index dispersion reports can be used to identify

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

220

quotes with price relatives that fall outside the range ofthe main bulk of quotes. These quotes can be identifiedfrom quote reports for the item, then investigated andappropriate action taken if necessary.12.51 Price quote report. This consists of a range of

information on an item that the index dispersion reporthas highlighted as warranting further investigation.Information listed may include current price, recentprevious prices and base price, together with locationsand types of shop. The report can be used to identify thequotes that require further investigation and also toinvestigate rejected prices.

Algorithms12.52 Algorithms can be created which may be used

to identify and invalidate price movements that differsignificantly from the norm for an item. For some sea-sonal items for which price movements are erratic, itmay be more appropriate to construct an algorithm tolook at price level rather than price change.12.53 An example is the Tukey algorithm. This oper-

ates as follows:

� The ratio of current price to previous valid price (theprice relative) is calculated for each price. (In the caseof items tested by price level rather than price change,this stage is omitted.)

� For each item, the set of all such ratios is sorted intoascending order, and ratios of 1 (unchanged prices)are excluded. (In the case of items tested by price levelrather than price change, the prices themselves aresorted.)

� The top and bottom 5 per cent of the list are removed(this 5 per cent is parameter 1).

� The ‘‘midmean’’ is the mean of what is left.

� The upper and lower ‘‘semi-midmeans’’ are the mid-means of all observations above or below the median.

� The upper (lower) Tukey limit is the midmean plus(minus) 2.5 times the difference between the midmeanand the upper (lower) semi-midmean. This figure of2.5 represents parameters 2 and 3. The upper andlower values can be set independently if desired butare currently set to be equal.

� If the upper limit is negative, it is set to zero. (If pricelevels are used, the lower limit is set to zero.)

� Price relatives, or price levels, outside the Tukey limitsare flagged as unacceptable and requiring amendmentor further investigation.

12.54 The Tukey algorithm has a number of advan-tageous characteristics. In particular, it produces intui-tively reasonable results; is consistent from month tomonth; is robust in the presence of outliers (in otherwords, adding in one or two rogue observations does notaffect very much the limits set by the algorithm); and isrobust as data volume changes (i.e. limits calculated froma subset of the data do not vary much from those cal-culated on the full data set).12.55 Whilst algorithms can be an efficient way of

highlighting problematical data, a word of cautionshould be expressed about using them. Analysts will

want to assure themselves that their use does not result insystematic bias in the index. This is a matter that mayalso need to be taken into account in any editing rou-tines, although it is less likely to be problematical in thecontext of manual editing.

Producing and publishingthe index

12.56 In regard to producing and publishing theindex, there are a number of organizational models thatcould be adopted for effective working. Considerationsto be taken into account in deciding on the appropriateorganizational structure include:

– the need for clarity of reporting lines;

– the need for a clear division of responsibilities;

– centralized or decentralized management of fieldwork(see above discussion on local collection, and theoutsourcing of fieldwork, paragraphs 12.6 to 12.14);

– production management versus technical develop-ment;

– compatibility with corporate structures in the nationalstatistical institute, for example, in relation to qualitymanagement, methodological research, and dissemi-nation.

12.57 In some cases, for instance where little in-house expertise in fieldwork practice exists, it may beadvantageous for fieldwork to be conducted by a dif-ferent organization in either the public or private sector.In these circumstances it is important that an effectivecontractual relationship exists with regard to the data.There should also be agreed delivery targets and per-formance measures to cover such things as data deliverytimetables, response rates and levels of accuracy. Con-sideration should also be given to the independentauditing of the contractor’s work on a sample basis.

Monthly compilation12.58 The system used for the regular computation

of the index must be sufficiently flexible to allow forchanges in the kind of data obtained. For example, localprice collection for purposively sampled products fromthe branches of a large supermarket chain may be super-seded by centrally collected prices for a statistical sampledrawn from complete sales data made available by thehead office of the chain. In these circumstances, amodular approach may be seen as an advantage.

12.59 Analytical computations provide comparisonsbetween the published index, or one or more sub-indices, and what they would have been using differentmethods or data. They help to explain why the index hasmoved as it has and they allow methodological experi-mentation. The following examples of such investiga-tions serve to make clear some of the computationalcapabilities and data that are required:

– alternative aggregations of sub-indices;

– the effects of different weights; the effects of introdu-cing newly significant product categories; and price-updating of weights;

ORGANIZATION AND MANAGEMENT

221

– number and duration of missing observations; how adifferent method of estimating them would affect theindex;

– comparison of indices computed with various sub-samples of the data as a means of estimating variance;variances of price ratios;

– computation of a standard reference index (one withno explicit quality adjustments) so that an implicitquality index is obtained;

– numbers of sampled products; rates of forced replace-ments; and lengths of time products remain in thesample;

– frequency distributions of quality adjustments.

12.60 To examine such matters, the database mustcontain not only prices but detailed descriptions ofproduct replacements, explanatory remarks attached toobserved prices, and so on. Generally, it will be foundthat historical databases will be too large to be stored liveon the system and therefore will need to be archived.Detailed documentation relating to the archived materialwill need to be kept to guard against loss of vital infor-mation caused by changes in computing staff or com-puters. Consideration should also be given to appointinga data custodian with responsibility for all archivedrecords.

Spreadsheets12.61 Spreadsheets may be used for compiling sub-

indices that require special procedures, or where dataare collected centrally, or on an uncertain timetableor to a different timetable from that of other datacollection, but effective control procedures need to beput in place. Examples of types of prices for which theuse of separate spreadsheets may be useful include:air fares, hotel accommodation, newspapers and carrental. Such use of a spreadsheet has the advantageof additional flexibility and scope for combiningresponsibility for data collection, data input and com-putation. The compiler’s specialized knowledge aboutthe markets or outlets where these prices may beobserved, combined with analytical tools applied tothe spreadsheet, will help the compiler to detect anyirregularities in the data, facilitate investigation ofwhether these reflect reporting or input errors, andallow for rapid rectification. The ability to jump be-tween numerical data entry and a chart displaying, forexample, current-month and previous-month entries,helps the rapid and simple detection of anomalies. Thesame person can then follow this up with the datasupplier.

12.62 As time passes, the resolution of problems thathave arisen and adaptation to new circumstances willresult in changes in the spreadsheet. Unless qualitymanagement controls are put in place, there is a dangerthat the spreadsheet will be understandable only by theperson responsible and that it will not be properlydocumented. If so, two unfortunate consequences canarise:

� If that person is absent, retires or moves to anotherjob, his or her successor will find it very difficult

to maintain the continuity and quality of the sub-index.

� New procedures introduced to deal with new circum-stances may be inconsistent with procedures used forother sub-indices for which other people are respon-sible.

12.63 Good documentation and good communica-tion with colleagues will diminish these risks. At a mini-mum, there should be an insistence that the spreadsheetsand changes in them are made understandable by theprovision of adequately explicative row and columnheadings or of notes attached to headings. Furthermore,changes in procedures or formulae, rebasing and theapplication of new weights should always be introducedby moving computation over to a new sheet within theworkbook, not by modifying the old sheet. The new sheetand the old sheet will then exist side by side so that theycan be compared.

12.64 Inadvertent changes may be prevented byusing passwords to cells containing formulae and bylocking cells containing input data once editing is com-pleted. Passwords should be known only to a limitednumber of people with authority to edit the spreadsheets.Regular back-up by copying the whole workbook toanother disk is also essential.

Introducing changes12.65 Various checks should be carried out when

introducing changes. These may include a comparison ofthe old and new basis using data from parallel running ofcollections (e.g. when handing over to a new collectioncontractor) or re-estimating backwards – for example,when new base prices are being imputed for a completerange of goods or services. Any anomalies can then beinvestigated further.

Disaster recovery12.66 A consumer price index will arguably be

the most important and highest-profile statistic that anational statistical institute produces and can affect thewidest range of users. There is often a legal obligation forthe CPI to be published within a short time period afterthe end of the month to which the data refer. For exam-ple, in the European Union, there is a legal requirementto publish within 30 days of the reference period theHarmonized Index of Consumer Prices (HICP), whichuses the CPI data sets from member States (although theEurostat timetable is for publication two weeks earlierthan this). Any delay in publication can have a sig-nificant impact on subsequentmonths, threatening futurepublications. Significant delays could take months tocatch up, in order to return to the existing tight pub-lication timetables. It is critical, therefore, that nationalstatistical institutes develop a robust and tested disasterrecovery plan, however unlikely the need to implement it.

12.67 There are a number of possible causes of dis-aster:

– failure of an external contractor to fulfil obligations tosupply information;

– failure of the computer system;

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

222

– major natural disaster or other event (e.g. terroristactivity) affecting the operations centres or head officeof the national statistical institute.

12.68 Where the collection is contracted out, one ofthe most important requirements of a disaster recoveryplan is to recruit an alternative permanent service sup-plier as soon as possible. It is probable that, on termi-nation of a contract with an external provider, thenational statistical institute could arrange to have theservices supplied by a third party, but only on a tem-porary basis prior to re-letting the contract through acompetitive tendering process.12.69 Additional money may need to be obtained for

implementing a computing disaster recovery plan. Con-sideration needs to be given as to whether outsourcingthe disaster recovery plan to a company specializing inthe provision of back-up support or maintaining an in-house capability is the best option. This will, in part,depend on the number of sites and locations at which thenational statistical institute operates. If the organizationhas a number of sites, some distance apart but linked bymodern communications infrastructure, then there is lesslikelihood that they will all be affected by a natural dis-aster.12.70 The managers of disaster recovery plans will

also need to consider:

– full specification of accommodation and associatedrequirements (e.g. personal computers, telephones)associated with each site;

– allocation of specific officers to specific duties for thedisaster recovery period and identification of eachindividual’s training needs;

– investigation of practicalities and associated expensesfor matters such as access to shared drives and sys-tems, including communication and the quality man-agement systems, from other sites;

– confirmation of costs, arrangement of site visits andliaison with procurement units in negotiating con-tracts.

Quality management andquality management systems12.71 Statistical offices are faced with the continuous

challenge of providing a wide range of outputs andservices to meet user, i.e. customer, needs. Thus a keyelement of quality is customer focus and the effectivedissemination of relevant, accurate and timely statistics.In addition, it can be argued that quality managementshould include effective customer education on the use ofsuch statistics. In these terms, success can be measuredby the achievement of a high level of satisfaction amongstwell-informed users.12.72 For the quality management of a CPI, it can

be argued that the priority area is quality control of theproduction process itself. For most national statisticalinstitutes, quality control of production will be an areawhich represents a high risk, given the complexity ofthe process and the financial implications of an error inthe index.

12.73 If the principles of organizing and managingthe collection of data, and subsequent processing ofinformation to produce a consumer price index, are to beadopted, then it is vital that a system is in place to ensurethat the data obtained, the processes involved in achiev-ing the specified outputs, and the formulation of thepolicies and strategies that drive them are managed inan effective, consistent manner. The processes should,wherever possible, be open to verification; and mech-anisms should be put in place to ensure that outputs meetrequirements – in other words, customer satisfaction.Taken together, these elements form the basis of aquality management system.

12.74 There are varying perceptions about themeaning of quality but an important common thread isthe requirement to react to and serve users of the CPIand to ensure continuous improvement in that service.Thus the implementation of an effective quality man-agement system requires a high level of understanding ofcustomer needs and the translation of this into a coherentstatistical and quality framework. Such a framework isalso necessary for putting together criteria for judgingsuccess. User needs can be canvassed either formallythrough negotiation of contractual obligations whichmay or may not be legally binding, or less formallythrough talking to customers on a one-to-one basis orthrough customer surveys.

12.75 In many countries, issues relating to the gov-ernance of the national statistical institute are set downin a ‘‘framework’’ or similar document. This defines thefunctions and responsibilities of the national statisticalinstitute, and generally guides and directs the work ofthe office. For instance, an objective stated in the frame-work document ‘‘to improve the quality and relevanceof service to customers – both in government and thewider user community’’ provides a powerful statementfor determining workplans.

12.76 This recognition of the importance of qualitycan be further endorsed by a published vision of thenational statistical institute as a key supplier of author-itative, timely and high-quality information. Such avision can be encapsulated by publishing objectives in anannual business plan. These objectives can includeimproving quality and relevance, thereby increasingpublic confidence in the integrity and validity ofoutputs.

12.77 Performance can be measured against a com-bination of a number of factors, including accuracy,timeliness, efficiency and relevance. There are a numberof practical examples and case studies of quality sys-tems, illustrating how different models may be applied.

Quality management systems12.78 Various standards of best-practice standards

can be exploited to help organizations to improve qualitymanagement. Some of these standards have the addedadvantage of being internationally recognized.

12.79 Total quality management. Total quality man-agement (TQM) is more closely identified with a man-agement philosophy rather than a highly specified andstructured system. The characteristics associated with

ORGANIZATION AND MANAGEMENT

223

TQM and an effective culture of quality in an organiz-ation include:

– clearly defined organizational goals;

– strong customer focus;

– strategic quality planning;

– process orientation;

– employee empowerment;

– information sharing;

– continuous quality improvement.

12.80 Benchmarking. Benchmarking is a process ofcomparing with others, and learning from them aboutwhat you do and how well you do it, with the aim ofbringing about improvements.

12.81 There are already a number of benchmarkingpartnerships operating within national statistical insti-tutes, some specifically considering the CPI. The Aus-tralian Bureau of Statistics has been particularly activein this area, and undertook an exercise in 1998–2000 inpartnership with the United Kingdom. Benchmarkingprojects have also been undertaken in New Zealand, theScandinavian countries and the United States.

12.82 Areas that can be considered when bench-marking a CPI collection may include:

– timelines, accuracy and coverage of collection;

– benefits of index methodologies for various items, e.g.geometric mean as against average of relatives;

– frequency of collection and publication;

– cost of collection per unit of commodity, etc.

12.83 European Foundation for Quality ManagementExcellence Model. The Excellence Model (1994) con-structed by the European Foundation for QualityManagement (EFQM) is a diagnostic tool for self-assessment. The model is widely used by governmentalorganizations across Europe to improve quality andperformance. It may be described as a tool that drives thephilosophy of TQM.

12.84 The EFQM Excellence Model focuses ongeneral business areas and assesses performance againsttwo sets of criteria – the first consists of five criteriacovering what the business area does (the enablers: lea-dership; people; policy and strategy; partnership andresources; and process), and the second consists of fourcriteria on what the business area achieves (the results:people results; customer results; society results; and keyperformance results). Evidence based on feedback fromfocus groups, questionnaires and personal interviews isused to score performance, and a resulting action planfor improvement is introduced which is then included inthe business plan.

12.85 Underlying the EFQM Excellence Model isthe realization that business excellence – measuredthrough customer satisfaction – is achieved througheffective leadership which drives policy and strategy,allocates resources compatible with that policy, andman-ages employees in such a way as to enable them tomanage the processes.

12.86 In the case of national statistical institutes,where some procedures are governed by statute or reg-ulation, the use of the EFQM Excellence Model enables

continuous improvement to be taken forward across arange of processes and functions. To work effectively, itneeds the commitment of senior managers, who must beresponsible for leading any self-assessment. However,unlike ISO 9000, where assessment is carried out byqualified auditors often from outside the work area (seebelow), the EFQM Excellence Model relies on the inputfrom all staff.

12.87 ISO 9000. The International Standard ISO9000 is an international quality standard for manage-ment systems (ISO, 1994). A quality system is a common-sense, well-documented business management systemthat is applicable to all business sectors. It helps to ensureconsistency and improvement of working practices,including the products and services produced.

12.88 The ISO standards were fully revised as ISO9001 in November 2000 to match current philosophiesof quality management and views regarding the struc-tures that need to be in place to ensure that continuousimprovement is maintained (ISO, 2000).

12.89 The revised standards give users the oppor-tunity to add value to their activities and to improvetheir performance continually by focusing on the majorprocesses within the organization. They will result in acloser alignment of the quality management system withthe needs of the organization and reflect the waythe organization runs its business activities. By meetingthe ISO 9000 standard, an organization will comemore into line with TQM and the EFQM ExcellenceModel.

Scope for greater use of qualitymanagement techniques

12.90 Both ISO 9000 and the EFQM ExcellenceModel have received a great deal of internationalrecognition over recent years. At the same time, the useof benchmarking networks has also grown in promi-nence. It is therefore pertinent to ask whether morecoordinated use should be made of these and otherquality management techniques at a strategic level infields of statistics where the focus is on internationalcomparability. This is particularly so with statistics thatare compiled for treaty purposes, for example by memberStates of the European Union following detailed meth-odological guidelines laid down in law.

12.91 The arguments are fivefold:

� It is paramount that such important non-optionalstatistics whose production and uses are enshrined inlegislation have the full trust of users.

� The quality of international comparisons is dependenton the weakest link, thus good-quality statistics fromone country may be of little value if not matchedby statistics of equally good quality from othercountries.

� There is a potential for misleading analysis and con-clusions arising from differences in the application ofstandard methodology.

� Empowerment in ensuring the establishment of ade-quate control processes is reduced when production isdelegated to member States.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

224

� There is limited scope for centralized validation andquality management when production is decentralized.

Performance management,development and training12.92 An effective performance management system

for individuals is just as important as applying such asystem to management structure. Performance man-agement can be seen as a continuous process designedto improve work outputs by focusing on what peopleactually achieve rather than the amount of effort theyput into the work. It should provide the link betweenthe objectives of the individuals, those of their teamand those of the wider organization, so that workplansare coherent across the organization, and everybodyknows what they are doing and why they are doingit. The performance management system should pro-vide clear objectives for monitoring and evaluation, toenable feedback on performance and also to assistwith the identification of the development needs ofindividuals. Performance management should be con-tinuous.

Training requirements12.93 Effective training will help motivate staff and

equip them to deliver a good-quality CPI. At its simplest,training will give a background understanding of thenature and uses of the index and how it is compiled.Training and development takes many different formsand may include:

– tutoring by the line manager or supervisor;

– attending an induction course or reading a manual;

– accompanying an experienced price collector.

12.94 A written training plan is useful in identifyingtraining and development needs in relation to the or-ganization’s goals and targets. It can also be used toidentify the resources required to deliver the trainingto meet these needs, and to evaluate whether the train-ing has been delivered effectively and objectives havebeen met.

Specific training for compilersand collectors12.95 Further training will be required for specific

skills, depending on the roles of the individuals and theirjobs. Training should continue beyond the inductionstage to cover changed procedures, and retraining whereperformance is unsatisfactory.12.96 Price collectors will need to be trained speci-

fically in field procedures, including relations withshopkeepers, the selection and definition of a valid price,special rules for certain individual items (including sea-sonal items), how to complete forms and, whereappropriate, how to use hand-held computers. Compi-lers of the index will need to be trained specifically onvalidation procedures and consistency checking, thecalculation of centrally collected indices, weighting

procedures and how to aggregate prices, as well as onthe treatment of seasonal items and special proceduresrelating to some sections (e.g. housing). It may also bebeneficial to provide training in local or national tradingor statistical regulations, economics, and commodityinformation.

12.97 Significant benefits may result from the inter-action between price collectors and index compilers.Benefits will also be gained from liaison between thenational statistical institute and commodity experts fromindustry. Such experts can advise on issues such as howto identify quality features on particular items, forexample electrical goods, personal computers, or cloth-ing and footwear.

12.98 It may be beneficial if statisticians from head-quarters are personally responsible for supervising pricecollection in the area where the head office is situated, sothat they can have first-hand experience of the problemsinvolved. This will put them in a position to provideassistance where difficulties arise. Equally, it is a goodidea to arrange for regular visits to headquarters bygroups of collectors and their supervisors. It is goodfor morale. Price collectors will, arguably, do a betterjob if they feel that they belong to a team, if they can seethat their work is appreciated and if they feel that theirproblems are understood. Visits to headquarters willhelp convey that the accuracy and conscientiousness oftheir contribution is recognized as being crucial to thequality of the index. Visits to head office by price col-lectors also will help the statisticians to keep in touchwith conditions in the field and, for example, to obtainmore information about new goods and aspects ofquality change.

12.99 Similarly, compilers of the index may wish tovisit the field occasionally and participate in or simplyobserve the price collection. This will provide them witha better appreciation of the practical problems associatedwith price collection and a better feel for data (and inconsequence for the quality of the index), together withthe skills required to help with price collection in theevent of an emergency.

Documentation12.100 A manual and other documents such as desk

instructions may serve for initial training. Later on suchdocuments should enable the collectors and compilers toremind themselves of all the relevant rules and procedures.The documentation should be well organized and wellindexed so that answers to problems can quickly be found.

12.101 Documentation should be checked by allconcerned and updated regularly. The pile of pieces ofpaper containing amendments should never grow large,but should be replaced by a new consolidated version.One way of achieving this is to have a loose-leafmanual so that individual pages can be replaced when-ever necessary. Another option is to keep an elec-tronic version that can be updated by nominatedindividuals. It is important that the updating ofdocumentation is done in a systematic and controlledway. A variety of software is available to help thestatistician to do this.

ORGANIZATION AND MANAGEMENT

225

12.102 The benefits of using standard electronicsoftware for documentation are threefold:

– more efficient production of documentation, becausethe software helps with the initial compilation of in-formation and reduces the need to print and circulatepaper copies;

– better-informed staff, because they have immediateelectronic access to the latest documentation, includ-ing desk instructions, with a search facility by subjectand author;

– better quality control, because authors can readilyamend and date-stamp updates and because access tonon-authors is restricted to ‘‘read only’’.

Reviews12.103 Training may be seen as an essential part of

continuous quality improvement. Staff may be invitedto operational reviews where all team members havethe opportunity to raise concerns and, where appropriate,tackle specific issues through individual or group training.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

226

13PUBLICATION, DISSEMINATION ANDUSER RELATIONS

Introduction13.1 The consumer price index (CPI) is one of the

most important statistical series. Where statistics arecategorized according to their potential impact, the CPIand its variants are always in the first rank. It followstherefore that it must be published, and otherwise dis-seminated, according to the policies, codes of practiceand standards set for such data.13.2 The CPI should therefore be:

– released as soon as possible;

– made available to all users at the same time;

– released according to pre-announced timetables;

– released separately from ministerial comment;

– made available in convenient form for users;

– accompanied by methodological explanation;

– backed up by professional statisticians and econo-mists who can answer questions and provide furtherinformation.

13.3 Above all, the CPI should meet the Funda-mental Principles of Official Statistics (United Nations,1994). These principles are published in several lan-guages on the websites of the United Nations and theUnited Nations Economic Commission for Europe(UNECE). They refer to dissemination and to all aspectsof statistical work. These and other standards are dis-cussed in this chapter.

Time series presentation oflevel and change13.4 It is common, though not universal, to give

greatest prominence to indices that show changes inaggregate prices between the month for which the mostup-to-date data are available and the same month oneyear earlier. It is also usual to compare this annualchange with the annual change shown one month pre-viously. The model presentation in Box 13.1 on page 230provides an example of this. It is also possible to focus onthe latest one-month change or to give some emphasis toquarter-on-quarter changes.13.5 The arguments for the choices shown in the

example are as follows. The 12-month comparison pro-vides an indication of price changes in a reasonably longtime frame, by reference to periods which may otherwisebe expected to be similar year to year. Thus, seasonalfactors are unlikely to be influential. Also, price changesthat are often decided centrally, such as those relating tothe tariffs of utilities, and changes in indirect taxes (whichhave a direct impact on prices), are usually on an annual

timetable and occur in the same month or months eachyear. There may nevertheless be one-off changes that canhave an influence on the index.

13.6 Some press releases may give prominence to themonth-on-previous-month change, especially for somecomponents of the CPI. Such data have to be presentedwith care to avoid suggesting, for example, that a 2 percent change in one month is similar to a 24 per centchange over a year.

13.7 It is also virtually universal to set a referencemonth (or longer period) in the past for which the priceindex is set at 100. All subsequent months then haveindex numbers which are percentages of the referencemonth or period. Indeed, it is that index which is used asthe basic figure from which the other changes are cal-culated.

13.8 Indices are usually shown only to one decimalplace, as are the other changes mentioned here, so figureshave to be rounded. Rounding in these circumstancescan, however, give a false impression of comparativechange and must thus be explained, especially whereprices are changing relatively little.

13.9 Care has also to be taken to differentiatebetween percentage points in the basic monthly index(which usually has 100 per cent set several years earlier)and, for example, percentage changes between onemonth and the next. If in one month the index is, forexample, 200 and the following month it is 201, then thechange can be described as one percentage point (abovethe period when the index was set at 100) or as half apercentage point (where the previous month is taken as100 per cent). Both are valid, but they are percentages ofdifferent points in the past. It is therefore important tospecify which is the base point of reference.

13.10 The reference period which is set at 100 isoften referred to as ‘‘the base period’’. But it is often arelatively arbitrarily chosen date, changed every fewyears, and not necessarily related to any point in timewhen methodologies may have changed or when a newbasket of goods and services was introduced. The statusof the referenceperiodshouldbemadeclear in themethod-ological explanation.

13.11 The CPI is, by definition, an index and there-fore not a level or a series of absolute changes in prices.Nevertheless, in the process of presenting the CPI,average prices are calculated for categories of goods andservices. It is thus possible to publish some average pricesfor groups of goods or services, and also to show theupper and lower bands of the prices from which theaverages have been calculated. Some users of the indexfind average price levels useful; these averages shouldtherefore be made available to researchers who may want

227

them. It has to be noted, however, that data on pricelevels may be less reliable than the price change indicesfor any given group of goods or services.

13.12 So far this chapter has referred only to thebroadest aggregates, without reference to subgroups ofprices or to variants of the CPI which may include orexclude certain items. All of the foregoing refers to themost common form of CPI, which is usually intended torefer to the ‘‘average consumer’’ in a specific country andto include virtually all consumer prices in that country.But it can equally refer to regions of a country or tosubgroups (such as pensioners), or to related or alter-native measures of price change. Related or alternativemeasures, and sub-aggregate indices, are discussed inparagraphs 13.24 to 13.37 below.

Seasonal adjustment andsmoothing of the index

13.13 The treatment of seasonal products and theestimation of seasonal effects are discussed in Chapter22. In the present chapter we discuss the disseminationof such adjusted or smoothed series.

13.14 Most series of economic statistics are shownseasonally adjusted, as well as unadjusted. Consumerprice indices are, however, not usually seasonally adjus-ted, although they sometimes are. Seasonal factors, forany series, are usually frequently recalculated using thelatest data, so seasonally adjusted series can be changedin retrospect, but unadjustedCPIs are not usually revised.

13.15 In comparing one month with the same montha year earlier, it is assumed that seasonal patterns aremuch the same from one year to the next. There may be,however, exceptional months when the usual seasonalchange is advanced or delayed. Such exceptional circum-stances should be noted as one of the likely causes of achange in the CPI or in one of its components.

13.16 Changes over periods of less than a year are ofcourse subject to seasonal factors and, in order to dif-ferentiate seasonal factors from other factors, it is nec-essary to make estimates of seasonal effects and to notethem as factors that have contributed to changes in theindex.

13.17 Although the CPI itself is not usually season-ally adjusted, some variants of the CPI may be seasonallyadjusted, perhaps because they are more subject to sea-sonality and because they can be revised in retrospectif necessary. If such variants are seasonally adjusted,it is important to explain why. Seasonal adjustmentusually leads to a smoother series than the originalunadjusted one. There are also other ways of smoothinga monthly series, for example using three-month movingaverages.

13.18 Statistical offices do not usually smooth theCPI series in their published presentations. Consumerprice changes are not usually so erratic from month tomonth as to disguise price trends. If there is an erraticchange, the producers of the index can usually explainthe reasons for it. In any case, where any seasonallyadjusted or smoothed series is published, it is importantto publish the unadjusted as well as the adjusted series, so

that the effect of the adjustment process is clear to userswho may wish to know what has happened to prices,whether or not the changes can be put down to seasonalfactors. Similarly, a full explanation should be given forthe reasons why a particular seasonal adjustment pro-cedure has been followed.

Analysis of contributionsto change

13.19 The CPI is an aggregate of many differentgoods and services whose prices are changing at differentrates, some of which may be going up while others aregoing down. Many users of the index want to knowwhich goods or services have contributed most to chan-ges in the index, and which prices may be out of step withgeneral price trends.

13.20 The statisticians who calculate the index arewell placed to provide analyses of the contributions tothe price change, and to do so at the same time as theindex is published. Sufficient detail should be madeavailable so that users can see for themselves what hashappened to various groups of prices. In addition, toassist journalists and others working under time con-straints, the statistician should indicate the goods orgroup of products whose changes in price are the maincontributors to the aggregate CPI, and also goods whosechanges in price are the most different from the aggre-gate. The statistics can be presented in the form of tablesand charts so that the trends may be compared. Simi-larly, statisticians should indicate any reasons for pricechanges which may not be immediately obvious but arenevertheless discernible from the published figures. Forexample, if there has been a sharp price rise or fall oneyear earlier, then it will affect the current year-on-yearchange, whatever happens to prices currently.

13.21 Analysis of contributions to change shouldalso refer to any pre-announced price changes, or majorchanges since the last price-reporting date, which willaffect the outlook for the index over the followingmonths.

Economic commentary andinterpretation of the index

13.22 In undertaking an analysis such as that de-scribed above, statisticians must be objective so thatusers of the data may differentiate clearly between thefigures themselves and the interpretation of them. It istherefore essential to take care to avoid expressingany judgement of the impact of current policy on pricechanges or the possible implications of price changes forfuture policies. Whether the figures should be seen asgood news or bad news is for the users to decide forthemselves. The statistician’s role here is to make it aseasy as possible for users to form their own judgementsfrom the perspective of their own economic or politicalviews.

13.23 There are several ways of avoiding any ap-parent or real lapses in objectivity in the analysis. Thefirst, and perhaps the most important, is to publish the

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

228

figures independently of any ministerial or other politi-cal comment. Another is to be consistent in the way theanalysis is presented. That is to say, the data should bepresented in much the same format every month (seeparagraphs 13.38 to 13.41 below). For example, tablesand charts should cover the same periods every month,and use the same baselines.

Presentation of related oralternative measures

Core inflation13.24 For the purposes of economic analysis, it is

desirable to construct measures of ‘‘core’’ or ‘‘under-lying’’ inflation which exclude movements in the inflationrate that are attributable to transient factors. In otherwords, measures of core or underlying inflation seek tomeasure the persistent or generalized trend of inflation.Central banks, for example, need to have measures of thegeneral trend of inflation when setting monetary policy.For this reason, economists and statisticians are in-creasingly interested in developing measures of ‘‘under-lying inflation’’.13.25 Several methods can be used to derive a mea-

sure of underlying inflation. Most measures focus onreducing or eliminating the influence of exceptionallyvolatile prices, or of exceptionally large individual pricechanges. Themost traditional approach is to exclude par-ticular components of the CPI on a discretionary basis.The items to be excluded would be based on the statis-tician’s knowledge of the volatility of particular items,depending on the economic conditions of the country.Items commonly excluded under this approach are freshmeat, fruit and vegetables, and petroleum. Many coun-tries also exclude imported goods, government charges,and government-controlled prices. In some countries, acalculation is made to exclude the effect of indirect taxessuch as VAT. Of course, care must be taken so as not toexclude so many items that the remainder becomes only asmall and unrepresentative component of the total.13.26 Other methods include smoothing techniques,

for example annualizing three-month average inflation.A more difficult method is to exclude outliers, that isthose items with the highest or lowest increases.

Alternative indices13.27 An example of an alternative index is a ‘‘tax

and prices index’’ in which income tax and sometimessocial security payments are taken into account. Such anindex estimates how much a taxpayer’s gross incomeneeds to change in order to maintain his or her spendingpower. It combines changes in direct (income) tax withchanges in consumer prices.13.28 Another example is an index which reflects

changes in prices excluding indirect taxes (such as salestaxes) and duties. When compared to the CPI itself, suchan index indicates the effects on prices of changes inindirect (e.g. sales) taxation.13.29 Both of these examples involve allowing for

taxes in one form or another. They are more complex

than the CPI itself, and do not have the intuitive attrac-tion of an index which aims at tracking the change inprices of a typical basket of consumer goods and services.As such, they should be presented as interesting andenlightening constructs based on the core index. It mustbe made clear that they are not replacements for, orsuperior to, the CPI itself.

13.30 A further example is the European Union’sHarmonized Indices of Consumer Prices (HICPs), whichare used to compare and aggregate price movementsacross European Union economies. The HICPs do notuse a common basket of goods for all the countriesin which they are calculated, because buying habitsare different from one country to another, but the con-cepts and methods are nevertheless harmonized in otherways. No European Union member uses the HICPs as itsnational CPI, and therefore member countries also pro-duce and publish their own indices. Although the HICPsare already used as an important indicator in the zonewithin Europe which uses the Euro as its underlying unitof currency, the HICPs are nevertheless relatively new,and are still under development. This is a case where thepresentation of an alternative index may raise seriousquestions about whether it may be superior to thenational CPI. It is therefore important to explain clearlythe underlying concepts (which generally differentiate theHICPs from national CPIs) and to explain in some detailthe reasons why the results are different. The HICPs werenot calculated before 1996, and therefore do not enableprice comparisons before that date. The starting dateshould be indicated if it is not obvious in any presentation.

13.31 Another concept is the cost of living index(COLI), which is usually defined as an index that indicatesthe changes in the costs associated not just with buyingthe same basket of goods, but with providing the sameutility or usefulness to the consumer. Countries do notusually attempt to calculate a COLI on a regular basis,but users frequently refer to the CPI as a cost of livingindex. It should be made clear, in any background notes,whether this is indeed the concept underlying the CPI.

Sub-aggregate indices13.32 Countries commonly calculate price indices

for hundreds of products (for example bread or foot-wear), based on thousands of individual price records.The number of possible sub-aggregates is therefore verylarge indeed.

13.33 One kind of sub-aggregation is the grouping ofsets of items or products which, when the sets are takentogether, comprise the whole of the CPI. An importantconsideration here is the relationship of products withinthe subgroups. For example, an index may be presentedfor food and, under the heading of food, indices may bepresented for subgroups such as cereals and vegetables.

13.34 One of the first considerations in present-ing such sub-aggregate data for related products isconsistency. That is to say, there should be a set of sub-aggregates for which indices are calculated and pre-sented each month. Users commonly attach greatimportance to being able to continue their analysis frommonth to month.

PUBLICATION, DISSEMINATION AND USER RELATIONS

229

13.35 Another consideration is international stan-dardization of the division of the index into groups ofgoods and services, which enables comparison betweencountries. Some countries also have their own sub-aggregate groupings whichmay predate the current inter-national standard. The generally accepted internationalstandard for the presentation of sub-aggregates is theClassification of Individual Consumption according toPurpose (COICOP). It is used, for example, in theHICPs. Because COICOP defines groups of items by thegeneral purpose for which they are used (e.g. ‘‘transport’’or ‘‘housing and household services’’), it combines goodsand services within the same subgroups. Many nationalclassifications are, however, composed of subgroups inwhich goods and services are never in the same subgroup.Where the national CPI is sub-aggregated by divisionsother than the international standard, it is advisableeither to present a breakdown also by COICOP or atleast to show how the national classification compares tothe international standard. COICOP and the relatedCentral Product Classification (CPC) are discussed inmore detail in Chapter 3 of this manual.

13.36 A further type of sub-aggregate index is an indexwhich is essentially the same as the CPI except that itexcludes certain items from it. The core index discussedearlier is an example. It could also be argued that theHICPs are such an index because they exclude certain non-

monetary expenditures. Some countries publish, in addi-tion to their all-items CPI, an index or indices whichexclude certain expenditures. An example is an index whichexcludes mortgage interest payments from housing costs.

13.37 In the presentation of all related or alternativemeasures, their definitions should be made clear. It isalso advisable to give the reasons for their publication.Most importantly, it should not be suggested that thesub-aggregate index is more meaningful than or superiorto the CPI itself.

Press release, bulletin andmethodological statement

13.38 The model presentation of a CPI in Box 13.1 isan example of the first page of a press release for a fic-titious country. Other formats are possible. For example,the presentation might include a seasonally adjustedindex. As indicated in the model, the presentation shouldcontain the following information:

– details of issuing office;

– date and time of release;

– percentage change in new month over the same monthone year earlier;

– comparison with change in previous month;

Box 13.1 Model presentation of consumer price index

Office of [name of country] StatisticsFriday 18 February 2000, for release at 11.00 a.m.

CONSUMER PRICE INDEX (CPI)JANUARY 2000: PRESS RELEASE

In January 2000, consumers were paying 1.0 per cent more than they did in January 1999 for the goods and services in the CPI basket.This 12-month change was lower than the 12-month change recorded in December (1.5 per cent) but higher than in November (0.9 percent).

Percentage change in the consumer price index over the same month of the previous year, for the last five years

Year

Per

cent

age

chan

ge o

ver

the

sam

e m

onth

of t

he p

revi

ous

year

0

1

2

3

4

5

1995 1997 1998 19991996 2000

Main contributions to the overall 1.0 per cent increaseThe largest increase was in the prices of clothing and footwear, with smaller increases in recreation and culture. Within the energy

group of prices, there was a significant increase in gas tariffs. There were falls in the prices of furnishings and household goods. Thechanges in product groups are shown in the table on page x of this release.

Issued by the Office of Xxxxx Statistics, address xxxxxxPress enquiries 1 111 1111; Public enquiries 2 222 2222 (name of a contact is helpful)Background notes on the CPI are given in the annex to this note.More notes and more details are given in our Internet site at XXX

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

230

– information on the product groups which contributedto the change and on any significant component price;

– reference to where more information can be found.

Note that no judgements are offered on policy or eco-nomic reasons for the price change, and no judgement isgiven on whether the change is good or bad.13.39 What is not obvious from just one example is

that the format of the press release should be the samefrom month to month. Using a consistent format isimportant in order to avoid appearing to choose a dif-ferent format to indicate a preferred trend, for examplefrom a selected starting date.13.40 Other pages of the press release should give the

monthly indices (base period equals 100) from which thepercentage changes are calculated. Similar indices shouldalso be given for major groups of goods and services.Charts may also be used to illustrate, for example, whichprices have contributed most or least to the overall CPI.13.41 If any other consumer price variant is also

being published, then the differences between the indicesshould be briefly explained, including any methodolo-gical differences. Such variants that require explanationinclude, for example, a national index based on theEuropean Union’s HICPs methodology, any regionalindices, or versions of the CPI that exclude particularcomponents of consumer expenditure such as housepurchase. The press release should include a short noteon methodology, similar to that given in Box 13.2. Moredetailed explanation could be given in a handbook.

International standardsconcerning the disseminationof consumer price indices13.42 There are many international standards which

apply, in general terms or specifically, to the CPI. Theintroduction to this chapter lists some of the broadprinciples which are reflected in many of the interna-tional standards in some form. One very general stan-dard, but by its nature a fundamental one, is the UnitedNations Fundamental Principles of Official Statistics. It isavailable on the web sites of the UNECE and the UnitedNations in several languages. It refers not just to dis-semination but to all aspects of statistical work.13.43 The International Monetary Fund (IMF)

standards are particularly pertinent in regard to dis-semination. There are two which refer to statistics includ-ing consumer price indices. One is the General DataDissemination System (GDDS), and the other is theSpecial Data Dissemination Standard (SDDS). TheGDDS provides a general framework, with some specificindicators defined as ‘‘core’’ and others defined as‘‘encouraged’’. The SDDS is based on the GDDS frame-work, but is more demanding and applies only to thosecountries that choose to subscribe to it in writing to theIMF Board. Both standards are available on the IMFweb site.13.44 Under the heading of quality, the GDDS

refers to the necessity to provide information on sourcesand methods, as well as on component details and

checking procedures. Under the heading ‘‘integrity’’, itrefers to declared standards of confidentiality, internalgovernment access before data release, identification ofministerial commentary, and information on revisionand advance notice of changes in methodology. Underthe heading ‘‘access by the public’’, it refers to theneed for pre-announced release dates and simultane-ous access for all users. In the tables of data categories,it refers to the CPI as a core indicator which shouldbe issued monthly, within one to two months of thedata collection date. All of these standards are reflectedin the present manual. The ILO has also publishedguidelines concerning dissemination practices for labourstatistics (ILO, 1998), which are available on the ILOweb site.

Timing of dissemination of theconsumer price index

13.45 The CPI should be released as soon as possible,but it is equally important to release the index accordingto a strict timetable. It is also important to publish thetimetable of release dates as far in advance as possible.Having a fixed release date, published well in advance, isimportant for two main reasons. First, it reduces thescope for the manipulation of the release date for politicalexpediency. Second, it gives confidence to users that therelease date is as soon as possible and has not beendelayed (or brought forward) for purely political reasons.A third advantage is that users know when to expect thefigures and can be prepared to use them.

Timeliness of release versusdata accuracy

13.46 The IMF’s GDDS, discussed in paragraphs13.43 and 13.44 above, recommends that the CPI be

Box 13.2 Model note on methodology – tobe included in press releases on consumer

price indices

What is the consumer price index (CPI) measuring and how isit done?

The all-items consumer price index (CPI) is the main measureof what is commonly called inflation. It measures the change inprices, on average, from month to month, of the goods and ser-vices bought by most households.

Prices are collected each month from shops and other sup-pliers of goods and services. The pattern of household expen-diture on these goods and services is derived from a regularhousehold budget (or expenditure) survey. The prices and spend-ing patterns are then combined to calculate the price indices forgroups of goods and services and for the all-items index.

The overall index, with all of its component indices, is publishedeach month in our CPI Bulletin. The Bulletin also contains moreinformation on the methodology used in calculating the CPI. Asmall booklet is also available. For a detailed account of themethodology used in calculating the CPI, please see the CPItechnical manual. For more information on these publications, andhow they may be obtained, please refer to our web site atwww:ous:gov or telephone the numbers given on the front of thispress release.

PUBLICATION, DISSEMINATION AND USER RELATIONS

231

released within one to twomonths of data collection eachmonth. It is usual, in practice, for most countries torelease the CPI in the middle of the month after themonth to which the index refers. This is possible because,in many cases, the data are collected mainly over a lim-ited period in the middle of the month to which the latestdata refer. Thus the statisticians have some time to checkand analyse the data, and to prepare the many tables andcharts in which the data will be disseminated.

13.47 The accuracy of the index is particularly impor-tant because so much depends on the CPI. In addition tothe economic policy implications of the index, the CPI isused in most countries in a variety of contracts. Perhapsthe best-known contractual use is the indexing of wagesand salaries. Also, partly because it is rare for more datato emerge after the CPI is published, and partly because ofthe way in which the index is used in contracts, it is veryrarely revised. This represents a major difference betweenthe CPI and other economic or socioeconomic aggregates.

13.48 It follows that, although timeliness is impor-tant, the timetable must allow time for the data to beproperly prepared and thoroughly checked. Afterthe release date, in most cases, a revision to the non-seasonally adjusted CPI would not be permissible. TheHICPs of the European Union are an exception and arerevised from time to time. If any series is revised, then ofcourse the changes must be fully described and explainedwhen the new data are released. If there is any method-ological change, this is usually known in advance. Usersshould be warned before any such change occurs.

Access to data13.49 With the CPI as with other statistics, users

should be allowed access to as much data as possible fortwo main reasons. First, some users find the detaileddata very useful in their analysis. Second, access to thedata inspires confidence in the data.

13.50 There are, however, limits on the quantity ofdata that can be made available to users. One reason isconfidentiality, which is addressed in the next section ofthis chapter. Another is the quantity of data that mostusers can absorb. A further reason is the cost of pub-lishing large quantities of data which few users may need.

13.51 In general, the CPI and its major componentsare deemed to be of such wide importance that they aremade available free through press releases. More detaileddata are, however, often published only in books andother media, and are charged for in order to recoversome of the dissemination costs. Similarly, special ana-lyses made at the request of particular users are usuallycharged for at a rate commensurate with the workinvolved.

13.52 The quantity of data to which users should begiven access through the various possible media is alsodiscussed in paragraphs 13.53 to 13.58 below.

Confidentiality13.53 Although, in general, as much data as possible

should be made available to users, there are reasons why

confidentiality is important in some instances. First,some data are supplied by retailers and others on theunderstanding that the data will be used only for thepurpose of aggregation with other data and will not bereleased in any other form. This can be especially impor-tant where the data are given voluntarily, as they oftenare. Second, only a sample of particular brands is pricedas representative of a much larger group of products. If itis known which brands are included in the index andwhich are not, then it might be possible to bias compo-nents of the index by manipulating a small number ofprices.

13.54 Even the knowledge that price data are, ormight be, collected on one particular day in the monthcould enable some component price indices to be biasedby retailers or others choosing to change prices on aparticular day. This is, however, only a short-run dangerand cannot be sustained.

Electronic dissemination13.55 The World Wide Web has several advantages

as a dissemination medium. For the data producer, dis-tribution costs are relatively small. No printing ormailing costs are involved. As soon as the data are on theWeb, they are available to all Web users at the same time.Putting a large amount of data on the Web costs littlemore than putting on a smaller amount. Web users candownload the data without re-keying, thus increasingspeed and reducing transmission or transposition errors.

13.56 Among the disadvantages of dissemination viathe World Wide Web is that not all data users have equalaccess to the Web. Another important disadvantageis that users may go straight to the data, without read-ing the metadata which may be crucial to the properunderstanding of the data. Also, it may be as easy for auser to disseminate the CPI widely by electronic means asit is for the statistical office, thus enabling users to pre-empt the producers by circulating the index in advance ofthe release time, perhaps without the metadata whichmay be essential to a proper understanding of the figures.

13.57 Ideally the CPI, complete with any essentialmetadata, should be released simultaneously to the pressand other users. One way in which some statistical officesare ensuring this is to bring the journalists togetherperhaps half an hour before the official release time,provide them with the printed press release, explain thedata and answer any questions. Then, at release time, thejournalists are permitted to transmit the figures to theiroffices for wider distribution.

13.58 In essence, care must be taken to ensure thatthe CPI is available at the same time to all users,regardless of the dissemination medium used.

User consultation

Different uses of consumerprice indices

13.59 The different uses of CPIs are discussed insome detail in Chapter 2. It is important to explain topotential users of the CPI which are suitable uses and

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

232

which are not. To this end, it is important to explain howthe CPI is constructed, and to provide details of itssources and methods. It is also important to make readilyavailable explanations of alternative indices or sub-indices, indicating how their uses differ from the uses ofthe CPI itself.

Presentation of methodology13.60 When the CPI is published each month, users

are anxious to see the main figures and to use them. Usersdo not generally want to be burdened with explanationsof the methodology underlying the data. Nevertheless,methodological explanations must be accessible to thosewho may want them, and in forms which are compre-hensible to users with different levels of expertise andinterest. Any significant changes in methodology must befully explained, and notified as far in advance as possibleof the change being made.13.61 In addition to a brief statement in press releases

(see paragraphs 13.38 to 13.41 above), methodologicalexplanations should be available on at least two levels.Non-experts should be able to refer to a booklet whichexplains the history, principles and practice underlyingthe CPI and any alternative measures which may also beavailable. A more thorough explanation of sources andmethods should also be readily available for those userswho are sufficiently interested and, for example, forstatisticians who may be working on the production ofthe CPI for the first time. The information must also bekept up to date despite the pressures to devote time to theoutput at the expense of documentation. As noted else-where, the ready availability of a full explanation ofsources and methods is essential to confidence and trustin the CPI.

Role of advisory committees13.62 For a statistical series as important as the CPI,

it is essential for there to be an advisory committee, or setof committees, representing users and producers. Thereare many contentious issues in the construction of theCPI. In many countries there have been fierce argumentsabout, for example, which components should be inclu-ded and excluded. The role of an advisory committee isto consider and to advise on contentious and other

issues. Perhaps an equally important role of an advisorycommittee is that its very existence provides reassurancethat the CPI can be trusted and is not a tool of govern-ment propaganda.

13.63 In those countries where advisory committeeshave not been the norm, there may be a fear on thepart of statisticians that including non-governmentalparticipants may raise expectations beyond what thestatisticians can deliver, thereby increasing dissatisfac-tion among the general public. In fact, the inclusion ofnon-governmental users can lead to a greater under-standing of the realities and the practical constraints tomeeting theoretical needs. This is the usual experienceof offices that already have advisory bodies whichinclude representatives of all the major constituencies,both inside and outside government. It is thereforeimportant that the advisory committee should comprisepeople such as academics, employers, trade union rep-resentatives and others who have an interest in the indexfrom differing points of view. It is also important thatthe reports of the advisory committee are made availableto the public fully and without undue delay.

Explaining index quality13.64 The CPI is regarded with suspicion at many

different levels. It usually refers to the average consumer,but each consumer has a different spending pattern fromthe spending patterns of others and may notice changesin one set of prices but not in others. More importantly,perhaps, there is criticism of the index because of suspi-cion that it does not keep track of newer types of goodsand services, changes in the quality of products, or newertypes of retailing.

13.65 In the light of such suspicion, it is impor-tant for the producers of the index to be willing to dis-cuss these issues and to explain how they are being dealtwith. As with other issues discussed here, the producersof the index must be open about their methods and theextent to which they can, or cannot, overcome thepotential or real problems which have been identified.It follows that the statisticians who produce the indexshould publish explanations of quality aspects, whe-ther or not the quality of the index is currently beingquestioned.

PUBLICATION, DISSEMINATION AND USER RELATIONS

233

14THE SYSTEM OF PRICE STATISTICS

Introduction14.1 This chapter focuses on the value aggregates for

goods and services that relate the major price indices,including the consumer price index (CPI), to one another.The chapter provides a deeper context for the domain ofthe CPI covered in Chapter 3 and the index weights dealtwith in Chapter 4. It also deepens the context for definingthe sample unit and the set of products, discussed inChapter 5.14.2 We begin by defining a value aggregate for a

domain of goods and services as the sum of the productsof the prices and quantities of those goods and services.A price index may be characterized as the factor givingthe relative change in this value aggregate arising fromchanges in prices. As such, all the major price indexformulae can be expressed as weighted averages of pricerelatives whose weights are the shares of items in thevalue aggregate. For the best-known price index for-mulae expressed as value aggregates of share-weightedaverages of price relatives, see Chapter 1, equation (1.2)and Chapter 15, equation (15.8) for the Laspeyres index.See Chapter 1, equation (1.3) and Chapter 15, equation(15.9) for the Paasche index, and Chapter 1, equations(1.11)–(1.12) and Chapter 15, equations (15.21) and(15.81) for the Walsh and Tornqvist indices. As thegeometric mean of the Laspeyres and Paasche indices,the Fisher ideal index of Chapter 1, equation (1.10)and Chapter 15, equation (15.12) is also a function ofexpenditure shares derived directly from the valueaggregate.14.3 To define a price index, we first need to know

several things about the value aggregate. The valueaggregate defines the following aspects of a price index:

� which commodities or items to include in the index;

� how to determine the item prices;

� which transactions that involve these items to includein the index;

� how to determine the weights, and from which sourcesthese weights should be drawn.

Besides the content of the value aggregates for the majorprice indices, we also discuss in this chapter theirvaluation and timing properties. These properties bearimportantly on how compilers define the prices andweights of price indices.14.4 The four principal price indices in the system of

economic statistics are the consumer price index (CPI),the producer price index (PPI), and the export andimport price indices (XPI and MPI). They are well-known and closely watched indicators of macroeco-nomic performance. They are direct indicators of the

purchasing power of money in various types of trans-actions and other flows involving goods and services.Consequently, these indices are important tools inthe design and conduct of the monetary and fiscal policyof the government. They also are used as deflators toprovide summary measures of the volume of goods andservices produced and consumed. They thus are alsoused to inform economic decisions throughout the pri-vate sector. They do not, or should not, comprise merelya collection of unrelated price indicators, but pro-vide instead an integrated and consistent view of pricedevelopments pertaining to production, consumption,and international transactions in goods and services. Byimplication, the meaningfulness of all of these indicesderives in no small measure from the meaningfulness ofthe value aggregates to which each refers. Although thereare other important price indices, most of which also arediscussed in this chapter, these four constitute thebackbone of the system of price statistics in most coun-tries, and they will be given special attention.

14.5 Paragraphs 14.8 onwards establish the rela-tionships among the four major price series by asso-ciating them with certain of the interlocking aggregatesdefined in the System of National Accounts 1993 (SNA1993). The system of national accounts (SNA) has gonethrough various versions over the years, the 1993 editionof this manual being the latest. We will use SNA to referto the system of national accounts generically, and SNA1993 to refer specifically to the most recent version, asappropriate. The CPI draws its coverage from a varietyof accounts in the SNA. At various points along the way,we note whether and how the composition of each valueaggregate in the national accounts relates to the aggre-gate on which the CPI may be defined. Besides the fourmain price indices and an array of additional useful priceindices, we briefly consider labour compensation indicesand purchasing power parities in the system of economicstatistics.

14.6 As noted in Chapter 2, the CPI is constructedfor a range of uses in various countries, but we canidentify two broad themes: the consumption (sometimescalled the cost of living) CPI, and the transactions (oftencalled the inflation) CPI. Advocates for the transactionsCPI often refer to it as an acquisitions CPI, following thelanguage of the earlier Consumer price indices: An ILOmanual (Turvey et al., 1989) which used this terminologyto distinguish alternative treatments of, for example,owner-occupied housing (p. 15). The term ‘‘acquisitionsCPI’’ has a different meaning in the SNA, referring tohouseholds’ consumption of goods and services securednot only by themselves, but also by non-profit institu-tions and government on their behalf. We thus use the

235

term ‘‘transactions’’ instead. In the ILO manual’s ter-minology, what we call a consumption CPI would havebeen called a ‘‘uses’’ CPI. Either is consistent with cur-rent SNA terminology.

14.7 Both types of CPI are oriented towards theprice experience of households, but, as its name implies,the consumption CPI focuses on the prices of items onwhich households make final consumption expendi-tures, while the transactions CPI focuses on the prices ofitems on which households make monetary finalexpenditures on consumption and capital formation.Consumption CPIs thus exclude capital formationexpenditures by households (for example, on theirown dwellings), but may include both monetary andimputed consumption expenditures (for example, theimputed rent paid by homeowners on their own dwell-ings). Transactions CPIs focus only on the prices ofitems on which households make monetary finalexpenditures, and thus may include household capitalformation expenditures (for example, net acquisitions ofdwellings), but categorically exclude expenditures thatmust be imputed in order to cover households’ effectiveconsumption of goods and services. In this chapter, wewill further explain the concepts of institutional sectorand type of transaction from the SNA that define thedistinction and the relationship between the consump-tion and transactions CPI. In each of the followingsections, as relevant, we will discuss the kinds ofexpenditures defining the items and weights appropriatefor each of these two main types, referring to the sum ofexpenditures corresponding to the consumption CPI asexpenditure aggregate #1 and that for the transactionsCPI as expenditure aggregate #2.

National accounts as a frameworkfor the system of price statistics

14.8 The system of national accounts is the coresystem of value aggregates for transactions and otherflows in goods and services. It is clearly of broad eco-nomic interest. Granted, the value aggregates of themajor price indices need not be coincident with the majorvalue aggregates in the national accounts. The nationalaccounts aggregates, however, represent the major flowsof goods and services and levels of tangible and intan-gible stocks in the economy. The major price indicestherefore should have a clear relationship to these ag-gregates. This chapter explains the value aggregates nowin common use by national authorities for the majorprice indices, or planned for future use, by assemblingthem from components identified in the SNA.

14.9 The SNA 1993 describes the system of nationalaccounts as follows:

1.1 The System of National Accounts (SNA) consistsof a coherent, consistent and integrated set of macro-economic accounts, balance sheets and tables based on aset of internationally agreed concepts, definitions, classi-fications and accounting rules. It provides a comprehen-sive framework within which economic data can becompiled and presented for purposes of economic analy-sis, decision taking and policy making.

The accounts cover the major economic activities takingplace within an economy, such as production, con-sumption, financing and the accumulation of capitalgoods. Some of the flows involved, such as income, sav-ing, lending and borrowing, do not relate to goods andservices and do not factor into price and quantity com-ponents. However, the SNA also contains a compre-hensive framework, the supply and use table, discussed inmore detail below, which establishes and displays theinterrelationships between all the main flows of goodsand services in the economy. The coverage and contentsof these flows are defined, classified and measured in aconceptually consistent manner. Within this table, thelinkages between major flows of goods and servicesassociated with activities such as production, consump-tion, distribution, importing and exporting can be seen ina simple and direct way. The table provides an idealframework for designing and organizing a system ofinternally consistent price statistics that relate to a set ofeconomically interdependent flows of goods and services.The table not only establishes the interrelationships be-tween consumer, producer, import and export pricesthemselves, but also their linkages with price indices formajor macroeconomic aggregates such as gross domesticproduct (GDP).

14.10 In this overview of price indices, we first take atop-level view of the major national accounts aggregates.We then begin a review of the underlying construction ofthese aggregates by considering first the types of eco-nomic agents in the economy that are recognized in thenational accounting system, and second, the economicaccounts kept on them involving goods and servicesflows that build up to the main aggregates. As theseaccounts are built up from their foundations, preciserelationships emerge between the well-known headlineprice indicators – the PPI, CPI, XPI and MPI – and theclosely watched national accounts aggregates.

Aggregate supply and useof goods and services

14.11 At the most aggregate level, the supply and useof goods and services in the national accounts is thesimple textbook macroeconomic identity equating totalsupply with total uses. Total supply is the sum of outputY, imports M, and taxes less subsidies on products T.Total use is the sum of intermediate consumption Z, thefinal consumption of households C and government G,capital formation I, and exports X:

Y+M+T=Z+C+G+I+X (14.1)

14.12 Rearranging this identity by subtracting inter-mediate consumption and imports from both sides, wearrive at the familiar alternative expressions for GDPfrom the production (value added) and expenditureapproaches:

(Y � Z)+T=Value added+T � C+C+I+X �M=Gross domestic product (14.2)

GDP is, of course, internationally recognized as thecentral national accounts aggregate for measuring

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

236

economic performance. It is essentially a measure ofproduction, as distinct from final demand. More precisely,it measures the value added of the productive activitiescarried out by all the economic agents resident in an eco-nomy. As imports are not included in GDP, a price indexfor GDP tracks internally generated inflation. Compilingindices for tracking the parts of relative change in GDPand its components that can be attributed to price andvolume change is among the most important objectives forthe development of price statistics in modern statisticalsystems.14.13 As explained in more detail later, the supply

and use table in the SNA is a comprehensive matrixcovering the economy as a whole that exploits the iden-tities (14.1) and (14.2) at a disaggregated level. Each rowof the matrix shows the total uses of a commodity, orgroup of commodities, while each column shows thetotal supplies from domestic industries and imports. Thetable provides an accounting framework that imposesthe discipline of both conceptual and numerical con-sistency on data on flows of goods and services drawnfrom different sources. The flows have to be defined,classified and valued in the same way, while any errorshave to be reconciled. The table provides a good basis forcompiling a set of interdependent price and quantityindices. In the following sections, we consider the variouselements or building blocks that make up the table beforeexamining the table as a whole.

Institutional units and establishments14.14 In building the accounting system and the

major aggregates Y,M, T, Z, C, G, I and X of equations(14.1) and (14.2), the SNA 1993 first organizes the econ-omy of a country into the kinds of entities or agents thatundertake economic activity. These agents are calledinstitutional units and comprise five types resident in theeconomy, as well as a single non-resident category, therest of the world. An institutional unit is said to beresident in an economy if its primary centre of economicinterest is located there. A centre of economic interest isoperationally defined in part by the duration of physicalpresence. For example, a household is resident in an eco-nomic territory if it lives within the territory’s boundariesfor a year or more. The five types of resident institutionalunits are: non-financial corporations; financial corpora-tions; general government; households; and non-profitinstitutions serving households (NPISHs). The SNA1993 associates with institutional units the ability to holdtitle to productive assets, and thus they represent thesmallest units on which complete balance sheets can becompiled.14.15 As noted earlier, institutional units can engage

in producing and consuming goods and services and incapital formation, accumulating goods and services asproductive tangible and intangible assets. To analyseproduction, the SNA 1993 identifies a smaller unit oragent than an institutional unit, called an establishmentor local kind of activity unit (LKAU). Within an insti-tutional unit, the establishment is the smallest unitorganized for production whose costs and output can beseparately identified. Generally, establishments special-

ize in the production of only a few types of output at asingle geographical location. To compile productivitystatistics, analysts also need detail on produced and non-produced non-financial assets (capital) by establishmentfrom multi-establishment institutional units. This isbecause, as we will see, these statistics use an industry oractivity classification of establishments rather than insti-tutional units. Some institutional units may own estab-lishments in more than one industry. On the other hand,an account of financial assets and liabilities by estab-lishment is not needed and not generally available fromthe accounts of institutional units owning multipleestablishments. The latter would be necessary to makeestablishment balance sheets.

14.16 The SNA 1993 classification of institutionalunits into sectors is shown in Box 14.1. The SNA 1993classification of institutional units does not strictlyfollow the legal status of institutional units, but rathertheir function. Hence, a government-owned non-finan-cial enterprise producing output sold at prices sub-stantially covering its costs and for which a balance sheetcan be compiled would be classified as a non-financialcorporation, along with non-financial corporations thatare corporate legal entities. For further details, see SNA1993, Chapter IV. Notice that the SNA 1993 institutionalsectors represent the units typically covered in economicand household censuses and surveys. The SNA focuseson the activities of institutional units that are resident ina nation or economic territory. It makes provision for therest of the world (S.2 in Box 14.1) only to capture thetransactions of resident institutional units with non-residents. Transactions of non-residents with other non-residents are out of scope for the national or regionalaccounts of a given country or region.

14.17 The classification of household institutionalunits into sectors is highly relevant for analysing theincidence of price change. As shown in Box 14.1, theSNA 1993 defines household subsectors according tothe major source of income: mixed income (mostlyprofits of household enterprises), compensation (wages,salaries and compensation in kind), or property income(rents, dividends and interest). There are not the onlysectors of households that may be of interest to usersof the CPI, however. In addition to the source ofincome, analysts often (perhaps more often) are inter-ested in the level of income. The shares of particulargoods and services in household expenditures are likelyto show more variation across income level than acrossmajor source of income. For example, to shed light onthe price experience of poor (low-income) householdswe would want to know whether there is a significantdifference in the shares of expenditure on specific goodsand services for poor as compared with non-poorhouseholds. A good example would be the relativeimportance of expenditures for used durable goods. Aswe will see, consumer durables are measured in the SNAon an acquisitions-less-disposals basis. While poorhouseholds normally would be net purchasers of suchgoods, richer households would tend to be net sellers. Achange in the prices of used goods thus would have avery different impact on the CPIs for the two groupsof households.

THE SYSTEM OF PRICE STATISTICS

237

Accounts of institutional units14.18 In equations (14.1) and (14.2), we identified the

basic aggregates comprising the total supply and use ofgoods and services in the economy, and derived GDP interms of these aggregates. To see how to separate theprice and volume components of supply and use, it isnecessary to build these basic aggregates up from theinstitutional sector accounts of the economy’s economicagents. In this process it is important to detail the pro-duction and consumption activities of these agents, aswell as the types of goods and services they produce andconsume. The framework organizing this information isthe supply and use table. As this table is built up, weeffectively also begin to accumulate data on the productshare weights s needed for computing price index for-mulae (Chapters 1, 3, and 15–17). The basic accounts ofthe SNA in which all of these aggregates are recordedat the level of institutional units are the production, useof income, capital, and external goods and servicesaccounts. These accounts organize the information forthe following top-level aggregates:

� Production account: output Y, intermediate con-sumption Z, and value added Y – Z;

� Use of income account: household consumption C andgovernment consumption G;

� Capital account: capital formation I;

� External goods and services account: exports X andimports M.

Recording transactions in goodsand services

14.19 Before turning to further elaboration on thesefour goods and services accounts, it is important tospecify how each entry in the value aggregates compris-ing them is to be recorded. The items i in the value ag-gregate equation (15.1) of Chapter 15 represent detailedgoods and services flows classified into categories oftransactions. There are two defining aspects of recordingtransactions: timing and valuation.

14.20 Regarding the timing of transactions, to asso-ciate each transaction with a date, the national accountsconsider a transaction to have been consummated whena liability to pay is created between the units involved.For flows of goods and services, this occurs when theownership of the good is exchanged or when the serviceis delivered. When change of ownership occurs or the

Box 14.1 Institutional sectors in the System of National Accounts 1993

S.1 Total economyS.11 Non-financial corporations

Ultimate subdivisions: public, national private and foreign controlledS.12 Financial corporations

Ultimate subdivisions: public, national private and foreign controlledS.121 Central bankS.122 Other depository corporations

S.1221 Deposit money corporationsS.1222 Other depository corporations, except deposit money corporations

S.123 Other financial intermediaries, except insurance corporations and pension fundsS.124 Financial auxiliariesS.125 Insurance corporations and pension funds

S.13 General governmentAlternate scheme n = 1, social security funds shown as a separate branch of government S.1314Alternate scheme n = 2, social security funds included as components of central, state, and local branches, and S.1314deletedS.13n1 Central governmentS.13n2 State governmentS.13n3 Local governmentS.1314 Social security funds

S.14 HouseholdsClassified according to the largest source of income receivedS.141 Employers (mixed income, owning an unincorporated enterprise with paid employees)S.142 Own account workers (mixed income,1 owning an unincorporated enterprise without paid employees)S.143 Employees (compensation of employees) 2

S.144 Recipients of property and transfer income3

S.1441 Recipients of property incomeS.1442 Recipients of pensionsS.1443 Recipients of other transfers

S.15 Non-profit institutions serving households (NPISHs)S.2 Rest of the world

1To understand how subsectors S.141 and S.142 of households are formed, an explanation of the term ‘‘mixed income’’ is in order. This, in turn, requires

consideration of the national accounts income concept of operating surplus. The operating surplus of an enterprise is the residual of the value of output less

purchases of goods and services, inputs, wages and salaries, employers’ social contributions (social security and pension payments), and taxes net of

subsidies payable on production that are unrelated to products. The mixed income of household unincorporated enterprises is algebraically defined identically

with the operating surplus of other enterprises. However, for unincorporated household enterprises, the compensation of the owners or proprietors of the

enterprise may not be included in the recorded compensation of employees item, and thus the difference between output and operating cost will include

compensation for the owners’ labour. The distinct terminology merely recognizes that the owners’ wages are often inextricably mixed with the operating

surplus for these units. 2Compensation of employees comprises wages and salaries, and employer-provided benefits comprising employers’ social con-

tributions. 3Property income comprises interest, dividends and rent.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

238

service is delivered, a transaction is said to have accrued.In general, this time need not be the same as the momentat which the payment actually takes place.14.21 There are two valuation principles in the

national accounts, one for suppliers and one for users.For suppliers, transactions in goods and services are tobe valued at basic prices. The basic price is the price perunit of good or service receivable by the producer. Weuse the term receivable to indicate that the price refers toan accrued transaction for the seller, and the term pay-able to indicate a transaction that has accrued to thepurchaser. As the producer does not receive taxes (if any)on products, but does receive subsidies (if any) on pro-ducts, taxes on products are excluded from the basicprice, while subsidies on products are included. Theproducer also does not receive separately invoicedtransport and insurance charges provided by other sup-pliers, or any distribution margins added by other, retailor wholesale service producers, and these are also ex-cluded from the basic price. In contrast, the user, aspurchaser, pays all of these charges, and users’ purchasesare therefore valued at purchasers’ prices, which addtaxes net of subsidies on products and margins forincluded transport, insurance and distribution services tothe basic price.14.22 The SNA 1993 distinguishes between taxes on

products and other taxes on production. Taxes net ofsubsidies on products T include all taxes payable per unitor as a fraction of the value of goods or services trans-acted. Included in T are excise, sales, and the non-refundable portion of value added taxes, duties onimports and taxes on exports. Subsidies on productsinclude all subsidies receivable per unit or as a fraction ofthe value of goods or services produced, including inparticular subsidies paid on imports and exports. Othertaxes on production comprise, for example, taxes on realproperty and taxes on profits. Other subsidies on pro-duction include, for example, regular payments by thegovernment to cover the difference between the costs andrevenues of loss-making enterprises. Of total taxes andsubsidies on production, only taxes and subsidies onproducts are considered in defining basic and purchasers’prices. By implication, there are no taxes payable onproducts included in either of the aggregates Y or M,

while subsidies receivable on products are included inthese aggregates.

14.23 Accordingly, output Y and imports M inequations (14.1) and (14.2) are valued at basic prices, towhich are added taxes less subsidies on products T toarrive at total supply. The reader may have noted thattransport, insurance and distributionmargins have some-how disappeared after having been introduced. Whetherthese services are included with the good or invoicedseparately does not affect the total expenditure on goodsand services by the purchaser. For the economy as awhole, these transactions cancel out, but when we con-sider industry or activity andproduct detail, theywill haveredistributive effects among goods and services products.This point is revisited in the discussion of the supply anduse table below.

14.24 The components of total uses are valued atpurchasers’ prices. This is straightforwardly interpretedfor the final consumption of households and govern-ment. For capital formation expenditures, the notion ofpurchasers’ prices also includes the costs of ‘‘setting up’’fixed capital equipment. For exports, purchasers’ pricesalso include export taxes net of subsidies, according tothe ‘‘free on board’’ (fob) value at the national frontier.We now discuss each of the four major goods and ser-vices accounts in turn.

Production14.25 An institutional unit engaged in production is

said to be an enterprise. By implication, any of the fivetypes of resident institutional units can be an enterprise.The production account for enterprises in the SNA 1993appears, with minor reordering of elements, essentiallyas shown in Table 14.1. An identical presentation alsoapplies to the establishments or local kind of activityunits (LKAUs) owned by enterprises. In fact, an estab-lishment can be defined operationally as the smallest unitfor which a production account can be constructed.There are cases in which an establishment or LKAUis synonymous with or at least inseparable from theinstitutional unit that owns it. This is true of single-establishment corporations and of household unin-corporated enterprises, for example. In other cases,an enterprise may own multiple establishments. The

Table 14.1 Production account for an establishment, institutional unit or institutional sector

SNA 1993 items in bold refer to flows in goods and services

Uses Resources

P.2 Intermediate consumption (purchasers’ prices) P.1 Output (basic prices)

B.1 Gross value-added (balances the account; that is, it isthe difference between output P.1 and intermediateconsumption P.2)

Of which, memorandum items breaking down totaloutput for classifying the market/non-market status ofthe producer unit:P.11 Market outputP.12 Output for own final useP.13 Other non-market output

THE SYSTEM OF PRICE STATISTICS

239

production account can also be produced for variousestablishment and enterprise groupings, including, ofcourse, institutional sectors, but also for establishmentindustry or activity groups. In the production accountand throughout the SNA 1993, the transaction codesbeginning with P refer to entries for transactions in goodsand services. The codes beginning with B refer to so-called ‘‘balancing items’’, which are defined residually asthe difference between a resources total and the sum ofitemized uses of those resources.

14.26 For classifying an establishment or LKAU,output is broken down into market output (P.11), whichis sold at ‘‘economically significant prices’’ substantiallycovering the cost of production, and two types of non-market output that are provided without charge or atprices so low they bear no relationship to productioncost. The two types of non-market output are output forown final use (P.12) and other non-market output (P.13).Output for own final use includes the production of, forexample, machine tools and structures (fixed capitalformation items) by an establishment for the use of theestablishment itself or other establishments in the sameenterprise, the imputed rental value of certain productiveassets owned by households, such as (and currentlylimited to) owner-occupied dwellings, and the produc-tion of certain other unincorporated household enter-prises, such as agricultural products produced by farmersfor consumption by their own families or employees.Other non-market output comprises the output of gen-eral government and non-profit institutions servinghouseholds distributed free of charge or sold at pricesthat are not economically significant. In constructing aprice index, we will necessarily be focusing on thosetransactions of establishment units that involve eco-nomically significant prices, and thus on market output(P.11). The prices collected for market output items mayalso, however, be used to value the own final use portionof non-market output (P.12). Our scope of coverage forprice indices thus extends to cover this component ofnon-market output as well.

14.27 A production unit’s resources derive from thevalue of its output, and its uses of resources are the costsit incurs in carrying out production. The productionaccount therefore uses both the basic price and pur-chasers’ price methods of valuation, as appropriate to aproduction unit in its roles as a supplier and a user ofproducts. For the supply (resources) of goods and ser-vices, products are valued at basic prices, the nationalcurrency value receivable by the producer for each unitof a product. The prices include subsidies, and excludethe taxes on products and additional charges or marginson products to pay for included retail and wholesaletrade services, and for included transport and insurance.For uses of goods and services, products are valued atpurchasers’ prices, the national currency value payableby the user for each unit of a product, including taxes onproducts as well as trade and transport margins, andexcluding subsidies on products.

14.28 Product detail in the production account. Inaddition to breaking output down into its market andnon-market components, output and intermediate con-sumption also can be broken down by type of product.

Classifying product types using, for example, the inter-national standard Central Product Classification (CPC),the production account for each establishment could bearranged to appear as in Table 14.2. Table 14.2 effec-tively gives the core structure of the report form of thetypical establishment survey providing source data onproduction for the national accounts.

14.29 Industry detail in the production account. Withthe values of total output by product, and total marketand non-market outputs in Table 14.2 for each estab-lishment, we then classify the establishment by its prin-cipal activity or industry, and market/non-market status.To reflect the information required for this classification,positions for the activity and market/non-market clas-sification codes of the establishment are shown at thetop of Table 14.2. The activity classification involvesprincipally, if not exclusively, sorting establishmentsaccording to the types of product produced (CPC orother product code, such as the Classification of Pro-ducts by Activity) for which the total output is greatest.The major categories of the International StandardIndustrial Classification of All Economic Activities(ISIC), Revision 3, are shown in Box 14.2 below.

14.30 As indicated in Table 14.2, The SNA 1993recommends use of the International Standard IndustrialClassification (ISIC) for all economic activities, the CPCfor domestic products, and the closely related Harmo-nized Commodity Description and Classification System(HS) for exported and imported products. Each countrymay adapt the international standard to its specific cir-cumstances. If the adaptation amounts to adding furtherdetail, the classification is said to be derived from theinternational standard. The Nomenclature generale desActivites economiques dans les Communautes europeennes(NACE, the General Industrial Classification of Eco-nomic Activities within the European Communities) isan industrial classification derived from the ISIC. Ifthe adaptation reorganizes the way in which detailedcategories are grouped compared with the interna-tional standard, but provides for a cross-classification atsome level of detail, it is said to be related. The NorthAmerican Industrial Classification System (NAICS) ofCanada, Mexico and the United States is an industrialclassification related to the ISIC. The European Com-mission’s PRODCOM classification of industrial pro-ducts is derived from its Classification of Products byActivity (CPA) which, in turn, is related to the inter-national standard CPC through a cross-classificationdefined at a high level of product detail.

14.31 The output aggregate of the producer price indexand the production account. The producer price index(PPI) is an index of the prices of the outputs of estab-lishments. The position of the PPI in the SNA 1993 isdefined by the relationship of its output value aggregateto those defined in the national accounts. In Box 14.2,we consider the composition of the PPI value aggregateaccording to its industry coverage, arguing that the PPI’sindustry coverage should be complete. Considering fur-ther market and non-market production within an indus-try group of establishments that are classified accordingto market status, the PPI’s coverage could extend both tothe market output (P.11) and output for own final use

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

240

(P.12) identified in Table 14.2 when this account is con-sidered for all establishments in the economy. Althoughthe latter is technically non-market output, it would bevalued at the basic prices the establishment would receivewere that own-use production to be sold.14.32 The expenditure aggregate of the consumer

price index and the production account. Consumptionfrom own production is a significant fraction of totalconsumption, comprising both goods and services. Forgoods produced by households, as noted in the SNA1993:

6.24 The System includes the production of all goodswithin the production boundary. At the time the pro-duction takes place it may not even be known whether, orin what proportions, the goods produced are destined forthe market or for own use. The following types of pro-duction by households are, therefore, included whetherintended for own final consumption or not:

(a) The production of agricultural products and theirsubsequent storage; the gathering of berries or otheruncultivated crops; forestry; wood-cutting and the col-lection of firewood; hunting and fishing;

(b) The production of other primary products such asmining salt, cutting peat, the supply of water, etc.;

(c) The processing of agricultural products; the pro-duction of grain by threshing; the production of flour bymilling; the curing of skins and the production of leather;the production and preservation of meat and fish prod-ucts; the preservation of fruit by drying, bottling, etc.; theproduction of dairy products such as butter or cheese; theproduction of beer, wine, or spirits; the production ofbaskets or mats; etc.;

(d) Other kinds of processing such as weaving cloth;dress making and tailoring; the production of footwear;the production of pottery, utensils or durables; makingfurniture or furnishings; etc.

The storage of agricultural goods produced byhouseholds is included within the production boundaryas an extension of the goods-producing process. Thesupply of water is also considered a goods-producingactivity in this context. In principle, supplying water is asimilar kind of activity to extracting and piping crude oil.

6.25 It is not feasible to draw up a complete, exhaustivelist of all possible productive activities but the above listcovers the most common types. When the amount of agood produced within households is believed to be quan-titatively important in relation to the total supply of thatgood in a country, its production should be recorded.Otherwise, it is not worthwhile trying to estimate it inpractice.

For services, the SNA 1993 notes housing services as thesole – but for most countries extremely important – itemof production for own consumption:

6.29 The production of housing services for their ownfinal consumption by owner-occupiers has always beenincluded within the production boundary in nationalaccounts, although it constitutes an exception to thegeneral exclusion of own-account service production. Theratio of owner-occupied to rented dwellings can vary sig-nificantly between countries and even over short periodsof time within a single country, so that both interna-tional and intertemporal comparisons of the produc-tion and consumption of housing services could bedistorted if no imputation were made for the value of

Table 14.2 Production account with product detail for an establishment or local kind of activity unit

SNA 1993 items in bold refer to flows in goods and servicesEstablishment ID: eeeeeeee Institutional unit ID: uuuuuuuuActivity/Industry code (ISIC): aaaa Institutional sector code: S.nnnnn

Market status: P.1n

Uses Resources

P.2 Intermediate consumption (purchasers’ prices),of which:

P.1 Output (basic prices), of which:

CPC 0 Agriculture, forestry and fishery products CPC 0 Agriculture, forestry and fishery productsCPC 1 Ores and mineral; electricity, gas, and water CPC 1 Ores and mineral; electricity, gas, and waterCPC 2 Food products, beverages and tobacco;

textiles, apparel and leather productsCPC 2 Food products, beverages and tobacco;

textiles, apparel and leather productsCPC 3 Other transportable goods, except metal

products, machinery and equipmentCPC 3 Other transportable goods, except metal

products, machinery and equipmentCPC 4 Metal products, machinery and equipment CPC 4 Metal products, machinery and equipmentCPC 5 Intangible assets; land; constructions;

construction servicesCPC 5 Intangible assets; land; constructions;

construction servicesCPC 6 Distributive trade services; lodging; food

and beverage serving services; transportservices; and utilities distribution services

CPC 6 Distributive trade services; lodging; foodand beverage serving services; transportservices; and utilities distribution services

CPC 7 Financial and related services; real estateservices; and rental and leasing services

CPC 7 Financial and related services; real estateservices; and rental and leasing services

CPC 8 Business and production services CPC 8 Business and production servicesCPC 9 Community, social and personal services CPC 9 Community, social and personal services

B.1 Gross value-added

Memorandum items breaking down total output forclassifying the market/non-market statusof the producer:P.11 Market outputP.12 Output for own final useP.13 Other non-market output

THE SYSTEM OF PRICE STATISTICS

241

own-account housing services. The imputed value of theincome generated by such production is taxed in somecountries.

The SNA imputes the value of such consumption at theequivalent market value of the output households pro-duce for their own purposes.

14.33 In some cases, however, the market equivalentmethod of valuing production for own consumption isnot viable because sufficiently similar market equiva-lents to the items supplied from own production are notavailable, or they are sufficiently rare that it is tooexpensive to obtain information on them or too unreli-able to base estimates on such information. In thesecases, production cost approaches are taken. The sourceof data for the production cost approaches is, in part,the household production account as regards goodsand services purchased for intermediate consumption.The ultimate source of primary information for thehousehold production account is principally the house-hold expenditure survey, though specialized surveys ofhousehold business activity may also be undertaken forthis purpose. For the shelter services provided by housesoccupied by their owners, for example, the productionaccount would be the source of expenditures on utilities,maintenance, and do-it-yourself repair items of inter-mediate consumption that would be used, in part, todetermine the cost to an owner-occupant of the serviceshe or she derives from his or her own dwelling. For theown production of agricultural produce, purchases ofseed, fertilizer, and small garden tools might be recordedas intermediate consumption. Particularly for the latter,however, it is often difficult to distinguish between theintermediate production expense of production for ownconsumption and final consumption expenditure formaintaining decorative landscaping.

Final consumption14.34 Consumption of goods and services in the

SNA 1993 is shown in the use of income account, whichappears essentially as in Table 14.3 for each institutionalunit. It is recalled that the accounts pertaining to goodsand services in the SNA 1993 that can be decomposedinto price and volume components, and that would thusdraw our interest as price index compilers, are desig-nated by the codes P.n. Items of final consumption aredesignated by P.3 with extensions: P.3 comprises indi-vidual consumption expenditure (P.31) and collectiveconsumption expenditure (P.32).

14.35 Individual consumption, actual consumption,and household consumption expenditures. The SNA dis-tinguishes individual from collective goods and services,a distinction that is equivalent to that between privateand public goods in economic theory. The distinctionis mainly relevant to services. Individual services areprovided to individual households and benefit thoseparticular households, whereas collective services areprovided to the community, for example services suchas public order, administration, security and defence.Many individual services, however, such as education,health, housing and transport, may be financed andpaid for by government or non-profit institutions andprovided free or at a nominal price to individual house-holds. A large part of government consumption expen-diture is not on public goods but on goods or servicessupplied to individual households. These individualconsumption expenditures by governments and NPISHsare described as social transfers in kind in the SNA 1993.

14.36 The concept of ‘‘household consumption’’ canhave three distinct meanings. First, it can mean the totalset of individual consumption goods and services actu-ally acquired by households, including those received as

Box 14.2 Coverage of industries or activities by the producer price index in terms of aggregate output value

The principal economic activities of the International Standard Industrial Classification of All Economic Activities (ISIC), Revision 3, are:

A Agriculture, hunting and forestryB FishingC Mining and quarryingD ManufacturingE Electricity, gas and water supplyF ConstructionG Wholesale and retail trade; repair of motor vehicles, motorcycles, and personal and household goodsH Hotels and restaurantsI Transport, storage and communicationsJ Financial intermediationK Real estate, renting and business activitiesL Public administration and defence; compulsory social securityM EducationN Health and social workO Other community, social and personal service activitiesP Private households with employed personsQ Extra-territorial organizations and bodies

These are characteristic of the activities identified in most national industrial classifications. In assembling data on the supply and useflows in the economy, a detailed industry production account such as given in Table 14.2 is effectively constructed for each type of activity inthe economy, whose major categories are shown in the ISIC list above. With the product output and expenditure detail in Table 14.2, we canshow more explicitly the typical goods and services coverage of the PPI within the output aggregate (P.1) of the production account for eachindustry. In most countries, PPIs cover industries that produce goods, such as the mining and manufacturing activities (C–D) and some-times also agriculture (A) and fishing (B), and construction (F), as well as the two industrial service activities – electricity, gas and watersupply (E) and transport, storage and communications (I). In principle, the PPI should cover the market output of all activities, and a numberof countries are currently working on extending PPI coverage to the remaining service-producing activities besides transport and utilities.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

242

social transfers in kind. Second, it can mean the subsetwhich households actually pay for themselves. To dis-tinguish between these two sets, the SNA describes thefirst as the actual final consumption of households andthe second as household final consumption expenditures.A third possible interpretation of household consump-tion is that it means the actual physical process ofconsuming the goods and services. It is this process fromwhich utility is derived and that determines the house-hold’s standard of living. The process of consuming orusing the goods or services can take place some timeafter the goods or services are acquired, as most con-sumer goods can be stored. The distinction betweenacquisition and use is most pronounced in the case ofconsumer durables that may be used over very longperiods of time. The treatment of durables is discussedfurther in Box 14.3.14.37 The existence of social transfers in kind is

not generally recognized in CPIs, although it is desirableto take account of them, especially when consideringchanges in the cost of living. Moreover, governmentsmay start to charge for services that were previouslyprovided free, a practice that has become increasinglycommon in many countries in recent years. The goodsand services provided free as social transfers could,in principle, be regarded as also being part of house-hold consumption expenditures but as having a zeroprice. The shift from a zero to positive price is thena price increase that could be captured by a consumerprice index.14.38 Monetary and imputed expenditures. Not all

household expenditures are monetary. A monetaryexpenditure is one in which the counterpart to the good

or service acquired is the creation of some kind offinancial liability. This may be immediately extinguishedby a cash payment, but many monetary expenditures aremade on credit. Household consumption expendituresalso include certain imputed expenditures on goods orservices that households produce for themselves. Theseare treated as expenditures because households incur thecosts of producing them (in contrast to social transfers inkind, which are paid for by government or non-profitinstitutions).

14.39 The imputed household expenditures recog-nized in the SNA include all those on goods that house-holds produce for themselves (mainly agricultural goodsin practice), but exclude all household services producedfor own consumption except for housing services pro-duced by owner-occupants. The imputed prices at whichthe included goods and services are valued are theirestimated prices on the market. In the case of housingservices, these are imputed market rentals. In practice,most countries follow the SNA by including owner-occupied housing in the CPI. Other imputed prices, suchas the prices of vegetables, fruit, or dairy or meat pro-ducts produced for own consumption, may be includedif they comprise a sufficiently large component ofhousehold consumption expenditure.

14.40 Product detail in the use of income account. Aswith the production accounts of the establishmentsowned by institutional units, we can consider extendingthe product detail of goods and services consumption inthe use of income account according to the type of pro-duct consumed. In order to maintain the integration ofthe system of price and volume statistics on consumptionwith those we have just covered on production, products

Table 14.3 Use of income account for institutional units and sectors

SNA 1993 items in bold refer to flows in goods and services

Institutional unit ID: uuuuuuuu Institutional sector code: S.nnnnn

Uses Resources

P.3 Final consumption expenditure (purchasers’ prices)1 B.6 Disposable income2

P.31 Individual consumption expenditure, of which:P.311 Individual consumption expenditure, except from

production on own account, and imputed consumptionexpenditure, household sector S.14 only

P.312 Imputed expenditure on owner-occupied housingservices, household sector S.14 only

P.313 Financial intermediation services implicitly measured (FISIM)P.314 Other imputed individual consumption expenditure

P.32 Collective consumption expenditure (generalgovernment sector S.13 only)

D.8 Adjustment for the change in the net equity of householdsin pension funds3

B.8 Saving (balances the account; that is, it is the difference between disposableincome B.6 and the sum of expenditures P.3 and adjustment D.8)

1By definition, corporations have no final consumption in the SNA 1993. Thus, item P.3 and its subdivisions appear with non-zero entries only for household,government, and non-profit institutions serving households (NPISH) units. 2 The SNA 1993 derives disposable income in a sequence of accounts producing thebalancing items: value added B.1 (production account), operating surplus B.2 and mixed income B.3 (generation of income account), balance of primary incomesB.5 (allocation of primary income account), and disposable income B.6 (secondary distribution of income account). Collapsing all of these steps, disposableincome B.6 is value added B.1 less (net) taxes on production and imports (payable) D.2 plus (net) subsidies D.3 (receivable), plus compensation of employeesreceivable, plus (net) property income (receivable) D.4, less (net) taxes on income and wealth (payable) D.5, less (net) social contributions (payable) D.61, plus(net) social benefits (receivable) D.62, less (net) other transfers (payable) D.7. 3 This adjustment reflects the treatment by the SNA 1993 of privately fundedpensions as owned by the household beneficiaries of such plans. It maintains consistency between the income and accumulation accounts in the system. It is notrelevant to price and volume measurement (see System of National Accounts 1993, Chapter IX, Section A.4 for further details).

THE SYSTEM OF PRICE STATISTICS

243

would be classified according to the same system as in theproduction account. We show the major categories of theCPC 1.0 within the components of final consumptionexpenditure in Table 14.4.

14.41 Although the discussion in this chapter main-tains a consistent classification of expenditure by productacross all goods and services accounts, other functionalclassifications of expenditure have been developed foreach institutional sector for specific purposes. Theinternational standard versions of these classificationsincluded in the SNA 1993 comprise the Classification ofIndividual Consumption according to Purpose (COI-COP), the Classification of the Purposes of Non-profitInstitutions Serving Households (COPNI), the Classifi-cation of the Functions of Government (COFOG), andthe Classification of the Purposes of Producers (COPP).The first column of Tables 14.4 and 14.5 is often com-piled using data from household expenditure surveys.These data are collected using functional classificationssuch as COICOP, rather than product classifications. Tofacilitate constructing the cross-economy framework ofthe SNA 1993 considered in this chapter, there is aconcordance between the CPC and the COICOP.

14.42 A hierarchy of household consumption aggre-gates. It is worth noting that all household consumption

expenditures (that is, of the households institutionalsectorS.14) are individual expenditures, bydefinition.Thefollowing hierarchy of household consumption aggre-gates that are relevant to CPIs may be distinguished inthe SNA:

P.41 Actual individual consumption, of which:

D.63 Social transfers in kind (the individual con-sumption expenditure P.31 of general gov-ernment S.13 and NPISHs S.15)

P.31 Individual consumption expenditure, of which:

P.311 Monetary consumption expenditure

P.312 Financial intermediation services im-plicitly measured (FISIM)

P.313 Imputed expenditure on owner-occupied housing services

P.314 Other imputed individual consumptionexpenditure

The codes P.311, P.312, P.313 and P.314 do not exist inthe SNA 1993 but are introduced for convenience here.These four sub-categories of household consumptionexpenditures are separately identified in Tables 14.4 and14.5. As already noted, D.63 is usually excluded fromthe expenditure coverage of CPIs.

14.43 It is worth noting the special treatment offinancial services in the SNA 1993. FISIM comprisesexpenditures on those market services provided by finan-cial institutions that are not separately distinguishedfrom interest charges. Expenditures on financial serviceson which there is an explicit charge are already coveredin P.311. Although FISIM P.312 requires an implicitmeasurement as the difference between a market interestrate and a reference rate, it is part of an observed interestpayment and thus is not considered an imputed expen-diture in the same sense as imputed rent P.313 and otherimputed expenditure P.314.

14.44 Our item P.314, other imputed individualconsumption, includes, besides households’ productionof goods for their own consumption, expenditures ongoods and services that employers make on behalf oftheir employees asnon-cash compensation.TheSNAcallsthis item D.12, employers’ social contributions, andconsiders it in the generation of income account. It isrecognized as a component of the labour services priceindex, but is not customarily included in the CPI despiteits dual role as an item of consumption (see paragraph14.75 below).

14.45 The expenditure aggregate of the consumerprice index and the use of income account. The detaileduse of income accounts for institutional sectors can beassembled into a consolidated framework by choosingcolumns from Table 14.4 for each sector and displayingthem together as in Table 14.5, which gives an economy-wide presentation of final consumption and saving.Table 14.5, for the total economy, shows individualconsumption as comprising the individual consumptionentries P.31 of the use of income accounts for house-holds, NPISHs and the general government sector. Italso aggregates the disposable income B.6 of all three. Itshows separately the final collective consumption ofgovernment P.32. The account in Table 14.5 has beenarranged specifically to show the consumption coverage

Box 14.3 Treatment of housing and consumerdurables in the system of national accounts and in

consumer price indices

Dwellings are fixed assets. Purchases of dwellings byhouseholds therefore constitute household gross fixed capitalformation and are not part of household consumption. Theycannot enter into a price index for household consumption. Fixedassets are used for purposes of production, not consumption.Dwellings have therefore to be treated as fixed assets that areused by their owners to produce housing services. The system ofnational accounts (SNA) actually sets up a production account inwhich this production is recorded. The services are consumed bythe owners. The expenditures on the services are imputed, theservices being valued by the estimated rentals payable on themarket for equivalent accommodation. The rentals have to coverboth the depreciation of the dwellings and the associated interestcharges or capital costs.

The existence of these imputed expenditures on owner-occupied housing services has always been recognized innational accounts and most countries have also included them intheir consumer price indices (CPIs), even though other imputedexpenditures are not included.

Consumer durables, such as automobiles, cookers and free-zers, are also assets that are used by their owners over longperiods of time. In principle, they could be treated in the sameway as dwellings and be reclassified as fixed assets that produceflows of services that are consumed by their owners. For certainanalytic purposes, it may be desirable to treat them this way. Todo so in the SNA, however, would not simply be a matter ofestimating the market rentals that would be payable for hiring theassets. It would also be necessary to set up production accountsin which the durables are used as fixed assets. This has tradi-tionally been regarded as too difficult and artificial. There are alsoobjections to extending further the range of imputed flows inclu-ded in the SNA and gross domestic product. In practice, there-fore, expenditures on durables are classified in the SNA asconsumption expenditures and not as gross fixed capital forma-tion, a practice that is carried over into CPIs.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

244

Table

14.4

Use

ofin

com

eaccountw

ith

pro

ductdeta

ilfo

rin

stitu

tionalunits

and

secto

rs

Left

colu

mns

(Uses)

show

deta

ilof

far

right

colu

mn

(Resourc

es);

SN

A1993

item

sin

bold

refe

rto

flow

sin

goods

and

serv

ices,

secto

rtitles

initalic

sin

dic

ate

wheth

er

the

colu

mn

appears

inth

euse

of

incom

eaccount

for

that

secto

r

Institu

tionalunit

ID:

uuuuuuuu

Institu

tionalsecto

rcode:S

.nnnnn

Uses

Resourc

es

P.31

Individualconsumptionexpenditure

P.32

Collectiveconsumption

expenditure

P.3

Finalconsumption

expenditure

(total,

purchasers’prices)

B.6

Dis

posable

incom

e

P.3

11

Moneta

ryconsum

ption

expenditure

P.3

12

Impute

dexpenditure

on

ow

ner-

occupie

d

housin

gserv

ices

P.3

14

Oth

er

indiv

idual

consum

ption

expenditure

1of

household

s

P.3

2C

olle

ctive

consum

ption

expenditure

:genera

l

govern

ment

S.1

3only

P.3

13

Fin

ancia

lin

term

edia

tion

serv

ices

implic

itly

measure

d(F

ISIM

)

CPC

0Agriculture

,fo

restry

and

fishery

pro

ducts

CPC

0Agriculture

,fo

restry

and

fishery

pro

ducts

CPC

0Agriculture

,fo

restry

and

fishery

pro

ducts

CPC

0Agriculture

,fo

restry

and

fishery

pro

ducts

CPC

1O

res

and

min

era

l;ele

ctric

ity,

gas,and

wate

r

CPC

1O

res

and

min

era

l;ele

ctric

ity,

gas,and

wate

r

CPC

1O

res

and

min

era

l;

ele

ctric

ity,gas,and

wate

r

CPC

1O

res

and

min

era

l;

ele

ctric

ity,gas,and

wate

r

CPC

2Food

pro

ducts

,bevera

ges

and

tobacco;te

xtile

s,appare

land

leath

er

pro

ducts

CPC

2Food

pro

ducts

,bevera

ges

and

tobacco;te

xtile

s,

appare

land

leath

er

pro

ducts

CPC

2Food

pro

ducts

,bevera

ges

and

tobacco;te

xtile

s,

appare

land

leath

er

pro

ducts

CPC

2Food

pro

ducts

,bevera

ges

and

tobacco;te

xtile

s,appare

l

and

leath

er

pro

ducts

CPC

3O

ther

transportable

goods,

exceptm

eta

lpro

ducts

,

machin

ery

and

equip

ment

CPC

3O

ther

transportable

goods,

exceptm

eta

lpro

ducts

,

machin

ery

and

equip

ment

CPC

3O

ther

transportable

goods,

exceptm

eta

lpro

ducts

,

machin

ery

and

equip

ment

CPC

3O

ther

transportable

goods,

exceptm

eta

lpro

ducts

,

machin

ery

and

equip

ment

CPC

4M

eta

lpro

ducts

,m

achin

ery

and

equip

ment

CPC

4M

eta

lpro

ducts

,m

achin

ery

and

equip

ment

CPC

4M

eta

lpro

ducts

,m

achin

ery

and

equip

ment

CPC

4M

eta

lpro

ducts

,m

achin

ery

and

equip

ment

CPC

6D

istrib

utive

trade

serv

ices;

lodgin

g;fo

od

and

bevera

ge

serv

ing

serv

ices;transport

serv

ices;and

utilit

ies

dis

trib

ution

serv

ices

CPC

6D

istrib

utive

trade

serv

ices;

lodgin

g;fo

od

and

bevera

ge

serv

ing

serv

ices;transport

serv

ices;and

utilit

ies

dis

trib

ution

serv

ices

CPC

6D

istrib

utive

trade

serv

ices;

lodgin

g;fo

od

and

bevera

ge

serv

ing

serv

ices;transport

serv

ices;and

utilit

ies

dis

trib

ution

serv

ices

CPC

6D

istrib

utive

trade

serv

ices;

lodgin

g;fo

od

and

bevera

ge

serv

ing

serv

ices;transport

serv

ices;and

utilit

ies

dis

trib

ution

serv

ices

CPC

7Fin

ancia

land

rela

ted

serv

ices;

realesta

teserv

ices;and

renta

l

and

leasin

gserv

ices

CPC

7Fin

ancia

land

rela

ted

serv

ices;re

alesta

te

serv

ices;and

renta

l

and

leasin

gserv

ices

CPC

7Fin

ancia

land

rela

ted

serv

ices;

realesta

teserv

ices;and

renta

l

and

leasin

gserv

ices

2

CPC

7Fin

ancia

land

rela

ted

serv

ices;

realesta

teserv

ices;and

renta

land

leasin

gserv

ices

CPC

7Fin

ancia

land

rela

ted

serv

ices;

realesta

teserv

ices;and

renta

land

leasin

gserv

ices

CPC

8Busin

ess

and

pro

duction

serv

ices

CPC

8Busin

ess

and

pro

duction

serv

ices

CPC

8Busin

ess

and

pro

duction

serv

ices

CPC

8Busin

ess

and

pro

duction

serv

ices

CPC

9C

om

munity,socia

land

pers

onalserv

ices

CPC

9C

om

munity,socia

land

pers

onalserv

ices

CPC

9C

om

munity,socia

land

pers

onalserv

ices

CPC

9C

om

munity,socia

land

pers

onalserv

ices

D.8

Adju

stm

entfo

rth

echange

inth

enetequity

ofhousehold

s

inpensio

nfu

nds

B.8

Savin

g

1The

‘‘Oth

erin

div

idualconsum

ption

ofhousehold

s’’

com

prises

D.1

2‘‘E

mplo

yers

’socia

lcontrib

utions,consum

ption

in-k

ind

ofgoods

and

serv

ices

supplie

dto

household

sby

their

em

plo

yers

inlie

uofcash

wages,and

consum

ption

from

the

household

s’ow

n

pro

ductio

nofgoods’’.

D.1

2appears

inth

eG

enera

tion

ofIn

com

eA

ccountand

isa

facto

rin

the

dis

cussio

nofth

eprice

index

ofla

bourserv

ices

toem

plo

yers

.A

mong

‘‘Em

plo

yers

’socia

lcontrib

utions’’

are

pro

vis

ion

ofhousin

g,transport,child

care

,m

edic

al

insura

nce

and

serv

ices,and

life

insura

nce

serv

ices.‘‘E

mplo

yers

’socia

lcontrib

utions’’

als

oin

clu

de

contrib

utions

topensio

npla

ns,w

hic

hare

notconsum

ption

exceptfo

ra

sm

all

part

attributa

ble

topensio

nadm

inis

tration

serv

ices.The

resid

ualpart

ofpensio

n

contrib

utions

isan

importantcom

ponentofhousehold

savin

g.

2In

additio

nto

the

realesta

te,re

nta

land

leasin

gserv

ices

ofhom

eow

ners

,th

eS

NA

199

3treats

financia

lserv

ices

consum

ption

expenditure

as

the

sum

ofm

easure

dand

impute

dcom

ponents

.

Measure

dexpenditure

scom

prise

explic

itserv

ice

charg

es

levie

dby

financia

linstitu

tions

fordeposit,lo

an,advis

ory

serv

ices

and

the

like,w

hile

impute

dexpenditure

sre

flectth

ein

com

efo

rgone

because

the

household

does

notle

nd

(keep

deposits

with

afinancia

l

institu

tion)

orborr

ow

ata

refe

rence

rate

.In

princip

le,th

ese

impute

dexpenditure

s,as

well

as

those

foroth

erim

pute

dconsum

ption,are

ofth

esam

em

ark

et-equiv

ale

ntvalu

ed

type

as

forow

ner-

occu

pie

dhousin

gserv

ices

and

could

be

covere

din

the

CP

I.

245

Table

14.5

Use

ofin

com

eaccountw

ith

pro

ductdeta

ilfo

rth

eto

taleconom

y

Left

colu

mns

show

deta

ilof

far

right

colu

mn;

SN

A1993

item

sin

bold

refe

rto

flow

sin

goods

and

serv

ices

Institu

tionalunit

ID:

uuuuuuuu

Instit

utionalsecto

rcode:

S.n

nnnn

B.6

Dis

posable

incom

e,Tota

leconom

y

S.1

,w

ith

uses

com

prisin

g:

P.31

Individualconsumptionexpenditure,TotaleconomyS.1

(purchasers’prices),

com

prisin

g:

P.32

Collectiveconsumption

expenditure,Total

economyS.1

(purchasers’

prices),

co

mp

risin

g:

P.3

Finalconsumption

expenditure,total

economyS.1,

of

wh

ich

P.31

Individualconsumptionexpenditure,Household

sectorS.14

P.31

Individualconsumption

expenditure,generalgovernment

S.13andNPISH1S.15sectors

P.32

Collectiveconsumption

expenditure,general

governmentsectorS.13

Consum

er

Price

Index

refe

rence

aggre

gate

#1

2

P.3

11

Moneta

ryconsum

ption

expenditure

P.3

13

Impute

dexpenditure

on

ow

ner-

occupie

dhousin

g

serv

ices

P.3

14

Oth

er

impute

din

div

idual

consum

ption

expenditure

:

household

sS

.14

D.6

3S

ocia

ltr

ansfe

rsin

kin

d

P.3

12

Fin

ancia

lin

term

edia

tion

serv

ices

implic

itly

measure

d(F

ISIM

)

CPC

0Agriculture

,fo

restry

and

fishery

pro

ducts

CPC

0Agriculture

,fo

restry

and

fishery

pro

ducts

CPC

0Agriculture

,fo

restry

and

fishery

pro

ducts

CPC

0Agriculture

,fo

restry

and

fishery

pro

ducts

CPC

0Agriculture

,fo

restry

and

fishery

pro

ducts

CPC

1O

res

and

min

era

l;ele

ctric

ity,

gas,and

wate

r

CPC

1O

res

and

min

era

l;ele

ctric

ity,

gas,and

wate

r

CPC

1O

res

and

min

era

l;ele

ctric

ity,

gas,and

wate

r

CPC

1O

res

and

min

era

l;

ele

ctric

ity,gas,

and

wate

r

CPC

1O

res

and

min

era

l;

ele

ctric

ity,gas,and

wate

r

CPC

2Food

pro

ducts

,bevera

ges

and

tobacco;te

xtile

s,appare

l

and

leath

erpro

ducts

CPC

2Food

pro

ducts

,bevera

ges

and

tobacco;te

xtile

s,appare

l

and

leath

erpro

ducts

CPC

2Food

pro

ducts

,bevera

ges

and

tobacco;te

xtile

s,appare

l

and

leath

erpro

ducts

CPC

2Food

pro

ducts

,bevera

ges,

tobacco;te

xtile

s,

appare

l,le

ath

erpro

ducts

CPC

2Food

pro

ducts

,bevera

ges,

tobacco;te

xtile

s,

appare

l,le

ath

er

pro

ducts

CPC

3O

thertransportable

goods,

exceptm

eta

lpro

ducts

,

machin

ery

and

equip

ment

CPC

3O

thertransportable

goods,

exceptm

eta

lpro

ducts

,

machin

ery

and

equip

ment

CPC

3O

thertransportable

goods,

exceptm

eta

lpro

ducts

,

machin

ery

and

equip

ment

CPC

3O

thertransportable

goods,exceptm

eta

l

pro

ducts

,m

achin

ery

and

equip

ment

CPC

3O

thertransportable

goods,

exceptm

eta

lpro

ducts

,

machin

ery

and

equip

ment

CPC

4M

eta

lpro

ducts

,m

achin

ery

and

equip

ment

CPC

4M

eta

lpro

ducts

,m

achin

ery

and

equip

ment

CPC

4M

eta

lpro

ducts

,m

achin

ery

and

equip

ment

CPC

4M

eta

lpro

ducts

,

machin

ery

and

equip

ment

CPC

4M

eta

lpro

ducts

,m

achin

ery

and

equip

ment

CPC

6D

istrib

utive

trade

serv

ices;

lodgin

g;fo

od

and

bevera

ge

serv

ing

serv

ices;transport

serv

ices;and

utilit

ies

dis

trib

ution

serv

ices

CPC

6D

istrib

utive

trade

serv

ices;lo

dgin

g;fo

od

and

bevera

ge

serv

ing

serv

ices;

transport

serv

ices;and

utilit

ies

dis

trib

ution

serv

ices

CPC

6D

istrib

utive

trade

serv

ices;

lodgin

g;fo

od

and

bevera

ge

serv

ing

serv

ices;transport

serv

ices;and

utilit

ies

dis

trib

ution

serv

ices

CPC

6D

istrib

utive

trade

serv

ices;lo

dgin

g;fo

od

and

bevera

ge

serv

ing

serv

ices;transport

serv

ices;and

utilit

ies

dis

trib

ution

serv

ices

CPC

6D

istrib

utive

trade

serv

ices;

lodgin

g;fo

od

and

bevera

ge

serv

ing

serv

ices;transport

serv

ices;and

utilit

ies

dis

trib

ution

serv

ices

CPC

7Fin

ancia

land

rela

ted

serv

ices;re

alesta

teserv

ices;

and

renta

land

leasin

gserv

ices

CPC

7Fin

ancia

land

rela

ted

serv

ices;re

alesta

te

serv

ices;and

renta

l

and

leasin

gserv

ices

CPC

7Fin

ancia

land

rela

ted

serv

ices;re

alesta

teserv

ices;

and

renta

land

leasin

gserv

ices

CPC

7Fin

ancia

land

rela

ted

serv

ices;

realesta

teserv

ices;and

renta

l

and

leasin

gserv

ices

CPC

7Fin

ancia

land

rela

ted

serv

ices;re

alesta

te

serv

ices;and

renta

l

and

leasin

gserv

ices

CPC

7Fin

ancia

land

rela

ted

serv

ices;re

alesta

te

serv

ices;and

renta

l

and

leasin

gserv

ices

CPC

8Busin

ess

and

pro

duction

serv

ices

CPC

8Busin

ess

and

pro

duction

serv

ices

CPC

8Busin

ess

and

pro

duction

serv

ices

CPC

8Busin

ess

and

pro

duction

serv

ices

CPC

8Busin

ess

and

pro

duction

serv

ices

CPC

9C

om

munity,socia

land

pers

onalserv

ices

CPC

9C

om

munity,socia

l

and

pers

onalserv

ices

CPC

9C

om

munity,socia

l

and

pers

onalserv

ices

CPC

9C

om

munity,socia

l

and

pers

onalserv

ices

CPC

9C

om

munity,socia

l

and

pers

onalserv

ices

D.8

Adju

stm

entfo

rth

echange

in

the

netequity

ofhousehold

s

S.1

4in

pensio

nfu

nds

B.8

Savin

g,

Tota

leconom

yS

.1

1N

on-p

rofit

instit

utio

ns

serv

ing

household

s.

2P.3

13

Fin

ancia

linte

rmedia

tion

serv

ices

implic

itly

measure

d(F

ISIM

)are

mark

etse

rvic

es

supplie

dto

house

hold

sby

financia

linst

itutio

ns

and

thus

are

inclu

ded

alo

ng

with

the

moneta

ryco

nsu

mptio

nexp

enditu

reofhouse

hold

s.FIS

IMare

insco

pe

for

inflatio

nortransa

ctio

nC

PIs

,fo

rexa

mple

.They

are

separa

tely

dis

tinguis

hed

from

non-F

ISIM

moneta

ryexpenditu

res

here

beca

use

they

require

implic

itm

easure

mentc

om

paring

am

ark

etin

tere

stra

tew

itha

refe

rence

rate

.O

therm

oneta

ryexp

enditu

res

are

measu

red

atl

easti

nprinci

ple

by

direct

observ

atio

n.

of the typical CPI, which comprises the first and secondcolumns and is labelled CPI reference aggregate #1. Thisaggregate corresponds with the practice of most, but notall, countries and comprises, as shown in Table 14.5, themonetary (non-imputed) individual consumption expen-diture of the household sector (P.311) plus the implicitrent paid by homeowners on their own residences(P.313). Box 14.3 contains further discussion on housingand durables in the CPI consumption expenditureaggregate.

Capital formation14.46 Capital formation comprises: the accumula-

tion of fixed tangible and intangible assets, such asequipment, structures and software; changes in inven-tories and work in progress; and acquisitions less dis-posals of valuables, such as works of art. These itemsare accounted for in the SNA capital account, whichappears, with minor resorting, essentially as in Table14.6 for each institutional unit. Net lending (+)/netborrowing (�) is the balancing item of the capitalaccount, making the uses on the left, comprising netacquisitions of stocks of various tangible and intangibleitems, add up to the resources on the right, comprisingthe sources of income financing them. From our earlierdiscussion on institutional units and establishments,it would be easy to conclude that the smallest eco-

nomic unit to which the capital account can apply isthe institutional unit. It was asserted earlier that onlyinstitutional units maintain balance sheets and can mon-itor the stock variables that are the focus of this account.Nevertheless, the physical capital assets forwhich changesare tracked in the capital account can and should becompiled, if possible, at the establishment/LKAU. Suchdata are particularly useful for productivity analysis,even though complete capital accounts cannot be com-piled at the establishment level.

14.47 Product detail in the capital account. As withthe other goods and services-related accounts in the SNA1993, the capital account’s goods and services items,designated by the codes P.5 with extensions, can beexpanded by product type. The account therefore can berearranged to show details of goods and services as inTable 14.7, which, as Table 14.6, may pertain to aninstitutional unit, an institutional sector aggregate, or thetotal economy. For an institutional unit, Table 14.6contains the core set of items in the report form of thetypical capital formation survey for the nationalaccounts. Our focus is on the CPI here, and thus onthe version of the form that typically would be part ofthe package a respondent would fill out in a house-hold expenditure survey. In addition to the CentralProduct Classification (CPC), version 1.0 shown here,the SNA 1993, Annex V contains a Non-financial assets

Table 14.6 Capital accountItems in bold refer to flows of goods and services

Institutional unit ID: uuuuuuuu Institutional sector: S.nnnnn

Uses Resources

P.5 Gross capital formation, of which: B.10.1 Changes in net worth due tosaving and capital transfers,of which:

P.51 Gross fixed capital formationP.511 Acquisitions less disposals of tangible fixed assetsP.5111 Acquisitions of new tangible fixed assetsP.5112 Acquisitions of existing tangible fixed assetsP.5113 Disposals of existing tangible fixed assets

P.512 Acquisitions less disposals of intangible fixed assetsP.5121 Acquisitions of new intangible fixed assetsP.5122 Acquisitions of existing intangible fixed assetsP.5123 Disposals of existing intangible fixed assets

P.513 Additions to the value of non-produced non-financial assetsP.5131 Major improvements to non-produced non-financial assetsP.5132 Costs of ownership transfer on non-produced non-financial assets

P.52 Change in inventoriesP.53 Acquisitions less disposals of valuables

B.8n Saving, netB.8 Saving (gross, from use of

income account)- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -K.1 Consumption of fixed capital (�) K.1 Consumption of fixed capital (�)- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -K.2 Acquisitions less disposals of non-produced non-financial assets

K.21 Acquisitions less disposals of land and other tangible non-produced assetsK.22 Acquisitions less disposals of intangible non-produced assets

D.9 Capital transfers receivable (+)D.91 Investment grantsD.92 Other capital transfers receivable

D.9 Capital transfers payable (�)D.91 Capital taxes payableD.91 Other capital transfers payable

B.9 Net lending ( +)/net borrowing (�)

THE SYSTEM OF PRICE STATISTICS

247

Table

14.7

Capitalaccountw

ith

pro

ductdeta

il

SN

A1993

item

sin

bold

refe

rto

flow

sin

goods

and

serv

ices

Institu

tionalunit

ID:

uuuuuuuu

Institu

tionalsecto

rcode:S

.nnnnn

B.1

0.1

Change

innet

wort

h

resultin

gfr

om

savin

gand

capital

transfe

rs,

with

uses

com

prisin

g

P.51

Grossfixedcapitalform

ation

P.52

Changein

inventories1

P.53

Acquisitions

lessdisposals

of

valuables2

P.5

Grosscapitalform

ation

P.511

Acquisitionsless

disposals

oftangible

fixedassets,

of

wh

ich

:3

P.512

Acquisitionsless

disposals

ofintangible

fixedassets,

of

wh

ich

:4

P.513

Additionsto

the

valueofnon-producednon-

financialassets,

of

wh

ich

:5

CPC

0Agriculture

,fo

restry

and

fishery

pro

ducts

CPC

0Agriculture

,fo

restry

and

fishery

pro

ducts

CPC

0Agriculture

,fo

restry

and

fishery

pro

ducts

CPC

0Agriculture

,fo

restry

and

fishery

pro

ducts

CPC

1O

res

and

min

era

l;

ele

ctric

ity,gas,

and

wate

r

CPC

1O

res

and

min

era

l;

ele

ctric

ity,gas,

and

wate

r

CPC

1O

res

and

min

era

l;

ele

ctric

ity,gas,

and

wate

r

CPC

2Food

pro

ducts

,

bevera

ges

and

tobacco;te

xtile

s,

appare

land

leath

er

pro

ducts

CPC

2Food

pro

ducts

,

bevera

ges

and

tobacco;te

xtile

s,

appare

land

leath

er

pro

ducts

CPC

2Food

pro

ducts

,bevera

ges

and

tobacco;te

xtile

s,

appare

land

leath

er

pro

ducts

CPC

3O

ther

transportable

goods,exceptm

eta

l

pro

ducts

,m

achin

ery

and

equip

ment

CPC

3O

ther

transportable

goods,exceptm

eta

l

pro

ducts

,m

achin

ery

and

equip

ment

CPC

3O

thertransportable

goods,exceptm

eta

l

pro

ducts

,m

achin

ery

and

equip

ment

CPC

4M

eta

lpro

ducts

,

machin

ery

and

equip

ment

CPC

4M

eta

lpro

ducts

,

machin

ery

and

equip

ment

CPC

4M

eta

lpro

ducts

,

machin

ery

and

equip

ment

CPC

4M

eta

lpro

ducts

,

machin

ery

and

equip

ment

CPC

5In

tangib

leassets

;

land;constructions;

construction

serv

ices

CPC

5In

tangib

leassets

;

land;constructions;

construction

serv

ices

CPC

5In

tangib

leassets

;

land;constructions;

construction

serv

ices

CPC

5In

tangib

leassets

;

land;constructions;

construction

serv

ices

CPC

5In

tangib

leassets

;

land;constructions;

construction

serv

ices

Of

whic

h:

P.5

11a

Resid

ential

str

uctu

res,

Household

secto

rS

.14

P.5

11b

Oth

er

capital

form

ation

inC

PC

5

K.1

Consum

ption

offixed

capital

K.2

Acquis

itio

ns

less

dis

posals

ofnon-p

roduced

non-fi

nancia

lassets

B.9

Net

borr

ow

ing(�

)/

net

lendin

g(+

)

1S

NA

1993

assetcode

AN

.12

Invento

ries.E

xclu

des

inta

ngib

leassets

,la

nd,and

constructions.

2S

NA

1993

assetcode

AN

.13

Valu

able

s.E

xclu

des

inta

ngib

leassets

,la

nd,constructions,and

construction

serv

ices.

3S

NA

1993

assetcode

AN

.111

Tangib

lefixed

assets

.E

xclu

des

inta

ngib

leassets

,la

nd,and

construction

serv

ices.

4S

NA

1993

assetcode

AN

.112

Inta

ngib

lefixed

assets

.E

xclu

des

land,constructio

ns,and

construction

serv

ices.

5S

NA

1993

assetcode

AN

.2N

on-p

roduced

assets

.E

xclu

des

inta

ngib

leassets

,

constructions,and

constructio

nserv

ices

classification identifying the specific tangible, intangible,produced, andnon-producedfixedassets, aswell as inven-tory and valuables items, recognized by the SNA 1993.14.48 The expenditure aggregate of the CPI and the

capital account. The CPI may be defined to include thehousehold sector’s final expenditure not only for con-sumption but also for capital formation. This brings intothe CPI expenditure aggregate the purchase of newresidential structures or expenditure on major improve-ments to existing residential structures. Consumer priceindex expenditure aggregate #2 is defined as the monetaryindividual consumption expenditure of households P.311in Table 14.5, which excludes all imputed expenditure,plus household expenditure on residential fixed capitalformation shown as item P.511a Residential structures,Household sector S.14 (shown in a box in Table 14.7).

External trade14.49 The external account of goods and services is

shown in Table 14.8. It contains the transactions of thenon-resident institutional units sector – S.2 Rest of theworld – with the five types of resident units taken togetherand determines the trade deficit (B.11) as imports(resources to rest of the world S.2) less exports (use orresources by the rest of the world). The external goods andservices account generally is taken from the balance ofpayments, which uses adjusted merchandise trade infor-mation from the customs services for goods P.61 andP.71, and assembles services data on P.62 and P.72from various sources. For further details, see Interna-tional Monetary Fund: Balance of payments manual (fifthedition, 1993). Although the account of external goodsand services is shown as an aggregate of the externaltransactions of all resident institutional units by the SNA1993, it may be possible to disaggregate it to distinguishthe external goods and services expenditures of institu-tional sectors, hence the institutional sector designationS.1.nnnn at the top of Table 14.8 to include this possibility.Our principal interest would be in the household sectorS.14 and its subsectors S.14nn, as these would relate tothe CPI.14.50 Product detail in the external account of goods

and services. As with the other accounts, the externalgoods and services account can be expanded to showproduct detail, as in Table 14.9. Regarding Table 14.9,the SNA 1993 states (SNA 1993, paragraph 15.68) thatimported goods should be valued at cost-insurance-freight (cif ) at the level of detailed products. On the otherhand, the SNA 1993 requires that, in total, imports ofgoods be valued free-on-board (fob) at the border of theexporting country, thus excluding insurance and trans-port in a single adjustment to total imports of goods cif(SNA 1993 paragraphs 14.36–14.41). That part offreight services on imports provided by non-residents isincluded in imports of transport services, and that partof insurance services provided on imports by non-resi-dents is added to imports of insurance services. Trans-port and insurance services provided by residents onimports are included in exports of transport and insur-ance services. This rather roundabout approach is takento imports by product because, as a practical matter, itmay be difficult to obtain insurance and freight charges

on imports from customs administrative data systems atthe product level of detail (see SNA 1993, paragraphs14.40–14.41). Recent developments in computerizedcustoms documentation have made the itemization ofinsurance and freight more straightforward, and theSNA 1993 does also allow for the possibility of deter-mining imports by product at their fob values, consistentwith the aggregate valuation of imports. If trade data arecollected by a survey of resident institutional units, thecore elements of the report form for such a survey wouldbe as given in Table 14.9.

14.51 The export and import price indices and theexternal account of goods and services. From the point ofview of the residents of an economic territory, exportsare a supply of goods and services to non-residents. TheSNA, however, records exports from the non-resident’spoint of view, as a non-resident use of goods and servicessupplied by residents. Accordingly, the relevant valua-tion principle for exports determining the behaviour ofthe non-resident user is the purchasers’ price. The SNAtakes the purchasers’ price to the non-resident user to bethe fob price at the frontier of the resident supplier’seconomic territory or country.

14.52 From the resident’s point of view, imports area use of goods and services supplied by non-residents.The SNA, however, records international trade from thenon-resident’s point of view, as the supply of goods andservices to residents by non-residents. Accordingly, therelevant valuation principle for imports determining thebehaviour of the non-resident supplier is the basic price.The SNA takes the basic price to the non-resident sup-plier to be the fob price at the frontier of the non-residentsupplier’s country in the rest of the world.

The supply and use table14.53 Arraying elements of resources and uses from

the production account, use of income account, capitalaccount, and external accounts of goods and services in aparticular configuration, we can derive a format for theproduction portion of an analytical presentation of thedata called a supply and use table (SUT). An SUT isshown in Table 14.10. It arrays various accounts relevantto monitoring developments in production and con-sumption within a country according to the supply anduses of goods and services.

14.54 In terms of the SNA 1993 codes, the supply ofgoods and services comes from:

� resident establishments (arranged in industries) in theform of domestic output (P.1), given by Y in equa-tions (14.1) and (14.2);

� the rest of the world as imports (P.7), given by M inequations (14.1) and (14.2), adjusted for trade andtransport margins and taxes less subsidies on products(D.21–D.31), given by T in equations (14.1) and(14.2);

and the uses of goods and services are for:

� current inputs into production by resident producers(arranged in industries) in the form of intermediateconsumption (P.2), given by Z in equations (14.1) and(14.2);

THE SYSTEM OF PRICE STATISTICS

249

Table 14.9 External account of goods and services with product detail

Resident institutional units classified into sectors S.1.nnnn with non-resident institutional units S.2;SNA 1993 goods and services items shown in bold

Uses Resources

P.6 Exports of goods and services P.7 Imports of goods and servicesExport price index uses aggregate Import price index supply aggregateP.61 Exports of goods P.71 Imports of goods

At fob values At fob values, of which:At cif values:1

CPC 0 Agriculture, forestry and fishery productsCPC 0 Agriculture, forestry and fishery productsCPC 1 Ores and mineral; electricity, gas, and waterCPC 1 Ores and mineral; electricity, gas, and waterCPC 2 Food products, beverages and tobacco;

textiles, apparel and leather productsCPC 2 Food products, beverages and tobacco;

textiles, apparel and leather productsCPC 3 Other transportable goods, except metal

products, machinery and equipmentCPC 3 Other transportable goods, except metal

products, machinery and equipmentCPC 4 Metal products, machinery and equipmentCPC 4 Metal products, machinery and equipment

Less: Adjustment to total imports of goods cif for insuranceand freight provided by both residents and non-residentsfor delivery to the first resident recipient.

P.62 Exports of services P.72 Imports of servicesCPC 5 Intangible assets; land; constructions;

construction services2CPC 5 Intangible assets; land; constructions;

construction services2

CPC 6 Distributive trade services; lodging;food and beverage serving services;transport services; and utilitiesdistribution services, of which:

CPC 6 Distributive trade services; lodging;food and beverage serving services;transport services; and utilitiesdistribution services, of which:

� Distributive trade services; lodging; foodand beverage serving services; transportservices; and utilities distribution services;except transport services on importsand exports rendered by residents

� Distributive trade services; lodging; foodand beverage serving services; transportservices; and utilities distribution services;except transport services on imports renderedby non-residents

� Transport services on imports andexports rendered by residents

� Transport services on imports andexports rendered by non-residents

CPC 7 Financial and related services; real estateservices; and rental and leasingservices, of which:

CPC 7 Financial and related services; real estateservices; and rental and leasing services,of which:

� Financial and related services; real estateservices; and rental and leasing services;except insurance services on importsrendered by residents

� Financial and related services; real estateservices; and rental and leasing services;except insurance services on imports renderedby non-residents

� Insurance services on imports renderedby residents

� Insurance services on imports rendered bynon-residents

CPC 8 Business and production services CPC 8 Business and production servicesCPC 9 Community, social and personal services CPC 9 Community, social and personal services

B.11 External balance of goods and services

1The SNA 1993 values imports fob, but it allows for the fact that while fob valuation by product would be consistent and preferred, compiling such data may beproblematic at the product level of detail. Imports of goods cif by product may be all that are available because the insurance and freight data are often notseparately compiled by product in customs systems (see SNA 1993, paragraph 15.68). Totals for these data may be obtained instead from resident and non-resident shippers in the process of compiling the balance of payments. Insurance and freight services provided by residents on imports are a services export.Regarding goods and services valuations in the import price and volume indices, see MPI in Tables 14.12 and 14.15, where it is explained that both fob andpurchasers’ price valuations are important in constructing the MPI as a deflator for imports fob. Imports at purchasers’ prices would be imports cif plus importtariffs, as well as domestic insurance and freight for delivery to the first domestic owner. 2Construction services only.

Table 14.8 External account of goods and services

Resident institutional units classified into sectors S.1.nnnn with non-resident institutional units S.2;SNA 1993 goods and services items shown in bold

Uses Resources

P.6 Exports of goods and services P.7 Imports of goods and servicesP.61 Exports of goods P.71 Imports of goodsP.62 Exports of services P.72 Imports of services

B.11 External balance of goods and services

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

250

Table

14.1

0The

supply

and

use

table

(SU

T)

Productionaccount:

double

outlin

es

and

no

shadin

g;Useofincomeaccount:

sin

gle

outlin

es

and

no

shadin

g;Capitalaccount:

dia

gonalshadin

g;Externalaccountofgoodsandservices:

vertic

alshadin

g

251

� final domestic consumption, including individualconsumption by resident households, resident non-profit institutions serving households (NPISHs), andthe government (P.31), and collective consumption bythe government (P.32), given by, respectively, C and Gin equations (14.1) and (14.2);

� capital formation by resident enterprises (P.5) (com-prising fixed capital formation (P.51), inventorychange (P.52), and acquisitions less disposals of valu-ables (P.53)), given by I in equations (14.1) and (14.2);

� export (P.6) and use by the rest of the world, given byX in equations (14.1) and (14.2).

14.55 Trade and transport margins do not appear inthe standard sequence of accounts in the SNA 1993because these accounts are not shown with productdetail. Although these margins are non-zero for indivi-dual products, they add up to zero in total because theamount added to the domestic supply of goods comesfrom the domestic supply of distribution, insurance andtransport services. Margins are thus shown in Table14.10 separately for margins on domestic production andimports (cif/fob adjustment), because the SUT displaysproduct detail down the columns. In the aggregate, ofcourse, these adjustments for trade and transport mar-gins on domestic production and the cif/fob adjustmentfor imports cancel each other out.

14.56 The SUT is primarily a matrix of flows ofgoods and services designed to highlight the relationshipbetween production and consumption, not betweeninstitutional units per se. For example, households mayundertake production in unincorporated enterprises forwhich activity appears in the production for own finaluse part of the SUT, but also consume goods and ser-vices, as represented in individual consumption. Thecurrent production transactions of the establishments ofall institutional units are grouped together and sum-marized in one part of the SUT, and the remainingtransactions are summarized and organized in anotherpart. Each institutional sector, including households(S.14), has its own SUT in principle. The SUT for thetotal economy (S.1) is the cell-by-cell sum of the insti-tutional sector SUTs.

The consumer price indexamong major price indices

14.57 It is instructive at this point to associate theSUT with the component aggregates and matrices of thefour major, headline price indices that are compiled bymost countries. In so doing, we form a more preciseimpression of the central purpose of the major priceindices in the overall economic statistical system repre-sented by the SNA 1993. The four main price indicesand their associated national accounts aggregates andmatrices in the SUT are:

– producer price index (PPI): output of resident pro-ducers (P.1);

– consumer price index (CPI): final consumption ofhouseholds (P.31) for CPI reference aggregate #1, plus

gross fixed capital formation of households (P.51) forCPI reference aggregate #2;

– export price index (XPI): exports (P.6);

– import price index (MPI): imports (P.7).

14.58 The location and coverage of these major priceindicators as they directly apply to goods and servicesvalue aggregates in the national accounts are shown dia-grammatically in Table 14.11. Chapter 15 characterizes aprice index as a function of price relatives and weights,noting that, other than the formula for the index itself, therequisite features of the relatives and weights would bedetermined by the value aggregate. These factors are:

– what items to include in the index;

– how to determine the item prices;

– what transactions that involve these items to includein the index;

– from what source to draw the weights used in theselected index formula.

Based on our survey of the goods and services accountsof the SNA 1993 culminating in the SUT, these parti-culars for each of the four major indices can summarizedas in Table 14.12.

Scope of the expenditure aggregatesof the consumer price index

14.59 As noted in paragraphs 14.6 and 14.7, thereare two principal expenditure sub-aggregates of the totalfinal expenditure of the households (S.14) institutionalsector employed in most national CPIs that we can nowsee are transparently linked to the SNA:

� CPI reference aggregate #1, comprising the consump-tion items:

P.311 Monetary consumption expenditure (Table14.5)P.313 Financial intermediation services implicitlymeasured (FISIM) (Table 14.5)P.312 Imputed expenditure on owner-occupied hous-ing services (Table 14.5)

� CPI reference aggregate #2, comprising the consump-tion and capital formation items:

P.311 Monetary consumption expenditure (Table14.5)P.313 Financial intermediation services implicitlymeasured (FISIM) (Table 14.5)P.511a Gross fixed capital formation in residentialstructures (Table 14.9)14.60 Proponents of CPI reference aggregate #1

generally take a consumption or cost of living view of theCPI, seeing household welfare as determined by the flowof goods and services, including the services of residentialstructures that are owned wholly or in part by theoccupants, that households consume. On this view,households’ fixed capital formation, which is effectivelylimited to the purchase of residences for own use, is abusiness-related activity of unincorporated enterprisesthat households own and thus not in the scope of theCPI. The customary version of aggregate #1 excludesnon-housing consumption from own production P.314.Although compensation in kind in the form of benefits

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

252

Table

14.1

1Location

and

covera

ge

ofm

ajo

rprice

indic

es:C

olu

mns

inth

esupply

and

use

table

The

eff

ective

covera

ge

of

the

majo

rin

dic

es

isshow

nby

shaded

are

as

To

tal

su

pp

ly

SN

A1993

transaction

P.1

Outp

ut,

of

whic

h,

esta

blis

hm

ents

pro

ducin

gprincip

ally

Tra

nsport

and

dis

trib

ution

marg

inadju

stm

ent

Taxes

less

subsid

ies

on

dom

estic

pro

ducts

P.7

Imports,fo

bC

if/fob

adju

stm

ent

Taxes

less

subsid

ies

on

imports

P.1

1M

ark

etoutp

ut

P.1

2O

utp

utfo

row

nfinaluse

P.1

3O

thernon-

mark

etoutp

ut

Imports

atpurc

hasers

’prices

PPIreference

aggregate

MPIreference

aggregate

Resourc

es

by

pro

duct:

Goods

Pro

duct�

Industr

yP

roduct�

Industr

yP

roduct�

Industr

yP

roduct�

1 +P

roduct�

1P

roduct�

1P

roduct�

1P

roduct�

1

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

---

Resourc

es

by

pro

duct:

Serv

ices

�

Tota

lre

sourc

es

0

Fin

al

uses

SN

A1993

transaction

P.3

1In

div

idualconsum

ption

P.3

2C

olle

ctive

consum

ption

P.5

Gro

ss

capitalfo

rmation

P.6

Exports,fo

b

SN

A1993

institu

tionalsecto

rH

ousehold

sS.1

4G

overn

ment

S.1

3

NPIS

Hs1

S.1

5

Govern

ment

S.1

3H

ousehold

sS.1

4All

institu

tional

secto

rs,

except

household

s

All

institu

tionalsecto

rs

Deta

iled

expenditu

recate

gories

P.3

11

Moneta

ryconsum

ption

P.3

13

FIS

IM2

P.3

12

Impute

dre

ntofow

ner-

occupants

P.3

14

Oth

er

impute

din

div

idual

consum

ption

expenditu

re

D.3

6Socia

ltransfe

rsin

kin

d

P.5

1G

ross

fixed

capital

form

ation

P.5

2In

vento

ries

P.5

3valu

able

s

Outp

ut(a

tbasic

prices)

sold

tonon-

resid

ents

Taxes

less

subsid

ies

on

export

pro

ducts

,transport

toin

tern

ational

ship

mentpoin

t

P.5

11a

Resid

ential

structu

res

for

ow

nuse

Oth

erfixed

capital

form

ation

CPIreference

aggregate

#1

CPIreference

aggregate

#2

XPIreference

aggregate

Expenditure

sby

pro

duct:

Goods

Pro

duct�

1P

roduct�

1P

roduct�

1P

roduct�

1P

roduct�

1P

roduct�

1P

roduct�

1P

roduct�

1P

roduct�

1P

roduct�

1P

roduct�

1

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

----

Expenditure

sby

pro

duct:

Serv

ices

Tota

lexpenditu

re

1N

on-p

rofitin

stitu

tions

serv

ing

household

s.

2Fin

ancia

lin

term

edia

tion

serv

ices

implic

itly

measure

d.

Table 14.12 Definition of scope, price relatives, coverage and weights for major price indices

Index Items to include Price determination Transactions coverage Sources of weights

PPI All types of domesticallyproduced or processedgoods and services thatare valued at marketprices

Basic prices, determined forgoods as the date whenavailable for sale (available forchange of ownership) orservice price whenservice rendered

Output of residententerprises, comprisingsales plus change infinished goods inventoriesfor goods, and salesfor services

The product by industrymatrices of Marketoutput P.11 and Outputfor own final use P.12in the expanded Industryproduction account andthe supply and usetable (SUT)

CPI Reference expenditureaggregate #1:

Purchasers’ prices,determined for goodson the change of ownershipdate and for serviceswhen used, including taxeson products, excludingsubsidies on products,and including transport anddistribution margins

Reference expenditureaggregate #1:

Reference expenditureaggregate #1:

All types of goods andservices purchasedexplicitly or implicitly byhouseholds for individualconsumption

Consumption expendituresof the Households sectorS.13 of institutional units,excluding consumptionfrom own production exceptfor imputed expendituresfor rental of owner-occupieddwellings

The product columnof the CPI consumptionsub-aggregate ofIndividual consumptionP.31 of the Householdsector S.13 in theexpanded Use ofIncome account andin the SUT

Reference expenditureaggregate #2 :

Reference expenditureaggregate #2 :

Reference expenditureaggregate #2 :

All types of goods andservices purchased explicitlyby households forindividual consumption,plus all types of goods andservices purchased explicitly byhouseholds for residential capitalformation

Reference expenditureaggregate #1, lessimputed expendituresfor rental of owner-occupied dwellings,plus net acquisition of ormajor improvements inresidential housing

The product columnof the monetary consumptionsub-aggregate of individualconsumption P.31 of theHousehold sector S.13 inthe expanded Use ofincome account plusthe product columnof acquisitions lessdisposals of fixed assetsP.511 for residentialhousing

XPI All types of transportable goodsand services purchased bynon-residents from residents.Goods exported without changeof ownership for significantprocessing by non-residentsand subsequent re-importare included

Purchasers’ prices at thenational frontier of the exportingcountry (fob), including exporttaxes and excluding exportsubsidies, and including transportand distribution margins fromthe production location tothe national frontier

All transportable goodsand services produced orprocessed by residents andpurchased by non-residentsexcept goods in transit orgoods exported andminimally processed bynon-residentsfor re-import

The product column ofExports P.6 in the expandedExternal account of goodsand services and the SUT

MPI All types of transportablegoods and servicespurchased by residents fromnon-residents. Goods importedwithout change of ownership forsignificant processing by residentsand subsequent re-export areincluded

Basic prices at thenational frontier of the exportingcountry (fob), excluding importtaxes and including importsubsidies, and excludingtransport and distributionmargins from the productionlocation to the national frontier1

All transportable goodsand services producedor processed by non-residents and purchasedby residents except goodsin transit or goods importedand minimally processedby residents for re-export

The product columnof Imports P.7 in theexpanded External accountof goods and servicesand the SUT

PPI = producer price index; CPI = consumer price index; XPI = export price index; MPI = import price index.

1In defining the import price index, however, the price index maker would, in fact, first consider an economic input price index valuing imported goods and services atthe purchasers’ price payable by their first resident owner. The import price index would be obtained by adjusting (multiplying) the import purchasers’ price index by a‘‘markdown’’ index tracking the movement in the ratio of imports fob to imports at purchasers’ prices. This is required for it to be properly matched in valuation withimports fob and yield the conceptually correct import volume index when used as an imports fob deflator.

254

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

provided by the employer is an important part of thisitem, households often are only vaguely aware of itsvalue, since the employer actually makes the payments tothe providers of the benefits. An argument neverthelesscould be made for including this item, as householdssometimes are able to exercise control over how this partof their compensation income is spent.14.61 Proponents of CPI reference aggregate #2

generally take a transactions or inflation view of the CPI,tailoring the index to measuring the rate of change in theprices of an expenditure aggregate broadly covering themonetary final expenditures that households make ongoods and services, including their capital formation inresidential structures via purchase of their own dwellingsand the major improvements they make to them.14.62 Both CPI concepts are useful. The cost of liv-

ing view provides a price index whose dual is the volumeof household consumption. The inflation view provides aprice index whose dual is the volume of households’ finalmonetary purchases, which represent the demand pres-sure they put on the markets in which they participate.Table 14.11 illustrates the coverage of both indices.

The consumer price index as a measureof inflation in market transactions14.63 Central banks take an interest in the major price

indices, particularly if they are implementing a monetarypolicy that targets inflation. Indeed, reference aggregate #2has been seen as a better measure of change in the prices ofactual transactions in goods and services than CPIs basedon reference aggregate #1, which gives substantial weightto the imputed rent of owner-occupied housing.14.64 Both reference aggregates for the CPI are an

important component of total final expenditure andGDP in virtually all countries, but the total value oftransactions in goods and services also includes inter-mediate consumption, so as an inflation index for totalgoods and services transactions, the CPI’s coverage israther limited under either definition #1 or #2 comparedwith, for example, the PPI, which covers, in principle,total output. Progress in extending the industry coverageof the PPI to cover all output-producing activities, ser-vices in particular, has, however, proceeded slowly owingto the technical difficulty of specifying service productsand measuring the associated prices. The combination ofthe PPI, covering output, and the import price indexprovides a price index for total market supply, and isseen by at least one monetary authority as a usefulinflation measure. Another central bank targets the totaldomestic supply price index, which is based on totalsupply less exports (that is, covering the aggregatecomprising output plus imports minus exports).14.65 The CPI’s purchasers’ price valuation princi-

ple also includes taxes less subsidies on products, whichmay not be desired in an inflation indicator for under-lying price change. Nevertheless, the CPI is the mostwidely available macroeconomic price statistic, and mayin many countries be the only available option for infla-tion measurement. Monetary authorities also may findthe CPI the most socially acceptable inflation targetprecisely because of its focus on households.

Treatment of cross-border shoppingin the consumer price index

14.66 Exports P.6 are not an expenditure of anyresident institutional unit and thus would not be thefocus of a price index covering its expenditure. By impli-cation, they would not appear in any CPI expenditureaggregate. Imports are, however, an expenditure of res-ident units and it is often relevant to consider the impor-tance of imports in the expenditure aggregates of suchunits. Inmany countries, imports acquired by householdsdirectly through cross-border shopping are a significantfraction of household consumption expenditure.

14.67 Of particular note here is that imported goodsP.71 and services P.72 in Table 14.8 for the householdsector would contain only the direct expenditures ofhouseholds on goods and services secured from non-residents, that is, in cross-border shopping. This shouldinclude purchases of transportable goods and services byhouseholds from non-resident suppliers through allmeans, including in person, by mail order and throughthe Internet. These expenditures in transactions withnon-residents are already covered in households’ indivi-dual consumption P.31 and capital formation P.5, so thepurpose of identifying imports P.7 in the context of theCPI is to identify the importance of transactions withnon-residents in the final expenditure aggregates ofhouseholds and that part of those aggregates covered bythe CPI expenditure aggregate.

14.68 Note that under both CPI reference aggregates#1 and #2 we would include expenditures on consump-tion goods and services provided by non-residents toresident households as the imported component ofIndividual consumption P.31. To assess the importanceof imports when considering CPI reference expenditureaggregate #2, we also would include households’ Fixedcapital formation P.51 expenditures on imported trans-portable goods such as building materials for residences,as well as residential construction services provided bynon-residents.

Other price indicators in thenational accounts

Price indices for total supply14.69 Consistent with our earlier discussion of the

coverage of the PPI, we define total market-valued out-put as the sum of market output P.11 and output for ownfinal use P.12. Total output P.1 is the sum of market-valued output and other non-market output P.13. Totalsupply at basic prices is the sum of output and importsP.7. Mark-up adjustments at the product level for tradeand transport margins on domestic production, insur-ance and freight on imports, and taxes D.21 less subsidiesD.31 on products would be added to total supply at basicprices to produce total supply at purchasers’ prices.

14.70 In decomposing total supply into price andvolume components, the total supply price index (SPI) atbasic prices can be seen to be a weighted mean of thetotal output price index YPI and the import price indexMPI. The YPI comprises in turn the PPI and an implicit

255

THE SYSTEM OF PRICE STATISTICS

deflator index (IDI) for other non-market output. Toobtain the price index for total supply at purchasers’prices, the SPI would be multiplied by an index of thetotal mark-up for trade, insurance, and transport mar-gins, and taxes net of subsidies on products. The marginsonly matter when developing supply price indices atpurchasers’ prices for individual products and productsub-aggregates. For all products, they cancel out, leavingonly taxes less subsides on products contributing to thetotal mark-up on total supply at basic prices. Totalsupply price indices at product levels of detail are usefulin compiling and reconciling discrepancies in supply anduse tables expressed in volume terms. In addition, theyare employed in producing industry price indices forintermediate consumption P.2, which are useful forcompiling gross domestic product (GDP) volume mea-sures from the production approach. Although princi-pally used as a compilation aid and in deflation of valueadded at basic prices via the double deflation approach(see paragraphs 14.71 and 14.73), supply price indicescould also serve as analytical indicators in their own rightbecause of their coverage of all goods and servicestransactions in the economy relating to production andexternal trade. As such, they may be useful as indicatorsfor the analysis and evaluation of economic policy,where broad coverage of transactions is required, forexample in formulating monetary policy.

Price indices for intermediateconsumption

14.71 In considering intermediate consumption priceindices (IPIs) for the total economy and for industry, theweights correspond to a column-wise reading of theintermediate consumption part of the SUT’s use matrix,which is derived from Table 14.2 and shown in Table14.10 as the region labelled P.2. Because the variousmargins on basic prices inherent in prevailing pur-chasers’ prices may vary from user industry to userindustry, the ideal sources for purchasers’ prices forintermediate consumption price indices would be enter-prise surveys. Unfortunately, such surveys are generally

burdensome and expensive. Instead, as noted in thediscussion above on price indices for total supply, theprice index of intermediate consumption by industry canbe derived from detailed product components of the SPI,which will result in indices of acceptable accuracy if thevariation in the total tax, subsidy, transport and dis-tribution margin is not too great from industry toindustry within product class. For the total economy, theprice index of intermediate consumption is obtained as aweighted average of industries’ intermediate input priceindices, where the weights are the share of each industry’sintermediate consumption in the total intermediateconsumption in the economy.

Price indices for final uses14.72 The price indices for final use comprise defla-

tors for individual consumption P.31, collective con-sumption P.32, gross fixed capital formation P.51,change in inventories P.52, acquisitions less disposals ofvaluables P.53 and exports P.6. Of the major priceindices discussed above, the CPI is the principal source ofdetailed (product level) information for P.31, while thePPI is a significant source of detailed information forP.51 and the principal source for the finished goodscomponent of P.52. When the CPI is defined on the basisof CPI reference expenditure aggregate #2, the CPIcould also be the source of data on capital formationin residential structures. The SPI may be the principalsource for the input inventories component of P.52 in theabsence of a detailed survey of the purchase price ofintermediate inputs, and the XPI is the deflator for P.6.The SPI can serve, as well, as a source of detailed productinformation for P.32, P.51 and P.53. We will designatethe deflator for total final uses as the final uses priceindex (FPI), which would be computed as a weightedmean (formula to be determined) of the componentindices just discussed.

Price indices for gross domestic product14.73 As noted above in the discussion of the

SPI and the intermediate consumption price index, the

Table 14.13 Generation of income account for establishment, institutional unit or institutional sector

SNA 1993 goods and services items shown in bold

Uses Resources

D.1 Compensation of employees B.1 Value added 1

D.11 Wages and salariesD.12 Employers’ social contributions

D.121 Employers’ actual social contributionsD.122 Employers’ imputed social contributions

D.2 Taxes on production and importsD.29 Other taxes on production2

D.3 Subsidies

D.39 Other subsidies on production (�)3

B.2 Operating surplus4

1From the production account. 2Taxes on production unrelated to products. 3Subsidies on production unrelated to products. 4Balancing item of the generation ofincome account.

256

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

GDP price index can be compiled in two ways, corre-sponding to the two goods and services methods ofcompiling GDP: the production approach and theexpenditure approach. Recall that the productionapproach derives from the definition of value addedimplicit in equation (14.2), as the difference betweenoutput P.1 (at basic prices) and intermediate con-sumption P.2 (at purchasers’ prices). The SNA 1993recommends the use of double deflation for valueadded, by which output at basic prices Y is deflatedby all the items YPI to obtain output volume, andintermediate purchases are deflated by an inter-mediate purchases price index to obtain intermediateinput volume. Real value added is then computed asthe difference between output volume and intermediateinput volume (see SNA 1993, Chapter XVI). Thisoperation is equivalent to deflating value added incurrent prices with a double deflation-type priceindex having a positive weight on the YPI and a nega-tive weight on the IPI. In the usual case just des-cribed, we have the value added deflator as a Paascheindex of the output price index YPI s,t and the inter-mediate input price index IPI s,t, where the weight onthe IPI s,t is

wtI=�P:2t

P:1t � P:2t

and the weight on the YPI s,t is 1� wtI . The corre-sponding volume index has the Laspeyres or ‘‘constantprice’’ form, which is equivalent to the double deflationmeasure of the volume of real value added divided bycurrent price value added in period s. The total valueadded at current basic prices divided by real value added,obtained via double deflation, yields the implicit deflatorfor value added at basic prices. Finally, the GDP deflatorat purchasers’ prices is the value added price index (atbasic prices for output and purchasers’ prices for inter-mediate input) multiplied by the index of the mark-up onvalue added of output taxes less output subsidies onproducts.14.74 Alternatively, the final expenditure deflator

FPI may be combined with the MPI using a doubledeflation-type approach. GDP volume is calculated fromexpenditure data by deflating imports P.7 by the MPI,and subtracting the result from the volume of final uses,calculated by deflating final uses by the FPI. The implicitGDP deflator would be the ratio of GDP at currentprices to GDP volume so calculated. The aggregate indexof GDP volume and the aggregate index of real valueadded should agree with one another, as should, byimplication, the implicit GDP deflator calculated fromthe two approaches.

Price indices for labour services14.75 Value added appears first in the production

account, calculated as the balancing item between out-put and intermediate consumption. This margin is usedto pay for, among other things, labour services. TheSNA 1993 provides for the income componentscomprising value added in the generation of income

account, shown in Table 14.13. The largest of theincome components itemized in this account is com-pensation of employees D.1, comprising wages andsalaries D.11 and employers’ social contributions D.12.D.1 represents a value aggregate for a flow of labourservices and is thus susceptible to decomposition intoprice and volume components. Table 14.14 shows thesame account expanded by type of labour service(occupation) for an establishment or industry. Theprice index for labour services (LPI ) measures devel-opments in total compensation, by occupation, withinindustry. The price of labour services in total compen-sation terms is of particular interest when comparedwith the GDP deflator, which indicates the relativepurchasing power of labour compensation in terms ofproduction for final consumption. This comparison isuseful in assessing cost-push pressures on output pricesand as an input into compiling measures of the pro-ductivity of labour. A second useful comparison isbetween the wages and salaries sub-index of the LPI andthe CPI. The ratio of the LPI to the CPI indicates thepurchasing power of wages in terms of consumptiongoods and services, and tracks the material welfare,particularly of the employees subsector S.143 of thehousehold institutional sector S.14 (see Box 14.1 onpage 238). In the LPI, the price of labour servicescomprises all the components of compensation ofemployees, including employers’ social contributions(benefits), as well as wages and salaries. The wages andsalaries sub-index of the LPI would be another exampleof a price index adjusted by a mark-up index. Ana-logously with the price index for total supply atpurchasers’ prices or for GDP by production inTable 14.10, the LPI would be adjusted in this case by a‘‘markdown index’’, deducting employers’ social con-tributions.

Framework for a system ofprice statistics for goodsand services

14.76 To summarize this overview of the main priceindicators and the national accounts, Table 14.15shows in tabular form the price indices needed for thevalue aggregates in the national accounts and theirrelation to the four main price indicators. Indices thatare functions of two other indices are shown with thenotation

f (I1; I2;w)

where f is an index formula, I1 and I2 are price indices,w is the weight of the second index, with the weightof the first index understood to be 1�w. For example,if f is the Laspeyres formula, then the output priceindex (YPI) would be calculated by making the fol-lowing substitutions: Ps;tL =YPI

s;t rs;t1 =PPIs;t, ws1=

1� wsD, rs;t2 =IDI

s;t, ws2=wsD. f could also be chosen as a

Paasche formula (with the same substitutions exceptfor change in the time superscript on the weightswt1=1�wtD and wt2= wtD), Fisher ideal formula, or otherindex formula.

257

THE SYSTEM OF PRICE STATISTICS

Table

14.1

4G

enera

tion

ofin

com

eaccountfo

resta

blis

hm

entand

industry

with

labourserv

ices

(occupational1)deta

il

SN

A1993

goods

and

serv

ices

item

sshow

nin

bold

Esta

blis

hm

ent

ID:

eeeeeeee

Activity/I

ndustr

ycode

(IS

IC):

aaaa

Institu

tionalunit

ID:

uuuuuuuu

Mark

et

sta

tus:P.1

nIn

stitu

tionalsecto

rcode:S.n

nnnn

Uses

Resourc

es

D.11

Wagesandsalaries

D.12

Employers’socialcontributions

D.1

Compensationofemployees

B.1

Valu

eadded

2

1:Legis

lato

rs,senio

roffi

cia

lsand

managers

1:Legis

lato

rs,senio

roffi

cia

lsand

managers

1:Legis

lato

rs,senio

roffi

cia

lsand

managers

2:P

rofe

ssio

nals

2:P

rofe

ssio

nals

2:P

rofe

ssio

nals

3:Technic

ians

and

associa

tepro

fessio

nals

3:Technic

ians

and

associa

tepro

fessio

nals

3:Technic

ians

and

associa

tepro

fessio

nals

4:C

lerk

s4:C

lerk

s4:C

lerk

s5:S

erv

ice

work

ers

and

shop

and

mark

etsale

sw

ork

ers

5:S

erv

ice

work

ers

and

shop

and

mark

etsale

sw

ork

ers

5:S

erv

ice

work

ers

and

shop

and

mark

etsale

sw

ork

ers

6:S

killed

agricultura

land

fishery

work

ers

6:S

killed

agricultura

land

fishery

work

ers

6:S

killed

agricultura

land

fishery

work

ers

7:C

raft

and

rela

ted

trades

work

ers

7:C

raft

and

rela

ted

trades

work

ers

7:C

raft

and

rela

ted

trades

work

ers

8:P

lantand

machin

eopera

tors

and

assem

ble

rs8:P

lantand

machin

eopera

tors

and

assem

ble

rs8:P

lantand

machin

eopera

tors

and

assem

ble

rs9:E

lem

enta

ryoccupations

9:E

lem

enta

ryoccupations

9:E

lem

enta

ryoccupations

0:A

rmed

forc

es

0:A

rmed

forc

es

0:A

rmed

forc

es

D.2

Taxes

on

pro

duction

and

imports

D.2

9O

ther

taxes

on

pro

duction

D.3

Subsid

ies

(�)

D.3

9O

ther

subsid

ies

on

pro

duction

B.2

Opera

ting

surp

lus

3

1Show

ing

majo

rgro

ups

ofIL

O:

Inte

rnational

Sta

ndard

Cla

ssifi

cation

of

Occupations

1988

(ISC

O-8

8)

(Geneva,1990).

2Fro

mth

epro

duction

account.

3Bala

ncin

gitem

ofth

egenera

tion

ofin

com

eaccount.

258

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

International comparisonsof expenditure on goodsand services14.77 The main price statistics discussed thus far

trace price developments of goods and services throughtime. Purchasing power parities (PPPs) compare pricelevels between different countries or geographical areasfor a given accounting period and are generally used toeliminate the effect of prices in different currency unitswhen comparing the levels of GDP between two coun-tries or areas. The price relatives in bilateral PPPs com-prise the ratios of the local currency prices of identicalgoods and services between the two countries or areas.The weights are proportional to the shares of these itemsin expenditure on GDPwithin the two countries or areas.The sources of price relatives are the same as those forthe final uses GDP deflator, and the weights are simplythe total final uses, net of imports fob, by product. Inorder to ensure that the PPP between area A and area Bis the reciprocal of the PPP between B and A, bilateral

PPPs need to be computed using symmetric index num-bers such as the Fisher.

14.78 A matrix of bilateral PPPs provides a meansof making not only direct bilateral comparisons, butalso bilateral comparisons between any two areas asthe product of a sequence of bilateral PPPs throughany set of intervening areas, beginning with the first areaand ending with the second. In order to ensure theconsistency of such multilateral comparisons – forexample, that a chain beginning with a given area andending with the same area produces a PPP of unity –bilateral PPPs are adjusted to produce a transitive set ofcomparisons.

14.79 The four main index series dealt with in thischapter are related to PPPs because the prices collectedfor the CPI, PPI, XPI and MPI, in addition to their usein these temporal indices and in the temporal GDP priceindex, can also be used in international comparisons ofexpenditures on consumption, capital formation andtrade. See Annex 4 on the International ComparisonProgram for further details on PPPs.

259

THE SYSTEM OF PRICE STATISTICS

Table

14.1

5A

fram

ew

ork

for

price

sta

tistics

SN

A1993

aggre

gate

SN

A1993

transaction

codes

1Valu

ation

and

needed

deta

ilS

NA

1993

sourc

eaccount

Price

index2

Derivation

from

oth

erprice

indic

es

Su

pp

ly

Market-valued

output

P.11+P.12

Basic

prices,productby

industry

Productionaccount

withindustryand

productdetail,total

economyS.1

Pro

du

cer

pri

ce

ind

ex

(PP

I)

Oth

er

non-m

ark

et

outp

ut3

P.1

3Basic

prices

(costofpro

duction),

pro

ductby

industry

Pro

duction

account

with

industry

and

pro

ductdeta

il,to

tal

econom

yS.1

Implic

itdeflato

rfo

roth

er

non-m

ark

et

outp

ut

(ID

I)

Derived

from

volu

me

indic

ato

r

Tota

loutp

ut

P.1

=P.1

1+

P.1

2+

P.1

3Basic

prices,

by

pro

duct

Pro

duction

account

with

industry

and

pro

ductdeta

il,to

tal

econom

yS.1

Outp

ut

price

index

(YP

I)Y

PI¼

f(P

PI,

IDI;

wm

),w

m¼

P:1

3

P:1

Imports

P.7

Basic

prices(goods

fobfrontierofexporting

country,including

thefreightand

insuranceon

importsprovided

bynon-residents),

byproduct

Externaltransactions

ingoodsand

servicesaccount

withproductdetail,

totaleconomyS.1

Imp

ort

pri

ce

ind

ex

(MP

I),

co

mp

risin

gan

imp

ort

pu

rch

asers

’p

rice

ind

ex

mu

ltip

lied

by

an

fob

/pu

rch

asers

’p

rice

mark

do

wn

ind

ex

Tota

lsupply

,basic

prices

P.1

+P.7

Basic

prices,by

pro

duct

Supply

and

use

table

,to

taleconom

yS.1

Supply

price

index

(SP

I)S

PI¼

f(M

PI,

YP

I;w

y),

wy¼

P:1

P:1

+P:7

Tota

ldom

estic

supply

P.1

+P.7�

P.6

Basic

prices,by

pro

duct

(P.1

and

P.7

);purc

hasers

’prices

(P.6

exports

fob,

see

‘‘Uses’’

entry

belo

w)

Supply

and

use

table

,to

taleconom

yS.1

Dom

estic

supply

price

index

(DS

PI)

SP

I¼

f(M

PI,

YP

I,X

PI;

wy,�

wx),

wy¼

P:1

P:1

+P:7�

P:6

,wx¼

P:6

P:1

+P:7�

P:6

Dom

estic

trade,

insura

nce,and

transport

marg

inadju

stm

ent

Basic

prices,fo

rserv

ices

pro

vid

ed

fortransport

and

dis

trib

ution

within

national

frontiers

,by

pro

duct

Supply

and

use

table

,to

tal

econom

yS.1

Supply

mark

-up

index

(SM

I)

SM

I¼

P:1

t +P:7

t +D:2

1t�

D:3

1t

P:1

t +P:7

t

P:1

s+P:7

s+D:2

1s�

D:3

1s

P:1

s+P:7

s

(in

the

aggre

gate

).Pro

duct-le

velto

tal

outp

utm

ark

-up

indic

es

would

als

oin

clu

de

trade

and

transport

marg

ins

inth

enum

era

tor

ofth

eabove

expre

ssio

n

Fre

ightand

insura

nce

on

imports

adju

stm

ent

Basic

prices

(for

serv

ices

pro

vid

ed

from

exporter

frontier

todom

estic

frontier,

regard

less

ofre

sid

ency

of

pro

vid

er),by

pro

duct

Supply

and

use

table

,to

taleconom

yS.1

Taxes

less

subsid

ies

on

pro

ducts

D.2

1�

D.3

1Payable

,by

pro

duct

Allo

cation

ofprim

ary

incom

eaccount,

genera

lgovern

mentsecto

rS.1

3

Tota

lsupply

,purc

hasers

’prices

P.1

1+

P.1

2+

P.7

+D

.21�

D.3

1Purc

hasers

’prices

SP

I�

SM

I

260

Uses

Inte

rmedia

teconsum

ption

P.2

Purc

hasers

’prices,

pro

ducts

by

industrie

sPro

duction

accountw

ith

pro

ductand

industry

deta

il,to

taleconom

yS.1

Inte

rmedia

teconsum

ption

price

index

(IP

I)

Usually

incorp

ora

tes

pro

duct-le

vel

info

rmation

from

the

tota

lsupply

price

index

atpurc

hasers

’prices

Indiv

idualconsum

ption

P.3

1Purc

hasers

’prices,

by

pro

duct

Use

ofin

com

eaccount

with

pro

ductdeta

il,to

tal

econom

yS.1

Household

consum

ption

price

index

(HP

I)

Incorp

ora

tes

the

CPI,

and

may

incorp

ora

tepro

duct-le

velin

form

ation

from

the

CPIand

PPI

regard

ing

goods

and

serv

ices

pro

duced

from

ow

nconsum

ption

and

pro

vid

ed

toin

div

iduals

by

NPIS

Hs

and

genera

lgovern

ment

Household

sectorS.14

P.31,exceptim

puted

consumptionand

consumptionfrom

productionfor

ownfinaluse,but

includingim

puted

rentofhome-owners

Purchasers’prices,

byproduct

Useofincomeaccount

withproductdetail,

household

sectorS.14,

withspecialsub-

classificationofP.31

Co

nsu

mer

pri

ce

ind

ex

(CP

I)an

do

ther

su

b-i

nd

ices

as

need

ed

Colle

ctive

consum

ption

P.3

2Purc

hasers

’prices,

by

pro

duct

Use

ofin

com

eaccount

with

pro

ductdeta

il,genera

lgovern

ment

secto

rS.1

3

Govern

ment

price

index

(GP

I)M

ay

incorp

ora

tepro

duct

indic

es

from

the

CP

Iand

PP

I

Gro

ss

fixed

capitalfo

rmation

P.5

1Purc

hasers

’prices,

by

pro

duct

Capitalaccountw

ith

pro

ductdeta

il,to

tal

econom

yS.1

Fix

ed

capital

form

ation

price

index

(KP

I)

May

incorp

ora

tepro

duct

indic

es

from

the

PP

I

Change

inin

vento

ries

P.5

2Purc

hasers

’prices,

by

pro

duct

Capitalaccountw

ith

pro

ductdeta

il,to

tal

econom

yS.1

Invento

ryprice

index

(NP

I)P

rice

index

of

invento

rysto

cks

Acquis

itio

ns

less

dis

posals

of

valu

able

s

P.5

3Purc

hasers

’prices,

by

pro

duct

Capitalaccount

with

pro

ductdeta

il,to

taleconom

yS.1

Valu

able

sprice

index

(VP

I)P

rice

index

of

valu

able

ssto

cks

Exports

P.6

Purchasers’prices

(fobdomesticfrontier),

byproduct

Externaltransactions

ingoodsand

servicesaccount

withproductdetail,

totaleconomyS.1

Exp

ort

pri

ce

ind

ex

(XP

I)

Tota

lfinal

uses

P.3

+P.5

+P.6

Purc

hasers

’prices,

by

pro

duct

Supply

and

use

table

,to

taleconom

yS.1

Tota

luses

price

index

(FP

I)F

PI¼

f(H

PI,

GP

I,K

PI,

NP

I,V

PI,

XP

I,~ ww

)w

here

4

~ ww¼

[wG,w

K,w

N,w

V,w

X]3

and

wG¼

P:3

2

P:3

+P:4

+P:5

+P:6

,

wK¼

P:5

1

P:3

+P:4

+P:5

+P:6

,

wN¼

P:5

2

P:3

+P:4

+P:5

+P:6

,

wV¼

P:5

3

P:3

+P:4

+P:5

+P:6:

wX¼

P:6

P:3

+P:4

+P:5

+P:6

261

Table

14.1

5A

fram

ew

ork

forprice

sta

tistics

(contd

.)

SN

A1993

aggre

gate

SN

A1993

transaction

codes

1Valu

ation

and

needed

deta

ilS

NA

1993

sourc

eaccount

Price

index

2D

erivation

from

oth

er

price

indic

es

Gro

ss

do

mesti

cp

rod

uct

Gro

ss

do

mesti

cp

rod

uct

GD

P=

P.3

+P.5

+P.6�

P.7

,or

By

pro

ductw

hen

assem

ble

dfrom

final

consum

ption

netof

imports

Supply

and

use

table

,to

tal

econom

yS.1

GD

Pdeflato

rG

DP

deflato

r¼

f(F

PI,

MP

I;w

M),

¼S

MI*�

f(S

PI,

IPI;

wI)

GD

P=

P.1�

P.2

+D

.21�

D.3

1w

here

5

By

industry

when

assem

ble

dfrom

valu

eadded

atbasic

prices,

with

industry

and

tota

lvalu

eadded

price

indic

es

adju

ste

dby

am

ark

-up

facto

rfo

rta

xes

netofsubsid

ies

on

pro

ducts

wM¼�

P:7

GD

P

wI¼�

P:2

4

GD

P

SM

I*¼

P:1

t�

P:2

t +D:2

1t�

D:3

1t

P:1

t�

P:2

t

P:1

s�

P:2

s+D:2

1s�

D:3

1s

P:1

s�

P:2

s

(in

the

aggre

gate

)In

dustry-levelvalu

eadded

mark

-up

indic

es

SM

I*w

ould

inclu

de

the

tota

ltrade

and

transport

marg

ins

on

outp

utin

the

num

era

tor

Com

pensation

of

em

plo

yees

D.1

By

industry

and

occupation

Genera

tion

ofin

com

eaccount,

tota

leconom

yS.1

Em

plo

ym

ent

cost

index

1P.1

1=

mark

etoutp

ut,

P.1

2=

outp

utfo

row

nfinaluse,D

.21

=ta

xes

on

pro

ducts

,and

D.3

1=

subsid

ies

on

pro

ducts

.2The

fourm

ajo

rprice

indic

es

are

show

nin

bold

.3This

cate

gory

com

prises

public

serv

ices

outp

utpro

vid

ed

free

ofcharg

eorateconom

ically

insig

nifi

cantprices

by

genera

lgovern

mentand

non-p

rofitin

stitu

tions

serv

ing

household

s(N

PIS

Hs).

This

outp

utis

valu

ed

atcostbecause

ithas

no

mark

etcom

para

tor.

Aprice

index

cannot

be

directly

constructe

dfo

rth

isaggre

gate

because

there

are

no

econom

ically

sig

nifi

cantprices

foroth

ernon-m

ark

etoutp

ut.

The

implic

itdeflato

rfo

roth

ernon-m

ark

etoutp

utP.1

3is

derived

by

div

idin

ga

directly

com

pile

dvolu

me

indic

ato

rin

toth

evalu

eofoth

er

non-m

ark

etoutp

ut.

4U

nlik

eour

oth

eraggre

gations

ofin

dic

es

thatin

volv

eth

ecom

bin

ation

oftw

ocom

ponentin

dic

es,w

eshow

the

FPIas

asim

ultaneous

aggre

gation

ofsix

price

indic

es

forth

ecom

ponents

offinaluses.Again

,fcan

be

any

ofth

ein

dic

es

introduced

inC

hapte

rs1

and

15,w

ith

the

weig

htofth

efirs

titem

,here

ofin

div

idualconsum

ption

P.3

1,dete

rmin

ed

as

one

min

us

the

restofth

ew

eig

hts

,and

the

price

rela

tives

giv

en

by

the

listofin

dex

arg

um

ents

.5The

negative

weig

htofth

esecond

index

arg

um

entofboth

ofth

ese

form

ula

efo

rG

DP

isan

indic

ation

thatth

ey

repre

senta

‘‘double

deflation-type’’

price

index

(see

SN

A1993,C

hapte

rXVI,

Section

E).

262

15BASIC INDEX NUMBER THEORY

IntroductionThe answer to the question what is the Mean of a given

set of magnitudes cannot in general be found, unless thereis given also the object for the sake of which a mean valueis required. There are as many kinds of average as thereare purposes; and we may almost say in the matter ofprices as many purposes as writers. Hence much vaincontroversy between persons who are literally at crosspurposes. (Edgeworth (1888, p. 347)).

15.1 The number of physically distinct goods andunique types of services that consumers can purchase isin the millions. On the business or production side ofthe economy, there are even more commodities that areactively traded. This is because firms not only producecommodities for final consumption, but they also produceexports and intermediate commodities that are demandedby other producers. Firms collectively also use millions ofimported goods and services, thousands of different typesof labour services and hundreds of thousands of specifictypes of capital. If we further distinguish physical com-modities by their geographical location or by the seasonor time of day that they are produced or consumed, thenthere are billions of commodities that are traded withineach year in any advanced economy. For many purposes,it is necessary to summarize this vast amount of price andquantity information into a much smaller set of numbers.The question that this chapter addresses is: how exactlyshould the microeconomic information involving possi-bly millions of prices and quantities be aggregated into asmaller number of price and quantity variables? This isthe basic problem of index numbers.15.2 It is possible to pose the index number problem

in the context of microeconomic theory; i.e., given thatwe wish to implement some economic model based onproducer or consumer theory, what is the ‘‘best’’ methodfor constructing a set of aggregates for the model? Whenconstructing aggregate prices or quantities, however,other points of view (that do not rely on economics) arepossible. Some of these alternative points of view areconsidered in this chapter and the next. Economicapproaches are pursued in Chapters 17 and 18.15.3 The index number problem can be framed as the

problem of decomposing the value of a well-defined setof transactions in a period of time into an aggregate priceterm times an aggregate quantity term. It turns out thatthis approach to the index number problem does not leadto any useful solutions. So, in paragraphs 15.7 to 15.17,the problem of decomposing a value ratio pertaining totwo periods of time into a component that measures theoverall change in prices between the two periods (this isthe price index) times a term that measures the overallchange in quantities between the two periods (this is the

quantity index) is considered. The simplest price index isa fixed basket type index; i.e., fixed amounts of the nquantities in the value aggregate are chosen and then thevalues of this fixed basket of quantities at the prices ofperiod 0 and at the prices of period 1 are calculated. Thefixed basket price index is simply the ratio of these twovalues where the prices vary but the quantities are heldfixed. Two natural choices for the fixed basket are thequantities transacted in the base period, period 0, or thequantities transacted in the current period, period 1.These two choices lead to the Laspeyres (1871) andPaasche (1874) price indices, respectively.

15.4 Unfortunately, the Paasche and Laspeyresmeasures of aggregate price change can differ, sometimessubstantially. Thus in paragraphs 15.18 to 15.32, takingan average of these two indices to come up with a singlemeasure of price change is considered. In paragraphs15.18 to 15.23, it is argued that the ‘‘best’’ average to takeis the geometric mean, which is Irving Fisher’s (1922)ideal price index. In paragraphs 15.24 to 15.32, instead ofaveraging the Paasche and Laspeyres measures of pricechange, taking an average of the two baskets is con-sidered. This fixed basket approach to index number the-ory leads to a price index advocated by Correa MoylanWalsh (1901; 1921a). Other fixed basket approaches are,however, also possible. Instead of choosing the basket ofperiod 0 or 1 (or an average of these two baskets), it ispossible to choose a basket that pertains to an entirelydifferent period, say period b. In fact, it is typical statis-tical agency practice to pick a basket that pertains to anentire year (or even two years) of transactions in a yearprior to period 0, which is usually a month. Indices ofthis type, where the weight reference period differs fromthe price reference period, were originally proposed byJoseph Lowe (1823), and indices of this type are studiedin paragraphs 15.64 to 15.84. Such indices are alsoevaluated from the axiomatic perspective in Chapter 16and from the economic perspective in Chapter 17.1

15.5 In paragraphs 15.65 to 15.75, another approachto the determination of the functional form or the formulafor the price index is considered. This approach is attrib-utable to the French economist Divisia (1926) and isbased on the assumption that price and quantity data areavailable as continuous functions of time. The theory ofdifferentiation is used in order to decompose the rate ofchange of a continuous time value aggregate into two

1Although indices of this type do not appear in Chapter 19, wheremost of the index number formulae exhibited in Chapters 15–18 areillustrated using an artificial data set, indices where the weight refer-ence period differs from the price reference period are illustratednumerically in Chapter 22, in which the problem of seasonal com-modities is discussed.

263

components that reflect aggregate price and quantitychange. Although the approach of Divisia offers someinsights,2 it does not offer much guidance to statisticalagencies in terms of leading to a definite choice of indexnumber formula.

15.6 In paragraphs 15.76 to 15.97, the advantages anddisadvantages of using a fixed base period in the bilateralindex number comparison are considered versus alwayscomparing the current period with the previous period,which is called the chain system. In the chain system, a linkis an index number comparison of one period with theprevious period. These links are multiplied together inorder to make comparisons over many periods.

The decomposition of valueaggregates into price andquantity components

The decomposition of value aggregatesand the product test

15.7 A price index is a measure or function whichsummarizes the change in the prices of many commod-ities from one situation 0 (a time period or place) toanother situation 1. More specifically, for most practicalpurposes, a price index can be regarded as a weightedmean of the change in the relative prices of the com-modities under consideration in the two situations. Todetermine a price index, it is necessary to know:

� which commodities or items to include in the index;� how to determine the item prices;

� which transactions that involve these items to includein the index;

� how to determine the weights and from which sourcesthese weights should be drawn;

� what formula or type of mean should be used toaverage the selected item relative prices.

All the above questions regarding the definition of a priceindex, except the last, can be answered by appealing tothe definition of the value aggregate to which the priceindex refers. A value aggregateV for a given collection ofitems and transactions is computed as:

V=Pni=1

piqi (15:1)

where pi represents the price of the ith item in nationalcurrency units, qi represents the corresponding quantitytransacted in the time period under consideration and thesubscript i identifies the ith elementary item in the groupof n items that make up the chosen value aggregate V.Included in this definition of a value aggregate is thespecification of the group of included commodities (whichitems to include) and of the economic agents engaging intransactions involving those commodities (which trans-actions to include), as well as principles of the valuationand time of recording that motivate the behaviour of theeconomic agents undertaking the transactions (determi-nation of prices). The included elementary items, their

valuation (the pi), the eligibility of the transactions and theitemweights (the qi) are all within the domain of definitionof the value aggregate. The precise determination of the piand qi is discussed in more detail elsewhere in this manual,in particular in Chapter 5.3

15.8 The value aggregateV defined by equation (15.1)refers to a certain set of transactions pertaining to a single(unspecified) time period. Now the same value aggregatefor two places or time periods, periods 0 and 1, is con-sidered. For the sake of convenience, period 0 is called thebase period and period 1 is called the current period andit is assumed that observations on the base periodprice and quantity vectors, p0 � [ p01, . . . , p0n ] and q

0 �[q01, . . . , q0n] respectively, have been collected.

4 The valueaggregates in the base and current periods are defined inthe obvious way as:

V 0 �Pni=1

p0i q0i ; V

1 �Pni=1

p1i q1i (15:2)

In the previous paragraph, a price index was defined as afunction or measure which summarizes the change in theprices of the n commodities in the value aggregate fromsituation 0 to situation 1. In this paragraph, a price indexP(p0, p1, q0, q1) along with the corresponding quantityindex (or volume index) Q(p0, p1, q0, q1) is defined to betwo functions of the 4n variables p0, p1, q0, q1 (thesevariables describe the prices and quantities pertaining tothe value aggregate for periods 0 and 1) where these twofunctions satisfy the following equation:5

V1=V 0=P( p0, p1, q0, q1) Q( p0, p1, q0, q1) (15:3)

If there is only one item in the value aggregate, then theprice index P should collapse down to the single priceratio, p11=p

01, and the quantity index Q should collapse

down to the single quantity ratio, q11=q01. In the case of

many items, the price index P is to be interpreted assome sort of weighted average of the individual priceratios, p11=p

01, . . . , p1n=p

0n.

15.9 Thus the first approach to index number theorycan be regarded as the problem of decomposing thechange in a value aggregate, V1/V 0, into the product of apart that is attributable to price change, P(p0, p1, q0, q1),and a part that is attributable to quantity change,Q(p0, p1, q0, q1). This approach to the determination ofthe price index is the approach that is taken in thenational accounts, where a price index is used to deflatea value ratio in order to obtain an estimate of quantitychange. Thus, in this approach to index number theory,the primary use for the price index is as a deflator.Note that once the functional form for the priceindex P(p0, p1, q0, q1) is known, then the corresponding

2 In particular, it can be used to justify the chain system of indexnumbers (discussed in paragraphs 15.86 to 15.97).

3Ralph Turvey has noted that some values may be difficultto decompose into unambiguous price and quantity components.Examples of difficult-to-decompose values are bank charges, gamblingexpenditures and life insurance payments.4Note that it is assumed that there are no new or disappearing com-modities in the value aggregates. Approaches to the ‘‘new goods prob-lem’’ and the problem of accounting for quality change are discussed inChapters 7, 8 and 21.5 The first person to suggest that the price and quantity indices should bejointly determined in order to satisfy equation (15.3) was Fisher (1911,p. 418). Frisch (1930, p. 399) called equation (15.3) the product test.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

264

quantity or volume index Q(p0, p1, q0, q1) is completelydetermined by P; i.e., rearranging equation (15.3):

Q( p0, p1, q0, q1)= V1=V 0� �

=P( p0, p1, q0, q1) (15:4)

Conversely, if the functional form for the quantity indexQ(p0, p1, q0, q1) is known, then the corresponding priceindex P(p0, p1, q0, q1) is completely determined by Q.Thus using this deflation approach to index numbertheory, separate theories for the determination of theprice and quantity indices are not required: if either P orQ is determined, then the other function is implicitlydetermined by the product test equation (15.4).15.10 In the next section, two concrete choices for

the price index P(p0, p1, q0, q1) are considered and thecorresponding quantity indices Q(p0, p1, q0, q1) thatresult from using equation (15.4) are also calculated.These are the two choices used most frequently bynational income accountants.

The Laspeyres and Paasche indices15.11 One of the simplest approaches to the deter-

mination of the price index formula was described ingreat detail by Lowe (1823). His approach to measuringthe price change between periods 0 and 1 was to specifyan approximate representative commodity basket,6 whichis a quantity vector q:[q1, . . . , qn] that is representativeof purchases made during the two periods under con-sideration, and then calculate the level of prices in period1 relative to period 0 as the ratio of the period 1 cost ofthe basket,

Pni=1p

1i qi, to the period 0 cost of the basket,Pn

i=1p0i qi: This fixed basket approach to the determina-

tion of the price index leaves open the question as to howexactly is the fixed basket vector q to be chosen.15.12 As time passed, economists and price statisti-

cians demanded a little more precision with respect to thespecification of the basket vector q. There are two naturalchoices for the reference basket: the base period com-modity vector q0 or the current period commodity vectorq1. These two choices lead to the Laspeyres (1871) priceindex7 PL defined by equation (15.5) and the Paasche(1874) price index8 PP defined by equation (15.6):

9

PL( p0, p1, q0, q1) �

Pni=1

p1i q0i

Pni=1

p0i q0i

(15:5)

PP( p0, p1, q0, q1) �

Pni=1

p1i q1i

Pni=1

p0i q1i

(15:6)

15.13 The formulae (15.5) and (15.6) can be rewrit-ten in an alternative manner that is more useful forstatistical agencies. Define the period t expenditure shareon commodity i as follows:

sti � ptiqti�Pn

j=1

ptjqtj for i=1, . . . , n and t=0, 1 (15:7)

Then the Laspeyres index (15.5) can be rewritten asfollows:10

PL( p0, p1, q0, q1)=

Pni=1

p1i q0i

�Pnj=1

p0j q0j

=Pni=1

( p1i =p0i ) p

0i q0i

�Pnj=1

p0j q0j

=Pni=1

( p1i =p0i )s

0i (15:8)

using definitions (15.7). The Laspeyres price index PLcan thus be written as an arithmetic average of the nprice ratios, p1i =p

0i , weighted by base period expenditure

shares. The Laspeyres formula (until very recently) hasbeen widely used as the intellectual base for consumerprice indices (CPIs) around the world. To implement it,a statistical agency needs only to collect information onexpenditure shares s0n for the index domain of definitionfor the base period 0, and then collect informationon item prices alone on an ongoing basis. Thus theLaspeyres CPI can be produced on a timely basis withouthaving quantity information for the current period.

15.14 The Paasche index can also be written inexpenditure share and price ratio form as follows:11

PP( p0, p1, q0, q1)=

1

Pni=1

p0i q1i

�Pnj=1

p1j q1j

( )

=1

Pni=1

p0i =p1ið Þp1i q1i

�Pnj=1

p1j q1j

( )

6 Lowe (1823, Appendix, p. 95) suggested that the commodity basketvector q should be updated every five years. Lowe indices are studied inmore detail in paragraphs 15.45 to 15.85.7 This index was actually introduced and justified by Drobisch (1871a,p. 147) slightly earlier than Laspeyres. Laspeyres (1871, p. 305) in factexplicitly acknowledged that Drobisch showed him the way forward.However, the contributions of Drobisch have been forgotten for themost part by later writers because Drobisch aggressively pushed for theratio of two unit values as being the ‘‘best’’ index number formula.While this formula has some excellent properties where all the ncommodities being compared have the same unit of measurement, it isuseless when, say, both goods and services are in the index basket.8Drobisch (1871b, p. 424) also appears to have been the first to defineexplicitly and justify the Paasche price index formula, but he rejectedthis formula in favour of his preferred formula, the ratio of unit values,and so again he did not gain any credit for his early suggestion of thePaasche formula.9Note that PL(p

0, p1, q0, q1) does not actually depend on q1 and PP (p0,

p1, q0, q1) does not actually depend on q0. It does no harm to includethese vectors, however, and the notation indicates that the reader is in therealm of bilateral index number theory; i.e., the prices and quantities fora value aggregate pertaining to two periods are being compared.

10 This method of rewriting the Laspeyres index (or any fixed basketindex) as a share weighted arithmetic average of price ratios is attri-butable to Fisher (1897, p. 517) (1911, p. 397) (1922, p. 51) and Walsh(1901, p. 506; 1921a, p. 92).11 This method of rewriting the Paasche index (or any fixed basketindex) as a share weighted harmonic average of the price ratios isattributable to Walsh (1901, p. 511; 1921a, p. 93) and Fisher (1911, p.397–398).

BASIC INDEX NUMBER THEORY

265

=1Pn

i=1

p1i =p0ið Þ�1s1i

� �

=Pni=1

p1i =p0i

� ��1s1i

� ��1(15:9)

using definitions (15.7). The Paasche price index PP canthus be written as a harmonic average of the n item priceratios, p1i =p

0i , weighted by period 1 (current period)

expenditure shares.12 The lack of information on currentperiod quantities prevents statistical agencies from pro-ducing Paasche indices on a timely basis.

15.15 The quantity index that corresponds to theLaspeyres price index using the product test in equation(15.3) is the Paasche quantity index; i.e., if P in equation(15.4) is replaced by PL defined by equation (15.5), thenthe following quantity index is obtained:

QP( p0, p1, q0, q1) �

Pni=1

p1i q1i

Pni=1

p1i q0i

(15:10)

Note that QP is the value of the period 1 quantity vectorvalued at the period 1 prices,

Pni=1p

1i q1i , divided by the

(hypothetical) value of the period 0 quantity vectorvalued at the period 1 prices,

Pni=1p

1i q0i . Thus the period

0 and 1 quantity vectors are valued at the same set ofprices, the current period prices, p1.

15.16 The quantity index that corresponds to thePaasche price index using the product test (15.3) is theLaspeyres quantity index; i.e., if P in equation (15.4) isreplaced by PP defined by equation (15.6), then thefollowing quantity index is obtained:

QL( p0, p1, q0, q1) �

Pni=1

p0i q1i

Pni=1

p0i q0i

(15:11)

Note that QL is the (hypothetical) value of the period 1quantity vector valued at the period 0 prices,

Pni=1p

0i q1i ,

divided by the value of the period 0 quantity vectorvalued at the period 0 prices,

Pni=1p

0i q0i . Thus the period

0 and 1 quantity vectors are valued at the same set ofprices, the base period prices, p0.

15.17 The problem with the Laspeyres and Paascheindex number formulae is that, although they areequally plausible, in general they will give differentanswers. For most purposes, it is not satisfactory forthe statistical agency to provide two answers to thequestion:13 What is the ‘‘best’’ overall summary measureof price change for the value aggregate over the two

periods in question? In the following section, we con-sider how ‘‘best’’ averages of these two estimates of pricechange can be constructed. Before doing so, we ask:What is the ‘‘normal’’ relationship between the Paascheand Laspeyres indices? Under ‘‘normal’’ economicconditions when the price ratios pertaining to the twosituations under consideration are negatively correlatedwith the corresponding quantity ratios, it can be shownthat the Laspeyres price index will be larger than thecorresponding Paasche index.14 A precise statement ofthis result is presented in Appendix 15.1.15 The diver-gence between PL and PP suggests that if a single esti-mate for the price change between the two periods isrequired, then some sort of evenly weighted average ofthe Laspeyres and Paasche indices should be taken asthe final estimate of price change between periods 0and 1. As mentioned above, this strategy will be pursuedin the following section. It should, however, be kept inmind that statistical agencies will not usually haveinformation on current expenditure weights, henceaverages of Paasche and Laspeyres indices can be pro-duced only on a delayed basis (perhaps using nationalaccounts information) or not at all.

Symmetric averages offixed basket price indices

The Fisher index as an average of thePaasche and Laspeyres indices

15.18 As mentioned above, since the Paasche andLaspeyres price indices are equally plausible but oftengive different estimates of the amount of aggregate pricechange between periods 0 and 1, it is useful to considertaking an evenly weighted average of these fixed basketprice indices as a single estimator of price changebetween the two periods. Examples of such symmetricaverages16 are the arithmetic mean, which leads to the

12Note that the derivation in the formula (15.9) shows how harmonicaverages arise in index number theory in a very natural way.13 In principle, instead of averaging the Paasche and Laspeyres indices,the statistical agency could think of providing both (the Paasche indexon a delayed basis). This suggestion would lead to a matrix of pricecomparisons between every pair of periods instead of a time series ofcomparisons. Walsh (1901, p. 425) noted this possibility: ‘‘In fact, if weuse such direct comparisons at all, we ought to use all possible ones.’’

14 Peter Hill (1993, p. 383) summarized this inequality as follows:

It can be shown that relationship (13) [i.e., that PL is greater than PP]holds whenever the price and quantity relatives (weighted by values) arenegatively correlated. Such negative correlation is to be expected for pricetakers who react to changes in relative prices by substituting goods andservices that have become relatively less expensive for those that havebecome relatively more expensive. In the vast majority of situationscovered by index numbers, the price and quantity relatives turn out to benegatively correlated so that Laspeyres indices tend systematically torecord greater increases than Paasche with the gap between them tendingto widen with time.

15 There is another way to see why PP will often be less than PL. If theperiod 0 expenditure shares s0i are exactly equal to the corres-ponding period 1 expenditure shares s1i , then by Schlomilch’s (1858)inequality (see Hardy, Littlewood and Polya (1934, p. 26)), it can beshown that a weighted harmonic mean of n numbers is equal to or lessthan the corresponding arithmetic mean of the n numbers and theinequality is strict if the n numbers are not all equal. If expenditureshares are approximately constant across periods, then it follows thatPP will usually be less than PL under these conditions (see paragraphs15.70 to 15.84).16 For a discussion of the properties of symmetric averages, see Diewert(1993c). Formally, an average mða, bÞ of two numbers a and b issymmetric if mða, bÞ ¼ mðb, aÞ. In other words, the numbers a and bare treated in the same manner in the average. An example of a non-symmetric average of a and b is (1/4)a+(3/4)b. In general, Walsh(1901, p. 105) argued for a symmetric treatment if the two periods (orcountries) under consideration were to be given equal importance.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

266

Drobisch (1871b, p. 425), Sidgwick (1883, p. 68) andBowley (1901, p. 227)17 index, PD � (1=2)PL+(1=2)PP,and the geometric mean, which leads to the Fisher(1922)18 ideal index, PF, defined as

PF (p0,p1,q0,q1) � PL(p

0,p1,q0,q1ÞPP(p0,p1,q0,q1) 1=2

(15:12)

At this point, the fixed basket approach to index numbertheory is transformed into the test approach to indexnumber theory; i.e., in order to determine which of thesefixed basket indices or which averages of them might be‘‘best’’, desirable criteria or tests or properties are neededfor the price index. This topic will be pursued in moredetail in the next chapter, but an introduction to the testapproach is provided in the present section because atest is used to determine which average of the Paascheand Laspeyres indices might be ‘‘best’’.15.19 What is the ‘‘best’’ symmetric average of PL

and PP to use as a point estimate for the theoretical costof living index? It is very desirable for a price indexformula that depends on the price and quantity vectorspertaining to the two periods under consideration tosatisfy the time reversal test.19 An index number formulaP(p0, p1, q0, q1) satisfies this test if

P( p1, p0, q1, q0)=1=P( p0, p1, q0, q1) (15:13)

i.e., if the period 0 and period 1 price and quantity dataare interchanged, and then the index number formula isevaluated, then this new index P(p1, p0, q1, q0) is equalto the reciprocal of the original index P(p0, p1, q0, q1).This is a property that is satisfied by a single price ratio,and it seems desirable that the measure of aggregateprice change should also satisfy this property so that itdoes not matter which period is chosen as the baseperiod. Put another way, the index number comparisonbetween any two points of time should not depend onthe choice of which period we regard as the base period:if the other period is chosen as the base period, then thenew index number should simply equal the reciprocal ofthe original index. It should be noted that the Laspeyresand Paasche price indices do not satisfy this timereversal property.

15.20 Having defined what it means for a priceindex P to satisfy the time reversal test, then it ispossible to establish the following result.20 The Fisherideal price index defined by equation (15.12) is the onlyindex that is a homogeneous21 symmetric average ofthe Laspeyres and Paasche price indices, PL and PP,and satisfies the time reversal test (15.13). The Fisherideal price index thus emerges as perhaps the ‘‘best’’evenly weighted average of the Paasche and Laspeyresprice indices.

15.21 It is interesting to note that this symmetricbasket approach to index number theory dates back toone of the early pioneers of index number theory,Arthur L. Bowley, as the following quotations indicate:

If [the Paasche index] and [the Laspeyres index] lieclose together there is no further difficulty; if they differby much they may be regarded as inferior and superiorlimits of the index number, which may be estimated astheir arithmetic mean . . . as a first approximation (Bow-ley (1901, p. 227)).

When estimating the factor necessary for the correc-tion of a change found in money wages to obtain thechange in real wages, statisticians have not been contentto follow Method II only [to calculate a Laspeyres priceindex], but have worked the problem backwards [tocalculate a Paasche price index] as well as for-wards. . . . They have then taken the arithmetic, geo-metric or harmonic mean of the two numbers so found(Bowley (1919, p. 348)).22

15.22 The quantity index that corresponds to theFisher price index using the product test (15.3) is theFisher quantity index; i.e., if P in equation (15.4) isreplaced by PF defined by equation (15.12), the follow-ing quantity index is obtained:

QF (p0,p1,q0,q1)� QL(p

0,p1,q0,q1)QP(p0,p1,q0,q1)

1=2(15:14)

Thus the Fisher quantity index is equal to the squareroot of the product of the Laspeyres and Paaschequantity indices. It should also be noted that QF (p

0, p1,q0, q1)=PF (q

0, q1, p0, p1); i.e., if the role of pricesand quantities is interchanged in the Fisher price indexformula, then the Fisher quantity index is obtained.23

15.23 Rather than take a symmetric average of thetwo basic fixed basket price indices pertaining to twosituations, PL and PP, it is also possible to return toLowe’s basic formulation and choose the basket vectorq to be a symmetric average of the base and currentperiod basket vectors, q0 and q1. This approach

17Walsh (1901, p. 99) also suggested the arithmetic mean index PD (seeDiewert (1993a, p. 36) for additional references to the early history ofindex number theory).18 Bowley (1899, p. 641) appears to have been the first to suggest theuse of the geometric mean index PF. Walsh (1901, pp. 428–429) alsosuggested this index while commenting on the big differences betweenthe Laspeyres and Paasche indices in one of his numerical examples:‘‘The figures in columns (2) [Laspeyres] and (3) [Paasche] are, singly,extravagant and absurd. But there is order in their extravagance; forthe nearness of their means to the more truthful results shows that theystraddle the true course, the one varying on the one side about as theother does on the other.’’19 See Diewert (1992a, p. 218) for early references to this test. If wewant the price index to have the same property as a single price ratio,then it is important to satisfy the time reversal test. However, otherpoints of view are possible. For example, we may want to use our priceindex for compensation purposes, in which case satisfaction of the timereversal test may not be so important.

20 See Diewert (1997, p. 138).21An average or mean of two numbers a and b, m(a, b), is homogeneousif when both numbers a and b are multiplied by a positive number l,then the mean is also multiplied by l; i.e., m satisfies the followingproperty: m(la, lb)=lm(a, b).22 Fisher (1911, pp. 417–418; 1922) also considered the arithmetic,geometric and harmonic averages of the Paasche and Laspeyres indi-ces.23 Fisher (1922, p. 72) said that P and Q satisfied the factor reversaltest if Q(p0, p1, q0, q1)=P(q0, q1, p0, p1) and P and Q satisfied the pro-duct test (15.3) as well.

BASIC INDEX NUMBER THEORY

267

to index number theory is pursued in the followingsection.

The Walsh index and the theory ofthe ‘‘pure’’ price index

15.24 Price statisticians tend to be very comfortablewith a concept of the price index that is based on pricingout a constant ‘‘representative’’ basket of commodities,q � (q1, q2, . . . , qn), at the prices of periods 0 and 1,p0 � ( p01, p02, . . . , p0n) and p1 � ( p11, p12, . . . , p1n) respec-tively. Price statisticians refer to this type of index as afixed basket index or a pure price index 24 and it corre-sponds to Sir George H. Knibbs’s (1924, p. 43)unequivocal price index.25 Since Lowe (1823) was the firstperson to describe systematically this type of index, it isreferred to as a Lowe index. Thus the general functionalform for the Lowe price index is

PLo( p0, p1, q) �

Pni=1

p1i qi

�Pni=1

p0i qi=Pni=1

si( p1i =p

0i ) (15:15)

where the (hypothetical) hybrid expenditure shares si26

corresponding to the quantity weights vector q aredefined by:

si � p0i qi�Pn

j=1

p0j qj for i=1, 2, . . . , n (15:16)

15.25 The main reason why price statisticians mightprefer a member of the family of Lowe or fixed basketprice indices defined by equation (15.15) is that the fixedbasket concept is easy to explain to the public. Note thatthe Laspeyres and Paasche indices are special cases ofthe pure price concept if we choose q=q0 (which leadsto the Laspeyres index) or if we choose q=q1 (whichleads to the Paasche index).27 The practical problem ofpicking q remains to be resolved, and that is the problemthat will be addressed in this section.

15.26 It should be noted that Walsh (1901, p. 105;1921a) also saw the price index number problem in theabove framework:

Commodities are to be weighted according to theirimportance, or their full values. But the problem ofaxiometry always involves at least two periods. There is afirst period, and there is a second period which is com-pared with it. Price variations have taken place betweenthe two, and these are to be averaged to get the amount oftheir variation as a whole. But the weights of the com-modities at the second period are apt to be different fromtheir weights at the first period. Which weights, then, arethe right ones – those of the first period? Or those of thesecond? Or should there be a combination of the two sets?There is no reason for preferring either the first or thesecond. Then the combination of both would seem to bethe proper answer. And this combination itself involvesan averaging of the weights of the two periods (Walsh(1921a, p. 90)).

Walsh’s suggestion will be followed and thus the ithquantity weight, qi, is restricted to be an average ormean of the base period quantity q0i and the currentperiod quantity for commodity iq1i , say m(q

0i , q

1i ), for i=

1, 2, . . . , n.28 Under this assumption, the Lowe priceindex (15.15) becomes:

PLo( p0, p1, q0, q1) �

Pni=1

p1i m(q0i , q

1i )

Pnj=1

p0j m(q0j , q

1j )

(15:17)

15.27 In order to determine the functional form forthe mean function m, it is necessary to impose some testsor axioms on the pure price index defined by equation(15.17). As above, we ask that PLo satisfy the timereversal test (15.13). Under this hypothesis, it is imme-diately obvious that the mean function m must be asymmetric mean;29 i.e., m must satisfy the followingproperty: m(a, b)=m(b, a) for all a>0 and b>0. Thisassumption still does not pin down the functional formfor the pure price index defined by equation (15.17). Forexample, the function m(a, b) could be the arithmeticmean, (1/2)a+(1/2)b, in which case equation (15.17)reduces to the Marshall (1887) and Edgeworth (1925)price index PME, which was the pure price index pre-ferred by Knibbs (1924, p. 56):

PME( p0, p1, q0, q1) �

Pni=1

p1i f(q0i+q1i )=2g

Pnj=1

p0j f(q0j+q1j )=2g(15:18)

15.28 On the other hand, the function m(a, b) couldbe the geometric mean, (ab)1/2, in which case equation

24 See section 7 in Diewert (2001).25 Suppose however that, for each commodity, Q0=Q, then the frac-tion,

P(P0Q)=

P(PQ), viz., the ratio of aggregate value for the

second unit-period to the aggregate value for the first unit-period isno longer merely a ratio of totals, it also shows unequivocally theeffect of the change in price. Thus it is an unequivocal price index forthe quantitatively unchanged complex of commodities, A, B, C, etc.It is obvious that if the quantities were different on the two

occasions, and if at the same time the prices had been unchanged, thepreceding formula would become

P(PQ0)=

P(PQ). It would still be

the ratio of the aggregate value for the second unit-period to theaggregate value for the first unit period. But it would be also morethan this. It would show in a generalized way the ratio of the quan-tities on the two occasions. Thus it is an unequivocal quantity indexfor the complex of commodities, unchanged as to price and differingonly as to quantity.Let it be noted that the mere algebraic form of these expressions

shows at once the logic of the problem of finding these two indices isidentical (Knibbs (1924, pp. 43–44)).

26Note that Fisher (1922, p. 53) used the terminology ‘‘weighted by ahybrid value’’, while Walsh (1932, p. 657) used the term ‘‘hybridweights’’.27Note that the ith share defined by equation (15.16) in this case is thehybrid share si � p0i q1i =

Pni=1p

0i q1i , which uses the prices of period 0 and

the quantities of period 1.

28Note that we have chosen the mean function m(q0i , q1i ) to be the same

for each item i. We assume that m(a, b) has the following two prop-erties: m(a, b) is a positive and continuous function, defined for allpositive numbers a and b and m(a, a)=a for all a>0.29 For more on symmetric means, see Diewert (1993c, p. 361).

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

268

(15.17) reduces to theWalsh (1901, p. 398; 1921a, p. 97)price index, PW:

30

PW ( p0, p1, q0, q1) �

Pni=1

p1iffiffiffiffiffiffiffiffiffiq0i q

1i

pPnj=1

p0j

ffiffiffiffiffiffiffiffiffiq0j q

1j

q (15:19)

15.29 There are many other possibilities for themean function m, including the mean of order r,[(1/2)ar+(1/2)br ]1/r for r 6¼ 0. Obviously, in order tocompletely determine the functional form for the pureprice index PLo, it is necessary to impose at leastone additional test or axiom on PLo(p

0, p1, q0, q1).15.30 There is a potential problem with the use of

the Edgeworth–Marshall price index (15.18) that hasbeen noticed in the context of using the formula to makeinternational comparisons of prices. If the price levels ofa very large country are compared to the price levels of asmall country using formula (15.18), then the quantityvector of the large country may totally overwhelm theinfluence of the quantity vector corresponding to thesmall country.31 In technical terms, the Edgeworth–Marshall formula is not homogeneous of degree 0 in thecomponents of both q0 and q1. To prevent this problemfrom occurring in the use of the pure price index

PK (p0, p1, q0, q1) defined by equation (15.17), it is asked

that PLo satisfy the following invariance to proportionalchanges in current quantities test:32

PLo( p0, p1, q0, lq1)=PLo( p

0, p1, q0, q1)

for all p0, p1, q0, q1and all l> 0 (15:20)

The two tests, the time reversal test (15.13) and theinvariance test (15.20), make it possible to determine theprecise functional form for the pure price index PLodefined by formula (15.17): the pure price index PK mustbe the Walsh index PW defined by formula (15.19).33

15.31 In order to be of practical use by statisticalagencies, an index number formula must be able to beexpressed as a function of the base period expenditureshares, s0i , the current period expenditure shares, s

1i , and

the n price ratios, p1i =p0i . The Walsh price index defined

by the formula (15.19) can be rewritten in the followingformat:

PW ( p0, p1, q0, q1) �

Pni=1

p1iffiffiffiffiffiffiffiffiffiq0i q

1i

pPnj=1

p0j

ffiffiffiffiffiffiffiffiffiq0j q

1j

q

=

Pni=1

�p1i =

ffiffiffiffiffiffiffiffiffip0i p

1i

p � ffiffiffiffiffiffiffiffis0i s

1i

pPnj=1

�p0j =

ffiffiffiffiffiffiffiffiffip0j p

1j

q � ffiffiffiffiffiffiffiffis0j s

1j

q

=

Pni=1

ffiffiffiffiffiffiffiffis0i s

1i

p ffiffiffiffiffiffiffiffiffiffiffiffip1i =p

0i

pPnj=1

ffiffiffiffiffiffiffiffis0j s

1j

q ffiffiffiffiffiffiffiffiffiffiffiffip0j =p

1j

q (15:21)

15.32 The approach taken to index number theoryin this section was to consider averages of various fixedbasket type price indices. The first approach was to takean even-handed average of the two primary fixed basketindices: the Laspeyres and Paasche price indices. Thesetwo primary indices are based on pricing out the basketsthat pertain to the two periods (or locations) underconsideration. Taking an average of them led to theFisher ideal price index PF defined by equation (15.12).The second approach was to average the basket quantityweights and then price out this average basket at theprices pertaining to the two situations under con-sideration. This approach led to the Walsh price index,PW, defined by equation (15.19). Both of these indicescan be written as a function of the base period expen-diture shares, s0i , the current period expenditure shares,s1i , and the n price ratios, p1i =p

0i . Assuming that the

statistical agency has information on these three sets ofvariables, which index should be used? Experience withnormal time series data has shown that these two indiceswill not differ substantially and thus it is a matter ofindifference which of these indices is used in practice.34

Both of these indices are examples of superlative indices,which are defined in Chapter 17. Note, however, thatboth of these indices treat the data pertaining to the twosituations in a symmetric manner. Hill35 commented onsuperlative price indices and the importance of a sym-metric treatment of the data as follows:

Thus economic theory suggests that, in general, asymmetric index that assigns equal weight to the twosituations being compared is to be preferred to either theLaspeyres or Paasche indices on their own. The precisechoice of superlative index – whether Fisher, Tornqvistor other superlative index – may be of only secondaryimportance as all the symmetric indices are likely toapproximate each other, and the underlying theoreticindex fairly closely, at least when the index numberspread between the Laspeyres and Paasche is not verygreat (Hill (1993, p. 384)).

30Walsh (1921a, p. 103) endorsed PW as being the best index numberformula: ‘‘We have seen reason to believe formula 6 better than for-mula 7. Perhaps formula 9 is the best of the rest, but between it andNos. 6 and 8 it would be difficult to decide with assurance.’’ His for-mula 6 is PW defined by equation (15.19) and his 9 is the Fisher idealdefined by equation (15.12). The Walsh quantity index,QW (p

0, p1, q0, q1), is defined as PW (q0, q1, p0, p1); i.e., the role of prices

and quantities in definition (15.19) is interchanged. If the Walshquantity index is used to deflate the value ratio, an implicit price indexis obtained, which is Walsh’s formula 8.31 This is not likely to be a severe problem in the time series context,however, where the change in quantity vectors going from one periodto the next is small.32 This is the terminology used by Diewert (1992a, p. 216); Vogt (1980)was the first to propose this test.33 See section 7 in Diewert (2001).

34Diewert (1978, pp. 887–889) showed that these two indices willapproximate each other to the second order around an equal price andquantity point. Thus for normal time series data where prices andquantities do not change much going from the base period to thecurrent period, the indices will approximate each other quite closely.35 See also Hill (1988).

BASIC INDEX NUMBER THEORY

269

Annual weights and monthlyprice indices

The Lowe index with monthly prices andannual base year quantities

15.33 It is now necessary to discuss a major prac-tical problem with the above theory of basket typeindices. Up to now, it has been assumed that thequantity vector q:(q1, q2, . . . , qn) that appeared inthe definition of the Lowe index, PLo(p

0, p1, q) definedby equation (15.15), is either the base period quantityvector q0 or the current period quantity vector q1 or anaverage of these two quantity vectors. In fact, in termsof actual statistical agency practice, the quantity vectorq is usually taken to be an annual quantity vector thatrefers to a base year, say b, that is prior to the baseperiod for the prices, period 0. Typically, a statisticalagency will produce a consumer price index at amonthly or quarterly frequency, but for the sake ofargument a monthly frequency will be assumed in whatfollows. Thus a typical price index will have the formPLo(p

0, pt, qb), where p0 is the price vector pertaining tothe base period month for prices, month 0, pt is the pricevector pertaining to the current period month for prices,say month t, and qb is a reference basket quantity vectorthat refers to the base year b, which is equal to or priorto month 0.36 Note that this Lowe index PLo(p

0, pt, qb) isnot a true Laspeyres index (because the annual quantityvector qb is not equal to the monthly quantity vector q0

in general).37

15.34 The question is: why do statistical agencies notpick the reference quantity vector q in the Lowe formulato be the monthly quantity vector q0 that pertains totransactions in month 0 (so that the index would reduceto an ordinary Laspeyres price index)? There are twomain reasons why this is not done:

� Most economies are subject to seasonal fluctuations,and so picking the quantity vector of month 0 as thereference quantity vector for all months of the yearwould not be representative of transactions madethroughout the year.

� Monthly household quantity or expenditure weightsare usually collected by the statistical agency using ahousehold expenditure survey with a relatively smallsample. Hence the resulting weights are usually sub-ject to very large sampling errors and so standardpractice is to average these monthly expenditure orquantity weights over an entire year (or in some cases,over several years), in an attempt to reduce thesesampling errors.

The index number problems that are caused by seasonalmonthly weights are studied in more detail in Chapter22. For now, it can be argued that the use of annualweights in a monthly index number formula is simplya method for dealing with the seasonality problem.38

15.35 One problem with using annual weights cor-responding to a perhaps distant year in the context of amonthly consumer price index must be noted at thispoint: if there are systematic (but divergent) trends incommodity prices and households increase their pur-chases of commodities that decline (relatively) in priceand reduce their purchases of commodities thatincrease (relatively) in price, then the use of distantquantity weights will tend to lead to an upward bias inthis Lowe index compared to one that used more currentweights, as will be shown below. This observation sug-gests that statistical agencies should strive to get up-to-date weights on an ongoing basis.

15.36 It is useful to explain how the annual quantityvector qb could be obtained from monthly expenditureson each commodity during the chosen base year b. Letthe month m expenditure of the reference populationin the base year b for commodity i be vb,mi and let thecorresponding price and quantity be pb,mi and qb,mirespectively. Of course, value, price and quantity foreach commodity are related by the following equa-tions:

vb,mi =pb,mi qb,mi where i=1, . . . , n and m=1, . . . , 12

(15:22)

For each commodity i, the annual total, qbi , can beobtained by price deflating monthly values and summingover months in the base year b as follows:

qbi=P12m=1

vb,mi

pb,mi=P12m=1

qb,mi ; i=1, . . . , n (15:23)

where equation (15.22) was used to derive the secondequation in (15.23). In practice, the above equations willbe evaluated using aggregate expenditures over closelyrelated commodities and the price pb,mi will be the monthm price index for this elementary commodity group i inyear b relative to the first month of year b.

15.37 For some purposes, it is also useful to haveannual prices by commodity to match up with theannual quantities defined by equation (15.23). Followingnational income accounting conventions, a reasonable39

36Month 0 is called the price reference period and year b is called theweight reference period.37 Triplett (1981, p. 12) defined the Lowe index, calling it a Laspeyresindex, and calling the index that has the weight reference period equalto the price reference period a pure Laspeyres index. Balk (1980c,p. 69), however, asserted that although the Lowe index is of the fixedbase type; it is not a Laspeyres price index. Triplett also noted thehybrid share representation for the Lowe index defined by equations(15.15) and (15.16). Triplett noted that the ratio of two Lowe indicesusing the same quantity weights was also a Lowe index. Baldwin (1990,p. 255) called the Lowe index an annual basket index.

38 In fact, the use of the Lowe index PLo(p0, pt, qb ) in the context of

seasonal commodities corresponds to Bean and Stine’s (1924, p. 31)Type A index number formula. Bean and Stine made three additionalsuggestions for price indices in the context of seasonal commodities.Their contributions are evaluated in Chapter 22.39 These annual commodity prices are essentially unit value prices.Under conditions of high inflation, the annual prices defined byequation (15.24) may no longer be ‘‘reasonable’’ or representative ofprices during the entire base year because the expenditures in the finalmonths of the high inflation year will be somewhat artificially blownup by general inflation. Under these conditions, the annual prices andannual commodity expenditure shares should be interpreted withcaution. For more on dealing with situations where there is highinflation within a year, see Hill (1996).

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

270

price pbi to match up with the annual quantity qbi is the

value of total consumption of commodity i in year bdivided by qbi . Thus we have:

pbi �P12m=1

vb,mi =qbi i=1, . . . , n

=

P12m=1

vb,mi

P12m=1

vb,mi =pb,mi

using (15:23)

=P12m=1

sb,mi ( pb,mi )�1� ��1

(15:24)

where the share of annual expenditure on commodity iin month m of the base year is

sb,mi � vb,miP12k=1

vb;ki

; i=1, . . . , n (15:25)

Thus the annual base year price for commodity i, pbiturns out to be a monthly expenditure weighted har-monic mean of the monthly prices for commodity i in the

base year, pb, 1i , pb, 2i , . . . , pb, 12i .15.38 Using the annual commodity prices for the

base year defined by equation (15.24), a vector of theseprices can be defined as pb � [ pb1, . . . , pbn]. Using thisdefinition, the Lowe index PLo(p

0, pt, qb) can be ex-pressed as a ratio of two Laspeyres indices, where theprice vector pb plays the role of base period prices ineach of the two Laspeyres indices:

PLo( p0, pt, qb) �

Pni=1

ptiqbi

Pni=1

p0i qbi

=

Pni=1

ptiqbi

�Pni=1

pbi qbi

Pni=1

p0i qbi

�Pni=1

pbi qbi

=

Pni=1

sbi ( pti=p

bi )

Pni=1

sbi ( p0i =p

bi )

=PL( pb, pt, qb)=PL( p

b, p0, qb) (15:26)

where the Laspeyres formula PL was defined by equa-tion (15.5). Thus the above equation shows that theLowe monthly price index comparing the prices ofmonth 0 to those of month t using the quantities of baseyear b as weights, PLo(p

0, pt, qb), is equal to the Las-peyres index that compares the prices of month t tothose of year b, PL(p

b, pt, qb), divided by the Laspeyresindex that compares the prices of month 0 to those ofyear b, PL(p

b, p0, qb). Note that the Laspeyres index inthe numerator can be calculated if the base year com-modity expenditure shares, sbi , are known along with theprice ratios that compare the prices of commodity i inmonth t, pti , with the corresponding annual averageprices in the base year b, pbi . The Laspeyres index in the

denominator can be calculated if the base year com-modity expenditure shares, sbi , are known along with theprice ratios that compare the prices of commodity i inmonth 0, p0i , with the corresponding annual averageprices in the base year b, pbi :

15.39 There is another convenient formula for eval-uating the Lowe index, PLo(p

0, pt,qb), and that is to usethe hybrid weights formula (15.15). In the present con-text, the formula becomes:

PLo( p0, pt, qb) �

Pni=1

ptiqbi

Pni=1

p0i qbi

=

Pni=1

( pti=p0i )p

0i qbi

Pni=1

p0i qbi

=Pni=1

ptip0i

� �s0bi

(15:27)

where the hybrid weights s0bi using the prices of month 0and the quantities of year b are defined by

s0bi �p0i q

biPn

j=1

pbj qbj

; i=1, . . . , n

=pbi q

bi ( p

0i =p

bi )Pn

j=1

pbj qbj ( p

0j =p

bj )

h i (15:28)

The second equation in (15.28) shows how the base yearexpenditures, pbi q

bi , can be multiplied by the commodity

price indices, p0i =pbi , in order to calculate the hybrid

shares.15.40 There is one additional formula for the Lowe

index, PLo(p0, pt, qb), that will be exhibited. Note that the

Laspeyres decomposition of the Lowe index defined bythe third term in equation (15.26) involves the long-termprice relatives, pti=p

bi , which compare the prices in month

t, pti , with the possibly distant base year prices, pbi , and

that the hybrid share decomposition of the Lowe indexdefined by the third term in equation (15.27) involves thelong-term monthly price relatives, pti=p

0i , which compare

the prices in month t, pti , with the base month prices, p0i .

Both of these formulae are unsatisfactory in practicebecause of sample attrition: each month, a substantialfraction of commodities disappears from the market-place. Thus it is useful to have a formula for updating theprevious month’s price index using just month-over-month price relatives. In other words, long-term pricerelatives disappear at too fast a rate to make it viable, inpractice, to base an index number formula on their use.The Lowe index for month t+1, PLo(p

0, pt+1, qb), can bewritten in terms of the Lowe index for month t,PLo(p

0, pt, qb), and an updating factor as follows:

PLo( p0, pt+1, qb) �

Pni=1

pt+1i qbi

Pni=1

p0i qbi

=

Pni=1

ptiqbi

Pni=1

p0i qbi

2664

3775Pni=1

pt+1i qbi

Pni=1

ptiqbi

2664

3775

=PLo( p0, pt+1, qb)

Pni=1

pt+1i qbi

Pni=1

ptiqbi

2664

3775

BASIC INDEX NUMBER THEORY

271

=PLo( p0, pt+1, qb)

Pni=1

pt+1i

pti

!ptiq

bi

Pni=1

ptiqbi

266664

377775

=PLo( p0, pt+1, qb)

Pni=1

pt+1i

pti

� �stbi

� �(15:29)

where the hybrid weights stbi are defined by:

stbi �ptiq

biPn

j=1

ptjqbj

; i=1, . . . , n (15:30)

Thus the required updating factor, going from month tto month t+1, is the chain link index

Pni=1s

tbi ( p

t+1i =pti),

which uses the hybrid share weights stbi corresponding tomonth t and base year b.

15.41 The Lowe index PLo(p0, pt, qb) can be re-

garded as an approximation to the ordinary Laspeyresindex, PL(p

0, pt, q0), that compares the prices of the basemonth 0, p0, to those of month t, pt, using the quantityvectors of month 0, q0, as weights. It turns out that thereis a relatively simple formula that relates these twoindices. In order to explain this formula, it is firstnecessary to make a few definitions. Define the ith pricerelative between month 0 and month as

ri � pti=p0i ; i=1, . . . , n (15:31)

The ordinary Laspeyres price index, going from month 0to t, can be defined in terms of these price relatives asfollows:

PL( p0, pt, q0) �

Pni=1

ptiq0i

Pni=1

p0i q0i

=

Pni=1

ptip0i

� �p0i q

0i

Pni=1

p0i q0i

=Pni=1

ptip0i

� �s0i=

Pni=1

s0i ri � r* (15:32)

where the month 0 expenditure shares s0i are defined asfollows:

s0i �p0i q

0iPn

j=1

p0j q0j

; i=1, . . . , n (15:33)

15.42 Define the ith quantity relative ti as the ratioof the quantity of commodity i used in the base year b,qbi , to the quantity used in month 0, q

0i , as follows:

ti � qbi =q0i ; i=1, . . . , n (15:34)

The Laspeyres quantity index, QL(q0, qb, p0), that

compares quantities in year b, qb, to the correspondingquantities in month 0, q0, using the prices of month 0,p0, as weights can be defined as a weighted average of

the quantity ratios ti as follows:

QL(q0, qb, p0) �

Pni=1

p0i qbi

Pni=1

p0i q0i

=

Pni=1

qbiq0i

� �p0i q

0i

Pni=1

p0i q0i

=Pni=1

qbiq0i

� �s0i

=Pni=1

s0i ti using definition (15:34)

� t* (15:35)

15.43 Using formula (A15.2.4) in Appendix 15.2 tothis chapter, the relationship between the Lowe indexPLo(p

0, pt, qb) that uses the quantities of year b asweights to compare the prices of month t to month 0,and the corresponding ordinary Laspeyres index PL(p

0,pt, q0) that uses the quantities of month 0 as weights isthe following one:

PLo( p0, pt, qb) �

Pni=1

ptiqbi

Pni=1

p0i qbi

=PL( p0, pt, q0)+

Pni=1

(ri�r*)(ti�t*)s0i

QL(q0, qb, p0)

(15:36)

Thus the Lowe price index using the quantities of year bas weights, PLo(p

0, pt, qb), is equal to the usual Laspeyresindex using the quantities of month 0 as weights,PL( p

0, pt, q0), plus a covariance termPn

i=1(ri � r*) (ti �t*)s0i between the price relatives ri � pti=p0i and thequantity relatives ti � qbi =q0i , divided by the Laspeyresquantity index QL(q

0, qb, p0) between month 0 and baseyear b.

15.44 Formula (15.36) shows that the Lowe priceindex will coincide with the Laspeyres price index ifthe covariance or correlation between the month 0 to tprice relatives ri � pti=p0i and the month 0 to year bquantity relatives ti � qbi =q0i is zero. Note that this cov-ariance will be zero under three different sets of conditions:

� if the month t prices are proportional to the month 0prices so that all ri=r*;

� if the base year b quantities are proportional to themonth 0 quantities so that all ti=t*;

� if the distribution of the relative prices ri is indepen-dent of the distribution of the relative quantities ti.

The first two conditions are unlikely to hold empirically,but the third is possible, at least approximately, if con-sumers do not systematically change their purchasinghabits in response to changes in relative prices.

15.45 If this covariance in formula (15.36) is nega-tive, then the Lowe index will be less than the Laspeyresindex. Finally, if the covariance is positive, then theLowe index will be greater than the Laspeyres index.Although the sign and magnitude of the covarianceterm,

Pni=1(ri � r*)(ti � t*)s0i , is ultimately an empirical

matter, it is possible to make some reasonable con-jectures about its likely sign. If the base year b precedes

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

272

the price reference month 0 and there are long-termtrends in prices, then it is likely that this covariance ispositive and hence that the Lowe index will exceed thecorresponding Laspeyres price index;40 i.e.,

PLo( p0, pt, qb) > PL( p

0, pt, q0) (15:37)

To see why the covariance is likely to be positive, sup-pose that there is a long-term upward trend in the priceof commodity i so that ri � r* � ( pti=p0i )� r* is positive.With normal consumer substitution responses,41 qti=q

0i

less an average quantity change of this type is likely to benegative, or, upon taking reciprocals, q0i =q

ti less an

average quantity change of this (reciprocal) type is likelyto be positive. But if the long-term upward trend in priceshas persisted back to the base year b, then ti � t* �(qbi =q

0i )� t* is also likely to be positive. Hence, the

covariance will be positive under these circumstances.Moreover, the more distant is the base year b from thebase month 0, the bigger the residuals ti � t* are likely tobe and the bigger will be the positive covariance. Simi-larly, the more distant is the current period month t fromthe base period month 0, the bigger the residuals ri � r*are likely to be and the bigger will be the positive co-variance. Thus, under the assumptions that there arelong-term trends in prices and normal consumer sub-stitution responses, the Lowe index will normally begreater than the corresponding Laspeyres index.15.46 Define the Paasche index between months 0

and t as follows:

PP( p0, pt, qt) �

Pni=1

ptiqti

Pni=1

p0i qti

(15:38)

As discussed in paragraphs 15.18 to 15.23, a reasonabletarget index to measure the price change going frommonth 0 to t is some sort of symmetric average of thePaasche index PP(p

0, pt, qt), defined by formula (15.38),and the corresponding Laspeyres index, PL(p

0, pt, q0),defined by formula (15.32). Adapting equation (A15.1.5)in Appendix 15.1, the relationship between the Paascheand Laspeyres indices can be written as follows:

PP( p0, pt, qt)=PL( p

0, pt, q0)+

Pni=1

(ri � r*)(ui � u*)s0i

QL(q0, qt, p0)

(15:39)

where the price relatives ri � pti=p0i are defined byequation (15.31) and their share-weighted average r* byequation (15.32) and the ui, u* and QL are defined asfollows:

ui � qti=q0i ; i=1, . . . , n (15:40)

u* �Pni=1

s0i ui=QL(q0, qt, p0) (15:41)

and the month 0 expenditure shares s0i are defined by theidentity (15.33). Thus u* is equal to the Laspeyresquantity index between months 0 and t. This means thatthe Paasche price index that uses the quantities of montht as weights, PP(p

0, pt, qt), is equal to the usual Laspeyresindex using the quantities of month 0 as weights,

PL(p0, pt, q0), plus a covariance term

Pni=1(ri � r*) (ui �

u*)s0i between the price relatives ri � pti=p0i and the quan-tity relatives ui � qti=q0i , divided by the Laspeyres quan-tity index QL(q

0, qt, p0) between month 0 and month t.15.47 Although the sign and magnitude of the co-

variance term,Pn

i=1(ri � r*)(ui � u*)s0i , is again anempirical matter, it is possible to make a reasonableconjecture about its likely sign. If there are long-termtrends in prices and consumers respond normally toprice changes in their purchases, then it is likely that thiscovariance is negative and hence the Paasche index willbe less than the corresponding Laspeyres price index;i.e.,

PP( p0, pt, qt)<PL( p

0, pt, q0) (15:42)

To see why this covariance is likely to be negative, sup-pose that there is a long-term upward trend in the priceof commodity i 42 so that ri � r* � ( pti=p0i )� r* is posi-tive. With normal consumer substitution responses, qti=q

0i

less an average quantity change of this type is likely to benegative. Hence ui � u* � (qti=q0i )� u* is likely to benegative. Thus, the covariance will be negative underthese circumstances. Moreover, the more distant is thebase month 0 from the current month t, the bigger inmagnitude the residuals ui � u* are likely to be and thebigger in magnitude will be the negative covariance.43

Similarly, the more distant is the current period month tfrom the base period month 0, the bigger the residualsri � r* will probably be and the bigger in magnitude willbe the covariance. Thus under the assumptions that thereare long-term trends in prices and normal consumersubstitution responses, the Laspeyres index will begreater than the corresponding Paasche index, with thedivergence likely to grow as month t becomes moredistant from month 0.

15.48 Putting the arguments in the three previousparagraphs together, it can be seen that under the

40For this relationship to hold, it is also necessary to assume thathouseholds have normal substitution effects in response to these long-term trends in prices; i.e., if a commodity increases (relatively) in price,its consumption will decline (relatively) and if a commodity decreasesrelatively in price, its consumption will increase relatively.41Walsh (1901, pp. 281–282) was well aware of consumer substitutioneffects, as can be seen in the following comment which noted the basicproblem with a fixed basket index that uses the quantity weights of asingle period: ‘‘The argument made by the arithmetic averagist sup-poses that we buy the same quantities of every class at both periods inspite of the variation in their prices, which we rarely, if ever, do. As arough proposition, we – a community – generally spend more on ar-ticles that have risen in price and get less of them, and spend less onarticles that have fallen in price and get more of them.’’

42 The reader can carry through the argument if there is a long-termrelative decline in the price of the ith commodity. The argumentrequired to obtain a negative covariance requires that there be somedifferences in the long-term trends in prices; i.e., if all prices grow (orfall) at the same rate, there will be price proportionality and the co-variance will be zero.43However, QL=u* may also be growing in magnitude, so the neteffect on the divergence between PL and PP is ambiguous.

BASIC INDEX NUMBER THEORY

273

assumptions that there are long-term trends in pricesand normal consumer substitution responses, the Loweprice index between months 0 and t will exceed thecorresponding Laspeyres price index, which in turn willexceed the corresponding Paasche price index; i.e., underthese hypotheses,

PLo(p0,pt,qb)>PL(p

0,pt,q0)>PP(p0,pt,qt) (15:43)

Thus, if the long-run target price index is an average ofthe Laspeyres and Paasche indices, it can be seen thatthe Laspeyres index will have an upward bias relative tothis target index and the Paasche index will have adownward bias. In addition, if the base year b is prior tothe price reference month, month 0, then the Lowe indexwill also have an upward bias relative to the Laspeyresindex and hence also to the target index.

The Lowe index and mid-year indices15.49 The discussion in the previous paragraph

assumed that the base year b for quantities precededthe base month for prices, month 0. If the currentperiod month t is quite distant from the base month 0,however, then it is possible to think of the base year bas referring to a year that lies between months 0 and t.If the year b does fall between months 0 and t, thenthe Lowe index becomes a mid-year index.44 It turnsout that the Lowe mid-year index no longer has theupward biases indicated by the inequalities in theinequality (15.43) under the assumption of long-termtrends in prices and normal substitution responses byquantities.

15.50 It is now assumed that the base year quantityvector qb corresponds to a year that lies between months0 and t. Under the assumption of long-term trends inprices and normal substitution effects so that there arealso long-term trends in quantities (in the oppositedirection to the trends in prices so that if the ith com-modity price is trending up, then the corresponding ithquantity is trending down), it is likely that the inter-

mediate year quantity vector will lie between the monthlyquantity vectors q0 and qt. The mid-year Lowe index,PLo(p

0, pt, qb), and the Laspeyres index going frommonth 0 to t, PL(p

0, pt, q0), will still satisfy the exactrelationship given by equation (15.36). Thus PLo(p

0, pt,qb) will equal PL(p

0, pt, q0) plus the covariance term

[Pn

i=1(ri � r*) (ti � t*)s0i ]=QL(q0, qb, p0), where QL(q

0, qb,

p0) is the Laspeyres quantity index going from month0 to t. This covariance term is likely to be negative sothat

PL( p0, pt, q0) > PLo( p

0, pt, qb): (15:44)

To see why this covariance is likely to be negative,suppose that there is a long-term upward trend in theprice of commodity i so that ri � r* � ( pit=p0i )� r* ispositive. With normal consumer substitution responses,qi will tend to decrease relatively over time and since q

bi

is assumed to be between q0i and qti , q

bi =q

0i less an aver-

age quantity change of this type is likely to be negative.Hence ui � u* � (qbi =q0i )� t* is likely to be negative.Thus, the covariance is likely to be negative underthese circumstances. Therefore, under the assump-tions that the quantity base year falls between months0 and t and that there are long-term trends in pricesand normal consumer substitution responses, the Las-peyres index will normally be larger than the corre-sponding Lowe mid-year index, with the divergenceprobably growing as month t becomes more distantfrom month 0.

15.51 It can also be seen that under the aboveassumptions, the mid-year Lowe index is likely to begreater than the Paasche index between months 0 andt; i.e.,

PLo( p0, pt, qb) > PP( p

0, pt, qt) (15:45)

To see why the above inequality is likely to hold, think ofqb starting at the month 0 quantity vector q0 and thentrending smoothly to the month t quantity vector qt.When qb=q0, the Lowe index PLo(p

0, pt, qb) becomes theLaspeyres index PL(p

0, pt, q0). When qb=qt, the Loweindex PLo( p

0, pt, qb) becomes the Paasche index PP(p0,

pt, qt). Under the assumption of trending prices andnormal substitution responses to these trending prices, itwas shown earlier that the Paasche index will be less thanthe corresponding Laspeyres price index; i.e., thatPP(p

0, pt,qt) was less than PL(p0, pt,q0), recalling the

inequality (15.42). Thus, under the assumption ofsmoothly trending prices and quantities between months0 and t, and assuming that qb is between q0 and qt, we willhave

PP( p0, pt, qt)<PLo( p

0, pt, qb)< PL( p0, pt, q0) (15:46)

Thus if the base year for the Lowe index is chosen to bein between the base month for the prices, month 0, andthe current month for prices, month t, and there aretrends in prices with corresponding trends in quantitiesthat correspond to normal consumer substitutioneffects, then the resulting Lowe index is likely to liebetween the Paasche and Laspeyres indices going frommonths 0 to t. If the trends in prices and quantities are

44The concept of the mid-year index can be traced to Hill (1998, p. 46):

When inflation has to be measured over a specified sequence of years, suchas a decade, a pragmatic solution to the problems raised above would beto take the middle year as the base year. This can be justified on thegrounds that the basket of goods and services purchased in the middleyear is likely to be much more representative of the pattern of con-sumption over the decade as a whole than baskets purchased in either thefirst or the last years. Moreover, choosing a more representative basketwill also tend to reduce, or even eliminate, any bias in the rate of inflationover the decade as a whole as compared with the increase in the CoLindex.

Thus, in addition to introducing the concept of a mid-year index, Hillalso introduced the terminology representativity bias. Baldwin (1990,pp. 255–256) also introduced the term representativeness: ‘‘Hererepresentativeness [in an index number formula] requires that theweights used in any comparison of price levels are related to thevolume of purchases in the periods of comparison.’’However, this basic idea dates back to Walsh (1901, p. 104; 1921a, p.

90). Baldwin (1990, p. 255) also noted that his concept of representa-tiveness was the same as Drechsler’s (1973, p. 19) concept of char-acteristicity. For additional material on mid-year indices, see Schultz(1999) and Okamoto (2001). Note that the mid-year index conceptcould be viewed as a close competitor to Walsh’s (1901, p. 431) multi-year fixed basket index where the quantity vector was chosen to be anarithmetic or geometric average of the quantity vectors in the span ofperiods under consideration.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

274

smooth, then choosing the base year half-way betweenperiods 0 and t should give a Lowe index that is approxi-mately half-way between the Paasche and Laspeyresindices; hence it will be very close to an ideal targetindex between months 0 and t. This basic idea has beenimplemented by Okamoto (2001), using Japanese con-sumer data, and he found that the resulting mid-yearindices approximated very closely to the correspondingFisher ideal indices.15.52 It should be noted that these mid-year indices

can only be computed on a retrospective basis; i.e., theycannot be calculated in a timely fashion, as can Loweindices that use a base year that is prior to month 0.Thus mid-year indices cannot be used to replace themore timely Lowe indices. The above material indicates,however, that these timely Lowe indices are likely tohave an upward bias that is even bigger than the usualLaspeyres upward bias compared to an ideal targetindex, which was taken to be an average of the Paascheand Laspeyres indices.15.53 All the inequalities derived in this section rest

on the assumption of long-term trends in prices (andcorresponding economic responses in quantities). If thereare no systematic long-run trends in prices, but onlyrandom fluctuations around a common trend in allprices, then the above inequalities are not valid and theLowe index using a prior base year will probably providea perfectly adequate approximation to both the Paascheand Laspeyres indices. There are, however, reasons forbelieving that there are some long-run trends in prices. Inparticular:

� The computer chip revolution of the past 40 yearshas led to strong downward trends in the prices ofproducts that use these chips intensively. As new usesfor chips have been developed over the years, theshare of products that are chip intensive has grownand this implies that what used to be a relativelyminor problem has become a more major problem.

� Other major scientific advances have had similareffects. For example, the invention of fibre optic cable(and lasers) has led to a downward trend in tele-communications prices as obsolete technologies basedon copper wire are gradually replaced.

� Since the end of the Second World War, a seriesof international trade agreements has dramaticallyreduced tariffs around the world. These reductions,combined with improvements in transport technolo-gies, have led to a very rapid growth of internationaltrade and remarkable improvements in internationalspecialization. Manufacturing activities in the moredeveloped economies have gradually been outsourcedto lower-wage countries, leading to deflationin goods prices in most countries around the world.In contrast, many services cannot be readilyoutsourced45 and so, on average, the price of servicestrends upwards while the price of goods trendsdownwards.

� At the microeconomic level, there are tremendousdifferences in growth rates of firms. Successful firmsexpand their scale, lower their costs, and cause lesssuccessful competitors to wither away with theirhigher prices and lower volumes. This leads to a sys-tematic negative correlation between changes in itemprices and the corresponding changes in item volumesthat can be very large indeed.

Thus there is some a priori basis for assuming long-rundivergent trends in prices. Hence there is some basisfor concern that a Lowe index that uses a base yearfor quantity weights that is prior to the base month forprices may be upwardly biased, compared to a moreideal target index.

The Young index15.54 Recall the definitions for the base year quan-

tities, qbi , and the base year prices, pbi , given by equations

(15.23) and (15.24) above. The base year expenditureshares can be defined in the usual way as follows:

sbi �pbi q

biPn

k=1

pbkqbk

; i=1, . . . , n (15:47)

Define the vector of base year expenditure shares inthe usual way as sb � [sb1, . . . , sbn]. These base yearexpenditure shares were used to provide an alternativeformula for the base year b Lowe price index going frommonth 0 to t, defined in equation (15.26) as PLo( p0, pt, qb)=

Pni=1s

bi p

ti=p

bi

� �� �=Pn

i=1sbi p

0i =p

bi

� �� �. Rather

than using this index as their short-run target index,many statistical agencies use the following closely rela-ted index:

PY ( p0, pt, sb) �

Pni=1

sbi pti=p

0i

� �(15:48)

This type of index was first defined by the Englisheconomist, Arthur Young (1812).46 Note that there is achange in focus when the Young index is used comparedto the other indices proposed earlier in this chapter. Upto this point, the indices proposed have been of the fixedbasket type (or averages of such indices) where a com-modity basket that is somehow representative for thetwo periods being compared is chosen and then ‘‘pur-chased’’ at the prices of the two periods and the indexis taken to be the ratio of these two costs. In contrast,for the Young index, representative expenditure sharesare chosen that pertain to the two periods underconsideration, and then these shares are used to calcu-late the overall index as a share-weighted average of theindividual price ratios, pti=p

0i . Note that this view of

index number theory, based on the share-weightedaverage of price ratios, is a little different from the viewtaken at the beginning of this chapter, which saw the

45 Some services, however, can be internationally outsourced; e.g., callcentres, computer programming and airline maintenance.

46 This formula is attributed to Young by Walsh (1901, p. 536; 1932,p. 657).

BASIC INDEX NUMBER THEORY

275

index number problem as that of decomposing a valueratio into the product of two terms, one of whichexpresses the amount of price change between the twoperiods and the other which expresses the amount ofquantity change.47

15.55 Statistical agencies sometimes regard theYoung index, defined above, as an approximation to theLaspeyres price index PL(p

0, pt, q0). Hence, it is ofinterest to see how the two indices compare. Definingthe long-term monthly price relatives going from month0 to t as ri � pti=p0i and using definitions (15.32) and(15.48):

PY ( p0, pt, sb)�PL( p0, pt, q0) �

Pni=1

sbiptip0i

� ��Pni=1

s0iptip0i

� �

=Pni=1

sbi�s0i� � pti

p0i

� �

=Pni=1

sbi�s0i� �

ri

=Pni=1

sbi�s0i� �

ri�r*� �

+r*Pni=1

sbi�s0i� �

=Pni=1

sbi�s0i� �

ri�r*� �

(15:49)

sincePn

i=1sbi=Pn

i=1s0i=1 and defining r* �Pn

i=1

s0i ri=PL( p0, pt, q0). Thus the Young index PY(p

0, pt, sb)is equal to the Laspeyres index PL(p

0, pt, q0), plus thecovariance between the difference in the annual sharespertaining to year b and the month 0 shares, sbi � s0i , andthe deviations of the relative prices from their mean,ri � r*.

15.56 It is no longer possible to guess at what thelikely sign of the covariance term is. The question is nolonger whether the quantity demanded goes down as theprice of commodity i goes up (the answer to this questionis usually ‘‘yes’’) but the new question is: does the share

of expenditure go down as the price of commodity i goesup? The answer to this question depends on the elasticityof demand for the product. Let us provisionally assume,however, that there are long-run trends in commodityprices and if the trend in prices for commodity i is abovethe mean, then the expenditure share for the commoditytrends down (and vice versa). Thus we are assuming highelasticities or very strong substitution effects. Assumingalso that the base year b is prior to month 0, then underthese conditions, suppose that there is a long-termupward trend in the price of commodity i so thatri � r* � ( pti=p0i )� r* is positive. With the assumed veryelastic consumer substitution responses, si will tend todecrease relatively over time and since sbi is assumed tobe prior to s0i ; s

0i is expected to be less than s

bi or s

bi � s0i

will probably be positive. Thus, the covariance is likelyto be positive under these circumstances. Hence withlong-run trends in prices and very elastic responses ofconsumers to price changes, the Young index is likely tobe greater than the corresponding Laspeyres index.

15.57 Assume that there are long-run trends incommodity prices. If the trend in prices for commodity iis above the mean, then suppose that the expenditureshare for the commodity trends up (and vice versa).Thus we are assuming low elasticities or very weaksubstitution effects. Assume also that the base year b isprior to month 0 and suppose that there is a long-termupward trend in the price of commodity i sothat ri � r* � ( pti=p0i )� r* is positive. With the assumedvery inelastic consumer substitution responses, si willtend to increase relatively over time and since sbi isassumed to be prior to s0i , it will be the case that s

0i is

greater than sbi or sbi � s0i is negative. Thus, the covar-

iance is likely to be negative under these circumstances.Hence with long-run trends in prices and very inelasticresponses of consumers to price changes, the Young indexis likely to be less than the corresponding Laspeyres index.

15.58 The previous two paragraphs indicate that, apriori, it is not known what the likely difference betweenthe Young index and the corresponding Laspeyres indexwill be. If elasticities of substitution are close to one,then the two sets of expenditure shares, sbi and s

0i , will be

close to each other and the difference between the twoindices will be close to zero. If monthly expenditureshares have strong seasonal components, however, thenthe annual shares sbi could differ substantially from themonthly shares s0i .

15.59 It is useful to have a formula for updating theprevious month’s Young price index using just month-over-month price relatives. The Young index for montht+1, PY(p

0, pt+1, sb), can be written in terms of theYoung index for month t, PY(p

0, pt, sb), and an updatingfactor as follows:

PY ( p0, pt+1, sb) �

Pni=1

sbipt+1i

p0i

� �

=PY ( p0, pt, sb)

Pni=1

sbi ( pt+1i =p0i )

Pni=1

sbi ( pti=p

0i )

47 Fisher’s 1922 book is famous for developing the value ratiodecomposition approach to index number theory, but his introductorychapters took the share-weighted average point of view: ‘‘An indexnumber of prices, then shows the average percentage change of pricesfrom one point of time to another’’ (Fisher (1922, p. 3)). Fisher wenton to note the importance of economic weighting: ‘‘The precedingcalculation treats all the commodities as equally important; conse-quently, the average was called ‘simple’. If one commodity is moreimportant than another, we may treat the more important as though itwere two or three commodities, thus giving it two or three times asmuch ‘weight’ as the other commodity’’ (Fisher (1922, p. 6)). Walsh(1901, pp. 430–431) considered both approaches: ‘‘We can either (1)draw some average of the total money values of the classes during anepoch of years, and with weighting so determined employ the geo-metric average of the price variations [ratios]; or (2) draw some averageof the mass quantities of the classes during the epoch, and apply tothem Scrope’s method.’’ Scrope’s method is the same as using theLowe index. Walsh (1901, pp. 88–90) consistently stressed the impor-tance of weighting price ratios by their economic importance (ratherthan using equally weighted averages of price relatives). Both the valueratio decomposition approach and the share-weighted averageapproach to index number theory are studied from the axiomaticperspective in Chapter 16.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

276

=PY ( p0, pt, sb)

Pni=1

pbi qbi ( p

t+1i =p0i )

Pni=1

pbi qbi ( p

ti=p

0i )

using definition (15:47)

=PY ( p0, pt, sb)

Pni=1

pbi qbi

ptip0i

� �pt+1i

pti

!

Pni=1

pbi qbi ( p

ti=p

0i )

=PY ( p0, pt, sb)

Pni=1

sb0ti ( pt+1i =pti)

� �(15:50)

where the hybrid weights sb0ti are defined by

sb0ti �pbi q

bi ( p

ti=p

0i )Pn

k=1

pbkqbk( p

tk=p

0k)

=sbi ( p

ti=p

0i )Pn

k=1

sbk( ptk=p

0k)

i=1, . . . , n

(15:51)

Thus the hybrid weights sb0ti can be obtained from thebase year weights sbi by updating them; i.e., by multi-plying them by the price relatives (or indices at higherlevels of aggregation), pti=p

0i . Thus the required updating

factor, going from month t to month t+1, is the chainlink index,

Pni=1s

b0ti ( pt+1

i =pti), which uses the hybridshare weights sb0ti defined by equation (15.51).15.60 Even if the Young index provides a close

approximation to the corresponding Laspeyres index, itis difficult to recommend the use of the Young index as afinal estimate of the change in prices going from period 0to t, just as it was difficult to recommend the use of theLaspeyres index as the final estimate of inflation goingfrom period 0 to t. Recall that the problem with theLaspeyres index was its lack of symmetry in the treat-ment of the two periods under consideration; i.e., usingthe justification for the Laspeyres index as a good fixedbasket index, there was an identical justification for theuse of the Paasche index as an equally good fixed basketindex to compare periods 0 and t. The Young indexsuffers from a similar lack of symmetry with respect tothe treatment of the base period. The problem can beexplained as follows. The Young index, PY(p

0, pt, sb),defined by equation (15.48) calculates the price changebetween months 0 and t treating month 0 as the base.But there is no particular reason to necessarily treatmonth 0 as the base month other than convention.Hence, if we treat month t as the base and use the sameformula to measure the price change from month t backto month 0, the index PY ( p

t, p0, sb)=Pn

i=1sbi ( p

0i =p

bi ),

would be appropriate. This estimate of price change canthen be made comparable to the original Young indexby taking its reciprocal, leading to the following rebasedYoung index,48 P*Y ( p

t, p0, sb), defined as

P*Y ( p0, pt, sb) � 1

�Xni=1

sbi ( p0i =p

ti)

=Xni=1

sbi ( pti=p

0i )�1

" #�1(15:52)

The rebased Young index, P*Y ( p0, pt, sb), which uses the

current month as the initial base period, is a share-weighted harmonic mean of the price relatives going frommonth 0 to month t, whereas the original Young index,PY(p

0, pt, sb), is a share-weighted arithmetic mean of thesame price relatives.

15.61 Fisher argued as follows that an index numberformula should give the same answer no matter whichperiod was chosen as the base:

Either one of the two times may be taken as the‘‘base’’. Will it make a difference which is chosen? Cer-tainly, it ought not and our Test 1 demands that it shallnot. More fully expressed, the test is that the formula forcalculating an index number should be such that it willgive the same ratio between one point of comparison andthe other point, no matter which of the two is taken as thebase (Fisher (1922, p. 64)).

15.62 The problem with the Young index is that notonly does it not coincide with its rebased counterpart,but there is a definite inequality between the two indices,namely:

P*Y ( p0, pt, sb)� PY ( p0, pt, sb) (15:53)

with a strict inequality provided that the period t pricevector pt is not proportional to the period 0 price vectorp0.49 A statistical agency that uses the direct Youngindex PY (p

0, pt, sb) will generally show a higher inflationrate than a statistical agency that uses the same raw databut uses the rebased Young index, P*Y ( p

0, ptI , sb).15.63 The inequality (15.53) does not tell us by how

much the Young index will exceed its rebased timeantithesis. In Appendix 15.3, however, it is shown thatto the accuracy of a certain second-order Taylor seriesapproximation, the following relationship holdsbetween the direct Young index and its time antithesis:

PY ( p0, pt, sb) � P*Y ( p0, pt, sb)+PY ( p

0, pt, sb)Var e

(15:54)

48Using Fisher’s (1922, p. 118) terminology, PY* (p0, pt, sb)� 1=

[PY (pt, p0, sb)] is the time antithesis of the original Young index,

PY(p0, pt, sb).

49 These inequalities follow from the fact that a harmonic mean of Mpositive numbers is always equal to or less than the correspondingarithmetic mean; seeWalsh (1901, p. 517) or Fisher (1922, pp. 383–384).This inequality is a special case of Schlomilch’s (1858) inequality; seeHardy, Littlewood and Polya (1934, p. 26). Walsh (1901, pp. 330–332)explicitly noted the inequality (15.53) and also noted that the corre-sponding geometric average would fall between the harmonic andarithmetic averages. Walsh (1901, p. 432) computed some numericalexamples of the Young index and found big differences between it andhis ‘‘best’’ indices, even using weights that were representative for theperiods being compared. Recall that the Lowe index becomes theWalshindex when geometric mean quantity weights are chosen and so theLowe index can perform well when representative weights are used.This is not necessarily the case for the Young index, even using repre-sentative weights. Walsh (1901, p. 433) summed up his numericalexperiments with the Young index as follows: ‘‘In fact, Young’smethod, in every form, has been found to be bad.’’

BASIC INDEX NUMBER THEORY

277

where Var e is defined as

Var e �Pni=1

sbi ei� e*� �2

(15:55)

The deviations ei are defined by 1+ei=ri/r* fori=1, . . . , n where the ri and their weighted mean r* aredefined by

ri � pti=p0i ; i=1, . . . , n; (15:56)

r* �Pni=1

sbi ri (15:57)

which turns out to equal the direct Young index,PY(p

0, pt, sb). The weighted mean of the ei is defined as

e* �Pni=1

sbi ei (15:58)

which turns out to equal 0. Hence the more dispersionthere is in the price relatives pti=p

0i , to the accuracy of a

second-order approximation, the more the direct Youngindex will exceed its counterpart that uses month t as theinitial base period rather than month 0.

15.64 Given two a priori equally plausible indexnumber formulae that give different answers, such asthe Young index and its time antithesis, Fisher (1922,p. 136) generally suggested taking the geometric aver-age of the two indices.50 A benefit of this averaging isthat the resulting formula will satisfy the time reversaltest. Thus rather than using either the base period 0Young index, PY(p

0, pt, sb), or the current period tYoung index, P*Y ( p

0, pt, sb), which is always below thebase period 0 Young index if there is any dispersion inrelative prices, it seems preferable to use the followingindex, which is the geometric average of the two alter-natively based Young indices:51

P**Y ( p0, pt, sb) � PY ( p

0, pt, sb)P*Y ( p0, pt, sb)

# $1=2(15:59)

If the base year shares sbi happen to coincide with boththe month 0 and month t shares, s0i and s

ti respectively,

it can be seen that the time-rectified Young indexP**Y ( p

0, pt, sb) defined by equation (15.59) will coincide

with the Fisher ideal price index between months 0and t, PF(p

0, pt, q0, qt) (which will also equal the Las-peyres and Paasche indices under these conditions).Note also that the index P**Y defined by equation(15.59) can be produced on a timely basis by a statisticalagency.

The Divisia index and discreteapproximations to it

The Divisia price and quantity indices15.65 The second broad approach to index number

theory relies on the assumption that price and quantitydata change in a more or less continuous way.

15.66 Suppose that the price and quantity data onthe n commodities in the chosen domain of definitioncan be regarded as continuous functions of (continuous)time, say pi(t) and qi(t) for i=1, . . . , n. The value ofconsumer expenditure at time t is V(t) defined in theobvious way as:

V(t) �Pni=1

pi(t)qi(t) (15:60)

15.67 Now suppose that the functions pi (t) and qi (t)are differentiable. Then both sides of the definition(15.60) can be differentiated with respect to time toobtain:

V 0(t)=Pni=1

p0i(t)qi(t)+Pni=1

pi(t)q0i(t) (15:61)

Divide both sides of equation (15.61) through by V(t)and using definition (15.60), the following equation isobtained:

V 0(t)

V(t)=

Pni=1

p0i(t)qi(t)+Pni=1

pi(t)q0i(t)

Pnj=1

pj(t)qj(t)

=Pni=1

p0i(t)

pi(t)si(t)+

Pni=1

q0i(t)

qi(t)si(t)

(15:62)

where the time t expenditure share on commodity i, si (t),is defined as:

si(t) �pi(t)qi(t)Pn

m=1

pm(t)qm(t)

for i=1, 2, . . . , n (15:63)

15.68 Divisia (1926, p. 39) argued as follows: sup-pose the aggregate value at time t, V(t), can be writtenas the product of a time t price level function, P(t) say,times a time t quantity level function, Q(t) say; i.e., wehave:

V(t)=P(t)Q(t) (15:64)

Suppose further that the functions P(t) and Q(t) aredifferentiable. Then differentiating the equation (15.64)yields:

V 0(t)=P0(t)Q(t)+P(t)Q0(t) (15:65)

50We now come to a third use of these tests, namely, to ‘‘rectify’’formulae, i.e., to derive from any given formula which does notsatisfy a test another formula which does satisfy it; . . . . This iseasily done by ‘‘crossing’’, that is, by averaging antitheses. If a givenformula fails to satisfy Test 1 [the time reversal test], its timeantithesis will also fail to satisfy it; but the two will fail, as it were, inopposite ways, so that a cross between them (obtained by geome-trical averaging) will give the golden mean which does satisfy (Fisher(1922, p. 136)).

Actually the basic idea behind Fisher’s rectification procedure wassuggested by Walsh, who was a discussant for Fisher (1921), whereFisher gave a preview of his 1922 book: ‘‘We merely have to take anyindex number, find its antithesis in the way prescribed by ProfessorFisher, and then draw the geometric mean between the two’’ (Walsh(1921b, p. 542)).51 This index is a base year weighted counterpart to an equallyweighted index proposed by Carruthers, Sellwood and Ward (1980, p.25) and Dalen (1992, p. 140) in the context of elementary index numberformulae. See Chapter 20 for further discussion of this unweightedindex.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

278

Dividing both sides of equation (15.65) by V(t) andusing equation (15.64) leads to the following equation:

V 0(t)

V(t)=P0(t)

P(t)+Q0(t)

Q(t)(15:66)

15.69 Divisia compared the two expressions for thelogarithmic value derivative, V0(t)/V(t), given by equa-tions (15.62) and (15.66), and he simply defined thelogarithmic rate of change of the aggregate pricelevel, P0(t)/P(t), as the first set of terms on the right-handside of (15.62). He also simply defined the logarithmicrate of change of the aggregate quantity level, Q0(t)/Q(t),as the second set of terms on the right-hand side ofequation (15.62). That is, he made the following defini-tions:

P0(t)

P(t)�Pni=1

si(t)p0i(t)

pi(t)(15:67)

Q0(t)

Q(t)�Pni=1

si(t)q0i(t)

qi(t)(15:68)

15.70 Definitions (15.67) and (15.68) are reasonabledefinitions for the proportional changes in the aggre-gate price and quantity (or quantity) levels, P(t) andQ(t).52 The problem with these definitions is that eco-nomic data are not collected in continuous time; they arecollected in discrete time. In other words, even thoughtransactions can be thought of as occurring in con-tinuous time, no consumer records his or her purchasesas they occur in continuous time; rather, purchases overa finite time period are cumulated and then recorded. Asimilar situation occurs for producers or sellers ofcommodities; firms cumulate their sales over discreteperiods of time for accounting or analytical purposes. Ifit is attempted to approximate continuous time byshorter and shorter discrete time intervals, empiricalprice and quantity data can be expected to becomeincreasingly erratic since consumers only make pur-chases at discrete points of time (and producers orsellers of commodities only make sales at discrete pointsof time). It is, however, still of some interest toapproximate the continuous time price and quantitylevels, P(t) and Q(t) defined implicitly by equations(15.67) and (15.68), by discrete time approximations.This can be done in two ways. Either methods ofnumerical approximation can be used or assumptionscan be made about the path taken through time by thefunctions pi (t) and qi (t) (i=1, . . . , n). The first strategyis used in the following section. For discussions of thesecond strategy, see Vogt (1977; 1978), Van Ijzeren(1987, pp. 8–12), Vogt and Barta (1997) and Balk(2000a).15.71 There is a connection between the Divisia

price and quantity levels, P(t) and Q(t), and the

economic approach to index number theory. This con-nection is, however, best made after studying the eco-nomic approach to index number theory. Since thismaterial is rather technical, it has been relegated toAppendix 15.4.

Discrete approximations to thecontinuous time Divisia index

15.72 In order to make operational the continuoustime Divisia price and quantity levels, P(t) and Q(t)defined by the differential equations (15.67) and (15.68),it is necessary to convert to discrete time. Divisia (1926,p. 40) suggested a straightforward method for doing thisconversion, which we now outline.

15.73 Define the following price and quantity (for-ward) differences:

DP � P(1)�P(0) (15:69)

Dpi � pi(1)� pi(0); i=1, . . . , n (15:70)

Using the above definitions:

P(1)

P(0)=P(0)+DPP(0)

=1+DPP(0)

� 1+

Pni=1

Dpiqi(0)

Pnm=1

pm(0)qm(0)

using (15.67) when t=0 and approximating p0i(0) by thedifference Dpi

=

Pni=1

fpi(0)+Dpigqi(0)

Pnm=1

pm(0)qm(0)

=

Pni=1

pi(1)qi(0)

Pnm=1

pm(0)qm(0)

=PL( p0, p1, q0, q1) (15:71)

where pt:[p1(t), . . . , pn(t)] and qt:[q1(t), . . . , qn(t)] for

t=0,1. Thus, it can be seen that Divisia’s discreteapproximation to his continuous time price index is justthe Laspeyres price index, PL, defined above by equa-tion (15.5).

15.74 But now a problem noted by Frisch (1936,p. 8) occurs: instead of approximating the derivatives bythe discrete (forward) differences defined by equations(15.69) and (15.70), other approximations could be usedand a wide variety of discrete time approximationscould be obtained. For example, instead of using for-ward differences and evaluating the index at time t=0, itwould be possible to use backward differences andevaluate the index at time t=1. These backward differ-ences are defined as:

Dbpi � pi(0)�pi(1); i=1, . . . , n (15:72)

This use of backward differences leads to the followingapproximation for P(0)/P(1):

52 If these definitions are applied (approximately) to the Young indexstudied in the previous section, then it can be seen that in order for theYoung price index to be consistent with the Divisia price index, thebase year shares should be chosen to be average shares that apply tothe entire time period between months 0 and t.

BASIC INDEX NUMBER THEORY

279

P(0)

P(1)=P(1)+DbP

P(1)=1+

DbPP(1)

� 1+

Pni=1

Dbpiqi(1)

Pnm=1

pm(1)qm(1)

using (15.67) when t=1 and approximating p0i(1) by thedifference Dbpi

=

Pni=1

fpi(1)+Dbpigqi(1)

Pnm=1

pm(1)qm(1)

=

Pni=1

pi(0)qi(1)

Pnm=1

pm(1)qm(1)

=1

PP( p0, p1, q0, q1)(15:73)

where PP is the Paasche index defined above by equation(15.6). Taking reciprocals of both sides of equation(15.73) leads to the following discrete approximation toP(1)/P(0):

P(1)

P(0)� PP (15:74)

15.75 Thus, as Frisch53 noted, both the Paascheand Laspeyres indices can be regarded as (equallyvalid) approximations to the continuous time Divisiaprice index.54 Since the Paasche and Laspeyres indicescan differ considerably in some empirical applica-tions, it can be seen that Divisia’s idea is not all thathelpful in determining a unique discrete time indexnumber formula.55 What is useful about the Divisiaindices is the idea that as the discrete unit of timegets smaller, discrete approximations to the Divisiaindices can approach meaningful economic indicesunder certain conditions. Moreover, if the Divisiaconcept is accepted as the ‘‘correct’’ one for indexnumber theory, then the corresponding ‘‘correct’’ dis-crete time counterpart might be taken as a weightedaverage of the chain price relatives pertaining to theadjacent periods under consideration, where the

weights are somehow representative of the two periodsunder consideration.

Fixed base versus chain indices15.76 In this section,56 we discuss the merits of

using the chain system for constructing price indicesin the time series context versus using the fixed basesystem.57

15.77 The chain system58 measures the change inprices going from one period to a subsequent periodusing a bilateral index number formula involving theprices and quantities pertaining to the two adjacentperiods. These one-period rates of change (the links inthe chain) are then cumulated to yield the relative levelsof prices over the entire period under consideration.Thus if the bilateral price index is P, the chain systemgenerates the following pattern of price levels for thefirst three periods:

1,P( p0, p1, q0, q1),P( p0, p1, q0, q1)P( p1, p2, q1, q2)

(15:75)

15.78 In contrast, the fixed base system of pricelevels, using the same bilateral index number formula P,simply computes the level of prices in period t relative tothe base period 0 as P(p0, pt, q0, qt). Thus the fixed basepattern of price levels for periods 0,1 and 2 is:

1,P( p0, p1, q0, q1),P( p0, p2, q0, q2) (15:76)

15.79 Note that in both the chain system and thefixed base system of price levels defined by the formulae(15.75) and (15.76), the base period price level is setequal to 1. The usual practice in statistical agencies is toset the base period price level equal to 100. If this isdone, then it is necessary to multiply each of the num-bers in the formulae (15.75) and (15.76) by 100.

15.80 Because of the difficulties involved in obtain-ing current period information on quantities (orequivalently, on expenditures), many statistical agenciesloosely base their consumer price index on the use of theLaspeyres formula (15.5) and the fixed base system.Therefore, it is of interest to look at some of the possibleproblems associated with the use of fixed base Laspeyresindices.

53 ‘‘As the elementary formula of the chaining, we may get Laspeyres’or Paasche’s or Edgeworth’s or nearly any other formula, according aswe choose the approximation principle for the steps of the numericalintegration’’ (Frisch (1936, p. 8)).54Diewert (1980, p. 444) also obtained the Paasche and Laspeyresapproximations to the Divisia index, using a somewhat differentapproximation argument. He also showed how several other populardiscrete time index number formulae could be regarded as approx-imations to the continuous time Divisia index.55 Trivedi (1981) systematically examined the problems involved infinding a ‘‘best’’ discrete time approximation to the Divisia indicesusing the techniques of numerical analysis. These numerical analysistechniques depend on the assumption that the ‘‘true’’ continuous timemicro-price functions, pi (t), can be adequately represented by a poly-nomial approximation. Thus we are led to the conclusion that the‘‘best’’ discrete time approximation to the Divisia index depends onassumptions that are difficult to verify.

56 This section is largely based on the work of Hill (1988; 1993, pp.385–390).57 The results in Appendix 15.4 provide some theoretical support forthe use of chain indices in that it is shown that under certain condi-tions, the Divisia index will equal an economic index. Hence any dis-crete approximation to the Divisia index will approach the economicindex as the time period gets shorter. Thus under certain conditions,chain indices will approach an underlying economic index.58 The chain principle was introduced independently into the eco-nomics literature by Lehr (1885, pp. 45–46) and Marshall (1887,p. 373). Both authors observed that the chain system would mitigatethe difficulties arising from the introduction of new commodities intothe economy, a point also mentioned by Hill (1993, p. 388). Fisher(1911, p. 203) introduced the term ‘‘chain system’’.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

280

15.81 The main problem with the use of fixed baseLaspeyres indices is that the period 0 fixed basket ofcommodities that is being priced out in period t canoften be quite different from the period t basket. Thus ifthere are systematic trends in at least some of the pricesand quantities59 in the index basket, the fixed baseLaspeyres price index PL(p

0, pt, q0, qt) can be quite dif-ferent from the corresponding fixed base Paasche priceindex, PP(p

0, pt, q0, qt).60 This means that both indicesare likely to be an inadequate representation of themovement in average prices over the time period underconsideration.15.82 The fixed base Laspeyres quantity index can-

not be used for ever: eventually, the base period quan-tities q0 are so far removed from the current periodquantities qt that the base must be changed. Chaining ismerely the limiting case where the base is changed eachperiod.61

15.83 The main advantage of the chain system isthat under normal conditions, chaining will reduce thespread between the Paasche and Laspeyres indices.62

These two indices each provide an asymmetric per-spective on the amount of price change that has occur-red between the two periods under consideration and itcould be expected that a single point estimate of theaggregate price change should lie between these twoestimates. Thus the use of either a chained Paasche orLaspeyres index will usually lead to a smaller differencebetween the two and hence to estimates that are closer tothe ‘‘truth’’.63

15.84 Hill (1993, p. 388), drawing on the earlierresearch of Szulc (1983) and Hill (1988, pp. 136–137),noted that it is not appropriate to use the chain systemwhen prices oscillate or bounce. This phenomenon canoccur in the context of regular seasonal fluctuations orin the context of price wars. However, in the context ofroughly monotonically changing prices and quantities,Hill (1993, p. 389) recommended the use of chainedsymmetrically weighted indices (see paragraphs 15.18 to15.32). The Fisher and Walsh indices are examples ofsymmetrically weighted indices.15.85 It is possible to be a little more precise about

the conditions under which to chain or not to chain.Basically, chaining is advisable if the prices and quan-tities pertaining to adjacent periods are more similar

than the prices and quantities of more distant periods,since this strategy will lead to a narrowing of the spreadbetween the Paasche and Laspeyres indices at eachlink.64 Of course, one needs a measure of how similarare the prices and quantities pertaining to two periods.The similarity measures could be relative ones or abso-lute ones. In the case of absolute comparisons, twovectors of the same dimension are similar if they areidentical and dissimilar otherwise. In the case of relativecomparisons, two vectors are similar if they are pro-portional and dissimilar if they are non-proportional.65

Once a similarity measure has been defined, the pricesand quantities of each period can be compared to eachother using this measure, and a ‘‘tree’’ or path that linksall of the observations can be constructed where themost similar observations are compared with each otherusing a bilateral index number formula.66 Hill (1995)defined the price structures between two countries to bemore dissimilar the bigger the spread between PL andPP; i.e., the bigger is {PL/PP, PP/PL}. The problem withthis measure of dissimilarity in the price structures of thetwo countries is that it could be the case that PL=PP (sothat the Hill measure would register a maximal degree ofsimilarity), but p0 could be very different from pt. Thusthere is a need for a more systematic study of similarity(or dissimilarity) measures in order to pick the ‘‘best’’one that could be used as an input into Hill’s (1999a;1999b; 2001) spanning tree algorithm for linkingobservations.

59 Examples of rapidly downward trending prices and upward trendingquantities are computers, electronic equipment of all types, Internetaccess and telecommunication charges.60Note that PL(p

0, pt, q0, qt) will equal PP(p0, pt, q0, qt) if either the two

quantity vectors q0 and qt are proportional or the two price vectors p0

and pt are proportional. Thus in order to obtain a difference betweenthe Paasche and Laspeyres indices, non-proportionality in both pricesand quantities is required.61Regular seasonal fluctuations can cause monthly or quarterly data to‘‘bounce’’—using the term coined by Szulc (1983, p. 548)—andchaining bouncing data can lead to a considerable amount of index‘‘drift’’; i.e., if after 12 months, prices and quantities return to theirlevels of a year earlier, then a chained monthly index will usually notreturn to unity. Hence, the use of chained indices for ‘‘noisy’’ monthlyor quarterly data is not recommended without careful consideration.62 See Diewert (1978, p. 895) and Hill (1988; 1993, pp. 387–388).63 This observation will be illustrated with an artificial data set inChapter 19.

64Walsh, in discussing whether fixed base or chained index numbersshould be constructed, took for granted that the precision of all rea-sonable bilateral index number formulae would improve, providedthat the two periods or situations being compared were more similar,and hence favoured the use of chained indices: ‘‘The question is really,in which of the two courses [fixed base or chained index numbers] arewe likely to gain greater exactness in the comparisons actually made?Here the probability seems to incline in favor of the second course; forthe conditions are likely to be less diverse between two contiguousperiods than between two periods say fifty years apart’’ (Walsh (1901,p. 206)).Walsh (1921a, pp. 84–85) later reiterated his preference for chained

index numbers. Fisher also made use of the idea that the chain systemwould usually make bilateral comparisons between price and quantitydata that were more similar, and hence the resulting comparisonswould be more accurate:

The index numbers for 1909 and 1910 (each calculated in terms of 1867–1877) are compared with each other. But direct comparison between 1909and 1910 would give a different and more valuable result. To use acommon base is like comparing the relative heights of two men bymeasuring the height of each above the floor, instead of putting themback to back and directly measuring the difference of level between thetops of their heads (Fisher (1911, p. 204)).

It seems, therefore, advisable to compare each year with the next, or,in other words, to make each year the base year for the next. Such aprocedure has been recommended by Marshall, Edgeworth and Flux. Itlargely meets the difficulty of non-uniform changes in the Q’s, for anyinequalities for successive years are relatively small (Fisher (1911, pp.423–424)).

65 (Diewert (2002b) takes an axiomatic approach to defining variousindices of absolute and relative dissimilarity.)66 Fisher (1922, pp. 271–276) hinted at the possibility of using spatiallinking; i.e., of linking countries that are similar in structure. Themodern literature has, however, grown as a result of the pioneeringefforts of Robert Hill (1995; 1999a; 1999b; 2001). Hill (1995) used thespread between the Paasche and Laspeyres price indices as an indicatorof similarity, and showed that this criterion gives the same results as acriterion that looks at the spread between the Paasche and Laspeyresquantity indices.

BASIC INDEX NUMBER THEORY

281

15.86 The method of linking observations explainedin the previous paragraph, based on the similarity of theprice and quantity structures of any two observations,may not be practical in a statistical agency context sincethe addition of a new period may lead to a reordering ofthe previous links. The above ‘‘scientific’’ method forlinking observations may be useful, however, in decidingwhether chaining is preferable or whether fixed baseindices should be used while making month-to-monthcomparisons within a year.

15.87 Some index number theorists have objected tothe chain principle on the grounds that it has no coun-terpart in the spatial context:

They [chain indices] only apply to intertemporalcomparisons, and in contrast to direct indices they arenot applicable to cases in which no natural order orsequence exists. Thus the idea of a chain index forexample has no counterpart in interregional or interna-tional price comparisons, because countries cannot besequenced in a ‘‘logical’’ or ‘‘natural’’ way (there is nokþ 1 nor k�1 country to be compared with country k)(von der Lippe (2001, p. 12)).67

This is of course correct, but the approach of Hill doeslead to a ‘‘natural’’ set of spatial links. Applying thesame approach to the time series context will lead to aset of links between periods which may not be month-to-month but it will in many cases justify year-over-yearlinking of the data pertaining to the same month. Thisproblem is reconsidered in Chapter 22.

15.88 It is of some interest to determine if thereare index number formulae that give the same answerwhen either the fixed base or chain system is used.Comparing the sequence of chain indices defined by theexpression (15.75) to the corresponding fixed base indi-ces, it can be seen that we will obtain the same answerin all three periods if the index number formula Psatisfies the following functional equation for all priceand quantity vectors:

P( p0, p2, q0, q2)=P( p0, p1, q0, q1)P( p1, p2, q1, q2)

(15:77)

If an index number formula P satisfies the equation(15.77), then P satisfies the circularity test.68

15.89 If it is assumed that the index number formulaP satisfies certain properties or tests in addition tothe circularity test above,69 then Funke, Hacker andVoeller (1979) showed that P must have the following

functional form, originally established by Konus andByushgens70 (1926, pp. 163–166):71

PKB( p0, p1, q0, q1) �

Qni=1

p1ip0i

� �ai

(15:78)

where the n constants ai satisfy the following restric-tions:

Pni=1

ai=1 and ai > 0 for i=1, . . . , n (15:79)

Thus under very weak regularity conditions, the onlyprice index satisfying the circularity test is a weightedgeometric average of all the individual price ratios, theweights being constant through time.

15.90 An interesting special case of the family ofindices defined by equation (15.78) occurs when theweights ai are all equal. In this case, PKB reduces to theJevons (1865) index:

PJ( p0, p1, q0, q1) �

Qni=1

p1ip0i

� �1n

(15:80)

15.91 The problem with the indices defined byKonus and Byushgens, and Jevons is that the individualprice ratios, p1i =p

0i , have weights (either ai or 1/n) that

are independent of the economic importance of com-modity i in the two periods under consideration. Putanother way, these price weights are independent of thequantities of commodity i consumed or the expenditureson commodity i during the two periods. Hence, theseindices are not really suitable for use by statisticalagencies at higher levels of aggregation when expendi-ture share information is available.

15.92 The above results indicate that it is not usefulto ask that the price index P satisfy the circularity testexactly. It is nevertheless of some interest to find indexnumber formulae that satisfy the circularity test to somedegree of approximation, since the use of such an indexnumber formula will lead to measures of aggregate pricechange that are more or less the same no matter whetherwe use the chain or fixed base systems. Fisher (1922,p. 284) found that deviations from circularity using hisdata set and the Fisher ideal price index PF defined byequation (15.12) above were quite small. This relativelyhigh degree of correspondence between fixed base and

67 It should be noted that von der Lippe (2001, pp. 56–58) is a vigorouscritic of all index number tests based on symmetry in the time seriescontext, although he is willing to accept symmetry in the context ofmaking international comparisons. ‘‘But there are good reasons not toinsist on such criteria in the intertemporal case. When no symmetryexists between 0 and t, there is no point in interchanging 0 and t’’ (vonder Lippe (2001, p. 58)).68 The test name is attributable to Fisher (1922, p. 413) and the conceptoriginated from Westergaard (1890, pp. 218–219).69 The additional tests referred to above are: (i) positivity and con-tinuity of P(p0, p1, q0, q1) for all strictly positive price and quantityvectors p0, p1, q0, q1; (ii) the identity test; (iii) the commensurability test;(iv) P(p0, p1, q0, q1) is positively homogeneous of degree one in the

components of p1, and (v) P(p0, p1, q0, q1) is positively homogeneous ofdegree zero in the components of q1.

70Konus and Byushgens show that the index defined by equation(15.78) is exact for Cobb–Douglas (1928) preferences; see also Pollak(1983, pp. 119–120). The concept of an exact index number formula isexplained in Chapter 17.71 The result in equation (15.78) can be derived using results in Eich-horn (1978, pp. 167–168) and Vogt and Barta (1997, p. 47). A simpleproof can be found in Balk (1995). This result vindicates IrvingFisher’s (1922, p. 274) intuition that ‘‘the only formulae which con-form perfectly to the circular test are index numbers which have con-stant weights . . . ’’. Fisher (1922, p. 275) went on to assert: ‘‘But,clearly, constant weighting is not theoretically correct. If we compare1913 with 1914, we need one set of weights; if we compare 1913with 1915, we need, theoretically at least, another set of weights.. . . Similarly, turning from time to space, an index number forcomparing the United States and England requires one set of weights,and an index number for comparing the United States and Francerequires, theoretically at least, another.’’

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

282

chain indices has been found to hold for other symme-trically weighted formulae, such as the Walsh index PWdefined by equation (15.19).72 In most time seriesapplications of index number theory where the base yearin fixed base indices is changed every five years or so, itwill not matter very much whether the statistical agencyuses a fixed base price index or a chain index, providedthat a symmetrically weighted formula is used.73 Thechoice between a fixed base price index or chain indexwill depend, of course, on the length of the time seriesconsidered and the degree of variation in the prices andquantities as we go from period to period. The moreprices and quantities are subject to large fluctuations(rather than smooth trends), the less the correspon-dence.74

15.93 It is possible to give a theoretical explanationfor the approximate satisfaction of the circularity test forsymmetrically weighted index number formulae. Anothersymmetrically weighted formula is the Tornqvist indexPT.

75 The natural logarithm of this index is defined asfollows:

lnPT ( p0, p1, q0, q1) �

Pni=1

1

2s0i+s1i� �

lnp1ip0i

� �(15:81)

where the period t expenditure shares sti are defined byequation (15.7). Alterman, Diewert and Feenstra (1999,p. 61) show that if the logarithmic price ratiosln ( pti=p

t�1i ) trend linearly with time t and the expendi-

ture shares sti also trend linearly with time, then theTornqvist index PT will satisfy the circularity testexactly.76 Since many economic time series on prices andquantities satisfy these assumptions approximately, theTornqvist index PT will satisfy the circularity testapproximately. As is seen in Chapter 19, the Tornqvistindex generally closely approximates the symmetricallyweighted Fisher and Walsh indices, so that for manyeconomic time series (with smooth trends), all three ofthese symmetrically weighted indices will satisfy thecircularity test to a high enough degree of approxima-

tion so that it will not matter whether we use the fixedbase or chain principle.

15.94 Walsh (1901, p. 401; 1921a, p. 98; 1921b,p. 540) introduced the following useful variant of thecircularity test:

1=P( p0, p1, q0, q1)P( p1, p2, q1, q2) . . .P( pT , p0, qT , q0)

(15:82)

The motivation for this test is the following. Use thebilateral index formula P(p0, p1, q0, q1) to calculate thechange in prices going from period 0 to 1, use the sameformula evaluated at the data corresponding to periods1 and 2, P(p1, p2, q1, q2), to calculate the change in pricesgoing from period 1 to 2, . . . , use P(PT�1, pT , qT�1, qT )to calculate the change in prices going from period T� 1to T, introduce an artificial period T+1 that has exactlythe price and quantity of the initial period 0 and useP(pT , p0, qT , q0) to calculate the change in prices goingfrom period T to 0. Finally, multiply all of these indicestogether. Since we end up where we started, the productof all of these indices should ideally be one. Diewert(1993a, p. 40) called this test a multiperiod identity test.77

Note that if T=2 (so that the number of periods is threein total), then Walsh’s test reduces to Fisher’s (1921,p. 534; 1922, p. 64) time reversal test.78

15.95 Walsh (1901, pp. 423–433) showed how hiscircularity test could be used in order to evaluate how‘‘good’’ any bilateral index number formula was. Whathe did was invent artificial price and quantity data forfive periods, and he added a sixth period that had thedata of the first period. He then evaluated the right-handside of equation (15.82) for various formulae,P(p0, p1, q0, q1), and determined how far from unity theresults were. His ‘‘best’’ formulae had products thatwere close to one.79

15.96 This same framework is often used to evaluatethe efficacy of chained indices versus their direct coun-terparts. Thus if the right-hand side of equation (15.82)turns out to be different from unity, the chained indicesare said to suffer from ‘‘chain drift’’. If a formula doessuffer from chain drift, it is sometimes recommendedthat fixed base indices be used in place of chained ones.However, this advice, if accepted, would always leadto the adoption of fixed base indices, provided thatthe bilateral index formula satisfies the identity test,P(p0, p0, q0, q0)=1. Thus it is not recommended thatWalsh’s circularity test be used to decide whether fixedbase or chained indices should be calculated. It is fair touse Walsh’s circularity test, as he originally used it as an

72 See, for example, Diewert (1978, p. 894)). Walsh (1901, pp. 424 and429) found that his three preferred formulae all approximated eachother very well, as did the Fisher ideal for his artificial data set.73More specifically, most superlative indices (which are symmetricallyweighted) will satisfy the circularity test to a high degree of approx-imation in the time series context. See Chapter 17 for the definition of asuperlative index. It is worth stressing that fixed base Paasche andLaspeyres indices are very likely to diverge considerably over a five-year period if computers (or any other commodity which has price andquantity trends that are quite different from the trends in the othercommodities) are included in the value aggregate under consideration(see Chapter 19 for some ‘‘empirical’’ evidence on this topic).74Again, see Szulc (1983) and Hill (1988).75 This formula was implicitly introduced in Tornqvist (1936) andexplicitly defined in Tornqvist and Tornqvist (1937).76 This exactness result can be extended to cover the case when thereare monthly proportional variations in prices, and the expenditureshares have constant seasonal effects in addition to linear trends; seeAlterman, Diewert and Feenstra (1999, p. 65).

77Walsh (1921a, p. 98) called his test the circular test, but since Fisheralso used this term to describe his transitivity test defined earlier byequation (15.77), it seems best to stick to Fisher’s terminology since itis well established in the literature.78Walsh (1921b, pp. 540–541) noted that the time reversal test was aspecial case of his circularity test.79 This is essentially a variant of the methodology that Fisher (1922,p. 284) used to check how well various formulae corresponded to hisversion of the circularity test.

BASIC INDEX NUMBER THEORY

283

approximate method for deciding how ‘‘good’’ a par-ticular index number formula is. To decide whether tochain or use fixed base indices, look at how similar theobservations being compared are and choose the methodwhich will best link up the most similar observations.

15.97 Various properties, axioms or tests that anindex number formula could satisfy have been intro-duced in this chapter. In the following chapter, the testapproach to index number theory is studied in a moresystematic manner.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

284

Appendix 15.1 The relationshipbetween the Paasche andLaspeyres indices

1. Recall the notation used in paragraphs 15.11 to 15.17,above. Define the ith relative price or price relative ri and theith quantity relative ti as follows:

ri �p1ip0i; ti �

q1iq0i; i=1; . . . ; n (A15:1:1)

Using formula (15.8) for the Laspeyres price index PL anddefinitions (A15.1.1), we have:

PL=Pni=1

ris0i � r* (A15:1:2)

i.e., we define the ‘‘average’’ price relative r* as the base periodexpenditure share-weighted average of the individual pricerelatives, ri.

2. Using formula (15.6) for the Paasche price index PP, wehave:

PP �

Pni=1

p1i q1i

Pnm=1

p0mq1m

=

Pni=1

ritip0i q0i

Pnm=1

tmp0mq0m

using definitions (A15:1:1)

=

Pni=1

ritis0i

Pnm=1

tms0m

=1Pn

m=1

tms0m

Pni=1

(ri�r*)(ti�t*)s0i

8>><>>:

9>>=>>;+r*

(A15:1:3)

using (A15.1.2) andPn

i=1s0i=1 and where the ‘‘average’’

quantity relative t* is defined as

t* �Pni=1

tis0i=QL (A15:1:4)

where the last equality follows using equation (15.11), thedefinition of the Laspeyres quantity index QL.

3. Taking the difference between PP and PL and usingequations (A15.1.2)–(A15.1.4) yields:

PP�PL=1

QL

Pni=1

(ri�r*)(ti�t*)s0i (A15:1:5)

Now let r and t be discrete random variables that take on the nvalues ri and ti respectively. Let s

0i be the joint probability that

r=ri and t=ti for i=1, . . . , n and let the joint probability be 0if r=ri and t=tj where i=j. It can be verified that the sum-mation

Pni=1(ri�r*) (ti�t*)s0i on the right-hand side of equa-

tion (A15.1.5) is the covariance between the price relatives riand the corresponding quantity relatives ti. This covariancecan be converted into a correlation coefficient.80 If this co-variance is negative, which is the usual case in the consumercontext, then PP will be less than PL.

Appendix 15.2 The relationshipbetween the Lowe andLaspeyres indices

1. Recall the notation used in paragraphs 15.33 to 15.48,above. Define the ith relative price relating the price of com-modity i of month t to month 0, ri, and the ith quantity rela-tive, ti, relating quantity of commodity i in base year b tomonth 0 ti as follows:

ri �ptip0i

ti �qbiq0i; i=1; . . . ; n (A15:2:1)

As in Appendix A15.1, the Laspeyres price index PL(p0, pt, q0)

can be defined as r*, the month 0 expenditure share-weightedaverage of the individual price relatives ri defined in (A15.2.1)except that the month t price, pti , now replaces period 1 price,p1i , in the definition of the ith price relative ri:

r* �Pni=1

ris0i=PL (A15:2:2)

2. The ‘‘average’’ quantity relative t* relating the quantitiesof base year b to those of month 0 is defined as the month 0expenditure share-weighted average of the individual quantityrelatives ti, defined in (A15.2.1):

t* �Pni=1

tis0i=QL (A15:2:3)

where QL=QL(q0, qb, p0) is the Laspeyres quantity index

relating the quantities of month 0, q0, to those of the year b, qb,using the prices of month 0, p0, as weights.

3. Using definition (15.26), the Lowe index comparing theprices in month t to those of month 0, using the quantityweights of the base year b, is equal to:

PLo( p0, pt, qb) �

Pni=1

ptiqbi

Pni=1

p0i qbi

=

Pni=1

pti tiq0i

Pni=1

p0i tiq0i

using (A15:2:1)

=

Pni=1

pti tiq0i

Pni=1

p0i q0i

8>><>>:

9>>=>>;

Pni=1

p0i tiq0i

Pni=1

p0i q0i

8>><>>:

9>>=>>;�1

=

Pni=1

ptip0i

� �tip

0i q0i

Pni=1

p0i q0i

8>><>>:

9>>=>>;,t* using (A15:2:3)

=

Pni=1

ritip0i q0i

Pni=1

p0i q0i

8>><>>:

9>>=>>;,t* using (A15:2:1)

=

Pni=1

ritis0i

t*=

Pni=1

(ri�r*)tis0it*

+

Pni=1

r*tis0i

t*

=

Pni=1

(ri�r*)tis0i

t*+

r*Pni=1

ritis0i

� �t*

80 See Bortkiewicz (1923, pp. 374–375) for the first application of thiscorrelation coefficient decomposition technique.

BASIC INDEX NUMBER THEORY

285

=

Pni=1

(ri�r*)tis0i

t*+r*[t*]

t*using (A15:2:3)

=

Pni=1

(ri�r*)(ti�t*)s0i

t*+

Pni=1

(ri�r*)t*s0i

t*+r*

=

Pni=1

(ri�r*)(ti�t*)s0i

t*+

t*Pni=1

ris0i�r*

� �t*

+r*

=

Pni=1

(ri�r*)(ti�t*)s0it*

+r* sincePni=1

ris0i=r*

=PL( p0, pt, q0)+

Pni=1

(ri�r*)(ti�t*)s0i

QL(q0, qb, p0)

(A15:2:4)

since using (A15.2.2), r* equals the Laspeyres price index,PL(p

0, pt, q0), and using (A15.2.3), t* equals the Laspeyresquantity index, QL(q

0, qb, p0). Thus equation (A15.2.4) tells usthat the Lowe price index using the quantities of year b asweights, PLo(p

0, pt, qb), is equal to the usual Laspeyres indexusing the quantities of month 0 as weights, PL(p

0, pt, q0), plus acovariance term

Pni=1(ri�r*)(ti�t*)s0i between the price rela-

tives ri � pti=p0i and the quantity relatives ti � qbi =q0i , dividedby the Laspeyres quantity index QL(q

0, qb, p0) between month 0and base year b.

Appendix 15.3 The relationshipbetween the Young indexand its time antithesis

1. Recall that the direct Young index, PY(p0, pt, sb), was

defined by equation (15.48) and its time antithesis,PY* (p0, pt, sb), was defined by equation (15.52). Define the ith

relative price between months 0 and t as

ri � pti=p0i ; i=1, . . . , n (A15:3:1)

and define the weighted average (using the base year weightssbi ) of the ri as

r* �Pni=1

sbi ri (A15:3:2)

which turns out to equal the direct Young index, PY(p0, pt, sb).

Define the deviation ei of ri from their weighted average r*using the following equations:

ri=r*(1+ei); i=1, . . . , n (A15:3:3)

If equation (A15.3.3) is substituted into equation (A15.3.2),the following equation is obtained:

r* �Pni=1

sbi r*(1+ei)

=r*+r*Pni=1

sbi ei sincePni=1

sbi=1 (A15:3:4)

e* �Pni=1

sbi ei=0 (A15:3:5)

Thus the weighted mean e* of the deviations ei equals 0.2. The direct Young index, PY(p

0, pt, sb), and its timeantithesis, P*Y (p

0, pt, sb), can be written as functions of r*,the weights sbi and the deviations of the price relatives ei asfollows:

PY ( p0, pt, sb)=r* (A15:3:6)

P*Y (p0, pt, sb)=

Pni=1

sbi fr*(1+ei)g�1� ��1

=r*Pni=1

sbi (1+ei)�1

� ��1(A15:3:7)

3. Now regard P*Y ( p0, pt, sb) as a function of the vector of

deviations, e:[e1, . . . , en], say P*Y (e). The second-order Taylor

series approximation to P*Y (e) around the point e=0n is givenby the following expression:81

P*Y (e) � r*+r*Pni=1

sbi ei+r*Pni=1

Pnj=1

sbi sbj eiej�r*

Pni=1

sbi [ei]2

=r*+r*0+r*Pni=1

sbiPnj=1

sbj ej

" #ei�r*

Pni=1

sbi [ei�e*]2

using (A15:3:5)

81 This type of second-order approximation is attributable to Dalen(1992; 143) for the case r*=1 and to Diewert (1995a, p. 29) for the caseof a general r*.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

286

=r*+r*Pni=1

sbi [0]ei�r*Pni=1

sbi [ei�e*]2 using (A15:3:5)

=PY ( p0, pt, sb)�PY (p0, pt, sb)

Pni=1

sbi [ei�e*]2

using (A15:3:6)

=PY ( p0, pt, sb)�PY (p0, pt, sb)Var e

where the weighted sample variance of the vector e of pricedeviations is defined as

Var e �Pni=1

sbi [ei�e*]2 (A15:3:9)

4. Rearranging equation (A15.3.8) gives the followingapproximate relationship between the direct Young indexPY(p

0, pt, sb) and its time antithesis P*Y (p0, pt, sb), to the accu-

racy of a second-order Taylor series approximation about aprice point where the month t price vector is proportional tothe month 0 price vector:

PY ( p0, pt, sb) � P*Y ( p0, pt, sb)+PY ( p0, pt, sb)Var e

(A15:3:10)

Thus, to the accuracy of a second-order approximation, thedirect Young index will exceed its time antithesis by a termequal to the direct Young index times the weighted variance ofthe deviations of the price relatives from their weighted mean.Thus the bigger is the dispersion in relative prices, the more thedirect Young index will exceed its time antithesis.

Appendix 15.4 The relationshipbetween the Divisia andeconomic approaches

1. Divisia’s approach to index number theory relied on thetheory of differentiation. Thus it does not appear to have anyconnection with economic theory. However, starting with Ville(1946), a number of economists82 have established that theDivisia price and quantity indices do have a connection withthe economic approach to index number theory. This con-nection is outlined in this appendix.

2. The economic approach to the determination of theprice level and the quantity level is first outlined. The partic-ular economic approach that is used here is attributable toShephard (1953; 1970), Samuelson (1953) and Samuelson andSwamy (1974).

3. It is assumed that ‘‘the’’ consumer has well-definedpreferences over different combinations of the n consumercommodities or items. Each combination of items can berepresented by a positive vector q:[q1, . . . , qn]. The con-sumer’s preferences over alternative possible consumptionvectors q are assumed to be representable by a continuous,non-decreasing and concave utility function f. It is furtherassumed that the consumer minimizes the cost of achieving theperiod t utility level ut:f (qt) for periods t=0,1, . . . ,T. Thus itis assumed that the observed period t consumption vector qt

solves the following period t cost minimization problem:

C(ut, pt) � minqPni=1

ptiqi : f (q)=ut=f (qt)

� �

=Pni=1

ptiqti ; t=0, 1, . . . ,T (A15:4:1)

The period t price vector for the n commodities under con-sideration that the consumer faces is pt. Note that the solutionto the period t cost or expenditure minimization problemdefines the consumer’s cost function, C(ut, pt).

4. An additional regularity condition is placed on theconsumer’s utility function f. It is assumed that f is (positively)linearly homogeneous for strictly positive quantity vectors.Under this assumption, the consumer’s expenditure or costfunction, C(u, p), decomposes into uc(p) where c(p) is theconsumer’s unit cost function.83 The following equation isobtained:

Pni=1

ptiqti=c( pt)f (qt) for t=0, 1, . . . ,T (A15:4:2)

Thus the period t total expenditure on the n commodities inthe aggregate,

Pni=1p

tiqti , decomposes into the product of

two terms, c(pt) f(qt). The period t unit cost, c(pt), can beidentified as the period t price level Pt and the period t levelof utility, f(qt), can be identified as the period t quantitylevel Qt.

5. The economic price level for period t, Pt:c(pt), definedin the previous paragraph, is now related to the Divisia pricelevel for time t, P(t), that was implicitly defined by the differ-ential equation (15.67). As in paragraphs 15.65 to 15.71, thinkof the prices as being continuous, differentiable functions of

82 See for example Malmquist (1953, p. 227), Wold (1953, pp. 134–147), Solow (1957), Jorgenson and Griliches (1967) and Hulten (1973),and see Balk (2000a) for a recent survey of work on Divisia price andquantity indices.83 See Diewert (1993b, pp.120–121) for material on unit cost functions.This material will also be covered in Chapter 17.

BASIC INDEX NUMBER THEORY

287

time, pi(t) say, for i:1, . . . , n. Thus the unit cost function canbe regarded as a function of time t as well; i.e., define the unitcost function as a function of t as

c*(t) � c[ p1(t), p2(t), . . . , pn(t)] (A15:4:3)

6. Assuming that the first-order partial derivatives of theunit cost function c(p) exist, calculate the logarithmic deriva-tive of c*(t) as follows:

d ln c*(t)

dt� 1

c*(t)

dc*(t)

dt

=

Pni=1

ci[ p1(t), p2(t), . . . , pn(t)]p0i(t)

c[ p1(t), p2(t), . . . , pn(t)](A15:4:4)

where ci[p1(t), p2(t), . . . , pN(t)]:@c[p1(t), p2(t), . . . , pn(t)]/@pi isthe partial derivative of the unit cost function with respect tothe ith price, pi, and p

0i(t) � dpi(t)=dt is the time derivative of

the ith price function, pi(t). Using Shephard’s (1953, p. 11)Lemma, the consumer’s cost minimizing demand for com-modity i at time t is:

qi(t)=u(t)cifp1(t), p2(t), . . . , pn(t)g for i=1, . . . , n

(A15:4:5)

where the utility level at time t is u(t)=f [q1(t), q2(t), . . . , qn(t)].The continuous time counterpart to equations (A15.4.2) aboveis that total expenditure at time t is equal to total cost at time twhich in turn is equal to the utility level, u(t), times the period tunit cost, c*(t):

Pni=1

pi(t)qi(t)=u(t)c*(t)=u(t)cf p1(t), p2(t), . . . , pn(t)g

(A15:4:6)

7. The logarithmic derivative of the Divisia price level P(t)can be written as (recall equation (15.67) above):

P0(t)

P(t)=

Pni=1

p0i(t)qi(t)

Pni=1

pi(t)qi(t)

=

Pni=1

p0i(t)qi(t)

u(t)c*(t)using (A15:4:6)

=

Pni=1

p0i(t)[u(t)cfp1(t), p2(t), . . . , pn(t)g]

u(t)c*(t)using (A15:4:5)

=

Pni=1

cifp1(t), p2(t), . . . , pn(t)gp0i(t)

c*(t)=

1

c*(t)

dc*(t)

dt

using (A15:4:4)

� c*0(t)

c*(t)(A15:4:7)

Thus under the above continuous time cost-minimizingassumptions, the Divisia price level, P(t), is essentially equal tothe unit cost function evaluated at the time t prices,c*(t):c[p1(t), p2(t), . . . , pN(t)].

8. If the Divisia price level P(t) is set equal to the unit costfunction c*(t):c[p1(t), p2(t), . . . , pN(t)], then from equation(A15.4.2), it follows that the Divisia quantity level Q(t) definedby equation (15.68) will equal the consumer’s utility functionregarded as a function of time, f *(t):f [q1(t), . . . , qn(t)]. Thus,under the assumption that the consumer is continuouslyminimizing the cost of achieving a given utility level where theutility or preference function is linearly homogeneous, it hasbeen shown that the Divisia price and quantity levels P(t) andQ(t), defined implicitly by the differential equations (15.67) and(15.68), are essentially equal to the consumer’s unit costfunction c*(t) and utility function f *(t) respectively.84 Theseare rather remarkable equalities since in principle, given thefunctions of time, pi(t) and qi(t), the differential equations thatdefine the Divisia price and quantity indices can be solvednumerically and hence P(t) and Q(t) are in principle observable(up to some normalizing constants).

9. For more on the Divisia approach to index numbertheory, see Vogt (1977; 1978) and Balk (2000a). An alternativeapproach to Divisia indices using line integrals may be foundin the forthcoming companion volume Producer price indexmanual (Eurostat et al., 2004).

84Obviously, the scale of the utility and cost functions are not uniquelydetermined by the differential equations (15.62) and (15.63).

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

288

16THE AXIOMATIC AND STOCHASTICAPPROACHES TO INDEX NUMBER THEORY

Introduction16.1 As was seen in Chapter 15, it is useful to be able

to evaluate various index number formulae that havebeen proposed in terms of their properties. If a formulaturns out to have rather undesirable properties, thiscasts doubts on its suitability as an index that could beused by a statistical agency as a target index. Looking atthe mathematical properties of index number formulaeleads to the test or axiomatic approach to index numbertheory. In this approach, desirable properties for anindex number formula are proposed, and it is thenattempted to determine whether any formula is con-sistent with these properties or tests. An ideal outcome isthe situation where the proposed tests are both desirableand completely determine the functional form for theformula.16.2 The axiomatic approach to index number the-

ory is not completely straightforward, since choices haveto be made in two dimensions:

� The index number framework must be determined.

� Once the framework has been decided upon, it mustbe decided what tests or properties should be imposedon the index number.

The second point is straightforward: different pricestatisticians may have different ideas about which testsare important, and alternative sets of axioms can lead toalternative ‘‘best’’ index number functional forms. Thispoint must be kept in mind while reading this chapter,since there is no universal agreement on what the ‘‘best’’set of ‘‘reasonable’’ axioms is. Hence the axiomaticapproach can lead to more than one best index numberformula.16.3 The first point about choices listed above

requires further discussion. In the previous chapter, forthe most part, the focus was on bilateral index numbertheory; i.e., it was assumed that prices and quantities forthe same n commodities were given for two periods andthe object of the index number formula was to comparethe overall level of prices in one period with the otherperiod. In this framework, both sets of price andquantity vectors were regarded as variables which couldbe independently varied so that, for example, variationsin the prices of one period did not affect the prices of theother period or the quantities in either period. Theemphasis was on comparing the overall cost of a fixedbasket of quantities in the two periods or takingaverages of such fixed basket indices. This is an exampleof an index number framework.16.4 However, other index number frameworks are

possible. For example, instead of decomposing a value

ratio into a term that represents price change betweenthe two periods times another term that representsquantity change, an attempt could be made to decom-pose a value aggregate for one period into a singlenumber that represents the price level in the period timesanother number that represents the quantity level in theperiod. In the first variant of this approach, the priceindex number is supposed to be a function of the ncommodity prices pertaining to that aggregate in theperiod under consideration, while the quantity indexnumber is supposed to be a function of the n commodityquantities pertaining to the aggregate in the period. Theresulting price index function was called an absoluteindex number by Frisch (1930, p. 397), a price level byEichhorn (1978, p. 141) and a unilateral price index byAnderson, Jones and Nesmith (1997, p. 75). In a secondvariant of this approach, the price and quantity func-tions are allowed to depend on both the price andquantity vectors pertaining to the period under con-sideration.1 These two variants of unilateral indexnumber theory will be considered in paragraphs 16.11to 16.29.2

16.5 The remaining approaches in this chapter arelargely bilateral approaches; i.e., the prices and quan-tities in an aggregate are compared for two periods.In paragraphs 16.30 to 16.73 and 16.94 to 16.129,the value ratio decomposition approach is taken.3 Inparagraphs 16.30 to 16.73, the bilateral price andquantity indices, P( p0, p1, q0, q1) and Q( p0, p1, q0, q1),are regarded as functions of the price vectors pertainingto the two periods, p0 and p1, and the two quantityvectors, q0 and q1. Not only do the axioms or tests thatare placed on the price index P( p0, p1, q0, q1) reflect‘‘reasonable’’ price index properties, but some testshave their origin as ‘‘reasonable’’ tests on the quan-tity index Q( p0, p1, q0, q1). The approach in paragraphs16.30 to 16.73 simultaneously determines the ‘‘best’’price and quantity indices.

16.6 In paragraphs 16.74 to 16.93, attention is shif-ted to the price ratios for the n commodities between

1 Eichhorn (1978, p. 144) and Diewert (1993d, p. 9) considered thisapproach.2 In these unilateral index number approaches, the price and quan-tity vectors are allowed to vary independently. In yet another indexnumber framework, prices are allowed to vary freely but quantitiesare regarded as functions of the prices. This leads to the economicapproach to index number theory, which is considered briefly inAppendix 15.4 of Chapter 15, and in more depth in Chapters 17and 18.3 Recall paragraphs 15.7 to 15.17 of Chapter 15 for an explanation ofthis approach.

289

periods 0 and 1, ri � p1i =p

0i for i=1, . . . , n. In the

unweighted stochastic approach to index number theory,the price index is regarded as an evenly weightedaverage of the n price relatives or ratios, ri. Carli(1764) and Jevons (1863; 1865) were the earlier pio-neers in this approach to index number theory, withCarli using the arithmetic average of the price relativesand Jevons endorsing the geometric average (but alsoconsidering the harmonic average). This approach toindex number theory will be covered in paragraphs16.74 to 16.79. This approach is consistent with astatistical approach that regards each price ratio ri asa random variable with mean equal to the underlyingprice index.

16.7 A major problem with the unweighted aver-age of price relatives approach to index number the-ory is that this approach does not take into accountthe economic importance of the individual commod-ities in the aggregate. Young (1812) did advocatesome form of rough weighting of the price relativesaccording to their relative value over the period beingconsidered, but the precise form of the required valueweighting was not indicated.4 It was Walsh (1901, pp.83–121; 1921a, pp. 81–90), however, who stressed theimportance of weighting the individual price ratios,where the weights are functions of the associatedvalues for the commodities in each period and eachperiod is to be treated symmetrically in the resultingformula:

What we are seeking is to average the variations in theexchange value of one given total sum of money in rela-tion to the several classes of goods, to which severalvariations [price ratios] must be assigned weights pro-portional to the relative sizes of the classes. Hence therelative sizes of the classes at both the periods must beconsidered (Walsh (1901, p. 104)).

Commodities are to be weighted according to theirimportance, or their full values. But the problem ofaxiometry always involves at least two periods. There is afirst period and there is a second period which is com-pared with it. Price variations5 have taken place betweenthe two, and these are to be averaged to get the amount oftheir variation as a whole. But the weights of the com-modities at the second period are apt to be different fromtheir weights at the first period. Which weights, then, arethe right ones – those of the first period or those of thesecond? Or should there be a combination of the two sets?There is no reason for preferring either the first or thesecond. Then the combination of both would seem to bethe proper answer. And this combination itself involvesan averaging of the weights of the two periods (Walsh(1921a, p. 90)).

16.8 Thus Walsh was the first to examine in somedetail the rather intricate problems6 involved in decidinghow to weight the price relatives pertaining to anaggregate, taking into account the economic importanceof the commodities in the two periods being considered.Note that the type of index number formula that Walshwas considering was of the form P(r, v0, v1), where r isthe vector of price relatives which has ith componentri=p1

i =p0i and vt is the period t value vector which has

ith component vti=ptiqti for t=0, 1. His suggested solu-

tion to this weighting problem was not completelysatisfactory but he did at least suggest a very usefulframework for a price index, as a value-weighted aver-age of the n price relatives. The first satisfactory solu-tion to the weighting problem was obtained by Theil(1967, pp. 136–137) and his solution is explained inparagraphs 16.79 to 16.93.

16.9 It can be seen that one of Walsh’s approachesto index number theory7 was an attempt to determinethe ‘‘best’’ weighted average of the price relatives, ri.This is equivalent to using an axiomatic approach to tryto determine the ‘‘best’’ index of the form P(r, v0, v1).This approach is considered in paragraphs 16.94to 16.129.8

16.10 The Young and Lowe indices, discussed inChapter 15, do not fit precisely into the bilateral frame-work since the value or quantity weights used in theseindices do not necessarily correspond to the values orquantities that pertain to either of the periods thatcorrespond to the price vectors p0 and p1. The axio-matic properties of these two indices with respect totheir price variables are studied in paragraphs 16.130to 16.134.

4 Walsh (1901, p. 84) refers to Young’s contributions as follows:

Still, although few of the practical investigators have actually employedanything but even weighting, they have almost always recognized thetheoretical need of allowing for the relative importance of the differentclasses ever since this need was first pointed out, near the commence-ment of the century just ended, by Arthur Young. . . . Arthur Youngadvised simply that the classes should be weighted according to theirimportance.

5 A price variation is a price ratio or price relative in Walsh’s termi-nology.

6 Walsh (1901, pp. 104–105) realized that it would not do simply totake the arithmetic average of the values in the two periods, [v0

i +v1i ]=2,

as the ‘‘correct’’ weight for the ith price relative ri since, in a period ofrapid inflation, this would give too much importance to the period thathad the highest prices and he wanted to treat each period symme-trically:

But such an operation is manifestly wrong. In the first place, the sizes ofthe classes at each period are reckoned in the money of the period, and ifit happens that the exchange value of money has fallen, or prices ingeneral have risen, greater influence upon the result would be given to theweighting of the second period; or if prices in general have fallen, greaterinfluence would be given to the weighting of the second period. Or in acomparison between two countries greater influence would be given to theweighting of the country with the higher level of prices. But it is plain thatthe one period, or the one country, is as important, in our comparisonbetween them, as the other, and the weighting in the averaging of theirweights should really be even.

However, Walsh was unable to come up with Theil’s (1967) solutionto the weighting problem, which was to use the average expenditureshare [s0i +s1i ]=2, as the ‘‘correct’’ weight for the ith price relativein the context of using a weighted geometric mean of the pricerelatives.7 Walsh also considered basket-type approaches to index numbertheory, as was seen in Chapter 15.8 In paragraphs 16.94 to 16.129, rather than starting with indices of theform P(r, v0, v1), indices of the form P( p0, p1, v0, v1) are considered.However, if the test of invariance to changes in the units of measure-ment is imposed on this index, it is equivalent to studying indices of theform P(r, v0, v1). Vartia (1976) also used a variation of this approach toindex number theory.

CONSUMER PRICE INDEX: THEORY AND PRACTICE

290

The levels approach to indexnumber theory

An axiomatic approach to unilateralprice indices16.11 Denote the price and quantity of commodity

n in period t by pti and qti respectively for i=1, 2, . . . , nand t=0, 1, . . . ,T. The variable qti is interpreted as thetotal amount of commodity i transacted within periodt. In order to conserve the value of transactions, it isnecessary that pti be defined as a unit value; i.e., ptimust be equal to the value of transactions in com-modity i for period t divided by the total quantitytransacted, qti . In principle, the period of time shouldbe chosen so that variations in commodity priceswithin a period are very small compared to theirvariations between periods.9 For t=0, 1, . . . ,T, andi=1, . . . , n, define the value of transactions in com-modity i as vti � ptiq

ti and define the total value of

transactions in period t as:

Vt �Pni=1

vti=Pni=1

ptiqti t=0, 1, . . . ,T (16:1)

16.12 Using the above notation, the followinglevels version of the index number problem is defined asfollows: for t=0, 1, . . . ,T, find scalar numbers Pt andQt such that

Vt=PtQt t=0, 1, . . . ,T (16:2)

The number Pt is interpreted as an aggregate period tprice level, while the number Qt is interpreted as anaggregate period t quantity level. The aggregate pricelevel Pt is allowed to be a function of the period tprice vector, pt, while the aggregate period t quantitylevel Qt is allowed to be a function of the period t

quantity vector, qt; hence:

Pt=c( pt) and Qt=f (qt) t=0, 1, . . . ,T (16:3)

16.13 The functions c and f are to be determinedsomehow. Note that equation (16.3) requires that thefunctional forms for the price aggregation function cand for the quantity aggregation function f be inde-pendent of time. This is a reasonable requirement sincethere is no reason to change the method of aggregationas time changes.

16.14 Substituting equations (16.3) and (16.2) intoequation (16.1) and dropping the superscripts t meansthat c and f must satisfy the following functional equa-tion for all strictly positive price and quantity vectors:

c( p) f (q)=Pni=1

piqi for all pi > 0 and for all qi > 0

(16:4)

16.15 It is natural to assume that the functions c( p)and f(q) are positive if all prices and quantities arepositive:

c( p1, . . . , pn) > 0; f (q1, . . . , qn) > 0

if all pi > 0 and all qi > 0 (16:5)

16.16 Let 1n denote an n-dimensional vector of ones.Then (16.5) implies that when p=1n, c(1n) is a positivenumber, a for example, and when q=1n, then f(1n) is alsoa positive number, b for example; i.e., (16.5) implies thatc and f satisfy:

c(1n)=a > 0; f (1n)=b > 0 (16:6)

16.17 Let p=1n and substitute the first equation in(16.6) into equation (16.4) in order to obtain the fol-lowing equation:

f (q)=Pni=1

qi

afor all qi > 0 (16:7)

16.18 Now let q=1n and substitute the secondequation in (16.6) into equation (16.4) in order to obtainthe following equation:

c( p)=Pni=1

pi

bfor all pi > 0 (16:8)

16.19 Finally substitute equations (16.7) and (16.8)into the left-hand side of equation (16.4) to obtain thefollowing equation:

Pni=1

pi

b

� � Pni=1

qi

a

� �=Pni=1

piqi for all pi > 0

and for all qi > 0 (16:9)

If n is greater than one, it is obvious that equation (16.9)cannot be satisfied for all strictly positive p and q vec-tors. Thus if the number of commodities n exceeds one,then there do not exist any functions c and f that satisfyequations (16.4) and (16.5).10

9 This treatment of prices as unit values over time follows Walsh (1901,p. 96; 1921a, p. 88) and Fisher (1922, p. 318). Fisher and Hicks bothhad the idea that the length of the period should be short enough sothat variations in price within the period could be ignored, as thefollowing quotations indicate:

Throughout this book ‘‘the price’’ of any commodity or ‘‘the quantity’’of it for any one year was assumed given. But what is such a price orquantity? Sometimes it is a single quotation for January 1 or July 1, butusually it is an average of several quotations scattered throughout theyear. The question arises: On what principle should this average be con-structed? The practical answer is any kind of average since, ordinarily, thevariations during a year, so far, at least, as prices are concerned, are toolittle to make any perceptible difference in the result, whatever kind ofaverage is used. Otherwise, there would be ground for subdividing theyear into quarters or months until we reach a small enough period to beconsidered practically a point. The quantities sold will, of course, varywidely. What is needed is their sum for the year (which, of course, is thesame thing as the simple arithmetic average of the per annum rates for theseparate months or other subdivisions). In short, the simple arithmeticaverage, both of prices and of quantities, may be used. Or, if it is worthwhile to put any finer point on it, we may take the weighted arithmeticaverage for the prices, the weights being the quantities sold (Fisher (1922,p. 318)).

I shall define a week as that period of time during which variations inprices can be neglected. For theoretical purposes this means that priceswill be supposed to change, not continuously, but at short intervals. Thecalendar length of the week is of course quite arbitrary; by taking it to bevery short, our theoretical scheme can be fitted as closely as we like to thatceaseless oscillation which is a characteristic of prices in certain markets(Hicks (1946, p. 122)).

10 Eichhorn (1978, p. 144) established this result.

THE AXIOMATIC AND STOCHASTIC APPROACHES TO INDEX NUMBER THEORY

291

16.20 Thus this levels test approach to index numbertheory comes to an abrupt halt; it is fruitless to look forprice and quantity level functions, Pt=c( pt) and Qt=f(qt),that satisfy equations (16.2) or (16.4) and also satisfy thevery reasonable positivity requirements (16.5).

16.21 Note that the levels price index function, c( pt),did not depend on the corresponding quantity vector qt

and the levels quantity index function, f (qt), did notdepend on the price vector pt. Perhaps this is the reasonfor the rather negative result obtained above. Hence, inthe next section, the price and quantity functions areallowed to be functions of both pt and qt.

A second axiomatic approach tounilateral price indices

16.22 In this section, the goal is to find functions of2n variables, c( p, q) and f( p, q), such that the followingcounterpart to equation (16.4) holds:

c( p, q)f ( p, q)=Pni=1

piqi for all pi > 0

and for all qi > 0 (16:10)

16.23 Again, it is natural to assume that the func-tions c( p, q) and f( p, q) are positive if all prices andquantities are positive:

c(p1, . . . ,pn;q1, . . . ,qn)> 0; f (p1, . . . ,pn;q1, . . . ,qn)> 0

if all pi > 0 and all qi > 0 (16:11)

16.24 The present framework does not distinguishbetween the functions c and f, so it is necessary torequire that these functions satisfy some ‘‘reasonable’’properties. The first property imposed on c is that thisfunction be homogeneous of degree one in its pricecomponents:

c(lp, q)=lc( p, q) for all l > 0 (16:12)

Thus, if all prices are multiplied by the positive numberl, then the resulting price index is l times the initial priceindex. A similar linear homogeneity property is imposedon the quantity index f; i.e., f is to be homogeneous ofdegree one in its quantity components:

f ( p, lq)=l f ( p, q) for all l > 0 (16:13)

16.25 Note that properties (16.10), (16.11) and(16.13) imply that the price index c( p, q) has the fol-lowing homogeneity property with respect to the com-ponents of q:

c( p, lq)=Pni=1

pilqif ( p, lq)

where l > 0

=Pni=1

pilqilf ( p, q)

using (16:3)

=Pni=1

piqi

f ( p, q)

=c( p, q) using (16:10) and (16:11) (16:14)

Thus c( p, q) is homogeneous of degree zero in its qcomponents.

16.26 A final property that is imposed on the levelsprice index c( p, q) is the following one. Let the positivenumbers di be given. Then it is asked that the price indexbe invariant to changes in the units of measurement forthe n commodities so that the function c( p, q) has thefollowing property:

c(d1p1, . . . ,dnpn;q1=d1, . . . ,qn=dn)

=c(p1, . . . ,pn; q1, . . . ,qn) (16:15)

16.27 It is now possible to show that properties(16.10), (16.11), (16.12), (16.14) and (16.15) on the pricelevels function c( p, q) are inconsistent; i.e., there doesnot exist a function of 2n variables c( p, q) that satisfiesthese very reasonable properties.11

16.28 To see why this is so, apply the equation(16.15), setting di=qi for each i, to obtain the followingequation:

c( p1, . . . , pn; q1, . . . , qn)=c( p1q1, . . . , pnqn;1, . . . , 1)

(16:16)

If c( p, q) satisfies the linear homogeneity property (16.12)so that c(lp, q)=lc( p, q), then equation (16.16) impliesthat c( p, q) is also linearly homogeneous in q so thatc( p, lq)=lc( p, q). But this last equation contradictsequation (16.14), which establishes the impossibility result.

16.29 The rather negative results obtained in para-graphs 16.13 to 16.21 indicate that it is fruitless topursue the axiomatic approach to the determination ofprice and quantity levels, where both the price andquantity vector are regarded as independent variables.12

Hence, in the following sections of this chapter, theaxiomatic approach to the determination of a bilateralprice index of the form P( p0, p1, q0, q1) will be pursued.

The first axiomatic approach tobilateral price indices

Bilateral indices and some early tests16.30 In this section, the strategy will be to assume

that the bilateral price index formula, P( p0, p1, q0, q1),satisfies a sufficient number of ‘‘reasonable’’ tests orproperties so that the functional form for P is deter-mined.13 The word ‘‘bilateral’’14 refers to the assump-tion that the function P depends only on thedata pertaining to the two situations or periods beingcompared; i.e., P is regarded as a function of thetwo sets of price and quantity vectors, p0, p1, q0, q1, that

11 This proposition is due to Diewert (1993d, p. 9), but his proof isan adaptation of a closely related result due to Eichhorn (1978, pp.144–145).12 Recall that in the economic approach, the price vector p is allowed tovary independently, but the corresponding quantity vector q is regar-ded as being determined by p.13 Much of the material in this section is drawn from sections 2 and 3of Diewert (1992a). For more recent surveys of the axiomatic approachsee Balk (1995) and von Auer (2001).14 Multilateral index number theory refers to the case where there aremore than two situations whose prices and quantities need to beaggregated.

CONSUMER PRICE INDEX: THEORY AND PRACTICE

292

are to be aggregated into a single number that sum-marizes the overall change in the n price ratios,p1

1=p01, . . . , p1

n=p0n.

16.31 In this section, the value ratio decompositionapproach to index number theory will be taken; i.e.,along with the price index P( p0, p1, q0, q1), thereis a companion quantity index Q(p0, p1, q0, q1) such thatthe product of these two indices equals the value ratiobetween the two periods.15 Thus, throughout this sec-tion, it is assumed that P and Q satisfy the followingproduct test:

V1=V0=P( p0, p1, q0, q1) Q( p0, p1, q0, q1): (16:17)

The period t values, V t, for t=0, 1 are defined byequation (16.1). As soon as the functional form for theprice index P is determined, then equation (16.17) can beused to determine the functional form for the quantityindex Q. A further advantage of assuming that theproduct test holds is that, if a reasonable test is imposedon the quantity index Q, then equation (16.17) canbe used to translate this test on the quantity indexinto a corresponding test on the price index P.16

16.32 If n=1, so that there is only one price andquantity to be aggregated, then a natural candidate for Pis p1

1=p01, the single price ratio, and a natural candidate for

Q is q11=q

01, the single quantity ratio. When the number of

commodities or items to be aggregated is greater than 1,then what index number theorists have done over theyears is propose properties or tests that the price index Pshould satisfy. These properties are generally multi-dimensional analogues to the one good price index for-mula, p1

1=p01. Below, some 20 tests are listed that turn out

to characterize the Fisher ideal price index.16.33 It will be assumed that every component of

each price and quantity vector is positive; i.e., pt� 0nand qt� 0n

17 for t=0, 1. If it is desired to set q0=q1,the common quantity vector is denoted by q; if it isdesired to set p0=p1, the common price vector is de-noted by p.16.34 The first two tests, denoted T1 and T2, are

not very controversial, so they will not be discussed indetail.

T1: Positivity:18 P( p0, p1, q0, q1) > 0

T2: Continuity:19 P( p0, p1, q0, q1) is a continuousfunction of its arguments

16.35 The next two tests, T3 and T4, are somewhatmore controversial.

T3: Identity or constant prices test:20

P( p, p, q0, q1)=1

That is, if the price of every good is identical during thetwo periods, then the price index should equal unity, nomatter what the quantity vectors are. The controversialaspect of this test is that the two quantity vectors areallowed to be different in the test.21

T4: Fixed basket or constant quantities test:22

P( p0, p1, q, q)=

Pni=1

p1i qi

Pni=1

p0i qi

That is, if quantities are constant during the two periodsso that q0=q1:q, then the price index should equal theexpenditure on the constant basket in period 1,Pn

i=1p1i qi, divided by the expenditure on the basket in

period 0,Pn

i=1p0i qi.

16.36 If the price index P satisfies Test T4 and Pand Q jointly satisfy the product test (16.17) above,then it is easy to show23 that Q must satisfy the identitytest Q( p0, p1, q, q)=1 for all strictly positive vectorsp0, p1, q. This constant quantities test for Q is alsosomewhat controversial since p0 and p1 are allowed tobe different.

Homogeneity tests16.37 The following four tests, T5–T8, restrict the

behaviour of the price index P as the scale of any one ofthe four vectors p0, p1, q0, q1 changes.

T5: Proportionality in current prices:24

P( p0, lp1, q0, q1)=lP( p0, p1, q0, q1) for l> 0

That is, if all period 1 prices are multiplied by thepositive number l, then the new price index is l times

15 See paragraphs 15.7 to 15.25 of Chapter 15 for more on thisapproach, which was initially due to Fisher (1911, p. 403; 1922).16 This observation was first made by Fisher (1911, pp. 400–406), andthe idea was pursued by Vogt (1980) and Diewert (1992a).17 The notation q� 0n means that each component of the vector q ispositive; q� 0n means each component of q is non-negative and q>0nmeans q� 0n and q=0n.18 Eichhorn and Voeller (1976, p. 23) suggested this test.19 Fisher (1922, pp. 207–215) informally suggested the essence of thistest.

20 Laspeyres (1871, p. 308), Walsh (1901, p. 308) and Eichhorn andVoeller (1976, p. 24) have all suggested this test. Laspeyres came upwith this test or property to discredit the ratio of unit values index ofDrobisch (1871a), which does not satisfy this test. This test is also aspecial case of Fisher’s (1911, pp. 409–410) price proportionality test.21 Usually, economists assume that, given a price vector p, the corre-sponding quantity vector q is uniquely determined. Here, the sameprice vector is used but the corresponding quantity vectors are allowedto be different.22 The origins of this test go back at least 200 years to the Massa-chusetts legislature, which used a constant basket of goods to index thepay of Massachusetts soldiers fighting in the American Revolution; seeWillard Fisher (1913). Other researchers who have suggested the testover the years include: Lowe (1823, Appendix, p. 95), Scrope (1833,p. 406), Jevons (1865), Sidgwick (1883, pp. 67–68), Edgeworth (1925,p. 215) originally published in 1887, Marshall (1887, p. 363), Pierson(1895, p. 332), Walsh (1901, p. 540; 1921b, pp. 543–544), and Bowley(1901, p. 227). Vogt and Barta (1997, p. 49) correctly observe that thistest is a special case of Fisher’s (1911, p. 411) proportionality test forquantity indexes which Fisher (1911, p. 405) translated into a test forthe price index using the product test (15.3).23 See Vogt (1980, p. 70).24 This test was proposed by Walsh (1901, p. 385), Eichhorn andVoeller (1976, p. 24) and Vogt (1980, p. 68).

THE AXIOMATIC AND STOCHASTIC APPROACHES TO INDEX NUMBER THEORY

293

the old price index. Put another way, the price indexfunction P( p0, p1, q0, q1) is (positively) homogeneousof degree one in the components of the period 1 pricevector p1. Most index number theorists regard thisproperty as a very fundamental one that the indexnumber formula should satisfy.

16.38 Walsh (1901) and Fisher (1911, p. 418; 1922,p. 420) proposed the related proportionality testP( p, lp, q0, q1)=l. This last test is a combination of T3and T5; in fact Walsh (1901, p. 385) noted that this lasttest implies the identity test, T3.

16.39 In the next test, instead of multiplying allperiod 1 prices by the same number, all period 0 pricesare multiplied by the number l.

T6: Inverse proportionality in base period prices:25

P(lp0, p1, q0, q1)=l�1P( p0, p1, q0, q1) for l> 0

That is, if all period 0 prices are multiplied by the positivenumber l, then the new price index is 1/l times the oldprice index. Put another way, the price index functionP( p0, p1, q0, q1) is (positively) homogeneous of degreeminus one in the components of the period 0 price vectorp0.

16.40 The following two homogeneity tests can alsobe regarded as invariance tests.

T7: Invariance to proportional changes incurrent quantities:

P( p0, p1, q0, lq1)=P( p0, p1, q0, q1) for all l > 0

That is, if current period quantities are all multiplied bythe number l, then the price index remains unchanged.Put another way, the price index function P( p0, p1,q0, q1) is (positively) homogeneous of degree zero inthe components of the period 1 quantity vector q1.Vogt (1980, p. 70) was the first to propose thistest26 and his derivation of the test is of some interest.Suppose the quantity index Q satisfies the quantityanalogue to the price test T5; i.e., suppose Q satisfiesQ( p0, p1, q0, lq1)=lQ( p0, p1, q0, q1) for l>0. Then,using the product test (16.17), it can be seen that P mustsatisfy T7.

T8: Invariance to proportional changes in basequantities:27

P( p0, p1, lq0, q1)=P( p0, p1, q0, q1) for all l > 0

That is, if base period quantities are all multiplied by thenumber l, then the price index remains unchanged. Putanother way, the price index function P( p0, p1, q0, q1) is(positively) homogeneous of degree zero in the compo-nents of the period 0 quantity vector q0. If the quantityindex Q satisfies the following counterpart to T8: Q( p0,p1, lq0, q1)=l�1Q( p0, p1, q0, q1) for all l>0, then using

equation (16.17), the corresponding price index P mustsatisfy T8. This argument provides some additionaljustification for assuming the validity of T8 for the priceindex function P.

16.41 T7 and T8 together impose the property thatthe price index P does not depend on the absolutemagnitudes of the quantity vectors q0 and q1.

Invariance and symmetry tests16.42 The next five tests, T9–T13, are invariance or

symmetry tests. Fisher (1922, pp. 62–63, 458–460) andWalsh (1901, p. 105; 1921b, p. 542) seem to have beenthe first researchers to appreciate the significance ofthese kinds of tests. Fisher (1922, pp. 62–63) spoke offairness but it is clear that he had symmetry proper-ties in mind. It is perhaps unfortunate that he didnot realize that there were more symmetry and invar-iance properties than the ones he proposed; if he had,it is likely that he would have been able to providean axiomatic characterization for his ideal priceindex, as is done in paragraphs 16.53 to 16.56. Thefirst invariance test is that the price index shouldremain unchanged if the ordering of the commodities ischanged:

T9: Commodity reversal test (or invariance tochanges in the ordering of commodities):

P( p0*, p1*, q0*, q1*)=P( p0, p1, q0, q1)

where pt* denotes a permutation of the components ofthe vector pt and qt* denotes the same permutationof the components of qt for t=0,1. This test is attribu-table to Fisher (1922, p. 63)28 and it is one of his threefamous reversal tests. The other two are the timereversal test and the factor reversal test, which areconsidered below.

16.43 The next test asks that the index be invariantto changes in the units of measurement.

T10: Invariance to changes in the units ofmeasurement (commensurability test):

P(a1p01, . . . , anp0

n; a1p11, . . . , anp1

n; a�11 q0

1, . . . , a�1n q0

n;

a�11 q1

1, . . . , a�1n q1

n)=P( p01, . . . , p0

n; p11, . . . , p1

n;

q01, . . . , q0

n; q11, . . . , q1

n) for all a1 > 0, . . . , an > 0

That is, the price index does not change if the units ofmeasurement for each commodity are changed. Theconcept of this test is attributable to Jevons (1863, p. 23)and the Dutch economist Pierson (1896, p. 131), whocriticized several index number formulae for not satis-fying this fundamental test. Fisher (1911, p. 411) firstcalled this test the change of units test; later, Fisher(1922, p. 420) called it the commensurability test.

25 Eichhorn and Voeller (1976, p. 28) suggested this test.26 Fisher (1911, p. 405) proposed the related test P( p0, p1, q0,lq0)=P( p0, p1, q0, q0)=

Pni=1p

1i q

0i =Pn

i=1p0i q

0i :

27 This test was proposed by Diewert (1992a, p. 216).

28 ‘‘This [test] is so simple as never to have been formulated. It is merelytaken for granted and observed instinctively. Any rule for averagingthe commodities must be so general as to apply interchangeably to allof the terms averaged’’ (Fisher (1922, p. 63)).

CONSUMER PRICE INDEX: THEORY AND PRACTICE

294

16.44 The next test asks that the formula be invar-iant to the period chosen as the base period.

T11: Time reversal test:

P( p0, p1, q0, q1)=1=P( p1, p0, q1, q0)

That is, if the data for periods 0 and 1 are interchanged,then the resulting price index should equal the reciprocalof the original price index. Obviously, in the one goodcase when the price index is simply the single price ratio,this test will be satisfied (as are all the other tests listed inthis section). When the number of goods is greater thanone, many commonly used price indices fail this test;e.g., the Laspeyres (1871) price index, PL defined byequation (15.5) in Chapter 15, and the Paasche (1874)price index, PP defined by equation (15.6) in Chapter 15,both fail this fundamental test. The concept of the test isattributable to Pierson (1896, p. 128), who was so upsetby the fact that many of the commonly used indexnumber formulae did not satisfy this test that he pro-posed that the entire concept of an index number shouldbe abandoned. More formal statements of the test weremade by Walsh (1901, p. 368; 1921b, p. 541) and Fisher(1911, p. 534; 1922, p. 64).16.45 The next two tests are more controversial,

since they are not necessarily consistent with the eco-nomic approach to index number theory. These testsare, however, quite consistent with the weighted sto-chastic approach to index number theory, discussedlater in this chapter.

T12: Quantity reversal test (quantity weightssymmetry test):

P( p0, p1, q0, q1)=P( p0, p1, q1, q0)

That is, if the quantity vectors for the two periods areinterchanged, then the price index remains invariant.This property means that if quantities are used to weightthe prices in the index number formula, then the period0 quantities q0 and the period 1 quantities q1 must enterthe formula in a symmetric or even-handed manner.Funke and Voeller (1978, p. 3) introduced this test; theycalled it the weight property.16.46 The next test is the analogue to T12 applied to

quantity indices:

T13: Price reversal test (price weights symmetry test):29

Pni=1

p1i q

1i

Pni=1

p0i q

0i

0BB@

1CCA,

P(p0,p1,q0,q1)=

Pni=1

p0i q

1i

Pni=1

p1i q

0i

0BB@

1CCA,

P(p1, p0, q0, q1)

(16:18)

Thus if we use equation (16.17) to define the quantityindex Q in terms of the price index P, then it can be seen

that T13 is equivalent to the following property for theassociated quantity index Q:

Q( p0, p1, q0, q1)=Q( p1, p0, q0, q1) (16:19)

That is, if the price vectors for the two periods areinterchanged, then the quantity index remains invari-ant. Thus if prices for the same good in the twoperiods are used to weight quantities in the constructionof the quantity index, then property T13 implies thatthese prices enter the quantity index in a symmetricmanner.

Mean value tests16.47 The next three tests, T14–T16, are mean value

tests.

T14: Mean value test for prices:30

mini ( p1i =p

0i : i=1, . . . , n) � P( p0, p1, q0, q1)

� maxi ( p1i =p

0i : i=1, . . . , n) (16:20)

That is, the price index lies between the minimum priceratio and the maximum price ratio. Since the price indexis supposed to be interpreted as some sort of an averageof the n price ratios, p1

i =p0i , it seems essential that the

price index P satisfy this test.16.48 The next test is the analogue to T14 applied to

quantity indices:

T15: Mean value test for quantities:31

mini (q1i =q

0i : i=1, . . . , n) � (V1=V 0)

P( p0, p1, q0, q1)

� maxi (q1i =q

0i : i=1, . . . , n) (16:21)

where V t is the period t value for the aggregate definedby equation (16.1). Using the product test (16.17) todefine the quantity index Q in terms of the price index P,it can be seen that T15 is equivalent to the followingproperty for the associated quantity index Q:

mini (q1i =q

0i : i=1, . . . , n) � Q( p0, p1, q0, q1)

� maxi (q1i =q

0i : i=1, . . . , n) (16:22)

That is, the implicit quantity index Q defined by P liesbetween the minimum and maximum rates of growthq1i =q

0i of the individual quantities.

16.49 In paragraphs 15.18 to 15.32 of Chapter 15, itwas argued that it is very reasonable to take an averageof the Laspeyres and Paasche price indices as a single‘‘best’’ measure of overall price change. This point of

29 This test was proposed by Diewert (1992a, p. 218).

30 This test seems to have been first proposed by Eichhorn and Voeller(1976, p. 10).31 This test was proposed by Diewert (1992a, p. 219).

THE AXIOMATIC AND STOCHASTIC APPROACHES TO INDEX NUMBER THEORY

295

view can be turned into a test:

T16: Paasche and Laspeyres bounding test:32

The price index P lies between the Laspeyres andPaasche indices, PL and PP, defined by equations (15.5)and (15.6) in Chapter 15.A test could be proposed where the implicit quantityindex Q that corresponds to P via equation (16.17) is tolie between the Laspeyres and Paasche quantity indices,QP and QL, defined by equations (15.10) and (15.11) inChapter 15. However, the resulting test turns out to beequivalent to test T16.

Monotonicity tests16.50 The final four tests, T17–T20, are mono-

tonicity tests; i.e., how should the price indexP( p0, p1, q0, q1) change as any component of the twoprice vectors p0 and p1 increases or as any component ofthe two quantity vectors q0 and q1 increases?

T17: Monotonicity in current prices:

P( p0, p1, q0, q1)< P( p0, p2, q0, q1) if p1 < p2

That is, if some period 1 price increases, then theprice index must increase, so that P( p0, p1, q0, q1) isincreasing in the components of p1. This property wasproposed by Eichhorn and Voeller (1976, p. 23) andit is a very reasonable property for a price index tosatisfy.

T18: Monotonicity in base prices: P( p0, p1, q0, q1)>

P( p2, p1, q0, q1) if p0 < p2

That is, if any period 0 price increases, then the priceindex must decrease, so that P( p0, p1, q0, q1) is decreas-ing in the components of p0. This very reasonableproperty was also proposed by Eichhorn and Voeller(1976, p. 23).

T19: Monotonicity in current quantities:

if q1 < q2, then

Pni=1

p1i q

1i

Pni=1

p0i q

0i

0BB@

1CCA�P(p0,p1,q0,q1)

<

Pni=1

p1i q

2i

Pni=1

p0i q

0i

0BB@

1CCA�P(p0,p1,q0,q2) (16:23)

T20: Monotonicity in base quantities: if q0 < q2, then

Pni=1

p1i q

1i

Pni=1

p0i q

0i

0BB@

1CCA�P( p0, p1, q0, q1)

>

Pni=1

p1i q

1i

Pni=1

p0i q

2i

0BB@

1CCA�P( p0, p1, q2, q1) (16:24)

16.51 Let Q be the implicit quantity index thatcorresponds to P using equation (16.17). Then it isfound that T19 translates into the following inequalityinvolving Q:

Q( p0, p1, q0, q1)<Q( p0, p1, q0, q2) if q1 < q2 (16:25)

That is, if any period 1 quantity increases, then theimplicit quantity index Q that corresponds to the priceindex P must increase. Similarly, we find that T20translates into:

Q( p0, p1, q0, q1) > Q( p0, p1, q2, q1) if q0 < q2 (16:26)

That is, if any period 0 quantity increases, then theimplicit quantity index Q must decrease. Tests T19 andT20 are attributable to Vogt (1980, p. 70).

16.52 This concludes the listing of tests. The nextsection offers an answer to the question of whether anyindex number formula P( p0,p1, q0,q1) exists that cansatisfy all 20 tests.

The Fisher ideal index and the testapproach

16.53 It can be shown that the only index numberformula P( p0, p1, q0, q1) which satisfies tests T1–T20 isthe Fisher ideal price index PF defined as the geometricmean of the Laspeyres and Paasche indices:33

PF (p0,p1,q0,q1)� fPL(p0,p1,q0,q1)Pp(p0,p1,q0,q1)g1=2

(16:27)

16.54 It is relatively straightforward to show thatthe Fisher index satisfies all 20 tests. The more difficultpart of the proof is to show that the Fisher index isthe only index number formula that satisfies these tests.This part of the proof follows from the fact that, if Psatisfies the positivity test T1 and the three reversaltests, T11–T13, then P must equal PF. To see this, rear-range the terms in the statement of test T13 into the

32 Bowley (1901, p. 227) and Fisher (1922, p. 403) both endorsed thisproperty for a price index. 33 See Diewert (1992a, p. 221).

CONSUMER PRICE INDEX: THEORY AND PRACTICE

296

following equation:

Pni=1

p1i q

1i

�Pni=1

p0i q

0i

Pni=1

p0i q

1i

�Pni=1

p1i q

0i

=P( p0, p1, q0, q1)

P( p1, p0, q0, q1)

=P( p0, p1, q0, q1)

P( p1, p0, q1, q0)

using T12, the quantity reversal test

=P( p0, p1, q0, q1)P( p0, p1, q0, q1)

using T11, the time reversal test (16:28)

Now take positive square roots of both sides of equation(16.28). It can be seen that the left-hand side ofthe equation is the Fisher index PF ( p0, p1, q0, q1)defined by equation (16.27) and the right-hand side isP( p0, p1, q0, q1). Thus if P satisfies T1, T11, T12 andT13, it must equal the Fisher ideal index PF.16.55 The quantity index that corresponds to the

Fisher price index using the product test (16.17) is QF,the Fisher quantity index, defined by equation (15.14) inChapter 15.16.56 It turns out that PF satisfies yet another test,

T21, which was Fisher’s (1921, p. 534; 1922, pp. 72–81)third reversal test (the other two being T9 and T11):

T21: Factor reversal test (functionalform symmetry test):

P( p0, p1, q0, q1)P(q0, q1, p0, p1)=

Pni=1

p1i q

1i

Pni=1

p0i q

0i

(16:29)

A justification for this test is the following: if P( p0,p1, q0, q1) is a good functional form for the price index,then, if the roles of prices and quantities are reversed,P(q0, q1, p0, p1) ought to be a good functional form for aquantity index (which seems to be a correct argument)and thus the product of the price index P( p0, p1, q0, q1)and the quantity index Q( p0, p1, q0, q1)=P(q0, q1, p0, p1)ought to equal the value ratio, V1/V 0 . The second partof this argument does not seem to be valid, and thusmany researchers over the years have objected to thefactor reversal test. Nevertheless, if T21 is accepted as abasic test, Funke and Voeller (1978, p. 180) showed thatthe only index number function P( p0, p1, q0, q1) whichsatisfies T1 (positivity), T11 (time reversal test), T12(quantity reversal test) and T21 (factor reversal test) isthe Fisher ideal index PF defined by equation (16.27).Thus the price reversal test T13 can be replaced by thefactor reversal test in order to obtain a minimal set offour tests that lead to the Fisher price index.34

The test performance of other indices16.57 The Fisher price index PF satisfies all 20 of the

tests T1–T20 listed above. Which tests do other com-

monly used price indices satisfy? Recall the Laspeyresindex PL defined by equation (15.5), the Paasche indexPP defined by equation (15.6), the Walsh index PW

defined by equation (15.19) and the Tornqvist index PT

defined by equation (15.81) in Chapter 15.16.58 Straightforward computations show that the

Paasche and Laspeyres price indices, PL and PP, fail onlythe three reversal tests, T11, T12 and T13. Since thequantity and price reversal tests, T12 and T13, aresomewhat controversial and hence can be discounted,the test performance of PL and PP seems at first sight tobe quite good. The failure of the time reversal test, T11, isnevertheless a severe limitation associated with the use ofthese indices.

16.59 The Walsh price index, PW, fails four tests:T13, the price reversal test; T16, the Paasche and Las-peyres bounding test; T19, the monotonicity in currentquantities test; and T20, the monotonicity in base quan-tities test.

16.60 Finally, the Tornqvist price index PT fails ninetests: T4 (the fixed basket test), the quantity and pricereversal tests T12 and T13, T15 (the mean value test forquantities), T16 (the Paasche and Laspeyres boundingtest) and the four monotonicity tests T17 to T20. Thusthe Tornqvist index is subject to a rather high failurerate from the viewpoint of this axiomatic approach toindex number theory.35

16.61 The tentative conclusion that can be drawnfrom the above results is that, from the viewpoint ofthis particular bilateral test approach to index num-bers, the Fisher ideal price index PF appears to be‘‘best’’ since it satisfies all 20 tests. The Paasche andLaspeyres indices are next best if we treat each testas being equally important. Both of these indices,however, fail the very important time reversal test. Theremaining two indices, the Walsh and Tornqvist priceindices, both satisfy the time reversal test but theWalsh index emerges as being ‘‘better’’ since it passes16 of the 20 tests whereas the Tornqvist only satisfies11 tests.36

The additivity test16.62 There is an additional test that many

national income accountants regard as very impor-tant: the additivity test. This is a test or propertythat is placed on the implicit quantity indexQ( p0, p1, q0, q1) that corresponds to the price indexP( p0, p1, q0, q1) using the product test (16.17). Thistest states that the implicit quantity index has the

34 Other characterizations of the Fisher price index can be found inFunke and Voeller (1978) and Balk (1985; 1995).

35 It is shown in Chapter 19, however, that the Tornqvist indexapproximates the Fisher index quite closely using ‘‘normal’’ time seriesdata that are subject to relatively smooth trends. Hence, under thesecircumstances, the Tornqvist index can be regarded as passing the 20tests to a reasonably high degree of approximation.36 This assertion needs to be qualified: there are many other tests thatwe have not discussed, and price statisticians might hold differentopinions regarding the importance of satisfying various sets of tests.Other tests are discussed by von Auer (2001; 2002), Eichhorn andVoeller (1976), Balk (1995) and Vogt and Barta (1997), among others.It is shown in paragraphs 16.101 to 16.135 that the Tornqvist index isideal when considered under a different set of axioms.

THE AXIOMATIC AND STOCHASTIC APPROACHES TO INDEX NUMBER THEORY

297

following form:

Q( p0, p1, q0, q1)=

Pni=1

p*i q

1i

Pnm=1

p*mq

0m

(16:30)

where the common across-periods price for commodityi, p*

i for i=1, . . . , n, can be a function of all 4n prices andquantities pertaining to the two periods or situationsunder consideration, p0, p1, q0, q1. In the literature onmaking multilateral comparisons (i.e., comparisonsbetween more than two situations), it is quite commonto assume that the quantity comparison between anytwo regions can be made using the two regional quan-tity vectors, q0 and q1, and a common reference pricevector, P* � ( p*

1, . . . , p*n):

37

16.63 Obviously, different versions of the additivitytest can be obtained if further restrictions are placedon precisely which variables each reference price p*

idepends. The simplest such restriction is to assume thateach p*

i depends only on the commodity i prices per-taining to each of the two situations under consideration,p0i and p1

i . If it is further assumed that the functionalform for the weighting function is the same for eachcommodity, so that p*

i =m( p0i , p

1i ) for i=1, . . . , n, then we

are led to the unequivocal quantity index postulated byKnibbs (1924, p. 44).

16.64 The theory of the unequivocal quantity index(or the pure quantity index)38 parallels the theory of thepure price index outlined in paragraphs 15.24 to 15.32 ofChapter 15. An outline of this theory is given here. Letthe pure quantity index QK have the following func-tional form:

QK ( p0, p1, q0, q1) �

Pni=1

q1i m( p0

i , p1i )

Pnk=1

q0km( p0

k, p1k)

(16:31)

It is assumed that the price vectors p0 and p1 are strictlypositive and the quantity vectors q0 and q1 are non-nega-tive but have at least one positive component.39 The pro-blem is to determine the functional form for the averagingfunction m if possible. To do this, it is necessary to imposesome tests or properties on the pure quantity indexQK. Aswas the case with the pure price index, it is very reasonableto ask that the quantity index satisfy the time reversal test:

QK ( p1, p0, q1, q0)=1

QK ( p0, p1, q0, q1)(16:32)

16.65 As was the case with the theory of the unequiv-ocal price index, it can be seen that if the unequivocalquantity index QK is to satisfy the time reversal test(16.32), the mean function in equation (16.31) mustbe symmetric. It is also asked that QK satisfy the follow-ing invariance to proportional changes in current pricestest.

QK ( p0, lp1, q0, q1)=QK ( p0, p1, q0, q1)

for all p0, p1, q0, q1and all l > 0 (16:33)

16.66 The idea behind this invariance test is this: thequantity index QK ( p0, p1, q0, q1) should depend only onthe relative prices in each period and it should notdepend on the amount of inflation between the twoperiods. Another way to interpret test (16.33) is to lookat what the test implies for the corresponding implicitprice index, PIK, defined using the product test (16.17).It can be shown that if QK satisfies equation (16.33),then the corresponding implicit price index PIK willsatisfy test T5 above, the proportionality in current pricestest. The two tests, (16.32) and (16.33), determine theprecise functional form for the pure quantity index QK

defined by equation (16.31): the pure quantity index orKnibbs’ unequivocal quantity index QK must be theWalsh quantity index QW

40 defined by:

QW ( p0, p1, q0, q1) �

Pni=1

q1i

ffiffiffiffiffiffiffiffiffip0i p

1i

pPnk=1

q0k

ffiffiffiffiffiffiffiffiffip0kp

1k

q (16:34)

16.67 Thus with the addition of two tests, the pureprice index PK must be the Walsh price index PW definedby equation (15.19) in Chapter 15 and with the additionof the same two tests (but applied to quantity indicesinstead of price indices), the pure quantity index QK

must be the Walsh quantity index QW defined byequation (16.34). Note, however, that the product of theWalsh price and quantity indices is not equal to theexpenditure ratio, V1/V 0. Thus believers in the pure orunequivocal price and quantity index concepts have tochoose one of these two concepts; they cannot bothapply simultaneously.41

16.68 If the quantity index Q( p0, p1, q0, q1) satis-fies the additivity test (16.30) for some price weights p*

i ,then the percentage change in the quantity aggregate,Q( p0, p1, q0, q1)�1, can be rewritten as follows:

Q( p0, p1, q0, q1)�1=

Pni=1

p*i q

1i

Pnm=1

p*mq

0m

�1=

Pni=1

p*i q

1i�

Pnm=1

p*mq

0m

Pnm=1

p*mq

0m

=Pni=1

wi(q1i�q0

i ) (16:35)37 Hill (1993, p. 395–397) termed such multilateral methods the blockapproach while Diewert (1996a, pp. 250–251) used the term averageprice approaches. Diewert (1999b, p. 19) used the term additive multi-lateral system. For axiomatic approaches to multilateral index numbertheory, see Balk (1996a; 2001) and Diewert (1999b).38 Diewert (2001) used this term.39 It is assumed that m(a, b) has the following two properties: m(a, b) isa positive and continuous function, defined for all positive numbers aand b, and m(a, a)=a for all a>0.

40 This is the quantity index that corresponds to the price index 8defined by Walsh (1921a, p. 101).41 Knibbs (1924) did not notice this point.

CONSUMER PRICE INDEX: THEORY AND PRACTICE

298

where the weight for commodity i, wi, is defined as

wi �p*iPn

m=1

p*mq

0m

; i=1, . . . , n (16:36)

Note that the change in commodity i going from situ-ation 0 to situation 1 is q1

i�q0i . Thus the ith term on the

right-hand side of equation (16.35) is the contribution ofthe change in commodity i to the overall percentagechange in the aggregate going from period 0 to 1. Busi-ness analysts often want statistical agencies to providedecompositions such as equation (16.35) so that they candecompose the overall change in an aggregate into sector-specific components of change.42 Thus there is a demandon the part of users for additive quantity indices.16.69 For the Walsh quantity index defined by

equation (16.34), the ith weight is

wWi�

ffiffiffiffiffiffiffiffiffip0i p

1i

pPnm=1

q0m

ffiffiffiffiffiffiffiffiffiffiffip0mp

1m

p ; i=1, . . . , n (16:37)

Thus the Walsh quantity index QW has a percentagedecomposition into component changes of the form ofequation (16.35), where the weights are defined byequation (16.37).16.70 It turns out that the Fisher quantity index QF,

defined by equation (15.14) in Chapter 15, also has anadditive percentage change decomposition of the formgiven by equation (16.35).43 The ith weight wFi

for thisFisher decomposition is rather complicated anddependsonthe Fisher quantity index QF( p

0, p1, q0, q1) as follows:44

wFi� w0

i +(QF )2w1i

1+QF; i=1, . . . , n (16:38)

where QF is the value of the Fisher quantity index,QF ( p0, p1, q0, q1), and the period t normalized price forcommodity i, wt

i , is defined as the period i price ptidivided by the period t expenditure on the aggregate:

wti �

ptiPnm=1

ptmqtm

; t=0, 1; i=1, . . . , n (16:39)

16.71 Using the weights wFidefined by equations

(16.38) and (16.39), the following exact decomposition is

obtained for the Fisher ideal quantity index:

QF ( p0, p1, q0, q1)�1=Pni=1

wFi (q1i�q0

i ) (16:40)

Thus the Fisher quantity index has an additive percen-tage change decomposition.45

16.72 Because of the symmetric nature of the Fisherprice and quantity indices, it can be seen that the Fisherprice index PF defined by equation (16.27) also has thefollowing additive percentage change decomposition:

PF ðp0, p1, q0, q1Þ�1=Pni=1

vFi ( p1i�p0

i ) (16:41)

where the commodity i weight vFiis defined as

vFi �v0i +(PF )2v1

i

1+PF; i=1, . . . , n (16:42)

where PF is the value of the Fisher price index,PF( p

0, p1, q0, q1), and the period t normalized quantityfor commodity i, vti , is defined as the period i quantity qtidivided by the period t expenditure on the aggregate:

vti �qtiPn

m=1

ptmqtm

; t=0, 1; i=1, . . . , n (16:43)

16.73 The above results show that the Fisher priceand quantity indices have exact additive decompositionsinto components that give the contribution to theoverall change in the price (or quantity) index of thechange in each price (or quantity).

The stochastic approach toprice indices

The early unweighted stochasticapproach

16.74 The stochastic approach to the determinationof the price index can be traced back to the work ofJevons (1863; 1865) and Edgeworth (1888) over 100years ago.46 The basic idea behind the (unweighted)stochastic approach is that each price relative, p1

i =p0i for

i=1, 2, . . . , n, can be regarded as an estimate of a com-mon inflation rate a between periods 0 and 1.47

It is assumed that

p1i

p0i

=a+ei; i=1, 2, . . . , n (16:44)

where a is the common inflation rate and the ei arerandom variables with mean 0 and variance s2. The least

42 Business and government analysts also often demand an analogousdecomposition of the change in price aggregate into sector-specificcomponents that add up.43 The Fisher quantity index also has an additive decomposition of thetype defined by equation (16.30) attributable to Van Ijzeren (1987,p. 6). The ith reference price p*

i is defined as p*i � (1=2)p0

i +(1=2)p1i =

PF ( p0, p1, q0, q1) for i=1, . . . , n and where PF is the Fisher price index.This decomposition was also independently derived by Dikhanov(1997). The Van Ijzeren decomposition for the Fisher quantity index iscurrently being used by the US Bureau of Economic Analysis; seeMoulton and Seskin (1999, p. 16) and Ehemann, Katz and Moulton(2002).44 This decomposition was obtained by Diewert (2002a) and Reinsdorf,Diewert and Ehemann (2002). For an economic interpretation of thisdecomposition, see Diewert (2002a).

45 To verify the exactness of the decomposition, substitute equation(16.38) into equation (16.40) and solve the resulting equation for QF. Itis found that the solution is equal to QF defined by equation (15.14) inChapter 15.46 For references to the literature, see Diewert (1993a, pp. 37–38;1995a; 1995b).47 ‘‘In drawing our averages the independent fluctuations will more orless destroy each other; the one required variation of gold will remainundiminished’’ (Jevons (1863, p. 26)).

THE AXIOMATIC AND STOCHASTIC APPROACHES TO INDEX NUMBER THEORY

299

squares or maximum likelihood estimator for a is theCarli (1764) price index PC defined as

PC( p0, p1) �Pni=1

1

n

p1i

p0i

(16:45)

A drawback of the Carli price index is that it doesnot satisfy the time reversal test, i.e., PC( p1, p0)=1/PC ( p0, p1).48

16.75 Now change the stochastic specification andassume that the logarithm of each price relative,ln( p1

i =p0i ), is an unbiased estimate of the logarithm of

the inflation rate between periods 0 and 1, b say. Thecounterpart to equation (16.44) is:

lnp1i

p0i

� �=b+ei; i=1, 2, . . . , n (16:46)

where b:ln a and the ei are independently distributedrandom variables with mean 0 and variance s2. The leastsquares or maximum likelihood estimator for b is thelogarithm of the geometric mean of the price relatives.Hence the corresponding estimate for the commoninflation rate a49 is the Jevons (1865) price index PJ

defined as follows:

PJ( p0, p1) �

Yni=1

ffiffiffiffiffip1i

p0i

n

s: (16:47)

16.76 The Jevons price index PJ does satisfy the timereversal test and hence is much more satisfactory thanthe Carli index PC. Both the Jevons and Carli priceindices nevertheless suffer from a fatal flaw: each pricerelative p1

i =p0i is regarded as being equally important and

is given an equal weight in the index number formulae(16.45) and (16.47). John Maynard Keynes was parti-cularly critical of this unweighted stochastic approach toindex number theory.50 He directed the following criti-

cism towards this approach, which was vigorouslyadvocated by Edgeworth (1923):

Nevertheless I venture to maintain that such ideas,which I have endeavoured to expound above as fairly andas plausibly as I can, are root-and-branch erroneous. The‘‘errors of observation’’, the ‘‘faulty shots aimed at a singlebull’s eye’’ conception of the index number of prices,Edgeworth’s ‘‘objective mean variation of general prices’’,is the result of confusion of thought. There is no bull’s eye.There is no moving but unique centre, to be called thegeneral price level or the objective mean variation ofgeneral prices, round which are scattered the moving pricelevels of individual things. There are all the various, quitedefinite, conceptions of price levels of composite com-modities appropriate for various purposes and inquirieswhich have been scheduled above, and many others too.There is nothing else. Jevons was pursuing a mirage.

What is the flaw in the argument? In the first place itassumed that the fluctuations of individual prices roundthe ‘‘mean’’ are ‘‘random’’ in the sense required by thetheory of the combination of independent observations.In this theory the divergence of one ‘‘observation’’ fromthe true position is assumed to have no influence on thedivergences of other ‘‘observations’’. But in the case ofprices, a movement in the price of one commoditynecessarily influences the movement in the prices of othercommodities, whilst the magnitudes of these compensa-tory movements depend on the magnitude of the changein expenditure on the first commodity as compared withthe importance of the expenditure on the commoditiessecondarily affected. Thus, instead of ‘‘independence’’,there is between the ‘‘errors’’ in the successive ‘‘observa-tions’’ what some writers on probability have called‘‘connexity’’, or, as Lexis expressed it, there is ‘‘sub-nor-mal dispersion’’.

We cannot, therefore, proceed further until we haveenunciated the appropriate law of connexity. But the lawof connexity cannot be enunciated without reference tothe relative importance of the commodities affected—which brings us back to the problem that we have beentrying to avoid, of weighting the items of a compositecommodity (Keynes (1930, pp. 76–77)).

The main point Keynes seemed to be making in theabove quotation is that prices in the economy are notindependently distributed from each other and fromquantities. In current macroeconomic terminology,Keynes can be interpreted as saying that a macro-economic shock will be distributed across all prices andquantities in the economy through the normal inter-action between supply and demand; i.e., through theworkings of the general equilibrium system. ThusKeynes seemed to be leaning towards the economicapproach to index number theory (even before it wasdeveloped to any great extent), where quantity move-ments are functionally related to price movements. Asecond point that Keynes made in the above quotationis that there is no such thing as the inflation rate; thereare only price changes that pertain to well-specified setsof commodities or transactions; i.e., the domain ofdefinition of the price index must be carefully speci-fied.51 A final point that Keynes made is that price

48 In fact, Fisher (1922, p. 66) noted that PC( p0, p1)PC ( p1, p0)� 1unless the period 1 price vector p1 is proportional to the period 0 pricevector p0; i.e., Fisher showed that the Carli index has a definite upwardbias. He urged statistical agencies not to use this formula. Walsh (1901,pp. 331, 530) also discovered this result for the case n=2.49 Greenlees (1999) pointed out that although (1/n)

Pni=1 ln( p1

i =p0i ) is

an unbiased estimator for b, the corresponding exponential of thisestimator, PJ defined by equation (16.47), will generally not be anunbiased estimator for a under our stochastic assumptions. To see this,let xi= ln p1

i =p0i . Taking expectations, we have: Exi=b= ln a. Define

the positive, convex function f of one variable x by f(x):ex. By Jen-sen’s (1906) inequality, Ef(x)� f(Ex). Letting x equal the randomvariable xi, this inequality becomes: E( p1

i =p0i )=Ef (xi) � f (Exi)=

f (b)=eb=eln a=a: Thus for each n, E( p1i =p

0i ) � a, and it can be seen

that the Jevons price index will generally have an upward bias underthe usual stochastic assumptions.50 Walsh (1901, p. 83) also stressed the importance of proper weightingaccording to the economic importance of the commodities in theperiods being compared: ‘‘But to assign uneven weighting withapproximation to the relative sizes, either over a long series of years orfor every period separately, would not require much additional trou-ble; and even a rough procedure of this sort would yield results farsuperior to those yielded by even weighting. It is especially absurd torefrain from using roughly reckoned uneven weighting on the groundthat it is not accurate, and instead to use even weighting, which ismuch more inaccurate.’’

51 See paragraphs 15.7 to 15.17 in Chapter 15 for additional discussionon this point.

CONSUMER PRICE INDEX: THEORY AND PRACTICE

300

movements must be weighted by their economicimportance, i.e., by quantities or expenditures.16.77 In addition to the above theoretical criticisms,

Keynes also made the following strong empirical attackon Edgeworth’s unweighted stochastic approach:

The Jevons–Edgeworth ‘‘objective mean variation ofgeneral prices’’, or ‘‘indefinite’’ standard, has generallybeen identified, by those who were not as alive as Edge-worth himself was to the subtleties of the case, with thepurchasing power of money – if only for the excellentreason that it was difficult to visualise it as anything else.And since any respectable index number, howeverweighted, which covered a fairly large number of com-modities could, in accordance with the argument, beregarded as a fair approximation to the indefinite stan-dard, it seemed natural to regard any such index as a fairapproximation to the purchasing power of money also.

Finally, the conclusion that all the standards ‘‘come tomuch the same thing in the end’’ has been reinforced‘‘inductively’’ by the fact that rival index numbers (all ofthem, however, of the wholesale type) have shown aconsiderable measure of agreement with one another inspite of their different compositions . . . On the contrary,the tables given above (pp. 53, 55) supply strong pre-sumptive evidence that over long period as well as overshort period the movements of the wholesale and of theconsumption standards respectively are capable of beingwidely divergent (Keynes (1930, pp. 80–81)).

In the above quotation, Keynes noted that the propo-nents of the unweighted stochastic approach to pricechangemeasurement were comforted by the fact that all ofthe then existing (unweighted) indices of wholesale pricesshowed broadly similar movements. Keynes showedempirically, however, that his wholesale price indicesmoved quite differently from his consumer price indices.16.78 In order to overcome the above criticisms of

the unweighted stochastic approach to index numbers, itis necessary to:

� have a definite domain of definition for the indexnumber;

� weight the price relatives by their economic impor-tance.52

Alternative methods of weighting are discussed in thefollowing sections.

The weighted stochastic approach16.79 Walsh (1901, pp. 88–89) seems to have been

the first index number theorist to point out that a sen-sible stochastic approach to measuring price changemeans that individual price relatives should be weightedaccording to their economic importance or their trans-actions value in the two periods under consideration:

It might seem at first sight as if simply every pricequotation were a single item, and since every commodity(any kind of commodity) has one price-quotationattached to it, it would seem as if price-variations of everykind of commodity were the single item in question. Thisis the way the question struck the first inquirers into

price-variations, wherefore they used simple averagingwith even weighting. But a price-quotation is the quota-tion of the price of a generic name for many articles; andone such generic name covers a few articles, and anothercovers many. . . . A single price-quotation, therefore, maybe the quotation of the price of a hundred, a thousand, ora million dollar’s worths, of the articles that make up thecommodity named. Its weight in the averaging, therefore,ought to be according to these money-unit’s worth(Walsh (1921a, pp. 82–83)).

But Walsh did not give a convincing argument on exactlyhow these economic weights should be determined.

16.80 Henri Theil (1967, pp. 136–137) proposed asolution to the lack of weighting in the Jevons index, PJ

defined by equation (16.47). He argued as follows.Suppose we draw price relatives at random in such away that each dollar of expenditure in the base periodhas an equal chance of being selected. Then the prob-ability that we will draw the ith price relative is equalto s0i � p0

i q0i =Pn

k=1p0kq

0k, the period 0 expenditure

share for commodity i. Then the overall mean (period 0weighted) logarithmic price change is

Pni=1s

0i

ln ( p1i =p

0i ):

53 Now repeat the above mental experimentand draw price relatives at random in such a way thateach dollar of expenditure in period 1 has an equalprobability of being selected. This leads to the overallmean (period 1 weighted) logarithmic price change ofPn

i=1s1i ln ( p1

i =p0i ):

54

16.81 Each of these measures of overall logarithmicprice change seems equally valid, so we could argue fortaking a symmetric average of the two measures in orderto obtain a final single measure of overall logarithmicprice change. Theil55 argued that a ‘‘nice’’ symmetricindex number formula can be obtained if the probabilityof selection for the nth price relative is made equal to thearithmetic average of the period 0 and 1 expenditureshares for commodity n. Using these probabilities ofselection, Theil’s final measure of overall logarithmicprice change was

lnPT ( p0, p1, q0, q1) �Pni=1

1

2(s0i +s1i ) ln

p1i

p0i

� �(16:48)

Note that the index PT defined by equation (16.48) isequal to the Tornqvist index defined by equation (15.81)in Chapter 15.

16.82 A statistical interpretation of the right-handside of equation (16.48) can be given. Define the ith

52 Walsh (1901, pp. 82–90; 1921a, pp. 82–83) also objected to the lackof weighting in the unweighted stochastic approach to index numbertheory.

53 In Chapter 19, this index is called the geometric Laspeyres index,PGL. Vartia (1978, p. 272) referred to this index as the logarithmicLaspeyres index. Yet another name for the index is the base weightedgeometric index.54 In Chapter 19, this index is called the geometric Paasche index, PGP.Vartia (1978, p. 272) referred to this index as the logarithmic Paascheindex. Yet another name for the index is the current period weightedgeometric index.55 ‘‘The price index number defined in (1.8) and (1.9) uses the nindividual logarithmic price differences as the basic ingredients. Theyare combined linearly by means of a two-stage random selectionprocedure: First, we give each region the same chance ½ of beingselected, and second, we give each dollar spent in the selected regionthe same chance (1/ma or 1/mb) of being drawn’’ (Theil (1967,p. 138)).

THE AXIOMATIC AND STOCHASTIC APPROACHES TO INDEX NUMBER THEORY

301

logarithmic price ratio ri by:

ri � lnp1i

p0i

� �for i=1, . . . , n (16:49)

Now define the discrete random variable, R say, as therandom variable which can take on the values ri withprobabilities ri � (1=2)[s0i +s1i ] for i=1, . . . , n. Notethat, since each set of expenditure shares, s0i and s1i , sumsto one over i, the probabilities ri will also sum to one. Itcan be seen that the expected value of the discrete ran-dom variable R is

E R½ � �Pni=1

riri=Pni=1

1

2(s0i +s1i ) ln

p1i

p0i

� �= lnPT ( p0, p1, q0, q1): (16:50)

Thus the logarithm of the index PT can be interpreted asthe expected value of the distribution of the logarithmicprice ratios in the domain of definition under con-sideration, where the n discrete price ratios in this domainof definition are weighted according to Theil’s prob-ability weights, ri � (1=2)[s0i +s1i ] for i=1, . . . , n.

16.83 Taking antilogs of both sides of equation(16.48), the Tornqvist (1936; 1937) and Theil priceindex, PT, is obtained.56 This index number formula hasa number of good properties. In particular, PT satisfiesthe proportionality in current prices test T5 and the timereversal test T11, discussed above. These two tests canbe used to justify Theil’s (arithmetic) method of formingan average of the two sets of expenditure shares in orderto obtain his probability weights, ri � (1=2)[s0i +s1i ]for i=1, . . . , n. Consider the following symmetric meanclass of logarithmic index number formulae:

lnPS( p0, p1, q0, q1) �

Pni=1

m(s0i , s1i ) ln

p1i

p0i

� �(16:51)

where m(s0i , s1i ) is a positive function of the period 0 and 1

expenditure shares on commodity i, s0i and s1i respec-tively. In order for PS to satisfy the time reversal test, it isnecessary that the function m be symmetric. Then it canbe shown57 that for PS to satisfy test T5, m must be thearithmetic mean. This provides a reasonably strong jus-tification for Theil’s choice of the mean function.

16.84 The stochastic approach of Theil has another‘‘nice’’ symmetry property. Instead of considering thedistribution of the price ratios ri= ln p1

i =p0i , we could

also consider the distribution of the reciprocals of theseprice ratios, say:

ti � lnp0i

p1i

= lnp1i

p0i

� ��1

=� lnp1i

p0i

=�ri for i=1, . . . , n

(16:52)

The symmetric probability, ri � (1=2)[s0i +s1i ], can stillbe associated with the ith reciprocal logarithmic priceratio ti for i=1, . . . , n. Now define the discrete randomvariable, T say, as the random variable which can takeon the values ti with probabilities ri � (1=2)[s0i +s1i ] fori=1, . . . , n. It can be seen that the expected value of thediscrete random variable T is

E T½ � �Pni=1

riti

=�Pni=1

riti using (16:52)

=�E R½ � using (16:50)

=� lnPT ( p0, p1, q0, q1) (16:53)

Thus it can be seen that the distribution of the randomvariable T is equal to minus the distribution of therandom variable R. Hence it does not matter whetherthe distribution of the original logarithmic price ratios,ri � ln p1

i =p0i , is considered or the distribution of their

reciprocals, ti � ln p0i =p

1i , is considered: essentially the

same stochastic theory is obtained.16.85 It is possible to consider weighted stochastic

approaches to index number theory where the dis-tribution of the price ratios, p1

i =p0i , is considered rather

than the distribution of the logarithmic price ratios, lnp1i =p

0i . Thus, again following in the footsteps of Theil,

suppose that price relatives are drawn at random insuch a way that each dollar of expenditure in the baseperiod has an equal chance of being selected. Then theprobability that the ith price relative will be drawn isequal to s0i , the period 0 expenditure share for com-modity i. Thus the overall mean (period 0 weighted)price change is:

PL( p0, p1, q0, q1)=Pni=1

s0ip1i

p0i

(16:54)

which turns out to be the Laspeyres price index, PL. Thisstochastic approach is the natural one for studying sam-pling problems associated with implementing a Laspeyresprice index.

16.86 Now repeat the above mental experiment anddraw price relatives at random in such a way that eachdollar of expenditure in period 1 has an equal prob-ability of being selected. This leads to the overall mean(period 1 weighted) price change equal to:

PPAL( p0, p1, q0, q1)=Pni=1

s1ip1i

p0i

(16:55)

This is known as the Palgrave (1886) index numberformula.58

16.87 It can be verified that neither the Laspeyresnor Palgrave price indices satisfy the time reversal test,T11. Thus, again following in the footsteps of Theil, itmight be attempted to obtain a formula that satisfies thetime reversal test by taking a symmetric average of the

56 The sampling bias problem studied by Greenlees (1999) does notoccur in the present context because there is no sampling involved indefinition (16.50): the sum of the ptiq

ti over i for each period t is

assumed to equal the value aggregate Vt for period t.57 See Diewert (2000) and Balk and Diewert (2001).

58 It is formula number 9 in Fisher’s (1922, p. 466) listing of indexnumber formulae.

CONSUMER PRICE INDEX: THEORY AND PRACTICE

302

two sets of shares. Thus consider the following class ofsymmetric mean index number formulae:

Pm( p0, p1, q0, q1) �Pni=1

m(s0i , s1i )p1i

p0i

(16:56)

where m(s0i , s1i ) is a symmetric function of the period 0

and 1 expenditure shares for commodity i, s0i and s1irespectively. In order to interpret the right hand-side ofequation (16.56) as an expected value of the price ratiosp1i =p

0i , it is necessary that

Pni=1

m(s0i , s1i )=1 (16:57)

In order to satisfy equation (16.57), however, m must bethe arithmetic mean.59 With this choice of m, equation(16.56) becomes the following (unnamed) index numberformula, Pu:

Pu( p0, p1, q0, q1) �

Pni=1

1

2(s0i +s1i )

p1i

p0i

(16:58)

Unfortunately, the unnamed index Pu does not satisfythe time reversal test either.60

16.88 Instead of considering the distribution of theprice ratios, p1

i =p0i , the distribution of the reciprocals of

these price ratios could be considered. The counterpartsto the asymmetric indices defined earlier by equations(16.54) and (16.55) are now

Pni=1s

0i ( p

0i =p

1i ) and

Pni=1s

1i

( p0i =p

1i ), respectively. These are (stochastic) price indices

going backwards from period 1 to 0. In order to makethese indices comparable with other previous forward-looking indices, take the reciprocals of these indices(which leads to harmonic averages) and the followingtwo indices are obtained:

PHL( p0, p1, q0, q1) � 1Pni=1

s0ip0i

p1i

(16:59)

PHP( p0, p1, q0, q1) � 1Pni=1

s1ip0i

p1i

=1

Pni=1

s1ip1i

p0i

� ��1

=PP( p0, p1, q0, q1) (16:60)

using equation (15.9) in Chapter 15. Thus the reciprocalstochastic price index defined by equation (16.60) turnsout to equal the fixed basket Paasche price index, PP.This stochastic approach is the natural one for studyingsampling problems associated with implementing aPaasche price index. The other asymmetrically weighted

reciprocal stochastic price index defined by the formula(16.59) has no author’s name associated with it but itwas noted by Fisher (1922, p. 467) as his index numberformula 13. Vartia (1978, p. 272) called this index theharmonic Laspeyres index and his terminology will beused.

16.89 Now consider the class of symmetricallyweighted reciprocal price indices defined as:

Pmr( p0, p1, q0, q1) � 1

Pni=1

m(s0i , s1i )

p1i

p0i

� ��1(16:61)

where, as usual, m(s0i , s1i ) is a homogeneous symmetric

mean of the period 0 and 1 expenditure shares oncommodity i. However, none of the indices defined byequations (16.59) to (16.61) satisfies the time reversaltest.

16.90 The fact that Theil’s index number formula PT

satisfies the time reversal test leads to a preference forTheil’s index as the ‘‘best’’ weighted stochasticapproach.

16.91 The main features of the weighted stochasticapproach to index number theory can be summarizedas follows. It is first necessary to pick two periodsand a transactions domain of definition. As usual,each value transaction for each of the n commoditiesin the domain of definition is split up into price andquantity components. Then, assuming there are nonew commodities or no disappearing commodities,there are n price relatives p1

i =p0i pertaining to the two

situations under consideration along with the corre-sponding 2n expenditure shares. The weighted sto-chastic approach just assumes that these n relativeprices, or some transformation of these price relatives,f ( p1

i =p0i ), have a discrete statistical distribution, where

the ith probability, ri=m(s0i , s1i ), is a function of the

expenditure shares pertaining to commodity i in thetwo situations under consideration, s0i and s1i . Differ-ent price indices result, depending on how the func-tions f and m are chosen. In Theil’s approach, thetransformation function f is the natural logarithm andthe mean function m is the simple unweighted arith-metic mean.

16.92 There is a third aspect to the weighted sto-chastic approach to index number theory: it has to bedecided what single number best summarizes the dis-tribution of the n (possibly transformed) price rela-tives. In the above analysis, the mean of the discretedistribution was chosen as the ‘‘best’’ summary mea-sure for the distribution of the (possibly transformed)price relatives; but other measures are possible. Inparticular, the weighted median or various trimmedmeans are often suggested as the ‘‘best’’ measure ofcentral tendency because these measures minimize theinfluence of outliers. Detailed discussion of thesealternative measures of central tendency is, however,beyond the scope of this chapter. Additional materialon stochastic approaches to index number theory andreferences to the literature can be found in Clementsand Izan (1981; 1987), Selvanathan and Rao (1994),

59 For a proof of this assertion, see Balk and Diewert (2001).60 In fact, this index suffers from the same upward bias as the Carliindex in that Pu( p

0, p1, q0, q1)Pu( p1, p0, q1, q0)�1. To prove this, note

that the previous inequality is equivalent to [Pu( p1, p0, q1, q0)]�1 �

Pu( p0, p1, q0, q1) and this inequality follows from the fact that a

weighted harmonic mean of n positive numbers is equal or less than thecorresponding weighted arithmetic mean; see Hardy, Littlewood andPolya (1934, p. 26).

THE AXIOMATIC AND STOCHASTIC APPROACHES TO INDEX NUMBER THEORY

303

Diewert (1995b), Cecchetti (1997) and Wynne (1997;1999).

16.93 Instead of taking the above stochasticapproach to index number theory, it is possible totake the same raw data that are used in this approachbut use an axiomatic approach. Thus, in the follow-ing section, the price index is regarded as a value-weighted function of the n price relatives and the testapproach to index number theory is used in order todetermine the functional form for the price index. Putanother way, the axiomatic approach in the nextsection looks at the properties of alternative descrip-tive statistics that aggregate the individual price rela-tives (weighted by their economic importance) intosummary measures of price change in an attempt tofind the ‘‘best’’ summary measure of price change.Thus the axiomatic approach pursued below canbe viewed as a branch of the theory of descriptivestatistics.

The second axiomatic approachto bilateral price indices

The basic framework and somepreliminary tests

16.94 As mentioned in paragraphs 16.1 to 16.10,one of Walsh’s approaches to index number theory wasan attempt to determine the ‘‘best’’ weighted average ofthe price relatives, ri.

61 This is equivalent to using anaxiomatic approach to try to determine the ‘‘best’’index of the form P(r, v0, v1), where v0 and v1 are thevectors of expenditures on the n commodities duringperiods 0 and 1.62 Initially, rather than starting withindices of the form P(r, v0, v1), indices of the formP( p0, p1, v0, v1) will be considered, since this frameworkwill be more comparable to the first bilateral axiomaticframework taken in paragraphs 16.30 to 16.73. As will

be seen below, if the invariance to changes in the units

of measurement test is imposed on an index of theform P( p0, p1, v0, v1), then P( p0, p1, v0, v1) can be writ-ten in the form P(r, v0, v1).

16.95 Recall that the product test (16.17) was used todefine the quantity index Q( p0, p1, q0, q1):V1/V 0P( p0,p1, q0, q1) that corresponded to the bilateral price indexP( p0, p1, q0, q1). A similar product test holds in thepresent framework; i.e., given that the functional formfor the price index P( p0, p1, v0, v1) has been determined,then the corresponding implicit quantity index can bedefined in terms of P as follows:

Q( p0, p1, v0, v1) �

Pni=1

v1i

Pni=1

v0i

� �P( p0, p1, v0, v1)

(16:62)

16.96 In paragraphs 16.30 to 16.73, the price andquantity indices P( p0, p1, q0, q1) and Q( p0, p1, q0, q1)were determined jointly; i.e., not only were axiomsimposed on P( p0, p1, q0, q1) but they were also imposedon Q( p0, p1, q0, q1) and the product test (16.17) was usedto translate these tests on Q into tests on P. In thissection, this approach will not be followed: only tests onP( p0, p1, v0, v1) will be used in order to determine the‘‘best’’ price index of this form. Thus there is a paralleltheory for quantity indices of the form Q(q0, q1, v0, v1),where it is attempted to find the ‘‘best’’ value-weightedaverage of the quantity relatives, q1

i =q0i .

63

16.97 For the most part, the tests which will beimposed on the price index P( p0, p1, v0, v1) in this sectionare counterparts to the tests that were imposed on theprice index P( p0, p1, q0, q1) in paragraphs 16.30 to 16.73.It will be assumed that every component of each priceand value vector is positive; i.e., pt� 0n and vt� 0n fort=0,1. If it is desired to set v0=v1, the commonexpenditure vector is denoted by v; if it is desired to setp0=p1, the common price vector is denoted by p.

16.98 The first two tests are straightforward coun-terparts to the corresponding tests in paragraph 16.34.

T1: Positivity: P( p0, p1, v0, v1) > 0

T2: Continuity: P( p0, p1, v0, v1) is a continuousfunction of its arguments

T3: Identity or constant prices test: P( p, p, v0, v1)=1

That is, if the price of every good is identical during thetwo periods, then the price index should equal unity, nomatter what the value vectors are. Note that the twovalue vectors are allowed to be different in the abovetest.

61 Fisher also took this point of view when describing his approach toindex number theory:

An index number of the prices of a number of commodities is an averageof their price relatives. This definition has, for concreteness, beenexpressed in terms of prices. But in like manner, an index number can becalculated for wages, for quantities of goods imported or exported, and,in fact, for any subject matter involving divergent changes of a group ofmagnitudes. Again, this definition has been expressed in terms of time.But an index number can be applied with equal propriety to comparisonsbetween two places or, in fact, to comparisons between the magnitudes ofa group of elements under any one set of circumstances and their mag-nitudes under another set of circumstances (Fisher (1922, p. 3)).

In setting up his axiomatic approach, Fisher imposed axioms onthe price and quantity indices written as functions of the two pricevectors, p0 and p1, and the two quantity vectors, q0 and q1; i.e., hedid not write his price index in the form P(r, v0, v1) and imposeaxioms on indices of this type. Of course, in the end, his idealprice index turned out to be the geometric mean of the Laspeyresand Paasche price indices and, as was seen in Chapter 15, each ofthese indices can be written as expenditure share-weighted avera-ges of the n price relatives, ri � p1

i =ip0i :

62 Chapter 3 in Vartia (1976) considered a variant of this axiomaticapproach.

63 It turns out that the price index that corresponds to this ‘‘best’’

quantity index, defined as P (q0, q1, v0, v1) �Pn

i=1 v1i =Pn

i=1 v0i Q

�(q0, q1, v0, v1)�, will not equal the ‘‘best’’ price index, P( p0, p1, v0, v1).Thus the axiomatic approach used here generates separate ‘‘best’’ priceand quantity indices whose product does not equal the value ratio ingeneral. This is a disadvantage of the second axiomatic approach tobilateral indices compared to the first approach studied above.

CONSUMER PRICE INDEX: THEORY AND PRACTICE

304

Homogeneity tests16.99 The following four tests restrict the behaviour

of the price index P as the scale of any one of the fourvectors p0, p1, v0, v1 changes.

T4: Proportionality in current prices:

P( p0, lp1, v0, v1)=lP( p0, p1, v0, v1) for l> 0

That is, if all period 1 prices are multiplied by the posi-tive number l, then the new price index is l times the oldprice index. Put another way, the price index functionP( p0, p1, v0, v1) is (positively) homogeneous of degree onein the components of the period 1 price vector p1. Thistest is the counterpart to test T5 in paragraph 16.37.16.100 In the next test, instead of multiplying all

period 1 prices by the same number, all period 0 pricesare multiplied by the number l.

T5: Inverse proportionality in base period prices:

P(lp0, p1, v0, v1)=l�1P( p0, p1, v0, v1) for l > 0

That is, if all period 0 prices are multiplied by thepositive number l, then the new price index is 1/l timesthe old price index. Put another way, the price indexfunction P( p0, p1, v0, v1) is (positively) homogeneous ofdegree minus one in the components of the period 0price vector p0. This test is the counterpart to test T6 inparagraph 16.39.16.101 The following two homogeneity tests can

also be regarded as invariance tests.

T6: Invariance to proportional changes in currentperiod values :

P( p0, p1, v0, lv1)=P( p0, p1, v0, v1) for all l> 0

That is, if current period values are all multiplied by thenumber l, then the price index remains unchanged. Putanother way, the price index function P( p0,p1,v0,v1) is(positively) homogeneous of degree zero in the compo-nents of the period 1 value vector v1.

T7: Invariance to proportional changes in baseperiod values:

P( p0, p1, lv0, v1)=P( p0, p1, v0, v1) for all l > 0

That is, if base period values are all multiplied by thenumber l, then the price index remains unchanged. Putanother way, the price index function P( p0, p1, v0, v1) is(positively) homogeneous of degree zero in the compo-nents of the period 0 value vector v0.16.102 T6 and T7 together impose the property that

the price index P does not depend on the absolute mag-nitudes of the value vectors v0 and v1. Using test T6 withl=1=

Pni=1v

1i and using test T7 with l=1=

Pni=1v

0i , it can

be seen that P has the following property:

P( p0, p1, v0, v1)=P( p0, p1, s0, s1) (16:63)

where s0 and s1 are the vectors of expenditure shares forperiods 0 and 1; i.e., the ith component of st issti � vti=

Pnk=1v

tk for t=0,1. Thus the tests T6 and T7

imply that the price index function P is a function of the

two price vectors p0 and p1 and the two vectors ofexpenditure shares, s0 and s1.

16.103 Walsh (1901, p. 104) suggested the spirit oftests T6 and T7 as the following quotation indicates:‘‘What we are seeking is to average the variations in theexchange value of one given total sum of money in relationto the several classes of goods, to which several variations[i.e., the price relatives] must be assigned weights propor-tional to the relative sizes of the classes. Hence the relativesizes of the classes at both the periods must be considered.’’

16.104 Walsh also realized that weighting the ithprice relative ri by the arithmetic mean of the valueweights in the two periods under consideration, (1/2)[v0i +v1

i ] would give too much weight to the expendituresof the period that had the highest level of prices:

At first sight it might be thought sufficient to add upthe weights of every class at the two periods and to divideby two. This would give the (arithmetic) mean size ofevery class over the two periods together. But such anoperation is manifestly wrong. In the first place, the sizesof the classes at each period are reckoned in the money ofthe period, and if it happens that the exchange value ofmoney has fallen, or prices in general have risen, greaterinfluence upon the result would be given to the weightingof the second period; or if prices in general have fallen,greater influence would be given to the weighting of thefirst period. Or in a comparison between two countries,greater influence would be given to the weighting of thecountry with the higher level of prices. But it is plain thatthe one period, or the one country, is as important, in ourcomparison between them, as the other, and the weightingin the averaging of their weights should really be even(Walsh (1901, pp. 104–105)).

16.105 As a solution to the above weighting prob-lem, Walsh (1901, p. 202; 1921a, p. 97) proposed thefollowing geometric price index:

PGW ( p0, p1, v0, v1) �Yni=1

p1i

p0i

� �w(i)(16:64)

where the ith weight in the above formula was defined as

w(i) � (v0i v

1i )

1=2

Pnk=1

(v0kv

1k)

1=2=

(s0i s1i )

1=2

Pnk=1

(s0ks1k)

1=2i=1, . . . , n (16:65)

The second equation in (16.65) shows that Walsh’sgeometric price index PGW ( p0, p1, v0, v1) can also bewritten as a function of the expenditure share vectors, s0

and s1; i.e., PGW ( p0, p1, v0, v1) is homogeneous of degreezero in the components of the value vectors v0 and v1

and so PGW ( p0, p1, v0, v1)=PGW ( p0, p1, s0, s1). ThusWalsh came very close to deriving the Tornqvist–Theilindex defined earlier by equation (16.48).64

64 Walsh’s index could be derived using the same arguments as Theil,except that the geometric average of the expenditure shares (s0i s

1i )

1=2

could be taken as a preliminary probability weight for the ith loga-rithmic price relative, ln ri. These preliminary weights are then nor-malized to add up to unity by dividing by their sum. It is evidentthat Walsh’s geometric price index will closely approximate Theil’sindex using normal time series data. More formally, regarding bothindices as functions of p0, p1, v0, v1, it can be shown thatPW( p0, p1, v0, v1) approximates PT( p0, p1, v0, v1) to the second orderaround an equal price (i.e., p0=p1) and quantity (i.e., q0=q1) point.

THE AXIOMATIC AND STOCHASTIC APPROACHES TO INDEX NUMBER THEORY

305

Invariance and symmetry tests16.106 The next five tests are invariance or symmetry

tests and four of them are direct counterparts tosimilar tests in paragraphs 16.42 to 16.46 above.The first invariance test is that the price index shouldremain unchanged if the ordering of the commoditiesis changed.

T8: Commodity reversal test (or invariance tochanges in the ordering of commodities):

P ( p0*, p1*, v0*, v1*)=P( p0, p1, v0, v1)

where pt* denotes a permutation of the components ofthe vector pt and vt* denotes the same permutation ofthe components of vt for t=0,1.

16.107 The next test asks that the index be invariantto changes in the units of measurement.

T9: Invariance to changes in the units ofmeasurement (commensurability test):

P(a1p01, . . . ,anp0

n;a1p11, . . . ,anp1

n; v01, . . . , v0

n; v11, . . . , v1

n)=P(p0

1, . . . , p0n; p

11, . . . ,p1

n; v01, . . . , v0

n; v11, . . . , v1

n)for all a1 > 0, . . . ,an > 0

That is, the price index does not change if the units ofmeasurement for each commodity are changed. Notethat the expenditure on commodity i during period t, vi

t,does not change if the unit by which commodity i ismeasured changes.

16.108 The last test has a very important implica-tion. Let a1=1=p0

1, . . . , an=1=p0n and substitute these

values for the ai into the definition of the test. The fol-lowing equation is obtained:

P( p0, p1, v0, v1)=P(1n, r, v0, v1) � P*(r, v0, v1) (16:66)

where 1n is a vector of ones of dimension n and r is avector of the price relatives; i.e., the ith componentof r is ri � p1

i =p0i . Thus, if the commensurability test

T9 is satisfied, then the price index P( p0, p1, v0, v1),which is a function of 4n variables, can be written as afunction of 3n variables, P*(r, v0, v1), where r is thevector of price relatives and P*(r, v0, v1) is defined asP(1n, r, v

0, v1).16.109 The next test asks that the formula be

invariant to the period chosen as the base period.

T10: Time reversal test: P( p0, p1, v0, v1)=1=P( p1, p0, v1, v0)

That is, if the data for periods 0 and 1 are interchanged,then the resulting price index should equal the reciprocalof the original price index. Obviously, in the one goodcase when the price index is simply the single price ratio,this test will be satisfied (as are all the other tests listed inthis section).

16.110 The next test is a variant of the circularity test,introduced in paragraphs 15.76 to 15.97 of Chapter 15.65

T11: Transitivity in prices for fixed value weights:

P( p0, p1, vr, vs)P( p1, p2, vr, vs)=P( p0, p2, vr, vs)

In this test, the expenditure weighting vectors, vr and vs,are held constant while making all price comparisons.Given that these weights are held constant, however, thetest asks that the product of the index going from period0 to 1, P( p0, p1, vr, vs), times the index going from period1 to 2, P( p1, p2, vr, vs), should equal the direct indexthat compares the prices of period 2 with those ofperiod 0, P( p0, p2, vr, vs). Obviously, this test is a many-commodity counterpart to a property that holds for asingle price relative.

16.111 The final test in this section captures the ideathat the value weights should enter the index numberformula in a symmetric manner.

T12: Quantity weights symmetry test:

P( p0, p1, v0, v1)=P( p0, p1, v1, v0)

That is, if the expenditure vectors for the two periods areinterchanged, then the price index remains invariant.This property means that, if values are used to weightthe prices in the index number formula, then the period0 values v0 and the period 1 values v1 must enter theformula in a symmetric or even-handed manner.

A mean value test16.112 The next test is a mean value test.

T13: Mean value test for prices:

mini ( p1i =p

0i : i=1, . . . , n) � P( p0, p1, v0, v1)

� maxi ( p1i =p

0i : i=1, . . . , n) (16:67)

That is, the price index lies between the minimum priceratio and the maximum price ratio. Since the price index isto be interpreted as an average of the n price ratios, p1

i =p0i ,

it seems essential that the price index P satisfy this test.

Monotonicity tests16.113 The next two tests in this section are mono-

tonicity tests; i.e., how should the price indexP( p0, p1, v0, v1) change as any component of the twoprice vectors p0 and p1 increases?

T14: Monotonicity in current prices:

P( p0, p1, v0, v1)< P( p0, p2, v0, v1) if p1 < p2

That is, if some period 1 price increases, then the priceindex must increase (holding the value vectors fixed), sothat P( p0, p1, q0, q1) is increasing in the components of p1

for fixed p0, v0 and v1.

T15: Monotonicity in base prices:

P( p0, p1, v0, v1)> P( p2, p1, v0, v1) if p0 < p2

That is, if any period 0 price increases, then the priceindex must decrease, so that P( p0, p1, q0, q1) is decreas-ing in the components of p0 for fixed p1, v0 and v1.65 See equation (15.77) in Chapter 15.

CONSUMER PRICE INDEX: THEORY AND PRACTICE

306

Weighting tests16.114 The above tests are not sufficient to deter-

mine the functional form of the price index; for example,it can be shown that both Walsh’s geometric price indexPGW( p0, p1, v0, v1) defined by equation (16.65) and theTornqvist–Theil index PT ( p0, p1, v0, v1) defined byequation (16.48) satisfy all of the above axioms. Thus, atleast one more test will be required in order to determinethe functional form for the price index P( p0, p1, v0, v1).16.115 The tests proposed thus far do not specify

exactly how the expenditure share vectors s0 and s1 areto be used in order to weight, say, the first price relative,p1

1=p01. The next test says that only the expenditure

shares s01 and s11 pertaining to the first commodity are tobe used in order to weight the prices that correspond tocommodity 1, p1

1 and p01.

T16: Own share price weighting:

P( p01, 1, . . . , 1; p1

1, 1, . . . , 1; v0, v1)

=f p01, p

11, v0

1

�Pnk=1

v0k

� �, v1

1

�Pnk=1

v1k

� �� �(16:68)

Note that vt1=Pn

k=1vtk equals st1, the expenditure share

for commodity 1 in period t. The above test says that ifall the prices are set equal to 1 except the prices forcommodity 1 in the two periods, but the expenditures inthe two periods are arbitrarily given, then the indexdepends only on the two prices for commodity 1 and thetwo expenditure shares for commodity 1. The axiomsays that a function of 2+2n variables is actually only afunction of four variables.66

16.116 Of course, if test T16 is combined with testT8, the commodity reversal test, then it can be seen thatP has the following property:

P(1, . . . , 1, p0i , 1, . . . , 1;1, . . . , 1, p1

i , 1, . . . , 1; v0; v1)

=f p0i , p

1i , v0

i

�Pnk=1

v0k

� �, v1

i

�Pnk=1

v1k

� �� �i=1, . . . , n

(16:69)

Equation (16.69) says that, if all the prices are set equalto 1 except the prices for commodity i in the two peri-ods, but the expenditures in the two periods are arbi-trarily given, then the index depends only on the twoprices for commodity i and the two expenditure sharesfor commodity i.16.117 The final test that also involves the weighting

of prices is the following one:

T17: Irrelevance of price change with tinyvalue weights:

P( p01, 1, . . . , 1; p1

1, 1, . . . , 1; 0; v02, . . . , v0

n; 0; v12, . . . , v1

n)=1

(16:70)

The test T17 says that, if all the prices are set equal to 1except the prices for commodity 1 in the two periods,and the expenditures on commodity 1 are 0 in the twoperiods but the expenditures on the other commoditiesare arbitrarily given, then the index is equal to 1.67 Thus,roughly speaking, if the value weights for commodity 1are tiny, then it does not matter what the price ofcommodity 1 is during the two periods.

16.118 Of course, if test T17 is combined with testT8, the commodity reversal test, then it can be seen thatP has the following property: for i=1, . . . , n:

P(1, . . . , 1, p0i , 1, . . . , 1; 1, . . . , 1, p1

i , 1, . . . , 1; v01, . . . , 0, . . . ,

v0n; v

11, . . . , 0, . . . , v1

n)=1 (16:71)

Equation (16.71) says that, if all the prices are setequal to 1 except the prices for commodity i in the twoperiods, and the expenditures on commodity i are 0during the two periods but the other expenditures inthe two periods are arbitrarily given, then the index isequal to 1.

16.119 This completes the listing of tests for theapproach to bilateral index number theory based onthe weighted average of price relatives. It turns outthat the above tests are sufficient to imply a specificfunctional form for the price index, as seen in the nextsection.

The Tornqvist–Theil price index and thesecond test approach to bilateral indices

16.120 In Appendix 16.1 to this chapter, it is shownthat, if the number of commodities n exceeds two and thebilateral price index function P( p0, p1, v0, v1) satisfies the17 axioms listed above, then P must be the Tornqvist–Theil price index PT( p

0, p1, v0, v1) defined by equation(16.48).68 Thus the 17 properties or tests listed in para-graphs 16.94 to 16.129 provide an axiomatic character-ization of the Tornqvist–Theil price index, just as the 20tests listed in paragraphs 16.30 to 16.73 provided anaxiomatic characterization of the Fisher ideal priceindex.

16.121 Obviously, there is a parallel axiomatic the-ory for quantity indices of the form Q(q0, q1, v0, v1) thatdepend on the two quantity vectors for periods 0 and 1,q0 and q1, as well as on the corresponding two expen-diture vectors, v0 and v1. Thus, if Q(q0, q1, v0, v1) satisfiesthe quantity counterparts to tests T1 to T17, then Qmust be equal to the Tornqvist–Theil quantity index

66 In the economics literature, axioms of this type are known asseparability axioms.

67 Strictly speaking, since all prices and values are required to bepositive, the left-hand side of equation (16.70) should be replaced bythe limit as the commodity 1 values, v0

1 and v11, approach 0.

68 The Tornqvist–Theil price index satisfies all 17 tests, but the proof inAppendix 16.1 does not use all these tests to establish the result in theopposite direction: tests 5, 13, 15 and one of 10 or 12 were not requiredin order to show that an index satisfying the remaining tests must bethe Tornqvist–Theil price index. For alternative characterizations ofthe Tornqvist–Theil price index, see Balk and Diewert (2001) andHillinger (2002).

THE AXIOMATIC AND STOCHASTIC APPROACHES TO INDEX NUMBER THEORY

307

QT (q0, q1, v0, v1) defined, as follows:

lnQT (q0, q1, v0, v1) �Pni=1

1

2(s0i +s1i ) ln

q1i

q0i

� �(16:72)

where as usual, the period t expenditure share on com-modity i, sti , is defined as vti=

Pnk=1v

tk for i=1, . . . , n

and t=0, 1:16.122 Unfortunately, the implicit Tornqvist–Theil

price index, PIT(q0, q1, v0, v1) that corresponds to the

Tornqvist–Theil quantity index QT, defined by equation(16.72) using the product test, is not equal to the directTornqvist–Theil price index PT( p

0, p1, v0, v1), defined byequation (16.48). The product test equation that definesPIT in the present context is given by the followingequation:

PIT (q0, q1, v0, v1) �

Pni=1

v1i

Pni=1

v0i

� �QT (q0, q1, v0, v1)

(16:73)

The fact that the direct Tornqvist–Theil price index PT isnot in general equal to the implicit Tornqvist–Theil priceindex PIT, defined by equation (16.73), is something ofa disadvantage compared to the axiomatic approachoutlined in paragraphs 16.30 to 16.73, which led to theFisher ideal price and quantity indices being considered‘‘best’’. Using the Fisher approach meant that it was notnecessary to decide whether the aim was to find a ‘‘best’’price index or a ‘‘best’’ quantity index: the theory out-lined in paragraphs 16.30 to 16.73 determined bothindices simultaneously. In the Tornqvist–Theil approachoutlined in this section, however, it is necessary to choosebetween a ‘‘best’’ price index or a ‘‘best’’ quantity index.69

16.123 Other tests are of course possible. A coun-terpart to Test T16 in paragraph 16.49, the Paasche andLaspeyres bounding test, is the following geometricPaasche and Laspeyres bounding test:

PGL(p0,p1,v0,v1)� P(p0,p1,v0,v1)�PGP(p0,p1,v0,v1) or

PGP(p0,p1,v0,v1)� P(p0,p1,v0,v1)�PGL(p0,p1,v0,v1)

(16:74)

where the logarithms of the geometric Laspeyres andgeometric Paasche price indices, PGL and PGP, aredefined as follows:

lnPGL( p0, p1, v0, v1) �Pni=1

s0i lnp1i

p0i

� �(16:75)

lnPGP( p0, p1, v0, v1) �Pni=1

s1i lnp1i

p0i

� �(16:76)

As usual, the period t expenditure share on commodity i,sti , is defined as vt1=

Pnk=1 v

tk for i=1, . . . , n and t=0, 1. It

can be shown that the Tornqvist–Theil price index

PT ( p0, p1, v0, v1) defined by equation (16.48) satisfiesthis test, but the geometric Walsh price indexPGW ( p0, p1, v0, v1) defined by equation (16.65) does not.The geometric Paasche and Laspeyres bounding test wasnot included as a primary test in this section because itwas not known a priori what form of averaging of theprice relatives (e.g., geometric or arithmetic or har-monic) would turn out to be appropriate in this testframework. The test (16.74) is an appropriate one if ithas been decided that geometric averaging of the pricerelatives is the appropriate framework, since the geo-metric Paasche and Laspeyres indices correspond to‘‘extreme’’ forms of value weighting in the context ofgeometric averaging and it is natural to require that the‘‘best’’ price index lies between these extreme indices.

16.124 Walsh (1901, p. 408) pointed out a problemwith his geometric price index defined by equation(16.65), which also applies to the Tornqvist–Theil priceindex, PT ( p0, p1, v0, v1), defined by equation (16.48): thesegeometric type indices do not give the ‘‘right’’ answerwhen the quantity vectors are constant (or proportional)over the two periods. In this case, Walsh thought that the‘‘right’’ answer must be the Lowe index, which is the ratioof the costs of purchasing the constant basket during thetwo periods. Put another way, the geometric indices PGW

and PT do not satisfy the fixed basket test T4 in para-graph 16.35. What then was the argument that led Walshto define his geometric average type index PGW? It turnsout that he was led to this type of index by consideringanother test, which will now be explained.

16.125 Walsh (1901, pp. 228–231) derived his test byconsidering the following very simple framework. Letthere be only two commodities in the index and supposethat the expenditure share on each commodity isequal in each of the two periods under consideration.The price index under these conditions is equal toP ( p1

0, p20;p1

1, p21;v1

0, v20;v1

1, v21)=P*(r1, r2;1/2, 1/2;1/2, 1/2):

m(r1, r2), where m(r1, r2) is a symmetric mean of thetwo price relatives, r1 � p1

1=p01 and r2 � p1

2=p02.

70 In thisframework, Walsh then proposed the following pricerelative reciprocal test:

m(r1, r�11 )=1 (16:77)

Thus, if the value weighting for the two commodities isequal over the two periods and the second price relative isthe reciprocal of the first price relative r1, then Walsh(1901, p. 230) argued that the overall price index underthese circumstances ought to equal 1, since the relativefall in one price is exactly counterbalanced by a rise in theother and both commodities have the same expendituresin each period. He found that the geometric mean satis-fied this test perfectly but the arithmetic mean led toindex values greater than 1 (provided that r1 was notequal to 1) and the harmonic mean led to index valuesthat were less than 1, a situation which was not at allsatisfactory.71 Thus he was led to some form of geometric

69 Hillinger (2002) suggested taking the geometric mean of the directand implicit Tornqvist–Theil price indices in order to resolve thisconflict. Unfortunately, the resulting index is not ‘‘best’’ for either setof axioms that were suggested in this section.

70 Walsh considered only the cases where m was the arithmetic, geo-metric and harmonic means of r1 and r2.71‘‘This tendency of the arithmetic and harmonic solutions to run intothe ground or to fly into the air by their excessive demands is clearindication of their falsity’’ (Walsh (1901, p. 231)).

CONSUMER PRICE INDEX: THEORY AND PRACTICE

308

averaging of the price relatives in one of his approachesto index number theory.16.126 A generalization of Walsh’s result is easy to

obtain. Suppose that the mean function, m(r1, r2),satisfies Walsh’s reciprocal test (16.77) and, in addition,m is a homogeneous mean, so that it satisfies the fol-lowing property for all r1>0, r2>0 and l>0:

m(lr1, lr2)=lm(r1, r2): (16:78)

Let r1>0, r2>0. Then

m(r1, r2)=r1

r1

� �m(r1, r2)

=r1mr1

r1,r2

r1

� �using (16:78) with l=

1

r1

=r1m 1,r2

r1

� �=r1 f

r2

r1

� �(16:79)

where the function of one (positive) variable f(z) isdefined as

f (z) � m(1, z) (16:80)

Using equation (16.77):

1=m(r1, r�11 )

=r1

r1

� �m(r1, r

�11 )

=r1m(1, r�21 ) using (16:78) with l=

1

r1(16:81)

Using equation (16.80), equation (16.81) can be re-arranged in the following form:

f (r�21 )=r�1

1 (16:82)

Letting z � r�21 so that z1=2=r�1

1 , equation (16.82)becomes:

f (z)=z1=2 (16:83)

Now substitute equation (16.83) into equation (16.79)and the functional form for the mean function m(r1, r2)is determined:

m(r1, r2)=r1 fr2

r1

� �=r1

r2

r1

� �1=2

= r1=21 r

1=22 (16:84)

Thus, the geometric mean of the two price relatives isthe only homogeneous mean that will satisfy Walsh’sprice relative reciprocal test.16.127 There is one additional test that should be

mentioned. Fisher (1911, p. 401) introduced this test inhis first book that dealt with the test approach toindex number theory. He called it the test of determi-nateness as to prices and described it as follows: ‘‘Aprice index should not be rendered zero, infinity, orindeterminate by an individual price becoming zero.Thus, if any commodity should in 1910 be a glut on

the market, becoming a ‘free good’, that fact ought notto render the index number for 1910 zero.’’ In thepresent context, this test could be interpreted as thefollowing one: if any single price p0

i or p1i tends to

zero, then the price index P( p0, p, v0, v1) should nottend to zero or plus infinity. However, with thisinterpretation of the test, which regards the values vtias remaining constant as the p0

i or p1i tends to zero,

none of the commonly used index number formulaewould satisfy this test. Hence this test should beinterpreted as a test that applies to price indicesP( p0, p1, q0, q1) of the type studied in paragraphs 16.30to 16.73, which is how Fisher intended the test toapply. Thus, Fisher’s price determinateness test shouldbe interpreted as follows: if any single price p0

i or p1i

tends to zero, then the price index P( p0, p, q0, q1)should not tend to zero or plus infinity. With thisinterpretation of the test, it can be verified that Las-peyres, Paasche and Fisher indices satisfy this test butthe Tornqvist–Theil price index does not. Thus, whenusing the Tornqvist–Theil price index, care must betaken to bound the prices away from zero in order toavoid a meaningless index number value.

16.128 Walsh was aware that geometric averagetype indices such as the Tornqvist–Theil price index PT

or Walsh’s geometric price index PGW defined byequation (16.64) become somewhat unstable72 as indi-vidual price relatives become very large or small:

Hence in practice the geometric average is not likely todepart much from the truth. Still, we have seen that whenthe classes [i.e., expenditures] are very unequal and theprice variations are very great, this average may deflectconsiderably (Walsh (1901, p. 373)).

In the cases of moderate inequality in the sizes of theclasses and of excessive variation in one of the prices,there seems to be a tendency on the part of the geometricmethod to deviate by itself, becoming untrustworthy,while the other two methods keep fairly close together(Walsh (1901, p. 404)).

16.129 Weighing all the arguments and tests pre-sented above, it seems that there may be a slight pre-ference for the use of the Fisher ideal price index as asuitable target index for a statistical agency, but, ofcourse, opinions may differ on which set of axioms is themost appropriate to use in practice.

The test properties ofthe Lowe and Young indices

16.130 The Young and Lowe indices were defined inChapter 15. In the present section, the axiomatic prop-erties of these indices with respect to their price argu-ments are developed.73

16.131 Let qb � [qb1, . . . , qbn] and pb � [pb1, . . . , pbn]denote the quantity and price vectors pertaining to somebase year. The corresponding base year expenditure

72 That is, the index may approach zero or plus infinity.73 Baldwin (1990, p. 255) worked out a few of the axiomatic propertiesof the Lowe index.

THE AXIOMATIC AND STOCHASTIC APPROACHES TO INDEX NUMBER THEORY

309

shares can be defined in the usual way as

sbi �pbi q

biPn

k=1

pbkqbk

i=1, . . . , n (16:85)

Let sb � [sb1, . . . , sbn] denote the vector of base yearexpenditure shares. The Young (1812) price indexbetween periods 0 and t is defined as follows:

PY ( p0, pt, sb) �Pni=1

sbiptip0i

� �(16:86)

The Lowe (1823, p. 316) price index74 between periods 0and t is defined as follows:

PLo( p0, pt, qb) �

Pni=1

ptiqbi

Pnk=1

p0kq

bk

=

Pni=1

sbiptipbi

� �Pnk=1

sbkp0k

pbk

! (16:87)

16.132 Drawing on the axioms listed above in thischapter, 12 desirable axioms for price indices of the formP( p0, p1) are listed below. The period 0 and t pricevectors, p0 and pt, are presumed to have strictly positivecomponents.

T1: Positivity: P( p0, pt) > 0 if all prices are positive

T2: Continuity: P( p0, pt) is a continuous functionof prices

T3: Identity test: P( p0, p0)=1

T4: Homogeneity test for period t prices:

P( p0, lpt)=lP( p0, pt) for all l > 0

T5: Homogeneity test for period 0 prices:

P(lp0, pt)=l�1P( p0, pt) for all l > 0

T6: Commodity reversal test: P( pt, p0)=P( p0*, pt*)

where p0* and pt*denote the same permutation of

the components of the price vectors p0 and pt75

T7: Invariance to changes in the units ofmeasurement (commensurability test)

T8: Time reversal test: P( pt, p0)=1=P( p0, pt)

T9: Circularity or transitivity test:

P( p0, p2)=P( p0, p1)P( p1, p2)

T10: Mean value test: min fpti=p0i : i=1, . . . , ng �

P( pt, p0)� max fpti=p0i : i=1, . . . , ng

T11: Monotonicity test with respect to period t prices:

P( p0, pt)< P( p0, pt*) if pt < pt*

T12: Monotonicity test with respect to period 0 prices:

P( p0, pt) > P( p0*, pt) if p0 < p0*

16.133 It is straightforward to show that the Loweindex defined by equation (16.87) satisfies all 12 of theaxioms or tests listed above. Hence the Lowe index hasvery good axiomatic properties with respect to its pricevariables.76

16.134 It is straightforward to show that the Youngindex defined by equation (16.86) satisfies 10 of the 12axioms, failing the time reversal test T8 and the circu-larity test T9. Thus the axiomatic properties of theYoung index are definitely inferior to those of the Loweindex.

74 This index number formula is also precisely Bean and Stine’s (1924,p. 31) Type A index number formula. Walsh (1901, p. 539) initiallymistakenly attributed Lowe’s formula to G. Poulett Scrope (1833),who wrote Principles of Political Economy in 1833 and suggestedLowe’s formula without acknowledging Lowe’s priority. But in hisdiscussion of Fisher’s (1921) paper, Walsh (1921b, p. 543–544) correctshis mistake on assigning Lowe’s formula:

What index number should you then use? It should be this:Pqp1=

Pqp0.

This is the method used by Lowe within a year or two of one hundredyears ago. In my [1901] book, I called it Scrope’s index number; but itshould be called Lowe’s. Note that in it are used quantities neither of abase year nor of a subsequent year. The quantities used should be roughestimates of what the quantities were throughout the period or epoch.

75 In applying this test to the Lowe and Young indices, it is assumedthat the base year quantity vector qb and the base year share vector sb

are subject to the same permutation.76 From the discussion in Chapter 15, it will be recalled that the mainproblem with the Lowe index occurs if the quantity weight vector qb isnot representative of the quantities that were purchased during thetime interval between periods 0 and 1.

CONSUMER PRICE INDEX: THEORY AND PRACTICE

310

Appendix 16.1 Proof of theoptimality of the Tornqvist–Theilprice index in the secondbilateral test approachThe tests (T1, T2, etc.) mentioned in this appendix are thosepresented in paragraphs 16.98 to 16.119.

1. Define ri � p1i =p

0i for i=1, . . . , n. Using T1, T9 and

equation (16.66), P( p0, p1, v0, v1)=P*(r, v0, v1). Using T6, T7and equation (16.63):

P( p0, p1, v0, v1)=P*(r, s0, s1) (A16:1:1)

where st is the period t expenditure share vector for t=0, 1.2. Let x:(x1, . . . , xn) and y:(y1, . . . , yn) be strictly

positive vectors. The transitivity test T11 and equation(A16.1.1) imply that the function P* has the followingproperty:

P*(x; s0, s1)P*(y; s0, s1)=P*(x1y1, . . . , xnyn; s0, s1): (A16:1:2)

3. Using test T1, P*(r, s0, s1)>0 and using test T14,P*(r, s0, s1) is strictly increasing in the components of r. Theidentity test T3 implies that

P*(1n, s0, s1)=1 (A16:1:3)

where 1n is a vector of ones of dimension n. Using a resultattributable to Eichhorn (1978, p. 66), it can be seen that theseproperties of P* are sufficient to imply that there exist positivefunctions ai(s

0, s1) for i=1, . . . , n such that P* has the follow-ing representation:

lnP*(r, s0, s1)=Pni=1

ai(s0, s1) ln ri (A16:1:4)

4. The continuity test T2 implies that the positive func-tions ai (s

0, s1) are continuous. For l>0, the linear homo-geneity test T4 implies that

lnP*(lr,s0,s1)=lnl+lnP*(r,s0,s1)

=Pni=1

ai(s0,s1) lnlri using (A16:1:4)

=Pni=1

ai(s0,s1) lnl+Pni=1

ai(s0,s1) lnri

=Pni=1

ai(s0,s1) lnl+lnP*(r,s0,s1)

using (A16:1:4) (A16:1:5)

Equating the right-hand sides of the first and last lines inequation (A16.1.5) shows that the functions ai(s

0, s1) mustsatisfy the following restriction:Pn

i=1

ai(s0, s1)=1 (A16:1:6)

for all strictly positive vectors s0 and s1.5. Using the weighting test T16 and the commodity

reversal test T8, equations (16.69) hold. Equation (16.69)

combined with the commensurability test T9 implies that P*satisfies the following equation:

P*(1, . . . , 1, ri, 1, . . . , 1;s0, s1)=f (1, ri, s0, s1); i=1, . . . , n

(A16:1:7)

for all ri>0 where f is the function defined in test T16.6. Substitute equation (A16.1.7) into equation (A16.1.4)

in order to obtain the following system of equations:

P*(1, . . . , 1, ri, 1, . . . , 1;s0, s1)=f (1, ri, s0, s1)=ai(s0, s1) ln ri

i=1, . . . , n (A16:1:8)

But equation (A16.1.8) implies that the positive continuousfunction of 2n variables ai(s

0, s1) is constant with respect to allof its arguments except s0i and s1i and this property holds foreach i. Thus each ai(s

0, s1) can be replaced by the positivecontinuous function of two variables bi(s

0i , s

1i ) for i=1, . . . , n.77

Now replace the ai (s0, s1) in equation (A16.1.4) by the bi(s

0i , s

1i )

for i=1, . . . , n and the following representation for P* isobtained:

lnP*(r, s0, s1)=Pni=1

bi(s0i , s

1i ) ln ri (A16:1:9)

7. Equation (A16.1.6) implies that the functions bi(s0i , s

1i )

also satisfy the following restrictions:

Pni=1

s0i =1; andPni=1

s1i =1 impliesPni=1

bi(s0i , s

1i )=1 (A16:1:10)

8. Assume that the weighting test T17 holds and sub-stitute equation (16.71) into equation (A16.1.9) in order toobtain the following equation:

bi(0, 0) lnp1i

p0i

� �=0; i=1, . . . , n (A16:1:11)

Since the p1i and p0

i can be arbitrary positive numbers, it can beseen that equation (A16.1.11) implies

bi(0, 0)=0; i=1, . . . , n (A16:1:12)

9. Assume that the number of commodities n is equalto or greater than 3. Using equations (A16.1.10) and(A16.1.12), Theorem 2 in Aczel (1987, p. 8) can be applied andthe following functional form for each of the bi(s

0i , s

1i ) is

obtained:

bi(s0i , s

1i )=gs0i +(1� g)s1i ; i=1, . . . , n (A16:1:13)

where g is a positive number satisfying 0<g<1.10. Finally, the time reversal test T10 or the quantity

weights symmetry test T12 can be used to show that g mustequal ½. Substituting this value for g back into equation(A16.1.13) and then substituting that equation back intoequation (A16.1.9), the functional form for P* and hence P isdetermined as

lnP( p0, p1, v0, v1)= lnP*(r, s0, s1)=Pni=1

1

2(s0i +s1i ) ln

p1i

p0i

� �:

(A16:1:14)

77 More explicitly, b1(s01, s

11) � a1(s

01, 1, . . . , 1;s11, 1, . . . , 1) and so on.

That is, in defining b1(s01, s

11), the function a1(s

01, 1, . . . , 1;s11, 1, . . . , 1) is

used where all components of the vectors s0 and s1 except the first areset equal to an arbitrary positive number such as 1.

THE AXIOMATIC AND STOCHASTIC APPROACHES TO INDEX NUMBER THEORY

311

17THE ECONOMIC APPROACH TO INDEX NUMBERTHEORY: THE SINGLE-HOUSEHOLD CASE

Introduction

17.1 This chapter and the next cover the econo-mic approach to index number theory. This chapterconsiders the case of a single household, while the fol-lowing chapter deals with the case of many households.A brief outline of the contents of the present chapterfollows.17.2 In paragraphs 17.9 to 17.17, the theory of the

cost of living index for a single consumer or household ispresented. This theory was originally developed by theRussian economist, A.A. Konus (1924). The relation-ship between the (unobservable) true cost of living indexand the observable Laspeyres and Paasche indices willbe explained. It should be noted that, in the economicapproach to index number theory, it is assumed thathouseholds regard the observed price data as given,while the quantity data are regarded as solutionsto various economic optimization problems. Manyprice statisticians find the assumptions made in theeconomic approach to be somewhat implausible. Per-haps the best way to regard the assumptions madein the economic approach is that these assumptionssimply formalize the fact that consumers tend to pur-chase more of a commodity if its price falls relative toother prices.17.3 In paragraphs 17.18 to 17.26, the preferences of

the consumer are restricted compared to the completelygeneral case treated in paragraphs 17.9 to 17.17. Inparagraphs 17.18 to 17.26, it is assumed that the func-tion that represents the consumer’s preferences overalternative combinations of commodities is homo-geneous of degree one. This assumption means that eachindifference surface (the set of commodity bundles thatgive the consumer the same satisfaction or utility) is aradial blow-up of a single indifference surface. With thisextra assumption, the theory of the true cost of livingsimplifies, as will be seen.17.4 In the sections starting with paragraphs 17.27,

17.33 and 17.44, it is shown that the Fisher, Walsh andTornqvist price indices (which emerge as being ‘‘best’’in the various non-economic approaches) are alsoamong the ‘‘best’’ in the economic approach to indexnumber theory. In these sections, the preference func-tion of the single household will be further restrictedcompared to the assumptions on preferences made in theprevious two sections. Specific functional forms for theconsumer’s utility function are assumed and it turns outthat, with each of these specific assumptions, the con-sumer’s true cost of living index can be exactly calcu-lated using observable price and quantity data. Each ofthe three specific functional forms for the consumer’s

utility function has the property that it can approximatean arbitrary linearly homogeneous function to the sec-ond order; i.e., in economics terminology, each of thesethree functional forms is flexible. Hence, using the ter-minology introduced by Diewert (1976), the Fisher,Walsh and Tornqvist price indices are examples ofsuperlative index number formulae.

17.5 In paragraphs 17.50 to 17.54, it is shown that theFisher, Walsh and Tornqvist price indices approximateeach other very closely using ‘‘normal’’ time seriesdata. This is a very convenient result since these threeindex number formulae repeatedly show up as being‘‘best’’ in all the approaches to index number theory.Hence this approximation result implies that it normallywill not matter which of these three indices is chosenas the preferred target index for a consumer priceindex (CPI).

17.6 The Paasche and Laspeyres price indices have avery convenient mathematical property: they are con-sistent in aggregation. For example, if the Laspeyresformula is used to construct sub-indices for, say, food orclothing, then these sub-index values can be treated assub-aggregate price relatives and, using the expenditureshares on these sub-aggregates, the Laspeyres formulacan be applied again to form a two-stage Laspeyres priceindex. Consistency in aggregation means that this two-stage index is equal to the corresponding single-stageindex. In paragraphs 17.55 to 17.60, it is shown that thesuperlative indices derived in the earlier sections are notexactly consistent in aggregation but are approximatelyconsistent in aggregation.

17.7 In paragraphs 17.61 to 17.64, a very interestingindex number formula is derived: the Lloyd (1975) andMoulton (1996a) price index. This index number formulamakes use of the same information that is required inorder to calculate a Laspeyres index (namely, base periodexpenditure shares, base period prices and current periodprices), plus one other parameter (the elasticity of sub-stitution between commodities). If information on thisextra parameter can be obtained, then the resulting indexcan largely eliminate substitution bias and it can becalculated using basically the same information that isrequired to obtain the Laspeyres index.

17.8 The section starting with paragraph 17.65 con-siders the problem of defining a true cost of living indexwhen the consumer has annual preferences over com-modities but faces monthly (or quarterly) prices. Thissection attempts to provide an economic foundation forthe Lowe index studied in Chapter 15. It also providesan introduction to the problems associated with theexistence of seasonal commodities, which are consideredat more length in Chapter 22. The final section deals

313

with situations where there may be a zero price for acommodity in one period, but where the price is non-zero in the other period.

The Konus cost of living indexand observable bounds

17.9 This section deals with the theory of the costof living index for a single consumer (or household)that was first developed by the Russian economist,Konus (1924). This theory relies on the assumption ofoptimizing behaviour on the part of economic agents(consumers or producers). Thus, given a vector of com-modity prices pt that the household faces in a given timeperiod t, it is assumed that the corresponding observedquantity vector qt is the solution to a cost minimizationproblem that involves the consumer’s preferenceor utility function f.1 Thus in contrast to the axiomaticapproach to index number theory, the economicapproach does not assume that the two quantity vec-tors q0 and q1 are independent of the two price vectors p0

and p1. In the economic approach, the period 0 quantityvector q0 is determined by the consumer’s preferencefunction f and the period 0 vector of prices p0 that theconsumer faces, and the period 1 quantity vector q1 isdetermined by the consumer’s preference function f andthe period 1 vector of prices p1.

17.10 The economic approach to index numbertheory assumes that ‘‘the’’ consumer has well-definedpreferences over different combinations of the n con-sumer commodities or items.2 Each combination ofitems can be represented by a positive quantity vectorq:[q1, . . . , qn]. The consumer’s preferences over alter-native possible consumption vectors, q, are assumed tobe representable by a continuous, non-decreasing andconcave3 utility function f. Thus if f (q1)>f (q0), then theconsumer prefers the consumption vector q1 to q0. It isfurther assumed that the consumer minimizes the cost ofachieving the period t utility level ut:f (qt) for periodst=0, 1. Thus we assume that the observed period tconsumption vector qt solves the following period t costminimization problem:

C(ut, pt) � minqPni=1

ptiqi : f (q)=ut � f (qt)� �

=Pni=1

ptiqti for t=0, 1 (17:1)

The period t price vector for the n commodities underconsideration that the consumer faces is pt. Note thatthe solution to the cost or expenditure minimizationproblem (17.1) for a general utility level u and generalvector of commodity prices p defines the consumer’s costfunction, C(u, p). The cost function will be used belowin order to define the consumer’s cost of living priceindex.

17.11 The Konus (1924) family of true cost of livingindices pertaining to two periods where the consumerfaces the strictly positive price vectors p0 � ( p01, . . . , p0n)and p1 � ( p11, . . . , p1n) in periods 0 and 1, respectively, isdefined as the ratio of the minimum costs of achievingthe same utility level u:f (q), where q:(q1, . . . , qn) is apositive reference quantity vector:

PK ( p0, p1, q) � C( f (q), p

1)

C( f (q), p0)(17:2)

Note that definition (17.2) defines a family of priceindices, because there is one such index for each refer-ence quantity vector q chosen.

17.12 It is natural to choose two specific referencequantity vectors q in definition (17.2): the observed baseperiod quantity vector q0 and the current period quan-tity vector q1. The first of these two choices leads to thefollowing Laspeyres–Konus true cost of living index:

PK ( p0, p1, q0) � C( f (q

0), p1)

C( f (q0), p0)

=C( f (q0), p1)Pni=1

p0i q0i

using (17:1) for t=0

=

minqPni=1

p1i qi : f (q)=f (q0)

� �Pni=1

p0i q0i

(17:3)

using the definition of the cost minimization problemthat defines C( f (q0), p1)

�

Pni=1

p1i q0i

Pni=1

p0i q0i

since q0 � (q01, . . . , q0n) is feasible for the minimization

problem

=PL( p0, p1, q0, q1)

where PL is the Laspeyres price index. Thus the (unob-servable) Laspeyres–Konus true cost of living index isbounded from above by the observable Laspeyres priceindex.4

17.13 The second of the two natural choices fora reference quantity vector q in definition (17.2) leadsto the following Paasche–Konus true cost of living

1 For a description of the economic theory of the input and outputprice indices, see Balk (1998a). In the economic theory of the outputprice index, qt is assumed to be the solution to a revenue maximizationproblem involving the output price vector pt.2 In this chapter, these preferences are assumed to be invariant overtime, while in the following chapter, this assumption is relaxed (one ofthe environmental variables could be a time variable that shifts tastes).3Note that f is concave if and only if f (lq1+(1�l)q2)� lf (q1)+(1�l) f (q2) for all 0� l� 1 and all q1 � 0n and q

2 � 0n. Note also thatq� 0N means that each component of the N-dimensional vector q isnon-negative, q� 0n means that each component of q is positive andq>0n means that q� 0n but q=0n; i.e., q is non-negative but at leastone component is positive.

4 This inequality was first obtained by Konus (1924; 1939, p. 17). Seealso Pollak (1983).

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

314

index:

PK ( p0, p1, q1) � C( f (q

1), p1)

C( f (q1), p0)

=

Pni=1

p1i q1i

C( f (q1), p0)using (17:1) for t=1

=

Pni=1

p1i q1i

minqPni=1

p0i qi : f (q)=f (q1)

� � (17:4)

using the definition of the cost minimization problemthat defines

Cð f ðq0Þ; p0Þ �

Pni=1

p1i q1i

Pni=1

p0i q1i

since q1 � (q11, . . . , q1n)

is feasible for the minimization problem and thus

C( f (q1), p0)�Pni=1

p0i q1i and hence

1

C( f (q1), p0)� 1Pn

i=1

p0i q1i

=PP( p0, p1, q0, q1)

where PP is the Paasche price index. Thus the (unobser-vable) Paasche–Konus true cost of living index is boun-ded from below by the observable Paasche price index.5

17.14 It is possible to illustrate the two inequalities(17.3) and (17.4) if there are only two commodities;see Figure 17.1. The solution to the period 0 costminimization problem is the vector q0. The straightline C represents the consumer’s period 0 budget con-straint, the set of quantity points q1, q2 such thatp01q1+p

02q2=p

01q01+p

02q02. The curved line through q

0 isthe consumer’s period 0 indifference curve, the set ofpoints q1, q2 such that f (q1, q2)=f (q

01, q

02); i.e., it is the set

of consumption vectors that give the same utility as theobserved period 0 consumption vector q0. The solutionto the period 1 cost minimization problem is thevector q1. The straight line D represents the consumer’speriod 1 budget constraint, the set of quantity pointsq1, q2 such that p11q1+p

12q2=p

11q11+p

12q12. The curved

line through q1 is the consumer’s period 1 indifferencecurve, the set of points q1, q2 such that f (q1, q2)=f (q11, q

12); i.e., it is the set of consumption vectors that

give the same utility as the observed period 1 con-sumption vector q1. The point q0* solves the hypothe-tical problem of minimizing the cost of achieving thebase period utility level u0:f (q0) when facing the period1 price vector p1= ( p11, p

12). Thus we have C[u

0, p1]=

p11q0*1 +p

12q0*2 and the dashed line A is the corresponding

isocost line p11q1+p12q2=C[u

0, p1]. Note that the hypo-thetical cost line A is parallel to the actual period 1 cost

line D. From equation (17.3), the Laspeyres–Konus trueindex is C[u0, p1]=[ p01q

01+p

02q02], while the ordinary Las-

peyres index is [ p11q01+p

12q02]=[ p

01q01+p

02q02]. Since the

denominators for these two indices are the same, thedifference between the indices is attributable to the dif-ferences in their numerators. In Figure 17.1, this differ-ence in the numerators is expressed by the fact that thecost line through A lies below the parallel cost linethrough B. Now if the consumer’s indifference curvethrough the observed period 0 consumption vector q0

were L-shaped with vertex at q0, then the consumerwould not change his or her consumption pattern inresponse to a change in the relative prices of the twocommodities while keeping a fixed standard of living. Inthis case, the hypothetical vector q0* would coincide withq0, the dashed line through A would coincide with thedashed line through B and the true Laspeyres–Konusindex would coincide with the ordinary Laspeyres index.However, L-shaped indifference curves are not generallyconsistent with consumer behaviour; i.e., when the priceof a commodity decreases, consumers generally demandmore of it. Thus, in the general case, there will be a gapbetween the points A and B. The magnitude of this gaprepresents the amount of substitution bias between thetrue index and the corresponding Laspeyres index; i.e.,the Laspeyres index will generally be greater than thecorresponding true cost of living index, PK(p

0, p1, q0).17.15 Figure 17.1 can also be used to illustrate the

inequality (17.4). First note that the dashed lines E and Fare parallel to the period 0 isocost line throughC. The point q1* solves the hypothetical problem ofminimizing the cost of achieving the current period utilitylevel u1:f (q1) when facing the period 0 price vectorp0=( p01, p

02). Thus we have C[u

1, p0]= p01q1*1 +p

02q1*2 and

the dashed line E is the corresponding isocost linep11q1+p

12q2=C[u

0, p1]. From equation (17.4), thePaasche–Konus true index is [ p11q

11+ p12q

12]=C[u

1, p0],while the ordinary Paasche index is [ p11q

11+p

12q12]=

[ p01q11+p

02q12]. Since the numerators for these two indices

are the same, the difference between the indices is attri-butable to the differences in their denominators. In Figure17.1, this difference in the denominators is expressed bythe fact that the cost line through E lies below the parallelcost line through F. The magnitude of this difference

q2

q1O A B C D E F

q0*

q1 = (q11,q2

1)

q0 = (q10,q2

0) q1*

Figure 17.1 The Laspeyres and Paasche bounds to thetrue cost of living index

5 This inequality is attributable to Konus (1924; 1939, p. 19); see alsoPollak (1983).

THE ECONOMIC APPROACH TO INDEX NUMBER THEORY: THE SINGLE-HOUSEHOLD CASE

315

represents the amount of substitution bias between thetrue index and the corresponding Paasche index; i.e., thePaasche index will generally be less than the correspond-ing true cost of living index, PK(p

0, p1, q1). Note that thisinequality goes in the opposite direction to the previousinequality between the two Laspeyres indices. The reasonfor this change in direction is attributable to the fact thatone set of differences between the two indices takes placein the numerators of the indices (the Laspeyres inequal-ities), while the other set takes place in the denominatorsof the indices (the Paasche inequalities).

17.16 The bound (17.3) on the Laspeyres–Konustrue cost of living index PK(p

0, p1, q0) using the baseperiod level of utility as the living standard is one-sided,as is the bound (17.4) on the Paasche–Konus true cost ofliving index PK(p

0, p1, q1) using the current period levelof utility as the living standard. In a remarkable result,Konus (1924; 1939, p. 20) showed that there exists anintermediate consumption vector q* that is on thestraight line joining the base period consumption vectorq0 and the current period consumption vector q1 suchthat the corresponding (unobservable) true cost of livingindex PK ( p

0, p1, q�) is between the observable Laspeyresand Paasche indices, PL and PP.

6 Thus we have theexistence of a number l* between 0 and 1 such that

PL � PK ( p0, p1, l*q0+(1�l*)q1)� PP orPP � PK ( p0, p1, l*q0+(1�l*)q1)� PL (17:5)

The inequalities (17.5) are of some practical importance.If the observable (in principle) Paasche and Laspeyresindices are not too far apart, then taking a symmetricaverage of these indices should provide a good approx-imation to a true cost of living index where the referencestandard of living is somewhere between the base andcurrent period living standards. To determine the precisesymmetric average of the Paasche and Laspeyres indices,appeal can be made to the results in paragraphs 15.18 to15.32 in Chapter 15, and the geometric mean of thePaasche and Laspeyres indices can be justified as beingthe ‘‘best’’ average, which is the Fisher price index. Thusthe Fisher ideal price index receives a fairly strong jus-tification as a good approximation to an unobservabletheoretical cost of living index.

17.17 The bounds (17.3)–(17.5) are the best that canbe obtained on true cost of living indices without mak-ing further assumptions. Further assumptions are madebelow on the class of utility functions that describe theconsumer’s tastes for the n commodities under con-sideration. With these extra assumptions, the con-sumer’s true cost of living can be determined exactly.

The true cost of living index whenpreferences are homothetic

17.18 Up to now, the consumer’s preference func-tion f did not have to satisfy any particular homogeneity

assumption. For the remainder of this section, it isassumed that f is (positively) linearly homogeneous.7 Inthe economics literature, this is known as the assump-tion of homothetic preferences.8 This assumption is notstrictly justified from the viewpoint of actual economicbehaviour, but it leads to economic price indices that areindependent of the consumer’s standard of living.9

Under this assumption, the consumer’s expenditure orcost function, C(u, p) defined by equation (17.1),decomposes as follows. For positive commodity pricesp� 0N and a positive utility level u, then, using thedefinition of C as the minimum cost of achieving thegiven utility level u, the following equalities can bederived:

C(u, p) � minqPni=1

piqi : f (q1, . . . , qn)� u� �

=minqPni=1

piqi :1

uf (q1, . . . , qn)� 1

� �dividing by u > 0

=minqPni=1

piqi : fq1

u, . . . ,

qn

u

� �� 1

� �using the linear homogeneity of f

=uminqPni=1

piqi

u: fq1

u, . . . ,

qn

u

� �� 1

� �

=uminzPni=1

pizi : f (z1, . . . , zn)� 1� �

letting

zi=qi

u=uC(1, p) using definition (17:1)

=uc( p) (17:6)

6For more recent applications of the Konus method of proof, seeDiewert (1983a, p. 191) for an application to the consumer context andDiewert (1983b, pp. 1059–1061) for an application to the producercontext.

7 The linear homogeneity property means that f satisfies the followingcondition: f (lq)=lf (q) for all l>0 and all q� 0n. This assumption isfairly restrictive in the consumer context. It implies that each indif-ference curve is a radial projection of the unit utility indifference curve.It also implies that all income elasticities of demand are unity, which iscontradicted by empirical evidence.8More precisely, Shephard (1953) defined a homothetic function to bea monotonic transformation of a linearly homogeneous function.However, if a consumer’s utility function is homothetic, it can alwaysbe rescaled to be linearly homogeneous without changing consumerbehaviour. Hence, the homothetic preferences assumption can simplybe identified with the linear homogeneity assumption.9 This particular branch of the economic approach to index numbertheory is attributable to Shephard (1953; 1970) and Samuelson andSwamy (1974). Shephard in particular realized the importance of thehomotheticity assumption in conjunction with separability assump-tions in justifying the existence of sub-indices of the overall cost ofliving index. It should be noted that, if the consumer’s change in realincome or utility between the two periods under consideration is nottoo large, then assuming that the consumer has homothetic pre-ferences will lead to a true cost of living index which is very close toLaspeyres–Konus and Paasche–Konus true cost of living indicesdefined by equations (17.3) and (17.4). Another way of justifying thehomothetic preferences assumption is to use equation (17.49), whichjustifies the use of the superlative Tornqvist–Theil index PT in thecontext of non-homothetic preferences. Since PT is usually numeri-cally close to other superlative indices that are derived using thehomothetic preferences assumption, it can be seen that the assumptionof homotheticity will usually not be empirically misleading in theindex number context.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

316

where c(p):C(1, p) is the unit cost function that corre-sponds to f.10 It can be shown that the unit cost functionc(p) satisfies the same regularity conditions that fsatisfies; i.e., c(p) is positive, concave and (positively)linearly homogeneous for positive price vectors.11 Sub-stituting equation (17.6) into equation (17.1) and usingut=f (qt) leads to the following equation:

Pni=1

ptiqti=c( p

t)f (qt) for t=0, 1 (17:7)

Thus, under the linear homogeneity assumption on theutility function f, observed period t expenditure on the ncommodities is equal to the period t unit cost c(pt) ofachieving one unit of utility times the period t utilitylevel, f (qt). Obviously, the period t unit cost, c(pt), canbe identified as the period t price level Pt and the periodt level of utility, f (qt), as the period t quantity level Qt.12

17.19 The linear homogeneity assumption on theconsumer’s preference function f leads to a simplifica-tion for the family of Konus true cost of living indices,PK(p

0, p1, q), defined by equation (17.2). Using thisdefinition for an arbitrary reference quantity vector q:

PK ( p0, p1, q) � C( f (q), p

1)

C( f (q), p0)

=c( p1)f (q)

c( p0)f (q)using (17:6) twice

=c( p1)

c( p0)(17:8)

Thus under the homothetic preferences assumption, theentire family of Konus true cost of living indices collapsesto a single index, c(p1)/c(p0), the ratio of the minimumcosts of achieving unit utility level when the consumer facesperiod 1 and 0 prices respectively. Put another way, underthe homothetic preferences assumption, PK(p

0, p1, q) isindependent of the reference quantity vector q.17.20 If the Konus true cost of living index defined

by the right-hand side of equation (17.8) is used as theprice index concept, then the corresponding implicitquantity index defined using the product test (i.e., theproduct of the price index times the quantity index is

equal to the value ratio) has the following form:

Q( p0, p1, q0, q1) �

Pni=1

p1i q1i

Pni=1

ptiqtiPK ( p

0, p1, q)

=c( p1)f (q1)

c( p0)f (q0)PK ( p0, p1, q)using ð17:7Þ

twice

=c( p1)f (q1)

c( p0)f (q0) c( p1)=c( p0)f g using (17:8)

=f (q1)

f (q0)(17:9)

Thus, under the homothetic preferences assumption, theimplicit quantity index that corresponds to the true costof living price index c(p1)/c(p0) is the utility ratio f (q1)/f (q0). Since the utility function is assumed to be homo-geneous of degree one, this is the natural definition for aquantity index.

17.21 In subsequent material, two additional resultsfrom economic theory will be needed: Wold’s Identityand Shephard’s Lemma. Wold’s (1944, pp. 69–71; 1953,p. 145) Identity is the following result. Assuming thatthe consumer satisfies the cost minimization assumptions(17.1) for periods 0 and 1 and that the utility function f isdifferentiable at the observed quantity vectors q0 and q1,it can be shown13 that the following equation holds:

ptiPnk=1

ptkqtk

=

@f (qt)

@qiPnk=1

qtk@f (qt)

@qk

for t=0, 1 and k=1, . . . , n (17:10)

where @f (qt)/@qi denotes the partial derivative of theutility function f with respect to the ith quantity qi,evaluated at the period t quantity vector qt.

17.22 If the homothetic preferences assumption ismade and it is assumed that the utility function islinearly homogeneous, then Wold’s Identity can besimplified into an equation that will prove to be veryuseful:14

ptiPnk=1

ptkqtk

=@f (qt)=@qif (qt)

for t=0, 1 and k=1, . . . , n (17:11)

10Economists will recognize the producer theory counterpart to theresult C(u, p)=uc(p): if a producer’s production function f is subject toconstant returns to scale, then the corresponding total cost functionC(u, p) is equal to the product of the output level u times the unit costc(p).11Obviously, the utility function f determines the consumer’s costfunction C(u, p) as the solution to the cost minimization problem in thefirst line of equation (17.6). Then the unit cost function c(p) is definedas C(1, p). Thus f determines c. But we can also use c to determinef under appropriate regularity conditions. In the economics literature,this is known as duality theory. For additional material on dualitytheory and the properties of f and c, see Samuelson (1953), Shephard(1953) and Diewert (1974a; 1993b, pp.107–123).12 There is also a producer theory interpretation of the above theory;i.e., let f be the producer’s (constant returns to scale) productionfunction, let p be a vector of input prices that the producer faces, letq be an input vector and let u=f (q) be the maximum output that canbe produced using the input vector q. C(u; p) � minq f

Pni=1 piqi:

f ðqÞ � ug is the producer’s cost function in this case and c(pt) can beidentified as the period t input price level, while f (qt) is the period taggregate input level.

13 To prove this, consider the first-order necessary conditions for thestrictly positive vector qt to solve the period t cost minimization pro-blem. The conditions of Lagrange with respect to the vector of qvariables are: pt=lt!f (qt), where lt is the optimal Lagrange multi-plier and !f (qt) is the vector of first-order partial derivatives of fevaluated at qt. Note that this system of equations is the price equals aconstant times marginal utility equations that are familiar to econo-mists. Now take the inner product of both sides of this equation withrespect to the period t quantity vector qt and solve the resultingequation for lt. Substitute this solution back into the vector equationpt=lt!f (qt) and equation (17.10) is obtained.14Differentiate both sides of the equation f (lq)=lf (q) with respect tol, and then evaluate the resulting equation at l=1. The equationPni=1fi(q)qi=f (q) is obtained where fi(q):@f (q)/@qi.

THE ECONOMIC APPROACH TO INDEX NUMBER THEORY: THE SINGLE-HOUSEHOLD CASE

317

17.23 Shephard’s (1953, p. 11) Lemma is the fol-lowing result. Consider the period t cost minimizationproblem defined by equation (17.1). If the cost functionC(u, p) is differentiable with respect to the componentsof the price vector p, then the period t quantity vectorqt is equal to the vector of first-order partial derivativesof the cost function with respect to the componentsof p:

qti=@C(ut, pt)

@pifor i=1, . . . , n and t=0, 1 (17:12)

17.24 To explain why equation (17.12) holds, con-sider the following argument. Because it is assumedthat the observed period t quantity vector qt solves thecost minimization problem defined by C(ut, pt), then qt

must be feasible for this problem so it must be the casethat f (qt)=ut. Thus, qt is a feasible solution for thefollowing cost minimization problem where the generalprice vector p has replaced the specific period t pricevector pt:

C(ut, p) � minqPni=1

piqi : f (q1, . . . , qn)� ut� �

�Pni=1

piqti

(17:13)

where the inequality follows from the fact that qt �(qt1, . . . , qtn) is a feasible (but usually not optimal) solu-tion for the cost minimization problem in equation(17.13). Now define for each strictly positive price vectorp the function g(p) as follows:

g( p) �Pni=1

piqti�C(ut, p) (17:14)

where, as usual, p:(p1, . . . , pn). Using equations (17.13)and (17.1), it can be seen that g(p) is minimized (over allstrictly positive price vectors p) at p=pt. Thus the first-order necessary conditions for minimizing a differenti-able function of n variables hold, which simplify toequation (17.12).

17.25 If the homothetic preferences assumption ismade and it is assumed that the utility function is linearlyhomogeneous, then using equation (17.6), Shephard’sLemma (17.12) becomes:

qti=ut @c( p

t)

@pifor i=1, . . . , n and t=0, 1 (17:15)

Combining equations (17.15) and (17.7), the followingequation is obtained:

qtiPnk=1

ptkqtk

=@c( pt)

@pi

�c( pt) for i=1, . . . , n and t=0, 1

(17:16)

17.26 Note the symmetry of equation (17.16) withequation (17.11). It is these two equations that will beused in subsequent material in this chapter.

Superlative indices: The Fisherideal index

17.27 Suppose the consumer has the following uti-lity function:

f (q1, . . . , qn) �ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPni=1

Pnk=1

aikqiqk,

s

where aik=aki for all i and k (17:17)

Differentiating f (q) defined by equation (17.17) withrespect to qi yields the following equation:

fi(q)=1

2

2Pnk=1

aikqkffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPnj=1

Pnk=1

ajkqjqk

s for i=1, . . . , n

=

Pnk=1

aikqk

f (q)(17:18)

where fi(q):@f (qt)/@qi. In order to obtain the firstequation in (17.18), it is necessary to use the symmetryconditions, aik=aki. Now evaluate the second equationin (17.18) at the observed period t quantity vectorqt � (qt1, . . . , qtn) and divide both sides of the resultingequation by f (qt). The following equations are obtained:

fi(qt)

f (qt)=

Pnk=1

aikqtk

f (qt)f g2for t=0, 1 and i=1, . . . , n (17:19)

Assume cost minimizing behaviour for the consumer inperiods 0 and 1. Since the utility function f defined byequation (17.17) is linearly homogeneous and differ-entiable, equation (17.11) will hold. Now recall thedefinition of the Fisher ideal quantity index, QF, definedearlier in Chapter 15:

QF ( p0, p1, q0, q1)=

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPni=1

p0i q1i

Pnk=1

p0kq0k

vuuuuutffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPni=1

p1i q1i

Pnk=1

p1kq0k

vuuuuut

=

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPni=1

fi(q0)q1if (q0)

s ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPni=1

p1i q1i

Pnk=1

p1kq0k

vuuuuutusing equation (17:11) for t=0

=

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPni=1

fi(q0)q1if (q0)

s , ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPnk=1

p1kq0k

Pni=1

p1i q1i

vuuuuut

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

318

=

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPni=1

fi(q0)q1if (q0)

s , ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPni=1

fi(q1)q0if (q1)

s

using equation (17:11) for t=1

=

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPni=1

Pnk=1

aikq0k

q1i

f (q0)f g2

s , ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPni=1

Pnk=1

aikq1k

q0i

f (q1)f g2

s

using equation (17:19)

=

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi1

f (q0)f g2

s , ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi1

f (q1)f g2

s

using equation (17:17) and cancelling terms

=f (q1)

f (q0)(17:20)

Thus under the assumption that the consumer engagesin cost-minimizing behaviour during periods 0 and 1and has preferences over the n commodities that corre-spond to the utility function defined by equation (17.17),the Fisher ideal quantity index QF is exactly equal to thetrue quantity index, f (q1Þ=f ðq0).1517.28 As was noted in paragraphs 15.18 to 15.23 of

Chapter 15, the price index that corresponds to theFisher quantity index QF using the product test (15.3) isthe Fisher price index PF, defined by equation (15.12).Let c(p) be the unit cost function that corresponds tothe homogeneous quadratic utility function f defined byequation (17.17). Then using equations (17.16) and(17.20), it can be seen that

PF ( p0, p1, q0, q1)=

c( p1)

c( p0)(17:21)

Thus, under the assumption that the consumer engagesin cost-minimizing behaviour during periods 0 and 1and has preferences over the n commodities that corre-spond to the utility function defined by equation (17.17),the Fisher ideal price index PF is exactly equal to thetrue price index, c(p1)/c(p0).17.29 A twice continuously differentiable function

f (q) of n variables q:(q1, . . . , qn) can provide a second-order approximation to another such function f *(q)around the point q*, if the level and all the first-orderand second-order partial derivatives of the two func-tions coincide at q*. It can be shown16 that the homo-geneous quadratic function f defined by equation (17.17)can provide a second-order approximation to an arbi-trary f* around any (strictly positive) point q* in theclass of linearly homogeneous functions. Thus thehomogeneous quadratic functional form defined byequation (17.17) is a flexible functional form.17 Diewert(1976, p. 117) termed an index number formulaQ(p0, p1, q0, q1) that was exactly equal to the truequantity index f (q1)/f (q0) (where f is a flexible functional

form) a superlative index number formula.18 Equation(17.20) and the fact that the homogeneous quadraticfunction f defined by equation (17.17) is a flexiblefunctional form show that the Fisher ideal quantityindex QF defined by equation (15.14) is a superlativeindex number formula. Since the Fisher ideal price indexPF satisfies equation (17.21), where c(p) is the unit costfunction that is generated by the homogeneous quad-ratic utility function, PF is also called a superlative indexnumber formula.

17.30 It is possible to show that the Fisher ideal priceindex is a superlative index number formula by a dif-ferent route. Instead of starting with the assumption thatthe consumer’s utility function is the homogeneousquadratic function defined by equation (17.17), it ispossible to start with the assumption that the consumer’sunit cost function is a homogeneous quadratic.19 Thus,suppose that the consumer has the following unit costfunction:

c( p1, . . . , pn) �ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPni=1

Pnk=1

bik pi pk

s

where bik=bki for all i and k: (17:22)

Differentiating c(p) defined by equation (17.22) withrespect to pi yields the following equations:

ci( p)=1

2

2Pnk=1

bik pkffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPnj=1

Pnk=1

bjk pj pk

s for i=1, . . . , n

=

Pnk=1

bik pk

c(q)(17:23)

where ci(p):@c(pt)/@pi. In order to obtain the firstequation in (17.23), it is necessary to use the symmetryconditions. Now evaluate the second equation in (17.23)at the observed period t price vector pt � ( pt1, . . . , ptn)and divide both sides of the resulting equation by c(pt).The following equation is obtained:

ci( pt)

c( pt)=

Pnk=1

bikptk

c( pt)f g2for t=0, 1 and i=1, . . . , n (17:24)

As cost-minimizing behaviour for the consumer in per-iods 0 and 1 is being assumed and, since the unit costfunction c defined by equation (17.22) is differentiable,equations (17.16) will hold. Now recall the definition of

15For the early history of this result, see Diewert (1976, p. 184).16 See Diewert (1976, p. 130) and let the parameter r equal 2.17Diewert (1974a, p. 133) introduced this term into the economicsliterature.

18 Fisher (1922, p. 247) used the term superlative to describe the Fisherideal price index. Thus, Diewert adopted Fisher’s terminology butattempted to give some precision to Fisher’s definition of super-lativeness. Fisher defined an index number formula to be superlativeif it approximated the corresponding Fisher ideal results using hisdata set.19Given the consumer’s unit cost function c(p), Diewert (1974a, p. 112)showed that the corresponding utility function f (q) can be defined asfollows: for a strictly positive quantity vector q, f (q) � 1=maxpfPn

i=1piqi:c( p)=1g.

THE ECONOMIC APPROACH TO INDEX NUMBER THEORY: THE SINGLE-HOUSEHOLD CASE

319

the Fisher ideal price index, PF, given by equation(15.12) in Chapter 15:

PF ( p0, p1, q0, q1)=

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPni=1

p1i q0i

Pnk=1

p0kq0k

vuuuuutffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPni=1

p1i q1i

Pnk=1

p0kq1k

vuuuuut

=

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPni=1

p1ici( p0)

c( p0)

s ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPni=1

p1i q1i

Pnk=1

p0kq1k

vuuuuutusing equation (17:16) for t=0

=

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPni=1

p1ici( p0)

c( p0)

s , ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPnk=1

p0kq1k

Pni=1

p1i q1i

vuuuuut

=

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPni=1

p1ici( p0)

c( p0)

s , ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPni=1

p0ici( p1)

c( p1)

s

using equation (17:16) for t=1

=

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi1

c( p0)f g2

s , ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi1

c( p1)f g2

s

using equation (17:22) and cancelling terms

=c( p1)

c( p0): (17:25)

Thus, under the assumption that the consumer engagesin cost-minimizing behaviour during periods 0 and 1and has preferences over the n commodities that corre-spond to the unit cost function defined by equation(17.22), the Fisher ideal price index PF is exactly equal tothe true price index, c(p1)/c(p0).20

17.31 Since the homogeneous quadratic unit costfunction c(p) defined by equation (17.22) is also aflexible functional form, the fact that the Fisher idealprice index PF exactly equals the true price indexc(p1)/c(p0) means that PF is a superlative index numberformula.21

17.32 Suppose that the bik coefficients in equation(17.22) satisfy the following restrictions:

bik=bibk for i, k=1, . . . , n (17:26)

where the n numbers bi are non-negative. In this specialcase of equation (17.22), it can be seen that the unit cost

function simplifies as follows:

c( p1, . . . , pn) �ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPni=1

Pnk=1

bibkpipk

s

=

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPni=1

bipiPnk=1

bkpk

s=Pni=1

bipi (17:27)

Substituting equation (17.27) into Shephard’s Lemma(17.15) yields the following expressions for the period tquantity vectors, qt:

qti=ut @c( p

t)

@pi=biu

t i=1, . . . , n; t=0, 1 (17:28)

Thus if the consumer has the preferences that corre-spond to the unit cost function defined by equation(17.22) where the bik satisfy the restrictions (17.26),then the period 0 and 1 quantity vectors are equalto a multiple of the vector b:(b1, . . . , bn); i.e., q

0= b u0

and q1=b u1. Under these assumptions, the Fisher,Paasche and Laspeyres indices, PF, PP and PL, allcoincide. The preferences which correspond to theunit cost function defined by equation (17.27) are,however, not consistent with normal consumer beha-viour since they imply that the consumer will not sub-stitute away from more expensive commodities tocheaper commodities if relative prices change goingfrom period 0 to 1.

Quadratic mean of order rsuperlative indices

17.33 It turns out that there are many other super-lative index number formulae; i.e., there exist manyquantity indices Q(p0, p1, q0, q1) that are exactly equalto f (q1)/f (q0) and many price indices P( p0, p1, q0, q1) thatare exactly equal to c(p1)/c(p0), where the aggregatorfunction f or the unit cost function c is a flexible func-tional form. Two families of superlative indices aredefined below.

17.34 Suppose the consumer has the followingquadratic mean of order r utility function.22

f r(q1, . . . , qn) �ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPni=1

Pnk=1

aikqr=2i q

r=2k

r

s(17:29)

where the parameters aik satisfy the symmetry condi-tions aik=aki for all i and k and the parameter r satisfiesthe restriction r=0. Diewert (1976, p. 130) showed thatthe utility function f r defined by equation (17.29) isa flexible functional form; i.e., it can approximate anarbitrary twice continuously differentiable linearlyhomogeneous functional form to the second order. Notethat when r=2, f r equals the homogeneous quadraticfunction defined by equation (17.17).

20 This result was obtained by Diewert (1976, pp. 133–134).21Note that it has been shown that the Fisher index PF is exact for thepreferences defined by equation (17.17), as well as the preferences thatare dual to the unit cost function defined by equation (17.22). Thesetwo classes of preferences do not coincide in general. However, if the nby n symmetric matrix A of the aik has an inverse, then it can be shownthat the n by n matrix B of the bik will equal A

�1. 22 The terminology is attributable to Diewert (1976, p. 129).

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

320

17.35 Define the quadratic mean of order r quantityindex Qr by:

Qr( p0, p1, q0, q1) �

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPni=1

s0i (q1i =q

0i )r=2r

sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPni=1

s1i (q1i =q

0i )r=2r

s (17:30)

where sti � ptiqti=Pnk=1p

tkqtk is the period t expenditure

share for commodity i as usual.17.36 Using exactly the same techniques as were

used in paragraphs 17.27 to 17.32, it can be shown thatQr is exact for the aggregator function f r defined byequation (17.29); i.e., the following exact relationshipbetween the quantity index Qr and the utility function f r

holds:

Qr( p0, p1, q0, q1)=f r(q1)

f r(q0)(17:31)

Thus under the assumption that the consumer engagesin cost-minimizing behaviour during periods 0 and 1and has preferences over the n commodities that corre-spond to the utility function defined by equation (17.29),the quadratic mean of order r quantity index QF isexactly equal to the true quantity index, f r(q1)/f r(q0).23

Since Qr is exact for f r and f r is a flexible functionalform, it can be seen that the quadratic mean of order rquantity index Qr is a superlative index for each r=0.Thus there is an infinite number of superlative quantityindices.17.37 For each quantity index Qr, the product test

(15.3) in Chapter 15 can be used in order to define thecorresponding implicit quadratic mean of order r priceindex Pr*:

Pr*( p0, p1, q0, q1) �

Pni=1

p1i q1i

Pni=1

p0i q0i Q

r( p0, p1, q0, q1)

=cr*( p1)

cr*( p0)

(17:32)

where cr* is the unit cost function that corresponds tothe aggregator function f r defined by equation (17.29).For each r=0, the implicit quadratic mean of order rprice index Pr* is also a superlative index.17.38 When r=2, Qr defined by equation (17.30)

simplifies to QF, the Fisher ideal quantity index, and Pr*

defined by equation (17.32) simplifies to PF, the Fisherideal price index. When r=1, Qr defined by equation(17.30) simplifies to:

Q1( p0, p1, q0, q1) �

Pni=1

s0i

ffiffiffiffiffiq1iq0i

s

Pni=1

s1i

ffiffiffiffiffiq0iq1i

s =

Pni=1

p1i q1i

Pni=1

p0i q0i

Pni=1

p0i q0i

ffiffiffiffiffiq1iq0i

s

Pni=1

p1i q1i

ffiffiffiffiffiq0iq1i

s

=

Pni=1

p1i q1i

Pni=1

p0i q0i

Pni=1

p0iffiffiffiffiffiffiffiffiffiq0i q

1i

pPni=1

p1iffiffiffiffiffiffiffiffiffiq0i q

1i

p

=

Pni=1

p1i q1i

Pni=1

p0i q0i

,Pni=1

p1iffiffiffiffiffiffiffiffiffiq0i q

1i

pPni=1

p0iffiffiffiffiffiffiffiffiffiq0i q

1i

p

=

Pni=1

p1i q1i

Pni=1

p0i q0i

=PW ( p0, p1, q0, q1) (17:33)

where PW is the Walsh price index defined previously byequation (15.19) in Chapter 15. Thus P1* is equal to PW,the Walsh price index, and hence it is also a superlativeprice index.

17.39 Suppose the consumer has the followingquadratic mean of order r unit cost function:24

cr( p1, . . . , pn) �ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPni=1

Pnk=1

bikpr=2i p

r=2k

r

s(17:34)

where the parameters bik satisfy the symmetry condi-tions bik=bki for all i and k, and the parameter r satisfiesthe restriction r=0. Diewert (1976, p. 130) showed thatthe unit cost function cr defined by equation (17.34) isa flexible functional form; i.e., it can approximate anarbitrary twice continuously differentiable linearlyhomogeneous functional form to the second order. Notethat when r=2, cr equals the homogeneous quadraticfunction defined by equation (17.22).

17.40 Define the quadratic mean of order r priceindex Pr by:

Pr( p0, p1, q0, q1) �

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPni=1

s0ip1ip0i

� �r=2r

sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPni=1

s1ip1ip0i

� �r=2r

s (17:35)

where sti � ptiqti=Pnk=1 p

tkqtk is the period t expenditure

share for commodity i as usual.17.41 Using exactly the same techniques as were

used in paragraphs 17.27 to 17.32, it can be shown thatPr is exact for the aggregator function defined byequation (17.34); i.e., the following exact relationshipbetween the index number formula Pr and the unit costfunction cr holds:

Pr( p0, p1, q0, q1)=cr( p1)

cr( p0)(17:36)

Thus, under the assumption that the consumer engagesin cost-minimizing behaviour during periods 0 and 1,and has preferences over the n commodities that cor-respond to the unit cost function defined by equation

23 See Diewert (1976, p. 130).

24 This terminology is attributable to Diewert (1976, p. 130), this unitcost function being first defined by Denny (1974).

THE ECONOMIC APPROACH TO INDEX NUMBER THEORY: THE SINGLE-HOUSEHOLD CASE

321

(17.34), the quadratic mean of order r price index PF isexactly equal to the true price index, cr(p1)/cr(p0).25

Since Pr is exact for cr and cr is a flexible functionalform, it can be seen that the quadratic mean of order rprice index Pr is a superlative index for each r=0.Thus there are an infinite number of superlative priceindices.

17.42 For each price index Pr, the product test (15.3)in Chapter 15 can be used in order to define the corre-sponding implicit quadratic mean of order r quantityindex Qr*:

Qr*( p0, p1, q0, q1) �

Pni=1

p1i q1i

Pni=1

p0i q0i Pr( p0, p1, q0, q1)

=f r*( p1)

f r*( p0)

(17:37)

where f r* is the aggregator function that correspondsto the unit cost function cr defined by equation(17.34).26 For each r=0, the implicit quadratic meanof order r quantity index Qr* is also a superlativeindex.

17.43 When r=2, Pr defined by equation (17.35)simplifies to PF, the Fisher ideal price index, and Q

r*defined by equation (17.37) simplifies to QF, the Fisherideal quantity index. When r=1, Pr defined by equation(17.35) simplifies to:

P1( p0, p1, q0, q1) �

Pni=1

s0i

ffiffiffiffiffip1ip0i

s

Pni=1

s1i

ffiffiffiffiffip0ip1i

s =

Pni=1

p1i q1i

Pni=1

p0i q0i

Pni=1

p0i q0i

ffiffiffiffiffip1ip0i

s

Pni=1

p1i q1i

ffiffiffiffiffip0ip1i

s

=

Pni=1

p1i q1i

Pni=1

p0i q0i

Pni=1

q0iffiffiffiffiffiffiffiffiffip0i p

1i

pPni=1

q1iffiffiffiffiffiffiffiffiffip0i p

1i

p

=

Pni=1

p1i q1i

Pni=1

p0i q0i

,Pni=1

q1iffiffiffiffiffiffiffiffiffip0i p

1i

pPni=1

q0iffiffiffiffiffiffiffiffiffip0i p

1i

p

=

Pni=1

p1i q1i

Pni=1

p0i q0i

=QW ( p0, p1, q0, q1) (17:38)

where QW is the Walsh quantity index defined pre-viously in footnote 30 of Chapter 15. Thus Q1* is equalto QW, the Walsh quantity index, and hence it is also asuperlative quantity index.

Superlative indices:The Tornqvist index

17.44 In this section, the same assumptions thatwere made on the consumer in paragraphs 17.9 to 17.17are made here. In particular, it is not assumed that theconsumer’s utility function f is necessarily linearlyhomogeneous as in paragraphs 17.18 to 17.43.

17.45 Before the main result is derived, a pre-liminary result is required. Suppose the function of nvariables, f (z1, . . . , zn):f (z), is quadratic; i.e.,

f (z1, . . . , zn) � a0+Pni=1

aizi+1

2

Pni=1

Pnk=1

aikzizk

and aik=aki for all i and k (17:39)

where the ai and the aik are constants. Let fi(z) denotethe first-order partial derivative of f evaluated at zwith respect to the ith component of z, zi. Let fik(z)denote the second-order partial derivative of f withrespect to zi and zk. Then it is well known that the sec-ond-order Taylor series approximation to a quadraticfunction is exact; i.e., if f is defined by equation (17.39),then for any two points, z0 and z1, the following equa-tion holds:

f (z1)�f (z0)=Pni=1

fi(z0) z1i�z0i� �

+1

2

Pni=1

Pnk=1

fik(z0) z1i�z0i� �

z1k�z0k� �

(17:40)

It is less well known that an average of two first-orderTaylor series approximations to a quadratic function isalso exact; i.e., if f is defined by equation (17.39) above,then for any two points, z0 and z1, the following equa-tion holds:27

f (z1)�f (z0)=1

2

Pni=1

fi(z0)+fi(z

1)� �

z1i�z0i� �

(17:41)

Diewert (1976, p. 118) and Lau (1979) showed thatequation (17.41) characterized a quadratic function andcalled the equation the quadratic approximation lemma.In this chapter, equation (17.41) will be called thequadratic identity.

17.46 Suppose that the consumer’s cost func-tion28 C(u, p), has the following translog functionalform:29

lnC(u, p) � a0+Pni=1

ai ln pi+1

2

Pni=1

Pnk=1

aik ln pi ln pk+b0 ln u

+Pni=1

bi ln pi ln u+1

2b00( ln u)

2 (17:42)

where ln is the natural logarithm function and theparameters ai, aik, and bi satisfy the following

25 See Diewert (1976, pp. 133–134).26 The function f r* can be defined by using cr as follows: f r*(q) �1=maxp f

Pni=1 piqi:c

r( p)=1g.

27 The proof of this and the foregoing relation is by straightforwardverification.28 The consumer’s cost function was defined by equation (17.6) above.29 Christensen, Jorgenson and Lau (1971) introduced this function intothe economics literature.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

322

restrictions:

aik=aki,Pni=1

ai=1,Pni=1

bi=0 andPnk=1

aik=0

for i, k=1, . . . , n (17:43)

These parameter restrictions ensure that C(u, p) definedby equation (17.42) is linearly homogeneous in p, aproperty that a cost function must have. It can be shownthat the translog cost function defined by equation(17.42) can provide a second-order Taylor series approx-imation to an arbitrary cost function.30

17.47 Assume that the consumer has preferencesthat correspond to the translog cost function and thatthe consumer engages in cost-minimizing behaviourduring periods 0 and 1. Let p0 and p1 be the period 0 and1 observed price vectors, and let q0 and q1 be the period0 and 1 observed quantity vectors. These assumptionsimply:

C(u0, p0)=Pni=1

p0i q0i and C(u

1, p1)=Pni=1

p1i q1i (17:44)

where C is the translog cost function defined above.Now apply Shephard’s Lemma, equation (17.12), andthe following equation results:

qti=@C(ut, pt)

@pifor i=1, . . . , n and t=0, 1

=C(ut, pt)

pti

@ lnC(ut, pt)

@ ln pi(17:45)

Now use equation (17.44) to replace C(ut, pt) in equation(17.45). After some cross multiplication, this becomesthe following:

ptiqtiPn

k=1

ptkqtk

� sti=@ lnC(ut, pt)

@ ln pi

for i=1, . . . , n and t=0, 1 (17:46)

or

sti=ai+Pnk=1

aik ln ptk+bi ln u

t for i=1, . . . , n and t=0, 1

(17:47)

where sti is the period t expenditure share on commodity i.17.48 Define the geometric average of the period 0

and 1 utility levels as u*; i.e., define

u* �ffiffiffiffiffiffiffiffiffiu0u1p

(17:48)

Now observe that the right-hand side of the equationthat defines the natural logarithm of the translog costfunction, equation (17.42), is a quadratic function of thevariables zi:lnpi if utility is held constant at the level u*.Hence the quadratic identity (17.41) can be applied, and

the following equation is obtained:

lnC(u*, p1)� lnC(u*, p0)

=1

2

Pni=1

@ lnC(u*, p0)

@ ln pi+@ lnC(u*, p1)

@ ln pi

� �ln p1i� ln p0i� �

=1

2

Pni=1

ai+Pnk=1

aik ln p0k+bi ln u

*

�+ai

+Pnk=1

aik ln p1k+bi ln u

*

�ln p1i� ln p0i� �

=1

2

Pni=1

ai+Pnk=1

aik ln p0k+bi ln

ffiffiffiffiffiffiffiffiffiu0u1p

+ai

�

+Pnk=1

aik ln p1k+bi ln

ffiffiffiffiffiffiffiffiffiu0u1p �

ln p1i� ln p0i� �

=1

2

Pni=1

ai+Pnk=1

aik ln p0k+bi ln u

0+ai

�

+Pnk=1

aik ln p1k+bi ln u

1

�ln p1i� ln p0i� �

=1

2

Pni=1

@ lnC(u0, p0)

@ ln pi+@ lnC(u1, p1)

@ ln pi

� �ln p1i� ln p0i� �

=1

2

Pni=1

s0i+s1i

� �ln p1i� ln p0i� �

using equation (17:47):

(17:49)

The last equation in (17.49) can be recognized as thelogarithm of the Tornqvist–Theil index number formulaPT, defined earlier by equation (15.81) in Chapter 15.Hence, exponentiating both sides of equation (17.49)yields the following equality between the true costof living between periods 0 and 1, evaluated at theintermediate utility level u* and the observableTornqvist–Theil index PT:

31

C(u*, p1)

Cðu*, p0)=PT ( p0, p1, q0, q1) (17:50)

Since the translog cost function which appears on theleft-hand side of equation (17.49) is a flexible functionalform, the Tornqvist–Theil price index PT is also asuperlative index.

17.49 It is somewhat mysterious how a ratio ofunobservable cost functions of the form appearing onthe left-hand side of the above equation can be exactlyestimated by an observable index number formula. Thekey to this mystery is the assumption of cost-minimizingbehaviour and the quadratic identity (17.41), alongwith the fact that derivatives of cost functions areequal to quantities, as specified by Shephard’s Lemma.In fact, all the exact index number results derived inparagraphs 17.27 to 17.43 can be derived using trans-formations of the quadratic identity along with Shep-hard’s Lemma (or Wold’s Identity).32 Fortunately,for most empirical applications, assuming that the30 It can also be shown that, if all the bi=0 and b00=0, then

C(u, p)=uC(1, p):uc(p); i.e., with these additional restrictions on theparameters of the general translog cost function, homothetic pref-erences are the result of these restrictions. Note that it is also assumedthat utility u is scaled so that u is always positive.

31 This result is attributable to Diewert (1976, p. 122).32 See Diewert (2002a).

THE ECONOMIC APPROACH TO INDEX NUMBER THEORY: THE SINGLE-HOUSEHOLD CASE

323

consumer has (transformed) quadratic preferences willbe an adequate assumption, so the results presented inparagraphs 17.27 to 17.49 are quite useful to indexnumber practitioners who are willing to adopt the eco-nomic approach to index number theory.33 Essentially,the economic approach to index number theory providesa strong justification for the use of the Fisher price indexPF defined by equation (15.12), the Tornqvist–Theil priceindex PT defined by equation (15.81), the implicitquadratic mean of order r price indices Pr* defined byequation (17.32) (when r=1, this index is the Walsh priceindex defined by equation (15.19) in Chapter 15) and thequadratic mean of order r price indices Pr defined byequation (17.35). In the next section, we ask if it matterswhich one of these formulae is chosen as ‘‘best’’.

The approximation properties ofsuperlative indices

17.50 The results of paragraphs 17.27 to 17.49 pro-vide price statisticians with a large number of indexnumber formulae which appear to be equally good fromthe viewpoint of the economic approach to index num-ber theory. Two questions arise as a consequence ofthese results:

� Does it matter which of these formulae is chosen?� If it does matter, which formula should be chosen?

17.51 With respect to the first question, Diewert(1978, p. 888) showed that all of the superlative indexnumber formulae listed in paragraphs 17.27 to 17.49approximate each other to the second order around anypoint where the two price vectors, p0 and p1, are equaland where the two quantity vectors, q0 and q1, are equal.In particular, this means that the following equalities arevalid for all r and s not equal to 0, provided that p0=p1

and q0=q1.34

PT ( p0, p1, q0, q1)=Pr( p0, p1, q0, q1)=Ps*( p0, p1, q0, q1)

(17:51)@PT ( p

0, p1, q0, q1)

@pti=@Pr( p0, p1, q0, q1)

@pti

=@Ps*( p0, p1, q0, q1)

@pti

for i=1, . . . , n and t=0, 1 (17:52)

@PT ( p0, p1, q0, q1)

@qti=@Pr( p0, p1, q0, q1)

@qti

=@Ps*( p0, p1, q0, q1)

@qti

for i=1, . . . , n and t=0, 1 (17:53)

@2PT ( p0, p1, q0, q1)

@pti@ptk

=@2Pr( p0, p1, q0, q1)

@pti@ptk

=@2Ps*( p0, p1, q0, q1)

@pti@ptk

for i, k=1, . . . , n and t=0, 1 (17:54)

@2PT ( p0, p1, q0, q1)

@pti@qtk

=@2Pr( p0, p1, q0, q1)

@pti@qtk

=@2Ps

*

( p0, p1, q0, q1)

@pti@qtk

for i, k=1, . . . , n and t=0, 1 (17:55)

@2PT ( p0, p1, q0, q1)

@qti@qtk

=@2Pr( p0, p1, q0, q1)

@qti@qtk

=@2Ps*( p0, p1, q0, q1)

@qti@qtk

for i, k=1, . . . , n for t=0, 1 (17:56)

where the Tornqvist–Theil price index PT is definedby equation (15.81), the implicit quadratic mean oforder r price indices Ps* is defined by equation (17.32)and the quadratic mean of order r price indices Pr isdefined by equation (17.35). Using the results in theprevious paragraph, Diewert (1978, p. 884) concludedthat ‘‘all superlative indices closely approximate eachother’’.

17.52 The above conclusion is, however, not trueeven though the equations (17.51) to (17.56) are true.The problem is that the quadratic mean of order r priceindices Pr and the implicit quadratic mean of order sprice indices Ps* are (continuous) functions of the para-meters r and s respectively. Hence, as r and s becomevery large in magnitude, the indices Pr and Ps* candiffer substantially from, say, P2=PF, the Fisher idealindex. In fact, using definition (17.35) and the limitingproperties of means of order r,35 Robert Hill (2002, p. 7)showed that Pr has the following limit as r approachesplus or minus infinity:

limr!+1

Pr( p0, p1, q0, q1)= limr!�1

Pr( p0, p1, q0, q1)

=

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffimini

p1ip0i

� �maxi

p1ip0i

� �s(17:57)

33 If, however, consumer preferences are non-homothetic and thechange in utility is substantial between the two situations being com-pared, then it may be desirable to compute separately the Laspeyres–Konus and Paasche–Konus true cost of living indices defined byequations (17.3) and (17.4), C(u0, p1)/C(u0, p0) and C(u1, p1)/C(u1, p0),respectively. In order to do this, it would be necessary to use econo-metrics and estimate empirically the consumer’s cost or expenditurefunction.34 To prove the equalities in equations (17.51) to (17.56), simply dif-ferentiate the various index number formulae and evaluate the deri-vatives at p0=p1 and q0=q1. Actually, equations (17.51) to (17.56) arestill true provided that p1=lp0 and q1=mq0 for any numbers l>0 andm>0; i.e., provided that the period 1 price vector is proportional to theperiod 0 price vector and that the period 1 quantity vector is pro-portional to the period 0 quantity vector. 35 See Hardy, Littlewood and Polya (1934).

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

324

Using Hill’s method of analysis, it can be shown that theimplicit quadratic mean of order r price index has thefollowing limit as r approaches plus or minus infinity:

limr!+1

Pr*( p0, p1, q0, q1)= limr!�1

Pr*( p0, p1, q0, q1, )

=

Pni=1

p1i q1i

Pni=1

p0i q0i

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffimini

p1ip0i

� �maxi

p1ip0i

� �s

(17:58)

Thus for r large in magnitude, Pr and Pr* can differsubstantially from PT, P

1, P1*=PW (the Walsh priceindex) and P2=P2*=PF (the Fisher ideal index).

36

17.53 Although Hill’s theoretical and empiricalresults demonstrate conclusively that not all superlativeindices will necessarily closely approximate each other,there is still the question of how well the more commonlyused superlative indices will approximate each other. Allthe commonly used superlative indices, Pr and Pr*, fallinto the interval 0� r� 2.37 Hill (2002, p. 16) summar-ized how far apart the Tornqvist and Fisher indices were,making all possible bilateral comparisons between anytwo data points for his time series data set as follows:

The superlative spread S(0, 2) is also of interest since,in practice, Tornqvist (r=0) and Fisher (r=2) are by farthe two most widely used superlative indexes. In all 153bilateral comparisons, S(0, 2) is less than the Paasche–Laspeyres spread and on average, the superlative spreadis only 0.1 per cent. It is because attention, until now, hasfocussed almost exclusively on superlative indexes in therange 0� r� 2 that a general misperception has persistedin the index number literature that all superlative indexesapproximate each other closely.

Thus, for Hill’s time series data set covering 64 com-ponents of United States gross domestic product from1977 to 1994 and making all possible bilateral compar-isons between any two years, the Fisher and Tornqvistprice indices differed by only 0.1 per cent on average.This close correspondence is consistent with the resultsof other empirical studies using annual time seriesdata.38 Additional evidence on this topic may be foundin Chapter 19.17.54 In the earlier chapters of this manual, it is

found that several index number formulae seem ‘‘best’’when viewed from various perspectives. Thus the Fisherideal index PF=P

2=P2* defined by equation (15.12)seemed to be best from one axiomatic viewpoint, theTornqvist–Theil price index PT defined by equation(15.81) seems to be best from another axiomatic per-

spective, as well as from the stochastic viewpoint, andthe Walsh index PW defined by equation (15.19) (whichis equal to the implicit quadratic mean of order r priceindices Pr* defined by equation (17.32) when r=1)seems to be best from the viewpoint of the ‘‘pure’’ priceindex. The results presented in this section indicate thatfor ‘‘normal’’ time series data, these three indices willgive virtually the same answer. To determine preciselywhich one of these three indices to use as a theoreticaltarget or actual index, the statistical agency will have todecide which approach to bilateral index number theoryis most consistent with its goals. For most practicalpurposes, however, it will not matter which of thesethree indices is chosen as a theoretical target index formaking price comparisons between two periods.

Superlative indices andtwo-stage aggregation

17.55 Most statistical agencies use the Laspeyresformula to aggregate prices in two stages. At the firststage of aggregation, the Laspeyres formula is used toaggregate components of the overall index (e.g., food,clothing, services); then at the second stage of aggrega-tion, these component sub-indices are further combinedinto the overall index. The following question thennaturally arises: does the index computed in two stagescoincide with the index computed in a single stage?Initially, this question is addressed in the context of theLaspeyres formula.39

17.56 Suppose that the price and quantity data forperiod t, pt and qt, can be written in terms of M sub-vectors as follows:

pt=( pt1 , pt2 , . . . , ptM ) and qt=(qt1 , qt2 , . . . , qtM )

for t=0, 1 (17:59)

where the dimensionality of the subvectors ptm and qtm isNm for m=1, 2, . . . ,M with the sum of the dimensionsNm equal to n. These subvectors correspond to the priceand quantity data for subcomponents of the consumerprice index for period t. Now construct sub-indices foreach of these components going from period 0 to 1. Forthe base period, set the price for each of these sub-components, say P 0m form=1, 2, . . .M, equal to 1 and setthe corresponding base period subcomponent quantities,sayQ0m form=1, 2, . . . ,M, equal to the base period valueof consumption for that subcomponent for m=1, 2, . . . ,M:

P0m � 1 and Q0m �PNmi=1

p0mi q0mi for m=1, 2, . . . ,M

(17:60)

Now use the Laspeyres formula in order to constructa period 1 price for each subcomponent, say P1m form=1, 2, . . . ,M, of the CPI. Since the dimensionality of

36Hill (2002) documents this for two data sets. His time series dataconsist of annual expenditure and quantity data for 64 components ofUnited States gross domestic product from 1977 to 1994. For this dataset, Hill (2002, p. 16) found that ‘‘superlative indexes can differ bymore than a factor of two (i.e., by more than 100 per cent), eventhough Fisher and Tornqvist never differ by more than 0.6 per cent’’.37Diewert (1980, p. 451) showed that the Tornqvist index PT is alimiting case of Pr, as r tends to 0.38 See, for example, Diewert (1978, p. 894) or Fisher (1922), which isreproduced in Diewert (1976, p. 135).

39Much of the material in this section is adapted from Diewert (1978)and Alterman, Diewert and Feenstra (1999). See also Balk (1996b) fora discussion of alternative definitions for the two-stage aggregationconcept and references to the literature on this topic.

THE ECONOMIC APPROACH TO INDEX NUMBER THEORY: THE SINGLE-HOUSEHOLD CASE

325

the subcomponent vectors, ptm and qtm , differs from thedimensionality of the complete period t vectors of pricesand quantities, pt and qt, it is necessary to use differentsymbols for these subcomponent Laspeyres indices, sayPmL for m=1, 2, . . . ,M. Thus the period 1 subcomponentprices are defined as follows:

P1m � PmL ( p0m, p1m, q0m, q1m) �

PNmi=1

p1mi q0mi

PNmi=1

p0mi q0mi

for m=1, 2, . . . ,M (17:61)

Once the period 1 prices for theM sub-indices have beendefined by equation (17.61), then corresponding sub-component period 1 quantities Q1m for m=1, 2, . . . ,Mcan be defined by deflating the period 1 subcomponentvalues

PNmi=1p

1mi q

1mi by the prices P1m

Q1m �

PNmi=1

p1mi q1mi

P1mfor m=1, 2, . . . ,M (17:62)

Now define subcomponent price and quantity vectorsfor each period t=0,1 using equations (17.60) to (17.62).Thus define the period 0 and 1 subcomponent pricevectors P0 and P1 as follows:

P0=(P01,P02, . . . ,P0M) � 1M and P1=(P11,P

12, . . . ,P1M)

(17:63)

where 1M denotes a vector of ones of dimension M andthe components of P1 are defined by equation (17.61).The period 0 and 1 subcomponent quantity vectors Q0

and Q1 are defined as follows:

Q0=(Q01,Q02, . . . ,Q0M) and Q

1=(Q11,Q12, . . . ,Q1M)

(17:64)

where the components of Q0 are defined in equation(17.60) and the components of Q1 are defined by equa-tion (17.62). The price and quantity vectors in equations(17.63) and (17.64) represent the results of the first-stageaggregation. Now use these vectors as inputs intothe second-stage aggregation problem; i.e., apply theLaspeyres price index formula, using the information inequations (17.63) and (17.64) as inputs into the indexnumber formula. Since the price and quantity vectorsthat are inputs into this second-stage aggregation pro-blem have dimension M instead of the single-stage for-mula which utilized vectors of dimension n, a differentsymbol is required for the new Laspeyres index: this ischosen to be P*L. Thus the Laspeyres price index com-puted in two stages can be denoted as P*L(P

0,P1,Q0,Q1).Now ask whether this two-stage Laspeyres index equalsthe corresponding single-stage index PL that was studiedin the previous sections of this chapter; i.e., ask whether

P*L(P0,P1,Q0,Q1)=PL( p

0, p1, q0, q1): (17:65)

If the Laspeyres formula is used at each stage of eachaggregation, the answer to the above question is yes:straightforward calculations show that the Laspeyres

index calculated in two stages equals the Laspeyresindex calculated in one stage.

17.57 Now suppose that the Fisher or Tornqvistformula is used at each stage of the aggregation. That is,in equation (17.61), suppose that the Laspeyres formulaPmL ( p

0m, p1m, q0m, q1m) is replaced by the Fisher formulaPmF ( p

0m, p1m, q0m, q1m) or by the Tornqvist formula PmT( p0m, p1m, q0m, q1m); and in equation (17.65), supposethat P*L(P

0,P1,Q0,Q1) is replaced by P*F (or by P*T ) and

PL (p0, p1, q0, q1) is replaced by PF (or by PT). Then is it

the case that counterparts are obtained to the two-stage aggregation result for the Laspeyres formula,equation (17.65)? The answer is no; it can be shownthat, in general,

P*F (P0,P1,Q0,Q1) 6¼ PF ( p0, p1, q0, q1) and

P*T (P0,P1,Q0,Q1) 6¼ PT ( p0, p1, q0, q1) (17:66)

Similarly, it can be shown that the quadratic mean oforder r index number formula Pr defined by equation(17.35) and the implicit quadratic mean of order r indexnumber formula Pr* defined by equation (17.32) are alsonot consistent in aggregation.

17.58 Nevertheless, even though the Fisher andTornqvist formulae are not exactly consistent inaggregation, it can be shown that these formulae areapproximately consistent in aggregation. More specifi-cally, it can be shown that the two-stage Fisher formulaP*F and the single-stage Fisher formula PF in the in-equality (17.66), both regarded as functions of the 4nvariables in the vectors p0, p1, q0, q1, approximate eachother to the second order around a point where the twoprice vectors are equal (so that p0=p1) and where thetwo quantity vectors are equal (so that q0=q1), and asimilar result holds for the two-stage and single-stageTornqvist indices in equation (17.66).40 As was seen inthe previous section, the single-stage Fisher andTornqvist indices have a similar approximation prop-erty, so all four indices in the inequality (17.66) approx-imate each other to the second order around an equal(or proportional) price and quantity point. Thus fornormal time series data, single-stage and two-stageFisher and Tornqvist indices will usually be numericallyvery close. This result is illustrated in Chapter 19 for anartificial data set.41

17.59 Similar approximate consistency in aggrega-tion results (to the results for the Fisher and Tornqvistformulae explained in the previous paragraph) can bederived for the quadratic mean of order r indices, Pr,and for the implicit quadratic mean of order r indices,Pr*; see Diewert (1978, p. 889). Nevertheless, the resultsof Hill (2002) again imply that the second-order

40 See Diewert (1978, p. 889). In other words, a string of equalitiessimilar to equations (17.51) to (17.56) holds between the two-stageindices and their single-stage counterparts. In fact, these equalities arestill true provided that p1=l p0 and q1=mq0 for any numbers l>0 andm>0.41 For an empirical comparison of the four indices, see Diewert (1978,pp. 894–895). For the Canadian consumer data considered there, thechained two-stage Fisher in 1971 was 2.3228 and the correspondingchained two-stage Tornqvist was 2.3230, the same values as for thecorresponding single-stage indices.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

326

approximation property of the single-stage quadraticmean of order r index Pr to its two-stage counterpartwill break down as r approaches either plus or minusinfinity. To see this, consider a simple example wherethere are only four commodities in total. Let the firstprice ratio p11=p

01 be equal to the positive number a, let

the second two price ratios p1i =p0i equal b and let the last

price ratio p14=p04 equal c, where we assume a<c and

a� b� c. Using Hill’s result (17.57), the limiting value ofthe single-stage index is:

limr!+1

Pr( p0, p1, q0, q1)= limr!�1

Pr( p0, p1, q0, q1)

=

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffimini

p1ip0i

� �maxi

p1ip0i

� �s=

ffiffiffiffiffiacp

(17:67)

Now aggregate commodities 1 and 2 into a sub-aggre-gate and commodities 3 and 4 into another sub-aggre-gate. Using Hill’s result (17.57) again, it is found that thelimiting price index for the first sub-aggregate is [ab]1/2

and the limiting price index for the second sub-aggre-gate is [bc]1/2. Now apply the second stage of aggrega-tion and use Hill’s result once again to conclude thatthe limiting value of the two-stage aggregation using Pr

as the index number formula is [ab2c]1/4. Thus the lim-iting value as r tends to plus or minus infinity of thesingle-stage aggregate over the two-stage aggregate is

[ac]1=2=[ab2c]1=4=[ac= b2]1=4. Now b can take on anyvalue between a and c, and so the ratio of the single-stage limiting Pr to its two-stage counterpart can take onany value between [c/a]1/4 and [a/c]1/4. Since c/a is lessthan 1 and a/c is greater than 1, it can be seen that theratio of the single-stage to the two-stage index can bearbitrarily far from 1 as r becomes large in magnitudewith an appropriate choice of the numbers a, b and c.17.60 The results in the previous paragraph show

that some caution is required in assuming that allsuperlative indices will be approximately consistent inaggregation. However, for the three most commonlyused superlative indices (the Fisher ideal PF, the Tornqvist–Theil PT and the Walsh PW), the available empiricalevidence indicates that these indices satisfy the con-sistency in aggregation property to a sufficiently highdegree of approximation that users will not be undulytroubled by any inconsistencies.42

The Lloyd–Moulton indexnumber formula17.61 The index number formula that will be dis-

cussed in this section on the single-household economicapproach to index number theory is a potentially veryuseful one for statistical agencies that are faced withthe problem of producing a CPI in a timely manner. TheLloyd–Moulton formula that will be discussed in thissection makes use of the same information that isrequired in order to implement a Laspeyres index exceptfor one additional piece of information.

17.62 In this section, the same assumptions aboutthe consumer are made that were made in paragraphs17.18 to 17.26 above. In particular, it is assumed that theconsumer’s utility function f (q) is linearly homo-geneous43 and the corresponding unit cost function isc(p). It is supposed that the unit cost function has thefollowing functional form:

c( p) � a0Pni=1

aip1�si

� �1=(1�s)if s 6¼ 1 or

ln c( p) � a0+Pni=1

ai ln pi if s=1 (17:68)

where the ai and s are non-negative parameters withPni=1ai=1. The unit cost function defined by equation

(17.68) corresponds to a constant elasticity of substitu-tion (CES) aggregator function, which was introducedinto the economics literature by Arrow, Chenery,Minhasand Solow (1961).44 The parameter s is the elasticity ofsubstitution; when s=0, the unit cost function definedby equation (17.68) becomes linear in prices and hencecorresponds to a fixed coefficients aggregator functionwhich exhibits zero substitutability between all com-modities. When s=1, the corresponding aggregator orutility function is a Cobb–Douglas function. When sapproaches plus infinity, the corresponding aggregatorfunction f approaches a linear aggregator functionwhich exhibits infinite substitutability between each pairof inputs. The CES unit cost function defined by equa-tion (17.68) is not a fully flexible functional form (unlessthe number of commodities n being aggregated is 2), butit is considerably more flexible than the zero substitut-ability aggregator function (this is the special case ofequation (17.68) where s is set equal to zero) that isexact for the Laspeyres and Paasche price indices.

17.63 Under the assumption of cost minimizingbehaviour in period 0, Shephard’s Lemma (17.12), tellsus that the observed first period consumption of com-modity i, q0i , will be equal to u

0 @c(p0)/@pi, where @c(p0)/

@pi is the first-order partial derivative of the unit costfunction with respect to the ith commodity price eval-uated at the period 0 prices and u0=f (q0) is the aggre-gate (unobservable) level of period 0 utility. Usingthe CES functional form defined by equation (17.68)and assuming that s=1, the following equations areobtained:

q0i=u0a0

Pnk=1

ak(p0k)r

� �(1=r)�1ai(p0i )

r�1

for r� 1�s 6¼ 0 and i=1,2, . . . ,n

=u0c(p0)ai(p0i )

r�1

Pnk=1

ak(p0k)r

(17:69)

42 See Chapter 19 for some additional evidence on this topic.

43 Thus homothetic preferences are assumed in this section.44 In the mathematics literature, this aggregator function or utilityfunction is known as a mean of order r; see Hardy, Littlewood andPolya (1934, pp. 12–13).

THE ECONOMIC APPROACH TO INDEX NUMBER THEORY: THE SINGLE-HOUSEHOLD CASE

327

These equations can be rewritten as:

p0i q0i

u0c( p0)=

ai( p0i )r

Pnk=1

ak( p0k)rfor i=1, 2, . . . , n (17:70)

where r:1�s. Now consider the following Lloyd (1975)and Moulton (1996a) index number formula:

PLM( p0, p1, q0, q1) �

Pni=1

s0ip1ip0i

� �1�s( )1=(1�s)for s 6¼ 1

(17:71)

where s0i is the period 0 expenditure share of commodityi, as usual:

s0i �p0i q

0iPn

k=1

p0kq0k

for i=1, 2, . . . , n

=p0i q

0i

u0c( p0)using the assumption of cost minimizing

behaviour

=ai( p0i )

r

Pnk=1

ak( p0k)rusing equation (17:70) (17:72)

If equation (17.72) is substituted into equation (17.71), itis found that:

PLM( p0, p1, q0, q1)=

Pni=1

s0ip1ip0i

� �r� �1=r

=Pni=1

ai( p0i )r

Pnk=1

ak( p0k)r

p1ip0i

� �r8>><>>:

9>>=>>;1=r

=

Pni=1

ai( p1i )r

Pnk=1

ak( p0k)r

8>><>>:

9>>=>>;1=r

=

a0Pni=1

ai( p1i )r

� �1=r

a0Pnk=1

ak( p0k)r

� �1=r

=c( p1)

c( p0)using r � 1�s

and definition (17:68): (17:73)

17.64 Equation (17.73) shows that the Lloyd–Moulton index number formula PLM is exact for CESpreferences. Lloyd (1975) and Moulton (1996a) inde-pendently derived this result, but it was Moulton whoappreciated the significance of the formula (17.71) forstatistical agency purposes. Note that in order to eval-

uate formula (17.71) numerically, it is necessary to haveinformation on:

� base period expenditure shares s0i ;� the price relatives p1i =p0i between the base period andthe current period; and

� an estimate of the elasticity of substitution betweenthe commodities in the aggregate, s.

The first two pieces of information are the standardinformation sets that statistical agencies use to evaluatethe Laspeyres price index PL (note that PLM reduces toPL if s=0). Hence, if the statistical agency is able toestimate the elasticity of substitution s based on pastexperience,45 then the Lloyd–Moulton price index canbe evaluated using essentially the same informationset that is used in order to evaluate the traditionalLaspeyres index. Moreover, the resulting CPI will befree of substitution bias to a reasonable degree ofapproximation.46 Of course, the practical problem withimplementing this methodology is that estimates of theelasticity of substitution parameter s are bound to besomewhat uncertain, and hence the resulting Lloyd–Moulton index may be subject to charges that it is notobjective or reproducible. The statistical agency willhave to balance the benefits of reducing substitution biaswith these possible costs.

Annual preferences andmonthly prices

17.65 Recall the definition of the Lowe index,PLo(p

0, p1, q), defined by equation (15.15) in Chapter 15.In paragraphs 15.33 to 15.64 of Chapter 15, it is notedthat this formula is frequently used by statistical agen-cies as a target index for a CPI. It is also noted that,while the price vectors p0 (the base period price vector)and p1 (the current period price vector) are monthlyor quarterly price vectors, the quantity vector q:(q1,q2, . . . , qn) which appears in this basket-type formula isusually taken to be an annual quantity vector that refersto a base year, b say, that is prior to the base period forthe prices, month 0. Thus, typically, a statistical agencywill produce a CPI at a monthly frequency that has theform PLo(p

0, pt, qb), where p0 is the price vector per-taining to the base period month for prices, month 0, pt

45 For the first application of this methodology (in the context of theCPI), see Shapiro and Wilcox (1997a, pp. 121–123). They calculatedsuperlative Tornqvist indices for the United States for the years 1986–95 and then calculated the Lloyd–Moulton CES index for the sameperiod, using various values of s. They then chose the value of s(which was 0.7), which caused the CES index to most closelyapproximate the Tornqvist index. Essentially the same methodologywas used by Alterman, Diewert and Feenstra (1999) in their study ofUnited States import and export price indices. For alternative methodsfor estimating s, see Balk (2000b).46What is a ‘‘reasonable’’ degree of approximation depends on thecontext. Assuming that consumers have CES preferences is not areasonable assumption in the context of estimating elasticities ofdemand: at least a second-order approximation to the consumer’spreferences is required in this context. In the context of approximatingchanges in a consumer’s expenditures on the n commodities underconsideration, however, it is usually adequate to assume a CESapproximation.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

328

is the price vector pertaining to the current periodmonth for prices, month t say, and qb is a referencebasket quantity vector that refers to the base year b,which is equal to or prior to month 0.47 The question tobe addressed in the present section is: can this index berelated to one based on the economic approach to indexnumber theory?

The Lowe index as an approximation toa true cost of living index17.66 Assume that the consumer has preferences

defined over consumption vectors q:[q1, . . . , qn] thatcan be represented by the continuous increasing utilityfunction f (q). Thus if f (q1)>f (q0), then the consumerprefers the consumption vector q1 to q0. Let qb be theannual consumption vector for the consumer in the baseyear b. Define the base year utility level ub as the utilitylevel that corresponds to f (q) evaluated at qb:

ub � f (qb) (17:74)

17.67 For any vector of positive commodity pricesp:[ p1, . . . , pn] and for any feasible utility level u, theconsumer’s cost function, C(u, p), can be defined in theusual way as the minimum expenditure required toachieve the utility level u when facing the prices p:

C(u, p) � minqPni=1

piqi : f (q1, . . . , qn)=u

� �: (17:75)

Let pb � [ pb1, . . . , pbn] be the vector of annual prices thatthe consumer faced in the base year b. Assume that theobserved base year consumption vector qb � [qb1, . . . , qbn]solves the following base year cost minimization pro-blem:

C(ub, pb) � minqPni=1

pbi qi : f (q1, . . . , qn)=ub

� �=Pni=1

pbi qbi

(17:76)

The cost function will be used below in order to definethe consumer’s cost of living price index.17.68 Let p0 and pt be the monthly price vectors that

the consumer faces in months 0 and t. Then the Konustrue cost of living index, PK(p

0, pt, qb), between months0 and t, using the base year utility level ub=f (qb) as thereference standard of living, is defined as the followingratio of minimum monthly costs of achieving the utilitylevel ub:

PK ( p0, pt, qb) � C( f (q

b), pt)

C( f (qb), p0)(17:77)

17.69 Using the definition of the monthly costminimization problem that corresponds to the costC( f (qb), pt ), it can be seen that the following inequality

holds:

C( f (qb),pt) �minqPni=1

ptiqi : f (q1, . . . ,qn)=f (qb1, . . . ,qbn)

� �

�Pni=1

ptiqbi (17:78)

since the base year quantity vector qb is feasible for thecost minimization problem. Similarly, using the defini-tion of the monthly cost minimization problem thatcorresponds to the month 0 cost C( f (qb), p0), it can beseen that the following inequality holds:

C( f (qb),p0)�minqPni=1

p0i qi : f (q1, . . . ,qn)=f (qb1, . . . ,q

bn)

� �

�Pni=1

p0i qbi (17:79)

since the base year quantity vector qb is feasible for thecost minimization problem.

17.70 It will prove useful to rewrite the twoinequalities (17.78) and (17.79) as equalities. This can bedone if non-negative substitution bias terms, et and e0,are subtracted from the right-hand sides of these twoinequalities. Thus the inequalities (17.78) and (17.79)can be rewritten as follows:

C(ub, pt)=Pni=1

ptiqbi�et (17:80)

C(ub, p0)=Pni=1

p0i qbi�e0 (17:81)

17.71 Using equations (17.80) and (17.81), and thedefinition (15.15) in Chapter 15 of the Lowe index, thefollowing approximate equality for the Lowe indexresults:

PLo( p0, pt, qb) �

Pni=1

ptiqbi

Pni=1

p0i qbi

=fC(ub, pt)+etgfC(ub, p0)+e0g

� C(ub, pt)

C(ub, p0)=PK ( p

0, pt, qb) (17:82)

Thus if the non-negative substitution bias terms e0 and et

are small, then the Lowe index between months 0 and t,PLo(p

0, pt, qb), will be an adequate approximation to thetrue cost of living index between months 0 and t,PK(p

0, pt, qb).17.72 A bit of algebraic manipulation shows that the

Lowe index will be exactly equal to its cost of livingcounterpart if the substitution bias terms satisfy thefollowing relationship:48

et

e0=C(ub, pt)

C(ub, p0)=PK ( p

0, pt, qb) (17:83)

47As noted in Chapter 15, month 0 is called the price reference periodand year b is called the weight reference period.

48 This assumes that e0 is greater than zero. If e0 is equal to zero, thento have equality of PK and PLo, it must be the case that e

t is also equalto zero.

THE ECONOMIC APPROACH TO INDEX NUMBER THEORY: THE SINGLE-HOUSEHOLD CASE

329

Equations (17.82) and (17.83) can be interpreted as fol-lows: if the rate of growth in the amount of substitutionbias between months 0 and t is equal to the rate ofgrowth in the minimum cost of achieving the base yearutility level ub between months 0 and t, then the obser-vable Lowe index, PLo(p

0, pt, qb), will be exactly equal toits true cost of living index counterpart, PK(p

0, pt, qb).49

17.73 It is difficult to know whether condition(17.83) will hold or whether the substitution bias termse0 and et will be small. Thus, first-order and second-order Taylor series approximations to these substitutionbias terms are developed in paragraphs 17.74 to 17.83.

A first-order approximation to thebias of the Lowe index

17.74 The true cost of living index between months 0and t, using the base year utility level ub as the referenceutility level, is the ratio of two unobservable costs,C(ub, pt)/C(ub, p0). However, both of these hypotheticalcosts can be approximated by first-order Taylor seriesapproximations that can be evaluated using observableinformation on prices and base year quantities. Thefirst-order Taylor series approximation to C(ub, pt)around the annual base year price vector pb is given bythe following approximate equation:50

C(ub, pt) � C(ub, pb)+Pni=1

@C(ub, pb)

@pi

� �[ pti�pbi ]

=C(ub, pb)+Pni=1

qbi [ pti�pbi ]

using assumption (17:76) and Shephard’s Lemma

(17.12)

=Pni=1

pbi qbi+Pni=1

qbi [ pti�pbi ] using (17:76)

=Pni=1

ptiqbi : (17:84)

Similarly, the first-order Taylor series approximation toC(ub, p0) around the annual base year price vector pb isgiven by the following approximate equation:

C(ub, p0) � C(ub, pb)+Pni=1

@C(ub, pb)

@pi

� �[ p0i�pbi ]

=C(ub, pb)+Pni=1

qbi ½p0i�pbi �

=Pni=1

pbi qbi+Pni=1

qbi ½p0i�pbi �

=Pni=1

p0i qbi (17:85)

17.75 Comparing approximate equation (17.84)with equation (17.80), and comparing approximateequation (17.85) with equation (17.81), it can be seenthat, to the accuracy of the first-order approximationsused in (17.84) and (17.85), the substitution bias terms et

and e0 will be zero. Using these results to reinterpret theapproximate equation (17.82), it can be seen that if themonth 0 and month t price vectors, p0 and pt, are not toodifferent from the base year vector of prices pb, then theLowe index PLo(p

0, pt, qb) will approximate the true costof living index PK(p

0, pt, qb) to the accuracy of a first-order approximation. This result is quite useful, since itindicates that if the monthly price vectors p0 and pt arejust randomly fluctuating around the base year prices pb

(with modest variances), then the Lowe index will serveas an adequate approximation to a theoretical cost ofliving index. However, if there are systematic long-termtrends in prices and month t is fairly distant from month0 (or the end of year b is quite distant from month 0),then the first-order approximations given by approx-imate equations (17.84) and (17.85) may no longer beadequate and the Lowe index may have a considerablebias relative to its cost of living counterpart. Thehypothesis of long-run trends in prices will be exploredin paragraphs 17.76 to 17.83.

A second-order approximation to thesubstitution bias of the Lowe index

17.76 A second-order Taylor series approximationto C(ub, pt) around the base year price vector pb is givenby the following approximate equation:

C(ub, pt) � C(ub, pb)+Pni=1

@C(ub, pb)

@pi

� �[ pti�pbi ]

+1

2

� �Pni=1

Pnj=1

@2C(ub, pb)

@pi@pj

� �[ pti�pbi ][ ptj�pbj ]

=Pni=1

pbi qbi+

1

2

� �Pni=1

Pnj=1

@2C(ub, pb)

@pi@pj

� �

� [ pti�pbi ][ ptj�pbj ] (17:86)

where the last equality follows using approximate equa-tion (17.84).51 Similarly, a second-order Taylor seriesapproximation to C(ub, p0) around the base year pricevector pb is given by the following approximate equation:

C(ub, p0) � C(ub, pb)+Pni=1

@C(ub, pb)

@pi

� �[ p0i�pbi ]

+1

2

� �Pni=1

Pnj=1

@2C(ub, pb)

@pi@pj

� �[ p0i�pbi ][ p0j�pbj ]

=Pni=1

p0i qbi+

1

2

� �Pni=1

Pnj=1

@2C(ub, pb)

@pi@pj

� �

� [ p0i�pbi ][ p0j�pbj ] (17:87)49 It can be seen that, when month t is set equal to month 0, et=e0 andC(ub, pt)=C(ub, p0), and thus equation (17.83) is satisfied and PLo=PK.This is not surprising since both indices are equal to unity when t=0.50This type of Taylor series approximation was used in Schultze andMackie (2002, p. 91) in the cost of living index context, but it essen-tially dates back to Hicks (1941–42, p. 134) in the consumer surpluscontext. See also Diewert (1992b, p. 568) and Hausman (2002, p. 8).

51 This type of second-order approximation is attributable to Hicks(1941–42, pp. 133–134; 1946, p. 331). See also Diewert (1992b, p. 568),Hausman (2002, p. 18) and Schultze and Mackie (2002, p. 91). Foralternative approaches to modelling substitution bias, see Diewert(1998a; 2002c, pp. 598–603) and Hausman (2002).

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

330

where the last equality follows using the approximateequation (17.85).17.77 Comparing approximate equation (17.86)

with equation (17.80), and approximate equation(17.87) with equation (17.81), it can be seen that, tothe accuracy of a second-order approximation, themonth 0 and month t substitution bias terms, e0 and et,will be equal to the following expressions involvingthe second-order partial derivatives of the consumer’scost function @2C(ub, pb)/@pi@pj evaluated at thebase year standard of living ub and at the base yearprices pb:

e0 � � 1

2

� �Pni=1

Pnj=1

@2C(ub, pb)

@pi@pj

� �[ p0i�pbi ][ p0j�pbj ] (17:88)

et � � 1

2

� �Pni=1

Pnj=1

@2C(ub, pb)

@pi@pj

� �[ pti�pbi ][ ptj�pbj ] (17:89)

Since the consumer’s cost function C(u, p) is a concavefunction in the components of the price vector p,52 it isknown53 that the n by n (symmetric) matrix of second-order partial derivatives [@2C(ub, pb)/@pi@pj] is negativesemi-definite.54 Hence, for arbitrary price vectors pb, p0

and pt, the right-hand sides of approximations (17.88)and (17.89) will be non-negative. Thus, to the accuracyof a second-order approximation, the substitution biasterms e0 and et will be non-negative.17.78 Now assume that there are long-run sys-

tematic trends in prices. Assume that the last month ofthe base year for quantities occurs M months prior tomonth 0, the base month for prices, and assume thatprices trend linearly with time, starting with the lastmonth of the base year for quantities. Thus, assume theexistence of constants aj for j=1, . . . , n such that theprice of commodity j in month t is given by:

pti=pbi+aj(M+t) for j=1, . . . , n and t=0, 1, . . . ,T

(17:90)

Substituting equation (17.90) into approximations(17.88) and (17.89) leads to the following second-orderapproximations to the two substitution bias terms, e0

and et:55

e0 � gM2 (17:91)

et � g(M+t)2 (17:92)

where g is defined as follows:

g � � 1

2

� �Pni=1

Pnj=1

@2C(ub, pb)

@pi@pj

� �aiaj � 0 (17:93)

17.79 It should be noted that the parameter g will bezero under two sets of conditions:56

� All the second-order partial derivatives of the con-sumer’s cost function @2C(ub, pb)/@pi @pj are equal tozero.

� Each commodity price change parameter aj is pro-portional to the corresponding commodity j base yearprice pbj .

57

The first condition is empirically unlikely since it impliesthat the consumer will not substitute away from com-modities of which the relative price has increased. Thesecond condition is also empirically unlikely, since itimplies that the structure of relative prices remainsunchanged over time. Thus, in what follows, it will beassumed that g is a positive number.

17.80 In order to simplify the notation in what fol-lows, define the denominator and numerator of themonth t Lowe index, PLo(p

0, pt, qb), as a and b respec-tively; i.e., define:

a �Pni=1

p0i qbi (17:94)

b �Pni=1

ptiqbi (17:95)

Using equation (17.90) to eliminate the month 0 pricesp0i from equation (17.94) and the month t prices p

ti from

equation (17.95) leads to the following expressions for aand b:

a=Pni=1

pbi qbi+Pni=1

aiqbi M (17:96)

b=Pni=1

pbi qbi+Pni=1

aiqbi (M+t) (17:97)

It is assumed that a and b58 are positive and that

Pni=1

aiqbi � 0 (17:98)

Assumption (17.98) rules out a general deflation in prices.17.81 Define the bias in the month t Lowe index,

Bt, as the difference between the true cost of livingindex PK(p

0, pt, qb) defined by equation (17.77) and the

52 See Diewert (1993b, pp. 109–110).53 See Diewert (1993b, p. 149).54A symmetric n by n matrix A with ijth element equal to aij is negativesemi-definite if, and only if for every vector z:[z1, . . . , zn], it is the casethat

Pni=1

Pnj=1aijzizj � 0.

55Note that the period 0 approximate bias defined by the right-handside of approximation (17.91) is fixed, while the period t approximatebias defined by the right-hand side of (17.92) increases quadraticallywith time t. Hence, the period t approximate bias term will eventuallyoverwhelm the period 0 approximate bias in this linear time trendscase, if t is allowed to become large enough.

56A more general condition that ensures the positivity of g is that thevector [a1, . . . , an] is not an eigenvector of the matrix of second-orderpartial derivatives @ 2C(u, p)/@pi@pj that corresponds to a zero eigen-value.57 It is known that C(u, p) is linearly homogeneous in the componentsof the price vector p; see Diewert (1993b, p. 109) for example. Hence,using Euler’s Theorem on homogeneous functions, it can be shownthat pb is an eigenvector of the matrix of second-order partial deriva-tives @2C(u, p)/@pi@pj that corresponds to a zero eigenvalue and thusPni=1

Pnj=1[@

2C(u, p)=@pi@pj ]pbi pbj=0; see Diewert (1993b, p. 149) for a

detailed proof of this result.58 It is also assumed that a�gM2 is positive.

THE ECONOMIC APPROACH TO INDEX NUMBER THEORY: THE SINGLE-HOUSEHOLD CASE

331

corresponding Lowe index PLo(p0, pt, qb):

Bt � PK ( p0, pt, qb)�PLo( p0, pt, qb)

=C(ub, pt)

C(ub, p0)

� �� b

a

� �using equations (17:94) and (17:95)

=[b�et][a�e0]

� �� b

a

� �using equations (17:80) and (17:81)

� [b�g(M+t)2]a�gM2

� �� b

a

� �using equations (17:91) and (17:92)

=gf(b�a)M2�2aMt�at2g

fa[a�gM2]g simplifying terms

=g

Pni=1

aiqbi t� �

M2�2Pni=1

pbi qbi+

Pni=1

aiqbi M� �

Mt�at2� �

fa[a�gM2]gusing equations (17:96) and (17:97)

=�g

Pni=1

aiqbi t� �

M2�2Pni=1

pbi qbi

� �Mt+at2

� �fa[a�gM2]g < 0

using equation (17:98): (17:99)

Thus, for t� 1, the Lowe index will have an upward bias(to the accuracy of a second-order Taylor seriesapproximation) compared to the corresponding truecost of living index PK(p

0, pt, qb), since the approximatebias defined by the last expression in equation (17.99) isthe sum of one non-positive and two negative terms.Moreover, this approximate bias will grow quadraticallyin time t.59

17.82 In order to give the reader some idea of themagnitude of the approximate bias Bt defined by the lastline of equation (17.99), a simple special case will beconsidered at this point. Suppose there are only twocommodities and that, at the base year, all prices andquantities are equal to 1. Thus, pbi=q

bi=1 for i=1, 2 andPn

i=1 pbi qbi=2. Assume thatM=24 so that the base year

data on quantities take two years to process before theLowe index can be implemented. Assume that themonthly rate of growth in price for commodity 1 isa1=0.002 so that after one year, the price of commodity1 rises 0.024 or 2.4 per cent. Assume that commodity 2falls in price each month with a2=�0.002 so that theprice of commodity 2 falls 2.4 per cent in the first yearafter the base year for quantities. Thus the relative priceof the two commodities is steadily diverging by about5 per cent per year. Finally, assume that @2C(ub, pb)/@p1@p1=@2C(ub, pb)/@p2@p2=�1 and @2C(ub, pb)/@p1@p2=@2C(ub, pb)/@p2@p1=1. These assumptions implythat the own elasticity of demand for each commodity is�1 at the base year consumer equilibrium. Making all of

these assumptions means that:

2=Pni=1

pbi qbi=a=b

Pni=1

aiqbi=0 M=24; g=0:000008

(17:100)

Substituting the parameter values defined in equation(17.100) into equation (17.99) leads to the followingformula for the approximate amount that the Loweindex will exceed the corresponding true cost of livingindex at month t:

�Bt=0:000008 (96t+2t2)

2(2�0:004608) (17:101)

Evaluating equation (17.101) at t=12, t=24, t=36,t=48 and t=60 leads to the following estimates for �Bt:0.0029 (the approximate bias in the Lowe index at theend of the first year of operation for the index); 0.0069(the bias after two years); 0.0121 (the bias after threeyears); 0.0185 (the bias after four years); 0.0260 (the biasafter five years). Thus, at the end of the first year of theoperation, the Lowe index will only be above the cor-responding true cost of living index by approximately athird of a percentage point but, by the end of the fifthyear of operation, it will exceed the corresponding costof living index by about 2.6 percentage points, which isno longer a negligible amount.60

17.83 The numerical results in the previous para-graph are only indicative of the approximate magnitudeof the difference between a cost of living index and thecorresponding Lowe index. The important point to noteis that, to the accuracy of a second-order approximation,the Lowe index will generally exceed its cost of livingcounterpart. The results also indicate, however, that thisdifference can be reduced to a negligible amount if:

� the lag in obtaining the base year quantity weights isminimized; and

� the base year is changed as frequently as possible.It should also be noted that the numerical results dependon the assumption that long-run trends in prices exist,which may not be true,61 and on elasticity assumptionsthat may not be justified.62 Statistical agencies shouldprepare their own carefully constructed estimates of thedifferences between a Lowe index and a cost of livingindex in the light of their own particular circumstances.

The problem of seasonalcommodities

17.84 The assumption that the consumer has annualpreferences over commodities purchased in the base year

59 IfM is large relative to t, then it can be seen that the first two termsin the last equation of (17.99) can dominate the last term, which is thequadratic in t term.

60Note that the relatively large magnitude ofM compared to t leads toa bias that grows approximately linearly with t rather than quad-ratically.61 For mathematical convenience, the trends in prices were assumed tobe linear, rather than the more natural assumption of geometric trendsin prices.62Another key assumption that was used to derive the numericalresults is the magnitude of the divergent trends in prices. If the pricedivergence vector is doubled to a1=0.004 and a2=�0.004, then theparameter g quadruples and the approximate bias will also quadruple.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

332

for the quantity weights, and that these annual pref-erences can be used in the context of making monthlypurchases of the same commodities, was a key one inrelating the economic approach to index number theoryto the Lowe index. This assumption that annual pref-erences can be used in a monthly context is, however,somewhat questionable because of the seasonal natureof some commodity purchases. The problem is that it isvery likely that consumers’ preference functions sys-tematically change as the season of the year changes.National customs and weather changes cause house-holds to purchase certain goods and services duringsome months and not at all for other months.For example, Christmas trees are purchased only inDecember and ski jackets are not usually purchasedduring summer months. Thus, the assumption thatannual preferences are applicable during each month ofthe year is only acceptable as a very rough approxima-tion to economic reality.17.85 The economic approach to index number

theory can be adapted to deal with seasonal preferences.The simplest economic approach is to assume that theconsumer has annual preferences over commoditiesclassified not only by their characteristics but also by themonth of purchase.63 Thus, instead of assuming that theconsumer’s annual utility function is f (q) where q is ann-dimensional vector, assume that the consumer’sannual utility function is F [ f 1(q1), f 2(q2), . . . , f 12(q12)]where q1 is an n-dimensional vector of commodity pur-chases made in January, q2 is an n-dimensional vector ofcommodity purchases made in February, . . . , and q12 isan n-dimensional vector of commodity purchases madein December.64 The sub-utility functions f 1, f 2, . . . , f 12

represent the consumer’s preferences when makingpurchases in January, February, . . . ., and December,respectively. These monthly sub-utilities can then beaggregated using the macro-utility function F in order todefine overall annual utility. It can be seen that theseassumptions on preferences can be used to justify twotypes of cost of living index:

� an annual cost of living index that compares the pricesin all months of a current year with the correspondingmonthly prices in a base year;65 and

� 12 monthly cost of living indices where the index formonth m compares the prices of month m in thecurrent year with the prices of month m in the baseyear for m=1, 2, . . . , 12.66

17.86 The annual Mudgett–Stone indices comparecosts in a current calendar year with the correspondingcosts in a base year. However, any month could bechosen as the year-ending month of the current year, and

the prices and quantities of this new non-calendar yearcould be compared to the prices and quantities of thebase year, where the January prices of the non-calendaryear are matched to the January prices of the base year,the February prices of the non-calendar year arematched to the February prices of the base year, and soon. If further assumptions are made on the macro-utilityfunction F, then this framework can be used in order tojustify a third type of cost of living index: a movingyear annual index.67 This index compares the cost overthe past 12 months of achieving the annual utilityachieved in the base year with the base year cost, wherethe January costs in the current moving year are matchedto January costs in the base year, the February costs inthe current moving year are matched to February costsin the base year, and so on. These moving year indicescan be calculated for each month of the current year andthe resulting series can be interpreted as (uncentred)seasonally adjusted (annual) price indices.68

17.87 It should be noted that none of the three typesof indices described in the previous two paragraphs issuitable for describing the movements of prices goingfrom one month to the following month; i.e., they arenot suitable for describing short-run movements ininflation. This is obvious for the first two types of index.To see the problem with the moving year indices, con-sider a special case where the bundle of commoditiespurchased in each month is entirely specific to eachmonth. Then it is obvious that, even though all theabove three types of index are well defined, none of themcan describe anything useful about month-to-monthchanges in prices, since it is impossible to compare likewith like, going from one month to the next, under thehypotheses of this special case. It is impossible to com-pare the incomparable.

17.88 Fortunately, it is not the case that householdpurchases in each month are entirely specific to themonth of purchase. Thus month-to-month price com-parisons can be made if the commodity space isrestricted to commodities that are purchased in eachmonth of the year. This observation leads to a fourthtype of cost of living index, a month-to-month index,defined over commodities that are available in everymonth of the year.69 This model can be used to justifythe economic approach described in paragraphs 17.66 to17.83. Commodities that are purchased only in certainmonths of the year, however, must be dropped fromthe scope of the index. Unfortunately, it is likely thatconsumers have varying monthly preferences over thecommodities that are always available and, if this isthe case, the month-to-month cost of living index (andthe corresponding Lowe index) defined over always-available commodities will generally be subject to sea-sonal fluctuations. This will limit the usefulness of the63This assumption and the resulting annual indices were first proposed

by Mudgett (1955, p. 97) and Stone (1956, pp. 74–75).64 If some commodities are not available in certain months m, thenthose commodities can be dropped from the corresponding monthlyquantity vectors qm.65 For further details on how to implement this framework, seeMudgett (1955, p. 97), Stone (1956, pp. 74–75) and Diewert (1998b, pp.459 – 460).66 For further details on how to implement this framework, see Diewert(1999a, pp. 50–51).

67 See Diewert (1999a, pp. 56–61) for the details of this economicapproach.68 See Diewert (1999a, pp. 67–68) for an empirical example of thisapproach applied to quantity indices. An empirical example of thismoving year approach to price indices is presented in Chapter 22.69 See Diewert (1999a, pp. 51–56) for the assumptions on preferencesthat are required in order to justify this economic approach.

THE ECONOMIC APPROACH TO INDEX NUMBER THEORY: THE SINGLE-HOUSEHOLD CASE

333

index as a short-run indicator of general inflation since itwill be difficult to distinguish a seasonal movement inthe index from a systematic general movement in pri-ces.70 Note also that if the scope of the index is restrictedto always-available commodities, then the resultingmonth-to-month index will not be comprehensive,whereas the moving year indices will be comprehensivein the sense of using all the available price information.

17.89 The above considerations lead to the conclu-sion that it may be useful for statistical agencies toproduce at least two consumer price indices:

� a moving year index which is comprehensive andseasonally adjusted, but which is not necessarily usefulfor indicating month-to-month changes in generalinflation; and

� a month-to-month index which is restricted to non-seasonal commodities (and hence is not comprehen-sive), but which is useful for indicating short-runmovements in general inflation.

The problem of a zero priceincreasing to a positive price

17.90 In a recent paper, Haschka (2003) raised theproblem of what to do when a price which was pre-viously zero is increased to a positive level. He gave twoexamples for Austria, where parking and hospital feeswere raised from zero to a positive level. In this situa-tion, it turns out that basket-type indices have anadvantage over indices that are weighted geometricaverages of price relatives, since basket-type indices arewell defined even if some prices are zero.

17.91 The problem can be considered in the contextof evaluating the Laspeyres and Paasche indices. Supposeas usual that the prices pti and quantities q

ti of the first n

commodities are positive for periods 0 and 1, but that theprice of commodity n+1 in period 0 is zero but is positivein period 1. In both periods, the consumption of com-modity n+1 is positive. Thus the assumptions on theprices and quantities of commodity n+1 in the two per-iods under consideration can be summarized as follows:

p0n+1=0 p1n+1 > 0 q0n+1 > 0 q1n+1 > 0 (17:102)

Typically, the increase in price of commodity n+1 fromits initial non-zero level will cause consumption to fall sothat q1n+1 < q

0n+1, but this inequality is not required for

the analysis below.17.92 Let the Laspeyres index between periods 0 and

1, restricted to the first n commodities, be denoted as PnLand let the Laspeyres index, defined over all n+1 com-modities, be defined as Pn+1L . Also let v0i � p0i q0i denote

the value of expenditures on commodity i in period 0.Then by the definition of the Laspeyres index definedover all n+1 commodities:

Pn+1L �

Pn+1i=1

p1i q0i

Pn+1i=1

p0i q0i

=PnL+p1n+1q

0n+1Pn

i=1

v0i

(17:103)

where p0n+1=0 was used in order to derive the secondequation above. Thus the complete Laspeyres indexPn+1L defined over all n+1 commodities is equal to theincomplete Laspeyres index PnL (which can be written intraditional price relative and base period expenditureshare form), plus the mixed or hybrid expenditurep1n+1q

0n+1 divided by the base period expenditure on

the first n commodities,Pni=1v

0i . Thus the complete

Laspeyres index can be calculated using the usualinformation available to the price statistician plus twoadditional pieces of information: the new non-zero pricefor commodity n+1 in period 1, p1n+1, and an estimateof consumption of commodity n+1 in period 0 (when itwas free), q0n+1. Since it is often governments thatchange the previously zero price to a positive price, thedecision to do this is usually announced in advance,which will give the price statistician an opportunity toform an estimate for the base period demand, q0n+1.

17.93 Let the Paasche index between periods 0 and1, restricted to the first n commodities, be denoted as PnPand let the Paasche index, defined over all n+1 com-modities, be defined as Pn+1P . Also let v1i � p1i q1i denotethe value of expenditures on commodity i in period 1.Then, by the definition of the Paasche index defined overall n+1 commodities:

Pn+1P �

Pn+1i=1

p1i q1i

Pn+1i=1

p0i q1i

=PnP+v1n+1Pn

i=1

p0i q1i

=PnP+v1n+1Pn

i=1

v1i =( p1i =p

0i )

(17:104)

where p0n+1=0 was used in order to derive the secondequation above. Thus the complete Paasche index Pn+1Pdefined over all n+1 commodities is equal to theincomplete Paasche index PnP (which can be written intraditional price relative and current period expenditureshare form), plus the current period expenditure oncommodity n+1, v1n+1, divided by a sum of currentperiod expenditures on the first n commodities, v1i , dividedby the ith price relative for the first n commodities, p1i =p

0i .

Thus the complete Paasche index can be calculated using

70One problem with using annual weights in the context of seasonalmovements in prices and quantities is that a change in price when acommodity is out of season can be greatly magnified by the use ofannual weights. Baldwin (1990, p. 251) noted this problem with anannual weights price index: ‘‘But a price index is adversely affected ifany seasonal good has the same basket share for all months of the year;the good will have an inappropriately small basket share in its inseason months, an inappropriately large share in its off seasonmonths.’’ Seasonality problems are considered again from a morepragmatic point of view in Chapter 22.

CONSUMER PRICE INDEX MANUAL: THEORY AND PRACTICE

334