Development and pilot testing of an instrument to assess nutrition messages on four micronutrients in Canadian

magazines

Lindsay Zalot, BAScDietetic InternThe Ottawa Hospital – Civic Campus

2

Outline

Introduction Rationale and Objective Methods Results Discussion Implications and Conclusions Acknowledgements References

3

Introduction

Health Canada provides credible information to the public while striving to prevent and reduce risks to individual health and promote healthier lifestyles. (Health Canada, 2007)

Recent changes in Canadian nutrition policies may be reflected in the media.

Women have specific nutritional needs and vulnerabilities and, as such, are at unique risk for various nutrition-related diseases and conditions that affect the duration and quality of their lives.

Intakes of calcium, vitamin D, iron and folate are marginal (Garriguet, 2004)

4

Introduction continued

Tracking Nutrition Trends VI data showed that 76% of respondents reported that they received their nutrition information from print media. (Canadian Council of Food and Nutrition, 2006)

Magazines are a primary source of nutrition information for 72% of Canadian women. (Hassan et al., 2007)

Women perceive conflicting dietary information as one of the barriers hindering their ability to meet dietary recommendations. (French et al., 2005)

There have been some Canadian studies conducted on messages women receive from the print media.

5

Nutrition and Media Messages

▪ Korinis et al (1998) examined calcium and weight loss information in American teen-focused verses women’s magazines over two 4-year periods and discovered that messages related to calcium was primarily offered to women past their best opportunity to affect bone mass.

▪ Tassan, Marchessault and Campbell (2007) compared the frequency of calcium and body weight messages in Chatelaine and Flare magazines and concluded that opportunities existed to provide improved calcium and osteoporosis coverage for women at a prime age to increase their bone density.

6

Rationale and Objective

A study that develops and tests a tool for examining the quantity and accuracy of nutrition messages geared towards women of reproductive age needs to be conducted.

The objective of this research project was to develop

and pilot test a coding instrument that utilized a content analysis methodology, to assess the quantity, types and accuracy of messages related to calcium, vitamin D, iron and folate, for use in a larger Health Canada (HC) study.

7

Content Analysis

Content Analysis is a research technique that systematically examines the frequencies and meanings of linguistic elements by using a set of qualitative and quantitative methods to collect and analyze data from print communication. (Pratt & Pratt, 1995; Kondracki et al., 2002)

8

Process of Content Analysis

Developing a research objective

Making conceptualization decisions

Deciding on operationalization measures and coding schemes

Determining the scope of the sample

Training coders and revising coding materials

Process of coding

Determining agreement between coders

Communicating the results(Neuendorf, 2002)

9

SampleThree Magazines: Canadian Living, Chatelaine and Homemakers (n=16 issues)

10

Sample continued

For the pilot study, our sub-sample of magazine issues (n=16) was an approximate 10% of all issues published within our sampling time frame (2003-2007).

Issues were randomly chosen using a random number generator and then coded and analyzed.

Nutrition, food and health messages regarding four micronutrients: calcium, vitamin D, iron and folate.

11

Instrument Development

Two instruments were created for our study: the coding sheet and the codebook, based on other validated instruments. (Barr, 1989; Guillen & Barr, 1994)

The coding sheet was the electronic data collection tool (Excel spreadsheet) used for each magazine issue.

The codebook was the set of rules that guided our decisions for inclusion and exclusion of criteria.

12

Coding the Messages

Messages in the magazine issues were coded.

A Message was defined as “the piece of written text or the image that was coded that pertained to one of the four micronutrients of interest, calcium, vitamin D, iron, or folate, in five message formats: advertisement, editorial, image, article, or letter to the editor. The length of the Message ranged from part of a sentence to multiple sentences.”

Messages included nutrient content claims, general statements on the physiological function of the nutrients, and nutrient analyses found in recipes.

13

Coding continued

An advertisement, editorial, image, article, or letter to the editor was considered to contain a ‘message’ if it contained at least one of the 22 pre-selected criteria chosen for inclusion on the coding sheets, which were operationally defined in the codebook.

Criteria that were included had to reflect the objective (i.e., quantity, types and accuracy of messages related to the 4 micronutrients of interest) for the pilot test.

14

Coding continued

Important criteria included:

Total number of messages Number of messages related to each

micronutrient What the nutrient was presented in relation to

(e.g., food, nutritional supplements, nutrient analysis, the physiological function of a nutrient, or non-dietary uses of the nutrient)

Evaluation for congruence to Canadian nutrition policy, and if each message was congruent

15

Coding Procedures

Two coders: Dr. Marcia Cooper and myself.

Training sessions allowed coders to develop and revise coding rules and definitions included in codebook and code sheet.

The goal was to have high intercoder reliability, as defined by agreement, in coding messages the same way.

16

Coding Procedures continued

Coding of the 16 magazine was conducted independently by the two coders during a period of 7 weeks in March and April, 2008.

Data were directly entered into electronic code sheets and results of selected elements were tallied for further analysis.

Due to the timeline of the research project, 12 out of the 16 coded magazine issues (4 Chatelaine, 3 Canadian Living and 5 Homemakers issues) were analyzed for agreement between coders.

17

Coding Procedures continued

To determine agreement, both coders’ coding sheets were compared to see if the same messages were identified, and how each message was coded for each element.

To resolve coding discrepancies, the primary source of information (i.e., the actual magazine issue) was consulted to view the message itself.

Only 8 of the 22 criteria collected on the code sheets were further analyzed for this project, with the remaining criteria to be assessed in the future.

18

The number of total messages regarding calcium, vitamin

D, iron and folate in the 12 magazine issues analyzed

Magazine Coded

# of total Message

s

Number of Messages relating to micronutrients:

Ca Vit D Fe Folate

Chatelaine(n=4)

100 86 21 40 24

Canadian Living (n=3)

179 147 10 132 128

Homemakers (n=5)

99 67 9 67 68

Total (n=12 issues)

378 300 40 239 220

19

Percentage agreement between coders on number of messages

Magazine Coded

Actual # of

Messages

# of Messages Coded by

MC

# of Messages Coded by

LZ

% Agreement

Chatelaine (n=4)

100 98 88 83

Canadian Living (n=3)

179 170 169 86

Homemakers (n=5)

99 88 98 85

Total (n=12) 378 356 355 n/a

20

Discussion

This instrument development and pilot testing study highlighted new information and exposed unique challenges to overcome before conducting the larger Health Canada study.

The coding sheet will be modified to incorporate the coders’ feedback on how well it captured the information the coders were trying to obtain. With 22 elements, the tool was too long

21

Implications for the Larger Health Canada Study

Next important step for the tools: to undergo validation. “When properly applied, content analysis methods are both

reliable and valid.” (Kondracki et al., 2002)

Intercoder and intracoder reliability will need to be further determined for the pilot study results before the actual study occurs.

To ensure consistent training, a strategy will be developed that will take a prospective coder through coding different messages for the various micronutrients in varying message formats.

22

Conclusion Content analysis is a detail-oriented methodology that

was carefully applied to gain new knowledge in creating a instruments to measure the quantity, accuracy and types of messages in Canadian women’s magazines.

The goal of content analysis is to be objective so that real messages are captured and subsequently coded accurately.

Print media is an important source of nutrition information for women and it is important to know what messages women receive.

23

Acknowledgements

Dr. Marcia Cooper, Health Canada – a supporter, mentor, helper, decision-maker

Dr. Laurie Wadsworth, St. Francis Xavier University– reviewed the coding instrument and was the peer reviewer for my paper

The donors who contributed their magazines

24

Selected References Health Canada. About mission, values, activities; 2007 [cited 2008 June 17].

Available from: http://www.hc-sc.gc.ca/ahc-asc/activit/about-apropos/index_e.html Garriguet D. Statistics Canada. Overview of Canadians’ eating habits 2004. Ottawa:

Health Statistics Division; 2006. Catalogue no. 82-620-MIE — No. 2. Canadian Council of Food and Nutrition. Tracking nutrition trends VI; 2006 [cited

2007 18 Oct]. Available from: http://www.ccfn.ca/pdfs/TNT_VI_Report__2006.pdf Hassan T, Marchessault G, Campbell M. Messages about calcium and weight in

Canadian women’s magazines. Can J Diet Pract Res 2007;68(2):103-106. French MR, Moore K, Vernace-Inserra F, Hawker GA. Factors That Influence:

Adherence to Calcium Recommendations. Can J Diet Pract Res 2005;66(1):25-29. Pratt CA, Pratt CB. Comparative content analysis of food and nutrition

advertisements in Ebony, Essence, and Ladies’ Home Journal. J Nutr Educ 1995:27:11-17.

Kondracki NL, Wellman NS, Amundson DR. Content analysis: review of methods and their application in nutrition education. J Nutr Educ Behav 2002;34:224-230.

Neuendorf KA. The Content Analysis Guidebook. Thousand Oaks, CA: Sage; 2002 Barr SI. Nutrition in food advertising: content analysis of a Canadian women’s

magazine, 1928-1986. J Nutr Ed 1989;21:64-72. Guillen EO, Barr SI. Nutrition, dieting and fitness magazines in a magazine for

adolescent women, 1970-1990. J Adolesc Health 1994;15:464-472.

25

THANK YOU!

Thanks for your attention!

I will be pleased to answer any questions you may have.