Nursing Research II-Prelims

7/28/2019 Nursing Research II-Prelims

http://slidepdf.com/reader/full/nursing-research-ii-prelims 1/122

Prepared by:

Lucky P. Roaquin, RN, MAN



At the end of the course and given actual situation, agroup of student will be able to:

(COGNITIVE DOMAIN) To describe the meaning of research and its

importance to nursing practice

To discuss the different techniques in data analysis of

the different research approaches To differentiate the different ways of interpreting data

gathered during the implementation process




(AFFECTIVE DOMAIN) To initiate appreciation on the topics provided

To enhance self-esteem through rigorous training onnursing research




(PSYCHOMOTOR DOMAIN) To create a competitive research project

To generate an effective evidenced-based researchimportant in the nursing profession



Preliminary Period

A. The Empirical Phase

- Measurement and the Assessment of QuantitativeData

- Reliability Test of Instruments

- Validity Test of Instruments

- Specificity & Sensitivity



The assignment of numbers to objects according tospecified rules to characterize quantities of someattribute

The purpose of assigning numbers is to differentiatebetween people with varying degrees of attribute



Advantages:

1. Removes subjectivity and guesswork

2. Obtains reasonably precise information

3. Language of communication



Errors of Measurement The difference between true and obtained scores

= ± where:

is the observed score is true component

is error component



Errors of Measurement

Common Influences:

a) Situational contaminants – conditions under whichthey are produced.

b) Transitory personal factors – by such personal statesor people’s motivation to cooperate, act naturally or

do their bestc) Response-set biases – enduring characteristics of

people



Errors of Measurement

Common Influences:

d) Administration variations – alterations in themethods of collecting data

e) Instrument clarity – poorly constructed directions

f) Item sampling – results from sampling of items

g) Instrument format – technical characteristics of aninstrument



Reliability of a quantitative instrument is a majorcriterion for assessing its quality

Reliability is the consistency with which it measuresthe target attribute and concerns accuracy that reflectstrue scores

Reliable measures maximize the true scores andminimize the error



Aspects of Reliability Testing

1. Stability

2. Internal consistency

3. Equivalence



Stability Extent to which the similar scores are obtained on

separate occasions.

It is how reproducible the findings would be if the samemeasurement were repeatedly made on the same object

Assessment of stability involve procedures that evaluatetest-retest reliability (researchers administer the same

measure to a sample twice and then compare thescores).

Reliability coefficient is an index of the magnitude of

the test’s reliability



Stability Correlation coefficient seeks to detect and to explain

relationship among phenomena

Perfect relationship if two variables are obviously related (r=1.00)

Positive relationship if relationship exists but notperfect (between r=0.00 to 1.00)

Negative relationship if two variables are inversely related (between r=0.00 to -1.00)



Stability The higher the coefficient, the more stable the measure

Reliability coefficients above .80 usually are consideredgood.

Test-retest measure can be influenced by experiencesbetween tests. And Stability estimates can be influenced

by memory of initial responses.



Internal Consistency The degree to which the subparts of a composite scale

are all measuring the same attribute or dimension

It requires one administration

Best means of assessing an especially important sourceof measurement error in sampling of items

The most widely used method for evaluating internalconsistency is coefficient alpha (Cronbach’s alpha)



Internal Consistency Cronbach’s alpha estimates the internal consistency of

a composite measure composed of several subparts

Cronbach’s alpha can be interpreted like other reliability coefficients: 0.00 and +1.00, higher values reflect higherinternal consistency



Equivalence Primarily concerns the degree to which two or more

independent observers or coders agree about scoring

If there is a high level of agreement, then theassumption is that measurement errors have beenminimized

Number of agreement

Number of agreement + disagreements



Interpretation of Reliability Coefficients Important indicators of an instrument’s quality

Coefficients in the vicinity of .70 may be adequate butcoefficients of .80 or greater are highly desirable

Reliability is the proportion of true variability to thetotal obtained variability

=

It is clearly why instruments with reliability lower than.70 is risky to use.



Factors Affecting ReliabilityHomogenous of sample

Items with no discriminating power

Number of items



Tips: More items tapping the same concept should be added

Greater precision in defining categories or greater clarity inexplaining the underlying construct for rating scales

Heterogeneity of the sample

Know the characteristics of the group with which it was

developed

In selecting an instruments, researchers need to determine which aspect of reliability is relevant







Types of Validity Tests

1. Face

2. Content

3. Criterion-Related

4. Construct

5. Convergent & Discriminant



Face Validity• Refers to whether the instrument looks like it is

measuring the target construct

• Not strong but helpful for a measure to have face validity if other types of validity have also been demonstrated

• It is easier to persuade people to participate in the study



Content Validity Concerns the degree to which an instrument has an

appropriate sample of items for the construct beingmeasured and adequately covers the construct domain

Relevant for both affective and cognitive measures

Based on judgment but it is common to use a panel of

experts to evaluate the content validity of newinstruments



Content Validity Content validity index (CVI) is an index of the degree

to which an instrument is content valid, based onaggregated ratings of a panel of experts

Experts are asked to evaluated individual items oroverall instrument

Two issues: (1) individual items are relevant andappropriate; and (2) items taken together adequately measure all dimensions



Content Validity Item CVI (I-CVI) is computed as the number of experts

giving a rating then divided by the number of experts.0.80 is considered an acceptable value

2 approaches in determining the Scale CVI (S-CVI): (a)universal agreement and (b) averaging approach



Criterion-Related Validity• The degree to which scores on an instrument are

correlated with some external criterion

• Key issue: whether the instrument is a useful predictorof other behaviors, experiences or conditions

• Requirement: availability of a reliable and valid

criterion with which measures on the instrument can becompared



Criterion-Related Validity• Criterion-related validity coefficient is computed by

correlating scores on the instrument and the criterion

•

Types: (1) predictive validity and (2) concurrent validity

• Predictive validity refers to the adequacy of aninstrument in differentiating between people’sperformance on a future basis

• Concurrent validity reflects an instrument’s ability todistinguish individuals who differ on a present criterion



Construct Validity • Key criterion for assessing the quality of a study

• Concerns inferences from study particulars to higher-

order constructs

• Hypothesis-testing that is linked to a theoreticalperspective about the construct



Construct Validity 4 Approaches in Construct Validity

1. Known Groups

2. Hypothesized Relationships

3. Convergent and Discriminant Validity

4. Factor Analysis



1. Known-Groups Technique Yields evidence of contrast validity

The instrument is administered to groups hypothesizedto differ on the critical attribute because of a known

characteristic

E.g.

Measurement of fear between primiparas andmultiparas



2. Hypothesized Relationships Involves testing hypothesized relationships, often on the

basis of theory or prior research

Involves hypotheses about the relationship between the

measure of the construct and a variable representinggroup membership

E.g.

According to theory, construct X is positively related to construct Y

Instrument A is a measure of construct X; instrument B is a measure of construct Y

Scores on A & B are correlated positively, as predicted

Therefore, it is inferred that A & B are valid measures of X & Y



3. Convergent and Discriminant Validity Convergence – is evidence that different methods of

measuring a construct yield similar results.

Discriminability – the ability to differentiate the

construct from other similar constructs. Multitrait-multimethod matrix method – significant

validation tool

The point of including two concepts in a single validation study is to gather evidence that the twoconcepts are distinct, rather than two different labels forthe same underlying attribute



4. Factor Analysis A method for identifying clusters of related variables

that is, dimensions underlying a broad construct

It is used to identify and group together different itemsmeasuring an underlying attribute

Constitutes another means of testing hypotheses about

the interrelationships among variables, and for lookingat the convergent and discriminant validity of a large setof items



Interpretation of Validity An instrument’s validity is not proved, established or

verified but rather is supported to a greater or lesserextent by evidence

Researchers do not validate an instrument but rather anapplication of it.

The more evidence that can be gathered that aninstrument is measuring what it is supposed to bemeasuring, the more confidence will have its validity.





Specificity and sensitivity are criteria that areimportant in evaluating instruments used as screeningor diagnostic tools (e.g. a scale to measure risk of osteoporosis)

Screening or diagnostic instruments can be self-report,observational, or biophysiologic measures



SensitivityThe ability of a measure to identify a “case” correctly to

screen in or diagnosis a condition correctly

Results yield to “true positives”

SpecificityMeasure’s ability to identify noncases correctly to screen

out those without the condition

Results yield to “true negatives”



Predictive values – the probability of an outcome afterthe results are known

Positive predictive value (PPV) – proportion of people with a positive result who have the target outcome ordisease.

Negative predictive value (NPV) – proportion of people

who have a negative test result who do not have thetarget outcome or disease.



Likelihood Ratio It summarizes between specificity and sensitivity in a

single number

Addresses the question, “How much more likely are weto find that an indicator is positive among those with theoutcome of concern compared to those for whom theindicator is negative?”



URINARY COTININE LEVEL

SELF-REPORTED

SMOKING

Positive(Cotinine >200ng/ml

Negative(Cotinine ≤200ng/ml

Total

Yes, smoked A (true positive)20

B (false positive)10

A + B30

No, did notsmoked C (false positive)20 D (true negative)50 C + D70

Total A + C40

B + D60

A+B+C+D100



Sensitivity = A/(A+C) = .50Specificity = D/(B+D) = .83

PPV = A/(A+B) = .67

NPV = D/(C+D) = .71LR+ = sensitivity/(1-specificity) = 2.99

LR- = (1-sensitivity)/specificity = .60



Receiver Operating Characteristic Curve (ROC curve) A method used in developing and refining a screening

instrument to determine the best cut-off point for

“caseness” The sensitivity of an instrument is plotted against the

false-positive rate over a range of different scores

Screening instrument is excellent if the points crowd to

the upper left corner. Area under the curve – the overall accuracy of an

instrument that can be calculated as the proportion of the area under the ROC curve



Receiver Operating Characteristic Curve (ROC curve)





Efficiencyo Reduces participant burden

o Diminishes number of items in a scale

o Spearman-Brown formula – estimate how reliable thescale would be with fewer items



Efficiency

E.g. Consider a 40-item scale to measure social supportthat has internal consistency reliability 0f 0.94to be

shorten in 30-item scale

=

1 + [ − 1 ]

Where k is the factor by which the instrument is beingdecreased; r is the reliability for the full scale; and isthe reliability for the shorter scale



Efficiency

Activity:

Solve the

for an 80-item scale to measureanxiety among patients undergoing dialysis that has aninternal consistency of .96 to be shorten in 65-item scale

= 0.95

Is it possible to shorten the full scale to a 65-item? Why? Why not?



Efficiency

Activity:

Solve the

for an 95-item scale to measure thehealth index of elderly that has an internal consistency of .79 to be shorten in 70-item scale

= 0.73

Is it possible to shorten the full scale to a 65-item? Why? Why not?



Comprehensibility – participants and researchersshould be able to comprehend the behaviors requiredto secure accurate and valid measures

Precision – an instrument should discriminatebetween people with different amounts of an attributeas precisely as possible

Range – the instrument should be capable of achievinga meaningful measure from the smallest expected value of the variable to the largest



Linearity – a researcher normally strives to constructmeasures that are equally accurate and sensitive overthe entire range of values

Reactivity – the instrument should, avoid affecting theattribute being measured





Let’s do some

brain damaging!



Enumerate the advantages of measurement and

explain why it is important in doing research.



Give at least 5 common influences on

measurement error and provide a concreteexample from each influence.



What is reliability? Cite an example wherein

reliability is of significant in doing research.



Enumerate the different levels of measurement.

Give actual examples from each levels.



Discuss stability.



How is validity done in evaluating instrument?

Enumerate the 4 types of validity testing.



Discuss equivalence. Explain the formula used

and cite an example.



What is coefficient alpha? Discuss the use of

coefficient alpha in determining the reliability ofan instrument.



Differentiate face validity from content validity.

Discuss the two approaches to calculate contentvalidity.



Explain specificity and sensitivity in evaluating

quantitative instruments. Illustrate an example indetermining specificity and sensitivity of an

instrument.



Discuss construct validity and the different ways

to gather evidence about construct validity



Enumerate and explain the other criteria for

assessing quantitative measures.



Enumerate the factors that affect reliability and

ways to minimize these factors.







The Steps in developing high-quality self-report scales:1. Focus on structured self-reported measures rather

than the observational ones.

2. Describe methods of developing multi-item scales

3. Exclude infrequently used scale types, such as

semantic differentials

4. Focus on classical measurement theory rather thanitem response theory



1. Conceptualizing the Construct

Latent variable – an unmeasured variable thatrepresents an underlying, abstract construct which is the

cause of the scores in measurement theory

The strength of the latent variable is presumed to triggera certain numeric value on the scale



1. Conceptualizing the Construct

Be an expert on the construct – first step in scaledevelopment

Identify and understand the different dimensions of theconstruct

Take account of the related constructs from the targetconstruct

Explicit conceptualization and understanding of thepopulation for whom the scale is intended



2. Deciding on the Type of Scale

Two categories of scales: Likert Scales and latent traitscales

Likert scales – declarative statements with a bipolarresponse scale that is often on an agree/disagreecontinuum

Latent trait scales – a scale developed within an item

response theory framework.



2. Deciding on the Type of Scale

E.g. Developing a scale to measure risk-taking behaviorin adolescents.

Rasch model – the model that considers item difficulty in assessing items for the scale

In a CMT scale, theitems might include

statements about risk-taking of similar

intensity, with which

the respondents wouldagree or disagree.

In IRT scale, the itemsthemselves would be

chosen to reflectdifferent levels of risk-

taking such as smoking,

using drugs, driving while texting.



3. Developing an Item Pool: Getting Started

Items need to be carefully crafted to reflect the latent variable they are designed to measure

Look for possible sources

Existing instruments – adaptation of an existinginstrument which require adding or deleting items

or may involve rewording of items.

The literature – this point, your would already be an“expert” on the construct, this is an obvious source

of ideas for items

Concept Analysis – clarification of conceptualdefinition that will facilitate conceptual clarity

among nurse researchers

In-depth qualitative approach – qualitative research will help understand the dimensions of a

phenomenon, and can give actual words for items

Clinical Observations – patients in clinical settingsmay be an excellent source of items



3. Developing an Item Pool: Getting Started

Devellis (2003) urged scale developers to get started writing scale items without a lot of editing and critical

review in the early stage.

The best way to begin is to develop a simple statement with the key construct mentioned in it



4. Making Decisions about Item Features

Make decisions about such as issues as the number of items to develop, the number and form of the response

options, whether to include positively and negatively worded items, and how to deal with time.




Number of Items

•

Domain sampling model – the model used indeveloping a scale in the classical measurement theory framework, which involves the random sampling of ahomogenous set of items from a hypothetical universeof items relating to the construct

• The idea is to generate a fairly exhaustive set of itempossibilities, given the construct’s theoretical demands




Number of Items

•

Redundancy is good in order to measure the constructof interest with a set of items that capture the centraltheme in slightly different ways so that irrelevantidiosyncrasies of individual items will cancel eachother out

• Generate a very large pool of items




Response Options

•

The goal is to array people on a continuum where variability is essential through offering numerousresponse options

• Numbers are used with verbal descriptors to facilitate

coding




Positive & Negative Stems

•

The objective was to include items that wouldminimize the possibility of an acquiescence responseset – the tendency to agree with statements

• Acquiescence can be minimized by putting the most

positive response options at the end of the list ratherthan at the beginning




Item Intensity

•

The intensity of the statements should be similar andfairly strongly worded

• There is no point in including items with which almosteveryone would either agree or disagree




Item Frames

•

Items should be worded in such a manner that every respondent is answering the same question

Clarity – scale developers should strive

for clear, unambiguous items. Wordsshould be carefully chosen with theeducational and reading of the target

population.

Jargon – be cautious about using terms

that might be well-known in healthcarecircles but not familiar to the average

person.

Length – avoid long sentences or phrases.Simple sentences are easiest tocomprehend

Double negatives – it is often preferableto word things affirmatively, but doublenegatives should be avoided.

Double barreled – avoid putting two ormore ideas in a single item.





1. Internal Review

Individual items should capture the construct,and are grammatical and well worded

Assess the scale’s readability (the ease with which the materials can be read by people with varying reading skills, often evaluated throughreadability formulas such as Flesch-Kincaidgrade level score)



1. Internal Review

A general principle is to avoid long sentences and words with four or more syllabus



2. Input from the Target Population

Initial pretest the set of items from the targetpopulation with a 10-20 sample

Cognitive questioning is an excellent techniquefor discovering how others process the wordsand ideas presented to them in structuredquestions.



2. Input from the Target Population

It is a good idea to peruse the pretest answers tosee if response patterns suggest the need for

item revisions

Focus groups can also be used to determine theclarity of the items, linguistically and culturally appropriateness, inoffensive and relevant toconstruct.



3. External Review by Experts

External review by a panel of experts should beundertaken to assess the scale’s content validity

Selecting and Recruiting the Experts• People with strong credentials such as

experts on clinical or personal experience,

published papers, or ongoing researchprogram on the topic.

• Should be knowledgeable about the key construct and the target population

Content Validation of Items• The panel is asked to rate the dimensions of

each items that includes clarity of wording,

relevance of the items and appropriatenessfor the target population

• Relevance is most often used

Content Validation of Items• Concerns an overall recommendation• To consider whether the items taken as a

whole adequately cover the constructdomain

• After agreement, ask for their opinions



3. External Review by Experts

External review by a panel of experts should beundertaken to assess the scale’s content validity

Content validation of the Scale• 2nd round of content validation that is

composed of 3-5 experts from new set of experts or from the subset of the 1st panel• S-CVI is then computed





Undertake a quantitative assessment of the items, which requires that they be administered to a fairly large development sample.

Important steps include the development of samplingplan and data collection strategy.



1. Developing a Sampling Plan

The sample should be a representative of the population

Recruit a sample from multiple sites to enhancerepresentativeness and to assess geographic variation ininterpreting the items

Having a sufficiently large sample is essential to ensurestability in the covariation among the items



2. Developing a Data Collection Plan

Choose an approach that best approximates how thescale typically would be administered after it is finalized

Include measures to assess response biases, especially social desirability.



3. Preparing for Data Collection

Care should be taken to make the instrument attractive,professional looking, and easy to understand.

SOs may be asked to evaluate the appearance of theinstrument before it is reproduced

Avoid proximity effect in order minimize artificially inflate estimates of internal consistency





1. Basic Item Analysis

The performance of each item on the preliminary scaleneeds to be evaluated empirically to determine the

correlation of items

The degree of inter-item correlation can be assessedby inspecting the correlation matrix of all the items.



1. Basic Item Analysis

Next is to compute for the item-scale correlations(computation of preliminary total scale scores and then

calculate the correlations between individual items andtotal scores on the scales they’re intended to represent)



2. Exploratory Factor Analysis

Factor analysis disentangles complexinterrelationships among items and identifies items that

“go together” as unified concepts

Factor analysis offers an objective, empirical methodof clarifying the underlying dimensionality of a large set

of measures





A. Factor Extraction

Condenses items into a smaller number of factors andused to identify the number of underlying dimensions

Estimates of initial factors are obtained using principal components analysis (the most commonly usedextraction method)




In principal components analysis, linear combinationsof the observed variables are formed.

The 1st principal component is the combination thataccounts for the largest amount of variance in thesample (1st extracted factor).




The 2nd principle component accounts for the nextlargest amount of variance and is uncorrelated with

the first (2nd extracted factor).

Successive components explain progressively smallerportions of the total sample variance, and all are

uncorrelated with each other.




To decide on how many factors we need to representthe data, we use 2 statistical criteria:

Eigenvalues (sum of the squared item weights for thefactor)

The Scree Test ( a sharp drop in the percentage of explained variance that indicates the appropriatetermination point)



The determination of the number of factors isusually done by considering only factors

with Eigen valuesgreater than 1.

Factors with a varianceless than 1 are no betterthan a single variable,since each variable isexpected to have a

variance of 1.

Total Variance Explained

Comp

onent

Initial Eigenvalues Extraction Sums of Squared

Loadings

Total % of

Variance Cumulativ

e % Total % of

Variance Cumulativ

e % 1 3.046 30.465 30.465 3.046 30.465 30.465 2 1.801 18.011 48.476 1.801 18.011 48.476 3 1.009 10.091 58.566 1.009 10.091 58.566 4 .934 9.336 67.902 5 .840 8.404 76.307 6 .711 7.107 83.414 7 .574 5.737 89.151 8 .440 4.396 93.547 9 .337 3.368 96.915 10 .308 3.085 100.000 Extraction Method: Principal Component Analysis.

Eigenvalues



The examination of the Scree plot provides a visual of the total variance associated witheach factor.

The steep slope shows the large factors.

The gradual trailing off (scree) shows the restof the factors usually lower than an Eigen value of 1.

In choosing the number of factors, inaddition to the statistical criteria, one should

make initial decisions based on conceptualand theoretical grounds.

At this stage, the decision about the numberof factors is not final.

Scree Test



B. Factor Rotation

In this step, factors are rotated

Un-rotated factors are typically not very interpretable(most factors are correlated with may variables)



B. Factor Rotation

Factors are rotated to make them more meaningful andeasier to interpret (each variable is associated with a

minimal number of factors)

Different rotation methods may result in theidentification of somewhat different factors



B. Factor Rotation

The most popular rotational method is Varimaxrotations

Varimax use orthogonal rotations yielding uncorrelatedfactors/components

Varimax attempts to minimize the number of variablesthat have high loadings on a factor. This enhances theinterpretability of the factors



B. Factor Rotation

Other common rotational method used include obliquerotations which yield correlated factors

Oblique rotations are less frequently used because theirresults are more difficult to summarize

Other rotational methods include:Quartimax (Orthogonal)

Equamax (Orthogonal)

Promax (oblique)





3. Making Final Decision

The final decision about the number of factors to chooseis the number of factors for the rotated solution that is

most interpretable

To identify factors, group variables that have largeloadings for the same factor




Plots of loadings provide a visual for variable clusters

Interpret factors according to the meaning of the variables




Plots of loadings provide a visual for variable clusters

Interpret factors according to the meaning of the variables



Assumption underlying factor analysis include:

The measured variables are linearly related to thefactors + errors This assumption is likely to be violated if items limited

response scales (two-point response scale likeTrue/False, Right/Wrong items)

The data should have a bi-variate normaldistribution for each pair of variables



Assumption underlying factor analysis include:

Observations are independent

The factor analysis model assumes that variables aredetermined by common factors and unique factors. All unique factors are assumed to be uncorrelated with each other and with the common factors



3. Scale Refinement & Validation

Revising the Scale

The analyses undertaken in the development study oftensuggest the need to revise or add items

Examine items with high factoring loading when addingnew items

Examine the content of the items remaining in the scale

Scoring and Transforming the ScaleItem scores are typically just added together to form asubscales scoring of items or to form a total scores

Adding together of items implies equally importantindicators of latent variable.

Development of construct scales that have equalnumber of items per subscale

Conducting a Validation Study Include the worth of instrument in a validation study

Confirmatory factor analysis (CFA) is a hypothesistesting approach to determine where the items belongsfrom a specific factors

Establishing Cutoff Points

Produce dichotomous of scale construct along thecontinuum

There must be a score that distinguishes those who passand those who fail

Highly intended for diagnostic and screening scales

Establishing Norms

This occurs if the expectation is that (a) the scale will be widely used by the population and help evaluate theirscores, and (b ) average scale scores vary markedly by members of well-defined subpopulations

Norms are most commonly established for key demographic characteristics such as age and gender



4. Translating Scales into Other Languages

Well-constructed scales with good psychometricproperties are increasingly likely to be translated for use

of other culture

Translations are often centered (original language) ordecentered approach (allow modification to the

wording of items in the original scale)




Both conceptual equivalence and semanticequivalence are critical to the success of a translated

effort.

The “gold standard” for semantic equivalence involvesback-translation, in which the scale is first translated

from the source language into the target population, andthen translated back to the source language




The translated version is then tested in a manner of similar to the original scale.

Evidence for semantic equivalence and psychometricsoundness comes from pre-tests of both original andtranslated scale with a sample of bilingual people, and

comparison of reliabilities, factor structures, and other validity estimates between the two scales



Date post:	03-Apr-2018
Category:	Documents
Upload:	kit-roaquin
View:	214 times
Download:	0 times

Nursing Research II-Prelims

Documents