Post on 26-Mar-2015
transcript
SUSAN B. FOWLER, PHD, RN, CNRN, FAHA
Getting Started: What’s Out inthe Literature and Is It Any
Good?
Literature
Research Critique
Is not synonymous with “criticize”Examines the strengths, weaknesses,
meaning, and significance of the study (substantive and theoretical dimensions)
Be objective and realistic in identifying the study’s strengths and weaknesses
Important Points
All studies have weaknesses or flawsResearch is critiqued to broaden
understanding, improve practice, and provide background for conducting a study
The critique process involves comprehension, comparison, analysis, and evaluation
Cochrane Systematic Reviews
Cochrane Library www3.interscience.wiley.com/cgi-bin/mrwhome/106568753/HOME
The Cochrane Collaboration. Oxford: Update Software; 1996-. Updated quarterly.
Cochrane Database of Systematic Reviews (Cochrane Reviews) 'Gold Standard' for high-quality systematic reviews Full-text included in Cochrane Library Cochrane Reviews includes complete reviews and
protocols (reviews that are still in progress) Cochrane Reviews abstracts are in PubMed
More Reviews
Database of Abstracts of Reviews of Effects (Other Reviews - DARE) prepared by the National Health Service Centre for
Reviews and Dissemination, University of York, England
Complements the Cochrane Reviews by offering a selection of quality assessed reviews in those subjects where there is currently no Cochrane review
Brief critical appraisals of previously published reviews of the effects of health care
Structured abstracts, not full-text DARE not indexed in PubMed, but original research
articles may be Also available at no charge on the web from
University of York | www.crd.york.ac.uk/crdweb/
Other Systematic Reviews and Meta-analyses
PubMed MEDLINE - Systematic Reviews | www.ncbi.nlm.nih.gov/entrez/query/static/clinical.shtml access: part of PubMed's Clinical Queries (linked on
PubMed Home and Advanced Search pages) Identifies systematic reviews and other similar types of
studies found in biomedical journals included in PubMed database
No evaluation of comparative quality of different reviews on a topic
Abstracts supplied by authors/journals. No separate evaluation of quality of research
Identify systematic reviews in the biomedical area - gathers together much larger collection than other evidence-based practice resources
Evidence Guidelines/Summaries
BMJ Clinical Evidence | www.clinicalevidence.com/ Compendium of evidence on the effects of clinical interventions Summarizes the current state of knowledge, including knowns and
unknowns, based on thorough search Categorizes interventions as beneficial, likely beneficial, no known
benefit, harmful ... DynaMed | www.ebscohost.com/dynamed
Best available evidence summaries for nearly 2000 topics Clinical reference tool developed for use at the 'point-of-care' Outline format to quickly identify key conclusions Updated daily
Other Resources USPSTF Guidelines | www.ahrq.gov/clinic/prevenix.htm AHRQ Evidence Reports | www.ahrq.gov/clinic/epcix.htm
FPIN Clinical Inquiries included in Journal of Family Practice | www.jfponline.com and American Family Physician | www.aafp.org/afp/
RCTs, Case Cohorts, Control Studies
PubMed | (pubmed.gov ) note: Filtered by publication type, but not quality Clinical Queries - Uses preconfigured search strategies to
retrieve research-based citations on clinical topics in the areas of therapy, diagnosis, etiology, or diagnosis
Use Limits to search by specific publication type (e.g. meta-analysis, randomized controlled trial, review)
Cochrane Library | www3.interscience.wiley.com/cgi-bin/mrwhome/106568753/HOMEThe Cochrane Collaboration. Oxford: Update Software; 1996-. Updated quarterly. Cochrane Central Register of Controlled Trials (Clinical
Trials) Bibliography of controlled trials as part of an international effort to
hand search the world's journals Includes reports published in conference proceedings and in many other
sources not in PubMed or other bibliographic databases
Content of Research Reports
The AbstractThe Introduction (review of the literature,
theoretical framework, significance of and need for the study)
The Methods SectionThe Results SectionThe Discussion SectionThe References
The Beginning
The title Could you tell what the article was about by reading
the title?
The abstract Includes a brief description of the problem How the study was done (methodology) The results Conclusions
Next Steps
The Problem Is it clear what the problem the authors are trying to solve?
Review of the Literature Do the articles relate to the problem? Is it organized, broad to specific? Does it tell a story? Comprehensive and current? What years are covered in the
ROL?Research Question/Aim/Purpose/Hypothesis
Do authors pose a research question or hypothesis? Is it explicitly stated what the aim or purpose of the study is?
Design Is the design stated? Does the design flow from the research question or aim? Did the authors explain why they choose this research design?
QuantitativeQuantitative QualitativeQualitative
Analysis of numerical data Experimental Quasi-experimental Comparative Longitudinal Correlational
Analysis of data such as words (e.g., from interviews), pictures (e.g., video), or objects (e.g., an artifact) Grounded theory - social
processes Phenomonology – lived
experience Ethnography – culture Historical
Design
The Journey Continues
Sample Is the population described? Is the sample method, or how the sample was chosen
described? Is the sample size right for the analysis? Was a power
analysis done to determine sample size? Were standards for protection of human subjects
discussed?Tool
Was the tool created by the authors or already established?
Is a copy of the tool included in the article? Is reliability discussed? Is validity discussed?
ValidityValidity ReliabilityReliability
Extent to which the tool measures what it was intended to Content Criterion related construct
Results are repeatable and consistent Internal consistency Test-retest Inter-rater
Tool, Instruments, Questionnaires, and More
Power
The probability that a statistical test will detect a significant difference that exists - the risk of a Type I error can be calculated using power analysis.
Level of significanceSample sizePower - acceptable level is .80Effect size - the degree to which the null
hypothesis is false
Effect Size
An estimate of how large the treatment effect is, that is how well the intervention worked in the experimental group compared to the control group (intervention studies)
The larger the effect size, the stronger are the experimental intervention’s effects.
Effect size for intervention studies: .2 = small effect .5 = medium effect .8 = large effect
Getting Closer to the End
Methodology Are methods of data collection sufficiently described? Is the time frame when the study occurred described?
Data Analysis Is information presented sufficient to answer the
research question(s)? Were statistical tests used to analyze the data? Were values obtained from the analysis? Was statistical significance reported? Are the results explained? Are tables and figures easy to understand and
informative?
Confidence Intervals
Probability that a value will fall within a range of variables
The larger the CI the less precise the measurement of that variable
A very wide interval may indicates that more data should be collected before anything very definite can be said about the parameter
95% CI most common
The End
Discussion Is a discussion section presented? If yes, are the results compared with the literature
review?
Conclusions Are conclusions clearly stated? Are conclusions directly related to the results? Do the findings add to the present nursing knowledge? Are study limitations identified? Did authors make recommendations for further
research? Carlson, J. (1999). J Emerg Nurs, 25, 330-332.
References
Relevance (extent to which the reference bears on the research question)
Primary sources (descriptions of studies written by the researchers)
Secondary sources (descriptions of studies written by someone other than the original researcher)
Opinion and anecdotal
Level of Significance
The researcher does not know when an error in statistical decision making has occurred. The researcher can control the risk of making a Type I Error by setting the level of significance.
Level of Significance (alpha level) is the probability of making a Type I Error or the probability of rejecting a true null hypothesis.
Level of Significance
Usual level set at .05 (willing to accept the fact that if the study were carried out 100 times, the decision to reject the null hypothesis would be wrong 5 times)
If researcher wants to have smaller risk of falsely rejecting null hypothesis, set at .01.
Decreasing risk of making Type I Error increases risk of Type II Error.
Findings are significant or not significant.
Clinical Significance
Related to practical importance of the findings
No common agreement in nursing about how to judge clinical significance Effect size? Difference sufficiently important to
warrant changing patient care?
Clinical Significance
Who should judge clinical significance? The patients and their families? The clinician or researcher? Society at large?
Clinical significance is ultimately a value judgment
Research Journals
Examples of peer reviewed / refereed journals: Nursing Research, Advances in Nursing Science, Applied Nursing Research, Clinical Nursing Research, Western Journal of Nursing Research
Example of Peer Reviewed online resource: Medscape
Peer Review – Look for statement in first few pages of journal that it is peer reviewed. Content is “reviewed” by experts in field before publication
Let’s Get to Work – Critique in Action!!