Scientific Research Scientific Research Methods Methods

Prof. Dr. Hüseyin PADEMProf. Dr. Hüseyin PADEM

International Burch UniversityInternational Burch University

Introduction and OverviewIntroduction and Overview One. Introduction and Overview:1One. Introduction and Overview:1 Two. Planning and Designing a research Study: 26Two. Planning and Designing a research Study: 26 Three. General Approaches for Controlling Artifact and Bias: 65Three. General Approaches for Controlling Artifact and Bias: 65 Four. Data Collection, assessment Methods, and Measurements Four. Data Collection, assessment Methods, and Measurements

Strategies: 95Strategies: 95 Five. General Tipes of Research Designs and Approaches: 123Five. General Tipes of Research Designs and Approaches: 123 Six. Validity: 158Six. Validity: 158 Seven. Data Preparation, Analyses, and Interpretation: 198Seven. Data Preparation, Analyses, and Interpretation: 198 Eight. Ethical Consideration in Research: 233Eight. Ethical Consideration in Research: 233 Nine. Disemination Research Results and Distilling Principles of Nine. Disemination Research Results and Distilling Principles of

Research Design And Methodology: 261Research Design And Methodology: 261

BooksBooks

Marczyk, G., DeMatteo, D., Festinger, G., 2005. Essentials of Marczyk, G., DeMatteo, D., Festinger, G., 2005. Essentials of Research Design and Methodology. John Wiley & sons, Inc., USAResearch Design and Methodology. John Wiley & sons, Inc., USA

Bordens, K.S., Abbott, B.B., 2008. Research Design and Methods. Bordens, K.S., Abbott, B.B., 2008. Research Design and Methods. McGraw Hill Pub. USA McGraw Hill Pub. USA

McBurney, D.H., White, T,L., 2010. Research Methods. Wadsworth McBurney, D.H., White, T,L., 2010. Research Methods. Wadsworth Cengage Learning Pub.USA Cengage Learning Pub.USA

Saunders, M., Lewis, P., Thornhill, A., 2007. Research Methods for Saunders, M., Lewis, P., Thornhill, A., 2007. Research Methods for Business Students. Pearson Education. UK.Business Students. Pearson Education. UK.

Lester, J.D., 1995. Writting Research Papers. Harper Collins College Lester, J.D., 1995. Writting Research Papers. Harper Collins College Pub. USAPub. USA

One. Introduction and OverviewOne. Introduction and Overview

Progress in almost every field of science depends on thProgress in almost every field of science depends on thee contributations made by systematic researchcontributations made by systematic research

The purpose of the research is to ansver questions and The purpose of the research is to ansver questions and acquire new knowledgeacquire new knowledge

Research makes valuable contributions so what we know Research makes valuable contributions so what we know and how we think about things and eventsand how we think about things and events

Research can be used for the purposes of description, Research can be used for the purposes of description, explanation, and predictionexplanation, and prediction

In recent years, the results of various research studies In recent years, the results of various research studies have taken center stage in the popular mediahave taken center stage in the popular media (Cancer, (Cancer, nutrition, stress … etc)nutrition, stress … etc)..

Science and scientist conjure up a variety of images in our Science and scientist conjure up a variety of images in our minds.minds.

Common image is that of a person in a white lab coat Common image is that of a person in a white lab coat surrounded by bubbling flasks and test tubes…surrounded by bubbling flasks and test tubes…

Simply put, Simply put, science science is a set of methods used to collect is a set of methods used to collect information about phenomena in a particular area of information about phenomena in a particular area of interest and build a reliable base of knowledge about them.interest and build a reliable base of knowledge about them.

A scientist is someone who does science. A scientist is a A scientist is someone who does science. A scientist is a person who adopts the methods of science in his or her person who adopts the methods of science in his or her quest for knowledge. However, the simple definition does quest for knowledge. However, the simple definition does not capture what scientist do.not capture what scientist do.

Science as a way of thinking: science is not just a means of Science as a way of thinking: science is not just a means of acquiring knowledge, it is also a way of thinking and of acquiring knowledge, it is also a way of thinking and of viewing the world. A scientist approaches a problem by viewing the world. A scientist approaches a problem by carefully defining its parameters, seeking out relevant carefully defining its parameters, seeking out relevant information, and subjecting proposed solutions to rigorous information, and subjecting proposed solutions to rigorous testing.testing.

How do scientists do science: in their quest for knowledge How do scientists do science: in their quest for knowledge about a phenomenon, scientists can use a wide variety of about a phenomenon, scientists can use a wide variety of techniques, each suited to a particular purpose.techniques, each suited to a particular purpose.

Basic And Applied researchBasic And Applied research

Basic research, is conducted to investigate issues Basic research, is conducted to investigate issues relevant to the confirmation of theoretical or relevant to the confirmation of theoretical or empirical positions. The major goal of basic empirical positions. The major goal of basic research is to acquire general information about research is to acquire general information about phenomenon. FE, chemistry, physics, botanic.phenomenon. FE, chemistry, physics, botanic.

Applied research, the focus of applied research is Applied research, the focus of applied research is to investigate a problem based in the real world. to investigate a problem based in the real world. Applied research can be found in the areas of Applied research can be found in the areas of clinical, environmental, and industrial psychologyclinical, environmental, and industrial psychology

One.. What Exactly is Research?One.. What Exactly is Research?

Research studies come in many different forms. Research studies come in many different forms. Two of the most common types of research:Two of the most common types of research:

A.Correlational research: The goal is to determine A.Correlational research: The goal is to determine whether two or more variables are related. whether two or more variables are related. (‘Variable’ is a term with that can take on different (‘Variable’ is a term with that can take on different values, such a weight, time, height …). There are values, such a weight, time, height …). There are several different types of correlation (Chapter 5)several different types of correlation (Chapter 5)

B.Experimental research: involves comparing two B.Experimental research: involves comparing two group on one outcome measure to test some group on one outcome measure to test some hypotesis regarding causation. One group is the hypotesis regarding causation. One group is the experimental group, other is the control groupexperimental group, other is the control group

One.. Overview of Science and The One.. Overview of Science and The Scientific MethodScientific Method

ScienceScience can be defined as a methodological can be defined as a methodological and and systematic approach to the acquisition of new knowledge.systematic approach to the acquisition of new knowledge. Systematic, control and method.Systematic, control and method.

Scientific knowledge is not based on the opinions, feelings, Scientific knowledge is not based on the opinions, feelings, or intuition of the scientist. Instead, scientific knowledge is or intuition of the scientist. Instead, scientific knowledge is based on objective data that were reliable obtained in the based on objective data that were reliable obtained in the context of a cerefully designed research study.context of a cerefully designed research study.

Scientific methods:Scientific methods: is best thought of as an approche to the is best thought of as an approche to the acquisition of new knowledge, and this approach effectively acquisition of new knowledge, and this approach effectively distinguishes science from nonscience. It isn’t a single distinguishes science from nonscience. It isn’t a single method.method.

Scientific methods most agree that it is characterized by Scientific methods most agree that it is characterized by the Empirical approach, observation, question, hipotheses, the Empirical approach, observation, question, hipotheses, experiments, analyses, conclusion and replicationexperiments, analyses, conclusion and replication

The Research Process – a Process Perspective

Figure 3.1 The research process

One.. Overview of Science and The One.. Overview of Science and The Scientific MethodScientific Method

Empirical Approach is an evidence-based approach that relies on direct observation and experimentation in the acquisition of new knowledge. In this approach, scientific decision are made based on data derived from direct observation and experimentation.

Observation refers to two distinct concepts –being aware of the world around us and making careful measurements.

- Observation of the world around us can offen provide with many ideas for research studies.

- Avoid making biased observation. - An important aspect of measurement is operational definitiation. What

is exercise?

Questions: After getting a research idea, next step involves translating the research idea into an answerable question. ‘answerable’ particulary important, and it should not be overlooked.

Hypotheses: is simply an educated and testable guess about the answer to your research question. A key feature of all hypotheses is that each must make a prediction

Experiment -or research study: Research would collect data for hypotheses

Analyses: generally calls for statistical techniques. Conclusions: After analyzing the data and determining whether to

reject the null hypothesis, the researcher is now in a position to draw some conclusions about the results of the study. It statistically significant effect or not

Replications: essentially means conducting the same research study a second time with another group .

Null hypothesis always predict that there will be no Null hypothesis always predict that there will be no difference between the groups. Rejecting the null difference between the groups. Rejecting the null hypothesis mean that there is a difference between groups. hypothesis mean that there is a difference between groups. *Ho is reject**Ho is reject*

There are 2 types of errors,There are 2 types of errors, Type 1. researcher concludes There is difference between Type 1. researcher concludes There is difference between

groups, in fact there is no difference *falce positive*.groups, in fact there is no difference *falce positive*. Type 2. researcher concludes There is not a difference Type 2. researcher concludes There is not a difference

between groups, in fact there is a difference *falce between groups, in fact there is a difference *falce negative*negative*

One.. One.. Goals of Scientific Research

Description: refers to the process of defining, classifying, or categorizing phenomena of interest.

Quantitative or Qualitative– Quantitative research: involves studies that make use of

statistical analises to obtain their findings. Key features include formal and systematic measurement and the use of statistics

– Qualitative research involves studies that do not attempt to quantify their results through statistical summary or analysis. Interviews and observations without formal measurement.

Nomothetic or Idiographic The Nomothetic approach uses the study of groups to

identify general laws that apply to large group of people. The idiographic approach is the study of an individual.

– The population: is all individual of interest to the researcher.

– The sample: is a subset of the population Positive correlation between two variables means that both

variables change in the same direction (either both increase or both decrease)

Negative (inverse) correlation between two variables means that as one variable increases, the other variable decreases.

The difference in emphasis in qualitative versus quantitative methods

Table 8.1 The difference in emphasis in qualitative versus quantitative methodsSource: Based on Reichardt and Cook (1979).

One..One.. Goals of Scientific Research

Prediction:To be able to predict future events before they occur, then guessing the answers before the answers.

Understanding/explanation: true understanding of a phenomenon is achieved only when researchers successfully identify the cause or causes of the phenomenon.

Prediction, Understanding/ explanation

CHOOSING A RESEARCH TOPIC

The first step in designing any research study is deciding what to study.

Researcher choose the topics that they study in a variety of vays, and their decisions are necessarily influenced by several factors. For example, scientific field (management, biology, physics, medicine…)

Training and experience, and it typically takes many years to develop an area of expertice.

It is highly unlike that a botanist would coose to study quantum physics or macroeconomics.

What about I or T. And team work (%50+%50).

How researchers choose the topics ‘1.Interest’

Interest: researchers typically coose research topics that are of interest to them.

A reseracher’s basic curiosity about an observed phenomenon typically provides sufficient motivation for choosing a research topic

How researchers choose the topics

2. Problem solving: In each of research studies, researchers are attempting to solve some specific problem, such as work-related stress…

3. Previous research: researchers also choose research topics based on the results of prior research, whether conducted by them or by someone else.

Password is ‘RESEARCH BEGETS RESEARCH’ 4. Theory: theories often serve as a good source

for research ideas. Research ideas should be based on some theory. Theory makes a prediction

LITERATURE REVIEV

Once a researcher has chosen a specific topic, the next step in planning phase of research study is reviewing the existing literature (book, journal articles…) in that topic area.

Fortunately, the development of comrehensive electronic databases has facilitated the process of conducting literature reviews.

Researcher can change the focus or methodology of their studies based on the types that have already been conducted.

FORMULATING A RESEARCH PROBLEM

Good research problem must meet three criteria: a. The research problem should describe the

relationship between two or more variable b. The research problem should take the form of a

question. c. The research problem must be capable of being

tested empirically (ie., with data derived from direct observation and experimentation)

ARTICULATING HYPOTHESESARTICULATING HYPOTHESES

Next step is articulating the hypotheses that will be testedNext step is articulating the hypotheses that will be tested Hypotheses attempt to explain, predict, and explore the phenomenon Hypotheses attempt to explain, predict, and explore the phenomenon

of interestof interest There are two importent points: first, all hypotheses must be There are two importent points: first, all hypotheses must be

falsfalsiifiable; fiable; second a hypotheses must make a second a hypotheses must make a predictionprediction (usually one (usually one or two variable)or two variable)

Null hypotheses (Ho)Null hypotheses (Ho): always predicts that there will be no : always predicts that there will be no differences between the groups being studied.differences between the groups being studied.

By contrast, the By contrast, the alternate hypotheses (H1)alternate hypotheses (H1) always predicts that there always predicts that there will be a difference between the groups.will be a difference between the groups.

In scientific research, null hypotheses is tested, and then the null In scientific research, null hypotheses is tested, and then the null hipotheses is either confirmed or refuged (some times rejected or not hipotheses is either confirmed or refuged (some times rejected or not rejected)rejected)

Control Group The control group is exposed to

the same conditions as the experimental group, except for

the variable being tested.

All experiments should have a control group.

Directional Hypotheses and Nondirectional Hypotheses: It’s second category of research hypotheses Researcher has some idea about how the groups being studied will

differ. Researchers use nondirectional hypotheses when they belive that the

groups will differ, but they do not have a belief regarding how the groups will differ. They don’t know the direction of differences. Examples: nondirectional hypotheses

1. H:there is a significant differences between night worker (product/h) and day’s worker (product/h).

By contrast, researchers use directional hypotheses when they belive that the groups being studied will differ, and they have a belief regarding how the groups will differ. They know the direction of differences.Example:Directional Hypotheses: H:night worker produce (product/h) more than day’s worker (product/h).

Do you know the difference Do you know the difference between the independent between the independent and dependent variables?and dependent variables?

CHOOSING VARIABLES TO STUDY

Independent Variables vs. Dependent Variables: The independent variable (independent) is the factor that is

manipulated or controlled by the researcher. Mostly researcher are interested in the effect of it.

The dependent variable (dependent) is a measure of the effect (if any) of the independent variable.

Categorical Variables vs. Continuous Variables Categorical variables are variables that can take on specific values

only within a defined range of value. Gender (male/female), hair colour (blonde/brunette/ redhead) marital status (single/married/divorced).

Continuous variables are variable that can theoretically take on any value along a continuum. Age, weight.

Quantitative Variables vs. Qualitative Variables Qualitative variables are variable that vary in kind. Attractive/not

attractive, helpful/not helpful, consistent/not consistent Quantitative variables are those that vary in amount.

Experimental research

Students

Group A

Group B

Two methods of instruction

Group A

Group B

Assessment Comparation

Which method is effective? A, or B

Independent variable

Dependent variable

RESEARCH PARTICIPANTS

Number, kind of participants? Selecting Study Participants Random selection: is a procedure through which a

sample of participants is chosen from the population of interest in such a way that each member of the population has an equel probability of being selected to participate in the study.

Random assignment: Assigning study participants to groups within the study (Population:400 male, 100 famile, group must be %75 M, %25 FM)

GENERAL APPROACHES FOR CONTROLLING ARTIFACT AND BIAS

The primary purpose of research design is to eliminate source of bias.

Confounds are general names that source of artifact and bias.

Reduce the impact of artifact and bias.

The strategies should be considered EARLY in the design phase.

A BRIEF INTRODUCTION TO VALIDITY

Validity refers to the conceptual an scientific soundness of research study or investigation, and the primary purpose of all forms of research is to produce valid conclusions.

Validity is related to research methodologyFour Types of Validity1. Internal validity: refers to the ability of research design to rule

out or make implausible alternative explanations of the results2. External validity: refers to the generalizability of the results of

a research study.3. Construct validity: refers to the basis of the causal relationship4. Statistical validity: refers to aspect of quantitative evaluation

that affect the accuracy of the conclusion drawn from the results of a study

Methods for controlling sources of artifact and bias

Statistical control Control and comparison group Random selection Random assignment Experimental design

For example, research pareticipants bring a wide variety of phisical, emotional traits into the research contex. These different characteristics can directly affect the results of a study.

Experimental Bias

Ironically, the researchers themselves are the first common source of artifact and bias.

These biases are particularly pravalent in studies in which a single researcher is responsible for generating the hypotheses, disigning the study and collecting and analizing the data.

Factors influencing research

Figure 2.3 Factors influencing researchSource: Forcese, Dennis P.; Richer, Stephen, Social Research Methods, 1st edition 1973. Electronically reproduced by permission of Pearson Education, Inc. Upper Saddle River, New Jersey.

Strategies for minimizing experimenter effects

Carefully control or standardize all experimental procedures.

Provide training and education to all the researchers involved in this study

Minimize dual or multiple roles within the study Checks, and balances and quality control procedures,

whenever possible. Automate procedures, whenever possible Conduct data collection Choice of statistical analyses. Limit the knowledge that researchers and participants are

not receiving experimental manipulation.

Approaches for limiting researcher’ knowledge of participant assignmentEach of the procedures seeks to reduce or minimize the

researcher’s knowledge about the participants and about which experimental conditions they are assigned to.

Double-blind technique: neither the participants nor the researchers know which experimental or control condition research participants are assigned to.

Blind technique: only the researcher be kept ‘blind’ regarding which treatment or control conditions the participants are in.

Partial-blind technique: this is similar to the blind technique, except that the researcher is kept blind regarding participant selection for only a portion of the study.

Participant effectsParticipant effects

““Participant effectsParticipant effects”” are a source of artifact and bias are a source of artifact and bias stemmingstemming from a variety of fac from a variety of facttors related to the unique ors related to the unique motives, attitudes, and behaviors that participants bring to motives, attitudes, and behaviors that participants bring to any research study.any research study.

The “good” participant might attempt to provide information The “good” participant might attempt to provide information and responses that might be helpful to the study, and responses that might be helpful to the study,

The “negative” participant might try to provide information that The “negative” participant might try to provide information that might confound or undermine it,might confound or undermine it,

The “faithful” participant might try to act without bias, The “faithful” participant might try to act without bias, while the “apprehensive” participant might try to distors his or while the “apprehensive” participant might try to distors his or

her responses in a way that portrays him or her in an overly her responses in a way that portrays him or her in an overly positive or fovorable lightpositive or fovorable light

Controlling participants effectsControlling participants effects

Double-blind technicqueDouble-blind technicque Deception technicque: when researchers use Deception technicque: when researchers use

deception, it usually takes the form of providing deception, it usually takes the form of providing participants with misinformation about the true participants with misinformation about the true hypotheses of interest or the focus of the study. hypotheses of interest or the focus of the study.

Inquiry: The researcher can simply ask the Inquiry: The researcher can simply ask the participants about any number of issues related to participants about any number of issues related to participant effects and the overall purpose and participant effects and the overall purpose and hypotheses of the study.hypotheses of the study.

Random assignment

Potential artifact_experimenter; bias_participant effects The basic purpose of random assignment is to obtain

equivalence among groups. RA is a control technique in which all participants have an

equal likelihood of being assigned to any of the experimental or control groups.

RA increases internal validity because it distrubutes or equalizes potential confounds across experimental and control groups.

Studies that use random assignment are referred to as true experiments, while studies that do not use random assignment are referred to as quasi experiments.

Holding variables constant

Common method for holding the influence of a specific variable or variables constant in a study is refered to as matching.

For exam: we have two group; experimental and control

Blocking: unlike matching, is an approach that allows the researchers to determine what specific impact the variable in question is having on the depent variable. There are two independent variables: experimental (impaired and adequate) and control (impaired and adequate).

Statistical approaches

Final method One statistical approach for determining

equivalence between groups is to use simple analyses of means and standard deviations for the variables of interest for each group in the study

A means is simply an average score A standard deviation is a measure of variability

indicating the average amount that scores vary from the mean.

Research Never Stops

Figure 2.2 The wheel of research

Four. Data Collection, Assesment Methods, And Measurement Strategies-Measurement

Measurement is a critical juncture between scientific theory and application and describe, explain and predict the phenomena.

Measurement is important in research design in two critical areas. First, measurement allows researchers to quantify abstract construct and variables. Second, the level of statistical sophistication used to analyze data derived from a study is directly dependent on the scala of mesaurement used to quantify the variables of interest.

Scales of Measurements Scales of Measurements

There are There are four main scalesfour main scales of measurement subsumed of measurement subsumed under the broader categories of under the broader categories of nonmetricnonmetric ((nonmetric nonmetric datadata, , cannot be quantcannot be quantified and used to describe and categorize) ified and used to describe and categorize) and and metricmetric (metric data are used to examine amounts and (metric data are used to examine amounts and magnitudes) measurement,magnitudes) measurement, 11..nominal scales, nominal scales, 22. . ordinal ordinal scales, scales, 33. . interval scales, and interval scales, and 44. . ratio scales. Nominal and ratio scales. Nominal and ordinal scales are nonmetric measurement scales.ordinal scales are nonmetric measurement scales.

Nominal scalesNominal scales, gender, religious and political affiliation, , gender, religious and political affiliation, marital status.marital status.

Ordinal scalesOrdinal scales, built on nominal measurement. Examples , built on nominal measurement. Examples would be finishing position of runners in race (greater than or would be finishing position of runners in race (greater than or less than)less than)

Interval scalesInterval scales of measurements are quantitative of measurements are quantitative in nature, built on ordinal measurement, numbers in nature, built on ordinal measurement, numbers scaled at equal distances, no absolute zero point, scaled at equal distances, no absolute zero point, examples include temperature measured in examples include temperature measured in Fahrenheit and Celsius.Fahrenheit and Celsius.

Ratio scalesRatio scales include height, weight, and time. include height, weight, and time. Numerous examples of ratio scale data exist in our Numerous examples of ratio scale data exist in our daily lives. It is possible to have to zero point.daily lives. It is possible to have to zero point.

Scales of Measurement

Table 6.1 Scales of measurement

Presentation:Writing the Final Report

Guidelines

1.Audience

2.Good communication

3.Language and flow

4.Form and style

5.Headings

1. Title page2. Table of contents3. Executive summary4. Introduction and problem statement5. Theoretical background6. Methodology7. Findings of empirical study8. Conclusions and recommendations9. Footnotes10. Bibliography or references11. Appendix

Presentation:Structure of the report

Presentation: An Example of a Table of Contents

Figure 13.1 An example of a table of contents

Getting started with writing

Practical hints

Create time for your writing Write when your mind is fresh Find a regular writing place Set goals and achieve them Use word processing Generate a plan for the report Finish each writing session on a high point Get friends to read and comment on your work

Writing for different audiences

Key differences between an ‘Academic’ report

and a ‘Consultancy’ or ‘Management’ report

The academic report: Tends to be longer Will be marked and graded Will contain contextual descriptions

The consultancy report: Has less focus on the development of theory Contains recommendations relating to the organisation’s

business

Report structure (1)

The abstract

Four short paragraphs that answer the questions:

1. What were my research questions and why were they important?

2. How did I go about answering the research questions?

3. What did I find out in response to these questions?

4. What conclusions can be drawn?

Adapted from Saunders et al. (2009)

Report structure (2)

Introduction - include

The research questions(s) and a clear statement of research objectives

Brief background and a guide to the storyline

Literature review - purpose

To set your study in the wider context To show how your study supplements existing work

Report structure (3)

Checklist

Complete the Checklist

for points to include in your method chapter

Developed from Robson (2002)

Report structure (4)

Results chapter(s) - purpose

To report the facts your research discovered To support the facts with quotes from participants

Discussion chapter- purpose

To interpret results and relate the findings to the original research goals and objectives

To indicate implications of the research

Report structure (5)

Using a matrix in the planning of the content for the results and conclusions chapters

Saunders et al. (2009)

Figure 14.1 Using a matrix in the planning of the content for the results and conclusions chapters

Report structure (6)

Conclusion chapter – purpose

To answer the research question(s) To meet the research objectives To consider the findings To present any contributions to the topic displayed

in the literature To reflect on any implications for future research

Report structure (7)

References

Use a convention that is accepted by your university (e.g. Harvard, APA)

Cite all sources referred to in the text Check all citations to prevent plagiarism

Appendices

Include only essential supporting material Include copies of interview schedules Keep appendices to a minimum

Organising the report content (2)

Main points to consider

Choosing the title

Telling a clear story

Helping the reader by-Dividing your workPreviewing and summarising chaptersUsing suitable tables and graphicsWriting in a suitable style

Writing style

Key points:

Clarity and simplicity – avoid jargon

Checking grammar and spelling

Preserving anonymity

Regularly revising each draft

Evaluating the first draft

Checklist

Complete the Checklist

to help you evaluate the first draft

Saunders et al. (2009)

Oral presentation

Three key stages:

Planning and preparation

Use of visual aids

Presenting

Summary: Chapter

Writing is a creative process and a powerful way to clarifying your thinking

A project report needs a clear structure that helps to develop the storyline

All the information should be readily accessible to the reader

Summary: Chapter

Use a clear writing style free and check for spelling and grammatical errors

Be prepared to rewrite the first draft several times

Remember to check the assessment criteria

Summary: Chapter

Failing to prepare for your presentation is preparing to fail

Visual aids help the audience understand your presentation

Remember to –

Tell them what you're going to say Say it Tell them what you said