Scientific Methods for Management Science

Alexander SettlesHigher School of Economicsasettles@hse.ru

Class Web Site

http://hse.ru/edu/courses/8375505.html

All reading and schedule posted there

Schedule

1. Introduction to the Class – September 7th

2. From Theory to Research & Introduction to Research Reviews – September 14th

3. Review of Most Citied Corporate Governance Research – September 21st

4. Research Design Choices and Causal Inferences – September 28th

5. Inferences Validity and Reliability – October 5th

6. Experiments & Quasi-experiments – October 12th

7. Review of Most Citied Management Research – October 19th

8. Survey Development & Qualitative Research Methods – October 26th

9. Models and Simulation Analysis & Levels of Analysis – November 2nd

Course Objectives

Learn how to use the scientific method

Discuss your topic with fellow students

Find literature sources on management

Develop scholarly writing skills Develop critical thinking skills Instill research ethics

Course Objectives

Investigate professors that are potential dissertation advisors

Learn about human subjects issues Develop bibliographic organization

and citation skills Prepare dissertation proposal

Purpose of this Seminar

Introduction to international research practices

Improve results of your dissertation work and encourage you take an entire year to work on your papers

Gain a working knowledge of methods of scientific analysis

Common mistakes of HSE students in pursuing research

Making the blanket statement that things in Russia are different so social science practices do not apply

Poor literature reviews No methods – summaries of work

experiences or essay format No data to support or reject

hypotheses No conclusions

Course Requirements

Homework Assignment – Research review

Research Proposal Exam Class attendance and participation

Overview of Course Structure

Introduction to the Class and Intro to Research Paper Structure

From Theory to Research Research Design Choices and Causal Inferences Validity and Reliability Research Ethics and Content Analysis Research Experiments Quasi-experiments Survey Development Qualitative Research Methods Models and Simulation Analysis Levels of Analysis

Terms

applied research basic research belief-based

explanation circular explanation

(or tautology) commonsense

explanations confirmation bias deductive reasoning

hypothesis pseudoscience rational method research Science scientific method variable

Is Management a Science?

Falsification and common sense

Falsification is a crucial concept. And the hypothetico-deductive method is the only method for theory verification in all the empirical sciences (Elster, 1983)

It is not incompatible with “common sense”

Theory construction is different from theory verification

Methodological ¨naïveté”

Naïve rationalism: the position that the aim of a scientific theory is to ‘explain’ observed phenomena

Naïve empiricism: belief that for a statement to be scientifically meaningful, it must be logically constructible out of terms which refer to immediate experience.

Naïve pragmatism: the idea that scientific knowledge should be immediately ‘useful’, possibly coming directly from experience, and that the sooner it is used, the better

Contents of a typical research paper

Title page : abstract, contents Introduction Literature Review Description of work done and methods. Results and Discussion. Conclusions and Recommendations. References and Bibliography. Appendices.

Introduction

Define what the problem is The questions you are addressing Outline personal/specific

considerations that lead to this investigation

How it differs from previous work What the report will contain Perhaps some (hint) of the

conclusions

Methods

Describe the method or approach.

Justify that it is appropriate.

Establish constraints or assumptions.

Enable others to repeat the work and check the conclusions.

Link with the research question.

Motivate the work - what is its importance?

Establish approaches used in previous research - the literature search.

Where to Begin: Knowing thingsWhere to Begin: Knowing things

Not so much about what we know, but HOW we know

Most of what we know is matter of belief and agreement “Everybody knows that…” But everybody “knew” the world was flat once

Other way of knowing…direct experience, observation But when experience conflicts with agreement… There is good chance that we’ll surrender our experience in

favor of agreement

Methodology: special approach to inquiry The science of finding out How social scientists find out about human social life

Errors in inquiryErrors in inquiry

Inaccurate observations Most daily observations are casual… not precise Scientific observation is a conscious activity Ex: instructor’s clothes; football toss

Overgeneralization Assuming that a few similar events are evidence of a general

pattern Scientists guard against this by REPLICATION of inquiry

Selective observation We assume a pattern exists then focus on future events that

fit the pattern

Illogical reasoning “Exception that proves the rule” WHAT?...how can that be logical?

Foundations of social science:Foundations of social science:logic and observationlogic and observation

Theory, not philosophy or belief Social theory has to do with what is, not with what should

be...not so for many centuries. Science cannot settle debates about values

Social regularities Social affairs do exhibit a high degree of regularity, despite

exceptions

Aggregates, not individuals Regularities that social scientists study generally reflect the

collective behavior of many individuals

A Variable Language Attributes: characteristics or qualities that describe an object Variables: logical groupings of attributes

Independent and Dependent VariablesIndependent and Dependent Variables

Two concepts are implicit in causal or deterministic models

A dependent variable “depends” on an independent variable

That is, a change in the independent variable will produce a change in the dependent variable

Dialectics of Social Research:Dialectics of Social Research:

Inductive and Deductive TheoryInductive and Deductive Theory

Inductive Reasoning that moves from the particular to the

general...from…

1. a set of observations to…

2. the discovery of a pattern that represents some degree of order among all the given events

Deductive Reasoning that moves from the general to the

specific...from…

1. a pattern that might be logically or theoretically expected to…

2. observations that test whether the pattern actually occurs

Dialectics of Social Research:Dialectics of Social Research:

Quantitative and Qualitative DataQuantitative and Qualitative Data

Most simply put, difference is the distinction between numerical and non-numerical data

Every observation is qualitative at the outset We quantify it to make it easier to aggregate,

compare and summarize the data

Both types of data are useful and legitimate in management research

Elements of Social TheoryElements of Social Theory

Law: universal generalization about classes of facts

Ex: law of gravity—bodies are attracted to each other in proportion to their mass and in inverse proportion to their distance

No social scientific laws that claim universal certainty

Theory: a systematic explanation for observations that relate to a particular aspect of social life...

For example someone might offer a theory of strategy, firm competitiveness, organization, etc.

Elements of Social Theory, Elements of Social Theory, p.2p.2

Proposition: specific conclusions about the relationships among concepts that are derived from axiomatic groundwork

Hypothesis: a specified testable expectation about empirical reality that follows from a more general proposition

Research is designed to test hypotheses Null hypothesis suggests that there is NO relationship

among the variables under study

Traditional model of scienceTraditional model of science

Theory

Operationalization Specification of the exact operations involved in

measuring a variable

For the researcher testing an hypothesis, the meaning of variables is exactly and only what the operational definition specifies

Must be specified with clarity in a way to make observation precise and rigorous

Observation Systematic and rigorous gathering of data to test the

hypothesis

Research DesignPurposes of Research

Exploration:Exploration: typically done for three purposes:

to satisfy the researcher’s curiosity and desire for better understanding

to test the feasibility of undertaking a more extensive study

to develop the methods to be employed in a subsequent study

Description:Description: describe situations and events Census is good example of descriptive research

Explanation:Explanation: the “why?” of events, situations, behavior, attitudes, etc.

Logic of Nomothetic Explanation

NomotheticNomothetic explanation refers to the explanation refers to the accounting of many variations in a given accounting of many variations in a given phenomenonphenomenon

In contrast to…In contrast to…

Idiographic Idiographic explanation that seeks an in-explanation that seeks an in-depth understanding of a single casedepth understanding of a single case

Criteria for Nomothetic Causality

Correlation:Correlation: the variables must be correlated the variables must be correlated

Time order:Time order: the cause takes place before the the cause takes place before the effecteffect

Non-spurious:Non-spurious: the variables are non-spurious the variables are non-spurious

Spurious relationship:Spurious relationship: a coincidental statistical a coincidental statistical correlation between two variables, shown to be correlation between two variables, shown to be caused by some third variablecaused by some third variable

Correlation

Some relationship---or correlation—between the Some relationship---or correlation—between the variables must exist before we can consider variables must exist before we can consider causalitycausality

Correlation:Correlation: empirical relationship between two empirical relationship between two variables such that…variables such that…

Changes in one are associated with changes in Changes in one are associated with changes in the otherthe other

Particular attributes of one variable are Particular attributes of one variable are associated with particular attributes of the associated with particular attributes of the otherother

False Criteria for Nomothetic Causality

Complete causationComplete causation Causation is incomplete and Causation is incomplete and

probabalisticprobabalistic

Exceptional casesExceptional cases Exceptional cases do not disprove Exceptional cases do not disprove

general overall pattern of causationgeneral overall pattern of causation

Majority of casesMajority of cases Causal relationship may be true even if Causal relationship may be true even if

they don’t apply to the majority of they don’t apply to the majority of casescases

Necessary and Sufficient Causes

NecessaryNecessary cause represents a condition that cause represents a condition that mustmust be present for the effect to follow be present for the effect to follow

Ex:Ex: must be female to become pregnant must be female to become pregnant

Ex:Ex: must take college courses to get a must take college courses to get a degree…but…degree…but…

Simply taking courses is not a sufficient causeSimply taking courses is not a sufficient cause Must take the right onesMust take the right ones

Necessary and Sufficient Causes

Sufficient Sufficient cause represents a condition that, cause represents a condition that, if it is present, if it is present, guarantees the effectguarantees the effect in in questionquestion

Not saying that sufficient cause is only Not saying that sufficient cause is only possible cause for effectpossible cause for effect

Ex:Ex: skipping exam in course would be skipping exam in course would be sufficient cause for failing, but students could sufficient cause for failing, but students could fail in other ways, toofail in other ways, too

So, cause can be sufficient but not necessarySo, cause can be sufficient but not necessary

Units of Analysis

No limit to what or whom can be studied

Common social science units of analysis:

Individuals Groups Organizations Social artifacts.

Important:Important: what you ““call”call” a given unit of analysis is almost irrelevant—but you must be clear what that unit “is”“is”

Country level effects or the firms that operate in that country? Firm level strategy or country level strategy? Efficiency of the hotel or the satisfaction of customers?

Reductionism

Tendency to explain everything in terms of a Tendency to explain everything in terms of a particular, narrow set of conceptsparticular, narrow set of concepts

Remember paradigms that predispose Remember paradigms that predispose researcher to a particular explanationresearcher to a particular explanation

Definition of order by coercion, shared values, Definition of order by coercion, shared values, exchangeexchange

Conceptualization, Operationalization & Measurement

ConceptualizationConceptualization The refinement and specification of abstract The refinement and specification of abstract

conceptsconcepts A specific agreed-upon meaning of the A specific agreed-upon meaning of the

concept under studyconcept under study

OperationalizationOperationalization The development of specific research The development of specific research

procedures (operations) that will result in procedures (operations) that will result in empirical observations representing those empirical observations representing those concepts in the real worldconcepts in the real world

Indicators and Dimensions

IndicatorIndicator An observation that we consider as a An observation that we consider as a

reflection of the variable under studyreflection of the variable under study Ex: attending church as an indicator or Ex: attending church as an indicator or

religiosityreligiosity

DimensionDimension A specific aspect of a conceptA specific aspect of a concept Ex:Ex: actionaction aspects of religiosity (attending aspects of religiosity (attending

church, giving money) and church, giving money) and contemplativecontemplative aspects (prayer, etc)aspects (prayer, etc)

Basic Research Outline*

The ProblemThe Problem

PresentPresent a clear, brief statement of the problem, with concepts defined where necessary

ShowShow that the problem is limited to bounds amenable to treatment or test

DescribeDescribe the significance of the problem with reference to specific criteria

Source: Miller, Delbert C. 1991. Handbook of Research Design and Social Handbook of Research Design and Social Measurement, 5th EditionMeasurement, 5th Edition. Newbury Park: Sage Publications, pp. 15-16.

Basic Research Outline

The Theoretical FrameworkThe Theoretical Framework

DescribeDescribe the relationship of the problem to a theoretical framework

DemonstrateDemonstrate the relationship of the problem to previous research

PresentPresent alternate hypotheses considered feasible within the framework of the theory.

Basic Research Outline

The Research Question/HypothesesThe Research Question/Hypotheses

Clearly stateClearly state the research questions or the hypotheses selected for test. (Null and alternate)

IndicateIndicate the significance of test hypotheses to the advancement of research and theory.

DefineDefine concepts or variables (preferably in operational terms).

DescribeDescribe possible mistakes and their consequences.

NoteNote seriousness of possible mistakes.

Basic Research Outline

Design of the Experiment or InquiryDesign of the Experiment or Inquiry

DescribeDescribe ideal design or designs with particular attention to the control of interfering variables

DescribeDescribe selected operational design

SpecifySpecify statistical tests including dummy variables

Basic Research Outline

Sampling ProceduresSampling Procedures

DescribeDescribe experimental and control samples

SpecifySpecify method of drawing or selecting sample

Basic Research Outline

Methods of Gathering DataMethods of Gathering Data

DescribeDescribe measures of quantitative variables showing reliability and validity when these are known. Describe means of identifying qualitative variables

IncludeInclude descriptions of questionnaires or schedules

DescribeDescribe interview procedure

DescribeDescribe use made of pilot study, pretest, trial run.

Basic Research Outline

Working GuideWorking Guide

PreparePrepare working guide with time and budget estimates

EstimateEstimate total person-hours and cost

Basic Research Outline

Analysis of ResultsAnalysis of Results

SpecifySpecify methods of analysis

Basic Research Outline

Interpretation of ResultsInterpretation of Results

DiscussDiscuss how conclusions will be fed back into theory…OR…

InformInform policy/practice.

Basic Research Outline

Publication or Reporting Publication or Reporting Plans...Communication PlansPlans...Communication Plans

MonographMonograph, Executive summary

TestimonyTestimony to policy makers.

PresentationsPresentations to institutions, non-governmental agencies, media, public.

JournalJournal publication