Frank Hodge, 1

Frank HodgeUniversity of Washington

Computer-Based Computer-Based Instruments:Instruments:

Design Issues and Benefits

Frank Hodge, 2

OvervieOvervieww

DesignDesign BenefitBenefitss

ConcluConcludede

“An expert is a person who has made all the mistakes that can be made in a very narrow field.”

--Niels Bohr

Frank Hodge, 3

OvervieOvervieww

DesignDesign BenefitBenefitss

ConcluConcludede

Instrument types─ Attachments─ Surveys─ Basic experiments─ Complex experiments

Frank Hodge, 4

OvervieOvervieww

DesignDesign BenefitBenefitss

ConcluConcludede

Coding─ You

─ Student helper

─ A professional─ In house─ Outside

Frank Hodge, 5

OvervieOvervieww

DesignDesign BenefitBenefitss

ConcluConcludede

Interface─ Basic─ Adding bells and whistles

Data collection─ Format

Frank Hodge, 6

OvervieOvervieww

DesignDesign BenefitBenefitss

ConcluConcludede

Attachments─ Example

─ Sedor (The Accounting Review, 2002)

Frank Hodge, 7

OvervieOvervieww

DesignDesign BenefitBenefitss

ConcluConcludede

Surveys─ Front end

─ Back end

─ Examples─ Hodge (Accounting Horizons,

2003)─ U of Washington’s WebQ─ External resources (e.g.,

SurveyMonkey.com)

Frank Hodge, 8

OvervieOvervieww

DesignDesign BenefitBenefitss

ConcluConcludede

Basic experiments─ Random assignment

─ Front end

─ Back end

─ Example─ Elliott WP─ External resources

Frank Hodge, 9

OvervieOvervieww

DesignDesign BenefitBenefitss

ConcluConcludede

Complex experiments─ Random assignment

─ Front end

─ Back end

Frank Hodge, 10

OvervieOvervieww

DesignDesign BenefitBenefitss

ConcluConcludede

Complex experiments─ Examples

─ Hodge, Kennedy & Maines (The Accounting Review, 2004)

─ Elliott, Hodge, Kennedy & Pronk (WP)

─ Hodge & Pronk (WP)

Frank Hodge, 11

Frank Hodge, 12

Frank Hodge, 13

OvervieOvervieww

DesignDesign BenefitBenefitss

ConcluConcludede

Questions to ask yourself─ Who will do the coding?

─ What do I want the front end to look like?

─ What data do I want to collect?

─ ∑BENEFITS > ∑COSTS

Frank Hodge, 14

OvervieOvervieww

DesignDesign BenefitBenefitss

ConcluConcludede

“Potential” benefits─ Not geographically constricted─ Not restricted to “normal” hours─ Ability to collect larger samples─ Experiment “reaches out” to

subjects─ Potential to reduce costs─ Making changes can be easier

Frank Hodge, 15

OvervieOvervieww

DesignDesign BenefitBenefitss

ConcluConcludede

“Potential” benefits─ Validity benefits─ Ability to gather “behind the

scenes” data─ Being asked to be on a panel…

Frank Hodge, 16

OvervieOvervieww

DesignDesign BenefitBenefitss

ConcluConcludede

Before deciding to use a computer-based experiment do a cost-benefit analysis.

∑BENEFITS*1/2 > ∑COSTS x 2

Frank Hodge, 17

OvervieOvervieww

DesignDesign BenefitBenefitss

ConcluConcludede

Will computer-based experiments become the norm in accounting?

“Prediction is very difficult, especially of the future.”

--Niels Bohr