FEELE LabFEELE LabFinance and Economic Finance and Economic

Experimental Laboratory at Experimental Laboratory at ExeterExeter

Todd KaplanTodd Kaplan

Dieter BalkenborgDieter Balkenborg

Tim Miller Tim Miller

FDTL5 Grant for Bringing Experimental Economics into the Classroom

Bringing Economic Bringing Economic Experiments into the Experiments into the

ClassroomClassroomObjective: Put experiments into the undergraduate economics curriculum at the University of Exeter.Thanks to:

Higher Education Academy in England for the £s.

Dept. of Economics at Exeter.

Economics Network (England)

Steve Gjerstad, Denise Hazlett, Charlie Holt

Why use classroom Why use classroom experiments?experiments?

Inspires students.Inspires students. Chamberlin 1948 ran a simple market experiment. Chamberlin 1948 ran a simple market experiment. This inspired Vernon Smith to go into This inspired Vernon Smith to go into

experimental economics.experimental economics. Todd Kaplan was a student at Caltech 20 years Todd Kaplan was a student at Caltech 20 years

ago. Experiments with Charlie Plott inspired him ago. Experiments with Charlie Plott inspired him to switch majors to econ.to switch majors to econ.

Memorable experience for students. Memorable experience for students. Brings students closer to research. Allows Brings students closer to research. Allows

research to help teaching and teaching to research to help teaching and teaching to help research.help research.

We have embedded 48 experiments into 18 We have embedded 48 experiments into 18 undergraduate modules at Exeter.undergraduate modules at Exeter.

Types of Classroom Types of Classroom Experiments Experiments HandRun.HandRun.

Quick raise hands (symmetric prisoners’ dilemma).Quick raise hands (symmetric prisoners’ dilemma). Sampling paper collection (2*2 games, currency Sampling paper collection (2*2 games, currency

attack).attack). More sophisticated (Pit market, Sloman’s Trade Game).More sophisticated (Pit market, Sloman’s Trade Game).

Computerized.Computerized. Web based: PHP or Java (Bertrand, Double Auction).Web based: PHP or Java (Bertrand, Double Auction). Locally based/installed (z-tree).Locally based/installed (z-tree).

Homework.Homework. Simple Q&A with feedback in class: (Rubinstein’s site).Simple Q&A with feedback in class: (Rubinstein’s site). More advanced Individual Choice experiments with More advanced Individual Choice experiments with

some immediate feedback (Monty Hall).some immediate feedback (Monty Hall). Play against a fictitious/robot/prior human player (Holt: Play against a fictitious/robot/prior human player (Holt:

Traveler’s dilemma).Traveler’s dilemma). Students play each other at designated time.Students play each other at designated time.

Research.Research. Single lecture (Chamberlain)Single lecture (Chamberlain) Complete semester (Selten/Mitzkewitz/Uhlich, Iowa Pol. Complete semester (Selten/Mitzkewitz/Uhlich, Iowa Pol.

Stock Market) Stock Market) Requirement to be a subject. (Psychology)Requirement to be a subject. (Psychology)

Hand run Hand run Advantages:Advantages:

Suitable for large lectures.Suitable for large lectures. Some take just minutes.Some take just minutes. Engaging for students .Engaging for students .

Disadvantages:Disadvantages: May require careful preparation, including room May require careful preparation, including room

structure. structure. May require assistants/volunteers.May require assistants/volunteers. May require lecturer w/ practice: Student May require lecturer w/ practice: Student

experience may vary. experience may vary. Data collection/entry takes time. Feedback may Data collection/entry takes time. Feedback may

be delayed. (the next lecture)be delayed. (the next lecture) One can only run for few rounds.One can only run for few rounds. Some students may not be participating.Some students may not be participating.

Handrun Hints:Handrun Hints: Techniques for low setup costs. Techniques for low setup costs.

At beginning of semester, cut simple strips of paper At beginning of semester, cut simple strips of paper (you may want to use two colours).(you may want to use two colours).

Bring several plastic bags from home to collect Bring several plastic bags from home to collect answers.answers.

Give verbal instructions/ display question.Give verbal instructions/ display question. Quick to collect and sample a few. Quick to collect and sample a few.

Display last year’s results. Display last year’s results. Know what you are doing beforehand, Know what you are doing beforehand,

particularly with assistants.particularly with assistants. Example: Guessing game.Example: Guessing game.

Guess a number 0 to 100.Guess a number 0 to 100. The guess closest to 2/3 the average number wins a The guess closest to 2/3 the average number wins a

prize.prize. Ties will be broken randomly.Ties will be broken randomly.

Graph of guesses (Exeter Graph of guesses (Exeter UGs)UGs)

Average was 36.7 (winning guess 24.5), equilibrium is 0!

Six guesses were above 66.66

02468

1012141618

0-10 10-20 20-30 30-40 40-50 50-60 60-70 70-80 80-90 90-100

Exeter UG 2nd round guesses

Average was 12.4 (winning guess 8.3)

One guess was above 66.66

05

1015202530354045

0-10 10-20 20-30 30-40 40-50 50-60 60-70 70-80 80-90 90-100

Todd’s quote “you shouldn’t ignore theory, even when it is wrong.”

ComputerizedComputerized Advantages:Advantages:

Readily available, great for beginners, Readily available, great for beginners, uniform experience.uniform experience.

Immediate results, data ready for evaluationImmediate results, data ready for evaluation Disadvantages:Disadvantages:

Experiments standardized, limited flexibility Experiments standardized, limited flexibility Room requirements, split large lectures, use Room requirements, split large lectures, use

tutorialstutorials May need trained teaching assistantMay need trained teaching assistant Equipment (PhP vs Java, handhelds)Equipment (PhP vs Java, handhelds) Needs one hour of teaching timeNeeds one hour of teaching time

Computer Experiment HintsComputer Experiment Hints

Try using two students per computer (cuts Try using two students per computer (cuts down on web surfing, email browsing and down on web surfing, email browsing and gossip).gossip).

Give instructions beforehand (helps foreign & Give instructions beforehand (helps foreign & dyslexic students).dyslexic students).

Let students play all treatments in easiest Let students play all treatments in easiest order. (order. (within subjectwithin subject treatments) treatments)

Example: Bertrand Competition.Example: Bertrand Competition. Students act as firms in a market.Students act as firms in a market. They choose prices each period.They choose prices each period. Customers go to the firm with the lowest price.Customers go to the firm with the lowest price.

Sample result: Bertrand Sample result: Bertrand GameGame

0

1

2

3

4

5

6

7

8

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29

Time

Price

Average Price Average Selling Price

Marginal Cost

“I learnt that collusion can take place in a competitive market even without any actual meeting taking place between the two parties.”

Two Firms

Fixed Partners

Two Firms

Random Partners

Five Firms

Random Partners

“Some people are undercutting bastards!!! Seriously though, it was interesting to see how the theory is shown in practise.”

HomeworkHomework Advantages:Advantages:

Saves lecture timeSaves lecture time Less hassleLess hassle Many periods possible (no time limit)Many periods possible (no time limit) Easy access to homework dataEasy access to homework data

Disadvantages:Disadvantages: Typically not interactiveTypically not interactive Participation rate can be low unless Participation rate can be low unless

incentives are in place.incentives are in place. Hint: use some sort of incentive.Hint: use some sort of incentive.

ResearchResearch Advantages:Advantages:

RESEARCHRESEARCH Low cost (saves recruitment costs).Low cost (saves recruitment costs).

Disadvantages:Disadvantages: Organization needs to be more careful Organization needs to be more careful

than standard teaching experiments.than standard teaching experiments. Limited communication between and to Limited communication between and to

students.students.

Challenges for Classroom Challenges for Classroom ExperimentsExperiments

StudentsStudents LecturersLecturers Module (Course) StructureModule (Course) Structure

ChallengesChallenges

Students:Students: They have limited time too (both inside and They have limited time too (both inside and

outside the classroom).outside the classroom). Maintain attention (negative externality of both Maintain attention (negative externality of both

data and speed).data and speed). Assess performance/motivation (earnings is Assess performance/motivation (earnings is

noisy, attendance has problems).noisy, attendance has problems). How can we write test (or homework) questions How can we write test (or homework) questions

that require student to attend experiment?that require student to attend experiment? Some may (correctly/incorrectly) resent being Some may (correctly/incorrectly) resent being

used for research experiments at a sacrifice of used for research experiments at a sacrifice of their studies.their studies.

ChallengesChallenges

Lecturers:Lecturers:

4 types: Eager, Interested but need 4 types: Eager, Interested but need support, hesitant, unwilling.support, hesitant, unwilling.

Need to want to try something new.Need to want to try something new. Need to sacrifice course time.Need to sacrifice course time. Need to think experiments are of Need to think experiments are of

value.value.

ChallengesChallenges

Classes:Classes: Many courses don’t have experiments Many courses don’t have experiments

that fit directly into them (macro / that fit directly into them (macro / finance).finance).

Same experiments for different courses Same experiments for different courses (avoid repeat).(avoid repeat).

Some experiments may require students Some experiments may require students to be in two courses. (Run an experiment to be in two courses. (Run an experiment in micro economics and analyze it for the in micro economics and analyze it for the statistics course.)statistics course.)

Rewards

Students• Enjoyable, Interactive• Better grades (Emerson & Taylor, 2004)• Concrete learning experience (rather than

abstract/mathematical formulas).• Some students do well in experiments but

poor on tests (such as those with poor math skills).

Lecturers• Richer teaching style -> evaluations• Promote experimental economics (preach

what we practice).

How to use experimentsHow to use experimentsSize can determine how:Size can determine how: Large lectures (>100): use short hand-run, Large lectures (>100): use short hand-run,

homeworkhomework Medium Lectures (40<#<100): make use of Medium Lectures (40<#<100): make use of

computerized experiments in tutorials.computerized experiments in tutorials. Small Lectures (<40). Possible to use Small Lectures (<40). Possible to use

computerized experiments in place of lecturescomputerized experiments in place of lecturesExperiments seem to work well for all levels of Experiments seem to work well for all levels of

students (even high school)students (even high school)Hints:Hints: Usually do experiments before covering the Usually do experiments before covering the

material in the course.material in the course. Let students participate in preparation, execution Let students participate in preparation, execution

and evaluation. (Especially in an experimental and evaluation. (Especially in an experimental class.)class.)

Relate some exam questions to experiments.Relate some exam questions to experiments. Don’t be too obsessed with preserving a research Don’t be too obsessed with preserving a research

environment. environment.

Exeter GamesExeter Games

On purpose same style as Veconlab On purpose same style as Veconlab (Holt).(Holt).

Goal is to complement selection of Goal is to complement selection of experiments.experiments.

14 computerized experiments.14 computerized experiments. Types are a mix of individual choice Types are a mix of individual choice

(homework) and interactive games. (homework) and interactive games.

Exeter Games ExperimentsExeter Games Experiments1.1. Bertrand Market (micro / IO)Bertrand Market (micro / IO)2.2. Call Option (finance, homework)Call Option (finance, homework)3.3. Currency attack (macro)Currency attack (macro)4.4. Diamond Dybvig bank run (macro /banking)Diamond Dybvig bank run (macro /banking)5.5. Insurance (economics of social policy, micro)Insurance (economics of social policy, micro)6.6. Holdup Problem (corporate finance)Holdup Problem (corporate finance)7.7. Kiotaki Wright fiat money (macro /banking)Kiotaki Wright fiat money (macro /banking)8.8. Lemon Game (micro / IO, homework)Lemon Game (micro / IO, homework)9.9. Monty Hall (statistics, homework)Monty Hall (statistics, homework)10.10. Network Externalities (micro / IO)Network Externalities (micro / IO)11.11. Price Discrimination (micro, homework)Price Discrimination (micro, homework)12.12. Team Draft (game theory)Team Draft (game theory)13.13. Warren Buffet investment (finance)Warren Buffet investment (finance)14.14. Introductory Statistics (statistics)Introductory Statistics (statistics)

How to find Exeter GamesHow to find Exeter Games

Google for “Feele” to find the FEELE Laboratory Google for “Feele” to find the FEELE Laboratory homepage.homepage.

Ignore the large link “Register here”, which is for Ignore the large link “Register here”, which is for subjects to register for one of our research subjects to register for one of our research experiments.experiments.

Instead scroll down to “Access our website of Instead scroll down to “Access our website of teaching experiments”.teaching experiments”. View list of experiments (short descriptions of each)View list of experiments (short descriptions of each) Experimenter access (soon to be you!)Experimenter access (soon to be you!) Participant access (your students use this to log in; you Participant access (your students use this to log in; you

use it to check that your experiment is set up correctly)use it to check that your experiment is set up correctly)

Brief descriptions of each of our experiments.

Click here to register as an experimenter and run your first experiment.

Start by registering your email address.

The Code Word is ‘deadsea’.

Username will be based on your initials.

Check your details and confirm.

Username and Password will be emailed to you, after which you can log in here.

Enter details from registration email and click here.

Click here to create your first experiment.

Choose a code word to allow students to log in to your experiment.

Your students will need to know this Access Code to be able to log in to your experiment.

Click here to configure, run and monitor your new experiment.

The new experiment is empty and you have to add the first session.

Click here to makes changes to the default configuration.

Enter the number of students in your class (default: 2).

Enter the number of rounds you want to run (default: 1).

Save your changes.

Useful feature: you can view the effects of your configuration changes on the subject instructions by clicking here.

Don’t forget: you must start the experiment before the first subject can log in!

Click here to monitor subjects as they log in and view the results of the experiment.

Open a NEW browser window (not a tab), locate the FEELE homepage and click here to log in as a test subject in your new experiment.

Your students will log in here.

Consists of your Username (tk2), a hyphen (-) and whatever code word you chose when you created your experiment (network).

The student enters his/her names here. (Note: you may enter “test” to suppress the display of subject instructions during testing.)

The student checks his/her details and confirms.

You now need to open a further 5 NEW browser windows (not tabs) and log in the remaining test subjects.

This is what a typical result set looks like.

Draws graph of results.

Exports results to Excel (space delimited).

Breakdown of test question responses.

Reminder of your configuration settings.

Reminder of subject instructions.

Subjects often want to know who won!

How to find “our” How to find “our” Wikiversity siteWikiversity site

Google for “Feele” to find the FEELE Google for “Feele” to find the FEELE Laboratory homepage.Laboratory homepage.

Scroll down to “Wikiversity Site for Scroll down to “Wikiversity Site for Economic Classroom Experiments” Economic Classroom Experiments” and click on “View Wikiversity Site”.and click on “View Wikiversity Site”.

On “our” Wikiversity siteOn “our” Wikiversity site

Why use experiments in teaching economicsWhy use experiments in teaching economics List of ExperimentsList of Experiments

Hand-run as well as computerizedHand-run as well as computerized Not just Exeter GamesNot just Exeter Games

CoursesCourses Hints for running economic classroom Hints for running economic classroom

experimentsexperiments Useful Links and Related LiteratureUseful Links and Related Literature

Links to popular Veconlab (Holt) and Econport Links to popular Veconlab (Holt) and Econport sites.sites.

Quick link to Wikiversity.

Links to other experiment sites here.

Quick link to Wikiversity.

ResourcesResources

Web based resources.Web based resources. Veconlab (Holt)Veconlab (Holt) Exeter Games Exeter Games Econport (Cox, Gjerstad)Econport (Cox, Gjerstad) Game Theory website (Rubinstein)Game Theory website (Rubinstein) Denise Hazlett’s websiteDenise Hazlett’s website ExperonomicsExperonomics