MIT’s Poor Predictions:

Isn’t there a better way to make

predictions and search for

opportunities?

Jeffrey FunkAuthor of Technology Change and the Rise of New Industries

(Stanford University Press, 2013)

jeffreyleefunk@gmail.com

The Context

MIT’s Technology Review produces a list of 10 breakthrough

technologies each year (2001, 2003-2014)

“We have chosen 10 emerging areas of technology that will soon

have a profound impact on the economy and on how we live and

work”

“The mission of MIT Tech Review is to equip its audiences with the

intelligence to understand a world shaped by technology”

These lists are based on conversations with academic

experts from a variety of scientific disciplines

predictions were based on the “educated predictions of our

editors (made in consultation with some of the technology’s top

experts)”

The Context (2)

In October 2016, I gathered market sales data for the

40 predictions done in 2001, 2003, 2004 and 2005 (ten a

year)

Reports by market research companies were major

sources of data

Reports were found by Googling market, size, and sales

for each technology,

sometimes changing the name of the technology or broadly

defining it in order to increase the chances of finding data

The Basic Conclusion

1 of 40 has greater than $100 Billion in sales: Data mining (i.e.,

big data)

3 others have sales greater than $10 Billion: power grid control

(i.e., smart grid), biometrics, distributed storage (i.e., cloud storage)

1 has sales between $5 and $10 Billion: micro-photonics (photonic crystals)

6 have sales between $1 and $5 Billion, 8 have sales between

$100 million and $1 Billion, 14 have sales < $100 million

Data could not be found for 7 technologies probably because

the names were too broad to gather data

Some important technologies were also missed………

MIT’s Technology Review Missed Some Big

Markets that Have Emerged in the 21st Century

Technology Global Market Size, 2015

Smart Phones $400 Billion

Cloud Computing $175 billion

Internet of Things $130 billion

E-commerce for apparel $65 billion

Tablet Computers $60 billion

Social Networking $24 billion

Fintech $20 billion

eBooks $15 billion (only U.S.)

Wearable Computing $14 billion

Technology Review’s

Predictions

Only one (Big Data) has

sales larger than $100

billion

No others have sales

larger than $50 billion

Only three others have

sales larger than $10

billion (biometrics,

cloud storage, smart

grid)

Technologies Chosen in Place of Missed Markets

2005

Airborne Networks

Quantum Wires

Silicon Photonics

Metabolomics

Magnetic-Resonance Force Microscopy

Universal Memory

Bacterial Factories

Enviromatics

Cell-Phone Viruses

Biomechatronics

2004

Universal Translation

Synthetic Biology

Nanowires

T-Rays

Distributed Storage

RNAi Interference

Power Grid Control

Microfluidic Optical Fibers

Bayesian Machine Learning

Personal Genomics

2003

Wireless Sensor Networks

Injectable Tissue Engineering

Nano Solar Cells

Mechatronics

Grid computing

Molecular imaging

Nanoprintlithography

Software assurance

Glycomics

Quantum cryptography

2001

Brain-Machine

Interface:

Flexible Transistors

Data Mining

Digital Rights

Management

Biometrics

Natural Language

Processing

Microphotonics

Untangling Code

Robot Design

MicrofluidicsOrange: <$100 Million sales

Blue: too broad and vague to gather data

Green: Over $10 Billion sales

Predictions/Forecasts Matter

Every successful business (and good decision) implies a successful forecast

Apple’s introduction of MP3 players, smart phones, tablet computers

Amazon’s introduction of cloud services and ebooks

Most companies would have financial problems if they made as bad of forecasts

(and thus product introductions) as did MIT’s Technology Review

Many funding agencies, companies and students follow MIT Technology Review

Thus, Tech Review’s poor predictions have adversely affected many funding, startup, and

career decisions

How could MIT, the leading authority on technology, have made such bad

predictions (and not fix their method of predicting)?

Isn’t there a better way to predict and thus search for opportunities?

MIT is the Leading Authority of

Technology in the World

According to Chronicle of Education, MIT is one of the top ten

universities each year

Recipients of patents, research money, and licensing income

Sources of startups

More than 80 Nobel laureates have been connected in some

way to MIT at some moment in their careers

Its professors are big promoters of techno-optimism: Race

Against the Machine, The Second Machine Age, Andrew McAfee

and Erik Brynjolfsson

As the world’s leading authority on technology, how could

MIT’s Technology Review made such bad forecasts?

Shouldn’t it take responsibility for the predictions and fix the

method?

Possible Reasons for the Bad Forecasts

Lack of accountability?

Major issue in forecasting literature (e.g., Philip Tetlock)

These slides can provide feedback and thus accountability to MIT’s

Technology Review

Not enough time has passed?

Perhaps, but missed markets is bigger issue

Different definition by Tech Review?

Breakthrough science or ideas and not technologies?

Profound impact on economy means something other than money?

If so, then why didn’t Tech Review use different term?

Let’s assume Tech Review meant what it wrote

Cognitive Biases is more Likely Explanation

People assess relative importance of issues, including new technologies

by ease of retrieving from memory

largely determined by extent of coverage in media

E.g., media talks about solar, wind, battery-powered vehicles, bio-fuels and thus many

assume they are quickly diffusing

Second, judgments and decisions are guided directly by feelings of liking and

disliking

One person invested in Ford because he “liked” their products – but was Ford stock

undervalued?

Many people “like” some technologies and dislike others without considering their existing

economics or whether they are improving

Cognitive biases are also biggest reason given by management scholars for

incumbent failure during technological change

Source: Daniel Kahneman, Thinking Fast and Slow, 2011. Kahneman received Nobel Prize for Economics in 2002

How Might Cognitive Biases Apply to Predictions?

MIT’s Technology Review didn’t pay attention to popular media when it

made predictions

But it used a network of engineers and scientists, who may be smarter

than popular media but nevertheless biased. Leading academic engineers

and scientists usually

research elemental technologies that follow “science-based model of tech change”

emphasize new scientific disciplines or ideas, consistent with “science-based model

of technology change”

optimistic about their technologies or those of their colleagues

and thus ignore products and services such as smart phones, tablet computers, and

cloud computing

Upshot is that MIT’s Technology Review chose a wide variety of science-

based technologies many of which will never become big markets

Most of These Technologies Sound More Like Scientific

Disciplines Than Products and Services2005

Airborne Networks

Quantum Wires

Silicon Photonics

Metabolomics

Magnetic-Resonance Force Microscopy

Universal Memory

Bacterial Factories

Enviromatics

Cell-Phone Viruses

Biomechatronics

2004

Universal Translation

Synthetic Biology

Nanowires

T-Rays

Distributed Storage

RNAi Interference

Power Grid Control

Microfluidic Optical Fibers

Bayesian Machine Learning

Personal Genomics

2003

Wireless Sensor Networks

Injectable Tissue Engineering

Nano Solar Cells

Mechatronics

Grid computing

Molecular imaging

Nanoprintlithography

Software assurance

Glycomics

Quantum cryptography

2001

Brain-Machine

Interface:

Flexible Transistors

Data Mining

Digital Rights

Management

Biometrics

Natural Language

Processing

Microphotonics

Untangling Code

Robot Design

MicrofluidicsOrange: <$100 Million sales

Blue: too broad and vague to gather data

Green: Over $10 Billion sales

Is there a Better Way to Find Opportunities?

Many products and services emerge from a process different than science-based process of technology change

Rapid improvements in ICs, other electronic components, and Internet services enable new products and services to become economically feasible and thus emerge ***

can be called Silicon Valley Process of Technology Change

academics call these technologies general purpose technologies

This is very different from science-based process in which advances in science

enable new concepts (radios, TVs, LEDs, LCDs, OLEDs)

facilitate improvements in performance and cost

While improvements in ICs depend on advances in science, we are more interested in products and services that they enable

***See my papers (What Drives Exponential Improvements, California Management Review: Funk J and Magee, 2015. Rapid Improvements

without Commercial Production, Research Policy), my recent book published by Stanford University Press (Technology Change and the Rise of

New Industries), and my slideshare account (http://www.slideshare.net/Funk98/presentations).

Rapid Improvements in Integrated Circuits Have Enabled

Many New Types of Hardware to Emerge

Similar Things Have Occurred with

Computers, Internet and Smart Phones

Rapid Improvements in Computers enabled new forms of

Software applications

Rapid Improvements in Internet enabled new forms

Content

Services

Software

Rapid Improvements in Smart Phones enabled new forms of

Apps

Services

Software

For MIT’s Predictions, Missed and Successful (>$10 Billion)

Predictions Emerged from Improvements in Electronic

Components, Internet Services, and Smart Phones

Missed Predictions

Smart Phones

Cloud Computing

Internet of Things

E-commerce for apparel

Tablet Computers

Social Networking

Fintech

eBooks

Wearable Computing

Successful Predictions

Data Mining (Big Data)

Biometrics

Smart Grid Control

Distributed Storage (Cloud

Storage)

Similar Results Found from Analysis of Wall

Street Journal’s Billion Dollar Startup Club

Consists of global start-ups that

have billion dollar valuations, are still private

have raised money in the past four years

have at least one venture-capital firm as an investor

These startups reflect positive forecasts by investors that the

technologies will experience market growth

119 of 143 on WSJ’s list as of May 2015 were Internet-related

Most emerged through improvements in Internet services, cloud

computing, and access devices (smart phones, tablet computers)

7 others emerged through improvements in electronic components

Category Number of

Startups

Internet

Related?

Software 41 Yes

E-Commerce 26 Yes

Consumer Internet 37 Yes

Financial 15 Yes

Hardware 10

BioTech, Bio-

Electronics

8

Energy 2

Space 1

Other 3

Total 143 119

Wall Street Journal’s Billion Dollar Startup Club

Only 11 startups had patents that

cited more than 10 science or

engineering papers

119 of 143 startups exploited

opportunities that emerged from

improvements in Internet services

and smart phones

10 (hardware) of 143 startups

exploited opportunities that

emerged from improvements in

electronic components

For More Information on the

Analysis, see:

http://www.slideshare.net/Funk98/finding-

billion-dollar-startup-club-opportunities-

67270449

We Need Foxes, Not Hedgehogs

Philip Tetlock and Dan Gardner distinguish between foxes and hedgehogs in

their book Superforecasting: The Art and Science of Prediction (2015)

Foxes make better forecasts because they focus on larger number

of factors than do hedgehogs

Hedgehogs make predictions based on “a few fundamental truths”

Foxes draw on diverse strands of evidence and ideas

Those who solely monitor advances in science can be

called hedgehogs

because they focus on a single issue, what is published in science

and engineering journals

Those who monitor Silicon Valley process of technology change can be

called foxes because they draw on a diverse set of factors

rapid improvements in various electronic components, computers, and the Internet

impact of these improvements on the emergence of new products and services

Implications for Debate Between

Techno-Optimists and Robert Gordon

Techno-optimists such as Erik Brynjolfsson*, Andrew McAfee, Ray Kurzweil, Peter Diamandis and others argue that

changes in computers, mobile phones, displays, the Internet, and artificial intelligence have reached an “inflection point” in which dramatic increases in productivity will soon occur

Some of them also argue that these increases are so fundamentally large that they might cause wide-scale unemployment

Robert Gordon

demonstrates that there were fewer improvements in standard of living between 1940 and 2010 than between 1870 and 1940 in the U.S.

Argues that few improvements will occur in near future

Erik Brynjolfsson and Andrew McAffe. The Second Machine Age, The Race Against the Machine

Above Analyses Provides Support for

Robert Gordon’s Argument

Small markets for Tech Review’s predictions and small

number of science-based opportunities exploited by

billion-dollar startup club suggest

few science-based technologies will become economically

feasible in near future

we can’t be optimistic about: carbon-nanotubes, graphene,

other nano-technology, hydrogen vehicles, hyperloop,

superconducting transmission lines, mag-lev trains, synthetic

food, and fusion

They may diffuse, but the probabilities are small

Above Analyses Suggests Robert

Gordon may be Right (2)

If we can’t be optimistic about science-based

technologies, we are left with technologies that emerge

from improvements in electronic components, Internet

services, and smart phones, such as

Internet of Things, Big Data, ride sharing, driverless vehicles,

drones, smart payment, mobile payments, online education,

augmented reality, and virtual reality

But they don’t impact on all sectors of economy

They probably won’t lead to large reductions in cost of homes,

food, electricity, water, gas, and appliances

Items that dominate budgets of low-income people

For more information on the analysis of

techno-optimists vs. Robert Gordon, see:

http://www.slideshare.net/Funk98/has-

technology-change-slowed

Conclusions

Predicting future “Breakthrough Technologies” is very difficult

Predictions made by MIT’s technology Review were not very accurate

MIT missed technologies with big markets while choosing many with small markets

Even the 7 technologies thought to be too vague is evidence of poor forecast (should use definable technologies)

Silicon Valley Process of Technology Change is a better process to monitor than science-based process of technology change

Can better help decision makers find opportunities

Can help us understand what types of technologies will impact on our lives and the design changes that they will likely bring