Introduction to Embedded Systems Research:Course Goals, Organization, and Logistics
Robert Dick
[email protected] of Electrical Engineering and Computer Science
University of Michigan
Fovea
Lens
Cornea
Variable Resolution
and Position Sampling
Change-Adaptive
Signalling
Spatial State
Cache
(Occipital
Place Area)
Analasys, e.g.,
Classification
Adequate
decision confidence?
Sampling Guidance
Y
N
Long-Term Memory
Decision /
Result
(b) Iterative, multi-round human vision system.
Image Signal
Processor
Application Processor
(CPU and/or GPU)
CloudDecision /
Result
(a) Conventional machine vision pipeline.
Image Sensor: typically
homogeneous RGGB or RCCC.
Demosaicing, binning,
denoising, gamma
correction, and compression.
Hardware Feature
Extraction Accelerator
or
Feature extraction on
raw captured data.
Runs CNN, LSTM or
other analysis algorithm.
May drop computation on less
important data, but already payed
Image Signal Processor transfer cost.
May render decision or (at high
energy cost) do feature extraction
and defer decision to cloud.
Minimalistic
Image Pre-Processor
Application Processor
(CPU and/or GPU)
CloudDecision /
Result
Image Sensor: capture only the
most important
data for decision accuracy.
Efficient gamma
correction
and binning.
Decide based on features.
Very high energy cost for
wireless data transfer.
Scene Cache
Capture ControllerHardware Feature
Extraction Accelerator
or
Adequate
decision confidence?
or
Y
N
Captures most relevant
and rapidly changing data.
Learns important sample locations
from prior rounds.
Maintains state built from
prior still-relevant samples.
Determine and
transmit
relevant data.
Decide based on features.
Very high energy cost for
wireless data transfer.
Issue commands to
capture important data.
(c) Goal: multi-round, energy-efficient, low-latency
continuous learning machine vision.
Feature extraction on sparse captured
data with similar distribution to processed data.
Continuously learn features and important
data based on prior captures.
1.1040
2.1041
385
2.1039
704
1.1039
36
2.1040
1734
0.1039
1
3.1040
4
4.1039
642
3.1039
4.1040
409
3.1041
396665
5.1042
108612774
5.1040
337
4.1041
10644
6.1044
117
6.1039
609
6.1045
164
6.1040
841434 723938
6.1042
209
5.1041
4 12 164
5.1045
529417
5.1044
140105 88
5.1039
154 10551 90677
7.1039
1248
7.1047
2106
7.1040
29773362 241966 22903106
6.1041
3119 1936 40
6.1047
4128 253
8.1050
4
8.1042
9.1050
24784
8.1039
89632
8.1044
2840
8.1040
1088
9.1039
10.1040
4
11.1039
957
10.1050
144
11.1040
156
10.1041
32 16
10.1047
87780
10.1045
2152
10.1039
1165
12.1040
145
13.1052
2404
12.1042
24
12.1045
33008
12.1044
8217
12.1041
8
12.1039
135427
14.1049
113
14.1040
229
13.1050
132
13.1042
74433
13.1041
17187
13.1040
2715
13.1039
170059 90
15.1040
16
15.1050
242 1225
14.1050
237
14.1042
6200
14.1041
4
14.1044
720
14.1052
20
14.1039
84 36939
16.1040
88
15.1049
6
15.1039
56
17.1054
2919
16.1050
129
16.1041
36
16.1047
49154
16.1045
2632
16.1049
27
16.1052
16
16.1039
222734
18.1048
133
18.1039
3439
18.1040
241
17.1050
16
18.1049
36 339224
17.1042
172832
17.1041
49620 448
17.1040
1376493 2883
17.1045
72
17.1044
1073
17.1049
55441826
17.1048
124 3648290
17.1052
3547
17.1039
2484110477445
17.1047
24
19.1040
72
19.1039
6 88 60631617
18.1041
28 76
18.1047
3305
18.1054
13744
20.1040
109
20.1039
269
19.1052
8
19.1047
11712
19.1049
10
19.1054
7520
21.1039
82
20.1049
5
20.1047
4896
20.1054
864
22.1040
4
22.1050
23.1040
4
22.1039
144
24.1058
3389
23.1050
76
24.1040
4
23.1042
17528
23.1041
4
23.1054
24
23.1044
6234
23.1058
261
23.1049
4
23.1052
2944
23.1039
3069658
25.1040
80
24.1050
4
24.1039
58
25.1039
26.1040
4
27.1039
489
26.1050
4
27.1040
4
26.1047
3808
26.1045
2248
26.1058
113 80
26.1049
11
26.1039
66
26.1054
840
28.1055
84266 1542
27.1042
1229
27.1041
29619
27.1058
12
27.1049
3984
27.1048
35337
29.1040
262
29.1056
164
29.1039
742
28.1050
4 2464
28.1042
2137912
29.1055
1128
28.1041
4 2633 4
28.1040
2716132 691
28.1058
24 84 32
28.1049
36176 3
28.1048
1192 48
28.1039
365 110957 24475
29.105029.104229.104129.104729.104529.105829.104929.105229.1054
2
3
4
5
6
7
8
0 1 2 3 4 5 6 7 8
Pow
er
(mW
)
Time (s)
35 40 45 50 55 60 65 70 75 80 85 90
-8 -6 -4 -2 0 2 4 6 8
-8
-6
-4
-2
0
2
4
6
8
35 40 45 50 55 60 65 70 75 80 85 90
Temperature (°C)
Position (mm)
Temperature (°C)
IntroductionCourse structure
Deadlines
Outline
1. Introduction
2. Course structure
3. Deadlines
2 R. Dick EECS 507
IntroductionCourse structure
Deadlines
Embedded system definition
An (application-specific) computerwithin something else
that is not generally regardedas a computer.
3 R. Dick EECS 507
Common embedded system requirements
Mobile: limits PCB and power supply size.
Wireless: power and reliability implications.
Reliable: consider cars.
First time correct: field repairs difficult.
Rapidly implemented: IP reuse, automation, corner cutting.
Low price: competition between many companies.
High-performance: massively parallel, using ASICs.
Low power: battery life and cooling costs.
Secure: complicates design analysis.
Integrated w. physical world: noise / security / control / other implications.
Hard real-time: deadlines must not be violated.
IntroductionCourse structure
Deadlines
Embedded system structure
5 R. Dick EECS 507
IntroductionCourse structure
Deadlines
Embedded systems market
Dominates general-purpose computing market in volume.
∼$200-billion/year.
Similar in monetary size to general-purpose computing market.
Growing at 6.3%/year, 5% for general-purpose computing.
Car example: 50% of value from embedded systems, zero a few decades ago.
6 R. Dick EECS 507
IntroductionCourse structure
Deadlines
Waves in computing
∼1980: personal computers.
∼1995: the internet.
∼2010: smartphones.
What’s next?
?: Wearables.
?: Internet-of-Things.
When? 2020–2025?
7 R. Dick EECS 507
IntroductionCourse structure
Deadlines
Prediction
Claim
Every new class of computer systems will initially be seen as a toy by most.
As it becomes socially and commercially important, nearly everybody will actas if it was always obvious . . .
. . . even those who claimed it would always be a toy.
Predicting change
If logic dictates change, ignore the naysayers.
But consider customers.
8 R. Dick EECS 507
IntroductionCourse structure
Deadlines
Embedded systems market fragmentation
Application-specific.
Divergent hardware and software.
Limited market size for each class.
Many small–moderate size companies.
Limited engineering staff.
Implication
Embedded systems engineers must have deep knowledge of many levels ofthe design process.
9 R. Dick EECS 507
IntroductionCourse structure
Deadlines
Course goals
My goals
Prepare you for independent research on embedded systems.
How?
Develop broad background in major embedded system related research areas.
Your goals?
10 R. Dick EECS 507
IntroductionCourse structure
Deadlines
Examples
Distinguish between objectives and design.
Understand design as an optimization problem.
Estimate energy and thermal characteristics of an embedded system.
Understand energy consumption, performance, and reliability characteristicsof many wireless communication standards.
Explain the architectural and algorithmic implications of focusing onparticular embedded domains, e.g., automotive, wearable, or mobile.
Identify main security vulnerabilities of a given embedded architecture.
Implement several algorithms appropriate for ultra low power machinelearning on resource-constrained platforms.
Etc.
11 R. Dick EECS 507
IntroductionCourse structure
Deadlines
Commitment
Will do my best to
structure the course to help you toward the goals,
revise structure based on my observations and your feedback,
treat you fairly, and
share my experience.
12 R. Dick EECS 507
IntroductionCourse structure
Deadlines
Some find me scary...
“You are a good professor and I like you, butI’m scared of you.”
— Project advisee at graduation, afterrequesting a photo with me.
Something about my demeanor (serious,high-energy, very blunt) makes me lessapproachable to many students.
13 R. Dick EECS 507
IntroductionCourse structure
Deadlines
... but I like helping students
Ignore it and come to office hours if you want to talk about the class,embedded systems related career, entrepreneurship, etc.
Serious + blunt 6= annoyed + grumpy.
Everybody schedules 15-minute meetings with me soon, in groups of two.
I’ll ask you about your goals for the course and career, and about yourbackground.
Ask me anything.
14 R. Dick EECS 507
IntroductionCourse structure
Deadlines
Amount of course structure
Graduate−
level
You are
here.
First−time
course
Repeated
course
High
HighUndergraduate
Low
Low
You can easily find more structured and predictable courses.
15 R. Dick EECS 507
IntroductionCourse structure
Deadlines
Main expectations
Time per week
10–15 hours/week for those without projects.
20–30 hours/week for those with projects.
Behave honorably.
Do the assignments.
Actively participate in the search for good ideas during paper discussions.
For project version: develop and evaluate a novel idea related to embeddedsystems.
16 R. Dick EECS 507
IntroductionCourse structure
Deadlines
This course is optional
No teaching assistants.
Relatively unstructured.
A lot of reading.
Read/understand the papers you don’t present.
Give several presentations.
Projects will take time; more time than one credit would suggest.
Nobody requires this course to graduate.
If it looks interesting, welcome.
If it looks too burdensome or unstructured, drop it.
17 R. Dick EECS 507
IntroductionCourse structure
Deadlines
Your teacher: Robert Dick
Associate Professor in EECS, joined in 2009.
∼100 research papers on embedded systemdesign, best paper awards, several products.
Founding CEO of direct-to-consumer Strydathletic wearable electronics company.
Visiting Professor at Tsinghua Univeristy.
Visiting Researcher at NEC Labs, America,where technology went into their smartphones.
Graduate studies at Princeton.
Organized IEEE Design and Test special issue onthe second half of this course.
18 R. Dick EECS 507
IntroductionCourse structure
Deadlines
Outline
1. Introduction
2. Course structure
3. Deadlines
19 R. Dick EECS 507
IntroductionCourse structure
Deadlines
Lectures
Attendance is required.
Email me if you need to miss due to illness or travel.
Tell me before class if you need to leave early due to an unavoidable conflict.
There will be a few guest lectures by experts on particular topics.
Initially, lectures will take the entire class period.
Later, much of the class period will be spent on research paper presentationand discussion.
20 R. Dick EECS 507
IntroductionCourse structure
Deadlines
Exams and quizzes
Midterm exam.
Final exam
Cumulative but focusing on material after midterm.
Quizzes
Accounting for at most 15% of the exam portion of the course grade.
Primary purpose: determine your current state of understanding so Ican best allocate lecture and reading assignment time.
21 R. Dick EECS 507
IntroductionCourse structure
Deadlines
Reading
One research paper per class on average.
Paper summaries
Summarize each paper using the template provided.
I grade a random subset.
Other students also provide feedback, but not grades.
Paper summary critiques
Critique randomly assigned summaries by other students.
I will spot check these and reserve the right to penalize poor ones.
Based on past experience, students almost always do well on these.
22 R. Dick EECS 507
IntroductionCourse structure
Deadlines
Paper presentations
Starts in late January.
Teams will present each paper.
You will need to read the paper in detail and prepare slides.
Summarize the paper briefly and indicate promising applications, areas ofdiscussion, and connections with other work and concepts.
You will have at least a week’s notice of assigned presentation requirements.
Others are expected to read the paper in full and come with questions andobservations.
I will also ask questions, and comment/summarize if necessary.
23 R. Dick EECS 507
IntroductionCourse structure
Deadlines
Projects
Required for four-credit version of course.
Extensions of research ideas introduced in the course or novel ideas.
Generally, theory or design concepts validated via prototypes.
Quality sufficient for a research conference publication.
Report should be at a level enabling submission to a conference if another100 hours were spent on revision.
Team or individual work are both permitted.
Project presentations
Teams will present their projects to the entire class.
The class provides feedback.
24 R. Dick EECS 507
IntroductionCourse structure
Deadlines
Competition and collaboration
Not constrained to a particular number of As.
If the class as a whole does very well, more As.
Converse also true.
Collaboration on understanding the papers and making projects great willgenerally increase the number of As.
But don’t hold back on hard questions. “I don’t know but will follow up viaPiazza” is a perfectly legitimate response. . .
. . . unless it is on a very basic concept from the paper you are leading.
25 R. Dick EECS 507
IntroductionCourse structure
Deadlines
Website
https://robertdick.org/iesr/.
Simple hand-written website to manage course handouts and assignments.
Grades will be shared through Canvas.
Piazza
For administrative notification and discussion.
Required.
Do not miss Piazza announcements.
Topics of interest thread
Questions about embedded systems related topics that might not berelated to items in the syllabus.
May influence lectures and course topics.
26 R. Dick EECS 507
IntroductionCourse structure
Deadlines
Grading scheme
Without project
Summaries of assigned reading and critiques: 35%
Presentation and questions: 35%
Exams: 30%
With project
Summaries of assigned reading and critiques: 30%
Presentation and questions: 30%
Exams: 20%
Project: 20%
27 R. Dick EECS 507
IntroductionCourse structure
Deadlines
Outline
1. Introduction
2. Course structure
3. Deadlines
28 R. Dick EECS 507
IntroductionCourse structure
Deadlines
Homework
All due 10 January.
Estimated total time: 30 minutes.
Look at every link in the website menu. More will appear during the course.
Download and review the slides from the first lecture.
Make a post on Piazza indicating your main areas of interest related toembedded systems.
Select a time for your meeting with me.
https://docs.google.com/spreadsheets/d/1GD66KyNmTKcyTvLRUX-SGrAMfkgXe4w2KBZ1pyW79VQ/edit?usp=sharing.
29 R. Dick EECS 507
