2012-10-20 1 1 Mobile&Smart@KMU Gu-Min Jeong, Dong-Byeong Kang, Sung-Soo Lim and Nikil Dutt ICS, UC Irvine, USA EECS, Kookmin University, Korea Mobile&Smart@KMU An Advanced Course Design for Mobile Embedded Software through Android Programming 2 Mobile&Smart@KMU r Introduction r Background and motivation § Recent trend of embedded systems and apps § Main objectives of mobile embedded SW programming § Point of views r MESA design § Course design § Weekly items § Project § Evaluation r Feedback and discussion r Concluding remarks
Transcript
2012-10-20
1
1Mobile&Smart@KMU
Gu-Min Jeong, Dong-Byeong Kang, Sung-Soo Lim and Nikil DuttICS, UC Irvine, USA
EECS, Kookmin University, Korea
Mobile&Smart@KMU
An Advanced Course Design for Mobile Embedded Software through Android
Programming
2Mobile&Smart@KMU
r Introduction
r Background and motivation
§ Recent trend of embedded systems and apps
§ Main objectives of mobile embedded SW programming
§ Point of views
r MESA design
§ Course design
§ Weekly items
§ Project
§ Evaluation
r Feedback and discussion
r Concluding remarks
2012-10-20
2
3Mobile&Smart@KMUMobile Embedded Systems Lab
4Mobile&Smart@KMU
Embedded System - Software
Embedded System- Hardware
Mobile Embedded System Lab@Kookmin Univ.
Auto-MobileMobile Phone Mobile Robot
Cyber Physical SystemEmbedded System
2012-10-20
3
5Mobile&Smart@KMU
• Embedded HW & SW• Automobile, Smartphone and Robot
r Industry aspect § No layers between companies § Value chains made by major companies
CPU/Semiconductor
OS
Device
Application SW/Platform
Content/Service Google
Apple
Samsung
Samsung
Amazon
Qualcomm Intel
MS
10Mobile&Smart@KMU
r App market aspect § App market has changed into a ‘mass market’. § Due to cross platform issues and others, server/web side
development has become more and more important § Developers should be accustomed to multi-language issues.
Early stage – direct connectionUsing APIs in Android platform(Bluetooth, SMS, etc.)
Nowadays – Server based modelSimilar to general web-based model 3G / Wi-Fi connection Server based model
Up to date technology – Hybrid model Smartphone + web Wi-Fi direct for P2P connection NFC pairing for Bluetooth
2012-10-20
6
11Mobile&Smart@KMU
r Mobile embedded SW programming to students§ What to obtain § 1st step : study § 2nd step : research, profit and contest
Study
Research
Profit
Contest
Mobile Embedded SW
Course
1st step 2nd step
12Mobile&Smart@KMU
r Student § Currently
• Increasing demand for the mobile embedded SW related courses • Already, many experts
⇒ Introductory course is meaningless ⇒ Advanced course design⇒ Should provide professional examples
r University§ Limited resource
• Budget • Course • Instructors • Assistants
§ Conservativeness : 4 years curriculum ⇒ Advanced course design for 1 quarter giving fundamental insights
2012-10-20
7
13Mobile&Smart@KMU
r Instructor § Interest of the students for embedded system § ES and CPS point of view
• Application perspective of embedded system • Cross-layer perspective of embedded system
§ Fundamental knowledge
CPU /Semiconductor
OS
Device
Application SW /Platform
Content/Service
CPU/Semiconductor
OS
Device
Application SW/Platform
Content/Service
14Mobile&Smart@KMUMobile Embedded Systems Lab
§ Course design§ Weekly items
2012-10-20
8
15Mobile&Smart@KMU
r What to consider
Limited resources Increasing demand Keeping up with Industry trend
BudgetInstructor
Lecture/lab hours
Design of new courses
Basic/Intermediate/Advanced courses Various aspects of
smartphone
Embedded system Various smartphone
platforms Cross platform
CloudN-screen
Mechanical platforms
CS 190 : Mobile Embedded Software with Android- One Quarter, Advanced course, Project course
16Mobile&Smart@KMU
r Pedagogical issues § Design one quarter (10 week) course§ Enhance the knowledge of related courses and to develop highly
advanced applications. § Review key mobile embedded system platforms and technologies§ Provide many sample applications and open source codes§ Lead the students to develop advanced applications.
2012-10-20
9
17Mobile&Smart@KMU
r Objective of the course design § Stimulating students’ interest for the embedded system§ Introducing recent trend of embedded system and smartphone
app§ Supporting various application examples which can be used
directly for the students’ application design§ Analyzing popular open source projects for Android§ Connecting to other courses § Introducing new keywords in embedded systems, cyber-physical
systems and app related issues§ Summarizing recent trends in related industries such as smart
cars, smart robots and others.
18Mobile&Smart@KMU
r Technical report§ Jeong, G.-M., Kang, D.-B., Lim, S.-S., and Dutt, N.,
‘Mobile Embedded Software with Android: Course Design and Experiences,’ Technical report, Center for Embedded Computer System, TR 12-10,2012.(http://www.cecs.uci.edu/technical_report/TR%2012-10.pdf)
2012-10-20
10
19Mobile&Smart@KMU
Exercise
Introduction to Embedded system, Smartphone app,and smartphone
Basic Android UIProgramming
Lecture
Theories and trend-Multimedia-Network-LBS/AR
Android programming-Multimedia-Network-Sensor
NDK/debugging
Advanced Topics- Embedded System- Smartphone app
Project
20Mobile&Smart@KMU
r Course design
2012-10-20
11
21Mobile&Smart@KMU
r Week 1§ Overview
AndSyncIntroduction Popular Apps
Introduction to the course
Head First Android Programming
Introduction to popular apps
Monday Introduction – Instructors and coursesExamples – Project examples
Wednesday Apps - Popular AppsResearch using Apps – Embedded system, Platform, Multimedia, NetworkSensor, AR, LBS, OPRoS, ROS, etc.
Friday H.O.T. : AndSync app
Lab hours Tool installation
Homework AndSync app
22Mobile&Smart@KMU
r Week 1§ AndSync
Num. 1
Num. 5
Num. 5
Num. 1
Using intent
AndSync Activity AndroidSynchronization Activity
TextView
Button
Main thread
Created threads
Handler
OnClickListnerAndroid:onClick
2012-10-20
12
23Mobile&Smart@KMU
r Week 2§ Overview
Monday Introduction – recent trend of embedded systems and smartphone
Wednesday Android programming overview
Friday H.O.T. : BMP Compare
Lab hours Homework of Week 1
Homework BMPCompare
BMPCompareEmbedded
SystemSmartphone
Android Overview
Introduction to embeddedsystems and smartphone
Comparison of Java, Android API, JNIUnderstanding platform concept
Android - An overview
24Mobile&Smart@KMU
r Week 2§ Platform convergence
MultimediaPlatform
ControlPlatform Cloud
ControlPlatform
Multimedia Platform
Cloud
2012-10-20
13
25Mobile&Smart@KMU
r Week 2§ Smartphone as an embedded system
6290(ARM 9)
L4 Microkernel
REX Wrapper
REX
AMSS
Baseband SW
Linux Kernel
Android
Application SW
Hummingbird(S5PC110)IPC Samsung
Galaxy S2010.6.
26Mobile&Smart@KMU
r Week 2§ BMPCompare
Tap for BMP handling- BMPJava- BMPAndroidAPI- BMPJNI
Reset original image on ImageView
Make image grayscale
Rotate image clockwise
Rotate image Counterclockwise
EditText for inputting to turn angle
ImageView for showing image
2012-10-20
14
27Mobile&Smart@KMU
r Week 3§ Overview
Monday Activity, View– structures of activity, kinds of view and view group
Wednesday Views and UIEvent Handler, MapView, AdapterView, CustomView, SurfaceView, menu, etc.
Friday Layout change, Address book, Block game, Animation example
Lab hours Homework of Week 2
Homework Layout change, Address book, Block game, Animation example
ViewsBlock game
Activity / View
Views and UI
Introduction to activity
View and layout examples
Introduction to Views and UI
28Mobile&Smart@KMU
r Week 3§ Layout change / AdapterView example
§
2012-10-20
15
29Mobile&Smart@KMU
r Week 3§ Animation using SurfaceView
§
Touch
Twinkle star according to timeBackground Image(BMP File)
4 star Image(PNG file)
☜
First direction is decided randomly
Disappear
30Mobile&Smart@KMU
r Week 4§ Overview
Monday Intent, Service– Intent, broadcast receiver– Local Service, Remote Service
Wednesday Debugging and performance analysis- DDMS- Traceview- Profiling- Debugging- Etc.
Friday Spiral image app, FMOD, etc.
Lab hours Homework of Week 7
Homework Spiral image
BMP SpiralNDK
DebuggingJNI/NDK Debugging
JNI and NDK NDK and debugging related examples
Debugging and performance analysis
2012-10-20
22
43Mobile&Smart@KMU
r Week 8§ Android app development using NDK
44Mobile&Smart@KMU
r Week 8§ Android profiling using TraceView
§ Making a spiral image
Rotate
2012-10-20
23
45Mobile&Smart@KMU
r Week 9§ Overview
Monday Power dissipation and energy
Wednesday Performance analysis and toolsOptimization
Friday Cloud computing
Lab hours Homework of Week 8
Homework No homework
Power and energy in
smartphone
Cloud computing
Performance Analysis and Optimization
Powerdissipationand energy
Cloud computing trend
Performance analysis and tools: a trend
46Mobile&Smart@KMU
r Week 9§ A performance analysis tool for Android
1. Kyong, J., Lee, M., Jeong, G.-M., Park, C., and Lim, S.-S. A systematic debugging andperformance analysis framework for Android platforms, Proc. International Workshop onPerformance, Applications, and Parallelism for Android and HTML5 (PAPAH 2012), 2012.
2012-10-20
24
47Mobile&Smart@KMU
r Week 9§ A power consumption example
1. Kim, H. Smart phone, where does the power go ? 2011 EU - KoreaConference on Science and Technology (EKC 2011), 2011.
48Mobile&Smart@KMU
r Week 9§ A cloud computing example
IaaSPaaS
SaaS
SaaS (Software as a Service)• Provide software using cloud• Word, or spreadsheet through web browser• E-mail services, Google Docs, Google Apps
PaaS (Platform as a Service)• Provide platform as a service • Provide tools for online application development
and operation• Provide new SaaS through PaaS• Google App Engine, MS Windows Azure
IaaS (Infrastructure as a Service)• Provide infrastructure(server storage,
network)• Provide server and physical resource of server
by IaaS provider• Power supply, cooling, management of fire-fighting
equipment except server by IaaS business• AWS(Amazon Web Service)
Cloud should be accessed in various environmentWe need-> Abstract server interface on user’s position -> Server virtualization technology is needed
IaaS
PaaS
2012-10-20
25
49Mobile&Smart@KMU
r Week 10§ Overview
Monday N-screen issue
WednesdayMechanical platform- AUTOSAR- ROS- OPRoS
Friday Cross platform / Hybrid app
Lab hours Project related issues
Homework No homework
N-screen Cross platform(HTML5)
Mechanical PF Cross platform(Design tools)
N-screen Cross platform issues
Mechanical platform (AUTOSAR, ROS, OPRoS, etc.)
50Mobile&Smart@KMU
r Week 10§ N-screen
2012-10-20
26
51Mobile&Smart@KMU
r Week 10§ Robot application development using OPRoS
52Mobile&Smart@KMU
r Week 10§ Cross platform
• Visual tool - Verivo
No access to Code
AppStudio(Window PC) AppServer
Createcross-device Native apps
iOS Android WM, etc.
App
Visual Development
tool
App App
Abstraction Layer
Abstraction Layer
Abstraction Layer
2012-10-20
27
53Mobile&Smart@KMU
r Week 10§ HTML 5 related issues
§ B2G and WAC/Tizen
Platform/OS
Browser
HTML5 / Web App
Web OS (B2G)
HTML5 / Web App
Platform/OS
PhoneGapApplication
(HTML5+Platform API Wrapper)
54Mobile&Smart@KMU
r Project schedule § 20 phones from SK Planet § 2 robots from KITECH
What to do Due date
Team Making teams(2-3 students per team) Week 1-3
Project plan Detailed project planFinal project plan (Week 5) Week 3-5
Development & discussion Weekly reports Week 3-10
Final report and presentation
Final reports-paper and app report (Week 10) Final presentation (Week 11) Week 10-11
Showcase Showcase and presentation Week 11
2012-10-20
28
55Mobile&Smart@KMU
r Tentative projects § Remote control of robots using smartphone§ Cell-bot / Cloud robotics § Black box application in automobile § Character recognition using smartphone§ AR(Augmented Reality) application using NYAR or AndAR tool kit § LBS application in UCI § LBS application in UCI using AR § Mobile album – connect UCI § Send my firework ! § App analyzer § Control My PC
56Mobile&Smart@KMU
r Tentative projects
Project Remote control of robots using smartphone
Examples
<Kookmin Univ.> <Keio Univ.> <Univ. of West of Eng.>
Description
Control the robot using smartphone Bluetooth / Wi-Fi / WCDMA connection N-screen between robots and smartphonesE-learning using robot for childrenOPRoS / ROS platform
Sample Applications
iRobiQ, KMU/Yujin robotHexapod Robot, University of the West of England Walky, Keio Univ.
2012-10-20
29
57Mobile&Smart@KMU
r Tentative projects
Project Augmented Reality using AR toolkits
Examples
<Popcode, U. Cambridge>
DescriptionAR applications using AR toolkitsNyAR, AndAR tool kits On/off-line applications using AR engine and OpenGL
Sample Applications
NyARAndARPopcode
58Mobile&Smart@KMU
r Project results Team name Theme Categories
Let’s Location-based social networking service LBS, SNS
Wifi Flash drive File sharing service between smartphone, PC and servers Network, File sharing
RoboKontrol Interaction app with OPRoS based robot Remote control, RobotGesture Recognition
Scheduleshare Schedule sharing between users with ‘BUMP’ APIs Schedule sharing
Gyaan Quiz taking app with SNS characteristics SNS, Game, Quiz
GBC (Ginger Bread Car) app interaction with Android car using OpenCV Robot, Pattern recognition
ShuttleScheduler UCI bus system app providing bus/walking navigation at UCI LBS, Navigation
BooksInteractive AR app for child books using Qualcomm Vuforia AR, Learning
Dungeon Master SNS based game which simulates combat between players and monsters Game, SNS
PhotoGam SNS based photo editing app SNS, Photo
KOBOT Remote control of an OPRoS based robot Remote control, RobotIFM (IntelligentFood Monitor)
A reminder app that tracks and monitors your groceries and diet
u-health, -Barcode Scanner
2012-10-20
30
59Mobile&Smart@KMU
r BooksInteractive§ Augmented reality based interactive book
60Mobile&Smart@KMU
r GBC§ Android phone based autonomous car
2012-10-20
31
61Mobile&Smart@KMU
r ShuttleScheduler§ Anteater Express shuttle system
62Mobile&Smart@KMU
r ScheduleShare§ Schedule trading by ‘bumping’
2012-10-20
32
63Mobile&Smart@KMU
r KOBOT§ OPRoS based robot § Interaction with smartphone
64Mobile&Smart@KMU
r Gyaan§ SNS based quiz app
2012-10-20
33
65Mobile&Smart@KMU
r Wi-Fi Flash Drive § File transfer app for Android phone
66Mobile&Smart@KMU
r AMASE@UCI§ Sponsored by
• SK Planet, KITECH, Google, Samsung, D&CI
2012-10-20
34
67Mobile&Smart@KMU
r Evaluation§ Total score
§ Project rubrics
Score
Homework 40
Project 60
Total 100
Percentage ScoreCreativity 20 12
Market 30 18Technology 30 18
Usability 20 12Total 100 60
68Mobile&Smart@KMUMobile Embedded Systems Lab
2012-10-20
35
69Mobile&Smart@KMU
r Students’ feedback § Received excellent feedback from the students
• students’ final reports• lecture management system
r Main concern of the students § Burden to the homework
• 2-3 times more than other courses.• Main reason
üLots of examples üNo official lab hours
70Mobile&Smart@KMU
r Towards future lectures
ExerciseDay 3
LectureDay 3
LectureDay 1
LectureDay 1
LectureDay 2
LectureDay 2
Week 1 -
Week 8
Week 9 &
Week 10
Lecture (Day 2)
LectureDay 3
Lecture(Day 1)
LectureDay 1
Lecture(Day ½)
LectureDay 2
Week 1 -
Week 8
Week 9 &
Week 10
Exercise(Day 3)
Presentation for the project
Lecture hours
Lecturehours
Official lab hours
UnofficialLab hours
unofficial lab hours
MESA lecture Future course design
2012-10-20
36
71Mobile&Smart@KMU
r Towards future lectures § Additional items
• More explanation for the Linux and Android platform itself • New open platforms and API sets such as Vuforia, Samsung
Galaxy Note APIs, SNS APIs, etc.• More examples for the cross-platform issues(HTML5, PhoneGap,
B2G, etc.)• More examples for advanced apps• Examples for mechanical platforms (ROS, OPRoS, etc.)
72Mobile&Smart@KMUMobile Embedded Systems Lab
2012-10-20
37
73Mobile&Smart@KMU
r Notes from the recent trend§ Increasing importance of SW platform and application § Platform similarity between industries § App market as a ‘mass market’
r Tradeoff between students and university § Advanced course within 1 quarter
r Advanced course design § Increase interest of the students for embedded systems using
Android § Hope to be a good examples for the course design in mobile
embedded system § Hope to be a good pedagogical path to reinforce and stimulate
interest in an Embedded Systems curriculum.
74Mobile&Smart@KMU
Thank you !
Special thanks to SK Planet, Google, KITECH, Samsung and D&CI
‘Mobile Embedded Software with Android: Course Design and Experiences,’CECS Technical report (http://www.cecs.uci.edu/technical_report/TR%2012-10.pdf)
