20
August 29, 2017 Sam Siewert CS415 Human Computer Interaction Lecture 1 – Introduction Part-2
August 29, 2017 Sam Siewert
CS415Human Computer Interaction
Lecture 1 – IntroductionPart-2
Questions on Ex #1?Posted on Canvas - Ex #1
Turn in on Canvas Next Week Friday
Focus is on 1D CLI - Not as Easy as it Sounds!
Start with Mini-Shell Example code (written as graduate student for undergraduate OS classes)
Make a “friendly” shell - add some features!
Why Did Microsoft Release powershell in 2006 and Open Source it in 2016?
Sam Siewert 2
NUC/Jetson Linux - Go To Lab, Verify LoginOption #1 – Use King 122 NUC / Jetson Lab– Learn Host and Embedded
Linux– Getting Started– Jetson - Used for Self-Driving
Cars– NUC - Way Cool!ssh -X [email protected]
– [from Host 121]
Option #2 – Virtual-Box Linux with Ubuntu 16.04 LTS– Use Windows or Macintosh
PC– Learn Linux, it’s Easy– Ryan Sutton has SE VM
Image you can Use!!
Sam Siewert 3
Questions on Lab Resources?B72-122-NUC1 [192.168.122.111] => B72-TK1-112 [192.168.122.112]
…
B72-122-NUC9 [192.168.122.191] => B72-TK1-192 [192.168.122.192]
Sam Siewert 4
Left Side - #1 to #4
#5 & #6
Left Side - #7 to #9
Downloading and Copying FilesGetting Files to NUC– Option #1 - Bring files to Lab
on USB Memory Stick, Plug into NUC (or Jetson)
– Option #2 - Download files from NUC Web browser (right click, Save-Link-as)
– Option #3 - Put files on a Shared “P:” Windows drive and access from NUC
Getting Files from NUC to Jetson (Testing)– Use “Connect to Server” from
“Places” and use SSH to Jetson IP, using “ubuntu”, “ubuntu”
– Drag and drop files from NUC folder to Jetson folder
Sam Siewert 5
Now Drag and Drop Files on NUC Desktop
Transfer files to/from NUCTake code with you on memory stick or copy to P: drive before you leave (backed up)Remove test files from JetsonCompile code native on Jetson (like PRClab)Possible to cross-compile, cross-debug if you want with NVIDIA tools
Sam Siewert 6
The Basic ElementsHuman – Models of Perception and Cognition
– Brain Models [Simulations]– Brain Scans [fMRI, EEG, fNIRS]– Brain Processing of Sensory Information [Perception]
Computer – Theory of Computation and Devices– What Can Be Computed? What Can’t? How Fast?– Traditional ALU, Stored Program Computer: Von Neumann
Architecture, Von Neumann Bottleneck– Turing Machines [Wikipedia], Church-Turing Thesis
[Wikipedia], Turing Test– Hypercomputation (Myth, Reality?)– Quantum Computing, D-Wave, ANN, Analog “Real”
Computing, Neuromorphic, Deep Learning (GPU)
Interaction – Physical Devices and Interfaces– Physics, Time and Space, Chemistry– Acoustics, Haptics, Optics, Olfaction, Gustation– 5 Senses – Hear, Feel, See, Smell, Taste– Fusion of Senses – Proprioception [Body movement]– Extended – Chronoception [Time], Nocioception [Pain]– 20+ Senses: Internal [interoceptive], External [exteroceptive]– Superhuman Sensing [E.g. Thermal imaging, Magnetic field]
Sam Siewert 7
Why is Human Vision > Computer?
Sam Siewert 8
Approximately 100+ Mega-Pixel(Rod/Cone Count)
James Cutting & Peter Vishton –Perceiving Layout
Cortex=10 Billion Neurons(High fan-out)
Total=100 Billion Neurons
Neuroscience. 2nd edition.Purves D, Augustine GJ, Fitzpatrick D, et al., editors.Sunderland (MA): Sinauer Associates; 2001.http://www.ncbi.nlm.nih.gov/books/NBK10848/
Red Epic 64563 Mega-Pixel
I/O Bus (x16 5Gbps = 8GB/sec)
CameraLink
Interface Card
Local Bus
CPU CPU
MemoryController
5 To 10 billion transistors1. Neuron > Transistor2. Better Programming? ROM?3. More Richly Interconnected4. Storage + Processing
> 1 Trillion Synapses
http://bluebrain.epfl.ch/Human Brain ModelsProject StatusVisualization
Biological Vision vs. Machine Vision(Why A Honey Bee is Better than HPC for CV)
Humans - 100 million Photoreceptors
– 10 billion Neurons (Cerebral Cortex)– Brain with 100 billion Neurons– Millisecond Transfer – Massively Parallel Analog + Digital Computation
Synapse Match is a Challenge– 7000 Connections from 10 Billion Neurons– 3 Year Olds Have 1015 Synapses
CPU to Digital Camera/HDD– Connects 10’s of millions of pixels – to Several Billion transistors – Through Sequential Logic and I/O Bus
Sam Siewert 9
960K Neurons in flight:Learns locations,complex odors,colors, and shapes;with high efficiency (500 Watt/Kg), 0.218g
Brain plasticity for learning, connectedness, concurrency, integrated sensing, power efficiency, and resiliency
2016 – 16 billion?
NVIDIA GK11028nm, (7.1 billion)
Pascal – 15 billion
Intel MICA 22nm(5 billion)
http://en.wikipedia.org/wiki/List_of_animals_by_number_of_neurons
Moore’s Law Graphic - Wikipedia
3D PerceptionWe Live in an Animated 3D World
We still Compute with 2D Desktops (WIMP)– Tablets, MS Surface, Notebooks, Laptops,
Desktops– Mobile SmartPhones, iWatch, Android Gear– Enabled by Key Devices (Pointer – Mouse,
Stylus, Touch-screen, Raster Graphics – CRT, LCD, OLED, VR CAVE)
3D Limited to CAD, VR Worlds, Movies, Visualization So Far …
3D Batch Processing – Printers and Scanners
What About Interactive 3D?
Sam Siewert 10
https://en.wikipedia.org/wiki/OLED
https://en.wikipedia.org/wiki/3D_film
1950’s
Real World 3D PerceptionMany Visual Cues Provide 3D Perception – Open Research (Perceiving Layout)
HCI Inventors Want to Exploit What Matters Most– Stereopsis and Disparity (Left Eye and Right Eye View)– Accommodation (Near and Far Field)– Correspondence (Common Features Noted with Each Eye)
Sam Siewert 11
http://www.ibm.com/developerworks/library/bd-mdasecurity/, Sam Siewert
“Play” with RGB Depth MappersScene Depth Mapping + RGB Planar Image– Scene Segmentation (in Z as well as XY)– Skeletonization– Hand Tracking and Gesture Recognition– Interaction with 3D Virtual Worlds
Sam Siewert 12
Data Glove vs. RGB Depth MapperHand and Finger Tracking Based on SegmentationDepth (Skeletonization)Processing is Somewhat IntenseNothing to Wear
Data GloveRotational Rates, Accelerometer, Finger Flexure Sensors
Sam Siewert 13
CreativeCam With Intel RealSense
3D SDK
M.S. research, CU Boulder,“A Hybrid Sign Language Recognition System”, Van Culver, 2004OpenCV and Monovision Camera with 5DT Data Glove
Interactive 3D Devices I/O?3D Input – Active Depth Mappers, Passive Stereopsis– ASUS Xtion Interactive Depth Mapper– PrimeSense Chip [Apple now] – Revitalized– Many Competing Chips and Devices Now– Passive – Compute Intensive, Less Robust
(Flat Wall Problem)– Time-of-Flight, LIDAR (Distance, Outdoor)
3D Output [Left, Right Eye Disparity]– 120 Hz, Left / Right Eye Parallax– Filtering, Polarization (Left/Right Eye)– Active Shutter Glasses– Entertainment and Visualization Success,
Interactive, Less So … Sam Siewert 14
“Fool the Eyes!”
3D Alternatives?Augmented Reality (Interactive Mix of Real/Virtual)
1. Project Virtual World onto Real World2. View Real World Through Goggles with Annotation3. Project Virtual Images into retina as Light-field Overlay
Heads-Up-Display – Aviation [Gunsight Generalization]
Pranav Mistry, MIT Media Lab, Samsung– Project onto Real World– We’ll Study Later
Google Glass VRD – Failed, “Glassholes”
MS Hololens [View Real World with Annotation]
Magic Leap [VRD] – MIT Tech Review Article
Oculus Rift – VR Headset [Dual OLED, Head Tracking]
Wearable AR/VR Doomed? Limited? Unhealthy?
Sam Siewert 15
C130 HUD
Before We Go FurtherStart with Basics - 1D CLI
Move to 2D WIMP
Then 3D Animation, AR and VR
Interaction Theory and Guidelines
Cognition and Perception Models
Then, Back to the Future! Sam Siewert 16
August 24, 2004 Sam Siewert
Linux Skills
Introduction SessionPart-2
Linux How-ToUbuntu LTS 14.04 on Virtual Box and Jetson– Local How-To Resources
LinuxLinux-Development-Getting-Started.pdfLinux-Basic-Makefile-by-Example.pdfLinux-Setting-up-VBOXSF-Automount-from-Host.pdfLinux-Programming-Top-Errors.pdf
– https://ubuntu-manual.org/– https://www.youtube.com/watch?v=SGGj3hc
6R8Q
Sam Siewert 18
Virtual Box VM Installation ofa New OS Like Ubuntu Linuxis a Safe Way to Learn on thePlatform you Know Best
Server/Desktop, VM, EmbeddedA Key Advantage of Ubuntu is that One Linux Distribution Can be Used for Server/Desktop, VM and Embedded Use– Server: PRCLab (prclab.pr.erau.edu)– VM: Virtual Box with Ubuntu 14.04
LTS– Embedded: Jetson TK1 Developer
System
Linux Kernel supports the NDK Layer in the AOS (Android Operating System)– https://developer.android.com/ndk/inde
x.html– https://developer.android.com/sdk– https://developer.android.com/tools/sd
k/ndk/index.html
Sam Siewert 19
A few Comments on OptionsNative Linux (Jetson) – Best Performance for Interactive Applications
VM Linux – Nice Option for Convenient Development and Test
Server Linux – Safe Haven that is Highly Available, Backed-Up
Moving Code Can be Done with SFTP or SCP– SFTP Requires ZIP or Archive to Move Whole Directories– SCP – Moves Structure Automatically
Sam Siewert 20
