Pervasive PixelsPervasive Pixels
Henning Schulzrinne (PI)Henning Schulzrinne (PI)Steven K. FeinerSteven K. Feiner
Gail KaiserGail KaiserJohn KenderJohn Kender
Kathleen McKeownKathleen McKeown
Department of Computer ScienceDepartment of Computer ScienceColumbia UniversityColumbia University
Proposed ResearchProposed Research
Goal: seamless mobile multimedia collaboration Goal: seamless mobile multimedia collaboration across distanceacross distance
Integrate advances across fieldsIntegrate advances across fields Collaborative workCollaborative work Graphical and visual interfacesGraphical and visual interfaces Spoken language understanding and generationSpoken language understanding and generation Vision sensing and understandingVision sensing and understanding Networking and securityNetworking and security
ContributionsContributions
Contextual information managementContextual information management use workflow to determine display contentuse workflow to determine display content multimedia summaries of past and present sessionsmultimedia summaries of past and present sessions
Harmonizing physical and virtual environmentsHarmonizing physical and virtual environments map changing virtual information onto physical displaysmap changing virtual information onto physical displays map layout of physical environment onto virtual spacemap layout of physical environment onto virtual space
Network servicesNetwork services clear, flexible interface to common servicesclear, flexible interface to common services authentication and privacy supportauthentication and privacy support operating system infrastructure for large displaysoperating system infrastructure for large displays
Features of Research InfrastructureFeatures of Research Infrastructure
Large, instrumented multi-display Large, instrumented multi-display workspacesworkspaces
Networked mobile devices of various Networked mobile devices of various capabilitiescapabilities
Transparent and automatic adaptability to Transparent and automatic adaptability to changes of place, platform or groupchanges of place, platform or group
Support for a wide range of hardware and Support for a wide range of hardware and software, from commercial to novelsoftware, from commercial to novel
Pervasive Pixels Teams
HarmonizingPhysical and VirtualEnvironments
NetworkServices
Project Leader
Schulzrinne
ContextualizedInformationManagement
Enabling communicationsSchulzrinne
DatabaseRoss
Summarization
Kender, McKeown
Environment ManagementFeiner
Capture and DisplayNayar, Kender
3D ModelingAllen
WorkflowKaiser
Virtual DisplayNieh
Access and PrivacyKeromytis
Pervasive Pixels Teams
HarmonizingPhysical and VirtualEnvironments
NetworkServices
Project Leader
Schulzrinne
ContextualizedInformationManagement
Enabling communicationsSchulzrinne
DatabaseRoss
Summarization
Kender, McKeown
Environment ManagementFeiner
Capture and DisplayNayar, Kender
3D ModelingAllen
WorkflowKaiser
Virtual DisplayNieh
Access and PrivacyKeromytis
Pervasive Pixels Teams
HarmonizingPhysical and VirtualEnvironments
NetworkServices
Project Leader
Schulzrinne
ContextualizedInformationManagement
Enabling communicationsSchulzrinne
DatabaseRoss
Summarization
Kender, McKeown
Environment ManagementFeiner
Capture and DisplayNayar, Kender
3D ModelingAllen
WorkflowKaiser
Virtual DisplayNieh
Access and PrivacyKeromytis
Pervasive Pixels Teams
HarmonizingPhysical and VirtualEnvironments
NetworkServices
Project Leader
Schulzrinne
ContextualizedInformationManagement
Enabling communicationsSchulzrinne
DatabaseRoss
Summarization
Kender, McKeown
Environment ManagementFeiner
Capture and DisplayNayar, Kender
3D ModelingAllen
WorkflowKaiser
Virtual DisplayNieh
Access and PrivacyKeromytis
Pervasive Pixels Teams
HarmonizingPhysical and VirtualEnvironments
NetworkServices
Project Leader
Schulzrinne
ContextualizedInformationManagement
Enabling communicationsSchulzrinne
DatabaseRoss
Summarization
Kender, McKeown
Environment ManagementFeiner
Capture and DisplayNayar, Kender
3D ModelingAllen
WorkflowKaiser
Virtual DisplayNieh
Access and PrivacyKeromytis
The Department of Computer ScienceThe Department of Computer Science
A growing facultyA growing faculty 5 new assistant professors (pictures and areas here)5 new assistant professors (pictures and areas here) 1 new full professor: Peter Belhumeur1 new full professor: Peter Belhumeur 1 new full professor 9/02: Julia Hirschberg1 new full professor 9/02: Julia Hirschberg
A large student populationA large student population 80 PhDs80 PhDs 155 MS155 MS 380 undergraduate majors380 undergraduate majors
Internal, fluid collaborationInternal, fluid collaboration Interdisciplinary connectionsInterdisciplinary connections
Building Critical MassBuilding Critical Mass
NetworkingNetworking Schulzrinne, Misra, YeminiSchulzrinne, Misra, Yemini links to EElinks to EE
VisionVision Allen, Belhumeur, Kender, NayarAllen, Belhumeur, Kender, Nayar links to computer graphicslinks to computer graphics
Human Computer InteractionHuman Computer Interaction graphical and visual user interfaces: Feiner, Kendergraphical and visual user interfaces: Feiner, Kender spoken and written language: Hirschberg, McKeownspoken and written language: Hirschberg, McKeown links to speech and multimedia in EElinks to speech and multimedia in EE
Areas of GrowthAreas of Growth
Security and systemsSecurity and systems Computer engineeringComputer engineering Computer graphicsComputer graphics TheoryTheory Sofware systems (e.g., programming Sofware systems (e.g., programming
languages)languages)
Administrative SupportAdministrative Support
Institutional cost share on this proposal: $1 Institutional cost share on this proposal: $1 million in totalmillion in total
$100,000/year for 5 years (Provost)$100,000/year for 5 years (Provost) $100,000/year for 5 years (Dean)$100,000/year for 5 years (Dean)
Support for growth: slots, spaceSupport for growth: slots, space Financial supportFinancial support
return in revenue (e.g., Columbia Video Network)return in revenue (e.g., Columbia Video Network)
Technical transfer supportTechnical transfer support dedication of Columbia Innovative Enterprise servicesdedication of Columbia Innovative Enterprise services
Current Departmental FacilitiesCurrent Departmental Facilities
Shared infrastructure of over 20 Sun and Pentium Shared infrastructure of over 20 Sun and Pentium serversservers
Two teaching laboratoriesTwo teaching laboratories 40 seat student lab (CLIC) of Intel Pentium PCs, dual 40 seat student lab (CLIC) of Intel Pentium PCs, dual
projection capability, wireless microphone, video mixer projection capability, wireless microphone, video mixer and multiple camerasand multiple cameras
30 seat Microsoft Research Lab30 seat Microsoft Research Lab
Nortel Meridian telephone switchNortel Meridian telephone switch Research infrastructure of > 500 nodes plus serversResearch infrastructure of > 500 nodes plus servers Staffed by 4 professional system administratorsStaffed by 4 professional system administrators
Research InfrastructureResearch Infrastructure
User-based personalization: user locationUser-based personalization: user location magnetic card readersmagnetic card readers active badge systemactive badge system triangulation on mobile devicestriangulation on mobile devices visual trackingvisual tracking
Stationary setupsStationary setups Seminar roomSeminar room 3 meeting rooms3 meeting rooms 12 faculty offices12 faculty offices 5 public areas5 public areas portable unitsportable units
Seminar roomSeminar room
High-resolution DV video High-resolution DV video camera, 2 pan-tilt speaker camera, 2 pan-tilt speaker cameras, Omnicam cameras, Omnicam omnidrectional audience omnidrectional audience camera, video switcher, 16 camera, video switcher, 16 ceiling mounted ceiling mounted microphones, electronic microphones, electronic whiteboard, 3 XGA high-whiteboard, 3 XGA high-brightness video projectorsbrightness video projectors
Meeting RoomMeeting Room
2 remote-2 remote-controlled pan-tilt controlled pan-tilt video cameras, video cameras, Omnicam, Omnicam, conference table conference table PZM boundary PZM boundary microphones, microphones, automatic audio automatic audio mixer, ceiling mixer, ceiling speakers, 3 high-speakers, 3 high-lumen XGA lumen XGA
video projectorsvideo projectors
Faculty OfficeFaculty Office
Mimio electronic whiteboard, XGA video projector, Mimio electronic whiteboard, XGA video projector, Ethernet speaker phone, wal-mounted pan-tilt video Ethernet speaker phone, wal-mounted pan-tilt video camera, PocketPCscamera, PocketPCs
Public areasPublic areas
Walk-by stations and laboratoriesWalk-by stations and laboratories XGA video projector, steerable ceiling-mounted dome XGA video projector, steerable ceiling-mounted dome
video camera, embedded ceiling mounted netowrked video camera, embedded ceiling mounted netowrked Pentium computer, celing speaker, Andrea DA-400 array Pentium computer, celing speaker, Andrea DA-400 array microphone, Microtouch IBID 600 whiteboard.microphone, Microtouch IBID 600 whiteboard.
Picture herePicture here
Design for walk-by collaborationDesign for walk-by collaboration
Figure 1 hereFigure 1 here
FunctionalitiesFunctionalities
ConferencingConferencing Internet conferencing server to mis IP and PSTN audio streamsInternet conferencing server to mis IP and PSTN audio streams
Interconnection with analog phoneInterconnection with analog phone Digital hybrid connects digital or analog sound to existing telephone Digital hybrid connects digital or analog sound to existing telephone
system in classroomsystem in classroom Netowrk voice-over-IP interface attached to Nortel Meridian PBX for 20 Netowrk voice-over-IP interface attached to Nortel Meridian PBX for 20
simultaneous conversationssimultaneous conversations
Multi-processor servers and IA64 compute and database server, Multi-processor servers and IA64 compute and database server, plus othersplus others
File storageFile storage Face, speaker and fingerprint recognitionFace, speaker and fingerprint recognition 2 printers and tape library system in year 4 for backup2 printers and tape library system in year 4 for backup
Remaining AgendaRemaining Agenda
Research OverviewResearch Overview Contextualized Information Management (Kaiser)Contextualized Information Management (Kaiser) Harmonizing Physical and Virtual (Feiner)Harmonizing Physical and Virtual (Feiner) Network Services (Schulzrinne)Network Services (Schulzrinne)
Genomics testbed (Noble)Genomics testbed (Noble) Responding to reviews (Kender)Responding to reviews (Kender) Telephony demo (Schulzrinne)Telephony demo (Schulzrinne) LunchLunch DemosDemos Visit with administratorsVisit with administrators DemosDemos Small group meetingsSmall group meetings