Public Display Applications
Mestrado Integrado em Engenharia InformáticaProjecto Integrador
Pedro Albuquerque SantosRui Neves MadeiraProf. Nuno Correia
March 19th, 2014Monte de Caparica, Almada, Portugal
2
Why am I here?
● I have finished my Masters last December● During my dissertation I developed a public display
application– I may have some experience to share with you about the
system I have built
– I can give you some tips and answer some questions you may have about this kind of system
● I will present the system I have developed– Pointing out any similarities that it has with CampusTV
– After that, I am open to discuss any doubts you may have
3
Motivation
● Testing the personalization framework that was the main focus of my Masters' dissertation
● But also the realization that:– We live surrounded by mobile devices
● e.g., smartphones, tablets, notebooks
– While large situated displays are also present● e.g., TVs, computer screens, advertising and information displays
● Such displays can be leveraged to improve the user experience and to develop and improve interaction mechanisms
4
Motivation
● There is a growing body of research around the area of public displays– It remains a complex and system-specific effort
– Strong focus on advertising content
● There is an opportunity to leverage the ubiquity of personal devices and public displays– Provide useful and personalized information to users
– Adding real-time interactivity to increase the system usefulness
● Which is not a major requirement of CampusTV
5
Related Work
● Benefits of large displays – e.g., peripheral awareness, productivity, immersiveness
● How people interact with public displays● Interaction techniques, input sources and ways to abstract them
– Bluetooth scanning has been used to sense user presence and to allow some degree of interaction, due to its low entry barrier
– Other input sources include SMS and e-mail
● There has also been some focus on group-based personalization and content broadcasting based on user preferences
6
Related Work
M. Finke, A. Tang, R. Leung, and M. Blackstock. “Lessons Learned: Game Design for Large Public Displays”. In: Proceedings of the 3rd International Conference on Digital Interactive Media in Entertainment and Arts. DIMEA ’08. ACM, 2008, pp. 26–33.
N. Kaviani, M. Finke, S. Fels, R. Lea, and H. Wang. “What Goes Where?: Designing Interactive Large Public Display Applications for Mobile Device Interaction”. In: Proceedings of the First International Conference on Internet Multimedia Computing and Service. ICIMCS ’09. ACM, 2009, pp. 129–138.
7
With that in mind...
● FCT4U was born– (Semi-)public situated display based system
– Provides useful and contextual information to people nearby in an university campus:
● e.g., students, teachers, researchers, etc.● Any similarity with CampusTV is pure coincidence
● A personalization approach to enhance the user experience was applied to it
● A user study to assess and validate the system's purpose and interaction modes was conducted at CITI
● It was presented at the UIC-2013 last December
8
Principles & Goals
● Situated displays can be:– Public, shared, interactive or proactive/personalized
● FCT4U combines all of these categories● It aims to:
– Provide information to users in the surrounding space while sharing the public display between up to 3 users
– Expand information, by using mobile and public displays simultaneously
– Promote sharing and interaction among users (e.g., short messages and content sharing)
● CampusTV's public displays are mainly public and shared, with less focus on interactiveness and proactiveness/personalization
9
FCT4U System
● The system is composed of:– Mobile application that runs on smartphones and tablets
● Android 2.3.3+ was chosen as the mobile platform due to its current market share
– Application running on a situated display● JavaFX-based application to support multiple platforms
– This was also where the “content manager” was placed for simplicity(the prototype used a single public display)
● JPA for data storage, using MySQL as the back-end
– The applications share the same web-based UI● Built with HTML5, JavaScript and CSS● It offers re-usability, portability and adaptability to different form factors
10
FCT4U System
● Bluetooth was used to sense users' presence– It offers a cheap and easy way of sensing the proximity of other
devices/users
– BlueCove (JSR-82 implementation) is used to implement a presence sensing routine
● There were attempts to use the RSSI to determine the approximate distance from the display
– It was the public display that sensed the mobile devices around● For CampusTV, I advise you to do the opposite so that you do not have to
register each user's Bluetooth MAC Address– As in: T. Kubitza, S. Clinch, N. Davies, and M. Langheinrich. “Using Mobile Devices to
Personalize Pervasive Displays”. In: SIGMOBILE Mob. Comput. Commun. Rev. 16.4 (2013), pp. 26–27.
11
FCT4U System
● Communication between the public display and mobile devices is done through the WebSocket protocol– It allows bi-directional communication, while bypassing common network
restrictions● A central accessible node that works as a commnication hub is needed● For that, a Java-based framework was used: https://github.com/Atmosphere
– It supports end-to-end encryption when run over TLS
– It can be used by pure web applications running on modern browsers● Some older browsers, such as Android's WebView, do not support WebSockets.● In such cases, a native application is needed to deal with WebSockets connections.
– A good option for CampusTV if you WANT to add bi-directional communication between the public displays and mobile devices
● e.g., allowing real-time interaction and/or pushing data among devices
12
Architecture
P2MUCA means Personalization Platform for Multimodal Ubiquitous Computing Applications
● For CampusTV you would REALLY need to decouple the “FCT4U Engine” from the “Public Display Application”– Multiple public displays would communicate with the FCT4U Engine through some
kind of mechanism (e.g., Java RMI, WebServices, REST, WebSockets, etc.)
– The messages sent by the mobile devices through the communication hub, would then be delivered to the “FCT4U Engine” or to the corresponding “Public Display Application”, depending on where (certain?) features are implemented
13
P2MUCA OAuth 2.0
14
Test Intallation
15
User InterfaceMobile Application
Public Display Application
16
Widgets
17
Widgets
18
Widgets
19
Mobile Application
● It extends the public display– Allows users to view information while away from the public display
● The latest information received from the display is cached
– Provides a safe place to show privacy sensitive information and to set preferences (including connecting to Facebook)
– Works as an input device
● It is used to associate the mobile device's Bluetooth MAC address with a certain user– On FCT4U the display reacted to the presence of the users, not the
other way around
– This is not needed if the mobile devices are the ones to scan for the public displays, with MAC addresses known beforehand
20
Messaging & Facebook
21
(Optional) Location Tracking
● It allows us to automatically determine where someone lives, works and lunches– Density Clustering (DBSCAN algorithm)
● Geo-fencing of public transport hubs is used to determine the preferred transportation mode
22
Personalization
● P2MUCA– Major focus of my MSc dissertation and also part of
Rui Neves Madeira's PhD
– Functionalities are exposed through an OAuth 2.0 protected HTTP API and a companion website is available for configuration
– Uses a generic personalization model● It can be configured through an XML configuration file or
a web-based GUI
23
Personalization Model
● Resources represent interaction streams, preferences, options or contextual information
● They can be aggregated into parameters using formulas– e.g., 0.2*messagesRead+0.3*messagesReceived+0.5*messagesSent
● Personalizations define multiple options from the combination of multiple parameters
● The framework also supports context segmentation● If you WANT to have some personalization on CampusTV, you may follow
similar ideas in a more system-specific effort
24
User Study
● A user study was conducted at CITI– The main focus was to evaluate if the idea of
system like FCT4U was well accepted
– Comparing different system interaction modes was also a major concern
– Personalization evaluation was not taken into consideration
● It was a short term test to present the system to potential users
25
User Study
● The public display application was deployed on a 46” touch-screen● Multiple Android devices were used for the mobile application● Participants went trough 3 scenarios
– Passive, mobile application interaction and touch-screen interaction
– A questionnaire was answered after each scenario
● 9 test subjects who work at CITI– Aged between 26 to 39 years (average of 30.5)
– 2 females
– 8 PhD students and a post-PhD researcher
– 2/3 have mobile devices have Android 2.3.3+ devices
26
User Study Results
● 7 out of 9 users gave positive scores to the system's usefulness (median of 6)
● They also thought that system could save them time and that it would be even better if was already tailored for their needs
● Passive Mode– Answers were mostly positive
● System responsiveness was considered adequate by most users● The information provided by the passive mode was considered just above
adequate (median of 5)
– The first widgets were the ones seen that users saw(i.e., Map, Weather, Lunch Menus)
27
User Study Results
● Mobile Application Interaction– It gave users control of what is shown on screen
– It was considered useful and easy to use(answers with medians of 6)
– Answers related with ease of use, user effort and recovering from mistakes got medians of 5 or 6
– But, in general, they considered the application pleasant to use and quick to master(medians of 6 and 7)
28
User Study Results
● Mobile Application Specific Features– Regarding the short messages services the
reactions were mixed
– Users also did not consider the system very collaborative, even though some recognize its potential to evolve in that direction
– Those are two areas to improve in the future
29
User Study Results
● Touch Screen Interaction– It was not as well accepted as the mobile application,
getting slightly lower scores on the same questions● It may be due to people not being used to use such a large touch-
screen in an upright position● Maybe they did not see an advantage over the mobile application● Nevertheless, 7 out of 9 users considered this mode easy to use
– Also, it is not practical when the application is not deployed within arms reach
● e.g., the displays recently installed on “DI's” building
30
User Study Results
● Users preferred to use the public display alone– But they were able to cope with sharing it
● System UI was rated with a a mean of 5.33– There were no negative scores, but most (6 out of
9) gave neutral or just slightly positive scores● This showed us that there is still room for improvement
– Other more specific metrics had higher scores
31
User Study Results
● Widgets– Favorite widget: Lunch Menus
– High rated widgets: Map and Weather
– Lower rated widgets: Messages and News
● Users preferred the passive mode– All scenarios had similar scores
● But most users said that they would use the passive mode and one of the other active interaction modes
– Personalization should increase their satisfaction
32
User Study Results
● Privacy Sensitive features– 7 out of 9 users were apprehensive about
● Location tracking● Bluetooth visibility
– If mobile devices are the ones that scan for displays this is mitigated
– Users want control over the information they share● But are interested in smart features● A compromise must be found
33
Conclusions and Future Work
● This presentation followed the development of FCT4U:– A (semi-)public situated display based system that proactively shows
information to nearby users in an university campus, while providing interactivity through a mobile application
– Some of FCT4U's design decisions and architecture may be used as the basis for similar systems (e.g., CampusTV)
– The findings from the user study gave us some insight about the user receptiveness to this type of systems
● It would be interesting to see how users will react to your systems
● A long-term user study must be conducted to help fine tune the personalization model configuration
● In general, we should continue to add features, improve existing ones and explore new ideas
34
Thank you for your attention
Contact: [email protected]
Download Presentation
Anything else you want to discuss?
