A Television News Graphical Layout Analysis Method Using Eye Tracking

Rui Rodrigues, Ana Veloso, Óscar Mealha Dept. of Communication and Art

University of Aveiro Aveiro, Portugal

ruirodrigues@ua.pt, aiv@ua.pt, oem@ua.pt

Abstract—This paper presents an analysis method used to evaluate the graphical layout of Television (TV) News. With the data gathered from an eye tracker, it is possible to discern viewers’ main focus of attention. However, in the case of video media, the visualisation tools of eye tracking software are insufficient for this discernment. This method uses eye tracking log files and combines them with algorithms developed using a spreadsheet application and visual data representation techniques. As a result of this combination, it is possible to identify the graphics which have greater visual attention, and subsequently, these representations supported the analysis of viewers’ main focus of attention (areas of interest). A case study was also applied for the validation of this method.

Keywords – Analysis Method, Television News, Eye Tracking, Graphical Layout, Visual Attention

I. INTRODUCTION TV News programs have seen significant changes over

the last few years. One of the most important changes can be found in the graphical layout, with the introduction of on-screen graphic elements (over the shoulders, tickers, lower thirds) during news broadcasts, creating new ways of transmitting information. This information reception context has increased in complexity such that visual stimuli are presented diversely in the visual space and in most situations, also simultaneous in time. These graphics are visual stimuli with content and have the intention to provide extra information or additional messages to complement the anchor and news video.

The research in this area is far from having convergent results and therefore, further investigation on TV News is necessary, specifically because graphics are a reality that will endure [1]. In this area, and according to Josephson & Holmes [2], the use of eye tracking can assume a very important role for the study of TV News, more specifically in the study of visual composition. The eye tracking technology allows the detection and registration of viewers’ eye movements [3], and in this context, eye tracking becomes a crucial technology to clarify the influence of visual composition. The visual composition considered in this study includes several elements such as ticker, over the shoulders, bugs, lower thirds and the anchor.

This paper is divided into five main sections. First, a theoretical framework that supports this study is presented, namely focusing eye tracking in TV News. Second the main

graphics used in TV News and there main function will be presented. Third, a systematization of the most common visualisation techniques used with eye tracking data. Fourth, the analysis method developed and demonstrated in a controlled study is reported. Fifth, conclusions are presented and future work is considered.

II. THEORETICAL FRAMEWORK During the last 30 years several authors have studied

TV news. Elements such as the attention as well as the news story messages were analysed in this genre of TV program [4-11]. However, with the progress of new technologies, as well as the development of digital scenography in TV news, the inclusion of eye tracking can be relevant in order to assess the experience and interaction of a user with a specific study object.

Still, few studies have been developed with the use of eye tracking in TV News. One of these studies was conducted in 2006 by Josephson & Holmes [12], with the aim of measuring the attention spent by participants in different areas of TV news (crawler, headline, title, globe and main area). This study presented three versions of TV news with: i) video and audio redundancy without textual content; ii) video and audio redundancy and unrelated textual content (crawler) and iii) video and audio redundancy with related and unrelated textual content (crawler and headline).

Matsukawa, Miyata and Ueda [13] conducted a study using eye tracking technology, which was aimed to understand the use of graphics in TV news and information redundancy. The study was divided into two moments: i) the first moment intended for viewers to watch a TV news broadcast with eye tracking technology; ii) the second moment consisted in understanding the degree to which viewers understood the content present in the TV news broadcast. This study was important to understand the relationship between graphics and reception of contents.

Another study developed in this area was the research project “Portuguese TV news scenography: from simplicity to profusion of news rooms studios” [14]. During this project, a preliminary experience was also developed with an eye tracker, which focused on the study of selective attention on TV News’ program of the Portuguese public television channel – RTP. The results obtained presented several indicators and evidence of the unexplored field of TV

2012 16th International Conference on Information Visualisation

1550-6037/12 $26.00 © 2012 IEEE

DOI 10.1109/IV.2012.66

357

information reception. Another study was conducted by Marques [15]. This author developed a study where the primary goal was to evaluate the impact of scenography as well as the visual and auditory attention of viewers when viewing news in a TV context.

III. GRAPHICAL LAYOUT IN TV NEWS There are different graphics for different purposes.

However, all these graphics have the common intention of endowing TV news broadcasts with a distinctive and unique identity, establishing a better connection with the viewer. Fig. 1 highlights the main graphics present in TV News: Bugs (A), Over the Shoulders (B), Lower Thirds (C) and Ticker (D).

Figure 1. Graphical Layout (RTP)i

Regarding Bugs (Fig. 1A), this still graphic appears normally in the left superior corner of the screen, and is typically used to place the TV Station's logo. The term “Bugs” is applied to this graphic because during the news emission, the position of this graphic is always the same [16]. Usually, the clock is associated with "Bugs", but sometimes the clock can be associated to "lower thirds" or the "ticker", as is shown in Fig. 1.

The graphic element "Over the Shoulders" (Fig. 1B), normally appears in the news that has higher impact, therefore revealing the relevance of the news story. The term "Over the Shoulders" is applied because it generally appears in the superior part of the shoulder of the anchor [16]. In some countries it appears on the right side of the anchor (Portugal, Spain, France), and in another countries it appears on the left side of the anchor (Germany, Switzerland, Austria). The "Over the Shoulders" is usually accompanied with an image or icon, with the intention of establishing a link with the news story being presented.

Concerning the "Lower Thirds" graphic (Fig. 1C), this appears normally in the lower area of the screen, as the name suggests. The "Lower thirds" have a major importance in TV news broadcasts since they have the intention of highlighting the news story being presented. This news story can be detached by key words of short sentences with great impact. The "Lower Thirds" can be divided into three types, depending of the informative textual lines that are present: i) One-tier lower third; ii) Two-tier lower third; iii) Three-tier lower third [16]. This graphic is a motion image.

Finally, the “Ticker”, sometimes referred to as a crawler (Fig. 1D), usually appears at the bottom of the screen and has the intention of displaying less important news. The "Ticker" is increasingly used to show other types of textual content, such as the short messages sent by viewers as well as advertising information [16]. This graphic is also a motion textual image.

Lastly, there are other graphics that are part of TV news broadcast, but their occurrence is less frequent. There are the cases of notice boards that occupy the entire screen, as well as the generic and the end credits.

IV. EYE TRACKING VISUALIZATION TECHNIQUES Visual representation of data and information is of

utmost importance since it helps with the interpretation and comprehension of abstract data that is non-understandable [17]. In the context of eye tracking data visualisation, techniques have been developed. Fig. 2 represents some of the visualisation techniques developed for eye tracking context.

Figure 2. Representation of common Visualisation Techniques used with

Eye Trackingii

The main visualisation techniques that are represented in Fig. 2 are:

• Gaze Plot (A) – This visualisation technique consists in showing the sequence (and their order) of the eye movements of a subject. This data representation technique uses variation in circle size to indicate different fixation times. In other words, the larger the circle, the longer the fixation. The lines that connect the circles represent saccadic movements (rapid eye movements that occur between fixations). In this visualisation technique, it is also possible to see the path and the order of eye movements with a timeline. However it is only possible to visualize a single subject at a time.

• Heat Maps (B) – This visualisation technique consists in mapping with colour levels the main visualisation zones of a specific visual field. A heat map will also display elements of greater interest by colouring “hot” (red and orange colours) and “cold” (green colours) spots/areas. The "hotter" the

358

colours, the more visualised the arethis visualisation technique it is alstwo types of metrics: i) number oand ii) fixation time.

• Bee Swarm (C) – This visualiconsists in showing in the form ofwhere a user focuses. It can alsopresent the attention points of Different colours are used to distinof each subject.

• Clusters (D) – Finally, clusters coareas of interest for posterioconcentration of fixation points. Ttechnique can be applied either fogroup of subjects in non-temporal m

Although eye tracking technology sosome effective visualisation techniques, inthese techniques have limitations. It is amention that these limitations are related to tin this study (Tobii Studio). With this sopossible to use the main visualisation techanalysis, because this software can only pvariable that appears in video media with tvisualisation technique. Taking into accounlimitations, several authors developed technprovide other solutions. One of those caseby Raiha et al. [18]. These authors proposedwhere temporal information played a key rodata representation and visualisation. Theytechnique as "time plot of the gaze data" [18analysis of web search results listings useAnother example that attempts to explore eyimportant tool of information visualisation wFlag et al. [19]. These authors developedvisualisation system that adjusts to the specia subject.

V. METHOD The method reported in this paper app

media, e.g. audio-visual content, and has thgoal: Identify the graphic elements, presewhich have greater visual attention. With thto differentiate viewer’s main focus ofrelevance of this method is related to presented in the theoretical framework as welimitations in the analysis of graphical layoseveral studies with eye tracking and TV NMoreover, the visualisation tools of the eye tare insufficient for the discernment of viattention in graphical layout and therefore, to develop an analysis method for representemporal media. Summarily, this representaan algorithm that correlates eye tracker loanalysis data described in a spreadsheet apmethod aims to represent the visual behavviewers when they watch TV news that usedynamical graphical layout.

ea was. Finally, in so possible to use of visualisations;

sation technique f points, the spot o simultaneously

different users. nguish the points

onsist in defining or analysis of This visualisation or a subject or a media. oftware includes n some contexts

also important to the software used

oftware, it is not hniques for video process the time the “bee swarm” nt some of those niques in order to es was developed d a new technique le in eye tracking y designated this 8]. Authors in the d this technique. ye tracking as an

was developed by d an information ific necessities of

plies to temporal e following main

ent in TV News, his, it is possible f attention. The

some evidence ell as some of the outs evidenced in

News [12, 13, 15]. tracking software iewer’s focus of it was necessary

nting this data in ation is based on og file data with pplicationiii. This viour of a set of s a superimposed

A. Method Description The analysis method that was d

three main phases: i) Collection of Phase 1); ii) Extraction of log file into a spreadsheet application (Application of algorithm (Fig. 3, Ph

Figure 3. Analysis Meth

The first phase consists in capmovements. This registration is doThe second phase deals with the coeye tracker’s log files. The relevextracted for analysis are: x-coordinduration of each fixation. This dataformat compatible with the analysapplication which is dependent of temporal media being studied. The ttwo moments: i) the application of ii) visual representation of the dalgorithms (3b). The first momeapplication of two algorithms to“fixation points algorithm”(FPa); algorithm” (FTa). Fig. 4 represenpredetermined analysis area, characwith the coordinates of its origin i(B0,0), the coordinates of the bottomthe coordinates of the top left corner

Figure 4. Bounding Area Ch

developed is divided into eye tracking data (Fig. 3, data from an eye tracker (Fig. 3, Phase 2); iii) ase 3).

hod Phases

pturing the viewers’ eye one using an eye tracker. ollection of data from the vant data that has to be nate, y-coordinate and the a is then converted into a sis data in a spreadsheet the characteristics of the

third phase is divided into f the algorithms (3a); and ata obtained with these

ent (3a) consists in the o the collected data: i)

and ii) “fixation time nts a bounding box of a cterized by its dimension in the bottom left corner m right corner (BB,R) and r (BT,L).

haracterization

359

FPa algorithm L = Total number of log file entries. ri(x,y) = log file data sample (x and y coordinate, and fixation time). cFP = Fixation Point count (number of points). Initial condition of cFP is 0 point counts.

;1)

)()((

,

),(0,0,),(1 0,0

+=<

<<<� =

FPFPLT

yxiRByxiL

i

ccB

ryBANDBrxBif

The first algorithm (FPa) calculates the total number of

fixation points within a predetermined bounded area of analysis. Defining a bounding area is important because there may be fixation points outside the intended area of analysis (Fig. 4). After defining the bounding area, all log file entries (L) are imported into a spreadsheet application. Each log file entry (ri(x,y)) is tested; if the “x AND y” coordinates of the data sample (ri(x,y)) are within the defined bounding area, the counting variable is increment once [cFP = cFP+1]. The final count number held by cFP corresponds to the total number of fixation points for the predetermined bounding area, previously determined for analysis according to temporal media study areas and temporal intervals.

FTa algorithm L = Total number of log file entries. ri(x,y) = log file data sample (x and y coordinate, and fixation time). cFT = Fixation time count (ms). Initial condition of cFT is 0 millionth of a second (ms).

;)()

)()((

),(,

),(0,0,),(0,01

FTyxiFTLT

yxiRByxiL

i

crtcB

ryBANDBrxBif

+=<

<<<� =

The second algorithm (FTa) consists in obtaining the

total fixation time of a predetermined bounding area of analysis. Similarly to the first algorithm (FPa), a bounded area must be defined and all log files entries (L) must be imported. For each log file entry (ri(x,y)), if the “x AND y” coordinates (ri(x,y)) are within the defined bounding area, then sum the respective time (t) of ri(x,y) [cFT = t(ri(x,y)) + cFT]. The sum value held by cFT indicates the total fixation time for the defined bounding area analysed. The second moment (3b) consists in visually representing the data obtained from these algorithms in a “bubble chart”. Here, the maximum and minimum values of the x and y coordinates are defined (i.e. the intended bounding area). The position of each bubble in the chart is defined by a x and y coordinate, and the size of the bubble by the respective fixation time: the larger the bubble, the longer the fixation. This visual representation of data can be considered a combination between "Gaze Plot" and "Bee Swarm", because it represents in terms of size the time spent in each fixation point (gaze plot), as well as the number of fixation points of one or more viewers (bee swarm). However, this visual representation of data does not represent the saccades or the visual path of the viewer over time (another characteristic of the "Gaze Plot"). Finally, the desired Areas of Interest on the TV screen layout can be adapted to these algorithms.

B. Study Context The analysis method described above was applied in an

empirical study. The study consisted in participants watching approximately 10 minutes of a sample of TV News on a Tobii T120 eye tracker monitor. A convenience and non-probabilistic sample of 80 Portuguese university students took part in the study. The participants’ age ranged from 18 to 30 years old (mean = 22.5), with 40 female and 40 male participants. A pre-session and post-session survey was used as an instrument for data collection. Results acquired from these surveys are discussed in [20].

A controlled study object was used throughout the study. The reason behind the production of this news sample was to prepare a set of news stories with unfamiliar content to all the participants in the study, in order to avoid possible bias from previous knowledge about the topic and graphical layout. Four (4) sets of sample news were used and each participant only visualized one of the sets, selected in a random order. All sets were composed of 6 news stories, with the same global length. Each one of the sets had nine minutes and fifty-two seconds (09:52) length. The topics were: Society, Sciences and Technology, Actuality (2 news stories), Culture and Sport. These topics were chosen because they are the most common thematics in Portuguese TV News’ and trigger a greater interest in viewers. The sample news used were assembled from stories developed at the University of Aveiro’s “3810” television program.

This experience presented two versions of TV News: i) a version without any graphics (clean feed version); and ii) a version with graphics overlay. In this second version, in addition to the anchor (AOI2), a set of four areas of interest (AOI) with graphic elements are used: Bugs (AOI1), over the shoulders (AOI3), lower thirds (AOI4), and ticker (AOI5), as can be seen in Fig. 5.

Figure 5. Graphical Layout (ex. Anchorwoman)

Finally, it is important to explain the procedure followed in this experience. Each participant´s session started in a room that was prepared to receive him/her in a comfortable and controlled environment, to whom was explained the purpose of the study. The first task of the study consisted in answering a pre-session survey. After answering the survey, participants were taken to an area where the eye tracker was

360

placed to watch one of the four sets of sample news. After watching the sample news, participants were asked to answer a post-session survey. The length of each session was approximately 15 to 20 minutes.

C. Results and Discussion The analysis method described in section 2.1 was

applied to this specific context (TV News), resulting in a visual representation, as can be seen in Fig. 6.

Figure 6. Areas of Interest (ex. Anchorman)

Fig. 6 represents a two-dimensional illustration of the viewers' main focus of attention during the reception of TV News. This representation consisted in associating to each viewer's visualisation an x and y position, as well as the fixation time for each point. To obtain these results, the algorithms described in section 2.1 were used and applied to the Areas of Interest (AOI) intended to analyse: AOI1, AOI2, AOI3, AOI4 and AOI5.

Specifically, the first step consisted in applying the analysis method to the entire screen, correlating the data gathered from the eye tracker (x and y position and fixation time) with the pre-determined analysis specifications created in a spreadsheet application iv . The x and y coordinates were defined in order to perform a visual representation of the points observed and bounded within the delimitations of the screen. In this case, the defined x and y values were: B0,0 = (0,0), BB,R = (756,0), BT,L = (0,520), as can be seen in Fig. 6.The process was repeated for the previously defined AOIs. The x and y values used for each AOI were:

• AOI1: B0,0 = (0,450); BB,R = (0,180), BT,L = (0,510); • AOI2: B0,0 = (180,90); BB,R = (470,90), BT,L =

(180,505); • AOI3: B0,0 = (520,180); BB,R = (180,720), BT,L =

(520,465); • AOI4: B0,0 = (55,80); BB,R = (450,80), BT,L =

(55,122); • AOI5: B0,0 = (0,0); BB,R = (756, 0), BT,L = (0,60).

The referred coordinates are represented by red dots in Fig. 6 and the visual data representation in this figure is only applied to a single participant. As a note, the number of fixations detected by each participant in the range of 10 minutes (the total duration of the sample news) is about 400-600 fixations. Therefore, if all participants’ fixations were processed, approximately 32.000-48.000 fixation points would have to be rendered and represented.

The main purpose of using this analysis method is to understand the main focus of attention during the reception of TV News as well as to analyse the differences between different graphics. Therefore, there was the intention of understanding which graphics were viewed the most in terms of number of fixations as well as fixation time. In this sense, from the results obtained with the application of this analysis method to each AOI and considering the total amount of fixation points of the screen, it is possible to have relative knowledge of the most visualized graphic elements. As can be observed in Fig. 7, the anchor is the viewers’ most focused element in terms of mean fixation time (41,0%). Concerning the graphic elements; the viewer’s attention was dispersed throughout the entire graphical layout. The ticker is the graphic with the highest mean fixation time (15,3%), while the remaining graphics present a very low percentage, especially the bugs (1,4%). In terms of fixation points, the percentages regarding the anchor were 30,1%, 1.9% for Bugs and 28.6% for the ticker. Concluding, in terms of visual behaviour, viewers spend more visual attention on the graphical elements that move (ticker and anchor) then the graphical elements without movement (bugs, lower thirds and over the shoulders).

Figure 7. Most Focused Elements in terms of Fixation Time (ex.

Anchorwoman)

The use of this analysis method can help in understanding how viewers process the various graphical elements presented in TV News and with that understand the viewers’ visual behaviour. Differences can be found between this analysis method and the main eye tracking visualisation techniques. With this analysis method it is possible to represent the data in temporal media, and applied on the TV screen layout, as well as in the AOI defined by the researcher.

361

VI. CONCLUSIONS AND FUTURE WORK The use of eye tracking technology has brought

innovative aspects to this study, since it enabled the collection of specific data concerning visual information based on subjects’ eye behaviour. However, the complexity of specific study objects in multiple contexts requires visualisation techniques that nowadays eye tracking software alone cannot always interpret. This implies that occasionally, visualisation techniques have to be developed in order to better understand data acquired with eye tracking. The main purpose of this paper was to understand the graphic elements of TV news which have greater visual attention. In this context, the analysis method explored in this paper facilitated the understanding of the relation between visual stimuli arrangement and information processing in TV news, allowing the representation and differentiation of viewers’ visual behaviour.

Finally, with the use of this analysis method, it will be possible to understand in a better way the main AOI visualized by viewers. With this knowledge it is possible comprehend some of display patterns and thereby, help TV stations with the improvement of their graphical layout, adapting it to viewers’ needs or characteristics. As future work this analysis method can be applied in other experimental versions of TV News, for example, study a graphical layout with only still graphics and another layout with only motion graphics and thereby explain if animated graphics affect the visual behaviour of viewers.

ACKNOWLEDGMENT We would like to thank Labs.SAPOv at the University of

Aveiro for lending their eye tracker equipment. To the Portuguese institution “Fundação para a Ciência e Tecnologia” (FCT) for funding the PhD grant SFRH/BD/69836/2010.

REFERENCES [1] S. McClellan and K. Kerschbaumer. (2001) Tickers and bugs:

Has TV gotten way too graphic? Broadcasting & Cable. [2] S. Josephson and M. Holmes, "Visual Attention to Repeated

Internet Images: Testing the Scanpath Theory on the World Wide Web," presented at the ETRA '02 Proceedings of the 2002 symposium on Eye tracking research & applications, New York, 2002.

[3] A. Duchowski, Eye Tracking: Theory and Practice, Second ed., 2007.

[4] L. Bergen, T. Grimes, and D. Potter, "How Attention Partitions Itself During Simultaneous Message Presentations," Human Communication Research, vol. 31, pp. 311-336, 2005.

[5] H.-B. Brosius, W. Donsbach, and M. Birk, "How do text-picture relations affect the informational effectiveness of

television newscasts?," Jornal of Broadcasting & Electronic Media, vol. 40, pp. 180-195, 1996.

[6] D. Drew and T. Grimes, "Audio-Visual Redundancy and TV News Recall," Communication Research, vol. 14, pp. 452-461, 1987.

[7] J. Fox, "A Signal Detection Analysis of Audio/Video Redundancy Effects in Television News Video," Communication Research, vol. 31, p. 524, 2004.

[8] J. Fox, A. Lang, Y. Chung, S. Lee, N. Schwartz, and D. Potter, "Picture This: Effects of Graphics on the Processing of Television News," Journal of Broadcasting and Electronic Media, vol. 48, pp. 646-674, 2004.

[9] L. Frings, I. Mader, and M. Hull, "Watching TV news as a memory task – brain activation and age effects," BMC Neuroscience, vol. 11, p. 7, 2010.

[10] D. Graber, "Seeing is remembering: How visuals contribute to learning from television news," Journal of Communication, vol. 40, pp. 134-156, 1990.

[11] T. Grimes, "Mild Auditory-Visual Dissonance in Television News May Exceed Viewer Attentional Capacity," Human Communication Research, vol. 18, pp. 268-298, 1991.

[12] S. Josephson and M. Holmes, "Clutter or content? How on-screen enhancements affect how TV viewers scan and what they learn.," ETRA 2006 Proceedings, pp. 155-162, 2006.

[13] R. Matsukawa, Y. Miyata, and S. Ueda, "Information Redundancy Effect on Watching TV News: Analysis of Eye Tracking Data and Examination of the Contents," Library and Information Science, vol. 62, pp. 193-205, 2009.

[14] J. Azevedo, L. Fernandes, and M. Saraiva, "O Telejornal sob o olhar da cenografia: Da experimentação à imagem de marca," Comunicação e Sociedade, vol. 15, p. 245, 2009.

[15] R. Marques, "Avaliação da Recepção da Informação Noticiosa em Televisão," Departamento de Comunicação e Arte da Universidade de Aveiro, Aveiro2009.

[16] C. Silva, "Gerador de Caracteres e Grafismos Multimédia para Emissões TV," Curso de Mestrado em Comunicação e Multimédia, Universidade de Trás-os-Montes e Alto Douro, Vila Real, 2009.

[17] B. Fry, "Computational Information Design," Media Arts and Sciences, School of Architecture and Planning, Massachusetts Institute of Technology, 2004.

[18] K.-J. Räihä, A. Aula, P. Majaranta, H. Rantala, and K. Koivunen, "Static Visualization of Temporal Eye-Tracking Data," in Human-Computer Interaction-INTERACT, ed. New York, 2005, pp. 946-949.

[19] A. Flag, M. Haraty, G. Carenini, and C. Conati, "The Role of Eye Tracking in Adaptive Information Visualization," presented at the International Conference on Intelligent User Interfaces, Palo Alto, California, USA, 2011.

[20] R. Rodrigues, "A cenografia das notícias televisivas em Portugal Um Estudo de Eye Tracking," Departamento de Comunicação e Arte, Universidade de Aveiro, 2010.

i Screenshot taken from “RTP News” (http://rtp.pt/play/) on 22/2/2012. ii A - http://tinyurl.com/6udygam; B - http://tinyurl.com/6v2q4tw; C - http://tinyurl.com/6nnypjy; D - http://tinyurl.com/72ln8u7. All Images accessed on 20/2/2012.

iii A spreadsheet is a computer application (based on rows and columns) that makes calculations and data presentations. iv The spreadsheet used in this study was the Microsoft Excel. v Accessed 2012/2/26: http://labs.SAPO.pt/ua/.

362