+ All Categories
Home > Documents > Task-based User-System Interaction

Task-based User-System Interaction

Date post: 26-Aug-2016
Category:
Upload: alan
View: 217 times
Download: 1 times
Share this document with a friend
9
Künstl Intell (2012) 26:141–149 DOI 10.1007/s13218-011-0166-z FACHBEITRAG Task-based User-System Interaction Shah Rukh Humayoun · Antonella Poggi · Tiziana Catarci · Alan Dix Received: 7 August 2011 / Accepted: 20 December 2011 / Published online: 18 January 2012 © Springer-Verlag 2012 Abstract In current electronic environments, the ever- increasing amount of personal information, means that users focus more on managing their information rather than using it to accomplish their objectives. To overcome this prob- lem, a user task-based interactive environment is needed to help users focus on tasks they wish to perform rather than spending more time on managing their personal informa- tion. In this paper, we present parts of our on-going work on task-based user-system interaction, which highlights the need for a shift from an information-centric to a task-centric environment. More precisely, we look into issues relating to modeling user tasks that arise when users interact with the environment to fulfill their goals through these sets of tasks. Keywords Task-centric environment · Task specification · Personal ontology · Task inference · TaMoGolog S.R. Humayoun ( ) · A. Poggi · T. Catarci Dipartimento di Informatica e Sistemistica “A. Ruberti”, “Sapienza” Università di Roma, Via Ariosto 25, 00185 Roma, Italy e-mail: [email protected] A. Poggi e-mail: [email protected] T. Catarci e-mail: [email protected] A. Dix School of Computer Science, The University of Birmingham, Birmingham, B15 2TT, UK e-mail: [email protected] A. Dix Talis, 43 Temple Row, Birmingham, B2 5LS, UK 1 Introduction In most current systems, the user’s interaction is generally focused on functionally defined applications (word process- ing, address management, Internet browsing) and on the storage, organization and retrieval of information in files or other databases themselves determined by the units of op- eration of the applications. However, real activity, whether for work or leisure crosses application boundaries, may in- volve portions of files, and interlinks fragments of both. In fact, in current electronic environments, users have to focus more on managing their ever-growing personal information rather than using it to accomplish their everyday tasks. For example, Jones discusses “signs of breakdown” that are “are evident in a sense of having lost control over the files of a computer desktop or the email messages in the inbox” [23]. Managing information basically amounts to saving it and possibly being able to find (and re-find) it for subsequent reuse. During recent years, the creation of tools supporting the user in these processes has been the ultimate goal of the so-called Personal Information Management (PIM) research Field [22, 2931]. However, while more service-oriented in- ternet applications lend themselves to a more flexible model of interaction, in fact the trend still seems to be for web- based alternatives to desktop applications rather than a shift- ing to activity- or task-centered paradigm. We can identify a few characteristics that differentiate current systems from what might be a more activity or task- centered view. These are summarized in Table 1. Of course there are good reasons for systems to be the way there are. Some reasons are commercial: monolithic applications lock users in to them; and cognitive: hierarchies are complicated but simpler than virtually any other structure. So any alterna- tive is bound to involve both theoretical and pragmatic chal- lenges. Our long term aim is to understand and contribute to-
Transcript
Page 1: Task-based User-System Interaction

Künstl Intell (2012) 26:141–149DOI 10.1007/s13218-011-0166-z

FAC H B E I T R AG

Task-based User-System Interaction

Shah Rukh Humayoun · Antonella Poggi ·Tiziana Catarci · Alan Dix

Received: 7 August 2011 / Accepted: 20 December 2011 / Published online: 18 January 2012© Springer-Verlag 2012

Abstract In current electronic environments, the ever-increasing amount of personal information, means that usersfocus more on managing their information rather than usingit to accomplish their objectives. To overcome this prob-lem, a user task-based interactive environment is needed tohelp users focus on tasks they wish to perform rather thanspending more time on managing their personal informa-tion. In this paper, we present parts of our on-going workon task-based user-system interaction, which highlights theneed for a shift from an information-centric to a task-centricenvironment. More precisely, we look into issues relating tomodeling user tasks that arise when users interact with theenvironment to fulfill their goals through these sets of tasks.

Keywords Task-centric environment · Task specification ·Personal ontology · Task inference · TaMoGolog

S.R. Humayoun (�) · A. Poggi · T. CatarciDipartimento di Informatica e Sistemistica “A. Ruberti”,“Sapienza” Università di Roma, Via Ariosto 25, 00185 Roma,Italye-mail: [email protected]

A. Poggie-mail: [email protected]

T. Catarcie-mail: [email protected]

A. DixSchool of Computer Science, The University of Birmingham,Birmingham, B15 2TT, UKe-mail: [email protected]

A. DixTalis, 43 Temple Row, Birmingham, B2 5LS, UK

1 Introduction

In most current systems, the user’s interaction is generallyfocused on functionally defined applications (word process-ing, address management, Internet browsing) and on thestorage, organization and retrieval of information in files orother databases themselves determined by the units of op-eration of the applications. However, real activity, whetherfor work or leisure crosses application boundaries, may in-volve portions of files, and interlinks fragments of both. Infact, in current electronic environments, users have to focusmore on managing their ever-growing personal informationrather than using it to accomplish their everyday tasks. Forexample, Jones discusses “signs of breakdown” that are “areevident in a sense of having lost control over the files of acomputer desktop or the email messages in the inbox” [23].Managing information basically amounts to saving it andpossibly being able to find (and re-find) it for subsequentreuse. During recent years, the creation of tools supportingthe user in these processes has been the ultimate goal of theso-called Personal Information Management (PIM) researchField [22, 29–31]. However, while more service-oriented in-ternet applications lend themselves to a more flexible modelof interaction, in fact the trend still seems to be for web-based alternatives to desktop applications rather than a shift-ing to activity- or task-centered paradigm.

We can identify a few characteristics that differentiatecurrent systems from what might be a more activity or task-centered view. These are summarized in Table 1. Of coursethere are good reasons for systems to be the way there are.Some reasons are commercial: monolithic applications lockusers in to them; and cognitive: hierarchies are complicatedbut simpler than virtually any other structure. So any alterna-tive is bound to involve both theoretical and pragmatic chal-lenges. Our long term aim is to understand and contribute to-

Page 2: Task-based User-System Interaction

142 Künstl Intell (2012) 26:141–149

Table 1 Characteristics ofactivity or task-centered view Current systems Activity/task-centered view

Computation granularity Monolithic application Simpler operations

Computation structure Functionally linked Thread of activity

Information granularity Single file or smaller unit lockedinside application

Cross-linkable down to names,numbers, etc.

Information structure Hierarchical Relational

wards the move from personal object-centric computers en-vironments, i.e. systems focusing on the management of per-sonal information, towards personal activity- or task-centricenvironments, i.e. systems focusing on the management ofpersonal interaction, which should support interactively theuser in executing his/her tasks so as to achieve his/her goals,rather than just managing the personal information. This pa-per is inspired by this vision of more activity- or task-centriccomputing and describes the user-task related parts of ourongoing steps towards this long term goal.

To gain an insight, let us consider the following example.In a hardware shop one may find a section with all hammers,a section with all types of paint, etc. However, a carpenterwho wants to build a wall has to select a mixture of the ap-propriate tools and construction materials and store them inthe van in order to use them to execute the required task. Theactivity of selecting and storing is in charge of the carpenter.It would be much easier for the carpenter to just enter intothe shop, say that he/she has to build a wall, and have every-thing needed ready in his/her van in the right order. In otherwords, it would be definitely beneficial for the user to movefrom a (static and rigid) object-centric world to a (dynamicand adaptive) activity-centric one. The paper is organized asfollows. In Sect. 2, we provide a motivating scenario andthen define an abstract architecture for this kind of environ-ments that shows the important components for laying thefoundations of our approach. In Sect. 3, we focus on themain parts of this paper and describe our approach towardmodeling user tasks, the role of Personal Ontology (PO) inthe selection and execution of these tasks, and address theproblem of inferring these user tasks. In Sect. 4, we pro-vide a brief overview of the related work. We discuss someother works done in this direction and conclude the paperin Sect. 5.

2 Overview of User Task-Centric InteractiveEnvironment

In this section, we start by presenting a scenario showinghow a task-centric environment would support the user inperforming his/her day-to-day tasks. Then, we provide anabstract model for such an environment.

2.1 A Motivating Scenario

Consider the following scenario. Alan is a professor at Lan-caster University. He collaborates with Antonella, from theUniversity of Rome for a research project. One day Alan de-cides to start organizing the forthcoming mission to Romefor the project meeting. This may happen either becauseAlan that day woke up in a black mood and wanted to cheerup thinking of “a pleasurable trip”, or because he receivedan email from Antonella, inviting all project partners to themeeting. In both cases, a personal task-centered environ-ment would propose to Alan the list of tasks that he is morelikely to perform. In particular, in the case he had receivedan email from Antonella, the environment would detect theoccurrence of both Antonella and the project name in theemail, e.g. as the sender and the object of the email respec-tively. We assume that the system has access to Alan’s Per-sonal Ontology (PO), data about Alan organized througha conceptualization of his own domain of interest includ-ing people, projects, event, meta-data about documents, andother more personal concepts [24]. By using a reasoningmechanism over the underlying Personal Ontology, the en-vironment would detect all user-level tasks closely and se-mantically related to the information detected, such as orga-nizing a mission to Rome (flight booking, transport book-ing, hotel booking, etc.), writing a research paper about theproject, inviting local project partners for a meeting in Lan-caster, and so on.

Let us suppose that Alan chooses from the list the taskof organizing a mission. By again accessing the knowledgeavailable from his PO, the system would suggest that Alanspecifies or confirms the data required to define a mission,e.g. the location, the date and the context of the mission.Also, it may propose some particular values for this data,that is either semantically related to Alan’s present context,e.g. Rome for the location since the task was activated afterthe reception of Antonella’s email. At this stage, as Alan hasspecified the necessary information, according to the currenttask definition, the environment would next propose that hebooks a plane to Rome, then support him in booking hisfavorite hotel in Rome, and so on.

Page 3: Task-based User-System Interaction

Künstl Intell (2012) 26:141–149 143

Fig. 1 Components oftask-centric environment

2.2 Abstract Model of the Environment

From a careful analysis of the previous scenario, it followsthat, in order for such an environment to be effective, itshould provide suitable mechanisms for profiling, semanticstorage of documents, task and context inference. Figure 1shows an architectural view of how different parts will fittogether to provide a platform for such an environment. Thesolid arrows in the figure represent “control” flows, whichinfluence the choice of next task to be executed, whereasthe dashed arrows represent information flows. Overall theleft-hand side of the architecture represents components thatgather data that can be used to establish context or to au-tomatically initiate or suggest action, the middle is aboutcontext inference and task execution, and the right-handside concerns task representation and inference. Elementsof this architecture have been described in previous work[6, 10–12], but this paper is principally concerned with theright-hand side.

In brief, the components in the shown architecture workas follows. The recognizers (data detectors) find suitablefragments of the raw information that are semanticallymeaningful and that can be used to initiate or feed into ac-tions/tasks. This raw information could be incoming emails,active files, web pages or other kinds of documents anddata that the user encounters. The matching between suit-able semantically meaningful fragments and the possibleactions/tasks can be done through different methods such aspredefined service descriptions or by analyzing user’s previ-ous history.

The inference engine learns from user interactions andmakes suggestions to pre-populate parameters of suggestedtasks. The task-sequence inference process records interac-tion in order to learn patterns and also to make suggestionsbased on prior learning drawing on a stored history of pastuser action, as well as on specific user criteria. Here thishistory is shown as separate store, but it will typically be

closely linked to, or part of, the personal ontology. We needto record what is done in order to both establish a senseof context and in order to be able to allow the system togain some understanding of the user’s ongoing activities andtasks.

Underlying all of these is the personal ontology itself andspreading activation [25]. These may influence the initialdata detectors or may influence the choices of tasks and thetask-sequence inference processes. These components feedinto each other. The various terms, names, emails, etc., in apersonal ontology can yield keywords to be matched againsttext or semi-structured sources. Also as users perform ac-tions, the way in which they use information and the re-sults of their activities can be used to enrich the ontology.Note that this ontology may have different levels of reason-ing, such as simple forms of forward chaining [10, 11] ora more sophisticated reasoning engine through support ofsome ontology language [4], and this could also influenceother aspects of the picture. It is important to note that thisis the planned picture and many of the individual compo-nents exist in our previous work, but not all the interactionsare currently in place [12]. In the forthcoming sections how-ever, due to space constraints and this paper’s theme, wefocus only on our work concerning user-level tasks in theproposed architecture.

3 User-Task Modeling

In this section, we focus towards user tasks in our proposedenvironment. In order to model properly user-level tasks, wefirst need to define them using some form of formal taskspecification language that should be well-defined and ex-pressive enough for modeling these tasks at appropriate ab-straction levels. Secondly, we need to specify how the POof a specific user affects tasks selection and then how it isupdated due to execution of the selected task. Lastly, wehighlight the problems and solutions related to inferring tasksequence in such an environment.

Page 4: Task-based User-System Interaction

144 Künstl Intell (2012) 26:141–149

3.1 Choosing a Task Modeling Notation

There are a wide variety of task-modeling notations in useor proposed in the literature. Some of these have arisen fromorigins outside of computing, notably Hierarchical TaskAnalysis (HTA) [37]. Others have arisen where computingnotations have been adapted for task description, for exam-ple, Task Action Grammar (TAG) [27], which was derivedfrom formal grammars, User Action Notation (UAN) [18],which describes the behavior of the user and the interfacewhile both of these perform some tasks together, and ConcurTask Trees (CTT) [28], which uses operators from LOTOS,a concurrency notation focused particularly on specifyingcomputer network protocols.

Despite their different origins, task modeling notationsalmost universally include a strong focus on the hierarchicaldecomposition and ordering of tasks and sub-tasks, as is evi-dent, for example, in the chapters and introduction in Diaperand Stanton’s exhaustive collection [9].

Most task-modeling notations, even when derived fromcomputing formalisms, are focused on use by human ana-lysts during design. However, we also require that the nota-tion be usable for collaborative automation, where the usermay provide input/control at various points, but where somesubtasks are executed by the computer system on behalf ofthe user. Furthermore, we wish to be able to infer task mod-els from past interactive use. That is, we require a notationthat is both executable and tractable. Although there are in-spirations from related areas such as agent-based systemsand work-flow automation, there are no de facto commonnotations as there are in early design stages.

We propose to use TaMoGolog (Task ModelingGolog) [20], a formal task specification language, for mod-eling user tasks. TaMoGolog was built [19] on top of theGolog-family [15, 16, 26] of high-level programming lan-guages. The TaMoGolog language is at the same time ex-pressive enough for actually being helpful to model user-level tasks at different abstraction levels, and well-definedenough, syntactically and semantically, for being effectively“usable” and “learnable” by the system due to using the sit-uation calculus [35] theory. As part of a task definition, thelanguage allows the specification of precondition axioms oftask, of postcondition effects to fluents due to task execu-tion, of a rich set of operators usable for constructing com-plex path scenarios, and of task context knowledge. (Note afluent is the name in Golog [15, 26] for what can be thoughtof as a form of variable/data value.)

3.2 TaMoGolog: A Formal Task Specification Language

TaMoGolog distinguishes tasks into three main categories[20]: unit tasks, denoted as μ, are considered to be per-formed in an atomic manner; waiting tasks, denoted as ω,wait either for a particular event to happen or for some

set of conditions to be fulfilled; and composite tasks, de-noted as Γ , handle the structural behavior of the path-scenarios. TaMoGolog provides a rich set of operators [19],mostly obtained from the Golog-family along with someadditional operators, useful for constructing complex taskstructures. The TaMoGolog set of operators consist of: wait-ing/testing condition φ?, sequence [Γ1;Γ2], internal nonde-terministic choice [Γ1|Γ2], external nondeterministic choice[agtΓ1|Γ2], conditional choice [if φ thenΓ1 elseΓ2], internalnondeterministic choice of arguments [πx.Γ (x)], externalnondeterministic choice of arguments [agt πx.Γ (x)], inter-nal iteration [Γ ]∗, external iteration [agt Γ ]∗, conditionalloop [whileφ doΓ ], concurrency [Γ1‖Γ2], concurrency withpriority [Γ1 � Γ2], concurrent iteration [Γ ]‖, external pri-ority concurrency [agt Γ1 <> Γ2], external concurrent it-eration [agt Γ ]‖, interrupt 〈φ → Γ 〉, and failure handling[Γ1 � Γ2].

TaMoGolog semantics is also based on Golog-family[15, 26], where a unit task can be performed if all the relatedprecondition axioms are true at the moment of its executionand, once executed, its effects are shown on related fluents.Also, unit and waiting tasks are performed in an atomic step,while composite tasks are performed in several steps, onestep for each unit and waiting task composing the compos-ite task itself.

In brief, a task structure specification through TaMoGologconsists of the following predicates and definitions: nameof the task structure; sets of unit, waiting, and compositetasks; definition of each waiting and composite task throughGolog procedure definition [26]; set of precondition axiomsfor each unit task; postcondition effects on fluents (relationaland functional) for each unit task; external entities (externalapplications/systems or human users collaborating with thesystem) and their responsible tasks’ definitions; initial val-ues of fluents; any optional domain and context knowledgerepresentation.

3.3 Modeling the Motivating Scenario

Let us go back to the motivating scenario of Sect. 2.1. Here,we are interested in defining tasks, especially at the userlevel, by using our task specification language. Through Ta-MoGolog, it is possible to define user-level tasks at dif-ferent abstraction levels. At the higher abstraction level, afew of the user-level tasks from the scenario can be rec-ognized as: flightBooking, hotelReservation, transportBook-ing, preparingPaper, composingEmail, and updatingSched-uleDairy. On the other side, we can also recognize a fewenvironment-level tasks that are performed as the effect ofthe execution of these user-level tasks such as autoUpdat-ingScheduleDiary, updatingPO, etc. The above user-leveltasks can be decomposed further into sub-tasks. For exam-ple, flightBooking may require the user to first fill a form or

Page 5: Task-based User-System Interaction

Künstl Intell (2012) 26:141–149 145

search for a flight, and then to select a specific flight. Moreprecisely, the flightBooking task structure may be defined asfollows (in TaMoGolog syntax):

proc flightBooking[User (fillingForm; searchingFlight)]*;selectingFlight; priceGeneration;[User (Pay-By-Card | Pay-By-Bank);showingPaymentForm; fillingPaymentForm;confirmPayment; completingPayment;showingFlightDetail;[User (orderPrinting; printingPage)]*

endProc

One important aspect of a TaMoGolog task specification isthat it allows the definition of precondition axioms and post-condition effects for each task considered atomic at that ab-straction level. For example, searchingFlight may have pre-condition axioms, requiring the task to take as input a validdate of flying, as well as valid departure and arrival airports,etc. Also, the flightBooking task execution may affect manyfluents, e.g., expressing that the user is allocated a seat inthe flight, or that the user’s bank balance will be updated bydeducting the flight price.

Specifying a user-level task at a certain abstraction levelin the environment depends highly on the depth of thecontrol by the environment on the service providers’ ap-plications/systems which, in fact, execute these tasks lo-cally in their own environments. These specific applica-tions/systems execute these tasks internally, while the en-vironment maintains its own internal log through gettingfeedback from the executing applications/systems. Withinthe service providers’ applications/systems, the task may becomposed of many sub-tasks but from the environment per-spective, the task is normally treated as atomic. It is pos-sible that a certain application/system provides some formof API or meta-level description which exposes the lowerlevel details such as sub-tasks, sub-task paths, pre- and post-condition axioms for each sub-task, etc.; in which case theenvironment can also keep track of these lower level details.The selection of an appropriate application/system for exe-cuting a user task depends on many factors such as user’sprevious actions or some specific filtering criteria manuallyspecified by the user. As an example, de Leoni et al. in [17]provides a framework for adaptive process management sys-tems where tasks are assigned to external service providersthrough some defined criteria and then the targeted task isexecuted by that selected service provider.

3.4 The Role of the Personal Ontology in User TaskSelection

The Personal Ontology (PO) reflects the user’s view ofhis/her own domain of interest. In particular, it allows themanagement of the whole collection of heterogeneous per-sonal data usually maintained in a personal computer (e.g.

contacts, documents, emails) by accessing them througha unified, integrated, virtual and yet user-tailored view ofhis/her data [24]. Therefore, while POs will contain somecommon elements and classes, each PO is specific to eachuser. It can be specified by means of a Description Logic,such as DL-LiteA [3, 32, 33], which, besides allowing theexpression of the most commonly used modeling constructs,also allows to answering expressive queries, i.e. conjunctivequeries, in polynomial time with respect to the size of thedata. This is clearly a distinguishing and desirable feature ofsuch a language in a context like ours, since the amount ofdata is typically huge in one’s personal computer.

Clearly, in order for a task-centric environment to be ef-fective, the PO has to be constantly updated to reflect usercurrent activities and interests and to allow to suggesting themost appropriate list of tasks according to the user personaldomain of interest and context.

Now, we go on to consider a few of the key issues re-lated to PO and user tasks. The first one is the triggering ofsuggestions of the most appropriate set of tasks and task-paths according to each user’s own perspective. To this aim,the PO plays an important role as it provides a basis along-side other criteria, such as the user’s previous actions trace.As an example from the motivating scenario, after gettingthe same email from Antonella, the suggested set of taskswill be different for Alan compared with the suggested setof tasks for Shah, who is a colleague of Antonella living inRome. In particular, given that Alan’s PO reflects that Alan’scurrent location is Lancaster, while the meeting location isRome, the environment would suggest that Alan book travel.To provide a better solution, the environment can use furthercriteria, possibly specified as precondition axioms. For ex-ample, there are two possible tasks for traveling, i.e., by airor by road, with preconditions (in TaMoGolog syntax): Pre-condition(flightBooking) ≡ distance > 500 and, respectively,Precondition(coachBooking) ≡ distance ≤ 500. As, in thiscase, the distance is greater than 500, the environment wouldsuggest that Alan book a flight and other tasks related to longtraveling. But in the case of Shah, as his location coincideswith the meeting location, the environment would suggestthat he just manage his schedule diary. For suggesting thebest set of tasks, different methods and techniques can beapplied such as spreading activation [25], which has beenused for modeling the user’s context using a PO.

The PO also provides a basis for filling the task input pa-rameters in order to reduce the user effort in executing tasks;this parameter filling can be performed in an automatic orsemi-automatic manner. For example, the user’s current lo-cation can be used by the environment to fill the departuredestination while booking travel to another city. On the otherhand, the execution of a task may also affect the user’s PO.The environment can use a task’s postcondition effects in or-der to update the information in the user’s PO. For example,

Page 6: Task-based User-System Interaction

146 Künstl Intell (2012) 26:141–149

as Alan arrives in Rome to attend the meeting, the informa-tion in his PO would be updated due to the postconditioneffect (in TaMoGolog syntax): Postcondition(flightBooking,currentLocation, date = flightDate and time = arrivalTime)≡ currentLocation = arrivalDestination, which says that theflightBooking task will change the current location of theuser to the arrival location on the day of flight after the ar-rival time. In this case, if Alan gets an email after arrivingin Rome from his friend, who lives in Rome, for a meet-ing then the environment would suggest just scheduling ameeting and probably booking a taxi service rather than giv-ing the previous list of suggestions. Note that the PO can beupdated following different approaches. A notable approachis the one presented in Calvanese et al. [5], where the au-thors propose to use ConGolog-like high-level programs (onwhich TaMoGolog also based on) to update ontologies.

3.5 How to Infer User Tasks

In task-centered environments, detecting and inferring usertasks is complicated, particularly where the user path selec-tion is highly dependent on the user’s personal ontology asthe task execution path can be different for each user fromthe same trigging event. In fact, particular problems relatedto inferring tasks in task-centered environments are:

• Interleaving—users often perform several tasks simulta-neously, perhaps while waiting for something to com-plete, or because they notice an alert, or get a telephonecall. In this case it is difficult to suggest correct sugges-tions. Before task inference can begin it is necessary todisentangle these, otherwise each sub-task is littered withthe “noise” of the others.

• Dynamicity—where the suggestions or task paths changeat run-time dynamically as updates happen in the user POdue to execution of some user- or system-level task orsome event occurs.

• Generalization—where the user entered data is simply se-quences of keystrokes, clicks on locations or basic datatypes, it is hard to generalize without very large numbersof examples.

The use of the PO helps considerably with these problems.In the case of the interleaving problem, we encourage a drill-down model of interaction where the user either selects pre-vious outputs of tasks and then drills forwards or responds totask suggestions. This creates an explicit link either directlybetween tasks or indirectly between them through ontologyrelationships. These links can then be used to separate outinterleaved tasks and sub-tasks by tracing lines of depen-dency, rather like pulling out a string of pearls from a jew-elery box (see Fig. 2). The defining of precondition axiomsand postcondition effects in task specification through Ta-Mogolog could also be very useful in this case. In the sec-ond case, there is need to change dynamically at run-time

Fig. 2 Untangling interleaved tasks using dependencies

the suggested sets of tasks in the light of changes in the userPO or as some other event occurs, e.g., as Alan books flightfor Rome the environment may also suggest making a meet-ing with a company in Rome with whom Alan was previ-ously working. This dynamic suggestion list can be gener-ated based on the PO, history of user actions in the past, andwith any given criteria. The inference job here is to suggestthe most likely unit tasks and entire task sequences so as tominimize the user’s effort in using the environment. Intu-itively, we intend to build a series of increasingly complexinference mechanisms, both in term of our development pro-cess and in terms of user experience. That is even if we havecomplex inference mechanisms available, these need to bepresented incrementally to the user.

Concerning the generalization problem, because we canhave a rich typing of task/action input and output datathrough the PO, we are in a much better position to general-ize. If we only know that a task requires a character string asinput, then given a future character string we may have manypossible previous tasks sequences where the initial actionsin these tasks require a string. In contrast, if we know thata character string is in fact a person name or a city name,then faced with a future person name (perhaps from a di-rectory lookup, or an email sender) it is easier to find pasttasks requiring as input a person name. In other words, ourgeneralization is not based on the representation in terms ofletters, but in terms of the elements of the ontology.

4 Related Work

The creation of a complete, ontology based task-centeredenvironment is a demanding task to complete and a resultof bringing together the state of the art in many researchdomains. Individual works exist related to personal infor-mation management and task inference but none combinesthe complete functionality necessary for a working task-centered environment. This section discusses some relatedwork in different areas of the task-centered environment.

Ontologies in the form of hierarchies of user interestshave been proposed in [38]. Gauch et al. [14] also pro-posed a system that adapts information navigation based ona user profile structured as a weighted concept hierarchy.

Page 7: Task-based User-System Interaction

Künstl Intell (2012) 26:141–149 147

The user may create his/her own concept hierarchy and useit for browsing web sites. Razmerita et al. [34] presented ageneric ontology-based user modeling architecture appliedin the context of a Knowledge Management System, whileDenaux et al. [8] discuss the usage and adaptation of inter-active ontology-based user modeling in learning informa-tion systems for the user’s personal learning content man-agement.

Regarding activity-centric applications, Phuwanartnuraket al. [21] explore the way people organize information insupport of projects. Folder hierarchies have also been usedto prepare and plan activities like planning a wedding. Thisuser behavior suggests that there is in fact a need for amore activity-based desktop environment. In the prototypeActivity-Centered Task Assistant (ACTA) [1], a user’s task,referred to as an ACTA activity, is represented as a pre-structured container, which can be created inside the emailfolder hierarchy. It is a task-specific collection containingstructured predefined elements called components, that em-body common resources of the task and appear as activitysub-folders. Even though ACTA activities are built relyingon user’s personal data, their approach is not comparable toours, since they do not consider tasks as a workflow of ac-tions (e.g. filling and sending the meeting invitation email),which can be inferred and semi-automatically executed.

The Semex System [2, 13] (short for Semantic Explorer)aims to offer users a flexible platform for personal informa-tion management. It enables the browsing personal of in-formation by association through the automatic creation ofassociations between data items on one’s desktop and alsoleveraging the associations created to increase users’ pro-ductivity. The work in [7] presents a concept, architectureand prototype for a semantic desktop search environment,which promises to exploit the information present in thesecontexts accumulated by user activities and additional back-ground knowledge. Gnowsis [36] is an Open Source project1

released under a BSD compatible license. It is created toprove the ideas of the Semantic Desktop and to have a ref-erence implementation at hand for other researchers to testthe system and build on it.

5 Conclusion

In this paper, we have presented our vision of a task-centricview of personal computing and some of the properties re-quired for this. In order to develop a fully operational task-centered environment, there are many open issues yet to beresolved. However, we have made several important stepstoward such an ambitious goal.

1http://www.gnowsis.org

First we showed how to model user tasks with the helpof the TaMoGolog task modeling language. There are sev-eral reasons for choosing TaMoGolog for specifying usertasks such as the ability to define tasks at different abstrac-tion levels accurately and unambiguously, the explicit rep-resentation of user and external systems interaction with theenvironment, and the use of precondition axioms and post-condition effects to update the PO to aid in task inference.Moreover, it is based on the Golog-family, hence it can alsoreuse existing approaches for automated planning and taskinference in the Golog-family, as well as those for updatingontologies as defined in [5] and for assigning tasks to exter-nal services providers as defined in [17].

We also described the role of the PO for suggesting thepossible lists of tasks and the effects on the PO due to theexecution of the selected task. Furthermore, we explainedproblems and solutions related to inferring task sequencesin such an environment. The desktop prototype applicationOn Time [6], implemented few of the environment compo-nents including visualization of the personal ontology, andspreading activation initiated by data found using data detec-tors on recently active files. The spreading activation is usedto propose actions based on the most active entities as wellas to pre-fill forms. However, in all of these, task sequencesupport is limited to at best rudimentary chaining throughdata detectors and the spreading activation. In the prelimi-nary formative user studies (with six users) of On Time usingcooperative evaluation techniques [39], users were satisfiedwith the overall system concepts but the evaluation tests didreveal a number of more peripheral usability issues, leading,for example, to the redesign of some icons, and also someissues related to visualization of the ontology.

As already stated, there is still work to be done in sev-eral directions related to several aspects of the task-centeredenvironment. The path to creating a fully functional task-centered environment will not be straightforward but it willcertainly be very fruitful.

References

1. Bellotti V, Thornton J (2006) Managing activities with TV-acta:TaskVista and activity-centered task assistant. In: Proc of the sec-ond SIGIR workshop on personal information management (PIM2006).

2. Cai Y, Dong XL, Halevy A, Liu JM, Madhavan J (2005) Personalinformation management with SEMEX. In: SIGMOD-05, pp 921–923

3. Calvanese D, Giacomo G, Lembo D, Lenzerini M, Poggi A, RosatiR (2006) Linking data to ontologies: the description logic DL-LiteA. In: Proc of the 2nd workshop on OWL: experiences anddirections (OWLED 2006)

4. Calvanese D, De Giacomo G, Lenzerini M, Rosati R (2007)Tractable reasoning and efficient query answering in descriptionlogics: the DL-lite family. J Autom Reason 39(3):385–429

Page 8: Task-based User-System Interaction

148 Künstl Intell (2012) 26:141–149

5. Calvanese D, De Giacomo G, Lenzerini M, Rosati R (2007) Ac-tions and programs over description logic ontologies. In: Descrip-tion logics, v 250

6. Catarci T, Giuliano R, Piva M, Poggi A, Terella F, Tracanna E(2008) The On-TIME user interface. In: Proc of the 5th conferenceof the Italian chapter of AIS (itAIS 2008), Paris, France

7. Chirita P, Gavriloaie R, Ghita S, Nejdl W, Paiu R (2005) Activitybased metadata for semantic desktop search. In: ESWC-05

8. Denaux R, Dimitrova V, Aroyo L (2004) Interactive ontology-based user modeling for personalized learning content manage-ment. In: Proc of AH’04: workshop proceedings. Part II, pp 338–347

9. Diaper D, Stanton N (eds) (2004) The handbook of task analysisfor human-computer interaction. Erlbaum, Hillsdale

10. Dix A, Catarci T, Habegger B, Loannidis Y, Kamaruddin A, Kati-fori A, Lepouras G, Poggi A, Ramduny-Ellis D (2006) Intelligentcontext-sensitive interactions on desktop and the web. In: Proc ofthe international workshop in conjunction with AVI 2006 on con-text in advanced interfaces (CAI’06), Venice, Italy, May 23. ACM,New York, pp 23–27

11. Dix A (2008) Tasks = data + action + context: automated taskassistance through data-oriented analysis. In: Forbrig P, Paternò F(eds) Proceedings of the 2nd conference on human-centered soft-ware engineering and 7th international workshop on task modelsand diagrams, Pisa, Italy, September 25–26. Lecture notes in com-puter science, vol 5247. Springer, Berlin, pp 1–13

12. Dix A, Lepouras G, Katifori A, Vassilakis C, Catarci T, Poggi A,Ioannidis Y, Mora M, Daradimos I, Akim NMd, Humayoun SR,Terella F (2010) From the web of data to a world of action. J WebSemant 8(4):394–408

13. Dong X, Halevy AY (2005) A platform for personal informationmanagement and integration. In: CIDR-05

14. Gauch S, Chaffee J, Pretschner A (2003) Ontology-based user pro-files for search and browsing, user modeling and user-adapted in-teraction. J Pers Res (Special issue on user modeling for web andhypermedia information retrieval)

15. de Giacomo G, Lespérance Y, Levesque HJ (2000) ConGolog,a concurrent programming language based on the situation cal-culus. J Artif Intell 121(1–2):109–169

16. de Giacomo G, Lespérance Y, Pearce AR (2010) Situation cal-culus based programs for representing and reasoning about gamestructures. In: KR2010

17. de Leoni M, Mecella M, de Giacomo G (2007) Highly dynamicadaptation in process management systems through executionmonitoring. In: BPM, pp 182–197

18. Hartson HR, Siochi AC, Hix D (1990) The uan: a user-orientedrepresentation for direct manipulation interface designs. ACMTrans Inf Sys 8:181–203

19. Humayoun SR (2011) Incorporating usability evaluation in soft-ware development environments. PhD thesis, Dipartimento di In-formatica e Sistemistica “A. Ruberti”, Sapienza Università diRoma

20. Humayoun SR, Catarci T, Dubinsky Y (2011) A dynamic frame-work for multi-view task modeling. In: Marti P, Soro A, Gam-berini L, Bagnara S (eds) Proc of the 9th ACM SIGCHI Italianchapter international conference on computer-human interaction:facing complexity (CHItaly). ACM, New York, pp 185–190

21. Phuwanartnurak AJ, Gill R, Bruce H (2005) Don’t take my foldersaway: organizing personal information to get things done. In: CHI-05

22. Jones W, Teevan J (eds) (2007) Personal information management.University of Washington Press, Seattle

23. Jones W (2007) Keeping found things found: the study and prac-tice of personal information management. Morgan Kaufmann, SanMateo

24. Katifori V, Poggi A, Scannapieco M, Catarci T, Ioannidis Y (2005)OntoPIM: how to rely on a personal ontology for personal infor-mation management. In: Proc of the 1st workshop on the semanticdesktop

25. Katifori A, Vassilakis C, Dix A (2010) Ontologies and the brain:using spreading activation through ontologies to support personalinteraction. Cogn Syst Res 11:25–41

26. Levesque HJ, Reiter R, Lespérance Y, Lin F, Scherl RB (1997)GOLOG: a logic programming language for dynamic domains.J Log Program 31(1–3):59–83

27. Payne SJ, Green TRG (1986) Task-action grammars: a model ofmental representation of task languages. Hum-Comput Interact2(2):93–133

28. Paterno F (1999) Model-based design and evaluation of interactiveapplications, 1st edn. Springer, London, ISBN 1852331550

29. Personal Information Management (2006) Commun ACM 49(1)30. The PIM Workshop (2005) An NSF-sponsored invitational work-

shop on personal information management, Washington, 27–29January. http://pim.ischool.washington.edu/pim05home.htm

31. The PIM Workshop (2006) An NSF-sponsored invitational work-shop on personal information management, Washington, 10–11August. http://pim.ischool.washington.edu/pim06/index.htm

32. Poggi A (2006) Structured and semi-structured data integration.PhD thesis, Dipartimento di Informatica e Sistemistica, Universitàdi Roma “La Sapienza”

33. Poggi A, Calvanese D, De Giacomo G, Lembo D, Lenzerini M,Rosati R (2008) Linking data to ontologies. J Data Semant, X 133–173

34. Razmerita L, Angehrn A, Maedche A (2003) Ontology based usermodeling for knowledge management systems. In: Proc of the usermodeling conference, pp 213–217

35. Reiter R (2001) Knowledge in action: logical foundations forspecifying and implementing dynamical systems. MIT Press,Cambridge

36. Sauermann L, Grimnes GA, Kiesel M, Fluit C, Maus H, Heim D,Nadeem D, Horak B, Dengel A (2006) Semantic desktop 2.0: thegnowsis experience. In: ISWC-06

37. Shepherd A (2001) Hierarchical task analysis. Taylor & Francis,New York

38. Trajkova J, Gauch S (2004) Improving ontology-based user pro-files. In: Proc of RIAO 2004, pp 380–389

39. Wright PC, Monk AF (1989) A cost-effective evaluation methodfor use by designers. Int J Man-Mach Stud 35:891–912

Shah Rukh Humayoun holds aPh.D. degree in Computer Engi-neering from Sapienza Universityof Rome. His research interests in-clude user-centered design (UCD),automated usability evaluation, andtask modeling. He is particularly in-terested in applying and mergingHCI methodologies and techniquesin software engineering in order tofill the gap between these two fields.Contact him at [email protected].

Page 9: Task-based User-System Interaction

Künstl Intell (2012) 26:141–149 149

Antonella Poggi holds a Ph.D. de-gree in Computer Science, obtainedat both Sapienza University of Romeand University of Paris-Sud. Sheis currently working at Sapienzaas research associate. Her main re-search interests include semanticdata integration, XML data integra-tion, ontology evolution, personalinformation management, Web ser-vice modeling, and data provenancemanagement. Contact her at [email protected].

Tiziana Catarci is full professorat Sapienza University of Romeand, since January 2010, also vice-rector for technologies and infras-tructures. Her interests range fromdatabases & information visualiza-tion to usability & user interfaces.She has published over 150 pa-pers in leading journals and confer-ences and 20 books. Dr. Catarci isassociate editor of VLDB Journal,World Wide Web Journal, and Jour-nal of Data Semantics. Contact herat [email protected].

Alan Dix is Professor of HumanComputer Interaction at The Uni-versity of Birmingham. His pub-lished works include a major text-book on Human-Computer Interac-tion; and he is currently completinga new book, TouchIT, on physical-ity and design. His interests rangefrom formal methods to creativityand intelligent lighting. Contact himat [email protected].


Recommended