Survey of Context Provisioning Middleware

transcript

1492 IEEE COMMUNICATIONS SURVEYS & TUTORIALS, VOL. 15, NO. 3, THIRD QUARTER 2013

Survey of Context Provisioning MiddlewareMichael Knappmeyer, Saad Liaquat Kiani, Eike Steffen Reetz, Nigel Baker, and Ralf Tonjes

.Abstract—In the scope of ubiquitous computing, one of the

key issues is the awareness of context, which includes diverseaspects of the user’s situation including his activities, physicalsurroundings, location, emotions and social relations, device andnetwork characteristics and their interaction with each other.This contextual knowledge is typically acquired from physical,virtual or logical sensors. To overcome problems of heterogene-ity and hide complexity, a significant number of middlewareapproaches have been proposed for systematic and coherentaccess to manifold context parameters. These frameworks dealparticularly with context representation, context managementand reasoning, i.e. deriving abstract knowledge from raw sensordata. This article surveys not only related work in these threecategories but also the required evaluation principles.

Index Terms—Middleware, Context Provisioning, ContextManagement, Context Representation, Evaluation, Simulation,Ubiquitous Computing.

I. INTRODUCTION

UBIQUITOUS Computing (UbiComp) paraphrases theparadigm of hardware and software components being

transparently interwoven by means of wireless communica-tion. Value added computer intelligence resulting from thesmart and autonomous networking of multiple devices hasmuch more potential than that originating from a single,isolated device. A key objective of these systems is to sig-nificantly simplify Human Computer Interaction (HCI) bydeploying sensors, processors and actuators in the fabric ofeveryday life, such that their presence and complexity ishidden from users [1]. UbiComp is commonly understood asthe next wave of an evolution chain of computing paradigms,which have gone through the personal computing (secondgeneration) and distributed computing (third generation) fromthe roots of mainframe computing. Figure 1 illustrates ourview of this evolution and highlights some of the featurespertinent to the realisation of UbiComp.

A. Context and Context-awareness

Context is information about a location, its environmentalattributes (e.g. noise level, light intensity, temperature, andmotion) and the people, devices, objects and software agentsthat it contains. Context may also include system capabili-ties, services offered and sought, the activities and tasks inwhich people and computing entities are engaged, and their

Manuscript received December 31, 2011; revised October 9, 2012.M. Knappmeyer, E.S. Reetz and R. Tonjes are with the Faculty of Engi-

neering and Computer Science, University of Applied Sciences Osnabruck,Germany. (e-mail: {m.knappmeyer, e.reetz, r.toenjes}@hs-osnabrueck.de).

S.L. Kiani and N. Baker are with the Faculty of Environment andTechnology, University of the West of England, Bristol, UK (e-mail:saad2.liaquat@uwe.ac.uk).

Digital Object Identifier 10.1109/SURV.2013.010413.00207

��

Internet of Things Participatory Sensing

Smart Sensors & Actuators

Context-Awareness

Personal Computing Mainframe Computing

Distributed Computing

Ubiquitous Computing

��

Augmented Reality

Grid Computing

Mobile Computing

��

Cloud Computing

Fig. 1. Evolution Chain and Features of Ubiquitous Computing

situational roles, beliefs, and intentions. Context-awareness isone of the key enablers to facilitate proactive support of usersin their current situation. Users do not have to define theirsituation explicitly by utilising time consuming and counter-intuitive input devices but it is implicitly recognised by the“smart” environment instead. The idea that computing devicescan sense and react to stimuli from users’ environment islabelled as context-aware computing.

User identity and location are the most prominent contextparameters widely utilised in various location-based services.Beyond that, as defined by Dey [2], context may comprise anyinformation relevant for describing the users’ interaction witheach other and with context-aware services and applications.In our digital world there is a large amount of distributedinformation available describing such interaction in a diverseway, e.g. the location of a person (GPS enabled smart phones),activity (phone-based gyroscope, accelerometer, foregroundapplications, digital calendar), social situation (location, timeof the day, proximity to friends) and evolving preferences(self-configured and history-based digital profiles).

Context-awareness is an interdisciplinary field of researchinvolving communication engineering and computer science,more precisely mobile communication, HCI (Human Com-puter Interaction) design, sensor data processing, feature ex-traction and artificial intelligence. In these communities alarge set of application domains benefiting from contextualawareness has been proposed. Use cases of high potential

KNAPPMEYER et al.: SURVEY OF CONTEXT PROVISIONING MIDDLEWARE 1493

Sensors Layer

Data Retrieval Layer

Context Processing Layer

Applications Layer

Context Management Layer

Fig. 2. Layered design of a context-aware middleware

include health care and well-being [3], e-learning and campuslife [4], tourism and travelling [5], office and other businessapplications [6], advertising and e-commerce [7], entertain-ment [8], gaming [9] and social community applications[10]. Moreover, smart places – an emerging research fieldin itself – heavily rely on context-awareness. A smart placeis a geographically bounded area providing smart interactionbetween computational devices and users located in the space.Smart offices, smart labs and smart homes are examples ofsmart places [11].

B. Context-aware Systems

The computing systems that are designed to providecontext-aware services have to perform a variety of distinctfunctions. These include collection of raw data about the usersand their environment, applying different reasoning techniqueson such data to synthesise higher-level context information,storage of the context information in a retrievable and in-dexed format to make available when required, managementand coordination of context and related information betweendifferent components of the systems, and to provide a platformfor building, hosting or enabling context-aware applicationsand services.

A number of design approaches have been attempted forcollective provisioning of these functions with the overallaim of making context information available about anything,any time, and anywhere. Context-aware systems are usuallydesigned as middleware adopting a layered design – eachfunctional layer hiding the details of the underlying layers(see Fig. 2). The primary benefit of this approach is theencapsulation of varying complexities of different functions.Each layer builds on the information made available by thelayer below it, e.g. the Context Processing Layer uses datacollected at the Data Acquisition Layer while the ApplicationsLayer interacts with the Context Processing Layer to retrievecontext and does not concern itself with the details of dataacquisition or synthesis process.

With such a variety of functions to perform, the design,development, operation and evaluation of context-aware sys-tems becomes a very complex task. Each functional layerin the middleware faces multidimensional challenges in con-tributing to the overall objective of context provisioning, e.g.with respect to data acquisition, to fully exploit awareness,

context has to be acquired from heterogeneous sources, beprocessed and aggregated, as well as associated to users,devices and smart artefacts. Context is not only fetched fromphysical sensors but also from virtual and logical sensors, e.g.from databases or web services. This diversity and generalapplicability of context-awareness substantiate the need forarchitectural and functional support of ubiquitous services.

The design and operation of a context provisioning mid-dleware is also laterally affected by the context model orrepresentation scheme as it can influence the expressivenessand utility of the contextual information. In addition, cur-rent trends and innovations facilitate rapid development anddeployment of new (smartphone) applications and servicesthrough the use of Service Creation Environments and specificSoftware Development Kits (SDK). Their context demands areunexpected and not easily predictable. The support of bothexisting context-aware applications/services and those emerg-ing in the future requires functional scalability and gradualextendibility of the middleware with regard to new contexttypes and new context processing capabilities. With theirincreasing processing, storage and communication capabilities,smartphones can be considered as personalised companionsfor interfacing with the digital world. Focussing on themas the main source of interaction allows for increasing thephysical size of smart spaces to urban or even global extent.Therefore physical scalability is another prerequisite of acontext provisioning middleware.

The complex operation of a context-aware system is il-lustrated as an autonomic communication cycle in Fig. 3.The cycle highlights the acquisition of raw data from sensorsand user profiles, undergoing an aggregation/processing stagethrough application of various reasoning mechanisms, theresultant information providing a basis for decision making,which can support adaptation in context-based services andeventually serve as an input to the next cycle of (higher-level) context generation. In general, decision and adaptationare considered to be rather application/service specific andtherefore tend to be realised by the actual service/applicationlogic. The collection of sensor data and its analysis can beperformed by a common middleware that is able to support ahuge variety of application/service domains. Individual func-tions can be logically organised in different layers (cf. Fig.2) of the context-aware middleware. The middleware as awhole thus acts a glue that binds these functions togetherto achieve the goal of context-provisioning. The role of acontext-aware middleware is therefore both central and criticalto the effectiveness of context-aware services in the realmof ubiquitous computing, without which the virtual and realworlds can not be bridged.

This article presents a retrospective view of the means bywhich existing context-aware systems carry out their functionsusing their constituent components for provision of contextualinformation and services. The comprehensive review and anal-yses provided in the following sections are focussed on howcontext-aware middleware systems undertake context mod-elling, management, provisioning and reasoning. Particularly,we analyse how such complex and interwoven systems, withthe difficult task of bridging the virtual and real worlds, canbe evaluated through multidisciplinary approaches.

C. Major Contributions

This article assembles the state of the art in the multifariousaspects of context-aware systems. The comprehensive reviewand analyses of these aspects contains significant contributionsin the following categories:

The concept of context: Elaboration of the conceptual defi-nition of context and context-awareness, including a viewof how the definition of context has evolved over time,especially regarding human computer interaction.

Context modelling: A review of the context modelling andrepresentation approaches, classifications of context mod-els, the significance of context meta data and discussionon the features exposed by various context models thataffect the utility and suitability of these models.

Context management: Identification of distinct functionscarried out by context provisioning middleware and areview of existing classification of these systems. Wepresent the basis of new classifications in terms of design,architecture and technology based parameters. A reviewof existing context provisioning middleware is presented,which serves as a basis for developing our novel three-tier classification based on the conceptual design model(layered, object-oriented, event-based, etc.) of a con-text provisioning middleware, the resulting architectureand implementation (central server, multiple-distributedservers, peer-to-peer, etc.) and an intersection of thesetwo categories. Our classification presents a novel viewfor examining context provisioning middleware.

Context reasoning: We discuss the role of context reasoningas a critical function provided by context provisioningmiddleware, highlight the variety of multi-disciplinarytechniques employed in context provisioning middlewareand review the most prominent of these techniques. Thecontext reasoning and processing approaches are cate-gorised with respect to their requirements and features,and examples of their application domains are providedas well.

Evaluation of context middleware: A major contribution ofthis article is the detailed discussion on the evaluationstrategies and mechanisms of context provisioning mid-dleware. We identify the challenges faced in evaluat-ing a multidisciplinary domain, review the approachesused in evaluation of contemporary context middlewareas a whole and that of their individual functions andcomponents, identify the recent trends and evolution ofevaluation methodologies in these systems. A comparisonof these evaluation methodologies, with respect to thetarget environment and requirements, is also presentedalong with examples from the state of the art.

Domain outlook: Finally, the domain of context provisioningis discussed in a broader perspective, highlighting theoverlap and its interaction with other research disciplines.Based on lessons learnt and recent trends, we sketch theexpected evolution and identify future research objec-tives.

Our objective is not only to present an in depth coverageof the approaches taken in fulfilling the critical functions ofcontext provisioning middleware, but also to present a guide

that comprehensively introduces the domain of context-awaresystems to new researchers. To achieve this objective, it isessential to present the background and definitions of pertinentdomain concepts, which we do so in Section II. The overallarticle structure is described in the following subsection.

D. Structure

We first of all elaborate the concepts of ubiquitous comput-ing, context, and context-awareness as required backgroundknowledge in Section II. Context modelling and representationare discussed from a classification perspective in Section IIIand the context management approaches adopted in existingsystems are discussed and analysed in Section IV. SectionV contains a discussion on the context reasoning and infer-ence mechanisms. Section VI presents a detailed review andanalysis on the evaluation techniques of these middlewaresystems. In this article decision taking is not covered, actuationand adaptation are only briefly addressed when discussing theevaluation (Section VI). The article finally concludes with asummary, presentation of future trends and discussion of openissues in Section VII.

II. BACKGROUND AND DEFINITIONS

A. Ubiquitous Computing

Ubiquitous Computing (UbiComp) evolved from mobilecommunication and relies on context-awareness as one ofits key features (cf. Fig. 1). The term has been shaped byMark Weiser [1]. Weiser’s vision encompasses a new viewand a paradigm shift with regard to the interaction of humanbeings with computational resources distributed pervasivelyacross the environment and becoming part of the fabric ofeveryday life. Interaction includes both novel input mecha-nisms (e.g. gesture recognition based on accelerometers) andoutput devices (e.g. displays embedded in furniture). Theuse of processing resources is facilitated and refreshing asa “walk in the woods” [1]. Computers are pushed into thebackground. Ambient intelligence [13] arises from interweav-ing small embedded devices by means of mostly wirelesscommunication. Overall, the HCI is simplified by autonomousadaptation to the users’ situation and demands – and not viceversa. UbiComp systems aid in overcoming the problem ofinformation overload.

When introduced, these UbiComp concepts appeared asa science fictional long-term target. However, due to theintermediate technological progress in increased processing,sensing, memory and communication capabilities, the visionhas already become true to a certain extent. The size ofhandheld devices, environmental sensors and smart-its [14] hasdecreased continuously due to the advancements in solid-statetechnology. Numerous protocols and radio access technologieshave been developed for supporting wireless communicationin various ranges (e.g. Wi-Fi, IEEE 802.11; Zigbee, IEEE802.15.4; Bluetooth, IEEE 802.15.1; 3GPP 3G/LTE). Today,smartphones, wireless sensors and networked desktop com-puters can easily be interwoven by always-on connectivity.However, there are a number of barriers to be overcome beforetruly context-aware applications and services can be deployedand made widely available.

Collect

Act &Adapt

Decide

Analyse &Aggregate

PhysicalSensors

VirtualSensors

UserProfiles

LogicalSensors

ContextProcessing

Context Modelling

ContextManagement

Reasoning/Inference

RiskAnalysis

DecisionTheoryHypothesis

generation

Mobile Device Actuation

ContentTagging &

Lookup

UserAssistance

Environment Actuation

Feature Extraction

Application & Service Logic

Context Provisioning Middleware

Fig. 3. The Context-Aware System Cycle (adapted from [12])

B. Context

According to Dey [2], context is any information that canbe used to characterise the situation of an entity (person,place, physical or computational object) that is consideredrelevant to the interaction between entity and application [2].This definition is by far the most cited in the literature.Zimmermann proposes five fundamental categories of contextinformation: time, location, activity, relations and individuality[15].

Regarding the definition of a situation in context-awaresystems, different views exist in the research community.Zimmermann defines it as “the state of a context at a certainpoint (or region) in space at a certain point (or interval) intime, identified by a name” [15]. Being a structured repre-sentation of a part of the context, it can be compared to asnapshot taken by a camera. Location and time can be usedas spatio-temporal coordinates. Giunchiglia follows a morephilosophical approach and sees context as a “subset of thecomplete state of an individual that is used for reasoning abouta given goal” [16]. Situation is then “the complete state ofthe universe at an instant of time”. With regard to situation-awareness, Billings [17] defines a situation as “an abstractionthat exists within our minds, describing phenomena that weobserve in humans performing work in a rich and usuallydynamic environment”. In summary, a situation may containan infinite variety of contextual information.

Context can be classified into the following (incomplete)list of elements:

• Spatial context: information about the location, e.g. abso-lute geographic coordinates, relative physical proximity(distance), street, city, main business of places in prox-imity (e.g. shopping mall, university campus);

• Temporal context: information about the absolute time,

relative time, day time (e.g. morning, afternoon, evening),weekend vs. business day, season;

• Device context: information about the users’ interactiondevice(s), i.e. processing capabilities, input sensors, visu-alisation capabilities (e.g. supported codecs, screen size);

• Network and communication context: information aboutnetwork characteristics, e.g. Wi-Fi access points in prox-imity, available bandwidth and throughput, supportedQuality of Service (QoS) class, delay, transmission costs[18];

• Environmental context: information referring to the phys-ical environment of an entity, e.g. noise level, air pressure,light intensity, pollution;

• Individuality and user profile context: information aboutthe preferences, interests and habits associated touniquely identified users;

• Activity context: information about what an entity does,which task it is currently involved in and what it intendsto do next;

• Mental context: information about internal states of mind,e.g. a user’s feelings and mood, level of stress;

• Interaction context: interaction may comprise both socialinteraction between several users and interaction betweenusers and an application or service.

Interestingly, some of these context categories are directlyor indirectly related to each other. For instance, consideringthe contextual illustrations from Fig. 4, the spatial contextinfluences the temporal context since time of day directlydepends on the current time zone. Various abstraction levelsare also notable within most of the presented context types,e.g. some of the context elements associate information to anytype of entity (user, device, room) whereas other elements areonly applicable to selected sets of entities. Location, identity,time and activity are considered being the most important [19]

Location

Proximity

Street/ business

Time of day

Day of week

Day of month

Season Noise

Brightness

Pollution

Network Sensors

Status/Activity Settings

Weather

QoS Social profiles

Emotional state

Activities

Spatial Temporal Environmental

Device/Network Physical/Mental User Profile

Fig. 4. Examples of various types of personal context

1998 2001 2004 2007 2010

Locati

Useran

d hisen

e.g. loc

, time,

rature

vices,

infras

tructu

d envir

t [22]

Charac

terist

tity’s

tion [2]

Locati

on, en

vices,

User’s

e and ac

tivity

) [15]

Knowled

terist

tity, dig

ital su

Fig. 5. A partial timeline of the evolution in the definition and understandingof ‘context’ (human-computer interaction) in literature.

in contemporary literature and context-aware systems. Fig.5 illustrates a partial evolution map, according to publishedliterature, of the definition and understanding of ‘context’ inhuman-computer interaction. In the scope of this article, wedefine context as follows:

Context is any information that provides knowledge andcharacteristics about an entity (a user, an application/service,a device, or a spatially bound smart place) which is relevantfor the interaction between the entities themselves and withthe digital world. Context can be categorised as being static,dynamic and rapidly changing.

C. Context-Awareness

The ability of a service, application or actuator to adapt to aspecific context is referred to as context-awareness. For a pieceof data to contribute to the adaptation ability of an applicationor service, it has to go through to complete lifecycle ofacquisition, reasoning, representation, organisation, and thenbe communicated to the application or service to act uponit. These lifecycle stages are illustrated in Fig. 6. Given theclassification of context types (cf. Section II-B), awarenesscan be categorised accordingly to define the specific focus,e.g. location-awareness, network-awareness. The term context-awareness was first introduced by Schilit and Theimer [20].Typically, a context-aware system follows the autonomic com-

Acquire Reason Represent Manage Act

Source data (or low-level context)

Aggregated information

Uniformly formatted context

Organised context (storage, historic)

Communication and actuation

states

ting s

Fig. 6. Lifecycle stages that a context datum goes through, with respect tothe functional processes in a context middleware.

munication feedback loop presented in Fig. 3 and comprisesthe following tasks [12], [15], [24]:

1) Perception/Acquisition, i.e. collection of relevant dataallowing conclusion to the context;

2) Reasoning/Inference, i.e. deduce more meaningful infor-mation from the raw data;

3) Learning from historic context information and actions;4) Context Representation, i.e. representation and mod-

elling of context information in a machine interpretableway;

5) Management and Diffusion of context information;6) Actuation, i.e. Triggering/adapting the service execution

or application behaviour based on the available contextinformation.

Moreover, three different categories of adaptation can beidentified [19]:

1) presentation of information and services to a user, i.e.information is filtered and services are selected basedon users’ context;

2) automatic execution of a service and adaptation of anactuator, i.e. the service execution logic and actuationbehaviour depend on users’ context;

3) tagging of context to content (or information in general),i.e. context is associated to content in order to retrieveit more easily later on.

In order to take various levels or complexity into accountthe following definition of context-awareness is proposed.

The adaptive behaviour of services, applications and actu-ators according to the detected context depends on the degreeof consciousness and context complexity. It can be subdividedinto three different stages: Context-based adaptation refersto applications querying context on-demand synchronouslyand changing the execution based on current context pa-rameters. Context-aware adaptation requires an event-basedasynchronous context diffusion in order to allow the executionlogic to adapt on sensed and propagated events of interest.Situation-aware adaptation is more complex and requires(primitive) context to be further aggregated and high-levelcontext to be inferred.

The different stages of this adaptive behaviour can beexplained with the following example: consider an applicationon a user’s mobile device that can notify the user about nearbyrestaurants in a certain location. Such an application may pollthe on-device GPS for coordinates and the time service for thetime of the day. When certain conditions are met, e.g. time ofday that can be considered a meal time and location is known,the application can query a location information service fornearby restaurants and notify the user. We term this behaviour

context-based because the application itself is not aware ofthe user’s context, but is merely looking for information thatcan fire certain rules in its logic.

The next stage of context-awareness comes when such anapplication no longer actively engages in information pollingbut rather registers its interest in particular types of contextinformation regarding an entity and is notified asynchronouslywhen certain conditions are met. It becomes aware of thevarious context elements when they exist. While these types ofapplications are economical in their logic flow and execution,they do however depend on the availability of asynchronousevent registration and notification in the middleware infras-tructure. For such applications to be practically useful, thereneeds to be a higher level of contextual understanding, e.g. theuser may not be in a situation to have lunch (driving, meeting,may already have had a meal).

A situation-aware adaptation scenario means that the ap-plication will base its recommendation (or lack thereof) onadditional information which may include establishment ofthe user’s current activities, historic context regarding mealstaken at this time and place in the past, his social situation,list of commitments from his digital calendar, etc.

The review of existing applications that provide contextrelated facilities, both research based prototypes and widelyused smartphone-based applications, suggests that the currentstage is predominantly context-aware adaptation.

III. CONTEXT MODELLING & REPRESENTATION

Context modelling is the process of designing a model ofreal world entities, their properties, state of their environmentand situations that can be used as a reference for acquir-ing, interpreting and reasoning contextual information. Thepurpose of creating a context model for use in a context-aware middleware is to provide a uniform, machine process-able context representation scheme, facilitate context sharingand inter-operability between different middleware layers andexternal applications. The uniformity of the model asserts acommon understanding between all software components andapplications in the middleware. This common understandingis vital because the acquisition, reasoning and utilisation ofcontext information is usually done by separate components,layers or applications in the middleware. Moreover, additionaldemands of a context model are raised due to the temporalnature of context, variable certainty of sensed information inthe environment and about users, the fluid nature of relation-ships between entities as well as the availability of new typesof knowledge about the environment.

Context modelling techniques and requirements have beensurveyed by Strang and Linnhoff-Popien [25] and Bettini etal. [26]. Context information needs to be represented andmodelled for being machine interpretable and exchangeableusing well-defined interfaces. The goals are to support easymanipulation (low overhead in keeping the model up-to-date),easy extension (cheap and simple mechanism for adding newtypes of information), efficient search and query access aswell as scalability. A number of different approaches for rep-resenting contextual knowledge and semantic information canbe found in literature. On the one hand the representation is

tightly coupled to the inference mechanism, e.g. probabilisticlogic requires the modelling of probabilities. On the other handthe representation is often tailored to the problem domain andto the specific goal of the system.

Strang and Linnhoff-Popien [25] identify generic require-ments: The modelling approach should (1) be able to cope withhigh dynamics and distributed processing and composition,(2) allow for partial validation independently of complex in-terrelationships, (3) enable rich expressiveness and formalismfor a shared understanding, (4) indicate richness and qualityof information (QoI), (5) must not assume completeness andunambiguousness, and (6) be applicable to existing infrastruc-tures and frameworks.

A. Classification of Context ModelsBolcini et al. [27] provide a framework for the analysis of

context models that caters for modelled aspects, representa-tion features, usage and context management features. Theyanalyse existing context modelling efforts and, based on theirlevel of fulfilment of analysis parameters, they are classifiedinto five ‘classes of use’ that include Context as a matter of (1)channel-device presentation, (2) location and environment, (3)user activity, (4) agreement and sharing and (5) data/serviceselection.

Moreover, context models can be classified into six differentmodel categories, namely key-value models, markup schememodels, graphical models, object oriented models, logic basedmodels and ontology based models [28]. Recently, chemistryinspired models have been proposed [29] as well. Table Iprovides an overview of the individual advantages and dis-advantages of different context modelling schemes, which arefurther discussed in the following paragrahs.

Key-value pairs form a simple tuple of information. Thecontext information is assigned to a unique key in order to al-low for easy lookup by applying a matching algorithm. Thesepairs are easy to manage but lack structuring and therefore donot allow for efficient context retrieval. Markup scheme modelsincorporate a hierarchical data structure of markup tags, at-tributes and content. Examples include the User Agent Profileand the Composite Capabilities/Preference Profile (CC/PP)[31], which are based on XML (Extendible Markup Language)and standardised by the World Wide Web Consortium (W3C).The Context Meta Language (ContextML) [30] is anothermarkup based scheme that not only represents the contextinformation but context metadata as well.

Graphical models (e.g. based on the Unified ModellingLanguage) allow for a picturesque description of a contextmodel [36] and for deriving an Entity Relationship model asrequired in rational databases. An extension is proposed byHenricksen and Indulska [32], introducing Object-Role Mod-elling (ORM). Object oriented models offer powerful capabil-ities of inheritance, reusability and encapsulation. Access ofcontextual information is provided by well defined interfaces[37]. Logic based models offer a high degree of formalism andtypically comprise facts, expressions and rules. They enableformal inference, e.g. by means of general probabilistic logic,description logic, functional logic or first-order predicate logic.

Ontological modelling refers to an abstract conceptual vi-sion of the world. The relations within could also be de-

TABLE ICONTEXT MODELLING AND REPRESENTATION APPROACHES

Context ModellingApproach Advantages Disadvantages H

Examples

Key-Value Fast processing, storage and lookup Lack of data structure � / / / /

Markup Scheme

Structured representation; fast reason-ing; schemes allow for a commondefinition and knowledge of availableparameters

Data processing slows down with in-creasing size of the scheme (�) � (�) (�) /

ContextML [30],CC/PP [31]

GraphicalReadable by machines and humans;clear relations between model compo-nents

Implementation of graphical modelstypically requires a technology model(e.g. object oriented design) for imple-mentation

(�) (�) (�) / ORM [32]

Object OrientedSupport of coalgebraic and abstractdata types, recursive types, encapsu-lated states, inheritance

Data exchange in a distributed mid-dleware requires an object orientedcommunication protocol

(�) / (�) � / JCAF [33]

Logic based Fast processing, high degree of for-malism

Typically restricted to the targeted rea-soning mechanism (cf. Section V);possible need of model conversion

� � /

Ontology basedImplicit reasoning mechanisms; co-herent and unambiguous knowledgerepresentation

A large ontology slows down the rea-soning; domain experts and ontologyengineering are required

/ (�) � / /Wang et al. [34],CoBrA [23], SOCAM[35]

Chemistry inspired

Ability to fuse environment anduser context to trigger services au-tonomously; similarity to chemicalbonding and chemical reactions

Complex model requiring event-basedcontext propagation (�) � (�) / / Ikram et al. [29]

* �= support; (�) = limited support; / = no support. Empty fields indicate a significant dependency on the chosen implementation.

scribed by object oriented methods. However, an ontology iscommonly described by using languages standardised by theW3C in the context of the semantic web. Most relevant arethe Resource Description Framework Schema (RDF-S) [38]and the Web Ontology Language (OWL) [39].

Korpipaa and Mantyjarvi [40] enumerate the followinggoals for designing a context ontology: simplicity, flexibility,extensibility, generality and expressiveness. Many researchershave come to the conclusion that ontologies are theoreticallythe best way to represent and model context due to theextendibility and unambiguousness [28], [34]. However, theremay be certain drawbacks as ontology engineering is a chal-lenging and interminable matter. With the size of the ontology,querying and processing the information embedded within be-comes slow, in particular if performed on resource constrainedmobile devices. The context model can be arranged in layersto cushion this effect. Wang et al. [34] propose ontologymodularisation, i.e. a generic upper ontology on top anddomain specific ontologies below. Full featured ontologicalrepresentations tend to decrease the inference performanceand are not suited for highly dynamic systems. Especiallyif resource constrained mobile devices are envisaged as themain source and consumer of context an appropriate alter-native must be chosen. Another argument for not applyingontological representation is its limited support for modellinguncertain and unavailable data.

B. Context Meta Data

Beside the context information itself, a model for represent-ing related meta data is crucial. Such context meta information

Context InstanceContext Instance

Context ScopeContext Scope

Context DataContext Data Context Parameter (Simple)

Context Parameter (Array)Context Parameter (Array)

Associated EntityAssociated Entity

Validity PeriodValidity Period

Entity TypeEntity Type

Detection TimeDetection Time

Expiration TimeExpiration Time

ProcessingProcessing Sensor SourceSensor Source

Processing ComponentProcessing Component

Entity IDEntity ID

Context Meta DataContext Meta Data

Quality of Context IndicatorQuality of Context Indicator

Context Parameter (Struct)Context Parameter (Struct)

1 0..n

Fig. 7. Context meta data in the ContextML schema [30])

may include a quality of information quantifier, the degree ofuncertainty, possibility, measurement accuracy, resolution orconfidence interval. As an example, Fig. 7 depicts the contextmeta data elements of the ContextML schema [30].

These attributes are especially helpful for certain inferenceand reasoning mechanisms (cf. Section V). When taking his-toric context into account, it is important to embed data relatedto the temporal domain such as time of detection and expirytime. Rapidly changing information can be differentiated fromrather static information (such as gender, year of birth). In ad-dition, if tracing back context inference processes is required,both context sources and context processing entities may becaptured and stored as meta data. Crucial – but out of scope ofthis article – is privacy and security information [41]. In their

work, Chang et al. [42] propose the modelling of a lifecyclefor contextual information and an appropriate representa-tion in meta data. The state of contextual information (e.g.ready, running, expired, suspended) enables flexible and fasttransitions when the context changes temporarily. However,other models such as ContextML only utilises two states ofcontext information; transitions only occur unidirectionallyand ultimately from valid to historic context. The followingdefinition clarifies this differentiation.

Context meta data refers to information explicitly attachedto the context data. Meta data may be subdivided into manda-tory and optional parameters and may comprise the detectiontime, validity period, source of information, quality of context,probability or uncertainty, associated entity, applied process-ing, etc.

A context instance refers to a specific instantiated objectof contextual information, including meta data. Due to limitedvalidity of context, each context instance only remains validfor a specific period of time. Any expired context instance isconsidered to be historic context.

IV. CONTEXT MANAGEMENT & PROVISIONING

The management and provisioning of context informationare essential elements for realising context-aware servicesand applications. A notable number of context managementapproaches have been presented; surveys of which have beenpublished for instance in [28], [43], [44].

A. Functionalities

The overall management is usually subdivided into severalfunctional tasks as illustrated in the core layer of Fig. 8. Theaccess to contextual information is typically aided by a contextmanagement system, toolbox, framework or middleware en-abling services and applications to acquire and utilise context.

Sensor Data Acquisition deals with how raw informationabout any context is fetched and used as input to the middle-ware. It is important that the system can cope with a varietyof heterogeneous sources and sensors simultaneously. Sensorsmay be physical, virtual, or logical in nature. Depending onthe intelligence and computational power, preprocessing andfiltering may be performed by the sensor nodes themselvesor as part of the middleware functionality. Both synchronousand asynchronous sources are generally supported. The benefitof Context Storage is twofold. Caching strategies allow forfaster provisioning of the required context since repeatedprocessing stages may be omitted. Moreover, the storage ofexpired context in a history database enables the analysis ofprevious situations. Such information can be used to determinehabits and long-term intentions taking successive sequences ofactivities towards the desired goal into account.

Context Lookup & Discovery provide means for an applica-tion, service or actuator to identify the available context andhow to acquire and query for it. Commonly used approachesinclude lookup tables, semantic queries or web service mech-anisms such as SOAP (Simple Object Access Protocol) andWSDL (Web Services Description Language). Context Diffu-sion & Distribution are related to the output of a middlewaresystem, i.e. how context information is made available to

the consumers. This encompasses not only the definition ofquery models (e.g. key-value based or SQL based) but alsothe mode of communication. Communication protocols maysupport event-driven asynchronous publish/subscribe mecha-nisms to notify the application layer about context changesof interest. Additionally, synchronous on-demand queries maybe supported by the middleware. Since context reveals privateinformation about users, Privacy, Security and Access Controlare crucial tasks in every middleware. However, due to theselected scientific focus, they are not considered in-depth inthis article.

Context Processing & Reasoning refer to the capabilityof inferring context from raw sensor data or from existingprimitive low-level context. The middleware may apply featureextraction, description logic, rule-based reasoning or proba-bilistic inference on behalf of the application layer, hencesaving battery consumption on mobile resource constraineddevices. A powerful middleware has to support modularity sothat numerous processing mechanisms and algorithms can beplugged in. Details of processing and reasoning techniquesemployed in contemporary systems are discussed later inSection V.

Functional, spatial, synchronisation and temporal decou-pling of these elementary tasks is targeted in most of thearchitectural principles as further discussed below. Functionaldecoupling is achieved whenever functionalities can be in-voked independently from each other. Spatial decouplingforesees a sender being unaware of the receivers’ address orpresence. Synchronisation decoupling prevents blocking in thesender and receiver components when exchanging context sothat their main flow of execution can continue to be carriedout. Temporal decoupling requires senders and receivers ofcontext not to be involved at the same time.

B. Existing Classifications

Hong et al. [44] classify context middleware approaches intothe following six categories: (1) agent-based, (2) reflective,(3) metadata based, (4) tuple space based, (5) adaptive andobjective based and (6) Open Services Gateway Initiative(OSGI) based. However, this grouping mixes up architecturalapproaches, context representation and applied technologies.Baldauf et al. [28] differentiate between (1) direct sensoraccess, (2) middleware infrastructure and (3) context serveras architectural models. Hence architectural design is focusedwithout going into detail about technological details. Wino-grad [45] proposes a distinction between (1) widgets provid-ing interfaces for hardware sensors, (2) networked servicesresembling the context server architecture and a (3) blackboardmodel representing a data-centric view.

Regarding the overall system design, Hong et al. [44] rec-ommend four layers (bottom-up): (1) Network InfrastructureLayer, (2) Middleware Layer, (3) Application Layer and (4)User Infrastructure Layer. Baldauf et al. [28] identify fivelayers focussing on functional stages: (1) Sensors, (2) Rawdata retrieval, (3) Preprocessing, (4) Storage/Management and(5) Application. Zimmermann [15] derives a layered frame-work comprising (1) Sensor Layer, (2) Semantic Layer, (3)Control Layer and (4) Actuation Layer where only the first

SENSORS

MIDDLEWARE

APPLICATIONSContext

Lookup & Discovery

Context Processing & Reasoning

Context Storage

Physical Sensors

Virtual Sensors

Logical Sensors

Physical Actuators

Media Tagging

Service Composition

Context-Aware Mobile Applications

Service Adaptation

Context Diffusion & Distribution

Privacy, Security and

Access Control

Sensor Data Acquisition

Sensor Gateways

Fig. 8. Middleware Functionalities

two layers are directly related to context management andprovisioning and the latter two deal with decision logic andservice adaptation.

Makris et al. [46] identified six categories of abstract contextaware functionalities. These functionalities are (1) contextacquisition, (2) context modelling, (3) context exchange, (4)context evaluation, (5) exploitation of context from businesslogic perspective and (6) context aware horizontal functional-ity dealing with security, privacy and trust issues.

Based on these existing classifications, the following sub-sections discuss concrete design approaches, categories ofarchitectural models and relevant implementations. We presentan overview of selected context management frameworks andmiddleware solutions, classify them according to our proposedcharacteristics and furthermore review their successive evolu-tion.

C. Design-based Classification of Context Provisioning Mid-dleware

As discussed in the previous paragraphs, a number ofdesign approaches have been attempted for the collectiveprovisioning of context-awareness related functions. Thesedesign approaches range from layered middleware designsto object-oriented and event-based middleware designs. Dif-ferent communication and coordination semantics betweenindividual components of the logical design result in variedarchitectures of context-aware systems as well. The layeredmiddleware design is prominent, where each functional layerhides the details of the underlying layers. The primary benefitof this approach is the encapsulation of varying complexitiesof different functions. Each layer builds on the informationavailable from the layer below it, e.g. a context processinglayer uses data collected at the data acquisition layer while theapplication platform layer interacts with the context processinglayer to retrieve context and does not concern itself with thedetails of data acquisition or synthesis process. Within thisdesign approach, the provision of functions is often separatedinto different architectural components, e.g. some functionsare provided by a central server while applications that usethese functions are deployed remotely. SOCAM [35] is aprominent example of a layered middleware design.

Some examples of context-provisioning middleware adoptan object oriented model, where functions of context-awareness are assigned to separate software objects. These

objects encapsulate the internal processing of the functionsand provide interfaces for interaction with other objects. Theprimary difference between the layered middleware and theobject-oriented design is that in the layered design the contextprocessing pipeline is vertical and operates in series amongstthe various layers, whereas an object-oriented approach allowsdifferent functions to interact ‘out of series’ and horizon-tally. This feature is demonstrated in CORTEX [47], whichshowcases the concept of sentient objects enabling context-awareness in ad-hoc environments by exchanging contextamongst themselves. Object oriented design also providesdevelopers with the benefit of extensibility through poly-morphism and inheritance, as demonstrated in JCAF [33],which exploits these benefits to provide an infrastructure and aprogramming API for developing and deploying context-awareapplications and services.

Event based middleware design is conceptually closest tothe nature of context processing because human context andthat of related computing entities is naturally reactive andfluctuates with stimuli. Context-aware middleware designedwith this approach operate on the production, detection, con-sumption of, and reaction to events. An event in such asystem is any change in state of any user or device context.System components are usually divided into context producersthat generate context related events and context consumersthat react to contextual events. An event service is usuallyemployed to process and route events between consumers andproducers and may also perform filtering and transformationon context events in complex context-aware systems. Eventconsumers register their interest in particular events eitherdirectly with the event producers or with the event service.Publish/subscribe and message notification based communi-cation semantics are used in the middleware developed usingthis design approach. Event based design affords better respon-siveness in context-aware systems as they are by design morenormalised to unpredictable and asynchronous environments.

Different middleware designs exist due to the different ap-proaches used by designers to realise the functions of context-awareness (acquisition, reasoning, communication etc.). Wehave classified some of the contemporary middleware accord-ing to their design philosophy (cf. Table II). The realisationof a design approach is carried out by specifying functionalresponsibilities to architectural components of a software sys-tem. While most of the existing middleware can be categorised

Sensors

Context

Application

Computing Device

Fig. 9. Direct sensor coupled application running on a single computingdevice

under one of the listed design categories, there is a furthercategorisation that can be elicited by examining these systemsfrom a deployment and component architecture perspective.This architectural categorisation, described in the followingparagraphs, is primarily based on how the components thatcarry out different context-awareness functions in the middle-ware interact amongst each other to achieve the overall goalof context provisioning.

D. Architectural Classification of Context Middleware

In some middleware systems, context related applicationsoperate by directly accessing the sensors states on whichthe context information is based (direct sensor coupling).Generally, in such middleware a context-aware applicationdirectly consumes information retrieved from sensors andthere are no dedicated context reasoning, communication orcoordination functions. As illustrated in Fig. 9, the role of themiddleware is limited to that of a hardware abstraction layer,with no other specialised functions available.

In context server based architectures, a central server per-forms the functions of collecting and synthesising contextfrom sensor data and other information sources. As illustratedin Fig. 10, clients of the server access it remotely to retrievecontext or raw data to process locally. Therefore, the contextserver performs most of the middleware functions. This isone of the most common architectures used in context-awaresystems. One of the reasons for the prevalence of this archi-tecture is the functional flexibility it affords to designers. Forexample, the central server may limit its functions to mere dataacquisition or it may provide context synthesis and applicationplatform on top of it.

In a peer-to-peer architecture the functional tasks ofcontext-awareness are carried out by peer components, eachpeer acting as both a server and a client. It is a distributeddesign approach where peers either individually carry outmutually exclusive subsets of context-awareness related func-tions or replicate the functionality in different geographicalor logical domains. These peer components usually utilise acentralised server for coordination of context with context con-suming applications e.g. Hydrogen [37] and Context Toolkit[48]. Server-less peer-to-peer architectures also exist (JCAF[33], CORTEX [47]) but may suffer from limitations due to

Context Storage Applications

Context Lookup and Discovery

Central Server

Context Processing and Reasoning

Context Diffusion and Distribution

Fig. 10. Central server based middleware architecture

the lack of a coordinating component e.g. the requirement ofhard-coding addresses of communication endpoints betweencontext services and clients in JCAF.

Different approaches to designing context-aware systemsexist due to the constraints imposed by a variety of factors.Leading factors effecting the design decision and resultantarchitecture include location of sensors, number and mobilitypatters of users, available resources, targeted scale of the sys-tem and methods of context acquisition and distribution. Directsensor coupled architecture has been used when sensors andcontext consuming applications are accessible within a singlesystem (e.g. a mobile device or a desktop computer) and thereis no pressing requirement for complex context reasoning.However, due to this tight coupling, the scale, scope and us-ability of these middleware to be exploited as holistic context-aware systems is severely restricted. Most sensors have limitedcommunication range and software components that accessdata from such sensors have to be located physically close tothe sensors. These limitations triggered the evolution towardsa context server approach where a central server acquires,processes and stores context while providing interfaces forlocal and remote applications to access context. Context serverarchitecture allows reuse of sensor data and relieves resourceconstrained devices from context acquisition and processing.While leveraging these benefits, context server architecturerequires consideration of new factors in the design that includeselection of appropriate network protocols, quality of serviceparameters, network performance, mobility management etc.

The basic central server architecture allows remote compo-nents to access context but the data acquisition function is stillrestricted by the limitation in communication range of sensorsand other information sources. To overcome this shortcoming,the central server architecture has evolved further in terms ofdistribution of its constituent components, e.g. distribution ofthe data acquisition function into multiple remote data acquisi-tion modules that acquire data from their assigned sources andpush it to the central server. This further distribution of func-tionality, illustrated in Fig. 11, has transformed context-awaremiddleware systems into truly distributed systems where eachfunction may be hosted on different machines in a network anda central server coordinates the flow of information and controlbetween these components. Other factors that have influencedthe adoption of this architecture include availability of a largenumber of distributed information sources and sensors, ded-

Applications

Context Lookup and Discovery

Context Storage

Central Server

Context Diffusion and Distribution

Context Processing and Reasoning

Fig. 11. Central server based middleware architecture with distributedcomponents

icated reasoning components, mobility of modern day usersand abundance and increased usage of mobile devices. TableII presents the architectural approach based categorisationof context-aware systems that have been classified earlieraccording to their design approaches (cf. Section IV-C) andprovides a consolidated view of the overlap between designand architectural approaches of these systems.

E. ImplementationNetwork based frameworks have utilised RMI (Java Remote

Method Invocation) or CORBA (Common Object Request Bro-ker Architecture) standards as technological basis for interac-tion. Examples for RMI comprise JCAF (Java Context Aware-ness Framework) [33] and SOCAM [35]. CORBA has beenused in Gaia [53]. Both CORBA and RMI can be considered asrudimentary technologies for realising distributed networkedapplications. Because of the relatively low abstraction level,it may outperform newer technologies as far as speed and alow footprint are concerned. However, development is morecomplex and the interoperability limited.

The OSGI framework is an open modular service plat-form based on Java that implements a dynamic componentmodel as an extension to the standalone runtime environment.OSGI interface principles have been applied in [59] and [60].Multiagent systems (MAS) evolved from Distributed ArtificialIntelligence and are comprised of individual agents. Each isconsidered a locus of problem-solving activity. By operatingasynchronously it has a certain level of autonomy and in-telligence. Cooperative agents collaborate towards achievingcommon goals [61]. Hence, agent-based systems have alsobeen successfully adopted in context provisioning middlewaredesigns, for instance in CoBra [62] and EgoSpaces [63].

Moreover, web service technologies have recently receivedgreater attention in development of context middleware. Webservices utilise standard protocols, such as HTTP (HypertextTransfer Protocol) and SOAP (Simple Object Access Proto-col). In addition, WSDL (Web Services Description Language)may be applied for describing the interfaces of architecturalcomponents semantically. In the traditional client-server view,interacting services invoke each other, the requesting one be-comes a client of the invoked service [43]. A full level of web

service implementation has been used in CA-SOA (Context-aware Service Oriented Architecture) [64], CoSWAMI [55],ESCAPE [65], inContext [66] and Omnipresent [67]. Theprimary advantage of web service technologies is their wideinteroperability across networks and devices. RepresentationalState Transfer (REST) [68] compliance allows for statelessinformation exchange, e.g. based on XML. These conceptsare not only applicable for context-aware web services butalso for other types of actuators and ubiquitous computingapplications.

F. Overview of Related Frameworks and Middleware

Table III provides an overview of related systems andtheir historic evolution. Furthermore, major achievements andlimitations are discussed in the following paragraphs.

ActiveBadge [50], one of the earliest localisation andcontext-aware systems, is based on a centralised locationserver and communicates through Remote Procedure Calls(RPC). Mainly due to its very limited scope, the heterogeneityof networks, context consumption devices and sensors is notsupported. Context representation is proprietary and detailsare not specified in related literature. The support for sensor,context and entity diversity as well as model extendibilityand scalability are weak since only localisation of infra-redbased sensors embedded in badges is targeted. The ActiveMap[20] system is comparably limited and again, location is thefocus. Due to utilisation of the publish/subscribe paradigm,the context management scalability capability is supposed tooutperform ActiveBadge, albeit no specific or comparativeperformance analysis has been published. Alongside users,the concept foresees support of generic objects. Hence, entityheterogeneity is addressed, however not discussed in detail.Localisation is arranged in hierarchies, for example room,building, regions. In addition to basic RPC communication,infra-red based multicast is supported. Cyberguide [49] wasdeveloped as a rapid prototyping system for assessing theutility of context-awareness in mobile devices and focussedon location and orientation context of users. Infra-red basedbeacons and GPS sensors are used for estimating the positionof a device. Support of sensors, context and context processingcapabilities are very narrow and static. GUIDE [57] is anotherframework supporting location-based travelling. The system,which uses HTTP-based communication, demonstrates animprovement over its predecessors by applying a distributedarchitecture in which separate servers manage individual cells.

The systems mentioned above have been primarily designedto illustrate the benefit of context-awareness based on aselected usage scenarios. Being very restricted in functionality,degree of awareness as well as acquisition and processingof contextual knowledge, they can be classified as early sys-tems that increased interest and understanding of ubiquitouscomputing. However, they cannot be referred to as contextprovisioning middleware. The need for generic and systematicsupport of context-aware applications and services emergedslowly and has been addressed in later systems.

The Context Toolkit [48] is one of the first demonstrationsof utilising a complex combination of different types of usercontext information. Its architecture is based on distributed

TABLE IICONTEXT-AWARE MIDDLEWARE DESIGN APPROACHES AND SYSTEM ARCHITECTURES MATRIX

Layered Object Oriented Event Based

Direct Sensor Coupling Cyberguide [49]

Central ServerActive Badge [50] CASS [51]CoBrA [23] Active Map [20]OPEN [52]

Central Server withDistributed Components

Gaia [53] C-CAST [54] C-CAST [54]PACE [32] CoWSAMI [55] ACE [56]SOCAM [35] GUIDE [57]

MobiLife [58]ACE [56]

Peer-to-PeerCORTEX [47] CORTEX [47]

Hydrogen [37] Context Toolkit [48]JCAF [33]

TABLE IIICONTEXT MANAGEMENT & PROVISIONING FRAMEWORKS

Middleware/Project Com

Active Badge [50] RPC ? / / / / / / /Active Map [20] RPC & o ? / / / (�) / / (�)Cyberguide [49] ? ? / / / / / / /GUIDE [57] web ? / / / / / / (�)Context Toolkit [48] web M � � (�) / / (�) (�)Gaia [53] RPC, CORBA o � � ? � ? (�) (�)CoBrA [23] ag, (web) Ont � � / ? ? / (�)SOCAM [35] RMI, OSGi Ont � � ? ? ? ? ?CASS [51] ? ? (�) (�) ? / ? ? /CORTEX [47] o M � (�) ? � ? (�) /JCAF [33] RMI OO � � (�) � ? (�) /PACE [32] RMI, web OO, M, G � � (�) ? ? (�) /CoWSAMI [55] web M � � (�) (�) ? (�) /MobiLife [58] web Ont � � � ? ? (�) (�)OPEN [52] ? Ont � � / ? / (�) ?ACE [56] web Ont � � (�) ? / / ?

‡ web = web based; ag = agent based; RPC = Remote Procedure Calls; RMI = Remote Method Invocation; CORBA = Common Object Request Broker Architecture; OSGi =OSGi based; o = other; ? = unknown, i.e. not discussed in available literature.

§ Ont = Ontology; M = Markup Scheme; G = Graphical; OO = Object Oriented; o = other; ? = unknown, i.e. not discussed in available literature.* �= support; (�) = limited support; / = no support; ? = unknown, i.e. not discussed in available literature.

widgets that hide complexity and heterogeneity of sensors. Thehybrid design further encompasses a central context interpreterand server. Context is represented by using a custom XMLschema. Subscription based context updates are communicatedbased on the TCP/IP networking suite through SMTP (SimpleMail Transfer Protocol) or HTTP. While the system is by de-sign not intended to be used over a large scale and is focussedon utilising context information bound to a geographical area,physical and functional scalability are partly supported.

Gaia [53] is the first context provisioning and managementframework entitled as a middleware by the authors. It modelsa smart space as a programmable entity, providing supportto context based applications via well established operatingsystem concepts. Gaia applies I/O operations and file systemmanipulation for interconnecting active space objects. Event-based communication is established through both RPC and

CORBA. Context diversity originates from the so called Con-text Providers and covers location, room conditions, weather,stock prices and executing applications. Gaia has not beenassessed in terms of large-scale use. The authors mentionfederating multiple active spaces but such an attempt has notbeen documented.

CoBrA [23] is architecturally based on distributed agentsbeing organised by a central broker. Context is representedin ontologies (RDF/OWL). The broker is responsible forproviding a shared model of context, removing inconsistenciesand masking heterogeneous context sensing sources. Com-munication is established through the Agent CommunicationLanguage (ACL) and HTTP. The concept of compartmen-talising context into domains exists in CoBrA but physicalscalability has not been evaluated. Functional scalability isproblematic due to a tight coupling of applied technologies

and the resulting need for following these strict guidelines.SOCAM [35] targets providing an architecture specified for

rapid prototyping of context-aware services. The frameworkencompasses Context Providers, Context Interpreter, ServiceLocation Service and Context-aware Mobile Services. ContextProviders acquire primitive data which is further processedby a central Context Interpreter. Context is modelled basedon ontologies (RDF/OWL). SOCAM is built on top of anOSGi service platform and also utilises RMI, hence facilitatingservice discovery and management through these technologies.The availability of both on-demand queries and subscriptionsis only briefly mentioned without any details and, therefore,the scalability capabilities are difficult to estimate.

CASS [51] focusses on providing higher level context ab-stractions through a knowledge base, sensor listener, contextstorage database and rule engine (for reasoning) deployed ona server. Context-aware applications on mobile devices canlisten for relevant changes in context data propagated throughthe central database. The rule engine monitors such modifi-cations accordingly. Context representation is not discussed,however queries are realised through SQL. While prototypeapplications demonstrate feasibility of CASS in keeping re-source intensive context processing tasks from mobile devices,it is not designed to be scalable. Firstly, both sensor nodes andmobile devices have to communicate with a central CASSserver and no discovery mechanism is built in. Secondly,support for mobility is lacking.

CORTEX [47] proposes a sentient object model to supportthe construction of ubiquitous applications in wireless mo-bile ad-hoc networks. Sentient objects are defined as entitiesconsuming, processing and producing events and effect actu-ators. Therefore, a publish-subscribe component for discoveryof neighbouring components and sharing data is provided.Context is represented in XML and serves as input andoutput of sentient objects. While CORTEX is suitable for anumber of mobile entities interacting with sentient objects,this communication is specifically designed to work in ad-hoc environments focussed around a particularly boundedarea. Event propagation is limited by the range of multicastprotocols, hence scalability is very limited.

The JCAF [33] system is based on a distributed, event-basedinfrastructure for development and deployment of context-aware applications. Each covered context domain is modelledby a dedicated context service being realised as Java EnterpriseApplication deployed on an Application Server. Inhabitantsof such a domain are programmed as entities. Context ismodelled “semantically-free” and utilises an object orientedrepresentation. Context monitor and actuator componentsserve for acquisition and publication of context. Componentsin JCAF use RMI for communication which limits deviceheterogeneity severely. Due to the inadequate central RMIregistry and unavailability of dynamic lookup facilities JCAFfails to support large-scale deployments.

The PACE middleware [32] provides a context managementcomponent along with a programming toolkit for developmentand deployment of context-aware applications. In addition, de-cision support and a flexible messaging framework are investi-gated. Architecturally a distributed set of context repositoriesis proposed, each repository managing a specific catalogue

which is implemented as relational database. In addition toJava RMI, an HTTP based web interface is provided. Theauthors explicitly state that scalability is not a targeted designgoal. Moreover, the middleware uses an event-based messagerouting scheme on top of RMI and HTTP communication.This configuration compounds the overall scalability due thefundamental differences in the communication protocols i.e.messaging notification of the routing scheme and object-oriented RPC of Java RMI.

CoWSAMI [55] addresses the issue of using context fromdynamic sources by decoupling context sources from contextaggregators and providing dynamic discovery. Context sourcesexpose web service interfaces, including SOAP and WDSLtechnologies. Users can define relational context views andmap them to the context source interfaces and aggregatorsproduce context to satisfy these mappings accordingly. Bothcontext and rules are encoded in XML. The application ofCoWSAMI is limited to a particular application domain ofmobile users and vehicles. Moreover, the loose coupling anddynamic discovery of context requires explicit involvementof users in defining rules and mappings, hence limiting theusability.

MobiLife [58] applies a role model comprising distributedContext Providers, Context Consumer and a central ContextBroker in its Context Management Framework. RESTfulHTTP interfaces are used for communication and ensurethe support of device heterogeneity. Context is representedin ontologies. Entity diversity is not discussed, however thedescribed application scenarios are clearly user centric. Thedesign can be used for a large-scale deployment, however asingle, central Context Broker can be identified as bottleneckin practice.

OPEN [52] tries to address the developers’ functionalrequirements as well as users’ simplicity requirements. There-fore, OPEN enables different layers of programming supportfrom simple program parametrisation to incremental and com-posite programming paradigms. Several cooperation patternsare woven into the programming framework for encouragingthe users’ cooperation. Context queries are implemented by anadopted SPARQL context request and user defined rules arestored and executed based on a JESS engine. The usabilityof the system has been evaluated based on a treasure-gamescenario. However, processing and administrative scalabilityaspects have not been investigated so far.

ACE [56] aims at closing the gap between the (1) manage-ment and retrieval of context and (2) the demands of higher-level context related tasks such as application triggers. Anapplication context model is proposed to allow applicationdevelopers to explicitly describe their context logic. The appli-cation context engine stores the described models and handlesthe whole lifecycle of the defined application. Experimentshave demonstrated the use of the framework but also identifiedcurrent limitations. The reasoning delay from the underlyinginference machine prevents its practical usage in a large-scalesystem.

To summarise, a diverse set of design approaches has beenadopted for developing context provisioning frameworks andmiddleware. The general trend towards large-scale systems aswell as the support of application and device heterogeneity

becomes obvious. However, recent work still fails to ad-equately display the desired features including middlewareextendability and the required support for both emerging andevolving context-aware applications and services. This is notonly related to the introduction of new context categoriesbut also to the aggregation of detected context in variousabstraction levels. Another prominent reason that hinderscontemporary context middleware from stepping out of thebounds of lab environments into the real world is the lack ofscalability and its appropriate evaluation.

V. CONTEXT PROCESSING & REASONING

In general terms, reasoning derives “conclusions from acorpus of explicitly stored information” [69]. In the originalphilosophical sense deductive, inductive, abductive, analogicaland fallacious reasoning can be differentiated, the first twovariants being most relevant: Deduction attempts to showthat a conclusion necessarily follows from a set of premisesor hypotheses. Induction tries to derive propositions aboutunobserved objects based on previous observations, eitherspecifically or generally [70]. Anagnostopoulos et al. [71]describe context reasoning as “a process for inferring newcontext, previously unidentified on the basis of a-priori knowncontext”. In the domain of context-aware and ubiquitouscomputing and for the scope of this article, Context Reasoningand Context Inference are synonymously defined as follows.

Context Reasoning and Context Inference refer to theautomated deduction of high-level context from lower-levelor primitive context. In addition to an aggregation of avail-able context information, reasoning/inference may derive newconclusions based on existing context (i.e. measurable andobservable facts) and an available knowledge base.

Consequently, a significant number of mechanisms origi-nating from the fields of artificial intelligence and knowledge-based systems can be adopted for context reasoning. Moreover,there is a strong relation to activity recognition (e.g. [72]) andplan recognition (e.g. [73]). Before reviewing the differentcontext reasoning mechanisms, we discuss the role of a contextmiddleware in the the following subsection.

A. Context Processing and the Middleware

The entire context detection process can be subdivided intosubsequent stages as depicted in Fig. 12. A specific utilisationof such a layered design is for example discussed in [74].From bottom to the top of the processing chain, the complexityof mechanisms increases while reducing the amount of datathrough aggregation. Real world events are initially sensedand sampled. Particularly if physical sensors are utilised, thecollected raw data is preprocessed afterwards, either on-the-fly or in bulk. Pre-processing may comprise removal of noiseand faulty data by applying filtering techniques, averagingsensor readings or fusion of redundant data. In the next step,features are extracted, for example by fuzzy logic or crispfilters, to derive low-level context. This primitive context mayalso be directly fetched from virtual and logical sensors andserves as input for inferring common-sense abstractions, forexample a user’s situation, his mental and physical activities,his intentions.

Real World

Raw Sensor Data

Low-level context (primitive context)

Mid-level context, e.g. user‘s activitiy

Sensing, Sampling

Preprocessing (e.g. filtering)

Classification, Labelling

High-level context, e.g. user‘s intentionTime Series Analysis

Preprocessed Sensor DataFeature extraction

Fig. 12. Context Processing Stages

The process of context reasoning is evidently a complexone, and the context middleware is tasked with maskingthis complexity from other components and applications inthe system. The encapsulation of complexity is importantbecause it provides decoupling between the functionally sepa-rate components of the middleware and external applications.Consider a simple example of a reasoning component in amiddleware that uses finite state machines (FSM) to determineif a person is free to take a phone call. This reasoning enginedepends on input from various physical or virtual sensors thatmay include location, motion, phone status, noise, activity,etc. Assuming a common context model exists within theoverall system, raw data from these sensors will serve asinput the the FSM and may or may not result in a statethat signifies a person’s ability to take a phone call withinthat context. A different reasoning component, which may bepresent in the same middleware, may use a different reasoningmechanism, e.g. rule-based or probabilistic reasoning. Therole of the middleware in such scenarios is to facilitate thedata acquisition to feed the acquired data to the reasoningcomponent without being concerned about what process thedata will go through during the reasoning stage. Similarly,once the reasoning and context inference has been carriedout, interested third party applications or other componentsof the middleware (e.g. context storage components) needonly request the deduced context information. While thisencapsulation of the context reasoning complexity is importantfor the vertical integration between the components/layers of amiddleware, it is still critical to review the variety of contextreasoning mechanisms that have been employed in existingcontext-aware systems.

Different reasoning mechanisms may be suited to differenttypes of context domains, offer varying degrees of confidencein the contextual information that is based on fuzzy or incom-plete data and have different computational costs associatedwith their application in practice. The factors are importantto consider because mirroring the state of human activitiesinto computational constructs is a challenging process. Kim

IF A THEN B IF B THEN C

Rule and Knowledge Base

Inference Engine

Input acquisition

Actions Knowledge acquisition

Fig. 13. Architecture of a Rule-based reasoning system

et al. [72] identify typical challenges when dealing withthe nature of human activities: (1) several activities may beperformed concurrently; (2) activities may be interleaved; (3)the interpretation of activities may be ambiguous; (4) multipleresidents may be present in a smart environment and followcollaborative activities. Identifying users’ activities refers toeither a classification problem or to a time-series analysistask [75]. Various reasoning mechanisms can be applied forrecognising high-level context information. The selection ofan adequate technique mostly depends on the targeted context,the available input information and the chosen context model.The following subsections present fundamental backgroundinformation and discuss related work.

B. Rule-based ReasoningAs in any other rule-based system, rule-based context rea-

soning requires a fact base (knowledge base) and a rule basein which rules are stored. For inferring high-level context,primitive context is asserted into the fact base. Rule-basedreasoning comprises a threefold cycle of (1) matching, (2)conflict-resolution and (3) acting. Whenever the fact baseis updated, the internal inference engine checks if the rulebase contains any matching premises. After solving potentialconflicts, e.g. though consideration of assigned priorities, therule fires, which may optionally change the fact base and/ortrigger an asynchronous context update message to externalcomponents. This mechanism is also referred to as forwardchaining. The general architecture of a rule-based reasoningsystem is shown in Fig. 13.

Rule-based reasoning comes with certain drawbacks. Alarge rule base easily becomes confusing and intractable.Particularly, if actions result in fact assertion, unforeseen andundesired chain reactions may occur which may not only haltthe system but possibly even destroy the previously derivedasserted facts and/or conclusions. Moreover, the storage of factknowledge requires a large amount of memory consumptionsince the input context parameters of all entities need to bewatched constantly. Most importantly, rule-based reasoningcan only be usefully applied in systems supporting event-basedcontext propagation. Boolean execution, i.e. rules fire or theydo not, prevents the support of impreciseness.

C. Description LogicDescription Logic (DL), a compound of logic based for-

malisms for knowledge representation and reasoning, is ap-

Description Logic

Reasoning

Knowledge Base

Fig. 14. Architecture of a Description Logic based system

plied in conjunction with ontological context representation.The semantic modelling of concepts (classes), roles (prop-erties, relationships) and individuals allows terminologicalknowledge to be specified in a machine interpretable manner.An ontology is a formal specification of a shared conceptu-alisation of a domain of interest [76]. The so called TBox(terminological box) contains axioms relating concepts toeach other, e.g. describing concept hierarchies. The ABox(assertional box) comprises ground sentences, i.e. it asso-ciates individual objects to concepts. Both TBox and ABoxconstitute the knowledge base whereas the TBox formalisesintensional knowledge and the ABox extensional knowledge.An architectural representation of the description logic basedsystem is shown in Fig. 14.

DL are variable-free fragments of First Order Logic. Severalclasses of formal expressiveness are differentiated. The small-est propositionally closed DL is ALC and comprises atomicconcepts, atomic rules, conjunction, disjunction, negation,existential restriction and value restriction [77]. S is usedif transitive roles are supported moreover. Additional lettersindicate further extensions, e.g. H for role hierarchy, O fornominals/singelton classes, I for inverse roles, N for numberrestrictions, Q for qualified number restrictions and F forfunctional number restrictions. SHIQ is the basis for OWL,SHOIQ for OWL-DL and SHIF represents the formalrange of OWL Lite.

Description logic based reasoning has been applied in [34]for inferring users’ activities in the home domain (e.g. watch-ing movies, having dinner, taking shower) based on an OWLrepresentation. In [62], ontologies describing places, locationsand activities have been defined and used for reasoning by anOWL inference engine.

Performance measurements of DL based reasoning havebeen conducted in [34] and [78]. Researchers share the opinionthat reasoning based on ontologies is computationally inten-sive and that response times largely depend on the size ofthe data set and rule set. Scalability is a problem addressedin [78], in which a hybrid approach combining relationaland semantic representation is proposed. In addition to thereasoning complexity, designing an ontology is a complexendeavour requiring domain knowledge and expert agreement.In addition to a base layer of (potentially too large) commonlydefined ontologies, more specific application dependant on-

Naive Bayes Maximum Entropy

Hidden Markov Linear-chain CRF

joint conditional

single class

sequence

single class

sequence

Generative directed models General CRF

joint conditional

linear sequence

general sequence

linear sequence

general sequence

Generative Probabilistic Models

Discriminative Probabilistic Models

Fig. 15. Probabilistic Reasoning Models

tologies need to be designed on top. Extra effort is needed ifuncertainty and unavailability of context are to be supported.

D. Probabilistic Logic

Probabilistic models can be distinguished as generativemodels or discriminative models as depicted in Fig. 15.Assuming input values x and the target class variable y, gen-erative models require joint probability distributions p(y, x)to be represented whereas discriminative models are based onconditional probabilities, i.e. p(y|x) [79]. Both categories ofprobabilistic models can be represented graphically. Each noderepresents a random variable. The conditional independence oftwo random variables is graphically represented by the absenceof an edge between the associated nodes. Hence, causal depen-dencies become human comprehensible. Let G = (V,E) be agraph with vertexes V and edges E. The vertexes V = X∪Yare depicted as circles, X being the set of input or observa-tion variables and Y the set of output variables. Generativemodels are represented as Directed Acyclic Graph, i.e. theiredges are directed. Discriminative models are undirected andlink random variables to so called factor nodes graphicallyrepresented as small filled rectangle. A factor Ψs comprisesall random variables to which the factor node is connected.

The Naıve Bayes (NB) model [80] is the simplest generativeapproach for classifying single class variables in dependenceof multiple feature values. The Hidden Markov Model (HMM)[81] extends NB for representing sequentially structured data,allowing modelling of temporal changes. Like the NB model,the Maximum Entropy model [82] is used for classifyinga single output variable based on a set of observed inputparameters. In contrast to NB, it contains conditional prob-abilities, hence it is discriminative. Linear-chain ConditionalRandom Fields (CRF) [79] can be interpreted as a linearlysequential extension to the Maximum Entropy model. Genericsequential analysis of data is also supported when using theunconstrained CRF models. According to Lafferty et al. [83]CRF tend to be more robust than generative models to theviolations of their independence assumptions.

In particular, physical sensors are likely to give imprecisemeasurements. Moreover, due to potential temporary lack of

communication facilities, sensor data may not be availableat all. This is why Probabilistic Reasoning is especiallyapplicable in context-aware environments. Furthermore, prob-abilistic models, i.e. causal dependencies, can be learnt fromtraining data, e.g. by applying the Expectation Maximizationor Maximum Likelihood methods. With regard to context-aware systems, such training data may be collected from userbased feedback. Sequential probabilistic models can be repre-sented as finite state machine (FSM) with specific transitionprobabilities between states. This is particularly useful, if thecontext-aware system tries to detect real-world activities thatusually occur in a particular order.

In related work, both CRFs and HMMs have been appliedfor recognition of human activity in the kitchen domain [72].A Naıve Bayesian Classifier is used in [74] to identify variousactivity context features taken from audio sensors and ac-celerometers; such as driving a car, running to the door, takingan elevator and listening to music. Oliver et al. [84] proposetheoretical concepts of Layered HMM allowing for inferenceat multiple levels of temporal granularity. Dynamic BeliefNetworks (DBN) (a generalisation of Bayesian Networksincorporating temporal dependencies) and Linear DynamicalSystems (LDS) (a more general form of HMMs withoutconstraints on number of state spaces) and their applicabilityto human activity recognition is discussed by Turaga et al.[85].

E. Other Reasoning Mechanisms

Wen-Yu et al. [86] use Situation Calculus to model ubiqui-tous information services. Situation Calculus [87] is a formalfirst-order language defining actions, objects, situations, pre-condition axioms and successor state axioms. Sharing its basicphilosophy with commonly applied FSM, Situation Calculuseliminates the need for a-priori definition of possible statessince situations are dynamically generated and an infinitenumber of instances is allowed. This makes Situation Calculuseligible to model an open dynamic world but makes it un-manageable and not applicable for context-aware applicationsthat expect a defined output state. Situation Calculus has beenapplied for incremental plan recognition in [73].

A combined approach of Fuzzy Logic and clustering is pre-sented in [88]. Imprecise reasoning about situational contextand unsupervised model learning are supported. Yin et al.[89] present an activity recognition algorithm segmenting low-level sensor data with a probabilistic model. Each segment ofsignals is represented as an LDS model where transitions aremodelled as Markov processes. The overall goal is to derivehigh-level activities from Wi-Fi radio signal strengths.

A notable number of approaches apply workflow-inspiredpatterns focussing on temporal flows. Bosse et al. [90] andBoth et al. [91] introduce the Temporal Trace Language (TTL),a formally specified language based on predicate logic whichshares similarities with Situation Calculus. Inference rules aretranslated into temporal rules for describing the reasoningbehaviour in temporal partial logic. Both forward reasoningand backward reasoning are supported for deriving a human’sprogress in task execution while observing his behaviour.

F. Overview of Reasoning Mechanisms

Table IV provides an overview of the approaches presentedabove. It rates the support of machine based model learning,sequential data analysis and whether the mechanism can copewith imprecise and unavailable input data. In summary, adiverse set of mechanisms has been successfully applied toderive a rich set of high-level context.

VI. EVALUATION OF UBIQUITOUS MIDDLEWARE

A. Challenges of Middleware Evaluation

Context-Awareness and Ubiquitous Computing are bothmultidisciplinary research areas, incorporating elements ofHCI design, artificial intelligence, communication engineer-ing, psychology and social/behavioural sciences. Correspond-ingly, the evaluation of such a complex and interwoven systemis extraordinarily challenging and requires multidisciplinarytechniques. Interesting studies and surveys have been con-ducted with focus on the evaluation of a UbiComp applicationfrom the user’s perception, for example in [98], [99]. Proposedmethodologies for context-aware systems are borrowed andextended from the background of HCI research.

Though the users’ experience is essential, the work pre-sented in this article concentrates on the evaluation of the con-text management middleware rather than on the applicationsthemselves. Hence, a context middleware is assumed whichis able to support the users in their everyday life with theirdevice(s) moving through their usual environment seamlessly.The middleware has the difficult task of bridging the virtualand real worlds, which comes with numerous challenges.One problem in evaluating a middleware supporting richcontext-awareness is that a generic one-size-fits-all approachis unrealistic [100]. In related work, evaluation often fails to beobjective and comparable. A common consensus is required,not only related to UbiComp applications but also to theunderlying middleware caring for holistic or partial awareness.

Neely et al. [100] explain the need for a common evalu-ation framework and for a combination of multidisciplinaryapproaches. We agree with their opinion that evaluation isalways goal oriented and needs to be tailored accordingly.The following sections survey existing methods, propose rec-ommendations for usage and discuss limitations.

B. Prototyping

Prototyping the conceptualised middleware is the mostcommonly applied evaluation technique. Usually, core func-tionalities are implemented and deployed in a controlledlab environment. In an early stage, lab staff and colleaguesare usually taken as test subjects. This community is rarelyextended to external participants. Selected applications orservices are realised to demonstrate the benefit of the systemunder test and prove its correct functionality. Testbeds havebeen developed, for instance, in the Active Badge project[50], for the Active Map Service [20], the Cyberguide [49]and GUIDE [57] project. Other examples comprise an OfficeAssistant [101] and Gaia [53]. This kind of evaluation tech-nique has its advantage in illustrating the final user experiencewhile hiding middleware complexity. Obviously, it has been

frequently used in early work where the innovative idea ofUbiComp and context-awareness had to be delivered to a non-technical audience. But still, recent evaluations have focusedon and are sometimes even restricted to building a smalldemonstrator (e.g. inContext [66], MobiLife [58]). A disad-vantageous side effect of achieving middleware transparencyin such evaluations is that the internal performance metricsdo not become visible. Testbeds seldom succeed in provingscalability. Showing the context and application flexibility andvariety requires enormous efforts (time and costs) since a lot ofdisjunct and complementary services need to be prototyped.However, prototyping is a suitable choice if either the enduser view is to be investigated or if the ability of rapidprototyping is to be assessed (e.g. Context Toolkit [102]).Nonetheless, the experiences while coding the concepts oftenhelp in identifying shortcomings and improving or extendingthe architectural/functional model. The main target is to showthe capabilities of the middleware based on experimental ap-plications. As with other HCI evaluation, the user experiencecan be explicitly (e.g. interviews, questionnaires) or implicitly(observation, application based feedback) evaluated, albeit thisview may not necessarily mirror the middleware capabilitiesbut only the look & feel and usability of the applications.

C. Field Trials

1) Basic Approaches: Numerous methodological ap-proaches have been used for evaluating UbiComp applicationsin the field, particularly their HCI design. Before discussingtheir relevance for middleware evaluation, the most importanttechniques for collecting and analysing user behaviour andexperience are briefly introduced; see [99], [103]–[105] formore details.

• Interviewing: Qualitative data is gathered by evaluatorsasking individual users or groups open-ended questionsabout their work, background, ideas, etc. The answersare captured by a combination of note taking, audio andvideo. Though this approach is inexpensive, automaticeveryday actions might not be reported. When beingconducted outside the natural environment, the user mightforget to mention important information and impacts.

• Direct Observation or Contextual Field Research: Trainedobservers directly follow the test subjects and note downtheir behaviour. This costly, time-consuming and disrup-tive technique may utilise photographic and video analy-sis and is inappropriate in private settings and for large-scale scenarios. One of the most relevant advantages isthat users do not have to recall their actions and thequality of data is independent from variations in theirindividual reporting.

• Self Reporting: Recall Surveys that are based on usersorally reporting their behaviour suffer from recall andselective reporting biases. Activities are often either notremembered or incorrectly reported. Alternatively, usersnote down their daily routines in Time Diaries. Thismethod provides less biased data but due to distractionand annoyance, the records usually do not provide com-plete information.

TABLE IVCONTEXT PROCESSING & REASONING APPROACHES

Reasoning Approach Requirements Impr

Applicability (Examples)

Rule-based Reasoning Event-based context propagation; scalable fact storage / (�) / Situation Recognition [92]

Description Logic Domain knowledge represented in ontologies / � /Home activities [34], locations [62],

unspecific/generic [93], [78], [94], [95]

Situation Calculus Formalised language model / � /Modelling ubiquitous information services

[86]Naıve Bayes Labelled training data � / � Activity Recognition [74]

Hidden Markov Models, LinearDynamical Systems

Labelled training data or knowledge about conditionalprobabilities, availability of historic context � � � Activity Recognition [85], [89]

Bayesian Networks Labelled training data � / � Generic High-level Context Reasoning [96]Dynamic Belief Networks Labelled training data, availability of historic context � � � Activity Recognition [85]

Conditional Random Fields Labelled training data or knowledge about conditionalprobabilities, availability of historic context � � � Activity Recognition [97]

* �= support; (�) = limited support; / = no support.

• Usability Testing: Conventionally, usability data is col-lected using a combination of methods (observation, in-terviews, questionnaires) in a controlled setting, usually alab, where equipment for recording is available. Tailoredto software applications executing on desktop computers,the primary goal is to determine whether an interface isusable by the intended user population.

• Lag Sequential Analysis (LSA): Originating from devel-opment psychology, the LSA technique allows for acqui-sition of quantitative data by observing users performingtheir normal activities. Evaluators can generate statisticssuch as frequency and conditional probabilities of events.A drawback is that users being recorded by video mayalter their behaviour, knowing they are being observed.

• Automatic Tracing: Instead of a human evaluator, thebehaviour is tracked by an autonomous systems withthe help of various sensors and devices. User input orfeedback is not supported.

• Wizard of Oz: In a Wizard of Oz experiment, subjectstypically interact with a computer system. This system isbelieved to work autonomously but its functionalities arein fact being performed by unrecognised human being(s).

• Obstacle Course Data Collection: Evaluators prepare ato-do list for the test subjects. Instead of just observingtheir natural behaviour, specific activities or goals aredefined which the user has to follow.

• Experience Sampling Method (ESM): This technique, alsoreferred to as Ecological Momentary Assessment, sharessimilarities with Self Reporting but avoids the retrospec-tive character. An alarm is presented to the user during orshortly after the behavioural activity has been conducted.Upon reception of the alarm, the user has to answerquestions or report what he is doing or feeling. This way,data does not suffer from recall bias. Commonly usedESM tools comprise PDAs, smart phones, paper booklets,

mobile phones, traditional phones, audio player/recorder,pagers, watches, cameras or custom devices [106]. ESMallows the collection of both structured qualitative data(by defining fixed responses) and unstructured qualitativedata (by asking open-ended questions).

The presented techniques are not only useful for UbiCompapplication assessment. The recognition of high-level context(i.e. reasoning or inference) can best be evaluated by the userswhose context is to be derived. Three different main targetscan be identified:

1) Evaluate the reasoning/inference algorithms qualita-tively and quantitatively, i.e. assess their accuracy bycomparing expected (calculated) context against actual(user reported) context;

2) Collect training and feedback data for reason-ing/inference algorithms that support supervised ma-chine learning;

3) Evaluate the overall user experience, particularly as faras adaptive service/application execution is concerned.

The evaluation of ubiquitous everyday assistance benefitsfrom in-situ studies, also referred to as in the wild or eth-nomethodological. The key idea is to collect data while usersare situated in their own natural environment rather thancreating an artificial one in the lab which might influence theirbehaviour [99]. Due to the characteristics of the UbiCompdomain (user mobility, invisible sensing systems, distributedand fragmented interaction across different applications anddevices [107]), the ESM technique appears most promising,in particular if combined with automatic tracing and logging.

Preferably, the data collection is performed by commoneveryday devices so that users do not have to wear additionalhardware. It is important to minimise distraction from their or-dinary workflow. Especially if long-term studies are envisaged,minimal obtrusiveness on the user experience is essential.When applying ESM, this can be achieved by supporting (1)

event-based context-aware triggering of questions (i.e. whenis the alarm activated?), (2) context-aware selection of thealarming mode (i.e. how is the alarm presented?, e.g. audiblevs. tactile) and (3) context-aware selection of questions andpossible answers. Bearing in mind the variety of context, ESMtools may further serve as appropriate human sensors forunderstanding subjective areas such as emotional state, timeuse, social interactions, intentions, abstracted activities, habitsand goals [108]. Another advantage of ESM is that evalua-tors can collect preliminary data in near real time and thecontext recognition accuracy can be measured quantitativelyby applying stochastic methods. However, the individuality ofusers and their cognitive perception of events and activities (cf.[109]) results in potential errors and is problematic for learninggeneric models. The same applies to long-term activitiesconsisting of short-term successive or even parallel actions.Multitasking, false starts and human error can be seriousdilemmas when assigning labels to time sequences [110].

2) ESM Tools: Various tools for the evaluation of applica-tions have been designed, primarily based on ESM. Four ofthem are briefly presented below.

The Context-Aware Experience Sampling Tool (CAES) [111]addresses HCI researchers and allows them to acquire feed-back from users in particular situations detected by sensors. Itruns on PocketPC devices and supports context-aware alarmtriggering for the ESM. The main target is to assess theapplication interface design while concentrating on definedsituations, hence decreasing the degree of interruption andannoyance. The flexibility of questions presented to the testsubjects is increased by enabling the dynamic upload of newprotocols. Sequences of questions, multiple choice and mul-tiple response questions are supported, as well as recurrencepatterns and randomisation. Audio and photographic feedbackcan be collected.

MyExperience [105] combines autonomic caption of objec-tive sensor data (passive tracing) and subjective user feedback(active context-triggered ESM). The open-source implemen-tation is based on .NET and available for Windows Mobiledevices. MyExperience utilises a three-tiered, event-drivenarchitecture of sensors, triggers and actions – hence supportingconditionally triggered report surveys and SMS/Email notifi-cations. Moreover, it allows for preliminary data collectionby providing direct remote access to collected data withoutany need for physical access to the device under study. AnXML schema has been designed for enabling the scriptingbased remote configuration or control of the evaluation client.Therefore, study customisation (i.e. definition of surveys,triggers and actions) is simplified and accelerated.

Momento [112] utilises an architecture that comprises a (1)desktop platform being used by the evaluators in order toconfigure and monitor an experiment, (2) a server componentand (3) mobile devices collecting sensor data and applyingESM. The desktop platform can be connected to the fixed ap-plications via a Context Toolkit. The mobile client is realisedbased on Java and C#. It can be configured via simple textfiles in which rules for ESM alarm triggering can be defined.

The ActivityDesigner [110] focuses on the test-driven designprocess of UbiComp applications. It refers to the domain ofActivity-Centered Design (ACD) and targets an activity-based

UbiComp prototyping. Long-term everyday user activities arecaptured over an extended time period. A graphical userfront-end is used to analyse automatically generated activityjournals and to design prototypes for defined storyboards(i.e. activities, actions, scenes and the transitions in between).These prototypes can be deployed in a virtual machine (high-end device) or by a web application based on the GoogleWeb Toolkit. In summary, the ActivityDesginer tries to supportthe interlinked process of (1) field observations, (2) activityanalysis/modelling, (3) interaction prototyping and (4) in-situtesting.

3) Trends & Evolution: The increasing penetration ofsmartphones and the availability of equipped sensors couldsignificantly accelerate the evolution of ESM based clientswithout the need for any extra hardware. The basic require-ments still remain the same: avoidance of massive batteryconsumption, systems crash and negative impact on the userexperience. One constraint of earlier work was the verylimited number of participants. In contrast, mobile applicationscan be developed and disseminated more easily nowadays.Distribution frameworks such as Apple App Store or GooglePlay Store allow large scale studies to be conducted. Thiswould turn the human sensor into a crowd sensor and enableparticipatory sensing. The more feedback is provided byunknown and unbiased test subjects the more fruitful andmeaningful the statistical analyses. However, adaptive filtertechniques are required to support a focus on situations andusers of special interest. Context-aware alarming could becombined with context-aware selection of test users. Bothhoped and unexpected effects have to be monitored [99].Another important aspect is the motivation of users. Long-termstudies may be burdensome if the number of ESM triggeredfeedback requests is too high. A socially or gaming inspiredreward mechanism may help. Finally, it is questionable howa test subject feels about being observed by strangers. Inthis regard, privacy, security and anonymisation have to besupported. Moreover, interested users would benefit fromimproved context recognition accuracy. In summary, field trialsoffer essential insight into otherwise hidden results. They arenot only useful for the evaluation of UbiComp applicationsbut also for the assessment and gradual optimisation of rea-soning/inference algorithms.

D. Emulation & SimulationIn the literature, especially in the scope of UbiComp and

context-awareness, the terms emulation and simulation areoften used synonymously without much differentiation. Ingeneral, the goal of emulation is to be able to substitutethe object that is emulated. The focus of a simulation is themodelling of essential features of the system under test. Ingeneral simulation does not necessarily lead to emulation. Inparticular, a simulation may run slower (or faster) than real-time whereas emulation requires the system under test to beat least partly available in a real deployment and invoked inreal time.

Transferred to the domain of context-aware middleware, atleast four basic categories of approaches appear useful:

1) Emulating Context: The system under test (middlewarecore) is used as is, i.e. prototyped system components

are directly invoked. But context does not originate fromreal world events and real users, instead it is emulated.

2) Emulating Middleware Components: The real life imple-mentation of the middleware core remains untouched.However, the middleware is extended by further com-ponents, e.g. processors, sources and sinks of contextinformation that are considered outside of the real pro-totype.

3) Emulating Actuation: In order to illustrate the adaptivebehaviour of context-aware applications or actuators, avirtualised environment can be added.

4) Middleware Simulation: The system under test is sim-ulated entirely. The contextual input as well as thefunctionalities of the system components are modelledin an abstract manner. There is no real life deploymentrequired outside the simulation environment.

These four approaches are analysed and discussed in moredepth below before related work is summarised.

1) Context Emulation: The emulation of context aims at en-suring controlled test conditions. In a UbiComp environmentit is impractical to send real users around in the real worldand ask them to follow a defined behaviour. That is why thereal world (or parts of it) are substituted by a virtual worlddesigned by the evaluators. The reaction of the middlewarebased on events and context changes in this virtual world isthen evaluated. The key challenge is to represent the physicalworld reasonably and realistically and not just randomly. Oneapproach is to define labelled and parameterised sequencesof context changes (e.g. “having breakfast at home”). Thesetemplates can then be executed by the emulation environmentto create a vivid setting autonomously. Alternatively, a Graphi-cal User Interface (GUI) allows for temporary manipulation ofselected parameters. The following targets can be addressed:

• Context Emulation allows for generating huge amountsof context information. Not only knowledge about theindividual user but also the number of users can beeasily increased in the virtual test world. Real context isduplicated and associated to pseudo entities (also knownas avatars) [113]. The same applies to device context.Therefore, a real middleware system can be probed inorder to analyse its scalability behaviour under almostrealistic circumstances.

• Context reasoning, aggregation and processing in general,can be evaluated if primitive context (e.g. acceleration,location) is emulated. The high-level context (e.g. activ-ity) output of the middleware is then compared againstexpectations. This requires context to be recordable andreplicable.

• The adaptation behaviour of applications and other actu-ators can be observed while manipulating the context.

It is essential to make recorded context profiles exchangeableso that the results of different approaches can be comparedwith each other. Efforts have been made by researchers atthe Massachusetts Institute of Technology (MIT) who sharedtheir sensor data collected in a smart live-in lab [114]. Arepresentation schema is for instance proposed in [115].The need for dataset exchange has also been emphasised inthe BoxLab project at MIT (http://datasets.mit.edu). In their

shared resources not only datasets are available for downloadsbut also annotation schemas. Current research focus is onsmart homes but extensions towards a smart world coveringeveryday activities is desirable.

2) Emulating Middleware Components: The target of emu-lating middleware components is the substitution of hardware(e.g. physical sensors) by software. This approach is usefulif new context types are to be added to the system. Beforeputting too much effort into the design and implementation,the behaviour of the middleware can be estimated beforehand.The effect of the number of components can be measured. Thisapplies not only to context sources and processors but alsoto context consumers or sinks. Especially in an event-drivensystem the number of consumers is expected to influencemetrics such as the notification time and network load.

3) Emulating Actuation: The evaluation of actuating com-ponents can be tricky in a UbiComp environment. Imagine asmart home adjusting the heating or light level according to itsinhabitants. The smarter and larger the space is, the more ex-pensive becomes classical prototyping. Therefore, evaluatorsexperimented with virtual three-dimensional graphic engines,e.g. those used in popular games including Half Life, QuakeIII Arena, and Unreal Tournament, to produce a realisticdemonstration of how the environment would react to contextevents.

Obviously this approach fails if we consider a large scale,e.g. an urban smart space. Even in a geographically smallarea, it is questionable if a realistic and fancy layout wouldadd value and insight to the middleware evaluation. Only ifthe emulation of actuation is combined with context emulation,these techniques might improve understanding and the look &feel of selected UbiComp spaces. Still the focus of emulatingactuation remains on the end user perspective and experiencerather than on the middleware.

4) Middleware Simulation: The methodology of a system-level simulation is borrowed from classical communicationengineering. The behaviour of the middleware, including allits distributed components, is modelled. Typically, only limitedcore functionalities and the exchanged messages are includedin the simulation model where Discrete Event Simulation canbe applied. When building a simulation model the followingquestions and challenges need to be addressed:

• The appropriate degree of abstractness must be identi-fied for the simulation model. A model with too muchdetails will increase development costs and simulationtime without providing more realistic results. The modeldepends on the specific goals of the simulation [116].

• The parameters (input values) and metrics (output values)must be chosen. In a UbiComp setting, the definitionof reasonable scenarios is extremely complex. Not onlythe user behaviour but also the application behaviour (orin general the requirements of context consumers) hasto be estimated. This comprises the number of contextrequests/subscriptions per unit time, the amount andfrequency of context changes.

• In addition, as far as a distributed middleware is tobe evaluated, knowledge about the behaviour (e.g. re-sponse time, processor and memory consumption) ofeach component is required. The key is finding simple

parameters that are able to represent the real systemwithout simplifying too much. Obviously it is difficult– if not impossible – to build a realistic simulationmodel without ever having implemented a prototype ofthe middleware and feeding the simulation parameterswith metrics determined in the real testbed.

The essential advantage of system-level simulations is thatparameters can be varied easily to investigate various scenar-ios. Parameters like the number of users, number of devices,the amount of context types, the context query frequencycan be used to estimate the large-scale performance of amiddleware. Network load, context traffic and response timesare interesting metrics to observe. If modelled well, the sim-ulation can help in identifying bottlenecks of the system andprovide guidelines for improving the concepts and algorithms.To facilitate the comparison of results obtained by differentresearchers, common simulation scenarios (sets of parameters)need to be defined in the future.

5) Simulation/Emulation Tools: Categories of emulationand simulation are not necessarily differentiated in relatedliterature. There is a body of research which concentrateson three-dimensional graphical user interfaces. Testers cannavigate around in the first person perspective and see the(emulated) actuation capabilities of the smart space based onemulated context changes, as expanded below.

UbiWise [117] is built on the Quake III Arena graphicsengine and allows multiple users to participate in interactiveUbiComp scenarios. The physical environment view is simu-lated by a subcomponent called UbiSim whereas a device viewis provided by a separated module, realised in Java, namedWise. As an example the authors emulate a UbiComp cameraand an actuating picture frame. Real web service interaction issupported as well. The emulated camera can be controlled inthe Wise view and it can be carried around in the UbiSim 3D-simulator environment. Spatial and environmental interactionsof devices and users are focused.

A Hybrid Test and Simulation Environment [118] has beendeveloped by researchers at Lancaster University. In con-trast to UbiWise, their approach does not provide a three-dimensional GUI interface. According to the system undertest, the interfaces of the evaluator components are basedon Web Services (HTTP and SOAP). Important featuresare centralised logging and the support of configuration andautomated test scripts. The simulation environment focuseson the evaluation of location-based applications under con-sideration of different radio access technologies. Quantitativemeasurements such as the response and transport delay havebeen taken.

CIVE [119] presents a context-based interactive System fordistributed virtual environments. It tries to bridge the gapbetween real and virtual cyber-world with focus on how todeliver real UbiComp interaction (e.g. gestures) into cyber sys-tems. The submodule ubi-UCAM generates contexts and userprofiling out of real world interaction, whereas NAVER caresfor a virtual heritage and sensing of users’ activities in thecyber environment. Another component called INTERFACEconnects those two.

TATUS [113] also aims at building a virtual ubiquitouscomputing environment. The complexity is reduced by em-

ulating sensors and actuators. Users’ devices are not emulatedbut the original ones are connected to the simulator. Further,rapid scenario and virtual world modelling are facilitated byproviding a design toolkit allowing for a reasonable level of re-alism, software flexibility, experiment usability and simulatorextensibility support. Moreover, a Wireless Network Simulatorhas been designed in order to evaluate the performance ofcommunication systems used in UbiComp environments. Itsmain purpose is to model channel conditions based on distanceand obstacles (human beings, walls, etc.).

UbiREAL [120] is another simulator addressing virtual 3Dspaces. Its developers extend previous work by generatingmore realistic context and supporting systematic testing. Bothvirtual and real devices can cooperate via Ethernet. Besidesa network simulator and the 3D GUI interface, UbiREALcomprises a simulator for physical quantities and a test suite.The former one models the change of physical conditionsbased on triggered actions (e.g. the room becomes warmerif the heating has been switched on). The latter one canbe used to define action rules and expected outcomes in aformal representation schema, hence allowing for autonomousvalidation of results.

CAST (Context-Awareness Simulation Toolkit) [121] con-centrates on a virtual home domain and allows creatingvirtual context. Security aspects are taken into considerationwhen sharing such context between simulator components.Conceptually a Virtual Device Manager is responsible foremulating sensors and context consumers. Pseudo user entitiescan be generated by a Virtual Person Manager and the VirtualHome-Map Editor allows for designing the virtual smartspace. The two-dimensional graphical presentation is based onMacromedia Flash and fails to provide a realistic impressionof the setting. The comic-like approach is not suited formiddleware evaluation either.

UbiHolo [122] is rather a UbiComp software paradigm thana simulator. However, the authors’ work deserves attentionbecause it contains one of the few approaches to applysystem-level simulation in the UbiComp domain. To evaluatethe large-scale performance of their concept, a peer-to-peersimulator called p2psim has been utilised.

Siafu [123] is a representative example of a Java-basedcontext emulator. Based on a two-dimensional GUI, it allowsthe comfortable manipulation of context temporarily. Sinceit relinquishes the three-dimensionality, it is suited to cover(spatially) larger areas such as urban smart spaces. Siafucan generate its own simulated context (e.g. by defined usermovements) and in parallel incorporate real world contextacquired from sensors. Datasets can be produced for machinelearning and effects of context changes can be demonstratedand visualised by plugging an application.

C-ProMiSE (A Context Provisioning Middleware with Sup-port for Evolving Awareness) [124] has been quantitativelyevaluated by discrete event simulation. The simulation modelshave been derived from black-box assessments of prototypedsystems components. The definition of various context queryscenarios and context update assumptions facilitates a system-atic analysis of the overall performance.

E. Comparison of Methodologies

This section has highlighted the dependence on multidisci-plinary approaches and challenges in evaluating middlewarefor ubiquitous computing. Techniques incorporating prototyp-ing, emulation, simulation and field trials have been discussedand their utilisation in various systems has also been presented.These techniques not only target different – and often mutuallyexclusive – objectives, but also require varying aspects offunctional availability in the middleware being evaluated.Table V provides a comparative summary the these evaluationtechniques in terms of targets and requirements along withexamples of their utilisation in existing middleware systems.

VII. SUMMARY AND OUTLOOK

A. Synopsis

This article has presented a survey of context provisioningmiddleware with emphasis on context modelling, contextmanagement, context processing and evaluation of contextmiddleware approaches. Related work has been analysed andcategorised correspondingly.

The main points of this article and key findings from theanalyses of various domain specific topics can be summarisedas follows:

1) Context-awareness is a key feature to enable the vi-sion of ubiquitous computing. A Context ProvisioningMiddleware can aid in collecting sensor data, detectthe user’s situation and make it available to context-aware services and applications. Its major requirementsare functional and physical scalability while ensuringcoherent and simple access to contextual information.

2) A broad definition of context helps in identifying a largeset of application domains. Early systems were limitedto spatial context whereas recent, more innovative sys-tems tend to increase the abstraction level, e.g. activityor plan recognition. Semantic context modelling utilisesontologies represented by semantic web technologies.Numerous researchers highlight their difficulties whendesigning ontologies and applying reasoning with largeontologies; therefore a layered approach has been pro-posed with common sense ontologies at the bottom anddomain specific knowledge on top.

3) Recent work in the area of context management architec-tures concentrates on web service approaches and triesto support modular extendibility to cope with emergingservices and applications; furthermore the interoper-ability and wide utilisation of web standards supportheterogeneous devices and (radio access) networks.

4) A number of different approaches, from various do-mains of computing and artificial intelligence, havebeen used in the complex functional task of contextreasoning/processing. Different reasoning mechanismsmay be suited to different types of context domainsand offer varying degrees of confidence in the processedcontextual information. The selection of an appropriatetechnique depends on the type of context being reasonedabout, available input data and the adopted contextmodel.

5) The approaches to context middleware evaluation areoften limited in focus to functional assessment, par-ticularly through prototype implementation. The sizeand number of playgrounds and test beds is increasing.However, context-aware systems have not yet steppedout of the laboratory environments into the real worldsignificantly.

B. Lessons Learnt

Based on the multi-spectrum review of the domain ofcontext-awareness in general and context-provisioning mid-dleware in particular, we summarise the lessons learnt in thissection:

• Because it deals with the practical application of wide-ranging computing concepts, ubiquitous computing ingeneral is becoming indistinguishable from other fieldsof computing. This aspect highlights both the crossdisciplinary applicability and conceptual complexity indesigning context-aware systems.

• We do not expect a single reasoning/inference technologyor a single context model to be able to cope withall application domains simultaneously. Instead, contextconversation and translation mechanisms as well as adiverse set of context processing technologies need tobe incorporated.

• The evaluation of context provisioning middleware isa complex endeavour. A fair and objective comparisonof concurrent context representation and managementapproaches is challenging mainly due to the fact that theamount of published measurements is insufficient. Herewe encourage the definition of useful metrics and a com-mon evaluation methodology that enables the quantitativejuxtaposition.

• The number of context sources and sinks is expectedto increase tremendously with Internet of Things (IoT)deployments and sensor equipped smartphones. There-fore, the scalability of a middleware, both in terms ofcomputation and administration, is a sincere concern.

• A systematic evaluation methodology definitely benefitsfrom prototyping, (global) field trials, context emulationand system-level simulations.

C. Trends and Evolution

In addition to academic research, industrial developmentshave recently gained a foothold into context-aware comput-ing. Dominant market forces, including Apple, Google andMicrosoft, not only build and equip smartphones but alsoentire software ecosystems that include software develop-ment kits and software distribution platforms. An obvioustrend is the seamless integration of social networking suitesand tools for sharing user-generated content. Correspondingfunctionalities are integrated in the operating system. Siri(www.apple.com/uk/ios/siri/) and Vlingo (www.vlingo.com/)are prominent examples of mobile applications that provide aspeech-based HCI and constitute the first practical realisationof a context-aware personal assistant.

User-generated content and voluntary disclosure of confi-dential information in social networks appears to be a more

TABLE VOVERVIEW OF EVALUATION STRATEGIES

EvaluationTechnique Targets Requirements Examples

PrototypingMost realistic proof-of-concept for end-to-end scenarios; provides input parametersfor simulations

Implemented applications and test personsActive Badge [50], Active Map Service [20],Cyberguide [49], GUIDE [57], Conference

Assistant [125], Office Assistant [101], Gaia [53]

Field Trials Assess Reasoning/Inference accuracy, col-lect feedback & training data

Context-aware experience sampling, dynami-cally reconfigurable by evaluators, easily dis-tributable and deployable on everyday devices

MyExperience [105], Momento [112], CAES[111] [103], ActivityDesigner [110]

ContextEmulation

Real-world context is extended or replacedby creating a virtual world; prototypes canbe fed with input in order to evaluatescalability

For evaluating context processing, reasonablemodels must be defined (e.g. taken fromrecorded real-life context and associated toavatars)

Siafu, UbiWise [117], TATUS [113], UbiREAL[120]

EmulatingMiddlewareComponents

Substitution of hardware (e.g. physical sen-sors) by software; reduction of costs inlarge scale UbiComp environments; Esti-mation of consequences when extendingthe middleware

Emulation models must reflect the behaviourof the real hardware UbiWise [117], TATUS [113]

EmulatingActuation

Providing a realistic outlook of context-based actuation in a smart environment

The influence of the actuation must be fedback into the system by producing emulatedcontext (e.g. heating ⇒ warmth)

CIVE [119], UbiWise [117], TATUS [113],UbiREAL [120]

MiddlewareSimulation

System-level analysis of large-scale be-haviour

Abstract Simulation model, reasonable param-eter assumptions (taken from prototypes) UbiHolo [122], C-ProMiSE [124]

expressive source of context than physical sensor readings, atleast for most of the application domains. Social and emotionalsensors might alternatively utilise the Experience SamplingMethod that has been discussed in this article. Sensor equippedsmartphones are the users’ primary source of interaction withthe digital world; furthermore they allow for opportunisticor participatory sensing, i.e. multiple phones may form acrowd sensor. Data exchange may occur globally via theInternet or by low range communication and the utilisationof opportunistic networking [126].

There is a growing realisation of the need for real-worldexperimental facilities in order to advance the domain ofubiquitous computing. The SmartSantander initiative [127]uniquely conducts a smart city experiment that provides asmart space at an urban scale across four European cities,in essence becoming one of the largest living labs in Europe.It remains to be seen if the IoT concepts and related tech-nologies will increase the size of such smart spaces to largergeographical areas or even globally.

Recent advances point towards a computing paradigm shiftin progress, which can be perceived as an intermediarystep towards realisation of ubiquitous computing. The firstgeneration of computing provided one computer to manyindividuals (mainframes), the second generation allowed forone computer per individual (PC) and the third generation(distributed/mobile) computing has made possible many com-puters per individual. The current transformations are pavingthe way for a many computers–many individuals correspon-dence through the Cloud computing paradigm. It is interestingto note that while earlier generations maintained a strongidentity association between the computer and the individual,future generations may no longer maintain that association dueto virtualised (Cloud) and community administered (sensors)computing resources. In such computing environments, webelieve that it will be the role of middleware to mask thenumerous computing resources from users and manage theidentity associations, which have accessibility, privacy andsecurity concerns, at the same time.

D. Open Issues and Future Work

The expected evolution discussed above motivates furtherresearch contributing to the vision of a generic and evolvingcontext provisioning middleware for utilisation in ubiquitouscomputing. Based on the current state of the art, the followingobjectives appear essential.

The paradigm of Cloud computing may aid in contextprocessing and context storage by offering virtual resourcesthat can be efficiently utilised on demand [128]. How tosupport and embed virtual resources in context modelling,context management and reasoning mechanisms is still anopen issue. Moreover, privacy and security concerns increasewith the available variety and amount of context – andits potential storage in the Cloud. Personal information al-lows virtually tracing users and their activities. Therefore,appropriate protection mechanisms are required. Particularlyuser management, authentication and authorisation need tobe systematically incorporated. Third-party context access hasto be bound to explicit grants or autonomous anonymisation.Further investigation is required to establish how to efficientlyand safely support these mechanisms in a Cloud-based contextprovisioning middleware.

Recent middleware designs tend to unidirectionally detectand publish irrelevant context that is not required by anyapplication/service at all or at least not in the provided gran-ularity or accuracy. Hence, processing resources and energyare wasted. The efficiency could be increased by autonomousremote (re-) configuration of sensing devices according to thecontext demands of consumers. These demands may either beexplicitly defined through context subscriptions or implicitlyderived and predicted from earlier queries. The resultingfeedback loop may be adequately addressed and analysed fromcontrol systems engineering point of view.

Most researchers have had difficulties in identifying a killerapplication for ubiquitous computing. Ubiquitous computingand context-awareness intend to pro-actively support usersduring their everyday life without any need for complicatedinteraction to the computing resources. This leads to the ap-

plication of ubiquitous computing and context-aware conceptsin a huge variety of useful application domains. However, weunderscore the quite obvious choice of an all encompassingpersonal digital assistant as the context-aware application ofchoice, which can provide an interface for different applicationdomains and further route the demands to other domainspecific services. In existing systems, the focus has been on theadaptation of the service/application logic and its execution.However, future research may investigate the possibility ofcontext-aware service composition and context-aware servicedeployment.

Another avenue of exploration is the level of control del-egation to context-aware systems. Even considering simplecontext-aware services, there is a significant amount of com-puting and processing in the background that the user isintentionally not made aware of. The system does not wantto be intrusive and distract the user from his current activityi.e. there is not necessarily the need for a computing-awareuser. However, the user needs to understand the system; andmore importantly, the user needs to be able to easily overruleautomatisms and thus enforce individualistic control. Thisneed for individualistic control is important because thoughhuman beings collectively share certain common-sense logicand knowledge, a particular human being behaves with acertain individuality. This individuality has other ramificationsas well, e.g. most reasoning techniques in existing context-aware systems apply probabilistic logic, which is based onprobabilistic models derived through machine learning. Thesetechniques depend on available training data that usuallyoriginates from an individual user and/or from a group ofusers. Hence, the training data is often tainted with individu-alistic noise, which may over represent random individualisticexpressions. A useful trade-off for training data has to beidentified to compensate for this individualistic noise and thusderive general context reasoning heuristics.

Finally, one of the early proponents of ubiquitous andcontext-aware computing, Abowd [129] states that ubiquitouscomputing has performed an intellectual disappearing actthrough its multi-disciplinary nature i.e. its ideas and conceptsalready pervade much of computing research and practice.This argument is not meant to close the chapter on focussedubiquitous computing research but rather to emphasise theimportance of this next generation of computing. Abowdargues that the intellectual agenda of ubiquitous computing hasbecome so profound that it is increasingly indistinguishablefrom the overall agenda of computing research today [129, p.2], echoing Weiser’s argument that the most profound researchtopics are those that disappear by weaving themselves intothe fabric of every research until they are indistinguishablefrom it [130]. This analysis best explains the diversity in thescope of the work discussed in this article, as it is increasinglydifficult to analyse the state of the art in context-aware systemswithout addressing the multi-disciplinary nature of the domainof ubiquitous computing.

VIII. CONCLUSION

This article has presented a comprehensive review andanalysis of how context-aware middleware systems undertakecontext modelling, management, reasoning and provisioning

related functions. By examining the state-of-the-art in the mul-tifarious aspects of context-awareness, the article has aimed atdeveloping an understanding of the functional diversity of themiddleware bridging the virtual and real worlds. We have alsoexamined how such complex and interwoven systems can beevaluated through multidisciplinary approaches. Based on thediscussions and analyses of various domain specific topics,this article has also presented the main trends and ongoingevolution of context provisioning. Open research issues havebeen identified and recommended for future work.

ACKNOWLEDGMENT

The work presented in this article has been partly fundedby the European research project “Context Casting (C-CAST)”in Framework Programme 7. Its main objective is to evolvemobile multimedia multicasting to exploit the increasing in-tegration of mobile devices with our everyday physical worldand environment.

REFERENCES

[1] M. Weiser, “The computer for the 21st century,” Scientific American,vol. 272, no. 3, pp. 78–89, 1995.

[2] A. K. Dey, “Understanding and using context,” Personal and Ubiqui-tous Computing, vol. 5, pp. 4–7, 2001.

[3] N. Kara and O. A. Dragoi, “Reasoning with contextual datain telehealth applications,” in Proc. Third IEEE InternationalConference on Wireless and Mobile Computing, Networking andCommunications, ser. WIMOB ’07. Washington, DC, USA:IEEE Computer Society, 2007, pp. 69–. [Online]. Available:http://dl.acm.org/citation.cfm?id=1318474.1318622

[4] Y.-M. Huang, Y.-H. Kuo, Y.-T. Lin, and S.-C. Cheng, “Toward interac-tive mobile synchronous learning environment with context-awarenessservice,” Computers & Education, vol. 51, no. 3, pp. 1205 – 1226,2008.

[5] W. Choi, H. Kim, J. Kim, and J. Chae, “A context-aware framework formobile navigation service,” in Computer and Information Technology,2007. CIT 2007. 7th IEEE International Conference on. IEEEComputer Society Press, Oct 2007, pp. 423–428.

[6] H. Chen, F. Perich, D. Chakraborty, T. Finin, and A. Joshi, “Intelligentagents meet semantic web in a smart meeting room,” in Proceedingsof the Third International Joint Conference on Autonomous Agentsand Multiagent Systems - Volume 2, ser. AAMAS ’04. Washington,DC, USA: IEEE Computer Society, 2004, pp. 854–861. [Online].Available: http://dx.doi.org/10.1109/AAMAS.2004.152

[7] J. Simoes and T. Magedanz, “Smart advertising in the home of thefuture,” International Journal of Computer Aided Engineering andTechnology, vol. 2, no. 2, pp. 164–180, 2010.

[8] B. Adams, D. Phung, and S. Venkatesh, “Extraction of socialcontext and application to personal multimedia exploration,” in Proc.the 14th annual ACM international conference on Multimedia, ser.MULTIMEDIA ’06. New York, NY, USA: ACM, 2006, pp. 987–996.[Online]. Available: http://doi.acm.org/10.1145/1180639.1180857

[9] S. Bjork, J. Holopainen, P. Ljungstrand, and R. Mandryk, “Special issueon ubiquitous games,” Personal and Ubiquitous Computing, vol. 6, no.5-6, pp. 358–361, 2002.

[10] A. Gupta, S. Paul, Q. Jones, and C. Borcea, “Automatic identificationof informal social groups and places for geo-social recommendations,”International Journal of Mobile Network Design and Innovation, vol. 2,no. 3, pp. 159–171, 2007.

[11] D. Cook and S. Das, Smart environments: technologies, protocols, andapplications. Wiley-Interscience, 2005.

[12] S. Dobson, S. Denazis, A. Fernandez, D. Gaıti, E. Gelenbe, F. Mas-sacci, P. Nixon, F. Saffre, N. Schmidt, and F. Zambonelli, “A surveyof autonomic communications,” ACM Trans. Autonomous and AdaptiveSystems (TAAS), vol. 1, no. 2, pp. 223–259, 2006.

[13] E. Aarts, R. Harwig, and M. Schuurmans, Ambient intelligence, Theinvisible future: the seamless integration of technology into everydaylife, P. J. Denning, Ed. New York, NY, USA: McGraw-Hill, Inc.,2001.

[14] H. Gellersen, “Smart-its: computers for artifacts in the physicalworld,” Commun. ACM, vol. 48, no. 3, pp. 66–, Mar. 2005. [Online].Available: http://doi.acm.org/10.1145/1047671.1047707

[15] A. Zimmermann, “Context Management and Personalisation,” Ph.D.dissertation, University of Aachen, 2007.

[16] F. Giunchiglia, “Contextual reasoning,” Epistemologia, special issueon I Linguaggi e le Macchine, vol. 16, pp. 345–364, 1993.

[17] C. Billings, “Situation awareness measurement and analysis: A com-mentary,” in Proc. International Conference on Experimental Analysisand Measurement of Situation Awareness, D. Garland and M. Endsley,Eds. Daytona Beach, FL: Embry-Riddle Aeronautical UniversityPress, 1995, pp. 1–6.

[18] P. Bellavista, A. Corradi, and C. Giannelli, “A unifying perspective oncontext-aware evaluation and management of heterogeneous wirelessconnectivity,” IEEE Commun. Surveys Tutorials, vol. 13, no. 3, pp. 337–357, quarter 2011.

[19] A. Dey, “Providing architectural support for building context-awareapplications,” Ph.D. dissertation, Georgia Institute of Technology,2000.

[20] B. Schilit and M. Theimer, “Disseminating active map information tomobile hosts,” IEEE Network, vol. 8, no. 5, pp. 22–32, 1994.

[21] N. S. Ryan, J. Pascoe, and D. R. Morse, “Enhanced realityfieldwork: the context-aware archaeological assistant,” in ComputerApplications in Archaeology 1997, ser. British ArchaeologicalReports, V. Gaffney, M. van Leusen, and S. Exxon, Eds.Oxford: Tempus Reparatum, October 1998. [Online]. Available:http://www.cs.kent.ac.uk/pubs/1998/616

[22] G. D. Abowd, E. D. Mynatt, and T. Rodden, “The human experience,”IEEE Pervasive Computing, vol. 1, no. 1, pp. 48–57, 2002.

[23] H. Chen, “An intelligent broker architecture for pervasive context-aware systems,” Ph.D. dissertation, University of Maryland, BaltimoreCounty, December 2004.

[24] C. Burghardt and T. Kirste, “Inferring intentions in genericcontext-aware systems,” in Proc. 6th international conferenceon Mobile and ubiquitous multimedia, ser. MUM ’07. NewYork, NY, USA: ACM, 2007, pp. 50–54. [Online]. Available:http://doi.acm.org/10.1145/1329469.1329475

[25] T. Strang and C. Linnhoff-Popien, “A context modeling survey,” in FirstInternational Workshop on Advanced Context Modelling, ReasoningAnd Management at UbiComp 2004, Nottingham, UK, September2004.

[26] C. Bettini, O. Brdiczka, K. Henricksen, J. Indulska, D. Nicklas,A. Ranganathan, and D. Riboni, “A survey of context modelling andreasoning techniques,” Pervasive and Mobile Computing, vol. 6, no. 2,pp. 161–180, 2010.

[27] C. Bolchini, C. A. Curino, E. Quintarelli, F. A. Schreiber, andL. Tanca, “A data-oriented survey of context models,” SIGMODRec., vol. 36, pp. 19–26, December 2007. [Online]. Available:http://doi.acm.org/10.1145/1361348.1361353

[28] M. Baldauf, S. Dustdar, and F. Rosenberg, “A survey on context-awaresystems,” International Journal of Ad Hoc and Ubiquitous Computing,vol. 2, no. 4, pp. 263–277, 2007.

[29] A. Ikram, N. Baker, M. Knappmeyer, E. Reetz, and R. Tonjesy,“An artificial chemistry based framework for personal and socialcontext aware smart spaces,” in Wireless Communications and MobileComputing Conference (IWCMC), 2011 7th International. IEEE,2011, pp. 2009–2014.

[30] M. Knappmeyer, S. Kiani, C. Fra, B. Moltchanov, and N. Baker,“Contextml: A light-weight context representation and context manage-ment schema,” in In Proc. IEEE International Symposium on WirelessPervasive Computing, May 2010, pp. 367 – 372.

[31] Ubiquitous Web Applications Working Group, “CompositeCapability/Preference Profiles (CC/PP): Structure and vocabularies2.0,” World Wide Web Consortium (W3C), W3C WorkingDraft, April 2007. [Online]. Available: http://www.w3.org/TR/2007/WD-CCPP-struct-vocab2-20070430/

[32] K. Henricksen and J. Indulska, “A software engineering frameworkfor context-aware pervasive computing,” in Pervasive Computing andCommunications, 2004. PerCom 2004. Proc. Second IEEE AnnualConference on. IEEE Computer Society, Mar 2004, pp. 77 – 86.

[33] J. E. Bardram, “The java context awareness framework (JCAF): aservice infrastructure and programming framework for context-awareapplications,” in Proc. Third international conference on PervasiveComputing, ser. PERVASIVE’05. Berlin, Heidelberg: Springer-Verlag, 2005, pp. 98–115. [Online]. Available: http://dx.doi.org/10.1007/11428572 7

[34] X. Wang, D. Zhang, T. Gu, and H. Pung, “Ontology based contextmodeling and reasoning using owl,” in Pervasive Computing and Com-

munications Workshops, 2004. Proc. Second IEEE Annual Conferenceon, March 2004, pp. 18 – 22.

[35] T. Gu, H. K. Pung, and D. Q. Zhang, “A service-oriented middlewarefor building context-aware services,” J. Network and Computer Appli-cations, vol. 28, pp. 1–18, January 2005.

[36] Q. Sheng and B. Benatallah, “Contextuml: a uml-based modeling lan-guage for model-driven development of context-aware web services,”in Mobile Business, 2005. ICMB 2005. International Conference on,W. Brookes, E. Lawrence, R. Steele, and E. Chang, Eds. IEEEComputer Society, July 2005, pp. 206 – 212.

[37] T. Hofer, W. Schwinger, M. Pichler, G. Leonhartsberger, J. Altmann,and W. Retschitzegger, “Context-awareness on mobile devices - thehydrogen approach,” in Proc. 36th Annual Hawaii InternationalConference on System Sciences (HICSS’03) - Track 9 - Volume 9,ser. HICSS ’03. Washington, DC, USA: IEEE Computer Society,2003, pp. 292.1–. [Online]. Available: http://dl.acm.org/citation.cfm?id=820756.821849

[38] D. Brickley and R. Guha, “Rdf vocabulary description language 1.0:Rdf schema,” W3C, W3C Recommendation, Feb 2004. [Online].Available: http://www.w3.org/TR/rdf-schema

[39] S. Bechhofer, F. van Harmelen, J. Hendler, I. Horrocks, D. L.McGuinness, P. F. Patel-Schneider, and L. A. Stein, “OWLweb ontology language reference,” World Wide Web Consortium(W3C), W3C Recommendation, February 2004. [Online]. Available:http://www.w3.org/TR/owl-ref/

[40] P. Korpipaa and J. Mantyjarvi, “An ontology for mobile devicesensor-based context awareness,” in Proc. 4th international andinterdisciplinary conference on Modeling and using context, ser.CONTEXT’03. Berlin, Heidelberg: Springer-Verlag, 2003, pp. 451–458. [Online]. Available: http://dl.acm.org/citation.cfm?id=1763142.1763181

[41] A. Ranganathan, J. Al-Muhtadi, and R. Campbell, “Reasoning aboutuncertain contexts in pervasive computing environments,” IEEE Per-vasive Computing, vol. 3, no. 2, pp. 62–70, 2004.

[42] H. Chang, S. Shin, and C. Chung, “Context Life Cycle ManagementScheme in Ubiquitous Computing Environments,” in Mobile DataManagement, 2007 International Conference on. IEEE ComputerSociety, May 2007, pp. 315–319.

[43] H. Truong and S. Dustdar, “A survey on context-aware web servicesystems,” International Journal of Web Information Systems, vol. 5,no. 1, pp. 5–31, 2009.

[44] J. Hong, E. Suh, and S. Kim, “Context-aware systems: A literaturereview and classification,” Expert Systems with Applications, vol. 36,no. 4, pp. 8509–8522, 2009.

[45] T. Winograd, “Architectures for context,” Human-Computer Interac-tion, vol. 16, no. 2, pp. 401–419, 2001.

[46] P. Makris, D. Skoutas, and C. Skianis, “A survey on context-awaremobile and wireless networking: On networking and computing envi-ronments’ integration,” IEEE Commun. Surveys Tutorials, 2012.

[47] C.-F. Sørensen, M. Wu, T. Sivaharan, G. S. Blair, P. Okanda,A. Friday, and H. Duran-Limon, “A context-aware middleware forapplications in mobile ad hoc environments,” in Proc. 2nd workshopon Middleware for pervasive and ad-hoc computing, ser. MPAC ’04.New York, NY, USA: ACM, 2004, pp. 107–110. [Online]. Available:http://doi.acm.org/10.1145/1028509.1028510

[48] A. K. Dey, D. Salber, M. Futakawa, and G. D. Abowd, “Anarchitecture to support context-aware applications,” Georgia Instituteof Technology, Technical Report GIT-GVU-99-23, 1999. [Online].Available: http://smartech.gatech.edu/handle/1853/3390

[49] G. D. Abowd, C. G. Atkeson, J. Hong, S. Long, R. Kooper, andM. Pinkerton, “Cyberguide: a mobile context-aware tour guide,”Wireless Networks, vol. 3, no. 5, pp. 421–433, Oct 1997. [Online].Available: http://dx.doi.org/10.1023/A:1019194325861

[50] R. Want, A. Hopper, V. Falcao, and J. Gibbons, “The active badgelocation system,” ACM Trans. Information Systems, vol. 10, no. 1, pp.91–102, Jan 1992.

[51] P. Fahy and S. Clarke, “Cass-middleware for mobile context-awareapplications,” in Workshop on Context Awareness, The Second Inter-national Conference on Mobile Systems, Applications, and Services(MobiSys). Boston, Massachusetts: ACM SIGMOBILE, June 2004,pp. 304–308.

[52] B. Guo, D. Zhang, and M. Imai, “Toward a cooperative programmingframework for context-aware applications,” Personal and UbiquitousComputing, vol. 15, no. 3, pp. 221–233, 2011.

[53] M. Roman, C. Hess, R. Cerqueira, A. Ranganathan, R. Campbell, andK. Nahrstedt, “A middleware infrastructure for active spaces,” IEEEPervasive Computing, vol. 1, no. 4, pp. 74 – 83, OIct-Dec 2002.

[54] M. Knappmeyer, N. Baker, S. Kiani, and R. Tonjes, “A context provi-sioning framework to support pervasive and ubiquitous applications,”in Proc. 4th European conference on Smart sensing and context, ser.EuroSSC’09. Berlin, Heidelberg: Springer-Verlag, 2009, pp. 93–106.

[55] D. Athanasopoulos, A. Zarras, V. Issarny, E. Pitoura, and P. Vassiliadis,“CoWSAMI: Interface-aware context gathering in ambient intelligenceenvironments,” Pervasive and Mobile Computing, vol. 4, no. 3, pp.360–389, 2008.

[56] J. Zhu, H. Pung, M. Oliya, and W. Wong, “A context realizationframework for ubiquitous applications with runtime support,” Com-munications Magazine, IEEE, vol. 49, no. 9, pp. 132–141, 2011.

[57] N. Davies, K. Cheverst, K. Mitchell, and A. Friday, “’Caches inthe Air’: Disseminating tourist information in the guide system,”in Proc. Second IEEE Workshop on Mobile Computer Systemsand Applications, ser. WMCSA ’99. Washington, DC, USA:IEEE Computer Society, 1999, pp. 11–. [Online]. Available:http://dl.acm.org/citation.cfm?id=520551.837524

[58] R. Kernchen, D. Bonnefoy, A. Battestini, B. Mrohs, M. Wagner, andM. Klemettinen, “Context-awareness in mobilife,” in Proc. 15th ISTMobile Summit. Mykonos, Greece: IST Mobile Summit, June 2006.

[59] T. Gu, H. Pung, and D. Zhang, “Toward an OSGi-based infrastructurefor context-aware applications,” IEEE Pervasive Computing, vol. 3,no. 4, pp. 66–74, 2005.

[60] Z. Yu, X. Zhou, Z. Yu, D. Zhang, and C.-Y. Chin, “An osgi-basedinfrastructure for context-aware multimedia services,” IEEE Commun.Mag., vol. 44, no. 10, pp. 136 –142, Oct 2006.

[61] V. Lesser, “Cooperative multiagent systems: A personal view of thestate of the art,” IEEE Trans. Knowl. Data Eng., vol. 11, no. 1, pp.133–142, 2002.

[62] H. Chen, T. Finin, and A. Joshi, “An ontology for context-aware per-vasive computing environments,” The Knowledge Engineering Review,vol. 18, no. 03, pp. 197–207, 2003.

[63] C. Julien and G. Roman, “Egospaces: Facilitating rapid development ofcontext-aware mobile applications,” IEEE Trans. Softw. Eng., vol. 32,no. 5, pp. 281–298, May 2006.

[64] I. Chen, S. Yang, and J. Zhang, “Ubiquitous provision of context awareweb services,” in Services Computing (SCC ’06), IEEE InternationalConference on. Chicago, IL: IEEE Computer Society, Sept 2006, pp.60–68.

[65] H.-L. Truong, L. Juszczyk, A. Manzoor, and S. Dustdar, “Escape– an adaptive framework for managing and providing contextinformation in emergency situations,” in Smart Sensing andContext, ser. Lecture Notes in Computer Science, G. Kortuem,J. Finney, R. Lea, and V. Sundramoorthy, Eds. Springer Berlin/ Heidelberg, 2007, vol. 4793, pp. 207–222. [Online]. Available:http://dx.doi.org/10.1007/978-3-540-75696-5 13

[66] H.-L. Truong, S. Dustdar, D. Baggio, S. Corlosquet, C. Dorn, G. Giu-liani, R. Gombotz, Y. Hong, P. Kendal, C. Melchiorre, S. Moretzky,S. Peray, A. Polleres, S. Reiff-Marganiec, D. Schall, S. Stringa,M. Tilly, and H. Yu, “incontext: A pervasive and collaborative workingenvironment for emerging team forms,” in Applications and the Internet(SAINT 2008), International Symposium on. IEEE Computer Society,Aug 2008, pp. 118 –125.

[67] D. de Almeida, C. de Souza Baptista, E. da Silva, C. Campelo,H. de Figueiredo, and Y. Lacerda, “A context-aware system basedon service-oriented architecture,” in Advanced Information Networkingand Applications (AINA ’06), 20th International Conference on, vol. 1.IEEE Computer Society, Apr 2006, p. 6.

[68] R. T. Fielding, “Architectural styles and the design of network-basedsoftware architectures,” Ph.D. dissertation, University of California,2000.

[69] R. Greiner, C. Darken, and N. I. Santoso, “Efficient reasoning,”ACM Comput. Surv., vol. 33, no. 1, pp. 1–30, Mar. 2001. [Online].Available: http://doi.acm.org/10.1145/375360.375363

[70] S. Russell, P. Norvig, J. Canny, J. Malik, and D. Edwards, Artificialintelligence: a modern approach, 3rd ed. Englewood Cliffs, NJ:Prentice Hall, Dec 2009.

[71] C. Anagnostopoulos, A. Tsounis, and S. Hadjiefthymiades,“Context awareness in mobile computing environments,”Wireless Personal Communications, vol. 42, pp. 445–464, 2007, 10.1007/s11277-006-9187-6. [Online]. Available:http://dx.doi.org/10.1007/s11277-006-9187-6

[72] E. Kim, S. Helal, and D. Cook, “Human activity recognition andpattern discovery,” IEEE Pervasive Computing, vol. 9, no. 1, pp. 48–53,2010. [Online]. Available: http://dx.doi.org/10.1109/MPRV.2010.7

[73] A. Goultiaeva and Y. Lesperance, “Incremental plan recognition in anagent programming framework,” in Working Notes of the AAAI Work-

shop on Plan, Activity, and Intention Recognition (PAIR), Vancouver,BC, July 2007.

[74] P. Korpipaa, M. Koskinen, J. Peltola, S.-M. Makela, and T. Seppanen,“Bayesian approach to sensor-based context awareness,” PersonalUbiquitous Comput., vol. 7, no. 2, pp. 113–124, Jul. 2003. [Online].Available: http://dx.doi.org/10.1007/s00779-003-0237-8

[75] P. Nurmi, P. Floreen, M. Przybilski, and G. Linden, “A framework fordistributed activity recognition in ubiquitous systems,” in ProceedingsInternational Conference on Artificial Intelligence (ICAI), vol. 1. LasVegas, Nevada, USA: CSREA Press, June 2005, pp. 650–655.

[76] T. R. Gruber, “A translation approach to portable ontologyspecifications,” Knowl. Acquis., vol. 5, no. 2, pp. 199–220, Jun. 1993.[Online]. Available: http://dx.doi.org/10.1006/knac.1993.1008

[77] M. Schmidt-Schauß and G. Smolka, “Attributive concept descriptionswith complements,” Artificial intelligence, vol. 48, no. 1, pp. 1–26,1991.

[78] D. Ejigu, M. Scuturici, and L. Brunie, “Semantic approach to contextmanagement and reasoning in ubiquitous context-aware systems,”in Digital Information Management (ICDIM’07), 2nd InternationalConference on, vol. 1. Lyon: IEEE, Oct 2007, pp. 500–505. [Online].Available: http://dx.doi.org/10.1109/ICDIM.2007.4444272

[79] R. Klinger and K. Tomanek, “Classical probabilistic models and con-ditional random fields,” Technische Universitat Dortmund, Dortmund,Germany, Algorithm Engineering Report TR07-2-013, Dec 2007.

[80] D. Lewis, “Naive (bayes) at forty: The independence assumption ininformation retrieval,” in Machine Learning: ECML-98, ser. LectureNotes in Computer Science, C. Nedellec and C. Rouveirol, Eds.Springer Berlin / Heidelberg, 1998, vol. 1398, pp. 4–15.

[81] L. Rabiner and B. Juang, “An introduction to hidden markov models,”IEEE ASSP Mag., vol. 3, no. 1, pp. 4–16, 1986.

[82] I. Csiszar, “MaxEnt, mathematics, and information theory,” Maximumentropy and Bayesian methods, pp. 35–50, 1996.

[83] J. Lafferty, A. McCallum, and F. Pereira, “Conditional random fields:Probabilistic models for segmenting and labeling sequence data,” inProc. 18th International Conference on Machine Learning. SanFrancisco, CA, USA: Morgan Kaufmann Publishers Inc., 2001, pp.282–289.

[84] N. Oliver, E. Horvitz, and A. Garg, “Layered representations for humanactivity recognition,” in Proc. Fourth IEEE International Conferenceon Multimodal Interfaces. IEEE Computer Society, 2002, pp. 3–8.[Online]. Available: http://dx.doi.org/10.1109/ICMI.2002.1166960

[85] P. Turaga, R. Chellappa, V. Subrahmanian, and O. Udrea, “Machinerecognition of human activities: A survey,” IEEE Trans. Circuits Syst.Video Technol., vol. 18, no. 11, pp. 1473–1488, Nov 2008. [Online].Available: http://dx.doi.org/10.1109/TCSVT.2008.2005594

[86] D. Wen-Yu, X. Ke, and L. Meng-Xiang, “A Situation Calculus-basedApproach To Model Ubiquitous Information Services,” Arxiv preprintcs/0311052, 2003.

[87] F. Pirri and R. Reiter, “Some contributions to the metatheory of thesituation calculus,” J. ACM (JACM), vol. 46, no. 3, pp. 325–361, May1999. [Online]. Available: http://doi.acm.org/10.1145/316542.316545

[88] C. B. Anagnostopoulos, P. Pasias, and S. Hadjiefthymiades, “Aframework for imprecise context reasoning,” in IEEE InternationalConference on Pervasive Services. IEEE Computer Society, 15-20July 2007, pp. 181–184. [Online]. Available: http://dx.doi.org/10.1109/PERSER.2007.4283913

[89] J. Yin, Q. Yang, D. Shen, and Z. Li, “Activity recognition via user-tracesegmentation,” ACM Trans. Sensor Networks (TOSN), vol. 4, no. 4, pp.1–34, 2008.

[90] T. Bosse, F. Both, C. Gerritsen, M. Hoogendoorn, and J. Treur,“Model-based reasoning methods within an ambient intelligent agentmodel,” in Constructing Ambient Intelligence, ser. Communicationsin Computer and Information Science, M. Muhlhauser, A. Ferscha,and E. Aitenbichler, Eds. Springer Berlin Heidelberg, 2008,vol. 11, pp. 352–370. [Online]. Available: http://dx.doi.org/10.1007/978-3-540-85379-4 40

[91] F. Both, M. Hoogendoorn, and J. Treur, “Model-based ambient analysisof human task execution,” in Proc. 1st international conference onPErvasive Technologies Related to Assistive Environments, ser.PETRA ’08. New York, NY, USA: ACM, 2008, pp. 92:1–92:8.[Online]. Available: http://doi.acm.org/10.1145/1389586.1389690

[92] L. Goix, M. Valla, L. Cerami, and P. Falcarin, “Situation inferencefor mobile users: a rule based approach,” in Mobile DataManagement (MDM ’07), International Conference on. Mannheim,Germany: IEEE, May 2007, pp. 299–303. [Online]. Available:http://dx.doi.org/10.1109/MDM.2007.63

[93] J. Serrano, J. Serrat, and A. Galis, “Ontology-based context informationmodelling for managing pervasive applications,” in Autonomic and

Autonomous Systems (ICAS ’06), International Conference on, P. Dini,D. Ayed, C. Dini, and Y. Berbers, Eds. California, USA: IEEEComputer Society, 19-21 July 2006.

[94] T. Gu, X. Wang, H. Pung, and D. Zhang, “An ontology-based contextmodel in intelligent environments,” in Proc. Communication Networksand Distributed Systems Modeling and Simulation Conference. SanDiego, California, USA: The Society for Modeling and SimulationInternational (SCS), 18–21 Jan 2004, pp. 270–275.

[95] E. Christopoulou, C. Goumopoulos, and A. Kameas, “An ontology-based context management and reasoning process for UbiCompapplications,” in Proc. 2005 Joint Conference on Smart Objects andAmbient Intelligence: Innovative Context-aware Services, Usages andTechnologies, ser. sOc-EUSAI ’05, ACM. New York, NY, USA:ACM, 2005, pp. 265–270. [Online]. Available: http://doi.acm.org/10.1145/1107548.1107613

[96] M. Knappmeyer, E. Wittkorn, S. Kiani, R. Tonjes, and N. Baker, “Con-text provisioning middleware with probabilistic reasoning support,” inProc. 20th Future Network and Mobile Summit, Warsaw, Poland, 2011.

[97] D. L. Vail, M. M. Veloso, and J. D. Lafferty, “Conditionalrandom fields for activity recognition,” in Proc. 6th internationaljoint conference on Autonomous agents and multiagent systems, ser.AAMAS ’07. New York, NY, USA: ACM, 2007, pp. 235:1–235:8.[Online]. Available: http://doi.acm.org/10.1145/1329125.1329409

[98] J. Scholtz and S. Consolvo, “Towards a discipline for evaluatingubiquitous computing applications,” Intel Research, Tech. Rep.IRS-TR-04-004, Jan 2004. [Online]. Available: http://www.seattle.intel-research.net/pubs/022520041200 232.pdf

[99] Y. Rogers, K. Connelly, L. Tedesco, W. Hazlewood, A. Kurtz, R. E.Hall, J. Hursey, and T. Toscos, “Why it’s worth the hassle: the valueof in-situ studies when designing ubicomp,” in Proc. 9th internationalconference on Ubiquitous computing, ser. UbiComp ’07. Berlin,Heidelberg: Springer-Verlag, 2007, pp. 336–353. [Online]. Available:http://dl.acm.org/citation.cfm?id=1771592.1771612

[100] K. Connelly, K. Siek, I. Mulder, S. Neely, G. Stevenson, and C. Kray,“Evaluating pervasive and ubiquitous systems,” Pervasive Computing,IEEE, vol. 7, no. 3, pp. 85 –88, july-sept. 2008.

[101] H. Yan and T. Selker, “Context-aware office assistant,” in Proceedingsof the 5th international conference on Intelligent user interfaces, ser.IUI ’00. New York, NY, USA: ACM, 2000, pp. 276–279. [Online].Available: http://doi.acm.org/10.1145/325737.325872

[102] A. K. Dey and G. D. Abowd, “Cybreminder: A context-aware systemfor supporting reminders,” in Proc. 2nd international symposiumon Handheld and Ubiquitous Computing, ser. HUC ’00. London,UK: Springer-Verlag, 2000, pp. 172–186. [Online]. Available:http://dl.acm.org/citation.cfm?id=647986.757284

[103] S. Intille, E. Tapia, J. Rondoni, J. Beaudin, C. Kukla, S. Agarwal,L. Bao, and K. Larson, “Tools for studying behavior and technologyin natural settings,” in UbiComp 2003: Ubiquitous Computing,ser. Lecture Notes in Computer Science, A. Dey, A. Schmidt,and J. McCarthy, Eds. Springer Berlin / Heidelberg, 2003, vol.2864, pp. 157–174. [Online]. Available: http://dx.doi.org/10.1007/978-3-540-39653-6 13

[104] S. Consolvo, L. Arnstein, and B. R. Franza, “User study techniques inthe design and evaluation of a ubicomp environment,” in Proc. the 4thinternational conference on Ubiquitous Computing, ser. UbiComp ’02.London, UK: Springer-Verlag, 2002, pp. 73–90. [Online]. Available:http://dl.acm.org/citation.cfm?id=647988.741490

[105] J. Froehlich, M. Y. Chen, S. Consolvo, B. Harrison, and J. A. Landay,“Myexperience: a system for in situ tracing and capturing of userfeedback on mobile phones,” in Proc. 5th international conferenceon Mobile systems, applications and services, ser. MobiSys ’07.New York, NY, USA: ACM, 2007, pp. 57–70. [Online]. Available:http://doi.acm.org/10.1145/1247660.1247670

[106] S. Consolvo and M. Walker, “Using the experience sampling methodto evaluate ubicomp applications,” Pervasive Computing, IEEE, vol. 2,no. 2, pp. 24 – 31, Apr-Jun 2003.

[107] A. Crabtree, S. Benford, C. Greenhalgh, P. Tennent, M. Chalmers, andB. Brown, “Supporting ethnographic studies of ubiquitous computingin the wild,” in Proc. 6th conference on Designing Interactive systems,ser. DIS ’06. New York, NY, USA: ACM, 2006, pp. 60–69. [Online].Available: http://doi.acm.org/10.1145/1142405.1142417

[108] S. S. Intille, L. Bao, E. M. Tapia, and J. Rondoni, “Acquiring in situtraining data for context-aware ubiquitous computing applications,” inProc. SIGCHI conference on Human factors in computing systems,ser. CHI ’04. New York, NY, USA: ACM, 2004, pp. 1–8. [Online].Available: http://doi.acm.org/10.1145/985692.985693

[109] J. Zacks and B. Tversky, “Event structure in perception and concep-tion,” Psychological Bulletin, vol. 127, no. 1, pp. 3–21, 2001.

[110] Y. Li and J. A. Landay, “Activity-based prototyping of ubicompapplications for long-lived, everyday human activities,” in Proc.the twenty-sixth annual SIGCHI conference on Human factors incomputing systems, ser. CHI ’08. New York, NY, USA: ACM,2008, pp. 1303–1312. [Online]. Available: http://doi.acm.org/10.1145/1357054.1357259

[111] S. S. Intille, J. Rondoni, C. Kukla, I. Ancona, and L. Bao,“A context-aware experience sampling tool,” in CHI ’03 extendedabstracts on Human factors in computing systems, ser. CHI EA ’03.New York, NY, USA: ACM, 2003, pp. 972–973. [Online]. Available:http://doi.acm.org/10.1145/765891.766101

[112] S. Carter, J. Mankoff, and J. Heer, “Momento: support forsituated ubicomp experimentation,” in Proc. SIGCHI conferenceon Human factors in computing systems, ser. CHI ’07. NewYork, NY, USA: ACM, 2007, pp. 125–134. [Online]. Available:http://doi.acm.org/10.1145/1240624.1240644

[113] E. O’Neill, M. Klepal, D. Lewis, T. O’Donnell, D. O’Sullivan,and D. Pesch, “A testbed for evaluating human interaction withubiquitous computing environments,” in First International Conferenceon Testbeds and Research Infrastructures for the Development ofNetworks and Communities (Tridentcom) 2005., ser. TRIDENTCOM’05. Washington, DC, USA: IEEE Computer Society, 2005, pp.60–69. [Online]. Available: http://dx.doi.org/10.1109/TRIDNT.2005.7

[114] S. S. Intille, K. Larson, E. M. Tapia, J. S. Beaudin, P. Kaushik,J. Nawyn, and R. Rockinson, “Using a live-in laboratory forubiquitous computing research,” in Proc. 4th international conferenceon Pervasive Computing, ser. PERVASIVE’06. Berlin, Heidelberg:Springer-Verlag, 2006, pp. 349–365. [Online]. Available: http://dx.doi.org/10.1007/11748625 22

[115] S. Hossein, S. Helal, and A. Mendez-Vasquez, Sensory DatasetDescription Language (SDDL) Specification, Mobile and PervasiveComputing Laboratory Department of Computer and InformationScience and Engineering Std., Rev. 1.0, April 2009. [Online].Available: http://www.icta.ufl.edu/persim/sddl

[116] A. M. Law and W. D. Kelton, Simulation Modelling and Analysis,3rd ed. Mcgraw-Hill Higher Education, Dec. 1999.

[117] J. Barton and V. Vijayaraghavan, “UBIWISE, a simulator forubiquitous computing systems design,” Hewlett-Packard Labs, PaloAlto, Tech. Rep. HPL-2003-93, 2003. [Online]. Available: http://www.hpl.hp.com/techreports/2003/HPL-2003-93.pdf

[118] R. Morla and N. Davies, “Evaluating a location-based application: Ahybrid test and simulation environment,” IEEE Pervasive Computing,vol. 3, no. 3, pp. 48–56, Jul-Sept 2004.

[119] S. Jang, Y. Lee, W. Woo, G. U. vr Lab, and S. Korea, “CIVE: Context-based interactive system for distributed virtual environment,” in The14th International Conference on Artificial Reality and Telexistence(ICAT 2004), Seoul, S. Korea, 30 Nov – 02 Dec 2004, pp. 495–498.

[120] H. Nishikawa, S. Yamamoto, M. Tamai, K. Nishigaki, T. Kitani,N. Shibata, K. Yasumoto, and M. Ito, “Ubireal: Realistic smartspacesimulator for systematic testing,” in UbiComp 2006: UbiquitousComputing, ser. Lecture Notes in Computer Science, P. Dourish andA. Friday, Eds. Springer Berlin / Heidelberg, 2006, vol. 4206, pp.459–476. [Online]. Available: http://dx.doi.org/10.1007/11853565 27

[121] I. Kim, H. Park, Y. Lee, H. Lee, and B. Noh, “Design of context-awareness simulation toolkit for ubiquitous computing,” in IndustrialElectronics, 2006 IEEE International Symposium on, vol. 4, July 2006,pp. 3220 –3225.

[122] J. Barbosa, R. Hahn, D. Bonatto, F. Cecin, and C. Geyer, “Evaluationof a large-scale ubiquitous system model through peer-to-peer protocolsimulation,” in Distributed Simulation and Real-Time Applications(DS-RT 2007), 11th IEEE International Symposium, Oct 2007, pp.175–181.

[123] M. Martin and P. Nurmi, “A generic large scale simulator for ubiq-uitous computing,” in Mobile and Ubiquitous Systems: Networking &Services, 2006 Third Annual International Conference on. IEEE,2006, pp. 1–3.

[124] E. Reetz, M. Knappmeyer, S. Kiani, A. Anjum, N. Bessis, andR. Tonjes, “Performance simulation of a context provisioning mid-dleware based on empirical measurements,” Simulation ModellingPractice and Theory, 2012.

[125] A. K. Dey, D. Salber, G. D. Abowd, and M. Futakawa, “Theconference assistant: Combining context-awareness with wearablecomputing,” in Proceedings of the 3rd IEEE International Symposiumon Wearable Computers, ser. ISWC ’99. Washington, DC, USA:IEEE Computer Society, 1999, pp. 21–. [Online]. Available:http://dl.acm.org/citation.cfm?id=519309.856496

[126] X. Gong, T. Chandrashekhar, J. Zhang, and H. Poor, “Opportunisticcooperative networking: To relay or not to relay?” Selected Areas

in Communications, IEEE Journal on, vol. 30, no. 2, pp. 307 –314,february 2012.

[127] L. Sanchez, J. Galache, V. Gutierrez, J. Hernandez, J. Bernat,A. Gluhak, and T. Garcia, “Smartsantander: The meeting point betweenfuture internet research and experimentation and the smart cities,” inFuture Network Mobile Summit (FutureNetw), 2011, june 2011, pp. 1–8.

[128] S. Kiani, A. Anjum, N. Antonopoulos, and M. Knappmeyer, “Context-aware service utilisation in the clouds and energy conservation,”Journal of Ambient Intelligence and Humanized Computing, pp. 1–21,2012.

[129] G. D. Abowd, “What next, ubicomp?: celebrating an intellectualdisappearing act,” in Proc. 2012 ACM Conference on UbiquitousComputing, ser. UbiComp ’12. New York, NY, USA: ACM, 2012,pp. 31–40. [Online]. Available: http://doi.acm.org/10.1145/2370216.2370222

[130] M. Weiser, “Some computer science issues in ubiquitous computing,”Communications of the ACM, vol. 36, no. 7, pp. 75–84, 1993.

Dr. Michael Knappmeyer received his Ph.D.(Computer Science) from the University of theWest of England, Bristol, UK, in 2012. His the-sis presented a Context Provisioning Middlewarewith Support for Evolving Awareness. In 2006 hereceived the Diplom-Informatiker degree from theUniversity of Applied Sciences Osnabruck, Ger-many. As research associate Michael participated inthe European research projects “C-MOBILE” and“Context Casting (C-CAST)”. In the latter he ledthe context reasoning activity and contributed to

the overall system architecture. His interests include smart spaces, contextmodelling, reasoning and mobile device management.

Dr. Saad Liaquat Kiani received his B.E. from theNational University of Sciences and Technology, Is-lamabad, Pakistan, in 2003. He received his M.S. inComputer Engineering from Kyung Hee University,South Korea, in 2007 and completed his Ph.D. inComputer Science at the University of the West ofEngland, Bristol, UK, in 2011. He is currently aSenior Lecturer in Networks and Mobile Computingat the University of the West of England. He isalso a visiting lecturer at Cardiff University’s Schoolof Computer Science and Informatics. His research

interests are in the areas of mobile and distributed computing, context-awaresystems and participatory sensing.

Eike Steffen Reetz has studied Electrical Engineer-ing with focus on communication technology at theUniversity of Applied Sciences Osnabruck (UASO)and received his “Diplom Ingenieur (FH)” degree inAugust 2007. He is currently working as a ResearchAssistant at the Mobile Communication ResearchGroup at UASO. His current research interests in-clude the evaluation of context provisioning systemsas well as testing of context aware services. Heparticipated in the European research projects “C-MOBILE” and “Context Casting (C-CAST)” and

is currently involved in the European research project “IoT.est”. Eike isworking towards his Ph.D. in Electrical Engineering with the research scopeof evaluation and testing of IoT-aware services.

Nigel Baker led the Mobile & Ubiquitous SystemsGroup, was the Co-Director of CCCS research cen-tre and Associate Professor (Reader) in ComputerScience at the University of the West of Englanduntil 2011. His first degrees were in Physics andNuclear & Particle Physics. His specialisms in thelast twenty years have been Real Time Systems,Computer Networks, Distributed Systems and inthe last decade Mobile Communications. He was avisiting researcher at CERN, Geneva for six years.He was also a Motorola Fellow until 2006 through

which he developed and led the Mobile Applications of Software Technologies(MAST) Programme whilst a recipient of a Royal Academy of EngineeringIndustrial Fellowship award with Motorola European Cellular InfrastructureDivision (ECID), Swindon. During this collaboration with Motorola heworked on several EU 5th, 6th and 7th Framework projects.

Prof. Dr.-Ing. Ralf Tonjes is heading the mobilecommunications group at the University of AppliedSciences Osnabruck. He studied communication en-gineering at the University of Hannover and biomed-ical engineering at the University of Strathclydein Glasgow. In 1998 he received his Dr.-Ing. de-gree (summa cum laude) in electrical engineeringfrom the University of Hannover. From 1998 to2005 he was with Ericsson Research, working onfuture mobile networks and representing Ericssonin standardisation. In 2005 Ralf Tonjes joined the

University of Applied Sciences of Osnabruck as full professor for MobileCommunications. He is a TPC member of several international conferencesand (co-) authored more than seventy scientific publications. His current re-search interests include wireless communication networks, Internet of things,context-aware service platforms and test automation.

Survey of Context Provisioning Middleware

Documents