A Reusable, Extensible Infrastructure for Augmented Field Trips

Mark J. Weal, Don Cruikshank, Danius T. Michaelides,David E. Millard and David C. De Roure

IAM Group, University of Southampton, UK

{mjw}{dgc}{dtm}{dem}{dder}@ecs.soton.ac.uk

Eva Hornecker, John Halloran and Geraldine FitzpatrickInteract Lab, Dept. of Informatics, University of Sussex, UK

{eh49}{johnhall}{G.A.Fitzpatrick}@sussex.ac.uk

Abstract

This paper describes a reusable pervasive informationinfrastructure developed as part of the Equator IRC, de-signed to allow the construction of literacy based eLearn-ing activities on top of material created as part of a moretraditional visitors system.

The architecture of the system is described along withdetails of the creation of the curated material and the sub-sequent adaption of the system by local primary schoolteachers to create a literacy experiences. Results of thefirst trials of the system are presented with conclusionsdrawn and discussion of future directions.

1. Introduction

Wireless handheld technologies are increasingly be-ing used in outdoor environments to provide engaginglearning experiences for children in authentic settings.Much of the focus of this work to date had been aboutsupporting scientific enquiry (Wireless Coyote [3], Gayet. al. [2], Ambient Wood [5]) and has often been set upas a one-off experience. The focus of this paper is to re-port on the first phase of a project that seeks to sup-port a more diverse range of learning activities, initiallycreative writing, and to provide a persistent infrastruc-ture that can easily be repurposed by teachers and oth-ers to create new experiences.

The Equator IRC (Interdisciplinary Research Col-laboration) is a large UK project investigating theboundary between the physical and digital worlds.Chawton House Library is an Elizabethan manor housethat once belonged to Jane Austen’s brother Edward,and is now owned by a charitable foundation that op-erates it as a study centre of early English women’s

writing. A wide variety of visitors to the house can beidentified including groups of schoolchildren using thegrounds for a number of possible curriculum based ex-periences, academics studying at the Centre, coach par-ties from such as the Jane Austen Society of America,and local Flower Societies.

In order to move towards a generic infrastructure tosupport a wide range of visitors we have been work-ing with the curators to produce a fairly standard vis-itors style system, and with teachers from WhiteleyPrimary School in Hampshire to create an augmentedfield trip for a group of Year 5 students (aged 10-11).An initial aim was to engage in a co-design process [4]with the curators at the house in order to create a ba-sic visitors system where information can be relayedto the visitors at appropriate points in the grounds ofthe house. The focus of this paper is describing the in-frastructure to support these experiences. The teachersfrom the school included the literacy advisor for the re-gion. All the teachers are working closely with the rel-evant areas of the curriculum.

Other learning experiences for children using loca-tive, context-based technologies have focused on scien-tific discovery, learning about nature, learning abouthistory etc., Here, the main emphasis of the school ex-perience has been on using the landscape as a writ-ing aide. The children explored the grounds and con-structed narratives around their discoveries. The teach-ers left instructions and activities for the children whichflashed up on the PDAs at various locations around thegrounds along with selected audio clips created by thecurators as part of the separate visitors system. Thepupils were asked to perform small tasks such as torecord a dialogue between two people arriving at thehouse for the first time. The children can also recordtheir own annotations for access later in the classroom

when writing their stories.The Chawton project architecture has been designed

to support a wide range of different users and as a re-sult a number of issues needed to be addressed. Howcan we enable the teachers and curators to create newexperiences and engage in co-design? How can very dif-ferent types of experiences be supported, i.e. experi-ences for different types of visitors to the house? Howcan we augment the grounds of a historic house with-out disturbing its atmosphere? How do we create an ex-tensible and persistent infrastructure, that can be ex-tended in terms of devices, infrastructure, content andtypes of experiences.

2. The Card Metaphor

To orchestrate the activities at Chawton we haveapplied a hypermedia technique called sculptural hy-pertext, coined by Mark Bernstein [1]. It describes theauthoring of a hypertext system where initially everynode is linked to every other node, and the author-ing process removes links to produce the final hyper-graph. These links are normally pruned using precondi-tions and assertions; in most systems describing them-selves as sculptural, link pruning is dynamic. This dif-fers from the more traditional calligraphic authoringsuch as hand authored Web pages, where new items arecreated unlinked and authors add links progressively toachieve the desired pattern of connection. This tech-nique has been used in other pervasive projects to or-chestrate the experience [6].

This system maintains a user context, or systemstate, and then decides which of the available cardsheld by the system fits with the current context. Theuser context includes user modelling of the participantsas well as other environmental factors, in the case ofthe Chawton trials, the participants location as wellas a global clock which controlled the overall experi-ence. When a card is experienced by the participant,the user context can change as a result of the asser-tions held on the card. This allows us to build in pre-requisites where a participant has to experience certaincards before they are able to access others. By encod-ing specific requirements on cards the experience de-signer gains control over the order in which events canhappen as well as various timing requirements whichmight exist. This can be viewed as a form of state ma-chine where the different context states are nodes inthe hypermedia and the links between states are dy-namically pruned based on the context information.

Different cards had different functionality associ-ated with them, for instance a simple information cardmight have a piece of audio attached to it to be played

automatically, or a capture card might ask the chil-dren to record a description of the particular part ofthe grounds they were in. Cards can then be arrangedinto sequences and decks for particular activities. Theteachers were able to put specific constraints on thecards to ensure that an activity lasted a minimumamount of time or that the next card was played af-ter a maximum allowed time for an activity.

By using a standardised ontology for representingthe cards, the results of an activity (i.e. the recordingof a description) can be made available to the otherusers of the system as information. In this way, the in-formation cards created by the curators could be in-cluded in sequences in the literacy scenario.

3. The Chawton Architecture

Figure 1 shows an overview of the Chawton archi-tecture. The pervasive infrastructure comprised at thebase level: wireless networking, location sensing tech-nology (GPS and RF pingers) as well as PDAs carriedby the children.

On top of this infrastructure sat the generic informa-tion system, an orchestration tool (constructed usingEQUIP, the EQUator Infrastructure Platform) along-side a triple store (the AKT 3store). The AKT 3Storeis a MySQL based triple store that provides the facil-ity to store and query RDF statements. It stored thecards used by the system during the trials. This wasresponsible for delivering information to participants,either visitors using the visit system or children tak-ing part in the literacy experience. Orchestration wasbased on participants current context and metadata at-tached to the content.

Figure 1. The Chawton project architecture.

The PDAs used in the trial were standard iPaq pock-etPCs which communicated with a central server viawireless using HTTP. Four battery powered wirelessaccess points were placed around the grounds to pro-

vide coverage. Interaction with the PDA was througha single application which displayed the various cardsand allowed the participants to listen to and record au-dio fragments, read textual instructions and make briefnotes. The PDA sent information to the server and re-turned an XML document containing instructions forthe software based on the matched context.

Location tracking was carried out using a combina-tion of RF pingers and GPS. Each of the PDAs had anRF pinger receiver attached via the serial port whichreceived pings from discrete pinger boxes placed aroundthe grounds. The pingers, with a range of around 5 me-ters, were developed by the University of Bristol in aprevious Equator project. This provided a mechanismfor identifying and highlighting specific objects or lo-cations such as the South Door or a specific tree thathad been struck by lightning.

GPS was used for larger expanses, such as the lawnarea or the walled gardens. GPS information was col-lected from a GPS receiver and sent via Bluetooth tothe PDA. Pinger locations were given priority over GPSlocations by the system in the event of both providingmatches. The PDA did no processing of location infor-mation but simply forwarded it to the central server.

The architecture was constructed around theEQUIP framework1. This provides a componentbased tuple space designed for the developmentof the types of pervasive experiences we are in-terested in. The dataspace holds the current stateof the system. As that state changes the compo-nents are informed of the changes in state they areinterested in. The dataspace operates on a broad-cast model but for simplicity the diagram indicatesthe flow of data around the components using ar-rows. The four main components that comprise theEQUIP system are:

Device Component: Containing the HTTP serverused to communicate to and from the hand helddevices. This processes information from the de-vice into the tuple space then returns the currentavailable deck for the device as an XML document.

Location Component: Parses the GPS and pingerinformation placed in the EQUIP tuple space.Where the information matches known locationsit enters the symbolic location for the device intothe tuple space.

Orchestration Component: Performs a queryof the cards in the 3Store based on the cur-rent context (comprised of the device, deck

1 http://www.crg.cs.nott.ac.uk/˜cmg/Equator/Downloads/docs/equip-tech.pdf

being used for the experience, current loca-tion, and previously viewed content). The resultis turned into a deck which is placed into the tu-ple space.

Logger Component: Logs all traffic through the tu-ple space.

4. The Curated Material

The curators generated content both for their ownexperiences (tours) but also for re-use in other experi-ences such as the literacy scenario. For the initial trials,content creation involved the in-situ recording of au-dio content by the curators as they walked around thegrounds. A post processing phase was required in or-der to segment the audio into individual clips and ppro-priate context metadata was added. The intention is toproduce a system that allows for ready revision and up-dating of the information content rather than aimed atproviding a highly polished specific tour of the grounds.

Locations were defined in two ways. Either by theplacement of a pinger used to indicate a region de-fined by being within range of the pinger or by walk-ing out a region using the hand held device. A staticdeck of cards was produced which included paired in-structions such as ‘Go to the walled garden and pressOK’, followed by the instruction ‘Walk slowly aroundthe perimeter of the walled garden and press OK whencompleted.’ The resultant system logs could then beautomatically processed into GPS regions that definedthe particular location.

5. The Literacy Scenario

The literacy scenario applied the infrastructureframework to the creation of a field trip for a groupof children with the aim of steering them through thegathering of material for them to later use when writ-ing a fictional story. The teachers devised a scenariowhere at various locations around the grounds the chil-dren would be given a range of different activities tocarry out. Some required them to write in their exer-cise books while others involved the recording of textand audio on the PDAs.

The teachers were able to make use of most of the lo-cations previously identified by the curators but wereable to add some specific locations that were appro-priate to the literacy exercise but had not featuredin the material created previously. As well as creat-ing the activities the teachers also selected informationcreated by the curators that could be given to the chil-dren to augment their understanding of the landscapeand house. It is this form of re-use that was one of the

main design goals of the infrastructure developed forthis project.

The teachers composed the literacy scenarios as se-quences of activities to take place at named locations.These activities had various specific properties to dowith the timing of the activity, the method of interac-tion used by the children and the order in which theytake place. The created scenarios were converted intoa deck of cards of the following types:

Information cards were selected from the informa-tion cards created by the curators and gave thechildren additional information about the variouslocations, perhaps after they’d been asked to thinkwhat it might have been used for.

Activity cards were designed to prompt the childrento think more deeply about their surroundings.They might be ask to make notes in their papernotebooks, reflect on what might have occurredat a particular location or think about how differ-ent characters might react in the current place.

Capture cards required the children to record infor-mation digitally. This could be a piece of audio,some dialogue between two imaginary charactersperhaps, or perahps a short piece of text. The chil-dren could then review or rerecord the informationif they were unhappy with it.

Sequence cards were used to group a set of tasks to-gether in a location. A simple sequence might bethat the children listened to an audio clip aboutthe stable block (Information), are asked to thinkabout who might have worked there (Activity) andthen finally record a short piece of dialogue tak-ing place between their imagined characters (Cap-ture).

As is true of most field trips, controlling the tim-ing of the various activities is important. By modellingorchestration information as part of the ontology theteachers were able to specify how long the childrenshould spend on particular activities and keep the over-all activity progressing.

The logging of activities is all carried out using thesame ontology allowing for easy reuse of the data col-lected during analysis post trials and also facilitat-ing the re-use of information during the trials. It ishoped that the information space will evolve over timewith the curated information increasing and experiencebuilders such as the teachers being able to make use ofcards from previous activities.

On returning to the school the children wrote theirstories based on material they had collected at thehouse. An electronic journal of their activities was pro-vided via the Web which allowed them to review their

experience by seeing the deck of cards that they col-lected during their exploration. This allowed them torelisten to audio clips and access their own recordingsand text.

6. Initial Trials

A group of children from Whiteley Primary Schoolwere brought out to Chawton House for a day of tri-als. They were given a guided tour of the insides ofthe house to set the scene before being divided intopairs for the later activities. Previously, at the school,they had been given a brief tutorial on how to use thePDA device. The literacy experience had two parts toit. In the first session the children explored the groundsfreely and at various locations were required to carryout a sequence of activities. Figure 2 shows the chil-dren exploring in pairs.

Figure 2. The device in use during the trial.

In the first phase the children gathered informationthrough listening to audio prepared by the curators,following instructions set out by the teachers and ob-serving the landscape, flora and fauna around them.Having completed the first phase the children weregathered together to briefly share their findings witheach other before moving on to the second phase wherethey went back to two locations of their choice and re-ceived further instructions on more specific activitiesto help in the creation of their stories.

6.1. Observations

Both the children and teachers felt that the dayhad been a success and met their expectations. Thehead teacher commented on how highly motivated thechildren had been by the approach. The children wereable, through the technology, to bring their experienceback to the classroom, as one teacher put it, ‘They allgrabbed little bits of Chawton House yesterday.’

A problem encountered by teachers on more tradi-tional field trips with clipboards is that the children of-ten see the activities as a race in which the first to com-plete all the tasks wins. The ability to pace the activi-ties by forcing the children to stop and reflect and notallowing them access to the next task until a fixed pe-riod of time had passed proved highly effective.

The children were readily accepting of the technol-ogy and quick to highlight what they liked about itduring interviews. They enjoyed the freedom of beingable to explore the various locations without having tostay to a fixed route. Comments made included thatthey preferred it to clipboards where the paper canblow around and become awkward to handle, they ap-preciated the authority of the voices speaking to themand enjoyed the fact they could relisten to informa-tion if they chose to. The children showed an aware-ness of some of the problems with the prototype sys-tem, namely speed of reaction and occasional softwareerrors, but this did not seem to distract them and theywere positive about the experience.

One of the main technical problems encountered hasbeen the speed of response to changes in location. Theasynchronous nature of the architecture may have ex-acerbated this along with the sheer speed of the chil-dren during the trials. As well as improving the speedof response, more directed tasks can be used to steerthe participants to specific locations if required.

The encoding of the experience was in this instancea two phase process, with discussion with the teachersleading to a scenario document that was then createdwithin the infrastructure by the researchers. This pro-cess has helped inform the requirements of a more au-tomatic authoring system which will allow the teachersto perform this task themselves and remove the needfor direct intervention.

As we had anticipated, this first trial has helped togive the curators and teachers a clearer idea of whatwe were working towards. Having seen the technologyin action and understood more fully what it is capa-ble of discussions about future trials have proved to beenthusiastic and highly productive.

7. Conclusions

The first trial with the Chawton prototype has al-lowed us to test the infrastructure and begin to in-vestigate the issues raised when teachers create fieldtrips using information provided by the curators of thehouse. We have focussed on a literacy experience forthe children drawing on the usage of the house itselfand the current curriculum for the group of childrenselected although the system is designed to be generic.

The use of sculptural hypermedia for orchestrationprovides a mechanism for encoding the complex inter-actions and dependencies and, with an appropriate con-text model, has been used to facilitate a rich experienceencompassing a variety of content and interaction mod-els. By using semantic web technology as the mediat-ing layer between the presented interfaces (authoringand user experiences) and the underlying infrastruc-ture, a unified approach can be taken that lends itselfto ready reuse of material and the growing of a cor-pus of information.

For the next phase of the project we are develop-ing tools to enable the curators and teachers to createtheir material more directly, and where possible in-situ.This will allow for the addition of richer material andthe creation of a wider range of activities to stretchthe infrastructure in different directions. We are plan-ning further trials in the near future with both the cu-rators and teachers as we begin to develop more ad-vanced authoring tools. These will support the easycreation of re-usable content and augmented field tripsmoving towards a generic infrastructure that supportsa wide range of activities.

Acknowledgments

The authors wish to acknowledge their partners: Chaw-

ton House Library, Whiteley Primary School and the Uni-

versity of Bristol. EPSRC IRC project “EQUATOR”

GR/N15986/01.

References

[1] M. Bernstein. Card shark and thespis: exotic tools for hy-pertext narrative. In Proceedings of Hypertext ’01, pages41–50. ACM Press, 2001.

[2] G.Gay,R.Reiger, andT.Bennington. Usingmobile com-puting to enhance field study. In Carrying the conversa-tion forward, pages 507–528. Erlbaum, 2002.

[3] W. Grant. Wireless coyote: A computer-supported fieldtrip. Comms ACM, 36(2):57–59, Feb. 1993.

[4] J. Halloran, E. Hornecker, G. Fitzpatrick, D. E. Millard,and M. J. Weal. The chawton house experience - aug-menting the grounds of a historic manor house. In Work-shop - Re-Thinking Technology in Museums: Towards aNew Understanding of People’s Experience in Museums,Limerick (Ireland), pages 54–65, June 2005.

[5] Y. Rogers, S. Price, D. S. Fraser, C. Randell, M. J.Weal, and G. Fitzpatrick. Ubi-learning Integrates indoorand outdoor experiences. Communications of the ACM,48(1):55–59, Jan. 2005.

[6] M.J.Weal,D.T.Michaelides,M.K.Thompson,andD.C.De Roure. Hypermedia in the ambient wood. New Re-view of Hypermedia and Multimedia, Special Issue on Hy-permedia beyond the Desktop, (9):137–156, 2003.