INFO310 0 Advanced Topics in Model-Based Information ... · T e hes e ar c bas i i t npu - o ut put...

INFO310 0 Advanced Topics in Model-Based Information Systems

Kandidat 108

Oppgaver Oppgavetype Vurdering Status

Introduction Dokument Automatisk poengsum Levert

Plagiarism and Declaration Dokument Automatisk poengsum Levert

1 Essay Filopplasting Manuell poengsum Levert

INFO310 0 Advanced Topics in Model-Based Information Systems

Emnekode INFO310Vurderingsform INFO310Starttidspunkt: 03.11.2016 14:00Sluttidspunkt: 09.11.2016 14:00Sensurfrist Ikke satt

PDF opprettet 16.11.2016 16:36Opprettet av Andreas Lothe OpdahlAntall sider 19Oppgaver inkludert NeiSkriv ut automatisk rettede Nei

1

Seksjon 1

1 OPPGAVE

Essay

Upload your file here. Maximum one file.

BESVARELSE

Filopplasting

Filnavn 9066477_cand-3238752_9157556

Filtype pdf

Filstørrelse 1088.801 KB

Opplastingstid 08.11.2016 15:03:31

Neste side

Besvarelsevedlagt

Kandidat 108

INFO310 0 Advanced Topics in Model-Based Information Systems Page 2 av 19

INFO310 – Advanced Topics in Model-Based Information Systems

Utilizing the data from biofeedback-capable gaming equipment Candidate: 108

Words: 5950 | Pages: 16

08.11.2016

Kandidat 108


1

Contents Introduction ............................................................................................................................................. 2

Affective gaming and biofeedback mechanisms ..................................................................................... 3

Real-time usages...................................................................................................................................... 4

Games (affective games) ..................................................................................................................... 4

E-sports ................................................................................................................................................ 5

Health .................................................................................................................................................. 6

Posterior usages ....................................................................................................................................... 6

Games .................................................................................................................................................. 7

E-sports ................................................................................................................................................ 7

Health .................................................................................................................................................. 7

Big data ................................................................................................................................................... 7

Personal data........................................................................................................................................ 8

Vitals data ............................................................................................................................................ 8

Game data ............................................................................................................................................ 8

Environmental data .............................................................................................................................. 9

Technological data .............................................................................................................................. 9

Ontologies ............................................................................................................................................... 9

Medical .............................................................................................................................................. 10

SS4RWWR .................................................................................................................................... 10

OBO Foundry’s ontologies............................................................................................................ 10

VSO ............................................................................................................................................... 10

MEDDRA ...................................................................................................................................... 11

Personal ............................................................................................................................................. 11

Game ................................................................................................................................................. 11

Framework ............................................................................................................................................ 12

Privacy ................................................................................................................................................... 13

Conclusion ............................................................................................................................................. 13

Literature ............................................................................................................................................... 14

Resources .............................................................................................................................................. 16

Illustrations ........................................................................................................................................ 16

Ontologies ......................................................................................................................................... 16

Social media contribution ...................................................................................................................... 16

Kandidat 108


2

Introduction It is over a half decade since what we know as the internet of things (IoT) was born, by the estimations

of Cisco (Evans, 2011). For some people the abbreviation IoT will make them think of their car, with

all the sensors that have been implemented over the years. For others, it can be smart TVs, smartphones,

coffee-machines, refrigerators, wearables or smart homes. A field that haven’t been affected too much

by the IoT yet, is the world of video games. E-sports, gaming communities and the number of people

playing and watching games are growing each day. It has grown such that several large sports clubs

around the world has started to pick up E-sports teams, as illustrated by Mike Kent’s article of

professional sports teams with e-sport teams (2016).

The last few years have given the gamers some technologies to play with, VR headsets are being released

one after the other, Oculus Rift, HTC Vive and the PlayStation VR are some examples. AR-glasses is

also worth mentioning, namely Google Glass and Microsoft HoloLens, although neither have been

released for the public masses yet. A new product that recently made its entrance to the world, will

hopefully pave the way for some exiting techniques and technology that the world of gaming and the

health sciences might benefit from, and drive VR-gaming to a whole new level.

The product is the NAOS QG (NQG), a gaming

mouse produced by the Swedish company Mionix.

Mionix is not new to the area of producing gaming

mice, but there is something special about the NQG,

where the QG stands for Quantified Gaming

(Mionix, 2015). What is special about it is that it has

additional sensors built into it, as seen in illustration

1, in addition to the normal optical laser sensor and

standard buttons. The NQG has a sensor for detecting

heart-beat and pulse and a sensor that notices

electrical properties in the skin, with a GSR

(Galvanic Skin Response) sensor. GSR captures

conductive changes in the skin, which changes based

on sweat (Peuscher, 2012).

Keyboards, mice and controllers are devices with

sensors and buttons. The “actuators” of the computer

makes use of the information given from these

devices, and translates it to navigating or performing

an action within the operating system, or sending the

information along to another application. These are

basic input-output (I/O) operations, that have existed

for many years. Extend these operations to utilize

input of other types of information and we can get

much more immersive experiences in gaming, and

contribute to e-health and science in the medical

fields. The data devices like the NQG produce, and

could in the future produce, what can we do with

them? And how do we utilize them in the best

manner possible?

Illustration 1. The Mionix NAOS QG

Kandidat 108


3

Affective gaming and biofeedback mechanisms To fully comprehend some of the usages of the data produced by devices like NQG, it is essential to

understand the concepts of affective gaming and biofeedback mechanisms.

Allanson et al defines affective gaming as “where the player’s current emotional state is used to

manipulate gameplay” (2005). In addition to the player’s conscious decisions and choices being utilized

to make changes in the game, we also utilize the unconscious emotional states of the player. The way

we access these unconscious emotional states is through what is called biofeedback mechanisms, which

means that we are monitoring the player’s vitals and physiological measurements. An important thing

Allanson et al mentions is that by using biofeedback mechanisms simply as another method of input as

a controller, it is merely a biofeedback-game. An affective game needs to utilize the biofeedback to

manipulate the game itself based on the emotional states of the user, just as if the game truly understood

the users emotional state and reactions.

Torres explains biofeedback in his master thesis as: “Biofeedback is the ability of self-regulating a

person’s biological or physiological functions by gaining greater awareness of them with the user of

instruments that provide information on those same systems” (2013). He then goes on explaining that

biofeedback can be divided into two subgroups: indirect and direct biofeedback: “Direct biofeedback

consists on conscious physiological function such as constraining a muscle, and indirect biofeedback

corresponds to an unconscious body action such as heart rate or respiration.” The indirect biofeedback

is what seems most suitable for affective gaming to utilize, as this is where the unconscious changes are,

which means that this is where the emotional states are possible to analyze.

Champion and Dekker made some interesting research on the topic of biofeedback mechanisms almost

ten years ago, which identified potential, problems and issues with the technology (2007). They used

sensors clipped to the user’s fingertips, and fed the biometric data into the game to dynamically change

visuals of the game, effects like screen shake and to generate the spawn of the NPC’s (Non-playable

character) which in this case was zombies. Additionally, they changed the speed of the playable

character based on heartbeat, and the volume of sound based on the user’s skin response, and more.

Kalyn et al did further work based on amongst other Champion and Dekker, and published a paper on

direct and indirect physiological control in games (2011). They basically made parts of the body replace

controller-input that changed things in the game, using both direct controllers like muscles to perform

actions, and indirect like heartbeat. The results they produced clearly showed them that indirect

controllers are not viable as a controller of actions in the game, which wasn’t surprising. The indirect

measurements should rather be used behind the scenes, changing scenery and other things that the player

isn’t in control of. The direct controllers have potential usages, for instance as being used for controlling

the player’s character.

In both projects the sensors had to be attached to the user, one could argue that most people wouldn’t

deal with the hassle of strapping on a bunch of sensors to access this “next level” of gameplay, and this

is where the NQG and similar devices make a big difference. These devices are also why it is time for

the themes of biofeedback mechanisms and affective gaming to re-emerge and make its entrance to the

world of video games. Major players in the game industries have been loking into these fields, for

instance Valve has been investigating how biofeedback mechanisms impacts cooperative player

experiences on major titles like Portal 2 and Left 4 Dead 2 (Perron and Schröter, 2009). There are likely

several reasons why affective gaming hasn’t been introduced to video games on a full scale yet, one of

which clearly is solved to some degree by the NQG. Mionix has also made it easy with a free and open

API, that developers could utilize (Mionix, 2015). The other reasons revolve around implementations,

mechanics, logic and how to best utilize the data. We will look closer at how to utilize the data, what to

use them for and how to use them in the most useful manner.

Kandidat 108


4

There are lots of ways to utilize and handle the data produced by these kinds of products, and there are

two main groups that will be in focus. Firstly, the real-time usages, what you can do with the data right

there and then. Secondly, the posterior usages, which is more of the analytic usages and what one can

do with the data generated from the real-time usages.

Real-time usages

Games (affective games) The game developers can take advantage of all these sensors to change the user-experience within the

game. It can be used to change the storyline, get the users emotional state to fire events, make the

NPC/AI characters adapt to the user and make the games optimize the gameplay further for the current

user. The game developers can build onto the standard I/O operations, with physical sensor giving input

and the game producing some output.

For the storylines, the developers could make use of the user’s reaction to a certain event, and to more

deeply understand how he/she responded and what emotional impact it had, with that information they

would be able to tailor the storyline in a direction the user might enjoy more. The game could build a

profile on that user, while constantly learning more information and adapting the game in a more

sophisticated way. There are also usages which doesn’t need to change the whole story of the game, for

instance one could also benefit from the sensors and fire events with perfect timing based on the actual

emotional state the user is in, which is more in the style of what for instance Champion and Dekker did.

In a horror game, the developers could program the game to fire off that jump-scare with perfect timing.

If the game has learned that the user’s average BPM (beat per minute) is around 70, it might fire off a

scary event when the user is approaching 120 BPM. This is just one example of how the game could

fire off certain events, and make the NPC/AI characters adapt to the user.

These aspects build on what the game-industry is already doing a lot of today, giving the users choices

to make in-game, and then change the story based on their choices. Which is a good thing, as it adapts

the game to the playstyle of the user. The user is although able to manipulate it to his liking, which is

hard to do with this sensor based game manipulation. With the game utilizing both conscious choices

and unconscious reactions/emotions, one could see even further personalized and unique gameplay.

Kandidat 108


5

E-sports This is what Mionix seemed to have in mind, when they released their campaign to make the mouse.

Illustration 2 shows us an image from their Kickstarter campaign, with a HUD displaying the vitals of

a team. It does not illustrate affective gaming, but regular biofeedback within a game.

Ingo Froböse is a professor at the German Sport University Cologne, and he has spent some time

studying e-sports, and has found that professional e-sports players are “real athletes” in the sense that

they experience a lot of the same physical strains that athletes in traditional sports do. For instance, they

sometimes achieve pulse rates close to what of a person running a marathon, and cortisol levels at the

level of a race-car driver (Schütz, 2016). In traditional sports on professional level the athletes often

have a support team, to help them improve their performance. Doctors, coaches, psychologists, personal

trainers, lifestyle coaches, there is a long list of professions that can help the athletes. The point is that

to improve most efficiently the athletes need help, and quantified gaming are a tool the “electronic

athletes’” can use for their improvement. The idea here is that on-screen while in the game, they have a

HUD (head-up display) that presents the player information about himself, and potentially his

teammates. So, they can see their own and the team-members’ physiological states. Which might result

in better team play, as one of the team members can assist one that is stressing out. On a personal level

the players can also identify scenarios that stresses them out, which could help them work on remaining

calmer in similar situations later. Stressing out can often lead to “hot-headed” actions and decisions,

which might produce negative results.

Hand in hand is the casting of e-sports events, they are able to benefit from these sensors as well, as they

are able to analyze deeper aspects of the game. They can see when a player is stressed and prone to make

mistakes, it adds another layer of information for them to serve to the audience. This has been done at

for instance Dreamhack (Swedish LAN party), where they showed the pulse rates, as can be seen in

illustration 3.

Illustration 2. How Mionix envisions a team being able to see each other’s vitals and stress levels.

Kandidat 108


6

Health There are of course also health applications to this technology, which both the medical field and the

players can benefit from. For instance, the games could incorporate some health functionality, in that

they could warn users of abnormalities from their averages, as well as monitor them over longer periods

of time. The horror games could even choose not to scare the player too much, if the person for instance

has a heart-condition or something similar. This also goes for operating systems; in that they could also

incorporate this technology. Additionally, the fact that players are able to monitor themselves could also

help them take breaks when they get too stressed out, and taking brakes from the computer in general

as most people in the medical fields recommend.

Posterior usages The data produced has the potential of being used for more than showing HUD’s and serving real-time

information, what can be extracted of the information posterior to the playing? For instance, for games

it would be necessary to implement some functions that would save the information, preferably both the

information given by the NQG, but equally important information about the game-states. As the data

about our vitals is itself not enough to utilize some of the concepts that will be described. But the vitals

data, alongside game-data, possibly combined with personal data and possibly others as well, will help

us puzzle together more pieces of the story.

If one were to use the data for game development optimization by learning how the users react to the

game, it would be key to identify the different aspects that are happening when a game is being played,

to fully utilize the potential.

Illustration 3. Heart rates visualized for the audience at Dreamhack in a StarCraft II match

Kandidat 108


7

Games The posterior-data could give detailed analytics to the developers, so that they could make improvements

to the existing game. They could learn about how to deal with problems and how to best solve them for

future games as well. The potential could be huge for the companies that release a lot of games, they

could create a huge game-knowledgebase generated by the games they make.

As mentioned earlier, one could make the games learn about the user and tailor the rest of the story to

player’s liking. Machine learning is also a relevant topic for the posterior data, this might be a tad far-

fetched for today’s technology, but image in a few years; Games that uses the posterior data could learn

not to only improve the local story for the local user, but also self-improve the game itself for all users.

E-sports For e-sports it is pretty much all about the analytics, how to improve the teams or the players

performance. Over time with the posterior data they could see improvements/decrease in performance,

analyze what types of teams they perform badly against, and why that is. As mentioned, all the answers

will not be given by simply utilizing the data about the player’s vitals, but it should be combined with

several other key elements.

Health E-health is no new term, dating back to the end of the last millennium. According to Eysenbach, it was

already in 1999 a “buzzword” for generally anything that could relate to computers and medicine (2001).

And some years later, a group found 51 different definitions of the word in different published works

(Enkin, et al., 2005).

In the future, people will most likely not have to go to their general practitioner to find some details

about their health, most likely our vital data will be sent directly to a machine or system for analysis.

The machine could then notify our doctor if there were something that needed attention, or it could send

an overview that our doctor would only need to brief through. In this scenario, the doctors would of

course have access to the data to see for themselves, at least that seems appropriate in the early stages

of the technology. As one might not make these systems analyze the data perfectly, in the early stages.

E-health-systems will be thought of as a system with parts where some device (A) monitors a person

(B) and sends the information along to system (C), and then a doctor/medical professional (D) can look

at it. (D) can then be the person’s general practitioner. There are already existing systems that does

similar things, for instance Qualcomm’s 2net system (Qualcomm, 2016).

Big data There are other sensors that in the future could be implemented, and taken advantage of. One could

utilize the camera to analyze facial expressions, maybe the microphone could be used for analyzing

respiratory rates by listening to the breath? A thermometer could help us better understand the user’s

surroundings, for instance. Without going into too much detail, as this is a somewhat fresh field and

identifying all the potential use cases and instruments that could benefit the various systems would likely

require teams of cross-discipline experts. I will mention some types of data that could be used, in the

upcoming paragraphs, merely to illustrate some of the potential. The game-knowledgebase mentioned

earlier, is an example of why these data could be used for big data generation, and then one could create

knowledgebases with the big data as a fundament. That there are enough amounts of data to generate so

called big data, is taken for granted, as this is hypothetical in the sense that “in some years, the sensors

will be implemented in all equipment and most applications will utilize the data.”

Kandidat 108


8

The question arises, what kind of data should we collect? It depends on the goal of what one wants to

use it for, game and health related big data is the two that might be closest to collecting the same data

as illustrated in illustration 4. The illustration shows some categories of data that is likely to be included

for learning about games, if one were to remove for instance the game data part, then it might look closer

to something to be used in a pure health perspective.

Personal data Data about the player, one might argue that it would be easiest for the game to link to some other

platforms to access what it needs. Age, height, weight, the general personal data, are examples of data

that could be collected, there is no doubt the data is powerful in discovering patterns and trends, and

generally being useful in the scenario of getting answers. It does although raise some potential privacy

concerns, so the player should be able to opt out, and the data collected shouldn’t be too specific.

Vitals data Data about our vitals from NQG-like devices could play a big role alongside wearables, for big data

generation for usages in the medical sciences. It involves heart rates, pulse, skin response and all these

things that we potentially can measure and build into devices. For the generation of big data solely

around health, it would likely be statistics/averages consisting of measurements at certain points,

mapped with other data, like timestamps, information revolving the measurement and such. Within a

game data system, it would likely be mapped with events and specific points within the game.

Game data Data about the game, its states, possibly generated player profiles based on choices the player has made

and the playstyle. Information about events mapped together with the vitals data, to more deeply

understand the effects the game has on the user.

Illustration 4. Some types of data that could be used for the purposes of big data generation

Kandidat 108


9

Environmental data Environmental data is another type of data that could be useful to look at for other purposes as well.

Environmental data is here thought of as things concerning the user’s environment, like the dates, times,

weather, temperatures and all these types of data. It could be useful to look at to get more context and

details that could explain why the user had that specific reaction, if for instance it were an unusual

reaction compared to the majority for an event within the game.

Technological data Data from sensors that is utilized, hardware, maybe also data about the software could prove itself useful

to collect. It could certainly be useful to see where the data is coming from, one could for instance

identify sensors that are not doing an optimal job. And come up with co-relations between the sensors

and the measurements they do, we will also see in the next section that sensors, like many electronic

devices, are prone to errors and might malfunction, which could be problematic.

The details around the types of data is somewhat vague, which is alright to illustrate potential data one

could collect, and how one should reason around why exactly that kind of data should be collected.

Specifying all the data that is important to collect is difficult without having any knowledge and

experience within neither the fields of game development or health. This is where one could bring groups

of experts from various fields together, with the fields depending on the usage. This group could then

work together to identify what data should be collected and how to best utilize it.

The big data itself can answer questions, but there is no reasoning with it. To be able to answer complex

questions, and do reasoning on the data, the big data collections should be mapped towards appropriate

ontologies.

Ontologies To most efficiently utilize the data collected, an appropriate ontology with which we could map the data

is essential. What types of ontologies are suiting for these kinds of data? The medical field has already

utilized semantic technologies for some time, and therefore have some useful parts to utilize. In the next

sections, we will see different existing ontologies that is applicable to some degree, now it doesn’t

necessarily mean that these are the best or only ontologies that are usable. It depends on what one wishes

to achieve, if it is within the medical fields, game development, psychology or social sciences. One can

tailor an ontology and reuse from these ontologies to represent what is most beneficial to one’s project.

Since ontologies have been used in the medical field for some time already, it is certainly the field with

the most available resources, and the wearable wave that came a few years ago haven’t decelerated the

growth. NQG-like devices have many similarities to wearables, although the users aren’t wearing the

mouse, it is still sensors built into a device which we “interact” with, most publicly available wearables

today take the shape of a watch, earbuds or similar. Wearables like the smart-watch is great for

monitoring everyday health, fitness, sleeping habits, and has a lot of real world appliances like

navigation, exercise motivators and so on. The appliances of devices like the NQG are to a degree

defined by the ecosystem in which they take a part. The NQG then takes the part of the PC and gaming

environment, naturally it is a part of the PC environment as it takes form of a computer mouse and it is

a part of the gaming environment, as it was intended to.

The point being, the data produced by wearables like the watch, who can be on you at most times, is for

instance highly valuable in the medical fields. The data produced by the NQG, can also be valuable for

Kandidat 108


10

the medical field as those who use it the most, might also be the persons in the more vulnerable positions

when it comes to health and fitness. One can argue that people who are concerned about health and

fitness are more likely to buy a smart-watch to monitor their vitals, than someone who isn’t as concerned

about health and spends more time in front of the computer monitor. In that way, NQG-like devices fills

an important gap where the smart-watches and fitness-bands might not reach a customer group.

In both cases, there are enormous amounts of data that could be produced, and semantic technologies

can help us make sense of it all. Drira et al wrote that the IDC (International Data Corporation) stated

in 2013 that less than 5% of information in the digital universe was analyzed, because one knew little

about the data. Drira wanted to illustrate that semantic enrichment can help us with that problem (2015).

Medical Fitting, usable and potential ontologies to use for the kind of data produced is majorly consisting of

ontologies for e-health, biology, physiological and medical fields. Remember that e-health was a

buzzword back in the start of this millennia, so there has been a lot of work in the medical fields, with

information technologies. Therefore, I will present some of the ontologies that seems appropriate to

potentially utilize for different projects.

SS4RWWR

In Ortuño and Rojas collection, Banos et al has an article revolving an ontology for sensor selection in

wearable activity recognition (2015). In the article, they argue about the error margins for these kinds

of sensors. And how most solutions and applications are created in such a manner that every tiny sensor

must function, which ideally they would, but that isn’t necessarily always the case. The sensors is often

small and to some extent fragile, and Banos et al presents an ontology which they named SS4RWWR

(Sensor Selection for Real-World Wearable Activity Recognition) to manage information about the

sensors, where are the sensors located on the body, what kind of data do they produce. They argue that

the ontology will help the system for instance provide an adequate type of data if one sensors were to

fail, from one of the other sensors that might capture something similar, or possibly even the same data.

Surely something of the like would be useful for the medical field, but also game developers could

borrow some of the features.

OBO Foundry’s ontologies

The OBO Foundry is a collective of ontology developers that produce several ontologies, with focus on

it being open and the ontologies sharing the same principles. A list of all their ontologies can be found

at obofoundry.org, as well as more information about their organization. The HDO (Human Disease

Ontology) and CDO (Cardiovascular Disease Ontology) are for instance two interesting ontologies,

which could be utilized to potentially warn the user about potential dangers or just contribute as a part

of an e-health-system like 2net. Another relevant ontology is the emotion ontology, which can describe

various emotions, moods and affective phenomena, as stated at their GitHub page (Hastings, 2015). The

ontology would be a great addition as to map the recorded vitals and related information to actual

emotional states.

In more general the OBCS (Ontology of Biological and Clinical Statistics) looks like a candidate to be

used for instance to generate Big Data for the Watson Project, or 2net-like systems. IBM has a project

where they allow hospitals to access Watson’s power, to diagnose patients (Otake, 2016). Surely the

collected data in these kinds of projects could be put to good use, with or without the OBCS in specific.

VSO

VSO (Vital Sign Ontology) covers blood pressure, body temperature, respiratory rates and pulse rates,

with the possibility to build further on it. It seems like a nice foundation for the type of data we can

collect now, and the data we could potentially collect in the future. Used with the emotion ontology, one

Kandidat 108


11

would perhaps be a few steps into a project that could help us understand the users’ reactions to the

game. Arbandi et al explains that VSO also follows the OBO Foundry’s guidelines, and interoperates

with OGMS (Ontology for General Medical Sciences) (2011).

MEDDRA

MEDDRA (Medical Dictionary for Regulatory Activities) is an ontology which revolves around

medical concepts, and contains such entities as heart rate, heartbeats increased and similar concepts. It

could prove to be useful for instance to make concepts a little easier to understand to non-medical

persons, as for instance the VSO are more specific about which artery the pulse was monitored from.

Personal Most ontologies revolving around personal information, seems to be more of the kinds that tracks

information for the user, to make various digital assistants. For instance the PIMO (Personal Information

Model Ontology), which has entities like things, tasks, events, locations and similar concepts. So

DBpedia’s “person” is what seems most appropriate to use with personal information, and one could

add additional information if there are concepts not covered. The privacy issues mentioned in the big

data part, is also existent here. One could co-relate the personal information with for instance medical

information and psychologic information. Some personal information should be stored, so the game

developers can ask questions on a deeper level concerning the users, and to do that one needs at least

some information about the users. For the medical fields, it could also be useful in seeing patterns

amongst users who share some attributes.

Game The GOP (Game ontology project) has a lot of usable information on describing, analyzing and studying

games (Whitehead, 2007), although it seems not much work has been done in the last few years. It

appears solid enough so that it is certainly possible to reuse, at the very least some of the parts and

concepts. It could also be utilized in the mechanics in affective games, as it contains inputs, game rules,

entity manipulations and things that could be adapted to the player. If one looks towards big data, one

would although need to define a sort of game event in addition, to map certain event or points in the

game towards the users emotional and biological states. For this one could utilize parts of the VGO

(Video Game Ontology) whose main goal is to “capture knowledge about events that happen in video

games and information about players” (Garijo et al, 2014). VGO capture some of what GOP doesn’t,

and VGO have defined different types of evens, which would be required to learn the most about the

gathered data. Both ontologies contain relevant information, that could be utilized.

Kandidat 108


12

Framework

Illustration 5 shows a diagram of a simple flowchart of what the data extraction could like in a

framework for a game-knowledgebase generation and how one could incorporate some of the

ontologies. The game would have to record its various events, and each time an event happened it would

have to send the data along. To make the most use out of the medical ontologies, Hodges et al showed

how medical ontologies could benefit from being able to share the information they model, and can

integrate that with wearable devices (2014). In this data extraction process, it is thought of as if the

ontologies could share that information.

The green circles are health related, and firstly we figure out what is being monitored, and where, then

the VSO can help us define it, and then MEDDRA can output a “something changed”, all the appropriate

data from each element is sent along. The GOP and VGO tells us what was going on in the game at the

time. The blue ones are about “the real world”, what devices are available, what is the user’s

surroundings/environment? Information about the person is the standard personals, but as this goes

through the event cycles in the game, one could build more detailed analysis of/profile on the user each

time. The emotion ontology would have to get the available information in order to analyze the users

emotional state, and then respond. From there one could generate a knowledgebase, and add onto it each

time, in order to later do reasoning on it. Or one could generate big data, and get statistics, visualizations

and so on. This is merely an example of what one of the parts of a framework for utilizing the data could

potentially resemble, if one wanted to know about how the user reacts to certain aspects within a game.

Illustration 5. Diagram of the simple data extraction flow description, what it might look like

Kandidat 108


13

The tricky part is to draw the lines of what data to collect, what data proves to be useful and what

doesn’t? For a full-scale project, you would likely need experts from a handful of different areas, in

order to clearly be able to define exactly what you want to utilize. A good starting point for further work

would although be to utilize the medical and game ontologies.

Privacy As in many other topics relating data collection, privacy is an essential issue, and it is also existent here.

Now there are certainly work-arounds that will make this less of a problem, for instance one could give

the users the options to opt out of giving personal data at all, or to allow the game to use it in real time,

but not save the data for usage in further analysis with big data. Asking the user for what data that the

user is willing to share, is what Android is doing, when the application asks for permission to read

information on the phone, similar concepts could be implemented in the games. It is also important to

remember to anonymize the data, very detailed information isn’t necessarily needed. For the e-health

systems like 2net, the doctor would benefit from knowing who the patient is, although it could be

implemented in such a way that the data is anonymized. Then, when the patient sees the doctor, the

doctor will get the appropriate data with for instance comments that he or some other doctor has made.

The summoning of the patient could be done electronically, so that the doctors schedule is automatically

generated for these types of situations.

Conclusion For now, we should build games as we always have, but adding the layers of affective gaming,

biofeedback mechanisms and data gathering on top of the game. In this way the game will function even

if the user does not have any of the required devices to utilize affective gaming. The technology and

techniques described will probably, need some time in to be realized. That could change over time, but

the first problem as of now is firstly that the NQG has just recently been publicly released. Secondly,

the equipment is more expensive due to the included sensors, so many might not see the usages yet and

will not buy it. Thirdly it might take some generations before all potential/appropriate sensors has been

incorporated into the mouse, with very accurate measurements and durable sensors. Additionally,

affective gaming might not become a natural part of games until VR and AR have a few generations of

products and see the use for incorporating biofeedback mechanisms, for instance.

Affective gaming with affective feedback would require some extra effort of the game developers, but

is certainly doable when it comes to implementations. The issue here might depend more on the available

technology and devices, and potential standards of the data transmitted. Implementing mechanics to

handle data from hundreds of different NQG-like devices, with different data inputs could prove to be

difficult.

Biofeedback gaming is a reality already, the NQG works at least for one person. Further work is to for

instance making it so that whole teams can watch each other’s vitals. And incorporate the tools needed

for them to analyze their games, for broadcasting/streaming many of the same or similar tools will help

them analyze the games they cast in much more detail.

Semantic technologies have the potential to improve normal and affective gaming, and biofeedback-

capable gaming equipment can help us on our way to these improvements. It can help game developers

understand the players better, and in general make better games. It could help psychologists better

understand our minds. It could improve our lives through e-health systems, and big data for medical

science. There are more usages for the data, that is guaranteed, the question remains of what these usages

are.

Kandidat 108


14

Literature

Allanson, J., Dix, A., Gilleade, K. (2005) Affective Videogames and Modes of Affective Gaming: Assist Me,

Challenge Me, Emote Me [Internet] Digital Games Research Association

Available from: http://www.digra.org/wp-content/uploads/digital-library/06278.55257.pdf

[Downloaded 25.10.16]

Arabandi, S., Brochhausen, M., Goldfain, A., Hogan, W., Smith, B. (2011) Vital Sign Ontology [Internet]

University of Buffalo

Available from: http://ontology.buffalo.edu/smith/articles/Vital_Sign_Ontology.pdf


Champion, E., Dekker, A. (2007) Please Biofeed the Zombies: Enhancing the Gameplay and Display of a

Horror Game Using Biofeedback [Internet] Digital Games Research Association

Available from: http://www.digra.org/wp-content/uploads/digital-library/07312.18055.pdf


Drira, K., Exposito, E., Mezghani, E., Silviera, M. (2015) A Semantic Big Data Platform for Integrating

Hetereogenous Wearable Data in Healthcare [Internet] Springer

DOI: 10.1007/s10916-015-0344-x


Enkin, M., Jadad, A., Oh, H., Rizo, C. (2005) What is eHealth(3): A Systematic Review of Published

Definition [Internet] Journal of medical internet research

DOI: 10.2196/jmir.7.1.e1


Evans, D. (2011) The Internet of Things [Internet] Cisco

Available from: http://www.cisco.com/c/dam/en_us/about/ac79/docs/innov/IoT_IBSG_0411FINAL.pdf


Eysenbach, G. (2001) What is e-health? [Internet] Journal of medical internet research

DOI: 10.2196/jmir.3.2.e20


Garijo, D., Parkkila, J., Poveda, M., Radulovic, F. (2014) The Video Game Ontology [Internet] Linkeddata.es

Available from: http://vocab.linkeddata.es/vgo/


Hastings, J. (2015) Emotion Ontology [Internet, GitHub page] Github.com

Available from: https://github.com/jannahastings/emotion-ontology


Hodges, J., Kritzler, Michahelles, F., Lueder, S., Wilde, E. (2014) Ontology alignment for wearable devices

and bioinformatics in professional health care [Internet] The Semantic Web Journal

Available from: http://www.semantic-web-journal.net/system/files/swj826.pdf


Kalyn, M., Lough, C., Mandryk, R., Nacke, L. (2011) Biofeedback Game Design: Using Direct and Indirect

Physiological Control to Enhance Game Interaction [Internet] University of Saskatchewan

Available from: http://hci.usask.ca/uploads/202-Nacke-Biofeedback_Game_Design.pdf


Kent, M. (2016) Full List of Professional Sports Teams in Esports [Internet] Dexerto.com

Available from: http://www.dexerto.com/news/2016/07/06/full-list-of-professional-sports-teams-in-

esports/ [Downloaded 21.10.16]

Kandidat 108


15

Mionix (2015) Quantified Gaming – know more, game better [Internet] Kickstarter.com

Available from: https://www.kickstarter.com/projects/206180815/quantified-gaming-know-more-game-

better [Downloaded 04.10.16]

Ortuño, F., Rojas, I. (2015) Bioinformatics and Biomedical engineering Switzerland: Springer

Collection of articles, article used:

Banos, O., Pomares, H., Rojas, I., Villalonga, C. Pages: 141 -152

An Ontolgoy for Dynamic Sensor Selection in Wearable Activity Recognition

Otake, T. (2016) IBM big data used for rapid diagnosis of rare leukemia case in Japan [Internet] Japan

Times

Available from: http://www.japantimes.co.jp/news/2016/08/11/national/science-health/ibm-big-data-

used-for-rapid-diagnosis-of-rare-leukemia-case-in-japan/#.WA9A-OiLQox


Perron, B., Schröter, F. (2009) Video Games and the Mind, Essays on Cognition, Affect and Emotion. North

Carolina: McFarland & Company, Inc., Publishers

Peuscher, J. (2012) Galvanic Skin Response (GSR) [Internet] Twente Medical Systems Internationa

Available from:

http://www.tmsi.com/products/accessories?task=callelement&format=raw&item_id=43&element=fe0c95f3-

af08-4719-bc51-36917715660d&method=download


Schütz, M. (2016) Science shows that eSports professionals are real athletes [Internet] Deutsche Welle

Available from: http://dw.com/p/1I4sT


Torres, V. (2013) Development of Biofeedback Mechanisms in a Procedural Environment Using Biometric

Sensors [Master Thesis]. University of Porto

Available from: https://arxiv.org/ftp/arxiv/papers/1310/1310.2102.pdf


Whitehead, J. (2007) An Ontology for Computer Games [Internet] University of California, Santa Cruz

Available from: https://classes.soe.ucsc.edu/cmps080k/Winter07/lectures/game-ontology-overview.pdf


Qualcomm (2016) 2net™ Platform [Internet] Qualcomm.com

Available from: http://www.qualcommlife.com/2net-platform


Kandidat 108


16

Resources

Illustrations Illustration 1: https://mionix.net/wp-content/uploads/2016/09/Naos-QG-Top-w-icons-2000px.png

Illustration 2: https://ksr-ugc.imgix.net/assets/002/992/678/d246a5617bec861b33101b137c2e425a_original.png

Illustration 3: http://quantifiedgaming.org/wp-content/uploads/2014/12/ujsdw.jpg

Illustration 4: Self-created

Illustration 5: Self-created

Ontologies Mentioned ontologies and where one can look at them:

MEDDRA: http://bioportal.bioontology.org/ontologies/MEDDRA

VSO: https://bioportal.bioontology.org/ontologies/VSO

EMOTION ONTOLOGY: https://github.com/jannahastings/emotion-ontology/tree/master/ontology

CVDO: https://raw.githubusercontent.com/OpenLHS/CVDO/master/cvdo.owl

HDO: http://www.disease-ontology.org/

GO: https://www.mindmeister.com/324669511/game-ontology-project

DBpedia person: http://dbpedia.org/ontology/Person

PIMO: http://www.semanticdesktop.org/ontologies/2007/11/01/pimo/

VGO:

https://github.com/dgarijo/VideoGameOntology/blob/master/GameOntologyv3.owl

http://vocab.linkeddata.es/vgo/

Social media contribution

There was no existing Wikipedia article on Affective gaming, so I made one:

https://en.wikipedia.org/wiki/Affective_gaming

Kandidat 108


Date post:	17-Oct-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times

INFO310 0 Advanced Topics in Model-Based Information ... · T e hes e ar c bas i i t npu - o ut put...

Documents