Day 2

Prior to today

• Participated in the one day class on Cognitive Ergonomics, read all material of that class

• studied the reader for today

• read the paper by vdVeer & Mariani

After today

• management plan and time schedule

• interviews with potential users

• video records for interaction analysis

• review relevant information from web and other document sources

• version 1 of Task Model 1

• collect ideas for task model 2

prepare presentations for next time:

• management plan

• task analysis methods and techniques used: why, how, issues

• task model 1

5. Task analysis of complex systems

Overview

5.1. Task analysis in the process of designing complex systems

5.2. Describing current work situations:

5.2.1. Analytic methods

5.2.2. Understanding implicit expert knowledge

5.2.3. Explicit group knowledge (video illustration- Boeing 747-400 accident)

5.2.4. Understanding implicit group knowledge (video ill. Examination system)

5.3. GTA

5.4. Envisioning the future system

5.5. Evaluating task model 2

5.1. Task analysis in the process of designing complex systems

Design of complex systems for users:

the virtual system and the mental model

Design for the mental model:

3 types of design activities• analysis• specification• evaluation

“Task analysis”, in modern approaches, indicates analysis of the whole task domain: business, work, users, stakeholders, situation

“Specification” focuses on details of technology

work organization/practice

Client

users’knowledge/behavior/needs

Technology

Task Model 1

Task Model 2

Scenario

Simulation

Prototype

Functionality

Dialog

Representation

Implementation

usabilitymeasuring

ethnography

psychologicalknowledgeacquisition/hermeneutics

problemanalysis/specification

specification/negotiation

constraints/opportunitiesfeedback

specification

early evaluation

earlyevaluation

UVM maintainingconsistency

Documents/artifacts

validity analysis

As soon as the system is implemented

Task analysis

– describing the current situation

• task model 1

– envisioning the future

• task model 2

Task analysis in literature

Several meanings• collecting knowledge about, and describing, current task domain (task

model 1)• specifying new task situation, envisioning use of novel technology (task

model 2)• specifying details of technology (existing or new) (the user virtual

machine)

approaches always consider type of task knowledge, and relate this to model intended for design process

Literature and approaches often consider only one or two, and often mix up meaning of what is being modeled

Important concepts and distinctions

task and goal:

• each task (activity) has a goal (state to reach)

• a goal may be reached through several different tasks

primary and secondary tasks

• a primary task is generic for the task domain irrespective of technology and situation

• a secondary task is related to the use of tools and technology

basic and unit tasks

• a users’ unit task is an atomic primary task

• a basic task is the simplest task available for delegation to technology

actions

• an action has no goal, its meaning is derived from the task it is part of

5.2. Describing current work situations: task model 1Brigitte Jordan (1996): sources of task knowledge

Explicit expert knowledge

• facts of task domain

• heuristics

• skills

Explicit group knowledge

• process models, rules

• instructions

• historic documents

• anecdotes

Implicit expert knowledge

• “expertise”

• intuition

• common sense

Implicit group knowledge

• “culture”

• COP (community of practise)

• work practice

5.2.1. Analytic methods:

conceptual framework• set of basic concepts (task, goal, condition, action)• relations between concepts

formal representations, based on conceptual framework• task hierarchies (HTA, based on Taylor)• task-action grammars (TAG, ETAG), • goals and operation rules (GOMS, CCT)

planned process• steps in collecting knowledge, modeling, specification

focus on the individual human and his knowledge• cognitive psychology

TKS (task knowledge structure) P. Johnson & H. Johnson (QMW college, London)

conceptual framework

• tasks, plans (task structures), strategies

• procedures (object - action relations)

• roles (sets of tasks assigned to certain individuals)

Psychological basis is conceptual knowledge in long term memory

• psychological techniques: interview, observation, card sorting

• document analysis

TKS (continued)

method:

• from individual TKSs to generic task model GTM (task model 1)

• from GTM to Specific task model STM (task model 2)

• from STM to spec. interface model SIM (functionality at conceptual level)

problems:

• experts often do not know details of others’ roles

• often there is no overview of all roles in an organisation

• objects (things) are only analysed at secondary level

ATOM - Paul Walsh (STC Technology, Harlow, UK)

Start of analysis is identification of “principal objects”

• active objects (agents)

• passive objects (things)

• environment (includes things, agents, technology

objects structured in class hierarchies and relations between types

actions establish a causal link between object states

actions are identified secondary, identified by type of verb:

• process verbs (d.g. “dies”)

• state verb (“waits”)

• action verb (“transports to”)

MAD - D.Scapin, S. Sebillotte (INRIA, France)

Conceptual framework

• tasks, goals of people - states to be reached

• functions of systems (delegated subtasks)

• task hierarchy, subtasks, elementary tasks

• constructors (PAR, ALT, PAR, LOOP, COND, OPT)

• initial state, final state (describe change due to effect of task)

• prior condition, post condition

Formalism

• task / goal tree (HTA)

MAD (continued)

The Graesser method:

• describe as complete and detailed as possible how you …. (name the global label of the task to be described)

“... because ...”

“... Therefore ...”

• keep track of hierarchical structure (need a second analyst)

• filling the gaps

“How ?”

“Why ?”

MAD (continued)

Generic levels of task knowledge

• abstract general task formulation

• specific context sub-tasks (expert task language)

• domain independent procedures: (“store a copy”, “send a message”)

• elementary actions (“ask for signature”, “put date on record”)

5.2.2. Understanding implicit expert knowledge

Implicit knowledge, behavior, skills, “expertise”

• individual knowledge that can not be verbalized

Technique:

• observation, registration

Hermeneutic interpretation

• “what would be my knowledge and mental process, if I would show the behavior that this expert is showing”

Reliability problem

• needs training and multiple interpreters

5.2.3. Explicit group knowledge

Explicit sources:

• official

– documented rules

– documented business plans

– documented laws, regulations

– memos

• in-official

– artifacts (incl. documents)

– stories, anecdotes (location dependent)

Validity:

example: rules, regulations, laws

– intention

– date of installation

– friction with novel developments

– "officially still valid"

– overruled

Example: analysing history documents(Video “Kegworth fatal error” - training video HRA)

An example: the Kegworth accident

Failure of a safety critical system: a Boeing 737-400 crash at East Midlands

Hardware failure: left motor flutter

• up-scaling problem

• no operational test

Video document shows original material, and actual actors, but cockpit scenes are played by actors, based on cockpit voice recorder (another historic document)

Analysing history documents (continued)

A combination of problems merged after the original hardware failure:

• functionality of the control system

• representation of the system towards the users

• training and information

• organisation problems

• human failure

watch the video and try to identify problems in this task domain

/ONDERWIJS/PAO-NFatalErrorCDR.mov

5.2.4. Understanding implicit group knowledge Ethnographic methods

No a priory conceptual framework• bootstrapping - hanging around and identifying relevant concepts• informal representations - diary and journalist type records• activity theory, anthropology, sociology

Goal: understanding complex settings• multiple activities, multiple actors, multiple agendas• in natural groups

Focus on the situation, on work practice• knowledge and meaning are developed in “communities of practice”• knowledge is “ability to participate meaningfully”• people co-construct knowledge and skills, partly implicit or “tacit”• situated tasks, situated actions, situated understanding

Ethnographic methods (continued)

Ethnography is fieldwork

ethnographer is participant observer

• part of the situation, and co-constructor of events

• novice, if possible apprentice, or “legitimate peripheral participant”

• accepted to be “nosy”

the two viewpoints of ethnographic analysis

• the view of the analyst: being surprised, recording novel phenomena

• the view of the aboriginal: understanding in situation and the relevance of resources and representations

• diary needed from day one

Ethnographic methods (continued)

Ethnographic methods

• hanging around - identifying basic situational phenomena, getting accepted

• informal questions, triggered by activities and situation, based on growing understanding

• observation of routine and expert behaviour

• interaction analysis

Interaction analysis

Focus on interactions (humans, artifacts, situation)• identify "hot spots”

– documents, stories, participant observation• collect registrations - multiple instances• content log

– participants, situation, artifacts, technology, time– verbal and non verbal (body) language, territory, space, messages

Video review sessions:• comments of original participants• explanations, thinking aloud• ask to identify important issues• propose hypotheses

Interaction analysis (continued)

Interpretation:

• patterns in interaction, resources

• verifiable observations: records of different instances

• generalization: theory

• supplementary interviews, archive, observations

Group sessions:

• interdisciplinary

• replays to generate hypotheses

• max 5 minutes for discussion

• identify analytic categories

• confront hypotheses with other instances of hot spot theory

Generalizing ethnographic knowledge

Surprise index

• events and phenomena should be as predictable for the ethnographer as they are for the original actors

Systematic coverage:

• space, territory, off location

– make sure all relevant locations are considered (coffee lounge, corridors)

• temporal (shifts, seasonal changes, off hours)

– related events during commuting, during lunch hour

• actors (supporting people, spouses, unofficial consultants)

Generalizing ethnographic knowledge (continued)

Orientation - systematic viewpoints: choose one or more

• person oriented record– daily activities, interactions

• object oriented record– movements, change of ownership

• setting oriented record– at location, during shifts, movements in the space

• task / process oriented record– focus on process

Generalizing ethnographic knowledge (continued)

• event structure

– preparation, start, re-arrangements, intro of new objects

• temporal patterns

– day, season, technology driven rhythm

– “acted” continuity (receptionist, pilot, underground train “driver”)

• interaction structure

– verbal, instrumental, body language

– interrupted conversation (continuation triggered by availability of info or resources or time)

– just in time finalization triggered by interruption

Generalizing ethnographic knowledge (continued)

• spatial organization

– meaning of positions (head of the table)

– fixed places, free choice, stable or re-arrangements

• artifacts and documents

– location and movements

– construction and revision (documents)

– ownership (allowed activities related to objects)

– symbolic meaning (dress code, stethoscope)

– non physical objects (PIN, password, anecdotes, jokes)

ownership, trajectory, availability and inspectability

Example ethnographic record(video “Math Examination” VU 1994)

Task analysis for the design of an automated examination system (preparing questionnaires, registering for exam, taking the exam, scoring results, administering marks)

An examination: setting oriented record

• roles: professor, supervisors, students

• objects: forms, boxes, writing equipment, calculating equipment

• situation: time structure, entrance, scenes (hot spots)

watch video and identify and analyse unexpected events

5.3. GroupWare task analysis (GTA) Conceptual framework for collecting and modeling task knowledge

http://www.cs.vu.nl/~gerrit/gta

agents (with description of relevant attributes)

• actors (individual humans, institutes and groups, machines, systems)

• roles and their representation (incl. symbols)

• organization (structure of actors, roles, and allocation rules)

work

• task / goal structure / actions (acts, communication acts)

• decomposition and temporal structure

• protocols and (expert) strategies - partly situated

situation

• objects in object structure (type relations and semantic relations)

• environment

• history

GTA representations

conceptual entities

• template

structures

• tree

• flow

media registrations

• video

• sound track

• picture

Task World Ontology (Martijn van Welie)

Task Agent

Role

Event

Object

name(string)goal(string)start_condition(string)stop_condition(string)initial_state(string)final_state(string)duration(integer)frequency(string)type(enum)user_actions(string)system_operations(string)

name(string)skills(string) attitude(string)miscellaneous(string)

name(string)goal(string)name(string)

attribute(Name,Value)*action(Name)*

name(string)description(string)

Contains

Responsible

Performed_by

PlaysTriggers

Subtask

Uses

Triggers

Used_by

Subrole

Is

Performed_by

EUTERPE GTA representations