Post on 05-Jun-2018
transcript
Virtuelle Realität
1
Part 16:
Social and Collaborative Virtual Environments
Virtuelle RealitätWintersemester 2006/07
Prof. Bernhard Jung
Prof. B. Jung Virtuelle Realität, WS 2006/07
IBM Ups Investment In Virtual World
IBM is set to invest $10 million over the next twelve months to increase its presence in the market for technologies that enable so-called virtual worlds such as Second Life, a 3-D Web environment in which people from various walks doff their everyday identities to assume online alter egos, a company spokesman said Friday. As part of the effort, IBM will expand its use of Second Life, which now has more than one million users, for virtual meetings with employees and business partners. Next week, company CEO Sam Palmisano will go "in world" to hold a virtual town hall on IBM's Second Life island. The company recently held a similar gathering with the press and analysts, and has hosted a virtual party for IBM alumni inside the online world. It's also building a 3-D replica of China's famed Palace Museum inside Second Life, which will be open to cyber tourists. IBM's virtual reality ambitions go far beyond Second Life, however, a spokesman says. …
[ http://www.techweb.com/wire/ebiz/193700609 November 10, 2006]
Virtuelle Realität
2
Prof. B. Jung Virtuelle Realität, WS 2006/07
Social Virtual Environments
Virtual RealityReal-time computer graphics, audio, haptics3D user interface"Presence": sense of "being there"
Social Virtual Environmentssense of "being with others"collaborative work, meetings, games, entertainment, …
Prof. B. Jung Virtuelle Realität, WS 2006/07
Overview
Conversation with othersGroupwareNew communication technologiesCollaborative Virtual Environments
"Ubiquitous Virtual Reality"
Further InformationJ. Preece, Y. Rogers & H. Sharp. Interaction Design – Beyound Human-Computer
Interaction. John Wiley & Sons. 2002. Alan Dix, Janet E. Finlay, Gregory D. Abowd, Russell Beale. Human-Computer
Interaction, 3rd Edition. Prentice Hall, 2004.T. Pfeiffer, M. Weber & B. Jung: Ubiquitous Virtual Reality - Accessing Shared Virtual
Environments through Videoconferencing Technology. Proceedings Theory and Practice of Computer Graphics Conference. Eurographics, 2005, pp. 209-216. Available at: www.informatik.tu-freiberg.de/~mweber
Virtuelle Realität
3
Prof. B. Jung Virtuelle Realität, WS 2006/07
Conversation with others
Various mechanisms and ‘rules’ we follow to hold a conversationmutual greetings
A: Hi thereB: Hi!C: HiA: All right?C: Good, how’s it going?A: Fine, how are you?C: OKB: So-so. How’s life treating you?
Prof. B. Jung Virtuelle Realität, WS 2006/07
Conversational rules
turn-taking to coordinate conversationA: Shall we meet at 8?B: Um, can we meet a bit later?
A: Shall we meet at 8?B: Wow, look at him?A: Yes what a funny hairdo!B: Um, can we meet a bit later?
Back channeling to signal to continue and followingUh-uh, umm, ahh
Virtuelle Realität
4
Prof. B. Jung Virtuelle Realität, WS 2006/07
More conversational rules
farewell ritualsBye then, see you, see you later….
implicit and explicit cuese.g. looking at watch, fidgeting with coat and bags explicitly saying “Oh dear, must go, look at the time, I’m late…”
Prof. B. Jung Virtuelle Realität, WS 2006/07
Breakdowns in conversation
When someone says something that is misunderstoodSpeaker will repeat with emphasis:A: “this one?”B: “no, I meant that one!”
(also a case of deixis: this and that refer to the visible situation)Also use tokens:Eh? Quoi? Huh? What?
Virtuelle Realität
5
Prof. B. Jung Virtuelle Realität, WS 2006/07
What happens in technology-mediated conversations?
Do same conversational rules apply?Are there more breakdowns?How do people repair them?
Phone?Email?Instant messaging?SMS texting?Video conferencing?
Prof. B. Jung Virtuelle Realität, WS 2006/07
Design implications
How to support conversations when people are ‘at a distance’ from each other
Many applications have been developedEmail, videoconferencing, videophones, instant messaging, chatrooms, collaborative virtual environments, media spaces
How effective are they?
Do they mimic or extend existing ways of conversing?
Virtuelle Realität
6
Prof. B. Jung Virtuelle Realität, WS 2006/07
What is groupware?
Software specifically designedto support group workingwith cooperative requirements in mind
NOT just tools for communicationGroupware can be classified by
when and where the participants are workingthe function it performs for cooperative work
Prof. B. Jung Virtuelle Realität, WS 2006/07
The Time/Space Matrix
Classify groupware by: when the participants are working,
at the same time or not where the participants are working,
at the same place or not
Common names for axes:time:
synchronous/asynchronousplace:
co-located/remote
differentplace
sameplace
sametime
differentplace
Virtuelle Realität
7
Prof. B. Jung Virtuelle Realität, WS 2006/07
Time/Space Matrix (ctd)
differentplace
sameplace
sametime
differenttime
face-to-faceconversation telephone
post-it note letter
Prof. B. Jung Virtuelle Realität, WS 2006/07
Classification by Function
Cooperative work involves:Participants who are workingArtefacts upon which they work(e.g. documents)
participants
artefacts of work
control andfeedback
P P
A
communication
understanding
direct
Virtuelle Realität
8
Prof. B. Jung Virtuelle Realität, WS 2006/07
What interactions does a tool support?
participants
artefacts of work
control andfeedback
P P
A
communication
understanding
direct
meeting and decisionsupport systems
– common understanding
computer-mediatedcommunication
– direct communicationbetween participants
shared applicationsand artefacts
– control and feedbackwith shared work objects
Prof. B. Jung Virtuelle Realität, WS 2006/07
Time/space matrix revisited
co-located remote
synchronous
asynchronous
co-authoring systems,shared calendars
argumentationtools
email andelectronic
conferences
shared work surfaces and editorsshared PCs and windows
video conferences,video-wall, etc.
Collaborative VEsmeeting rooms
CAVEs
Second Life
Hologram Messaging(still SciFi)
Virtuelle Realität
9
Prof. B. Jung Virtuelle Realität, WS 2006/07
Synchronous computer-mediated communication
Conversations are supported in real-time through voice and/or typingExamples include video conferencing, chatrooms, collaborative virtual environments
BenefitsCan keep more informed of what is going on Video conferencing allows everyone to see each other providing some support for non-verbal communicationChatrooms can provide a forum for shy people to talk more
Problems:Video lacks bandwidth so judders and lots of shadowsDifficult to establish eye contact with images of othersPeople can behave badly when behind the mask of an avatar
Prof. B. Jung Virtuelle Realität, WS 2006/07
Video conferencing: wall, kiosk, booth
Virtuelle Realität
10
Prof. B. Jung Virtuelle Realität, WS 2006/07
Will video be a success using G3 mobile phones?
The VP-210" VisualPhone: a mobile video phone developed by the japanese company Kyocera Corporation Source: http://www.kyocera.co.jp/news/1999/9905/0003-e.asp
• Will the judder, sudden jerks and shadowsdisappear?
• Will it be possible to establish eye contactand read lips on such a small image?
• Will people find it socially acceptable totalk to an image of someone in the palm oftheir hands?
Prof. B. Jung Virtuelle Realität, WS 2006/07
Asynchronous computer-mediated communication
Communication takes place remotely at different timesEmail, bulletin boards, newsgroups, online forums, blogs, …Recipients of email:
direct in To: fieldcopies in Cc: field
delivery identical – difference is social purposeBenefits include:
Read any place any timeFlexible as to how to deal with itPowerful, can send to many peopleCan make saying things easier
Problems include:FLAMING!!!SpammingMessage overloadFalse expectations as to when people will reply
Virtuelle Realität
11
Prof. B. Jung Virtuelle Realität, WS 2006/07
txt is gr8
instant messaging1996 – ICQ small companynow millionsmore like conversation
SMSpeople adapt their communication styles to medium
y is it we al lv shrt msgsoriginally a feature of internal management protocolshort messages (160 chars) and text with numbersno-one predicted mass adoption!!now phones with cameras for MMS
Hi, u there
want to meet later
yeh, had a good night last night?uhu
Prof. B. Jung Virtuelle Realität, WS 2006/07
Computer-mediated communication combined with other activity: Meeting and decision support systems
In design, management, and research, we want to:generate ideasdevelop ideasrecord ideas
Primary emphasis: common understanding
Three types of systemsargumentation tools
asynchronous co-locatedrecording the arguments for design decisions
meeting roomssynchronous co-locatedelectronic support for face-to-face meetings
shared drawing surfacessynchronous remoteshared drawing board at a distance
ArgTool
Virtuelle Realität
12
Prof. B. Jung Virtuelle Realität, WS 2006/07
Typical meeting room
sharedscreen
Prof. B. Jung Virtuelle Realität, WS 2006/07
Meeting rooms
synchronous co-located
electronic support for face-to-face meetingsindividual terminals (often recessed)large shared screen (electronic whiteboard)special softwareU or C shaped seating around screen
Various modes:brainstorming, private use, WYSIWIS
WYSIWIS – ‘what you see is what I see’all screens show same imageany participant can write/draw to screen
Virtuelle Realität
13
Prof. B. Jung Virtuelle Realität, WS 2006/07
Meeting capture
use ordinarywhiteboarddetector andspecial pensLCD projectionon whiteboardlow-cost alternativeto dedicated meeting room
Prof. B. Jung Virtuelle Realität, WS 2006/07
New communication technologies
Move beyond trying to support face-to-face communication Provide novel ways of interacting and talking
Examples include:SMS texting via mobile phonesOnline chatting in chatrooms Media spacesCollaborative virtual environments
Virtuelle Realität
14
Prof. B. Jung Virtuelle Realität, WS 2006/07
Hypermirror(Morikawa and Maesako, 1998)
allows people to feel as if they are in the same virtual place even though in physically different spaces
(woman in white sweater is in a different room to the other three)
People in different places are superimposed
on the same screento make them appear as if
in same space
Prof. B. Jung Virtuelle Realität, WS 2006/07
Creating personal space in Hypermirror
2) Two in this room are invadingthe ‘virtual’ personal spaceof the other person by appearing to bephysically on top of them
3) Two in the room move apart to allow person in other space more ‘virtual’personal space
Virtuelle Realität
15
Prof. B. Jung Virtuelle Realität, WS 2006/07
Everyone happy
Prof. B. Jung Virtuelle Realität, WS 2006/07
Clearboard (Ishii et al, 1993)
ClearBoard - transparent board that shows other person’s facial expression on your board as you draw
Virtuelle Realität
16
Prof. B. Jung Virtuelle Realität, WS 2006/07
Collaborative Virtual Environments (CVEs)
meet others in a virtual worldparticipants represented – embodiment artefacts too …
computer (e.g. spreadsheet) and ‘real’ (virtually) objectstext?
consistent orientation or easy to read
MUDs (Multi-user domains)2D/3D places to meet on the webusers represented as avatars
Prof. B. Jung Virtuelle Realität, WS 2006/07
Collaborative Virtual Environments
The rooftop garden in BowieWorld, a Collaborative Virtual environment (CVE), supported by Worlds.com. Users take part by “dressing up” as an avatar. There are 100s of avatars to choose from, including penguins and real persons. Once an avatar has entered a world they can explore it and chat to other avatars.
Source: www.worlds.com
Virtuelle Realität
17
Prof. B. Jung Virtuelle Realität, WS 2006/07
Collaborative Virtual Environments
Second Life (SL) is a privately owned, partly subscription-based 3-D virtual world, made publicly available in 2003 by San Francisco-based Linden Lab,[2]and founded by former RealNetworks CTO Philip Rosedale. Users can visit this virtual world almost as if it were a real place. They explore, meet new people, participate in individual and group activities, and, if they decide to visit often, they learn new skills and mature socially (in the sense of learning the culture of the virtual world). Though sometimes referred to as a game, Second Life does not have points, scores, winners and losers, levels, and end-strategy, or most of the other characteristics of a game. Source: http://en.wikipedia.org/wiki/Second_Life
Prof. B. Jung Virtuelle Realität, WS 2006/07
Videogames: Virtual Thievery Criminals break into accounts, steal artifacts and sell them for thousands of dollars.
Dec. 11, 2006 issue - Keep a close eye on your magic wand, or somebody will steal it. World of Warcraft and other Massively Multiplayer Online Role-Playing Games have recently become the target of criminals who seek in-game currency, or gold, because of its real-world value. Cyberthieves break into players' accounts, steal the artifacts and characters acquired during the course of the game, and sell them—sometimes for thousands of dollars. Whereas gold farmers in China have sought to acquire gold legitimately by playing WOW for hours on end, theft is a relatively new phenomenon, says Mikko Hyppönen, chief research officer at F-secure, a cybersecurity firm in Helsinki. "We really didn't see this a year ago."…One reason criminals have gone for game crime is because it's less risky than breaking into online bank accounts. Chances are your local police won't even be able to keep a straight face when you report that someone stole your magical potions.Thieves also benefit from lax virtual-property laws in the United States, where most of these games are made. Designers are reluctant to push for legal recognition of virtual property for fear of being held liable for theft. "Game designers certainly don't want to facilitate hacking, but they often are concerned about their own potential liability for a loophole that creates a hack," says John Fairfield, associate professor of law at Indiana University. "So therefore they don't share the incentives to publicize and enforce as strongly as the owner of the virtual property would." The game companies argue that they own the virtual property, not the players. "Because players don't own it, players can't fight back when it's stolen," says Fairfield. "And that makes players easy victims." Until the matter of ownership gets cleared up, this new crime wave will be tough to stop.
http://www.msnbc.msn.com/id/15994153/site/newsweek/ Dec. 11, 2006 issue
Virtuelle Realität
18
Prof. B. Jung Virtuelle Realität, WS 2006/07
Collaborative Virtual Environments
Prof. B. Jung Virtuelle Realität, WS 2006/07
Collaborative Virtual Environments
Virtuelle Werkstatt – Collaborative Virtual Prototyping in VRUniversität Bielefeld, Universität zu Lübeck
Virtuelle Realität
19
Prof. B. Jung Virtuelle Realität, WS 2006/07
Collaborative Virtual Environments
PASION - Psychologically Augmented Interaction Over NetworksUniversität Bielefeld http://www.techfak.uni-bielefeld.de/ags/wbski/PASION/
"Interactive Social Displays im virtuellenRaum erlauben die Interaktion mittels Bild, Ton und erweitertenMöglichkeitenzwischen Individuenund Gruppen."
Ubiquitous Virtual Reality
Accessing Shared Virtual Environments through Videoconferencing Technology
Thies Pfeiffer,SFB 360, University of Bielefeld, Germany
Matthias Weber, and Bernhard JungTechnical University of Freiberg, Germany
Talk presented atEurographics Conference on Theory and Practice of Computer Graphics, 2005
Virtuelle Realität
20
Prof. B. Jung Virtuelle Realität, WS 2006/07
What if...
... one could communicate to distant places?telegraphy, phone, chat
... one could communicate on the road?mobile phones, sms
... one could see each other in remote communication?
Prof. B. Jung Virtuelle Realität, WS 2006/07
Videoconferencing Solutions
DesktopWebCam & MicrophoneNetMeeting, iChat, GnomeMeeting
Stand-AlonePolycom VSX 3000D-Link DVC-2000TelePortec Teleporter(3D, Volumetric Display) VSX 3000
DVC-2000Teleporter
Virtuelle Realität
21
Prof. B. Jung Virtuelle Realität, WS 2006/07
What if...
... one could communicate to distant places?telegraphy, phone, chat
... one could communicate on the road?mobile phones, sms
... one could really see each other?videoconferencing
... the remote place is not real?
Prof. B. Jung Virtuelle Realität, WS 2006/07
Videoconferencing in VR
2D-Videoimage:AliceStreet
3D-Avatars:blue-cColiseum
…
(see www.kolabora.com/news/2004/02/20/threedimensional_virtual_conferencing_rooms_become.htm)
(see Gross et al., blue-c: ASpatially Immersive Display and 3DVideo Portal for Telepresence,Siggraph 2003)
(see Baker et al., Computation and Performance Issues InColiseum, An Immersive Videoconferencing System,ACM Multimedia 2003)
Virtuelle Realität
22
Prof. B. Jung Virtuelle Realität, WS 2006/07
What if...
... one could communicate to distant places?telegraphy, phone, chat
... one could communicate on the road?mobile phones, sms
... one could really see each other?videoconferencing
... the other place is not real?shared virtual environments
... the communication partner is not real?chatter bots, telephony systems
Prof. B. Jung Virtuelle Realität, WS 2006/07
Chatting Avatars
(Permanent Exhibition at the Heinz-Nixdorf Museum, developedby the AI group at University of Bielefeld, picture taken fromhttp://www.techfak.uni-bielefeld.de/ags/wbski/labortag2002.html)
Virtuelle Realität
23
Prof. B. Jung Virtuelle Realität, WS 2006/07
What if...
everything is possible?
Distant mobile communicationAudio and videoNatural communicationStandard clientsSeamless transition betweenhuman-human and human-machine interaction
When one could interact with an artificial system using
Prof. B. Jung Virtuelle Realität, WS 2006/07
Motivation
Using standard software and protocolsUsing different platformsSupporting different input modalities using video, audio and data
→ Using videoconferencing technology
Goal: viewing and controlling Virtual Environments (VEs) ubiquitously
Virtuelle Realität
24
Prof. B. Jung Virtuelle Realität, WS 2006/07
History of Videoconferencing
1924 - Two-Way Videophone demonstration by Herbert Eugene Ives
(b&w, 1929 color)1936 - First official Videophone connection
(Berlin-Leipzig, Deutsche Reichspost) 1956 - Picturephone® by AT&T1971 - First trans-atlantic videophone (Ericsson)1992 - INRIA Videoconferencing System
(IP-based software)1996 - ITU-T standard H.323 v11996 - Microsoft NetMeeting v2.0
(with video) 2000 - First MPEG-4 streaming video cell phone (Samsung)2001 - First trans-atlantic tele surgery2004 - ITU-T recommends H.264 video codec
Prof. B. Jung Virtuelle Realität, WS 2006/07
Videoconferencing
Connecting people with each otherChannels for multi-modal communication
Audio: speech, soundsVideo: gestures, facial expressionsData: text-chat
BidirectionalPoint-to-point or multipointDifferent protocols:
H.323, SIP, proprietary (e.g. Skype)
Virtuelle Realität
25
Prof. B. Jung Virtuelle Realität, WS 2006/07
3D Conferencing
Connecting people within VEsAll conferencing participants use special purpose VR software to share the VEMixed reality
Prof. B. Jung Virtuelle Realität, WS 2006/07
Videoconferencing in AR/VR
Studierstube(Vienna University of Technology, Austria):
video transferred via Videoconferencing (H.323)special protocol for other data, like trackingspecial client needed that computes trackingno standard software
(see Istvan Barakonvi, W. Frieb, Dieter Schmalstieg, Augmented Reality Videoconferencing for CollaborativeWork, Proc. of the 2nd Hungarian Conference on Computer Graphics and Geomtery, Budapest, Hungary, May 2003)
Virtuelle Realität
26
Prof. B. Jung Virtuelle Realität, WS 2006/07
Concept
Thin ClientStandard videoconferencing software
Thick ServerRuns the VEProvides Videoconferencing Gateway
Sending content to participantsUtilizing natural communication interface
Accepts or initiates calls
Prof. B. Jung Virtuelle Realität, WS 2006/07
Concept (2)
Virtuelle Realität
27
Prof. B. Jung Virtuelle Realität, WS 2006/07
Example Application Scenarios
Information DesksTechnical Support
Educational SystemsUnified Messaging ServicesInteractive EntertainmentCooperative Work...
Prof. B. Jung Virtuelle Realität, WS 2006/07
Virtuelle Realität
28
Prof. B. Jung Virtuelle Realität, WS 2006/07
Implementation
Research prototype of a videoconferencing interfaceServer is an extension of an existing application for virtualconstruction via natural speechTested client platforms include Windows XP / NetMeeting, Linux / GnomeMeeting and Pocket PC / PocketBone
Prof. B. Jung Virtuelle Realität, WS 2006/07
Server
Virtual construction application (in-house)VR framework Avango (OpenSource)
SGI OpenGL PerformerOpenH323 (OpenSource)ESMERALDAspeech recognition software (in-house)
Virtuelle Realität
29
Prof. B. Jung Virtuelle Realität, WS 2006/07
Scenegraph (1)
Prof. B. Jung Virtuelle Realität, WS 2006/07
Scenegraph (2)
Virtuelle Realität
30
Prof. B. Jung Virtuelle Realität, WS 2006/07
Scenegraph (3)
Prof. B. Jung Virtuelle Realität, WS 2006/07
Scenegraph (4)
Virtuelle Realität
31
Prof. B. Jung Virtuelle Realität, WS 2006/07
Scenegraph (5)
Prof. B. Jung Virtuelle Realität, WS 2006/07
Types of Communication
Human – humanAudio: speech, soundsVideo: mimics, facial expressions, gesturesText: chat
Machine – humanAudio: Speech-synthesis, soundsVideo: images, movies, real-time graphicsText: status messages
Human – machineAudio: speech-recognitionText: typed natural language, commandsVideo: computer-vision, face-detection & recognition, gesture recognition
Virtuelle Realität
32
Prof. B. Jung Virtuelle Realität, WS 2006/07
Open Questions
Robust Speech RecognitionEvaluation: speech recognisers <> codecs
Visual InteractionGesturesFace Recognition / Tracking
Usability studiesDifferent devicesDifferent combinations of modalities
Prof. B. Jung Virtuelle Realität, WS 2006/07
Conclusion
Videoconferencing interface to VEsThin client solutionScales from low to high-end devices
(cell phone, handheld, PC, standalone)Supports broad range of applicationsUses natural interface
→ Make your VE ubiquitously accessible!