Emerging Business Strategy, IBS [email protected] / 416.513.5656 1 An Introduction to XML, IVR and...

1

Emerging Business Strategy, IBS [email protected] / 416.513.5656

An Introduction to XML, IVR and VoiceXML

30 January 2002

Ian GRAHAM

Emerging Business Strategy, Bank of Montreal

E: <[email protected]> or <[email protected]>

T: (416) 513.5656 / F: (416) 513.5590

Web: http://www.utoronto.ca/ian/talks/

2


Overview

1. What is XML– Gotta know that if VoiceXML is to make any sense – XML for universal data

• Based on the ways people wanted to use HTML and the Web

2. A bit about IVR

3. VoiceXML

4. Future stuff

3


A syntax for “encoding” text-based data (words, phrases, numbers, ...)

A text-based syntax. XML is written using printable characters (no explicit binary data)

Extensible. XML lets you define your own tags (essentially data types), within the constraints of the syntax rules

Universal format. The syntax rules ensure that all XML processing software MUST identically handle a given piece of XML data.

If you can read and process it, so can anybody else

1. What is XML?

4


What is XML: A Simple Example

<?xml version="1.0" encoding="iso-8859-1"?> <partorders xmlns=“http://myco.org/Spec/partorders”> <order ref=“x23-2112-2342” date=“25aug1999-12:34:23h”> <desc> Gold sprockel grommets, with matching hamster </desc> <part number=“23-23221-a12” /> <quantity units=“gross”> 12 </quantity> <deliveryDate date=“27aug1999-12:00h” /> </order> <order ref=“x23-2112-2342” date=“25aug1999-12:34:23h”> . . . Order something else . . . </order></partorders>

XML Declaration (“this is XML”) Binary encoding used in file

5


Example Revisited

<partorders xmlns=“http://myco.org/Spec/partorders” > <order ref=“x23-2112-2342” date=“25aug1999-12:34:23h”> <desc> Gold sprockel grommets, with matching hamster </desc> <part number=“23-23221-a12” /> <quantity units=“gross”> 12 </quantity> <deliveryDate date=“27aug1999-12:00h” /> </order> <order ref=“x23-2112-2342” date=“25aug1999-12:34:23h”> . . . Order something else . . . </order></partorders> Hierarchical, structured information

tags attribute of thisquantity element

element

6


XML Data Model - A Tree

text

partorders

order

order

desc

part

quantity

delivery-date

date=

ref=

date=

ref=

xmlns=

<partorders xmlns="...">

<order date="..."

ref="...">

<desc> ..text..

</desc>

<part />

<quantity />

<delivery-date />

</order>

<order ref=".." .../>

</partorders>

text

7


XML: Why it's this way

Simple (like HTML)– But not quite so simple– Strict syntax rules, to eliminate syntax errors– syntax defines structure (hierarchically), and names structural parts

(element names) -- it is self-describing data

Extensible (unlike HTML; vocabulary is not fixed)– Can create your own language of tags/elements – Strict syntax ensures that custom tags can be reliably processed

Designed for a distributed environment (like HTML)– Can have data all over the place: can retrieve and use it reliably

Can mix different data types together (unlike HTML)– Can mix one set of tags with another set: resulting data can still be

reliably processed

8


Mixing language dialects together

<?xml version="1.0" encoding="iso-8859-1"?><html xmlns="http://www.w3.org/1999/xhtml1" xmlns:mt=“http://www.w3.org/1998/mathml” ><head> <title> Title of XHTML Document </title></head><body><div class="myDiv"> <h1> Heading of Page </h1> <mt:mathml> <mt:sup> ...… MathML markup … </mt:mathml> <p> more html stuff goes here </p></div> </body></html>

mt: prefix indicates'type' mathml (a differentlanguage)

Default ‘type’is xhtml

9


Defining Specific Language Dialects

Two ways of doing so:– XML Document Type Declaration (DTD) -- Part of core XML spec.– XML Schema -- New XML specification (2001), stronger constraints

on XML documents.

Adding dialect specifications implies two classes of XML data:– Well-formed An XML document that is syntactically correct– Valid An XML document that is both well-formed and

consistent with a specific DTD (or Schema)

Most current dialects defined using DTDs.

Schemas often used for type validation.

10


Classes of XML Dialects

Many XML 'dialects,' optimised for different roles --

– presentation stuff people read or look at– metadata for describing things; for use by other software– utilities generic XML tools -- XSLT, Schemas,...– s/w development as a software development tool– distributed apps. distributed applications, data delivery, Web

services

A) Presentational Language (for people/applications)

– MathML -- for mathematics SMIL -- for multimedia (RealPlayer)

– XHTML -- new HTML WML -- Wireless WAP-phones– SVG -- for graphics XUL -- user interface (Netscape

6)– VoiceXML -- voice interfaces

B) Metadata– RDF -- Resource Description Framework

11



C) Utilities– XSLT -- transform one XML document into another. E.g:

XSLT style sheet in

XMLparser

XSLT processor

text

partorders

order

order

desc

part

quantity

delivery-date

document “objects” fordata and style sheet

XMLparser

XML data in

partorders

xza

order

foo bee

data out (XML)

12



C) Utilities (cont) - XML Schemas – Define validation rules for a specific type of XML document

• Can define hierarchical nesting rules for elements, allowed attributes and attribute status (like DTDs)

• Can define stronger typing constraints on element content, attributes (e.g., Integers, Integer ranges, reals, real ranges, strings, tokens, etc.)

13



D) S/w development– ANT -- XML-based build configuration file (Java applications)– beanML -- language specifying composition and state of a Java-bean

based application. Processing a BML script results in a running application configured as described in the script.

– use XML as a tool for managing data inside an application

– Extract data from databases and store in intermediate XML "object"– Useful for application development (sometimes!)

DB 1 DB 2 DB 3

XML

App XML "server"

14


Classes of use for XML

E.1) For machine-machine communication– Financial information exchange

• FpML, FinXML, OFX/IFX, FixML, GOLD, XBRL, SwiftML

– Directory services metadata• dirXML, DSML (Directory Services Markup Language), ….

– Other business transactions• FRML (first retail markup language), …. ebXML (generic business)

– News, data syndication (exchanging data between machines)• XMLnews, ICE, NewsML, RSS, WDDX

E.2) Manage the connection between machines – Control of machine-machine applications

• XML-RPC, SOAP

– Brokering of Web "Services"• Biztalk, UDDI, ebXML

15


XML Messaging + Processing

FactorySupplier

Supplier

Supplier

Place order(XML/edi) using SOAP

Response(XML/edi) using SOAP

SOAP interface

16


XML Software

XML parser -- Reads in XML data, checks for syntactic (and possibly DTD/Schema) constraints, and makes data available to an application. There are three 'generic' parser APIs

– SAX Simple API to XML (event-based)– DOM Document Object Model (object/tree based)– JDOM Java Document Object Model (object/tree based)

If XML is not well formed, or is invalid (inconsistent with DTD) then the parser MUST throw a fatal exception, and stop.

Lots of XML parsers and interface software available (Unix, Windows, Psion, Palm, OS/390 | Z/OS, etc.)

SAX-based parsers are fast

DOM slower, more memory intensive

17


XML Parser Processing Model

XML dataparser

parserinterface

XML-basedapplication

18


XML Processing: SAX

A) SAX: Simple API for XML– http://www.megginson.com/SAX/index.html– An event-based interface– Parser reports events whenever it sees a tag/attribute/text node/other– Programmer attaches “event handlers” to handle the event

Advantages– Simple to use– Very fast (not doing very much before you get the tags and data)– Low memory use (doesn’t read an XML document entirely into

memory)

Disadvantages– Not doing very much for you -- you have to do everything yourself– Not useful if you have to dynamically modify the document once it’s in

memory (since you’ll have to do all the work to put it in memory yourself!)

19


XML Processing: DOM

B) DOM: Document Object Model– http://www.w3.org/DOM/– An object-oriented interface– Parser generates an in-memory tree corresponding to the document– DOM interface defines methods for accessing and modifying the tree

Advantages– Very useful for dynamic modification of, access to the tree– Useful for querying (I.e. looking for data) that depends on the tree

structure [element.childNode("2").getAttributeValue("boobie")]– Same interface for many programming languages (C++, Java, ...)

Disadvantages– Can be slow (needs to produce the tree), and can take up lots of

memory– DOM programming interface is a bit awkward, not terribly object

oriented

20


DOM Parser Processing Model

XML dataparser

parserinterface

application

text

partorders

order

order

desc

part

quantity

delivery-date

Document “object”

DOM

21


XML Processing: JDOM

C) JDOM: Java Document Object Model– http://www.jdom.org– A Java-specific object-oriented interface– Parser generates an in-memory tree corresponding to the document– JDOM interface has methods for accessing and modifying the tree

Advantages– Very useful for dynamic modification of the tree– Useful for querying (I.e. looking for data) that depends on the tree

structure– Much nicer Object Oriented programming interface than DOM

Disadvantages– Can be slow (make that tree...), and can take up lots of memory– New, and not entirely cooked (but close) – Only works with Java, and not (yet) part of Core Java standard

22


XML Processing: XSLT

D) XSLT eXtensible Stylesheet Language -- Transformations– http://www.w3.org/TR/xslt– An XML language for processing XML– Does tree transformations -- takes XML and an XSLT style sheet as

input, and produces a new XML document with a different structure

Advantages– Very useful for tree transformations -- much easier than DOM or SAX

for this purpose– Can be used to query a document (XSLT pulls out the part you want)

Disadvantages– Can be slow for large documents or style sheets– Can be difficult to debug style sheets (poor error detection)

23


Part 2. IVR overview

24


What is IVR

IVR -- Interactive Voice Response All the horrible stuff you have to deal with

– Answering/managing voice mail– Telephone banking– Call handling at customer support lines

– And sometimes, even, surfing the Web

25


Roots of IVR

Interface restrictions– Input: Voice and 15-key keypad (DTMF)– Output: Audio information and audio prompts

User restrictions– All session needs to be in user’s memory (nothing to ‘look at’)– Limited information ‘framing’– Many ‘dialogs’ means hard to keep/reference/know context– Same UI must handle experienced and novice (first-time) users

• Need to account for learning during a session

Application context– Often used within telephony applications, and so must be able to do

call control (re-routing + queuing of calls)– Integrate with call centres– Auto-forwarding of information (e.g., to a call centre)

26


Provisioning of IVR

Many IVR technology providers

Same basic approaches and ideas, but– Different software,– Different APIs, development tools, architectures

No common framework for ‘scripting’ the dialog sessions that control the user interface

– But always the same basic ideas and approaches (next slide)

Known problems with historic approach – Portability (can’t move application elsewhere)– Integrability (can’t easily extend to new non-pure IVR apps; not

easily integrable with Web)

27


IVR - Common Issues

Dual mode interfaces– First time through, user gets detailed instructions or prompts– Later on, the instructions are sparser, more succinct

Barging in– Sometimes the user should be able to barge in on a dialog, and

interrupt it – And sometimes not!

Hierarchy of control layers– ‘Escape’ mechanisms to jump from current dialog to “root” of session

Need to manage session, store session state, and pass data to other applications

– So an IVR session has some kind of logic/rules, variables for storing data, etc.

28


3. VoiceXML

29


What is VoiceXML

“... designed for creating audio dialogs that feature synthesized speech, digitized audio, recognition of spoken and DTMF key input, recording of spoken input, telephony, and mixed-initiative conversations.”*

“Its major goal is to bring the advantages of web-based development and content delivery to interactive voice response applications”*

DTMF component designed for integration with existing telephone systems

XML abstraction layer on top of existing IVR systems

In principle, not restricted to phones

– But in practice -- what else would you use??

* VoiceXML 1.0 Specification -- http://www.w3.org/TR/voicexml

30


History of VoiceXML

VoxML

Motorola (1998)

PML

Bell/Lucent + others (1998)

SpeechML

IBM (1998)

TalkML

H.P. (1998)

VoiceXML1.0

VoiceXML2.0

VoiceXML forum (2000)

W3C (2002 -in draft)

? (defunct)

1999

31


Scope for VoiceXML 1.0

Output of synthesized speech (text-to-speech) Playback of audio files Creation of audio files (recording and file creation) Recognition of spoken input (voice recognition) Recognition of DTMF (touch-tone dialing) input Telephony features such as call transfer and disconnect Logical mechanisms for driving and controlling a dialog session

Support and implementation of features is often contingent on third-party hardware and software.

32


Model for VoiceXML Applications

Base-level management of interaction between a person and a backend system of some sort. Here is the operational model:

Implementation(phone hardware)

VoiceXMLserver

Documents / dataresources

User

Incoming

Outgoing Response

Request

• CGI’s• servlets• application code• resource files

• UI control• hardware

33


Practical VoiceXML Implementations

Commercial systems usually bundle the first two parts together:

VoiceXML “solution” Application Resources

User

Incoming

Outgoing Response

Request

Computer (Sun, Intel) running UNIX/linux/NT/2000 equippedwith phone cards (Dialogic, etc.)

34


VoiceXML System: Component Technologies

VoiceXMLserver

Telecom boardsPBX

CT Integration

Speech synthesis (TTS)

Speech recognition (STT)

Speech grammars

Voice Biometrics

Software utilities

VoiceXML servers serve as integratorsof various hardware and software

Callcentre

35


Technical Integration issues

Standardized APIs for driving hardware, accessing software:

Telephony APIs (telecom boards + drivers)– TAPI (Microsoft)– JTAPI (Java) – TSAPI (Novell)

Speech APIs (to various pieces of speech software)– SAPI (Microsoft) [ ASAPI (AT&T) extensions ]– Java Speech API (Java)– Lots of proprietary stuff for specific speech components

Computer-Telephony CCT APIs– No common API standard (usually has a RPC component)

36


VoiceXML Integrators

Many, many companies– Some sell integrated solutions kits– Some are voice ASPs (provide leased hosting solutions)

I am sure you don’t care about this.

However, several provide developer sites where you can– Develop and test VoiceXML applications– But, you need your own Web site for hosting the VoiceXML (and

other) files

Two companies that provide developer sites and phone lines:– Tellme.com http://studio.tellme.com– VoiceGenie (Toronto) http://developer.voicegenie.com

37


VoiceXML 1.0 Components:

Covers various UI components, such as:– Forms (lists user can select from) and associated variables: includes

menus – Links (that link to other VoiceXML documents) controlled by logic– Playing audio prompts– Produce text-to-speech prompts/dialogs (if supported)

Covers control and logic mechanisms, including– Event handlers (including catching and throwing exceptions)– Variables– Resource fetching (e.g., access an audio file to play as a prompt)– Control grammars, including

• DTMF (touch-tone driven - built into VoiceXML)• Voice driven (using external standards for grammars)

38


Location of Resource Files

Files are not typically on the VoiceXML server, but are located remotely:

VoiceXML server Documents / resources

User Webserver

• CGI’s, servlets• application code /

resource files (e.g.session.vxml, gram.jsgf)

• data resources files (e.g.:voice.au, sound.aiff)

• VoiceXMLprocessor

HTTP

• Referenced via URLs

39


Management of Resource Files

Since VoiceXML files are referenced by URI, and often located remotely, you need optimization at the VoiceXML server:

– Resource caching (store local copies)– Model for cache coherency (how often to check for changes)– Pre-fetch hinting (when can this be done)– Timeout handling (if you can’t retrieve the resource)

When VoiceXML references a URI, the language provides attributes that define the cache model to use:

– caching=“safe” or “fast” (“safe” always goes for new

version) – fetchtimeout=“secs” (how long to wait before

throwing error)– fetchhint=“prefetch” or “safe”

(safe only gets stuff when needed)

Dropped in VoiceXML 2.0

40


VoiceXML Example

<?xml version="1.0"?><vxml version="1.0" application="app-root.vxml"> <form id="say_goodbye"> <field name="answer" type="boolean"> <prompt>Shall we say <value expr="application.bye"/>? </prompt> <filled> <if cond="answer"> <exit/> </if> <clear namelist="answer"/> </filled> </field> </form></vxml>

<?xml version="1.0"?><vxml version="1.0"> <var name="bye" expr="'Ciao'"/> <link next="operator_xfer.vxml"> <grammar> operator </grammar> </link></vxml>

app-root.vxml

main.vxml

From: VoiceXML 1.0 Specification -- http://www.w3.org/TR/voicexml

41


Processing Model: The Example

main.vxml

root

app-root.vxml

main.vxml main.vxml

root

app-root.vxml

operator_xfer.vxml

bootstrapfrom main.vxml

load root

Run dialog in ‘main’using root grammar

transfer transfercontext to new dialog

load new resource

42


VoiceXML Examples

<vxml version="1.0" application="app-root.vxml"> <form id="say_goodbye"> <field name="answer" type="boolean"> <prompt>Shall we say <value expr="application.bye"/> ? </prompt> <filled> <if cond="answer"> <exit/> </if> <clear namelist="answer"/> </filled> </field> </form>

<var name="bye" expr="'Ciao'"/> <link next="operator_xfer.vxml"> <grammar> operator </grammar> </link>

app-root.vxml

main.vxml

S: Shall we say Ciao?” U: “Boobie” S: I did not understand that U: “Bleeble” S: I did not understand that U: “Operator” S: [Transfer to

operator_xfer.vxml]

Defaultsystemprompt

43


Example: Including recorded audio

Can do so whenever you have a prompt. Some examples:<prompt> Welcome to Birdland.<audio src=“http://www.xxx.org/birdland.wav”>We have every record by Charlie Parker …</prompt>

– Synthesized voice with audio file in the middle

<prompt> <audio src=“http://audio.files.com/yeah.au” ></prompt>

– No text - just an audio file.

<prompt> <audio src=“…”>Text alternative to audio</audio></prompt>

– Synthesized text used if audio file unavailable

44


Recording Audio

Can be done, but is stored locally in a variable.

Can’t permanently store data on the VoiceXML server -- it must be moved over to the “application/resources” server.

– Typically use HTTP methods to do this.

<record> element lets you specify this, and, also indicate– Prompt and error response– variable name for storing it– ‘dead air’ time to indicate end of recording– DTMF indicator for end of message– How long the message can be– Audio format for the data

45


Recording Audio

<?xml version="1.0"?> <vxml version="1.0"> <form> <record name="greeting" beep="true"

maxtime="10s" finalsilence="4000ms" dtmfterm="true" type="audio/wav">

<prompt> At the tone, please say your greeting. </prompt> <noinput> I heard nothing ... </noinput> </record>

<field name="confirm" type="boolean"> <prompt> Your greeting is <value expr="greeting"/>. </prompt> <prompt> To keep it, say yes. To discard it, say no. </prompt> <filled> <if cond="confirm"> <submit next="save_greeting.pl" method="post" namelist="greeting"/> </if> <clear/> </filled> </field></form> </vxml>

46


VoiceXML 2.0

Is basically a ‘cleanup’ of VoiceXML 1.0– Better consistency with XML architecture, including definition of a

VoiceXML namespace URI ( http://www.w3.org/2001/vxml )– Support for error logging, – some changes to syntax, and some deprecated elements and

attributes– mandated support for a W3C sponsored speech grammar format – clarification of meaning, cleanup of caching control models

Basically the same language, with a few minor tweaks and improvements

47


JSML - JSpeech Markup Language

Developed by Sun and SpeechWorks, as a markup language for text-to-speech dialogs.

Based on the Java Speech API Markup Languagehttp://java.sun.com/products/java-media/speech/

Text annotation to provide hints to speech synthesizers– Aimed at making TTS speech more natural, more understandable

Feature set:– hints to word pronunciation– hints to phrasing, emphasis, pitch and speaking rate– “marker” elements -- notifications from the speech synthesizer to

applications when marker is reached.

48


JSML - JSpeech Grammar Format

Developed by Sun and SpeechWorks, as a syntax for expressing speech grammars

Based on the Java Speech Grammar API Grammar Formathttp://java.sun.com/products/java-media/speech/

49


Future of the “Voice” web and VoiceXML

VoiceXML1.0

VoiceXML2.0

VoiceXML forum (2000)

W3C (2002 -in draft)

Speech synthesis (SSML) [WD]

Speech reco. grammar [WD]

NLP [prelim]

Speech semantics [prelim]

Pronunciation lexicon [early]

Call control [early]

Voice Browser interoperation [early]

W3C

SALT

Microsoft-led (2002)

Speech ApplicationLanguage Tags

JSML

Sun/SpeechWorks (1999)

JSGML,gram. langs

VoiceXML 3? [????]

50


Multiple Future XML Standards

Future standards will be targeted at the various ‘different’ issues associated with Voice interfaces (of which IVR is a subset)

– Speech synthesis hints to appropriate spoken nature of text. Overlaps with audio Cascading Style Sheets, to some degree

– call controlcontrol and management of call transfers, holding, Abstraction on top of PBX controllers.

– Voice browser interoperationPassing of data and ‘application context’ between different voice (or other) applications

– VoiceXML 3?A specific voice-driven session management language, which will embrace these other languages as components.

51


Microsoft’s SALT

Speech Application Language Tags– Microsoft, Cisco, Intel, Comverse, SpeechWorks, Philips

A “lightweight” set of tags designed to be used with HTML and XHTML to enable lightweight telephony applications driven from regular Web documents.

Targeted at supporting multimodal access

No version to look at yet ….

Draft not due until later this year.

52


Short Glossary

Call control – managing connectivity and routing

of phone calls CTI

– Computer Telephony Integration DTMF

– Dual Tone Multi Frequency IVR

– Interactive Voice Response JSGF

– Java API speech grammar format -- A proposed standard for representing speech grammars

JSML– Java Speech Markup Language

NLU– Natural Language Understanding

PBX– Private Branch Exchange

PSTN – Public Switched Telephone

Network SLU

– Spoken Language Understanding

STT– Speech to Text (simple

recognition) SV

– Speaker Verification (biometric identification)

TTS– Text to Speech (synthesis)

53


An Introduction to XML, IVR and VoiceXML

30 January 2002

Ian GRAHAM

Emerging Business Strategy, Bank of Montreal

E: <[email protected]> or <[email protected]>

T: (416) 513.5656 / F: (416) 513.5590

Web: http://www.utoronto.ca/ian/talks/

Date post:	20-Dec-2015
Category:	Documents
View:	217 times
Download:	0 times