Software Development Processes, Reuse and Knowledge Sources in Spoken Dialog System 11761 Dialog...

Software Development Processes, Reuse and Knowledge Sources

in Spoken Dialog System

11761 Dialog SeminarRohit Kumar

Friday, Dec 1, 2006

• Some basic ideas / Ground rules– Practical Dialogue Hypothesis– Domain Independence Hypothesis

• Rapid Prototyping of Dialogue Systems– Dialogue system frameworks– Re-use

• SLDS development methodologies & Best practices– Iterative Method– (DISC) Dialogue Engineering Life Cycle– Model View Controller (MVC) approach– Rapid Dialogue Prototyping (RDP) Methodology– User involvement in development– Methodologies i have used : 2 cases

• Some “potential” rules of thumb ?

• Knowledge Sources view of Dialogue Systems– Overview– Template based – Tools and Methods required

• Some ideas

Also included: Full list of readings






• Some ideas

Also included: Full list of papers surveyed

Some basic ideas / Ground rules

• Spoken Dialog System development is costly and time consuming

• Two optimistic hypothesis (Allen et. al., 2000)– Practical Dialogue Hypothesis

“The conversational competence required for practical dialogues, while still complex is significantly simpler to achieve than general human conversational competence”

– Domain Independence Hypothesis“Within the genre of practical dialogue the bulk of complexity in the language interpretation and dialogue management is independent of the task being performed”

– Agreeable ?! (I say “Yes”)






• Some ideas


Rapid Prototyping of Dialogue Systems

• Dialogue System Frameworks– Can be considered a consequence or a partial proof of the domain

independence hypothesis

• Various Frameworks– TRIPS– Adriane / Ravenclaw / Olympus (CMU)– Communicator (SRI, CMU, U. Colorado, …)– Speech Builder (MIT)– CSLU Rapid Application Developer– Industrial Frameworks: Voice Platforms (IBM, Microsoft, Nuance…)– Tutorial Dialog: TuTalk– Many more…





(Denecke, 2002)

• Able to develop very simple dialog systems with 8 – 10 hours of effort using the Adriane framework

– Domain independent Interaction Patterns implemented• Question Undo, Correction, State

– Development comprises of 7 steps• Backend Application• Databases• Ontology• Dialogue Goals• Database Conversion Rules• Parsing Grammars• Generation Templates


Observations

Effort calculated as per the knowledge source. Rapid prototyping can be considered a method in which knowledge is added to a framework to customize it.

All design exhibit lack of coverage in Language resources. Direct consequence of not doing any data collection ahead of time.


• To reduce development costs: Re-Use– Can be considered to be the driving force being the Domain

Independence Hypothesis

(Hanna et. al. 2005)• While Domain Independence Hypothesis is acceptable, the

challenge is to develop such an domain independent system– Part of the challenge is difference in terms of complexity of applications

which in turn requires different degrees and forms of domain independence

• Object oriented methodology to support extensibility and re-use of (dialog) components– All discourse object contain capabilities

– On request, domain spotter polls and gets scores from each discourse object indicating object’s ability for successfully handle the request

– Supports maintainability and re-use

– Idea of hierarchies of increasing degrees of specialism


• Dynamic Linking of discourse objects– Keeps frames sizes small

– Facilitates delegation and delimitation of agent responsibility

– Open ended structure which can grow and adapt depending on objectives set by user on run time



• SLDS development methodologies & Best practices– (DISC) Dialogue Engineering Life Cycle– Iterative Method– Model View Controller (MVC) approach– Rapid Dialogue Prototyping (RDP) Methodology– User involvement in development– Methodologies i have used : 2 cases



• Some ideas


• SLDS Development Methodologies

• Dialogue Engineering Life Cycle– Based on DISC project on best practices for development and

evaluation of SLDSs

– Includes: Life Cycle model, Grid (set of issues and options), Set of Evaluation Criteria


• Grid comprises of 6 aspects of SLDS– For each aspect, the grid lists various issues along with options (based

on the available state of the art at that time) along with pros and cons

– Choices to be made by the developers

• Essentially and Iterative Life Cycle with 2 legs– Development Phases

• Grid Issues are addressed in these phases

– Evaluation• Evaluation criteria derived from choices made for the various Grid issues are

applied here

• This life cycle does not talk much about the phase after integration i.e. maintenance.

• Also scope for re-use is not discussed


• Evaluation– Analysis: Sufficiency and clarity of documents– Design: Design goals and constraints are sound, non-contradictory and feasible– Simulation: WOZ with questionnaires– Construction: Glass box and black box tests– Integration: Collected user system interaction is analysed


• Iterative Method– Work iteratively from 2 angles

• Conceptual Design• Framework customization

• Solid Lines– Creative Progression

• Dotted Line– How levels of the method

correlate

• E.g. Interpretation Module– Conceptual design starts with Parsing theory and Requirement

Specifications

– Framework customization starts with Parser and builds on corpus created from WOZ or other approaches


• Iterative Method: Recommended Pre-requisites– Classification of possible dialogues and identification of main use-cases

– Specification of requirements for the system behavior in each class of dialogues

• Considering Dialog Management in Iterative Development– Design

• Modularization• Knowledge Representation• Interfaces

– Customization• Tools• Framework templates• Code Patterns

• Identified Design choices– Dialogue History, User Request Handling, Sub-Dialogue Control


• Model View Controller Approach: Very Industrial– Derived from well established GUI development practice

• Possible to upload the user interface without modifying the application data or control logic



• Rapid Dialogue Prototyping Methodology (RDPM)– Focuses on Dialogue components only

• 5 steps– Producing the Task model

• Draw relations between attributes and task to be performed

– Deriving the initial dialogue model• Generic Dialogue Nodes (GDN)

– Simple (Static): User inputs

– List (Dynamic): Selects from menu

– Internal: Start/resets, etc.

• Local / Global dialogue flow management strategy

– Using a WoZ experiment to instantiate the initial dialogue model

– Using an Internal field test to refine the dialogue model

– Using an external field test to evaluate the final dialogue model


(Atwell et. al., 2000)• Experience with user involvement in development

– Particular case for CALL applications

• Involve users at every stage• Two distinct classes of users

– Prospective End-Users

– Meta Level Experts

• Interesting findings reported in the paper within the application domain which was possible due to user involvements at all stages


• Methodologies I have used

– Case 1: ConQuest dialogue system development

– Case 2: CycleTalk dialogue system development

• Sorry no time to make the slides for the above. But lets discuss



• SLDS development methodologies & Best practices– (DISC) Dialogue Engineering Life Cycle– Iterative Method– Model View Controller (MVC) approach– Rapid Dialogue Prototyping (RDP) Methodology– User involvement in development– Methodologies i have used : 2 cases



• Some ideas






• Knowledge Sources view of Dialogue Systems– Overview– Tools and Methods required

• Some ideas


• Knowledge Sources View

• Dialogue system development can be seen as a process of putting knowledge into an existing framework in order to customize the framework

• Tools and methods must– Provide visual languages for specification of structured dialogue data

– Represent all relevant data and the dependences between them

– Support formalism for defining constraints on the models and informing the user of violation of these models

– Generate code that can be interpreted be generic dialogue system

– Support the reuse for formerly developed domain models

Development Team & Requirements

User Logs

Randomlycreated Corpus

Dialog TaskSpecification

GrammarTemplates

VoiceDomain

KnowledgeLexicon

LanguageModel

AcousticModel

1

2

10

34

5

6

7

8

911

1213

1415

16

• Knowledge Sources View

• Dialogue system development can be seen as a process of putting knowledge into an existing framework in order to customize the framework

• Tools and methods must– Provide visual languages for specification of structured dialogue data

– Represent all relevant data and the dependences between them

– Support formalism for defining constraints on the models and informing the user of violation of these models

– Generate code that can be interpreted be generic dialogue system

– Support the reuse for formerly developed domain models





• Knowledge Sources view of Dialogue Systems– Overview– Tools and Methods required

• Some ideas– Template based


Some Ideas: Template based development

• Somewhere in between the spectrum of domain independence and domain dependence

– Based on extensive in domain re-use

– Changes in knowledge sources with every re-use

– Issues• Incorporating knowledge changes• Propagation of knowledge changes while maintaining constraints

– Hence view of the template system development would be• Which knowledge source is to be changed ?

• Readings– An architecture for Generic Dialogue Shell, J. Allen, D. Byron, M. Dzikovska,

G. Ferguson, L. Galescu, A. Stent– Rapid Prototyping for Spoken Dialogue Systems, Matthias Denecke– Developing Extensible and Reusable Spoken Dialogue Components: An

examination of the Queen’s Communicator, Philip Hanna, Ian O’Neil, Xingkun Liu, Michael McTear

– Iterative implementation of Dialogue System Modules, Lars Degerstedt Arne Jonsson

– The Dialogue Engineering Life Cycle, Laila Dybkjaer, Niels Ole Bernsen– DISC Website, www.disc2.dk– Rapid Dialogue Prototyping Methodology, Trung H. Bui, Martin Rajman,

Miroslav Melichar– User Guided System Development in Interactive Spoken Language

Education, E. Atwell et. al.– OpenSpeech Dialog: More powerful applications at lower cost,

SpeechWorks white paper– A survey of knowledge sources in dialogue systems, Annika Flycht-Erikssn– Universal Dialogue Specification for Conversational Systems, Anke Kolzer

http://www.disc2.dk/

Date post:	01-Apr-2015
Category:	Documents
Upload:	sheila-carle
View:	215 times
Download:	1 times

Software Development Processes, Reuse and Knowledge Sources in Spoken Dialog System 11761 Dialog...

Documents