Speech MashupsDan Burnett, Voxeo

SpeechTek 2010

What are mashups?

<XML>JSON

HTTP

Why VoiceXML?

Why not VoiceXML?

Tropo scripting

answer/reject/redirect/hangupcall/transfer/conference ask/say/record/log

startCallRecording/stopCallRecording

Weather app

• Collect postal code from caller

• Fetch weather from Yahoo using YQL

• Speak weather to the caller

Weather app

require 'open-uri'require 'json'

answer

hangup

Weather app

require 'open-uri'require 'json'

answersay 'Welcome to the Ruby Yahoo weather reader'

say 'Thats all. Goodbye.'hangup

Weather apprequire 'open-uri'require 'json'

answersay 'Welcome to the Ruby Yahoo weather reader'

ask 'Enter the ZIP code for a weather check'say 'Thats all. Goodbye.'hangup

Weather apprequire 'open-uri'require 'json'

answersay 'Welcome to the Ruby Yahoo weather reader'

options = { :choices => "[5 DIGITS]", :onChoice => lambda { |choice|

#Set the URI and our YQL select statement, then encode as a URI

#Fetch the JSON from the YQL API and convert the resulting #JSON data to a Ruby hash

#Get the relevant weather channel details and throw them into a hash

#Speak back the results

} }

ask 'Enter the ZIP code for a weather check', optionssay 'Thats all. Goodbye.'hangup

Weather apprequire 'open-uri'require 'json'

answersay 'Welcome to the Ruby Yahoo weather reader'

options = { :choices => "[5 DIGITS]", :onChoice => lambda { |choice|

#Set the URI and our YQL select statement, then encode as a URI yahoo_url = 'http://query.yahooapis.com/v1/public/yql?format=json&q=' query = "SELECT * FROM weather.forecast WHERE location = " + choice.value url = URI.encode(yahoo_url + query) #Fetch the JSON from the YQL API and convert the resulting #JSON data to a Ruby hash weather_data = JSON.parse(open(url).read) #Get the relevant weather channel details and throw them into a hash weather_results = weather_data["query"]["results"]["channel"]

#Speak back the results

} }

ask 'Enter the ZIP code for a weather check', optionssay 'Thats all. Goodbye.'hangup

Weather apprequire 'open-uri'require 'json'

answersay 'Welcome to the Ruby Yahoo weather reader'

options = { :choices => "[5 DIGITS]", :onChoice => lambda { |choice|

#Set the URI and our YQL select statement, then encode as a URI #Fetch the JSON from the YQL API and convert the resulting #JSON data to a Ruby hash #Get the relevant weather channel details and throw them into a hash

#Speak back the results say weather_results["description"] say "The wind chill is #{weather_results["wind"]["chill"]} degrees, " + "the wind speed is #{weather_results["wind"]["speed"]}" say "The forecast is #{weather_results["item"]["forecast"][0]["text"]}, " + "with a high of #{weather_results["item"]["forecast"][0]["high"]} degrees, " + "and a low of #{weather_results["item"]["forecast"][0]["low"]} degrees." } }

ask 'Enter the ZIP code for a weather check', optionssay 'Thats all. Goodbye.'hangup

Weather apprequire 'open-uri'require 'json'

answersay 'Welcome to the Ruby Yahoo weather reader'

options = { :choices => "[5 DIGITS]", :repeat => 3, :timeout => 7, :onBadChoice => lambda { say 'Invalid entry, please try again.' }, :onTimeout => lambda { say 'Timeout, please try again.' }, :onChoice => lambda { |choice|

#Set the URI and our YQL select statement, then encode as a URI #Fetch the JSON from the YQL API and convert the resulting #JSON data to a Ruby hash #Get the relevant weather channel details and throw them into a hash

#Speak back the results

} }

ask 'Enter the ZIP code for a weather check', optionssay 'Thats all. Goodbye.'hangup

Demo

• Upload script

• Link to a telephony application name

• Add phone numbers/IM/Jabber/SMS

Tropo Web API

answer/reject/redirect/hangupcall/session/transfer/conference ask/say/record/on/result/log

startCallRecording/stopCallRecording

JSON (JavaScript Object Notation)

tropo

Weather app

{"tropo": [ {"say": [{"value": "Welcome to the Yahoo weather reader"}]}, {"ask": {"say": [{"value": "Enter the ZIP code for a weather check"}, {"value": "Invalid entry, please try again.", "event": "nomatch"}, {"value": "Timeout, please try again.", "event": "timeout"}]}, "repeat": 3, "timeout": 7, "choices": {"value": "[5 DIGITS]"}, }]}

----------------- Receive recognition result ------------------------

{"tropo": [ {"say": [{"value": "The wind chill is ..."}, {"value": "The forecast is ..."}, {"value": "Thats all. Goodbye."}]}}]}

Other mashups• Yahoo Local over phone, SMS, and IM

https://www.tropo.com/docs/scripting/t_php-localsearch.htm

• SFO BART train arrival estimatehttps://www.tropo.com/docs/scripting/t_ruby-barteta.htm

• Find location of someone based on telephone numberhttps://www.tropo.com/docs/scripting/t_groovy-location-mashup.htm

• Look up political campaign contributionstext: [2-letter state id] [name] to +1.240.242.7944 .. ie "ut craig m smith"(also on jabber at tdata@tropo.im)

• Look up bills in the NY Senate via IM or Twitterhttp://www.voiceingov.org/blog/?p=1005

• Volunteer firefighter arrival time at fire station

Summary