Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 1
Mocking Web ServicesHow do you test online services… whenyou’re not online?
Scott DavisDavisworld Consulting
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 2
Introduction
My name is Scott Davis
JBoss At Work(O’Reilly)
Google Maps API(Pragmatic Bookshelf)
Pragmatic GIS(Pragmatic Bookshelf)
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 3
We’re going to talk about how to test WebServices
SOAP
REST
JSON
A big part of testing Web Services is mockingthem up
A mock is a “stand-in” for the real service
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 4
Why Mock?
There are several reasons why you mightnot want to beat on the real thing duringtesting
Metered service / $$$
Bandwidth / Lag-time
Offline testing
• Laptop at 20,000 feet
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 5
Let’s get started by mocking up the mostpopular of the bunch…
SOAP
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 6
What Is SOAP?
SOAP is one of the most common WSimplementations
SOAP is a protocol for exchanging XML-based messages over a computer network,normally using HTTP. SOAP forms the foundation layer of the web services stack,providing a basic messaging framework that more abstract layers can build on.SOAP facilitates the Service-Oriented architectural pattern.
The name "SOAP" was originally an acronym for Simple Object Access Protocol, butthe full name was dropped in Version 1.2 of the SOAP specification…
(Source: http://en.wikipedia.org/wiki/SOAP)
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 7
SOAP Specifics
Transport:
Commonly HTTP POST, but can also be SMTP,JMS, etc.
Request format:
XML (SOAP Envelope, Header, Body)
Response Format
XML (SOAP Envelope, Header, Body)
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 8
What Is WSDL?
SOAP is the message format
WSDL describes the WS interface
The Web Services Description Language (WSDL) is an XML format published fordescribing Web services.
WSDL describes the public interface to the web service. This is an XML-basedservice description on how to communicate using the web service; namely, theprotocol bindings and message formats required to interact with the web serviceslisted in its directory. The supported operations and messages are describedabstractly, and then bound to a concrete network protocol and message format.
(Source: http://en.wikipedia.org/wiki/WSDL)
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 9
To see SOAP in action, let’s use the freeGoogle WS API:
With the Google Web APIs service, software developers can query billions of webpages directly from their own computer programs. Google uses the SOAP andWSDL standards so a developer can program in his or her favoriteenvironment - such as Java, Perl, or Visual Studio .NET.
To start writing programs using Google Web APIs:1. Download the developer’s kit2. Create a Google Account3. Write your program using your license key
(Source: http://www.google.com/apis/)
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 10
What we’retrying toaccomplish isperformingthis queryprogrammat-ically
XML resultsinstead ofHTML
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 11
The WSDL (in the developer’s toolkit wedownloaded) describes the WS
The URL
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 12
The available method calls:
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 13
The binding:
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 14
The request arguments:
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 15
The response format:
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 16
Here is the SOAP Request:
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 17
And here is the SOAP Response:
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 18
…and the first response element:
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 19
Before we can mock up the Google WebService, we need to get our hands on somereal output
SOAPClient4XG
cURL
Apache JMeter
Apache AXIS TCPMonitor
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 20
SOAPClient4XGhttp://www-128.ibm.com/developerworks/xml/library/x-soapcl/
“SOAP Client for XML Geeks”
• 100 lines of code
• Allows you to make SOAP requests from thecommand line
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 21
SOAPClient4XG
java -jar sc4xg.jar SC4XG http://api.google.com/search/beta2 sampleSearch.xml
URL to SOAP service
SOAP Request
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 22
cURL
http://curl.haxx.se/download.html
The ubiquitous command-line tool forperforming HTTP GETs and POSTs is availablefor every platform
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 23
cURL
curl --request POST --header "Content-Type: text/xml" --data @sampleSpelling.xml http://api.google.com/search/beta2
URL to SOAP service
SOAP Request
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 24
Apache Jmeter
http://jakarta.apache.org/jmeter/
This load/stress testing tool has built-in supportfor SOAP
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 25
JMeter Request
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 26
JMeter Response
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 27
Apache AXIS TCPMonitor
http://ws.apache.org/axis/
You might already be using this as your SOAPservice provider, but there is a hidden gemburied deep inside…
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 28
TCPMonitor
curl --request POST --header "Content-Type: text/xml" --data @sampleSpelling.xml http://localhost:8989/search/beta2
URL to SOAP service
(Notice the URL points tolocalhost:8989)
SOAP Request
java -classpath $AXIS_HOME/lib/axis.jar org.apache.axis.utils.tcpmon
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 29
TCPMonitor Setup
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 30
TCPMonitor Response
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 31
Now that we have some valid output, howcan we set up a mock Google web service?
JSP
Groovlets
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 32
JSP
Gives you the flexibility to simply copy/pastein the XML/SOAP response
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 33
Groovy
Groovy is a scripting language that runs atopJava
You can run it as an interpreted (uncompiled)scripting language
You can compile it down to bytecode and mix itin with your existing Java codebase
• http://groovy.codehaus.org
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 34
Groovlets = Groovy + Servlets
Once you addgroovy.servlet.GroovyServlet toweb.xml, you can add Groovy scripts on the fly
• No redeploy necessary
• GroovyServlet compiles and runs the scriptsautomagically
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 35
Web.xml
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 36
WEB-INF/lib
Just add a few choice JARs toWEB-INF/lib…
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 37
soap.groovy
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 38
While the JSP and Groovlet examples lookvirtually identical at first glance, the Groovletis far more robust
You have the full programming language at yourfingertips
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 39
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 40
Now that we have a mock Google webservice, how can we call it?
DNS trickery
WSDL trickery
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 41
DNS Trickery
Add “127.0.0.1 api.google.com” to /etc/hosts
Create a “search” webapp
Create a “beta2” subdirectory
Or do sneaky URL redirects using TCPMon
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 42
WSDL Trickery
The WSDL (in the developer’s toolkit wedownloaded) describes the WS
The URL
Flip this to http://localhost
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 43
SOAP Conclusion
SOAP is only one implementation of WS.
There is a “kinder, gentler” form of WS availableto you
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 44
What Is REST?
The other white meat:
Representational State Transfer (REST) is a software architectural style fordistributed hypermedia systems like the world wide web. The term originated in a2000 doctoral dissertation about the web written by Roy Fielding, one of theprincipal authors of the HTTP protocol specification, and has quickly passed intowidespread use in the networking community.
Systems that follow Fielding's REST principles are often referred to as RESTful;REST's most zealous advocates call themselves RESTafarians.
(Source: http://en.wikipedia.org/wiki/Representational_State_Transfer)
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 45
There are two “types” of RESTful WS outthere:
Pure REST (following Fielding’s principles)
Popular REST (essentially what all of themainstream websites provide)
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 46
Popular REST is a reaction to the complexityof SOAP
Transport: HTTP GET
Request format: URL + QueryString
Response format: XML
• Sometimes it is informally called POX (Plain Old XML)in homage to POJOs (Plain Old Java Objects).
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 47
To see a “Popular” RESTful WS in action,let’s go over to Yahoo!
How do I get started?
1. Online DocumentationRead the online documentation and FAQs.
2. Get an Application IDTo access Yahoo! Search Webservices, you will need to get an application ID. Like abrowser's User-Agent string, the Application ID uniquely identifies your applicationand has no e ect on rate limiting.
3. Download the SDKThe development kit includes BSD licensed examples and libraries for variouslanguages: Perl, Python and PHP, Java, JavaScript, and Flash.
(Source: http://developer.yahoo.net/search/index.html)
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 48
Yahoo offers close to 20 different RESTfulservices (Search, Maps, Weather, Traffic,Finance, Shopping)
Here is the same web search we did in Google:• http://api.search.yahoo.com/WebSearchService/V1/webSearch?appid=00000000&query=jboss+at+work
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 49
POX
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 50
How do we get valid sample output?
All of the tools we examined earlier, plus:
• The Browser
• wget
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 51
The Browser
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 52
wget
http://www.gnu.org/software/wget/
Does basically the same thing as cURL, only thesyntax is a bit simpler for GETs
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 53
wget
wget -O out.xml“http://api.search.yahoo.com/WebSearchService/V1/webSearch?appid=000000&query=jboss+at+work”
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 54
Many popular websites offer both SOAP andRESTful interfaces
Amazon• http://www.amazon.com/gp/browse.html/102-2243700-0003349?node=3435361
eBay
• http://developer.ebay.com/common/api
A good search term to discover WS offerings is“[website] api”
Ebay api
Amazon api
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 55
Amazon mocking
http://awszone.com/
Amazon has a much more complicatedRESTful API
Awszone helps tremendously
Once you have the requests built and the sampleoutput, mocking it up is a breeze
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 56
AWS Main
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 57
AWS ItemSearch
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 58
AWS Code Generator
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 59
REST Conclusion
With the popularity of AJAX on the rise (andthe continuing weakness of native browserXML support), there is a third WS “flavor”gaining steam:
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 60
What Is JSON?
We don’t need no stinkin’ XML…
JSON (pronounced Jason), which stands for "JavaScript Object Notation", is alightweight computer data interchange format. JSON is a subset of the object literalnotation of JavaScript but its use does not require JavaScript.
JSON's simplicity has resulted in its widespread use, especially as an alternative toXML in Ajax. One of the claimed advantages of JSON over XML as a data interchangeformat in this context is that it is much easier to write a JSON parser. In Javascriptitself, JSON can be parsed trivially using the eval() procedure. This was importantfor the acceptance of JSON within the AJAX programming community because ofJavaScript's ubiquity among web browsers.
(Source: http://en.wikipedia.org/wiki/JSON)
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 61
JSON specifics:
Transport: HTTP GET
Request format: URL + QueryString
Response format: JSON
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 62
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 63
From a JavaScript perspective, once youhave a JSON string, all you need to do iseval() it:
Eval(menuDef);
No need for marshalling framework like Castoror XMLBeans• Ruby uses a superset of JSON called YAML
YAML is a recursive acronym meaning "YAML Ain't Markup Language". Early in itsdevelopment, YAML was said to mean "Yet Another Markup Language", retronymedto distinguish its purpose as data-centric, rather than document markup.
(Source: http://en.wikipedia.org/wiki/YAML)
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 64
Even though JSON has “JavaScript” in itsname, there are bindings for over 15 otherprogramming languages
See http://www.json.org/ for open sourceparsers
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 65
Getting sample output
wget
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 66
wget
Yahoo! Supports JSON output by simplyadding another query parameter:
output=json
For more information:• http://developer.yahoo.com/common/json.html
wget -O out.txt "http://api.search.yahoo.com/WebSearchService/V1/webSearch? appid=000000& query=jboss+at+work& output=json"
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 67
We talked about how to test Web ServicesSOAP
REST
JSON
There are many tools that helped us out along theway:
SOAPClient4XG
cURL
Apache Jmeter
Apache Axis TCPMonitor
Firefox / Safari / IE / et al
wget
Colorado Software Summit: October 22 – 27, 2006 © Copyright 2006, Davisworld.org
Scott Davis — Mocking Web Services Page 68
Conclusion
Thanks for your time!
Questions?
Email: [email protected]
Download slides:
• http://www.davisworld.org/presentations