Authors:Jason Lu
Michael S. PeñaSashmit Bhaduri
Center for Lifelong Continuing EducationUniversity of Texas at Austin
Austin Texas, U.S.A
Distributed Bidding System (DBS)
An Amazon EC2 Cloud Application
Introduction Motivation Relationships Architecture Implementation Scenarios Design Considerations Future Work Conclusion
DBS Presentation Outline
• E-commerce“..healthy 12.6% increase to $176.2 billion in 2010,
U.S. online retail sales are expected to reach $278.9 billion in 2015”- Forrester Research Inc. online
• Bidding System & Other sitesEx. Ebay, dealighted.com, slickdeals.net, etc..
• Need of Distributed Systems
• Cloud Computing Ex. Amazon EC2, Microsoft Azure, Google,
Salesforce, Rackspace, and Zoho
Introduction
Goals : IS vs. IS NOTo IS: Scalability, Consistency, Adequate
Performance Systemo IS NOT: Persistent, Security, Real Auction
System
Focuso Distributed algorithm vs Consumer friendly
Motivation
Cloud Computing Comparisono Free tier licensingo Free accountso Rich APIs supporto Most widely utilized Cloud service
Motivation (Cont.)
Java TCP Client/Server
Distributed Resource Allocationo 1st explored Lamport Mutual Exclusion Algorithmo Used Ricart and Agrawala’s Algorithm
Transactions, No Leader Election
Relationships
Architecture
End user (web browser) -> BidClientServlet Sequence of Operations:◦ Open HTTP connection to HTTP server (running on :8081)◦ HTTP server passes control to BidClientServlet◦ BidClientServlet processes request, perhaps connecting to a
random BidServer depending on the specific action◦ BidClientServlet generates HTML, sends back to web
browser
Architecture
HTTP API (specified via RFC1738/RFC3986-encoded params in URL):◦ Login – performs login e.g, action=login&username=foobar
Sets HTTP Cookie to track username (w/o cookie, goes back to login page automatically)
Nothing sent to BidServers – Web Browser only◦ Auction List – shows auction list
Connects to a BidServer Gets ItemIds, ItemInfo, TopBid for each Auction Disconnects, generate HTML table w/ info
◦ Detail/Place Same HTML output page Connects to a BidServer If Placing Bid, sends PlaceBid Gets ItemInfo, all Bids Disconnects, generate HTML w/ results of ItemInfo/Bid
List◦ Logout – logout, clears cookie
Architecture
TechnologiesoOpenJDK 6o BidServers, BidClientServlet
o Apache Tomcat 6.0o Catalina/J2EEo BidClientServlet (running within HTTP server)
o Deployment:o Amazon’s Elastic Compute Cloud
o Amazon EC2 API Tools (utilizes EC2 Java API)o Ubuntu 10.10 using Linux 2.6.x
o OS for EC2 instanceso Python 2.6o openssh
Implementation
GUI Screenshot Login Page:
Implementation (Cont.)
GUI ScreenshotImplementation (Cont.)
GUI ScreenshotImplementation (Cont.)
Communication
Bid Server
1
Bid Server
2
Bid Client
Command DescriptionHello Establish initial Lamport Time StampRequestCS Ask for Ricart/Agrawala Mutex for read/write
by item IdItemBids Get Bid history of a specific Item by item IdItemTopBid Get highest bid of a specific item by item Id.
Command Description
ItemIds Get IDs of all items ready for auctionItemInfo Get description of the item by item id
ItemTopBid Get highest bid of a specific item by item Id.
ItemBids Get bid history of a specific item by item id
PlaceBid Place a bid on a specific item by item id
ProtocolServer-Server Command
Request Response
Hello Request=Hello|TimeStamp=10 Response=Hello|TimeStamp=11
RequestCS Request=requestCS|TimeStamp=3928|ItemId=8|ReadOnly=true|SrcId=2
Response=requestCS|TimeStamp=3929|ItemId=8
ItemBids Request=ItemBids|TimeStamp=18|ItemId=4
Response=ItemBids|TimeStamp=21|ItemId=4|Bid=2000000|Bidder=jason|BidTime=05/09/2011 00:01:58
ItemTopBid N/A Response=ItemTopBid|ItemId=8|Bid=4000000|Bidder=sashmit|BidTime=05/09/2011 00:01:58
Client-Server Command
Request Response
ItemIds Request=ItemIds Response=ItemIds|ItemId=1|ItemId=2|ItemId=3|ItemId=4|ItemId=5|ItemId=6|ItemId=7|ItemId=8|ItemId=9|ItemId=10
ItemInfo Request=ItemInfo|ItemId=3 Response=ItemInfo|ItemId=3|ItemTitle=GM|ItemDesc=2005 Blue|AuctionEndTime=06/03/2011 23:59:59
ItemTopBid Request=ItemTopBid|ItemId=4 Response=ItemTopBid|ItemId=4|Bid=2000000|Bidder=jason|BidTime=05/08/2011 21:20:59
ItemBids Request=ItemInfo|ItemId=1 Response=ItemBids|ItemId=1|Bid=1000000|Bidder=jason|BidTime=05/09/2011 00:01:58|Bid=1100000|Bidder=sashmit|BidTime=05/09/2011 00:01:58|Bid=1200000|Bidder=jason|BidTime=05/09/2011 00:01:58|Bid=1300000|Bidder=jason|BidTime=05/09/2011 00:01:58|Bid=2000000|Bidder=jason|BidTime=05/09/2011 00:01:58
PlaceBid Request=PlaceBid|ItemId=1|Bid=2001000|Bidder=sashmit|BidTime=05/09/2011 00:10:49
NA
Scenario: Bid Server joins the cluster
Scenario: Bid Client Retrieves Auction Info
Scenario: Client Place A Valid Bid
Performance◦ Ricart & Agrawala’s mutex instead of Lamport’s to have
2(N-1) instead of 3(N-1) messages.◦ CS applies to each item instead of globally so that mutex
conflict is low ◦ Applies ReadOnly and ReadWrite CS on item so that
multiple reads, which is the majority of the request, can happen at the same time.
◦ Partition Item info so that some data doesn’t need to be locked even for reading. Read static data: no synchronization Read dynamic data: Read Only mutex Write dynamic data: Read/Write mutex
Design Considerations
Scalability◦ Queue grows with # of processes with pending
requests on the same item instead of # of all processes.
◦ Even though the mutex is applied on item level, # of mutex data structures only grows with the # of items the current process actively requests CS on.
Resilience (Fault tolerant)◦ Handles Node join and leave
Design Considerations
Distributed bidding system that works o Includes Scalability, Consistency, Adequate
Performance
Features have great promiseo Class Project Consumer applicationo No evaluations / comparison to real-world
applications
Fun project with high potential
Conclusion
MySql Databaseo Initial database, but not interfaced
Nice to have Client Featureso Adding auctionso Auction categorieso Real accountso Auction alerts
Nice to have Cloud Featureso Auction Server Portioningo Auto-Scaling/Load Balancing Cloudo Auto-Deployment
Future Work
Q & A
EC2 management console
Backup Slides
EC2 Running Instances
Backup (Cont.)
EC2 EBS Volumes
Backup (Cont.)
EC2 Health Services
Backup (Cont.)
Deployment script Easier solution than creating own AMI
Backup (Cont.)
