Date post: | 23-Jan-2018 |
Category: |
Technology |
Upload: | distil-networks |
View: | 193 times |
Download: | 0 times |
Antoine Zammit Elias Terman
VP of MarketingVP of Technology
Don’t Let Bad Bots
Deflate Your Conversion
Rates and Brand
+28 More Brands!
Good Bots, Bad Bots, and Human Traffic
The Open Web Application Security Project (OWASP) is an important standards body in the application security community. Their annual top
10 threats list is the basis for many web application security programs. They are now expanding their scope to include automated threats -
bots.
SUBSET OF THREATS NAME DEFINING CHARACTERISTICS
Account Aggregation
Account Creation
Credential Cracking
Credential Stuffing
Use by an intermediary application that collects together multiple accounts and interacts on their behalf
Create multiple account for subsequent misuse
Identify valid login credentials by trying different values for username and/or passwords
Mass log in attempts to verify the validity of stolen username/password pairs
Carding
Card Cracking
Cashing Out
Multiple payment authorisation attempts used to verify the validity of bulk stolen payment card data
Identify missing start/expiry dates and security codes for stolen payment card data by trying different values
Buy goods or obtain cash utilising validated stolen payment card or other user account data
Footprinting
Vulnerability Scanning
Fingerprinting
Probe and explore application to identify its constituents and properties
Crawl and fuzz application to identify weaknesses and possible vulnerabilities
Elicit information about the supporting software and framework types and versions
OTHER
Ad Fraud
CAPTCHA Bypass
Denial of Service
Expediting
Scalping
Scraping
Skewing
Sniping
Spamming
Token Cracking
False clicks and fraudulent display of web placed advertisements
Solve anti-automation tests
Target resources of the application and database servers, or individual user accounts, to achieve denial of service
Perform actions to hasten progress of usually slow, tedious or time-consuming actions
Obtain limited-availability and/or preferred goods/services by unfair methods
Collect application content and/or other data for use elsewhere
Repeated link clicks, page requests or form submissions intended to alter some metric
Last minute bid or offer for goods or services
Malicious or questionable information addition that appears in public or private content, databases or user messages
Mass enumeration of coupon numbers, voucher codes, discount tokens, etc
100% OF OWASP AUTOMATED THREATS (BOTS) TARGET TRAVEL INDUSTRY
PAYMENT
CARDHOLDER
DATA
ACCOUNT
CREDENTIALS
VULNERABILITY
IDENTIFICATION
This work is licensed under the Creative Commons Share-Alike License for OWASP Automated Threat Handbook Web Applications by Distil Networks
The bad bot landscape
How bad bots impact the travel industry
Web/screen scraping and spinning (hoarding)
Increased GDS pull costs
Decreased SEO, slowdowns, and downtime
Account takeover, credit card fraud, and points fraud
Skewed conversion metrics and look-to-book ratios
WMPH Vacations Case Study
Q&A
Agenda
Advanced Persistent Bots
Basic scripts running
in command line
Headless browsers,
advanced scripts,
Cycle IPs and User
Agents
Real browser
automation, malware
APBs
75%
More Bad Bots Claim to Be Mobile
The amount of bad bots claiming to be
mobile browsers jumped 42.78% in 2016
Mobile App Tools Used by Bot Operators
Mobile Device Farms Mobile Device Emulators Debugging Software
Mobile device emulators that mimic
human users
Testing systems that mimic human users
on mobile devices (e.g. AWS Device
Farm, Google Firebase Testing Lab)
Debugging software used for
tampering with SDKs/reverse
engineering the app
About Distil Networks
Industry Expertise
● Invented the category
● The recognized leader
● 70 airline customers
The Most Effective Technology
● Wider: Web, API, and Mobile
● Deeper: Catch more bots
● Smarter: Without impacting users
Vigilant and Dedicated Partner
● Not A Solution, Your Solution
● Unprecedented access
● An extension of your team
Bot Defense as Adaptable and Vigilant as the Threat Itself
Travel Industry Leaders Rely on Distil...
True or False?
You have good visibility and control
over unwanted website traffic and
transactions.
Poll
Question
You’ve Been Scraped
OWASP AUTOMATED THREAT: SCRAPING
Scraper Bot Sophistication
CompetitorsContent Theft
Competitive Intel
Price Scraping
AggregatorsStart-ups
Unauthorized Middlemen
Hackers / FraudstersContent for Fake Pages
Search EnginesGoogle
Bing
Yahoo
Baidu
Who is behind Web Scraping?
What Kind of Data is Being Scraped?
Customer data
Pricing info
Editorial content
GDS API pulls
SEO strategies
Booking engine inputs
Spinning (Hoarding) by Unauthorized Middlemen
Middlemen using mobile device emulators to continuously
hold seats in the airline booking engine, but not buying
Resell on a secondary market once a buyer is found
Monetary damage:
➔ Empty seats on planes
➔ Loss of add-on sales like upgrades, travel insurance,
etc. (about $20 to $40 of additional revenue per sale
for airlines*)
AIRLINE
CUSTOMER USE CASE
Spinning via
Mobile App
Emulators
Source: http://www.eyefortravel.com/mobile-and-technology/scraping-single-biggest-threat-travel-industry*
Application Denial of Service
OWASP AUTOMATED THREAT: DENIAL OF
SERVICE
Denial of Service Bot
Sophistication
DDoS vs. Application Denial of Service
Application Denial of Service
Attacks the application directly
Hard to spot because it won’t show up
as an anomaly on your firewall and
may not impact load balancer
DDoS
Attacks the ISP hosting your
application
Easier to spot because it floods
upstream infrastructure to point where
packets never arrive at the web server
Account Takeover, Credit Card
Fraud, and Loyalty Points Fraud
Bad Bots Love Login Pages
OWASP AUTOMATED THREATS:
CREDENTIAL CRACKING, CREDENTIAL STUFFING
Account Takeover Bot
Sophistication
How Credential Stuffing Works
Over 1 billion
usernames, passwords
combinations exist in the
wild
Credential stuffing exploits
our propensity to reuse
passwords across multiple
sites.
Account Based Fraud
OWASP AUTOMATED THREATS:
CARDING, CARD CRACKING, CASHING OUT
Account Exploitation Bot
Sophistication
Travel Rewards Fraud
Dark Web listings that indicate typical price
ranges for airline and hotel loyalty accounts:
Airline loyalty accounts: $3.20 - $208
Hotel loyalty accounts: $1.50 - $45
Source: http://blog.cxloyalty.com/the-cost-of-loyalty-accounts-on-the-dark-web-how-to-protect-members
72 percent of loyalty program managers say they
have experienced an instance of loyalty program
fraud firsthand
Skewed Analytics and Look-to-Book Ratios
OWASP AUTOMATED THREAT: SKEWING
Sophistication level of bots
that skew analytics
Sophisticated Bots Appear as Human in Analytic Data
53% of bots able to load external Assets (e.g. JavaScript)
These bots will skew marketing tools such as (Google
Analytics, A/B testing, conversion tracking, etc.)
Skewed Analytics Leads to Misinformed Business Decisions
Inaccurate analytic data results in
Poor funnel analysis & optimization
Poor conversion rates
Inaccurate KPI tracking
Skewed look-to-book ratios
Difficulty in planning server expansion
The bad bot problem I'm most
concerned about:
A. Web scraping
B. Account-based fraud
C. Skewed analytics / look-to-book
D. Slowdowns and downtime
Poll
Question
About WMPH Vacations
At a Glance
Founded 2004 / 140 employees
More than 600,000 clients booked
9 corporate brands
30 websites
Award-Winning Mobile App
Reservation systems serve both direct
customers and 45 agents
Private label solutions
WMPH Technology Stack
30 different web properties
Mobile iCruise App for IOS & Android
Standardized web application stack
Employee Intranet
10 Virtual Servers on AWS
Cloud-based Phone System using 8x8 technology
Entire company is now over 90% cloud-based
API calls into everything from small cruise lines to
large Global Distribution Systems
WMPH Bot Challenges
Bad Bot Challenges
Aggressive web scraping caused site
slowdowns
API scraping almost took a cruise partner
offline
Constant barrage of SQL injection attack
attempts caused lots of noise in logs
Spam on cruise inquiry forms polluted
backend systems
Bots skewed conversion metrics
Tried Several Approaches to Solve the Problem...
Put CAPTCHAs on Forms Looked for Patterns Blocked IPs in AWS ELB
Creates a poor user experience Bots appear human in logs Defeated by distributed IP attacks
Defeated by advanced bots Labor intensive Defeated by low and slow crawlers
Defeated by CAPTCHA farms Distributed attacks hard to pinpoint Defeated by peer-to-peer / proxies
Reduces conversions rates Reactive in nature Reactive in nature
WMPH Vacations Selection Criteria
Bot Detection and Mitigation Solution Requirements
Block web scrapers without impacting human visitors or
good bots like Googlebot
Increase website availability and speed
Simple setup
Little or no maintenance; “self-optimizing” solution
Protect APIs powering our websites and mobile apps
Protect our web and mobile API servers
Fingerprint device
Verify browser
Verify device
Verify human
Verify Mobile Device ID
Verify mobile app
Verify device
Verify human
Stop bot operators (using mobile device farms,
device emulators, etc.) from accessing the API
servers that power our mobile apps
Prevent scrapers from hitting our
APIs through our website or by going
directly to our API servers
WMPH Results with Distil
40% increase in response times; no slowdowns
since deploying Distil
Improved partner relationships
Leads up 100% – No more spam – Only serving
CAPTCHAs to bots
Conversion rates up 22%
Self-tuning, proactive approach saving 20 hours
per month
Protecting login of company intranet
iCruise.com Traffic Overview
iCruise.com Traffic Overview
iCruise.com Traffic Overview
iCruise.com Click Fraud Report
Best Practices and Lessons Learned
IT and marketing need to partner on solving
the bad bot problem.
Review the Distil logs daily.
Blacklist aggressive bot IP numbers
Report aggressive IPs to their respective
IPSs. Follow up, and follow up, and follow
up.
Distil support will give you a list of urls being
hit by the bad bots. This will help you
determine what they are trying to do.
Don’t whitelist your office IP right away.
www.distilnetworks.com/trial/
Offer Ends: October 31st at 5PM
Two Months of Free Service + Traffic Analysis
Antoine Zammit Elias Terman
VP of MarketingVP of Technology