Date post: | 15-Jan-2015 |
Category: |
Technology |
Upload: | sukbum-hong |
View: | 1,801 times |
Download: | 6 times |
DNS DDoS Analysis and Defense
2013. Sep. Sukbum Hong ([email protected])
Please let me know, if there is any error, question, or comment.
Page 1
Contents • DNS Risks • DNS DDoS Attack History • DNS DDoS Attack Types and Defense
(1) Bandwidth Consuming attack (2) Massive A type Query attack (3) Massive other query type attack (4) Amplification attack open-resolver project (5) non-existant(NXDOMAIN) query attack (6) RRL defense
• Conclusion
Page 2
What’s happen if DNS was attacked? Why Risk?
• Every service(web,e-mail,intranet) will be shutdown if DNS is down
• All domains in DNS server will be impacted -. Generally One DNS server services several thousands of domains together
• Hard to change the IP address if get attacked -. generally, DNS TTL is 1Day or 2Days -. need to change the NAMEHOST(whois) as well as DNS A record
• Hard to defense attack as DNS packet is simple -. impossible to distinguish the legitimate traffic from illegitimate traffic
• Hard to defense attack as DNS packet is UDP -. No protocol based ACL
• Hard to block the source IP using rate-limit based
-. As attacker can spoof the Source IP address
Page 3
“Operation Global Blackout” on 31st March :: 2012/03
http://pastebin.com/NKbnh8q8 The principle is simple; a flaw that uses forged UDP packets is to be used to trigger a rush of DNS queries all redirected and reflected to those 13 IPs. The flaw is as follow; since the UDP protocol allows it, we can change the source IP of the sender to our target, thus spoofing the source of the DNS query. The DNS server will then respond to that query by sending the answer to the spoofed IP. Since the answer is always bigger than the query, the DNS answers will then flood the target ip. It is called an amplified because we can use small packets to generate large traffic. It is called reflective because we will not send the queries to the root name servers, instead, we will use a list of known vulnerable DNS servers which will attack the
root servers for us.
Page 4
GoDaddy Outage Takes Down Millions of customer Sites :: 2012/09/10
http://techcrunch.com/2012/09/10/godaddy-outage-takes-down-millions-of-sites/
http://www.wired.com/wiredenterprise/2012/09/godaddy-moves-to-verisign/
Amid Outage, GoDaddy Moves DNS
to Competitor VeriSign
Page 5
Knocked Spamhaus offline with 120G or 300Gbps attack :: 2013/03/19
http://blog.cloudflare.com/the-ddos-that-knocked-spamhaus-offline-and-ho
Officially 120Gbps, Unofficially 300Gbps attack
Page 6
300Gbps almost broke the Internet?
http://blog.cloudflare.com/the-ddos-that-almost-broke-the-internet
there's an online attack underway.
The biggest in history??
Enough to slow down the internet……………….???
http://gizmodo.com/5992652/that-internet-war-apocalypse-is-a-lie
That Internet War Apocalypse Is a Lie
Just to put it in perspective the traffic estimates for the DDOS attack were as high as 300 Gbps at the target. That would easily overwhelm the average hosting center, but not a core component of the Internet. For example, DECIX, the German Internet exchange in Frankfurt, regularly handles 2.5 Tbps at peak on any given day: http://www.de-cix.net/about/statistics/
While it may have severely affected the websites it was targeted at, the global Internet as a whole was not impacted by this localized incident.
Page 7
NetworkSolutions outage because of DNS DDOS attack
http://blogs.cisco.com/security/hijacking-of-dns-records-from-network-solutions/
Over 5,000 domains including linkedin.com in NSI DNS customers were changed with Unknown DNS.
ns1624.ztomy.com(204.11.56.20) . It was in the process to move to Prolexic to defense the DDoS attack
July 16, 2013
http://blogs.cisco.com/security/network-solutions-customer-site-compromises-and-ddos/
Linkedin.com homepage was redirected to DomainSale Homepage
$ traceroute -q 1 ns1624.ztomy.com. ....... 15 209.200.136.34 (209.200.136.34) 118.578 ms 16 unknown.prolexic.com (72.52.18.126) 239.717 ms 17 204.11.56.20 (204.11.56.20) 235.350 ms
July 16, 2013
June 20, 2013
Service outage for 24 hours because of ddos attack
Page 8
Hosting Service outage :: June
:: Hosting Provider in Sydney
http://www.tppwholesale.com.au/support/service-alerts/more-information-recent-ddos-attacks
we took the drastic step of rate limiting DNS queries using the Arbor Pravail equipment to stem the flow of the attack. but due to the aggressive filtering nature there will be some false positives and some customers who will be denied services despite being legitimate users. This kind of rate limiting is not ideal or a long term solution and will result in some further inconvenience. Our long term strategy is to further cluster, load balance and segregate name services to provide greatly enhanced scale, fault tolerance and capacity. This had not been required prior to this attack.
:: DNS hosting provider based in Toronto
It was difficult to differentiate the real traffic from the DDoS traffic
“This is the ‘nightmare scenario’ for DNS providers, because it is not against a specific domain which we can isolate and mitigate, but it’s against easyDNS itself”
http://blog.easydns.org/2013/06/04/post-mortem-of-the-june-3-4th-ddos/
We can recommend DNSMadeEasy, DNSimple or No-Ip, then there's Route53 (our users had good results with easyRoute53 overnight). - See more at: http://blog.easydns.org/2013/06/04/post-mortem-of-the-june-3-4th-ddos/#sthash.wRlXg4iO.dpuf
:: DNS hosting Provider in Florida
“The attacker essentially flooded us with ‘ANY’ queries for a variety of domains managed by our DNS service, with the intention of amplifying these small queries into significantly larger responses aimed at a specific network.”
Page 9
.CN root DNS outage
8/25(Sun) :: 00:00 DDoS attack to .CN root DNS server
02:00 Mitigated the attack
04:00 doos attack again
01:00 service recovered
No information about Attack size, how to attack,etc.
cn. 172800 IN NS a.dns.cn. cn. 172800 IN NS b.dns.cn. cn. 172800 IN NS c.dns.cn. cn. 172800 IN NS d.dns.cn. cn. 172800 IN NS e.dns.cn. cn. 172800 IN NS ns.cernet.net.
google.cn. 86400 IN NS ns2.google.com. google.cn. 86400 IN NS ns3.google.com. google.cn. 86400 IN NS ns1.google.com. google.cn. 86400 IN NS ns4.google.com. ;; Received 109 bytes from 203.119.27.1#53(c.dns.cn) in 75 ms
Around ~30% .CN traffic was downed even long TTL cache.
TTL 1 hour 1D(24h)
1m 1680(1.7%) 70
2m 3360 140
30m 50,000(50%) 2100
1h 100,000(100%) 4200(4.2%)
How much LDNS can lose the DNS cache?
Assumption : # of LDNS is 0.1M
Page 10
DNS DDoS Attack Types and Defense :: Bandwidth Consuming attack
Packet size based Filtering at Network Level?
30Gbps
20Gbps
1Gbps based network
Solution1 :: PBR impossible as eating too much CPU Router(config)# access-list 111 remark "DNS PBR“ Router(config)# access-list 111 permit udp any host dns.ip.addr eq 53 Router(config)# route-map dnsddos permit 10 Router(config-route-map)# match ip address 111 Router(config-route-map)# match length 512 1500 Router(config-route-map)# set interface Null 0 Router(config-if)# ip route-cache policy Router(config-if)# ip policy route-map dnsddos
route 173.X.X.X/32-DNS-DROP { match { destination 173.X.X.X/32; port 53; packet-length [ 99971 99985 ]; } then discard;}
http://blog.cloudflare.com/todays-outage-post-mortem-82515
-. Direct attack from Zombies
-. Normal traffic should be UDP not TCP TCP :: Zone transfer, when response over 512byte
-. Defense :: distributing the DNS infra
-. Defense :: Packet size based filtering if within the Infra size
-. Defense :: efficient by filtering the fragmented packet in upstream ISP
Page 11
DNS DDoS Attack Types and Defense :: Massive A type Query Attack
1.2.3.4.59873 > 10.10.1.2.53: 53495+ A? www.example.com. (44)
2.3.4.5.46922 > 10.10.1.2.53: 20009+ A? www.example.com. (44)
3.4.5.6.59873 > 10.10.1.2.53: 33495+ A? www.example.com. (44)
4.5.6.7.46922 > 10.10.1.2.53: 40009+ A? www.example.com. (44)
............................?
-. A Kind of QPS attack
-. Direct attack from Zombies
-. If source ip is not spoofed, we can filter rate-limited based policy
-. How to filter ? If source ip is randomly changed? and if the packet is exactly same with normal query traffic?
Victim :: 1.1.1.1
www.example.com IP Address?
How to differentiate attack or normal query?
Page 12
DNS DDoS Attack Types and Defense :: other Query type Attack
$ dig anonsc.com any
anonsc.com. IN A 123.45.67.59
anonsc.com. IN A 123.45.67.60
anonsc.com. IN A 123.45.67.61
anonsc.com. IN A 123.45.67.62
……………….
;; MSG SIZE rcvd: 3271
Ex: Direct attack case
Ex: Amplification attack case
Page 13
tcpdump example when ANY query $ tcpdump -X port 53 -n (or tshark port 53 –n –x)
19:38:14.172255 IP 114.xx.xx.xx.60249 > 61.110.xxx.xxx.domain: 22765+ ANY? cdnetworks.co.kr. (34) 0x0000: 4500 003e 0000 4000 3311 9310 726f 3e14 E..>[email protected]>. 0x0010: 3d6e c6ad eb59 0035 002a fb7f 58ed 0100 =n...Y.5.*..X... 0x0020: 0001 0000 0000 0000 0a63 646e 6574 776f .........cdnetwo 0x0030: 726b 7302 636f 026b 7200 00ff 0001 rks.co.kr.....
[0001] means A record type
[0001] means IN
TYPE HEX
A 00010001
ANY 00ff0001
MX 000f0001
NS 00020001
PTR 000c0001
SOA 00060001
AAAA 001c0001
TXT 00100001
HINFO 000d0001
$iptables -A INPUT -p udp --dport 53 -m string --algo bm --hex-string '|00FF0001|' -m recent --set --name dnsany
$iptables -A INPUT -p udp --dport 53 -m string --algo bm --hex-string '|00FF0001|' -m recent --name dnsany --rcheck --seconds 60 --hitcount 5 -j DROP
DNS DDoS Attack Types and Defense :: other Query type Attack
Default # of ipt_recent is 100, so need to maxmize the value in advance $ rmmod ipt_recent modprobe $ ipt_recent ip_list_tot=4095
Page 14
Massive spoofed query as if source ip is Victim Source:: 1.1.1.1:53 or 1024: / Dst :: open Resolver:53
Massive response from open resolver Source:: Open_Resolver:53 / dst :: 1.1.1.1:53(or 1024~)
command
Distributed reflective, amplified attack
Prepare big size response packet in advance
$ dig re.vr.lt txt 60byte
;; MSG SIZE rcvd: 4000(byte)
x ~70 times Victim :: 1.1.1.1
re.vr.lt DNS server :: 2.2.2.2
Page 15
Distributed reflective, amplified attack
Victim IP
Resolver DNS
Packet generating from Zombie PC
Packet from Victim Side
Resolver DNS
Page 16
Distributed reflective, amplified attack
IP (tos 0x0, ttl 49, id 25778, offset 0, flags [DF], proto: UDP (17), length: 65) 96.31.66.143.80 > 61.110.198.173.53: [udp sum ok] 59637+ [1au] TXT? re.vr.lt. ar: . OPT UDPsize=4096 (37)
IP (tos 0x0, ttl 64, id 34118, offset 0, flags [+], proto: UDP (17), length: 1500) 61.110.198.173.53 > 96.31.66.143.80: 59637 q: TXT? re.vr.lt. 1/4/1 re.vr.lt. TXT[|domain]
IP (tos 0x0, ttl 64, id 34118, offset 1480, flags [+], proto: UDP (17), length: 1500) 61.110.198.173 > 96.31.66.143: udp
IP (tos 0x0, ttl 64, id 34118, offset 2960, flags [none], proto: UDP (17), length: 1039) 61.110.198.173 > 96.31.66.143: udp
# dig re.vr.lt txt +bufsize=4096
;; MSG SIZE rcvd: 3878
Page 17
Distributed reflective, amplified attack
# tcpdump -w dns.pcap -nn host 96.31.66.143
Only 1st response is DNS and the rests are Fragmented UDP packets
EDNS0 사용(Extension Mechanism for DNS) :: rfc2671
DNS 요청자는 RFC 2671에 정의된 EDNS0(DNS 확장 메커니즘)을 사용하여 UDP 패킷의 크기를 알리고 UDP 패킷 크기의 원래 DNS 제한(RFC 1035)인 512(8진수)보다 큰 패킷 전송을 이용할 수 있습니다. DNS 서버는 UDP 전송 계층에서 요청을 받으면 OPT RR(리소스 레코드)에서 요청자의 UDP 패킷 크기를 확인하고 요청자가 지정한 최대 UDP 패킷 크기에 허용되는 만큼 리소스 레코드가 포함되도록 응답의 크기를 조절합니다.
$ man dig
+bufsize=B
Set the UDP message buffer size advertised using EDNS0 to B bytes. The maximum and minimum sizes of this buffer are 65535 and 0 respectively. Values outside this range are rounded up or down appropriately.
Page 18
Distributed reflective, amplified attack
http://dns.measurement-factory.com/surveys/openresolvers/ASN-reports/latest.html
http://www.chaz6.com/files/resolv.conf :: list of public ipv4/ipv6 dns cache servers
152,600 x Open Resolver !!!
http://openresolverproject.org/breakdown.cgi
2013-09-01 results 27,166,819 gave the correct answer to the A? for the DNS name queried
152,600 x 4,000 byte x 8(bit) = 4.8Gbps??
http://openresolverproject.org/searchby-asn.cgi?search?asn=XXXX ASN
Assumption :: if one zombie can query 152,600 open resolver in a second if one open resolver can generate 4,000 byte answer then, one DNS query can be 4.8Gbps traffic
Page 19
How to do at each Backbone/Access level?
20Gbps
40Gbps
Access Control List How to Filter? BackBone NET Level Access NET Level
Filtering big size UDP packet against the DNS server
Access Control based on Source Port and Destination Port Src:53 / Dst:53 ??
Access Control Filtering for Fragmented packets
Source IP Validation
SRC based Ratelimit
Signature based Filtering
Auth. DNS Server(ns1/ns2..)
Page 20
Signature Based Filtering against Amplification
How to filter if we get massive response packets , i,e. amplification attack
According to below image, we can see that QUESTION means 00010000 which means Questions :1, ANSWER:0
$ iptables -A INPUT -p udp --dport 53 -m string --algo bm --from 31 --to 32 --hex-string ! '|00010000|' -j DROP
# iptables -m string -h
string match options:
--from Offset to start searching from
--to Offset to stop searching
--algo Algorithm
[!] --string string Match a string in a packet
[!] --hex-string string Match a hex string in a packet
Page 21
Massive QUERY for $RANDOM.domain.com :: Non-Existent host
Objective :: The DNS server spends its time searching for something that doesn't exist instead of serving legitimate requests.
The result is that the cache on the DNS server gets filled with bad requests, and clients can't find the servers they are looking for.
• source IP based rate-limit if the source ip is not spoofed
• query type(ANY,TXT,CNAME,etc) based rate-limit or filtering
• it maybe problem if standard A query type with spoofed random source IP
NXDOMAIN query attack
Nov 21 09:09:58 s332-kt9-sel named[4942]: client 170.160.126.199#1234: query (cache) 'www.ceyxyl.biz/A/IN‘ Nov 21 09:09:58 s332-kt9-sel named[4942]: client 172.105.101.71#1234: query (cache) 'www.tcgexy.org/A/IN' Nov 21 09:09:58 s332-kt9-sel named[4942]: client 177.112.102.240#1234: query (cache) 'www.etueqt.org/A/IN' Nov 21 09:09:58 s332-kt9-sel named[4942]: client 59.34.42.184#1234: query (cache) 'www.nisyjr.com/A/IN' Nov 21 09:09:58 s332-kt9-sel named[4942]: client 93.3.157.3#1234: query (cache) 'www.inrxpx.biz/A/IN'
Page 22
http://www.redbarn.org/dns/ratelimits :: Default function since centos 6.x since 2013.
http://ss.vix.su/~vjs/rl-arm.html
rate-limit {
[ responses-per-second number ; ]
[ referrals-per-second number ; ]
[ nodata-per-second number ; ]
[ nxdomains-per-second number ; ]
[ errors-per-second number ; ] // SERVFAIL, FORMERR excluding nxdomains
[ all-per-second number ; ] // normally at least 4~5 times bigger than other value
[ window number ; ]
[ log-only yes_or_no ; ]
[ qps-scale number ; ] // responses-per-second, errors-per-second, nxdomains-per-second ,all-per-second values are reduced by the ratio
[ ipv4-prefix-length number ; ] // default Is /24, need to change /32
[ slip number ; ]
}
e,g. qps-scale 250; responses-per-second 20; and a total query rate of 1000 queries/second for all queries from all DNS clients including via TCP,
then the effective responses/second limit changes to (250/1000)*20 or 5. Responses sent via TCP are not limited but are counted to compute the query per second rate.
RRL(Response Rate Limiting) defense
Page 23
--- named.conf --- rate-limit { nxdomains-per-second 1; ipv4-prefix-length 32; slip 2; };
RRL(Response Rate Limiting) defense
1 CLIENT -> SERVER DNS Standard query A 0.809928333227621.example.com 2 SERVER -> CLIENT DNS Standard query response, No such name 3 CLIENT -> SERVER DNS Standard query A 0.990417249591218.example.com 4 SERVER -> CLIENT DNS Standard query response 5 CLIENT -> SERVER TCP 41702 > 53 [SYN] Seq=0 Win=5840 Len=0 MSS=1460 TSV=3124747279 TSER=0 WS=7 6 SERVER -> CLIENT TCP 53 > 41702 [SYN, ACK] Seq=0 Ack=1 Win=14480 Len=0 MSS=1460 TSV=3737413786 TSER=3124747279 WS=6 7 CLIENT -> SERVER TCP 41702 > 53 [ACK] Seq=1 Ack=1 Win=5888 Len=0 TSV=3124747282 TSER=3737413786 8 CLIENT -> SERVER DNS Standard query A 0.990417249591218.example.com 9 SERVER -> CLIENT TCP 53 > 41702 [ACK] Seq=1 Ack=50 Win=14528 Len=0 TSV=3737413788 TSER=3124747282 10 SERVER -> CLIENT DNS Standard query response, No such name 11 CLIENT -> SERVER TCP 41702 > 53 [ACK] Seq=50 Ack=110 Win=5888 Len=0 TSV=3124747284 TSER=3737413788 12 CLIENT -> SERVER TCP 41702 > 53 [FIN, ACK] Seq=50 Ack=110 Win=5888 Len=0 TSV=3124747284 TSER=3737413788 13 SERVER -> CLIENT TCP 53 > 41702 [FIN, ACK] Seq=110 Ack=51 Win=14528 Len=0 TSV=3737413791 TSER=3124747284 14 CLIENT -> SERVER TCP 41702 > 53 [ACK] Seq=51 Ack=111 Win=5888 Len=0 TSV=3124747287 TSER=3737413791
3 way handshake
4 way handshake
Truncated: Message is truncated
Jul 1 14:11:10 SERVER named-sdb[15282]: limit responses to xx.xx.xx.xx/32 for xxxx.com IN A (00014672)
http://www.circleid.com/posts/20130820_a_question_of_dns_protocols/
17% of visible resolvers do not successfully followup with a TCP connection following the reception of a truncated UDP response. Also performance issue.
Page 24
Anycast based Distribution
http://www.root-servers.org/ j.root-servers.net(192.58.128.30)
Google case:8.8.8.8
On the Internet, anycast is usually implemented by using
BGP to simultaneously announce the same destination IP
address range from many different places on the Internet.
This results in packets addressed to destination addresses
in this range being routed to the "nearest" point on the net
announcing the given destination IP address.
excerpted from Wikipedia.org
Page 25
Conclusion :: Technical Requirements for DNS DEFENSE
Tolerant against Massive QPS attack (~Mqps)
pass only valid dns packet
rate-limit per query type
ip rate-limit based on source ip or query type,etc
filter bad flag combinations
filter multiple request type in a packet
filter based on packet size(length,range)
Source IP validation
Using Multiple DNS provider
No solution against the Massive standard Query with randomly spoofed IP Address