1 Security Busters: Web Browser security vs. rogue sites Nikos VIRVILIS 1 , Alexios MYLONAS 1, 2 , Nikolaos TSALIS 1 , Dimitris GRITZALIS 1 1 Information Security & Critical Infrastructure Protection Laboratory Dept. of Informatics, Athens University of Economics & Business 76 Patission Ave., Athens, GR-10434, Greece 2 Faculty of Computing, Engineering and Sciences, Staffordshire University Beaconside, Stafford, ST18 0AD, United Kingdom {nvir, amylonas, ntsalis, dgrit}@aueb.gr, [email protected]Abstract: URL blacklists are used by the majority of modern web browsers as a means to protect users from rogue web sites, i.e. those serving malware and/or hosting phishing scams. There is a plethora of URL blacklists/reputation services, out of which Google’s Safe Browsing and Microsoft’s Smart- Screen stand out as the two most commonly used ones. Frequently, such lists are the only safeguard web browsers implement against such threats. Inevitably, as with any blacklist implementation, there is a time window between the creation of a rogue web site and the time it gets submitted to the blacklist(s), in which users are not protected. In this paper, we examine the level of protection that is offered by popular web browsers on iOS, Android and desktop (Windows) platforms, against a large set of phishing and malware URL. The results reveal that most browsers – especially those for mobile devices - offer limited protection against such threats. As a result, we propose and evaluate an architecture, which can be used to significantly improve the level of protection offered to the users, regardless of the web browser or platform they are using. Keywords: Android, iOS, Malware, Mobile Device, Phishing, Smartphone, Security, Windows, Web Browser. 1. INTRODUCTION Nowadays, people spend more than 60 hours per week accessing online content, either through their mobile devices or personal computers [1]. The use of web applications and services is continuou- sly increasing, with smartphones now been the prime way users are accessing the web [2]. However, while users browse the web they might visit rogue web sites, namely sites that serve malicious code (malware) and/or host phishing scams. Phishing can be deemed as one of the most popular and profitable attacks, having almost 450,000 attacks in 2013 and estimated losses of over 5.9B$. NIST defines phishing [3] as: “Phishing refers to use of deceptive computer-based means to trick individuals into disclosing sensitive personal information. Phishing attacks aid criminals in a wide range of illegal activities, including identity theft and fraud. They can also be used to install malware and attacker tools on a user’s system.” Malware attacks are also on the rise, with Kasperky Labs announcing that over 3 billion malware attacks were detected in 2013, with a total of 1.8 million malicious or potentially unwanted programs
Transcript
1
Security Busters: Web Browser security vs. rogue sites
Abstract: URL blacklists are used by the majority of modern web browsers as a means to protect users from rogue web sites, i.e. those serving malware and/or hosting phishing scams. There is a plethora of URL blacklists/reputation services, out of which Google’s Safe Browsing and Microsoft’s Smart-Screen stand out as the two most commonly used ones. Frequently, such lists are the only safeguard web browsers implement against such threats. Inevitably, as with any blacklist implementation, there is a time window between the creation of a rogue web site and the time it gets submitted to the blacklist(s), in which users are not protected. In this paper, we examine the level of protection that is offered by popular web browsers on iOS, Android and desktop (Windows) platforms, against a large set of phishing and malware URL. The results reveal that most browsers – especially those for mobile devices - offer limited protection against such threats. As a result, we propose and evaluate an architecture, which can be used to significantly improve the level of protection offered to the users, regardless of the web browser or platform they are using.
Keywords: Android, iOS, Malware, Mobile Device, Phishing, Smartphone, Security, Windows, Web Browser.
1. INTRODUCTION
Nowadays, people spend more than 60 hours per week accessing online content, either through
their mobile devices or personal computers [1]. The use of web applications and services is continuou-
sly increasing, with smartphones now been the prime way users are accessing the web [2]. However, while users browse the web they might visit rogue web sites, namely sites that serve malicious code
(malware) and/or host phishing scams.
Phishing can be deemed as one of the most popular and profitable attacks, having almost 450,000
attacks in 2013 and estimated losses of over 5.9B$. NIST defines phishing [3] as: “Phishing refers to use of deceptive computer-based means to trick individuals into disclosing sensitive personal
information. Phishing attacks aid criminals in a wide range of illegal activities, including identity theft
and fraud. They can also be used to install malware and attacker tools on a user’s system.” Malware attacks are also on the rise, with Kasperky Labs announcing that over 3 billion malware
attacks were detected in 2013, with a total of 1.8 million malicious or potentially unwanted programs
2
used in these attacks [4]. Nefarious web sites (e.g. adult websites, ones hosting pirated software, gamb-ling, etc.), are not the only ones that can expose users to such threats. Multiple benign sites (e.g. social
media websites, search engines, news sites, etc.) have also been used to deliver such attacks, after been
compromised (i.e., watering hole attacks) [5]. As a result, the likelihood that users will be exposed to
such threats, is considerable – if not high. Normal users, i.e. not technically and security savvy ones, rely on web browsers (or ‘browsers’)
to protect them from web sites (or ‘sites’) that serve malware (hereinafter referred as ‘malicious sites’)
or phishing attacks (hereinafter referred as ‘phishing sites’). In this paper, we evaluate the level of protection offered to normal users by the most popular web browsers on the desktop and smartphone
platforms, by accessing 1400 phishing and 1400 malicious URLs for a period of one month. Our work
focuses on Android and iOS for the smarpthone platform, and Windows for the desktop platform. Our
results reveal that most smartphone browsers offer very limited (if any) protection against phishing and malicious sites. On the other hand, most desktop browsers offer an acceptable level of protection
against phishing sites, but not against malicious sites. In this context, we have proposed and implement-
ed a secure proxy, that aims to raise the level of protection that is offered to normal users, regardless of the web browser or platform they are using.
In brief, this paper makes the following contributions: It provides a comparison of the phishing and malware protection offered by the popular desktop
browsers on Windows and mobile browsers on Android and iOS.
It highlights the limited efficacy – and in specific cases the total lack – of protection mechanisms
on mobile browsers, which has a significantly impact on user security. For the phishing tests, it provides a comparison of the browser detection rate with the results of
our previous work [6] and evaluation of how browser detection has changed in the meantime.
It provides an implementation of a secure proxy, based on the aggregation of multiple blacklists and AntiVirus (AV) engines.
It provides an evaluation of the secure proxy, demonstrating that it can offer an adequate level of
enhanced protection to normal users against rogue sites.
The remainder of the paper is structured as follows: Section 2 presents related work. Section 3
describes our research methodology. Section 4 presents our test results. In Section 5 we present and
evaluate the secure proxy. The paper concludes with an analysis and a detailed discussion of our find-
ings in Section 6.
2. BACKGROUND
2.1 Phishing
The main defense against phishing/malware attacks is based on lists (i.e. 'blacklists'). Lists are used
by browsers to identify if a requested URL has been reported as malicious. One of the most popular blacklists is Google’s Safe Browsing [7], which protects users from both phishing and malicious web
sites. Safe Browsing is currently used by Google Chrome, Mozilla Firefox and Apple Safari browsers.
Internet Explorer is using Microsoft’s proprietary blacklist, called SmartScreen [8]. Other browsers
also use their own proprietary lists and/or aggregate information from third parties. For instance, Opera uses a combination of blacklists from Netcraft [9] and PhishTank [10], as well as a malware blacklist
from TRUSTe [11].
Although almost every blacklist’s implementation is different, all of them do follow a basic concept, i.e., when the browser loads a URL, a URL check occurs against a local or remote database. If the URL
is listed in the database (i.e., it has been reported), a warning is raised to the user, advising her to stop browsing (Figure 1). Limited information is available on how these blacklists get updated and main-
tained, as this may enable attackers to bypass them more easily. However, a considerable part of the
submissions to blacklists are performed manually by users [10].
Figure 1 - Phishing warnings (screenshots)
A number of approaches has been proposed by the research community in an effort to protect users from phishing attacks. The research varies from surveys, regarding user awareness, to experiments of
the effectiveness of current security mechanisms and proposals of novel ones. More specifically, the
work in [12], [13] and [14] focuses on phishing with regards to its properties, characteristics, attack types, and available counter-measures. The work in [14] and [15] presents a survey on user training
methods, as well as their effectiveness against phishing attacks (as user participation plays a major role
in phishing protection).
Literature has also focused on the use of visual indicators to protect users from phishing. In [16] an overview of the warning indicators, as well as its advances over the last decade, is presented. Also,
[17] has surveyed users’ interaction regarding security indicators in web browsers. A study on the
effectiveness of browser security warnings was published in [18], focusing on the Google Chrome and Mozilla Firefox browsers. The authors collected over 25M user reactions with phishing and malware
security warnings, measuring the user reactions to these warnings. A similar study in [19] analyzed the
impact on the users' decision based on the choice of background color in the warning and the text
descriptions that were presented to them. In [20], the authors focused on the effectiveness of phishing blacklists, and in particular on their
update speed and coverage. The authors used 191 phishing sites that had been active for 30 min or less,
4
and compared 8 anti-phishing toolbars. Less than 20% of the phishing sites were detected at the begin-ning of the test. Similarly, in [21] the authors proposed the use of 'Anti-Phish', a browser extension for
the Mozilla Firefox browser that detects web site-based phishing attacks.
A Novel-Bayesian classification, based on textual and visual content was proposed in [22], where
the authors used a text classifier, an image classifier, and a fusion algorithm in order to defend against known properties of phishing attacks. The authors of [13] provide a methodology that aims to distin-
guish malicious and benign web pages, based on layout similarity between malicious and benign web
pages. In [23], the authors analyzed 300 phishing URLs and measured the effectiveness of desktop
browsers in detecting them. Opera browser offered the highest level of protection, by blocking 94.2%
of the phishing sites. In [24], authors tested the effectiveness of anti-phishing add-ons for Internet
Explorer, Google Chrome and Mozilla Firefox. In their evaluation Google Chrome outscored the other browsers. Finally, [11] tested popular desktop web browsers (i.e., Firefox, Chrome, Opera, IE, Safari),
focusing on the time required for browsers to block a malicious site (i.e., from the generation of the
malicious domain until the time this was blocked by the browsers). The initial results (day 1) ranged from 73.3% (IE) to 93.4% (Safari), while the final results (day 7) varied from 89.3% (IE) to 96.6%
(Firefox).
A number of anti-phishing mechanisms has been proposed for smartphones: In [25], the authors investigate the viability of QR-code-initiated phishing attacks (i.e. QRishing) by conducting two sepa-
rate experiments. A similar approach was presented in [26], where the authors worked on how notifi-
cation customization may allow an installed Trojan application to launch phishing attacks or anony-
mously post spam messages. Finally, previous work on browser security [27] revealed that security controls, which are typi-
cally found on desktop browsers, are not provided in their smartphone counterparts. Moreover, the
analysis revealed that the implementation of the security controls (among them the security control against phishing attacks) was not hindered by restrictions from the security architecture of the
smartphone platform (i.e. the application sandbox). This shows that level of security that is offered by
smartphone browsers can be enhanced to protect their users, as the latter are exposed to the same web threats as when they are using a desktop web browser.
Related literature regarding the effectiveness of anti-phishing mechanisms on the smartphone
platform is limited. Our work in [6] revealed significant differences in the effectiveness of blacklist of
Android, iOS, and desktop browsers against phishing sites.
2.2 Malware
Although the majority of browsers are based solely on the use of blacklists for detecting malicious
URLs, Internet Explorer and Google Chrome on Windows perform additional steps, in an effort to
detect malicious downloads. IE analysis is twofold: Firstly, it checks if the URL in included in its SmartScreen blacklist. In
such a case the URL will be blocked and a warning will be raised. If the URL is not included in the list
and a download is triggered, then IE will perform a number of additional steps in order to detect po-tentially malicious files. To this end, IE facilitates the Application Reputation feature [30]. The appli-
cation reputation service warns users if a downloaded file is not trusted, based on: (a) the existence of
a digital signature (verified software publisher) and (b) the number of users that have already down-loaded the file. Although such process inevitably creates false positives for non-popular downloads
that lack a trusted digital signature, it also provides a level of protection against malware infection.
Figure 2 - Malware warnings (screenshots)
Figure 3 - Internet Explorer's warning message for a malicious download
Google Chrome’s “Download Protection” feature [28] also supports protection against malicious
downloads. Chrome first checks if the URL that triggered the download, which can be different from the one the user has visited (due to redirections), is a flagged as malicious (i.e. is already included in
the Safe Browsing list). If the URL is not blacklisted, Chrome checks if the downloaded file, based on
its hash, is included in a whitelist of trusted files. If both checks fail, Chrome sends to Google servers
for further analysis the following data: a) the URL that triggered the download and the IP address of the hosting server, b) the referrer URL, (c) the size of the downloaded file and its SHA-256 hash, and
d) the file’s digital signature (if it is available). Based on this data, the Chrome will receive three
possible responses: benign, unknown, or malicious. Depending on this response, Chrome may issue a warning to the user (Figure 4).
Figure 4 - Google Chrome's warning message for a malicious download
A number of mechanisms to mitigate web-driven malware infections has been proposed by the
research community. In specific, Content-Agnostic Malware Protection (CAMP) has been proposed as
a means for detecting malware. It is currently used in Google Chrome browser [29]. Microsoft has implemented a similar reputation based system since Internet Explorer 9 [8], [30].
Furthermore, non-browser specific models have been proposed for the detection of malicious
domains through DNS monitoring [31], while [32], discusses large-scale, passive DNS analysis
techniques to detect domains that are involved in malicious activity.
6
A zero-day anti-malware solution is proposed in [33], which uses a combination of whitelists and blacklisting. The authors discuss that such whitelists do not need signature updates, and provide
protection against sophisticated zero-day malware attacks, by enforcing software restriction policies,
which allow only legitimate executables, processes and applications to be executed.
In addition, a number of models have been suggested in the literature, focusing on malware detec-tion, namely: (a) the AMICO project [34], which detects malware downloads in live web traffic using
statistical analysis to identify characteristics of malware distribution campaigns, (b) the ZOZZLE [35],
which detects and prevents JavaScript malware from been deployed in the browser, by using a Bayesian classifier, and (c) the EFFORT system [36], which improves the efficiency and effectiveness of
existing approaches regarding the detection of malware serving bots.
Last but not least, multiple models have been proposed that rely on machine learning techniques:
(a) for malware detection [37], [38], [31], [39] and (b) for detection of drive-by downloads [40], [41], [42], [43], [44], [45].
The industry offers a large number of content filtering solutions, ranging from software based
solutions, to Cloud services and hardware appliances. Multiple software solutions exist, such as McAfee’s Site Advisor and Symantec’s Safe Web [46], [47]. They are usually offered for free or at
low cost. However, they require the installation of third-party software/browser extensions and are
browser and platform dependent with limited support for mobile platforms. Commercial content filtering appliances and Cloud Services, e.g. OpenDNS [48], are popular in enterprises and are usually
platform and browser agnostic. However, their effectiveness is hard to measure due to the use of
proprietary technologies. They also come at a significant cost, which limits their use only to business
users. Finally, there are a number of online services for analyzing suspicious executables. One of the
most popular is VirusTotal [49], which utilizes a large number of popular antivirus engines for analy-
zing suspicious files (c.f. Table 19 in the Appendix for a list of AV engines). Users can upload files for analysis, or query the service for files that have already been analyzed by searching for the hash of a
file. VirusTotal also supports URL scans. Upon submission of a URL, the service will query a large
number of blacklists (c.f. Table 20 in the Appendix) and report if there are any matches.
3. METHODOLOGY
3.1 Test environment
To evaluate the protection that desktop and smartphone browsers offer to normal users, i.e. non
security and technically savvy ones, we accessed 1400 phishing and 1400 malicious URLs and marked
whether or not the browsers warned us about the risk of our action. As technology that can be used to fully automate this process is not currently available, a security savvy user was required to verify if a
web page that had not been reported by the browser was indeed a malicious or a benign web page.
As this was a cumbersome task, we set up the architecture presented in Figure 5. The URLs that were used in our evaluation were stored in the URL Collection. The URL Container parsed daily the
URL Collection and selected the URLs that had been reported in the last 24 hours. Then, two HTML
files were created for each threat (namely, one for phishing URLs and one for malicious URLs), which
were formatted as Snippet 1. Finally, a researcher used each browser in the test devices to access each URL in the HTML files and capture the results. The test devices included: (a) an iPhone 5S, with iOS
v7.1.1 and a Sony Xperia Tipo with Android v4.0.4, for the smartphone platform, as well as (b) a
Our experiments included the most popular browsers in the desktop and smartphone platforms
(c.f. Appendix, Tables 11-12). More specifically, we evaluated the following desktop browsers: (a)
Internet Explorer 11, (b) Chrome v35, (c) Firefox v29, and (d) Opera v22. Our effort in the smartphone
platform focused on iOS and Android, as these are the two most popular operating systems, comprising
almost 90% of the global smartphone market share [50]. Similarly, the most popular browsers for each mobile platform were selected, namely: (a) Safari
Mobile (built-in on iOS 7.1.1), (b) Chrome Mobile v35, (c) Opera Mini v7.0, “Browser” or “Internet”
(i.e. the default browser for Android 4.0.4), (d) Firefox Mobile v30, and (e) Opera Mobile v22. Al-though some of the desktop browsers have mobile counterparts, their availability in the two smartphone
platforms is heterogeneous, as shown in Table 1.
Table 1 – Browser availability on tested platforms
iOS 7.1.1 Android 4.0.4 Windows 7
Safari Mobile X
Chrome Mobile X X
Opera Mini X X
Browser† X
8
Firefox Mobile X
Opera Mobile X
Chrome X
Firefox X
Internet Explorer X
Opera X
† 'Browser' (or Internet in newer versions) is the pre-installed browser in Android
3.2 Phishing Tests
To evaluate the protection offered by the above mentioned web browsers against phishing attacks,
we collected phishing URLs that were reported by PhishTank [10]. PhishTank was selected as it is a popular online service that lists phishing attacks that are submitted and verified by an active commu-
nity. PhishTank publishes daily a list of verified phishing URLs, i.e., ones that have been confirmed as
fraudulent and online. However, as the state of a phishing URL is dynamic, a confirmed URL might
be cleaned or be taken down shortly after its submission. Our tests were conducted from June to July 2014 and included manual browsing to 1400 phishing
URLs. Although some parts of the evaluation could have been automated (e.g. when the request re-
turned an HTTP Error Code or the browser raised a warning), it was considered necessary to manually verify the existence of false negatives. In specific, we manually examined whether a URL that was not
blocked by the browser and if it was either (a) a legitimate or not responsive/non accessible site (i.e.
not an active phishing site) or (b) actual false negative, i.e., an active phishing site. In this context, each URL in our collection was manually categorized into one of the following categories:
a. Blacklisted: The URL was blocked by the web browser, i.e. the user received a warning indicating
a known phishing site.
b. False Negative: A phishing URL that was manually verified as fraudulent, but was not in the browser’s blacklist, i.e. the browser generated no warning.
c. Non-Phishing/Timeout/Error: The URL had been suspended/taken down/cleaned when we
accessed it.
It should be noted that our data set included only verified phishing URLs (i.e. a human operator from
PhishTank has manually verified them as phishing), as our main objective was to test the efficacy of
browser blacklists in blocking such attacks. As all URLs were actual phishing pages at the time of our analysis, the false positive rate of all blacklists was zero and thus out of the scope of this work.
Finally, we compared the results of this analysis with our previous work in [8], where we evaluated
the protection that is offered by the same browsers by collecting phishing URLs from PhishTank in the
period from January to March 2014.
3.3 Malware Tests
An online service, which reports malware hosting websites and is comparable to PhishTank (i.e.
well-known service for reporting verified URLs on a daily basis with a strong community support), is
not currently commercially or openly available. Therefore, and in order to gather our URL collection for this test, we used the open source “Collective Intelligence Framework” (CIF) [51]. CIF allows the
collection and analysis of malicious threat information from a large number of trusted sources (c.f.
Appendix for a list of these sources), which can be used for incident response, and intrusion detection and mitigation.
Similarly to the previous experiment, our tests included manual browsing to a total of 1400 URLs that hosted malicious software. For the scope of our analysis, we categorized every URL in one of the
following categories:
a. Blacklisted: The browser blocked either the URL or the file that was downloaded (or issued a
warning that the file could be potentially dangerous). b. False Negative: A URL that was not blocked by the browser and triggered the download of a
potentially malicious file, without any further alert being raised by the browser (e.g. due to reputation-
based analysis of the downloaded file). c. Non-Phishing/Timeout/Error: The URL had either been cleaned or suspended/taken down. This
category also included the URLs that did not trigger a download.
Similarly, in order to test the efficacy of browser blacklists in blocking such attacks, our data set included only verified malicious URLs (at the time of our analysis). As a result, the false positive rate
of all blacklists was zero and thus out of the scope of this work.
4. EXPERIMENTAL RESULTS
This section presents our experimental results. For each platform we list the browsers that offer protection against (a) phishing sites and (b) sites hosting malicious software (hereinafter referred as
‘malicious sites’) and analyze their success rate in blocking them. For the first evaluation we re-exa-
mine the protection that the popular browsers offer in the desktop and smartphone platform and com-pare it with our previous results in [6].
4.1 Protection against phishing
4.1.1 iOS browsers
In iOS, Mobile Safari, i.e., the pre-installed web browser, uses Google’s Safe Browsing to provide
anti-phishing protection. Our evaluation revealed that the implementation of this anti-phishing control suffers from a significant design weakness, as the Safe Browsing blacklist is only updated when the
iOS device is synchronized with iTunes. Considering that some iOS users may not synchronize their
devices frequently, they may end up with an outdated blacklist. Furthermore, the list is updated only
once per day. Thus, any phishing site that has been created in the meantime - even if it has been reported to Safe Browsing list - will not be blocked. As a result, iOS users receive considerably limited protec-
tion against phishing attacks. This holds true, since our results revealed that without performing the
synchronization step, Safari Mobile did not block any of the URL in our collection. In this context, we synced the iOS test device right before starting the evaluation of Mobile Safari.
Chrome Mobile on iOS offers phishing protection since January 2014. However, this option is
not enabled by default, but requires the user to enable the “Reduce Data Usage” option, which uses Google’s servers to fetch the requested URL. The contents of the web page are downloaded and com-
pressed and the URL is checked against the Safe Browsing list. This feature is privacy intrusive (as all
traffic is transferred through a third party), does not work for SSL/TLS pages or in Incognito mode
(private browsing), and is not enabled by default. We have excluded it from our evaluation as: a) we regard that it is less likely that normal users (i.e. not security and savvy ones) will enable security
controls, as smartphone users tend to be oblivious about their security and privacy [52] and (b) the
control is not ‘easily configurable’ [53], i.e., the label of the control is not intuitive, thus confusing
10
even security savvy users that control focuses on performance and not security. Finally, Opera Mini did not support phishing protection.
Table 2 - Support of Phishing Protection in iOS
URL
iOS Browser
Black-listed
False negatives
Non-phishing
Safari Mobile 542 370 488
Chrome Mobile N/A N/A N/A
Opera Mini N/A N/A N/A
N/A: Browser does not support anti-phishing mechanisms
4.1.2 Android browsers
Android users also receive limited protection against phishing attacks, as the default Android browser
(commonly known its users as “Browser” or “Internet” in newer versions of Android) supports no phishing protection. The same holds true for the Chrome Mobile and Opera Mini browsers. It is to be
noted that these are the most popular browsers on Android, according to the number of downloads on
Google Play (c.f. Table 11 in the appendix). On the other hand, Firefox Mobile or Opera Mobile do implement phishing countermeasures.
Moreover, our results suggest that both of them perform very well, achieving similar results with their
desktop counterparts. Specifically, Firefox and Opera on Windows have blocked 86.7% and 77.9% of
the phishing URLs, while Firefox Mobile and Opera Mobile blocked 85.4% and 75.9%, respectively (c.f. Fig. 6) Nevertheless, if one considers that: (a) not all users feel the need and/or are capable to
install a third-party browser on their devices [52] and (b) the pre-installed browser offers no protection,
then a very large number of Android users is not adequately protected from phishing attacks.
Table 3 - Support of Phishing Protection in Android
URL
Android Browser
Black-
listed False
negatives Non-
phishing
Firefox Mobile 1196 48 156
Opera Mobile 1062 110 228
Chrome Mobile N/A N/A N/A
Opera Mini N/A N/A N/A
Android Browser† N/A N/A N/A † 'Browser' (or Internet in newer versions) is the pre-installed browser in Android
N/A: Browser does not support anti-phishing mechanisms
4.1.3 Desktop browsers
Our analysis revealed that all desktop browsers offered anti-phishing protection using either Goog-le’s Safe Browsing list (i.e. Chrome and Firefox) or their own proprietary blacklists (i.e. Opera and
Internet Explorer). Chrome blocked the most phishing URLs, followed by Firefox. Although their
results are similar - which is expected, as they use the same blacklist - Chrome outperforms Firefox, as it blocks roughly 5% more phishing sites and has a lower false negative rate.
During our experiments, we encountered another issue with the synchronization of blacklists. That
was also raised by [11] and affected Firefox. In specific, we noticed that if the browser was not executed
for a few minutes each day before we started our evaluation, then Safe Browsing blacklist would not
be updated. This resulted to a large number of false negatives. The case stems from the way the Safe Browsing protocol updates its local database [54]. Interestingly, we did not face this problem with
Chrome or any of the smartphone browsers using the Safe Browsing list. To avoid the synchronization
issue, we executed the browser for at least 10 minutes and browsed randomly on the web, so as to give
it enough time to update its blacklist. Opera ranked third by blocking roughly 10% less phishing URL than Firefox and having slightly
more false negatives. Finally, Internet Explorer offered the lowest level of protection among the desk-
top browsers, having the smallest percentage of blocked URLs (less than 50%).
Table 4 - Support of Phishing Protection in Windows 7
URL
Browser
Black-listed
False negatives
Non-phishing
Firefox 1215 83 102
Chrome 1302 18 80
Opera 1090 118 192
Internet Explorer 678 138 584
4.1.4 Overview of results for phishing protection
Table 5 summarizes the availability of anti-phishing protection per operating system and brow-
ser (as of July 2014). Our results revealed that only a subset of the browsers in smartphones offer anti-phishing protection. Thus, their users are not protected from such attacks. This is particularly true for
Android users, where the pre-installed browser does not offer anti-phishing protection and users have
to install a third-party browser. In iOS, the pre-installed browser offers anti-phishing protection but its effectiveness is questionable (c.f. Section 4.1.1). On the contrary, all desktop browsers provided some
anti-phishing protection, even though their effectiveness differed significantly.
Table 5 - Support of anti-phishing mechanisms
OS Browser name Phishing
protection†
Android
Browser†† N
Chrome Mobile N
Firefox Mobile Y
Opera Mobile Y
Opera Mini N
iOS
Chrome Mobile N
Opera Mini N
Safari Mobile Y
Windows 7
Chrome Y
Firefox Y
Internet Explorer Y
Opera Y † Y: Security control available, N: Security control unavailable †† 'Browser (or Internet in newer versions) is the pre-installed browser in Android
Figure 6 includes some of the results of our experiments. More specifically, it presents: (a) the percentage of URLs that each browser blocked, (b) the percentage of manually verified phishing URLs
that were not blocked by the browser(s) (false negatives), and (c) the percentage of URLs that were
12
not in the browser’s blacklist and were manually verified during our analysis as non-malicious sites (i.e. URL that had been cleaned, or domains that had been taken down or were not accessible when we
visited them). The browsers that did not support any anti-phishing mechanism are not included in the
chart, as their detection rate is zero.
Figure 6 - Percentage of blocked URLs (Blue), phishing URLs that were not filtered out (false negatives) (Red), URLs not
in blacklist and not hosting a phishing attack (Green) (n=1400)
4.1.5 Comparison with previous evaluation
Herein, we compare the current anti-phishing protection (i.e. June-July 2014, Q2 2014) that the
popular browsers offer in the desktop and smartphone platforms, with our previous work in [6], which
were conducted between January and March 2014 (Q1 2014). Since the phishing ‘ecosystem’ is dyna-mic, i.e. phishing sites are short-lived with an average life expectancy of 23 hours [11], our aim is to
investigate how this dynamic nature is reflected in the browsers’ anti-phishing protection over a period
of time. Our results are depicted in Figure 7 (c.f. Tables 14-16 in the Appendix for detailed results). This
figure reveals differences in the browsers’ anti-phishing protection during the two evaluation periods.
Overall, the browsers blocked fewer phishing URL in Q2, with the exception of Firefox Mobile on Android. More specifically, Safari Mobile’s (iOS) detection dropped almost by half. This stresses again
the problematic implementation of the Safe Browsing protocol in iOS. Furthermore, our analysis
showed a small drop in the performance of the desktop versions of Firefox and Opera, with respect to
the blacklisted URLs and false negatives. Finally, Internet Explorer blacklisted less URLs and was prone to more false negatives, and Opera Mobile had more false negatives.
Our experiments results revealed that none of the iOS browsers offered any protection against ma-
licious sites. In the case of Mobile Safari, that was rather unexpected, as the browser uses the Safe Browsing list for the detection of phishing URLs. However, it does not use Safe Browsing for the
detection of malicious sites. Opera Mini did not utilize any blacklist for the detection malicious sites.
Neither did so Chrome mobile (not enabled by default and excluded due to the shortcomings that we mentioned previously). As a result, iOS users are not protected when visiting malicious web sites with
any of the above mentioned web browsers.
Table 6 - Support of Malware Protection in iOS
URL
iOS Browser
Black-listed
False negatives
Non-phishing
Safari Mobile (iOS)
N/A N/A N/A
Chrome Mobile N/A N/A N/A
Opera Mini N/A N/A N/A
N/A: Browser does not support anti-malware mechanisms
4.2.2 Android browsers
Our results showed that Android users are also largely unprotected against malicious sites. More
specifically, the pre-installed web browser (“Browser” or “Internet”), Chrome Mobile and Opera Mini browsers offered no protection against malicious sites. As summarized in Table 7, the only browsers
that utilized malware blacklists were Firefox Mobile and Opera Mobile. Nonetheless, our experiments
revealed that the level of protection that they offered was very limited, as they blocked roughly 10-12% of the malicious URLs in the experiments. It is noteworthy to mention that Firefox Mobile blocked
more malicious URLs than its desktop counterpart, as it highlights the issues in the implementation of
the Safe Browsing protocol.
0%10%20%30%40%50%60%70%80%90%
100%
Q1 Blacklisted
Q1 False-negative
Q1 Non-phishing
Q2 Blacklisted
Q2 False-negative
Q2 Non-Phishing
14
Overall, our results show that Android users are widely exposed to malicious sites. This finding is more worrying if one considers the increasing number of attacks against Android and the exponential
growth of Android malware [4], [55], [56].
Table 7 - Support of Malware Protection on Android
URL
Android Browser
Black-listed
False negatives
Non-phishing
Firefox Mobile (Android) 139 641 620
Opera Mobile (Android) 166 683 551
Chrome Mobile N/A N/A N/A
Opera Mini N/A N/A N/A
Browser† N/A N/A N/A † 'Browser' (or Internet in newer versions) is the pre-installed browser in Android
N/A: Browser does not support anti-malware mechanisms
4.2.3 Desktop Browsers
The experiments suggest that the level of protection against malicious sites that is provided by the
popular desktop browsers is considerably low. More specifically, Internet Explorer (IE) blocked most of the malicious sites and was the least prone to false negatives (i.e. malicious downloads that were not
blocked). This confirms the findings from similar research conducted by the industry in [11]. Never-
theless, even though IE outperformed all other browsers, it still suffered by a considerable number of false negatives. This highlights that the application reputation that is used by IE offers an extra line of
defense against malware; however, it is far from being perfect.
Chrome had a higher number of false negatives than IE, even though it offers a similar control
against malicious downloads. Opera ranked third. Firefox offered the least protection in our experi-ments by blocking only 5% of the malicious URLs.
At the time of our experiments, Firefox and Opera did not perform any further analysis on the
downloaded files. Their decision was purely formed on the basis of the URL. Newer versions of Firefox are expected to perform analysis on the downloaded executables using the same technique as Google
Chrome [57]. This is expected to raise the level of the protection offered by the browser against
malicious sites. We believe that its performance will be similar with Chrome, as both browsers will use the same service for the classification of the downloaded files.
Table 8 - Malware Protection Statistics in Windows
URL
Browser
Black-listed
False negatives
Non-phishing
Firefox 70 729 601
Chrome 280 552 568
Opera 180 816 404
Internet Explorer 573 420 407
4.2.4 Overview of results for malicious sites
In contrast to anti-phishing protection, the results of our experiments suggest that the level of pro-
tection against malicious sites, which is provided by the popular browsers on the desktop and smart-phone platforms, is considerably low.
Our results revealed that only a subset of the browsers in iOS and Android offer any protection against malicious sites. The browsers that did not filter phishing URLs also did not filter malicious
URLs. Interestingly, Safari Mobile did not filter malicious URLs, even though it implements Safe
Browsing to offer anti-phishing protection. The results are summarized in Table 9.
Table 9 - Support of anti-phishing mechanisms
OS Browser name Malicious sites protection†
iOS
Safari Mobile N
Chrome Mobile N
Opera Mini N
Android
Browser†† N
Firefox Mobile Y
Chrome Mobile N
Opera Mobile Y
Opera Mini N
Windows 7
Firefox Y
Chrome Y
Opera Y
Internet Explorer Y † Y: Security control available, N: Security control not available †† 'Browser is the pre-installed browser in Android
The results of our analysis are depicted in Figure 8, which presents: (a) the percentage of blocked URLs per browser, (b) the percentage of URLs that were not blocked by the browsers and triggered
the download of a potentially malicious file (false negatives), and (c) the percentage of URLs that were
not filtered by the browser and were manually verified as non-malicious sites (i.e. the domain/URL
had been cleaned/taken down, or a download was not triggered). The browsers that did not provide protection against malicious sites are not included in the chart, as their detection rate is zero.
Figure 8 - Percentage of blocked URLs (Blue), malicious URLs that were not filtered out (false negatives)
(Red), URLs not in blacklist and not triggering a download (Green) (n=1400)
11,9% 9,9%
40,9%
12,9%20,0%
5,0%
48,8% 45,8%
30,0%
58,3%
39,4%
52,1%
39,4%44,3%
29,1% 28,9%
40,6% 42,9%
Opera Mobile(Android)
Firefox Mobile(Android)
IE (Windows) Opera(Windows)
Chrome(Windows)
Firefox(Windows)
Malicious URL Detection
Blocked False Negatives Non Malicious
16
5. SECURE PROXY
As it is evident from the experimentation and the analysis of the results, there are three problems
that need to be addressed in order to strengthen our defenses against of rogue sites:
a. The limited effectiveness of blacklists against malicious sites.
b. The limited effectiveness of reputation based systems, like the ones implemented on Internet Explorer and Chrome to defend against malicious downloads.
c. The unavailability of security controls in popular smartphone browsers.
In addition, the restrictions imposed by smartphone sandboxes result in the unavailability and poor
performance of security software in smartphones. Moreover, the literature has showed that the unawa-
reness of smartphone users with regards to their security and privacy [52], [53]. In this context, we propose and implement a secure proxy, as a proof-of-concept security
mechanism, which proves the efficacy of aggregating multiple data sources in the detection of rogue
sites. Currently, and due to the heterogeneity and restrictions of smartphones and their security models,
a different security control might be incompatible (e.g. as in iOS) or infeasible to be implemented due to resource restrictions (e.g. in older Android smartphones). The proposed proxy is browser and
platform agnostic (i.e. does not require the installation of third party software) and can protect the user,
regardless of the browser one is using. Finally, and in contrast with the commercial and closed source content filtering solutions, we are proposing an open architecture which can be built with a fraction of
the cost.
5.1 Architecture
We have developed a secure forward HTTP proxy, which uses VirusTotal’s public API for analyzing the requested URLs and downloaded files. We selected this service due to (a) its popularity,
(b) to the wide number of AV engines and blacklist providers that it offers, and (c) the availability of
a free API. Nevertheless, any similar service could be used in place of VirusTotal. For each requested URL, our proxy queries VirusTotal in order to identify if the URL is blacklisted
by any of the blacklist providers. If it is, then the request is blocked and a warning message is returned
to the user. If the URL is not blacklisted, then the proxy returns the HTTP response (page contents) to
the browser. If a download is triggered, the proxy calculates the SHA-256 hash of the file and queries VirusTotal. Once more, if the hash is known and is reported as malicious by any AV vendor, then the
download will be blocked and a warning will be raised. If the hash is unknown, (i.e. the file had not
been analyzed in the past), then the proxy uploads the contents of the file for analysis and allows the
user to download the file if no AV engines flag it as malicious. The steps are summarized in Figure 9.
5.2 Proxy Evaluation
The evaluation of the secure proxy is focused on the detection of malicious sites, as our results
showed that the majority of the browsers provide a weak protection against such threats. We have also
tested our proxy against the complete collection of phishing URLs. However, this test was only a
verification of the correct operation of the proxy, as PhishTank, which was the source of the phishing URLs, is one of the blacklist providers that VirusTotal is using to detect phishing attacks and, thus, the
results would be biased.
Figure 9 - Proposed Architecture
To perform our evaluation, we accessed, daily, the same list of malicious URLs that the browsers
were tested against, redirecting the requests through our proxy. We decided to use a script, which
simulated the web requests instead of using an actual browser, in order to make sure that no browser specific countermeasure would interfere with our test results, as well as to automate the process. We
kept track of the total number of blocked URLs and compared them with the results of the browser that
achieved the highest blocking rate in our evaluation. The proxy was configured to block downloaded files (either based on the hash or the actual file analysis), when the number of detections reported by
VirusTotal were one or more. This parameter is configurable. In any real-word implementation, the
proxy could be configured to block a request, according to the security policy (or risk appetite) of the
organization that was deployed to e.g. potentially having a higher blocking threshold. We also kept statistics for (a) the number of URLs that were blocked based solely on URL analysis,
(b) the number of downloads that were blocked due to hash analysis, and (c) the number of downloads
that were blocked due to the actual analysis of the files by multiple AV engines.
5.2.1 URL-only analysis
As summarized in Table 10, the aggregation of multiple blacklists achieved encouraging results. This holds true as the use of multiple blacklists outperformed Internet Explorer by 12.3%, which was
the browser that blocked most malicious sites.
18
Table 10 - Comparison of the percentage of blacklisted URLs with the percentage of browser that achieved highest results
in our evaluation (referred as best performing browser)
Rogue URLs (n=1400) Secure proxy Best Performing browser
Malicious sites 53.2% 40.9% (Internet Explorer)
5.2.2 Hash-based analysis
Our results revealed that, while the aggregation of multiple blacklists for malicious sites provides higher protection than any individual browser, it still fails to detect almost half of the malicious sites
in our collection (false negatives). Browsing to these sites (namely 46.8%, 655 malicious sites in our
collection) triggered the download of 460 unique files (based on their SHA-256 hash), all of which
were PE executables. During our evaluation the secure proxy downloaded these files and queried VirusTotal for their hashes. Our analysis revealed that 57.3% of the submitted hashes (i.e. 264 out of
460) were unknown to VirusTotal, i.e., the files had not been submitted for analysis before. The
detection rate for the rest of the files, namely known hashes (42.7% of the files), is summarized in Figure 10 (for the detailed results see Table 17 in the Appendix).
The number of the AV engines, which are executed during the analysis of a file in VirusTotal, is
not always the same (ranges between 49-54 antivirus engines). Figure 10 summarizes the results based
on the detection ratio for each file. This ratio was calculated with z/n, where z represents the number of antivirus engines that flagged the file (hash) as malicious and n is the total number of antivirus
engines that were executed. The results indicated that the detection rate of approximately half of the
malicious files was in the range of 6-38% of the antivirus engines. Moreover, only the 27.4% of the malicious files were detected by the majority of the antivirus engines (namely, a detection ratio of 75-
91%).
Figure 10 - Detection percentage (hashes).
X-axis lists the percentage of AV engines that flagged the file as malicious. Y-axis lists
the percentage of the samples.
5.2.3 File-based analysis
For the 57.3% of the file hashes that were unknown to the service, our proxy uploaded the actual files for analysis. All of the uploaded files were reported as malicious. Based on the detection reports,
the majority of them (~71.0%) were detected by 46-50% of the antivirus engines. This means that for
the majority of the malicious files, assuming that the end user has only one AV product in use, there is
roughly 50% chance of the file been detected. Furthermore, ~6% of the malicious files were in the 19-45% of the detection rate and the rest 23% fell in the 51-62%. This suggests that these files, although
not been reported based on their hash, were undoubtedly malicious and the end-user should not be
allowed to download them. These findings are summarized in Figure 11 (c.f. Table 18 in the Appendix).
Figure 11 - Detection Percentage (executables).
X-axis lists the percentage of AV engines that flagged the file as malicious. Y-axis lists
the percentage of the samples.
5.2.4 Performance evaluation
Our proxy performs three types of queries to VirusTotal, which incur delays, namely: (a) URL
query, to identify if the URL is reported as rogue by any of the blacklist providers used by VirusTotal,
(b) Hash query, to identify if the hash of the file is known to be malicious (i.e. the file had been analyzed in the past and reported as malicious), and (c) File query, where the actual file is uploaded to VirusTotal
for analysis.
Our analysis revealed that for URL queries the proxy received a response from VirusTotal and
allowed or blocked the request in 648ms (average). For each triggered download, the average query time based on the hash was 516ms (hash query). If the hash was known, there would be no further
delay. The download would be blocked if there were one or more detections or allowed, if no detections
were reported. The longest delay occurs when the hash is unknown, as the file needs to be uploaded (file query) for further analysis. The delay depends on various parameters, i.e. the size of the file, the
network speed, and the load on the VirusTotal service, with the latter been the most time consuming
part. In our evaluation, the average size of the malicious executables that we gathered was 848KB and
20
our Internet connection was a 20Mbit (2048Kbit upload) ADSL line. On average, our file queries were completed in under 41sec (time required to upload a file and get the detection report).
6. DISSCUSSION AND CONCLUSIONS
This paper evaluates the level of protection that is offered by popular web browsers on Windows,
iOS and Android platforms by visiting a total of 2800 rogue URLs (1400 phishing and 1400 malicious
URLs). We tested daily each browser, against the reported URLs of the last 24 hours and measured the effectiveness of its blacklist (i.e. number of blacklisted URLs, number of false negatives, number of
cleaned/non-accessible sites). As this process is not currently automated, the protection offered by each
browser was manually evaluated by a researcher. Inevitably, this incurs a considerable cost for this evaluation, which is reflected in the number of URLs that were tested. This introduces a potential bias
in our results, which describe the level of protection in the period that the tests took place, namely
June-July 2014.
However, we consider that our results, while not generalizable, provide adequate indications about the level of protection offered to the users. This holds true, as our findings that refer to the desktop
browsers are in accordance with the results in [11]. Moreover, the results of this work regarding the
anti-phishing protection are similar to our previous evaluation [6].
6.1 Protection of desktop and mobile browsers
Overall, our results revealed that desktop browsers performed better in comparison to their smart-
phone counterparts, both against phishing and malicious sites. This is a worrisome finding if we
consider the proliferation of smartphones, as well as the increased web browsing with these devices. One could argue that this is expected, as smartphones lack the processing capabilities of desktops and
laptops. Nonetheless, this is only partly true today, as most smartphones have similar resources as a 3-
4 year old laptop (e.g. dual core CPU, 1-2 GB or RAM, etc.). In addition, our previous work has shown that the unavailability of important security controls - such as blacklists for phishing and malicious
sites in which this paper focuses - does not stem from the (API) restrictions that are imposed from the
smartphones operating system (i.e. sandbox profile) [53]. The reason that this happens is still unclear;
however, it falls out of the scope of this work. Our work focuses on the popular desktop browsers (Windows) and their smartphone counterparts
that are available in iOS and Android. While there are other browsers that we did not examine, such as
Safari on Mac OS X and Internet Explorer Mobile on Windows Phone, we consider our results as representative. This holds true as Windows is the most popular operating system for desktops and
laptops, as well as Android and iOS users constitute the 94% of the smartphone users (78.4% and
15.6% respectively) [58]. In addition, as iPads and Android tablets use a similar operating system (iOS, Android), and in most cases the exact same browser versions, our findings are considered to reflect the
protection that is offered on a larger user base.
The results of our work suggest that the current browser security controls offer inadequate
protection against phishing and malicious sites. Although all desktop browsers support such counter-measures, their effectiveness varies significantly. Furthermore, desktop browsers tend to be more ef-
ficient in blocking phishing sites than malicious ones. The situation is worse on smartphones, where
only a subset of the available browsers offers anti-phishing and anti-malware protection, leaving their users widely exposed. More specifically, in our evaluation Chrome outscored the other desktop brow-
sers with respect to anti-phishing protection. On the other hand, Internet Explorer offered the highest
protection against malicious sites. In the evaluation of smartphone browsers, Firefox Mobile outscored
all other tested browsers in Android. Also, Safari Mobile was the only iOS browser that offered phish-ing protection.
Surprisingly, Safari Mobile did not support detection of malicious URLs. Apple may have assumed
that using such a blacklist would not increase the level of protection for iOS users, based on the fact
that iOS devices only execute code signed by Apple. This assumption seems flawed as: (a) a significant percentage of users jailbreak their iOS devices, thus the device can also execute unsigned code [59],
(b) if iOS users do not receive any warning when visiting a malicious site, they can unwillingly put
other users at risk by forwarding/sharing the URL, and (c) files that are downloaded to an iPhone might be synchronized to a computer, resulting to its infection. Finally, our experiments in iOS revealed that
the anti-phishing protection that Safari Mobile offers is less effective that other browsers using the Safe
Browsing list, mainly due to the infrequent updates of the list.
6.2 Proposed countermeasure
Currently, it appears that no web browser offers strong/adequate protection in both platforms
(desktop, smartphone). As such, we have implemented and evaluated, as a proof of concept, a secure
proxy, which correlates multiple anti-phishing and anti-malware blacklists as a potential countermeasure. Our work suggests that such browser agnostic architectures, are currently the only
solution for protecting normal users, who browse the web with their mobile browser, which is
particularly true for browsers that lack the respective built-in countermeasures.
It should be noted that the proxy is proposed as a proof-of-concept tool, to highlight the significant benefit that the aggregation of multiple blacklists offers in the detection of rogue websites.
Thus, our evaluation regards issues such as privacy or performance as out of scope of this work.
However, as discussed in the next paragraphs, both issues can be addressed in a real-world implementation.
The evaluation showed that the proposed control, which is based on the aggregation of multiple
feeds from different blacklists, achieves significantly better results in blocking rogue URLs than any
individual browser. Our work focused on reducing the number of false negatives (i.e. phishing or malicious URLs that were not blocked by the browsers), as this could result in a successful attack. The
control, however, may be prone to false positives, as some sites may have erroneously added to a
blacklist without proper verification. In this case, a URL might be blocked until it is removed from all blacklists, causing a temporal nuisance. Nonetheless, popular blacklists (e.g. Safe Browsing) allow the
site administrators to request their web site’s removal from the blacklist when it has been cleaned,
which will reduce the inaccessibility period. Furthermore, a production level implementation, could be configured based on the user’s or organization’s risk appetite, to only block a site if the number of
detections (blacklists which list the URL as malicious) exceeds a threshold, thus reducing potential
false positives. However, this falls outside the scope our work. As discussed earlier, Internet Explorer and Chrome use reputation-based systems for analyzing
downloaded files. Our results reveal that these countermeasures, although they significantly increase
the detection rate (compared to browsers which are using solely URL checking), they still fail to block
a large number of malicious files. Similarly to these two browsers, one of our secure proxy’s checks is hash-based analysis. In specific, the secure proxy queries the file’s hash in VirusTotal and blocks the
download if it is reported as malicious by (at least) one AV engine. Hash-based analysis with the
aggregation of multiple AV engines adds an extra layer of protection against malicious sites and - as URL only analysis - does not introduce significant delays. In the experiments queries for URL-only
analysis and hash analysis introduced average delays of 648ms and 516ms per request, respectively.
The benefit of doing an online check is that the URL or hash will always be checked against the
most up-to-date list. It also enables the end user to use a browser that does not offer any build-it countermeasures and still be protected. This is especially useful for smartphone devices, where only a
22
limited number of browsers offers built-in anti-phishing and/or anti-malware protection. In addition, avoiding resource intensive operations on smartphones reduces the battery consumption, which is a
common problem with modern smartphones.
The inherent drawback of this approach – as in any countermeasure with an online, centralized
architecture, is concerns for user privacy. This is true as each URL that the user is visits has to be submitted to a third party for analysis, thus exposing the user’s browsing history/profile. Even though
this falls outside the scope of this work, this can be mitigated by maintaining a local blacklist/whitelist,
thus avoiding the need for a central architecture (e.g. as in our case, a proxy server) - similarly to the way Safe Browsing protocol works. The browsers that implement the protocol, can keep a local
database of reported URLs, which is updated frequently while the browser is running. As a result, all
lookups use the local database, thus avoiding unnecessary delays and privacy issues. Based on the fact
that VirusTotal is owned by Google, it seems fairly easy to include the aggregated results from all blacklists providers to a single list (e.g. imported into Safe Browsing list). Still, this will require the
browser vendors to implement/adopt the Safe Browsing protocol and make sure that they avoid any
synchronization issues. The tests we performed have shown the significant benefit of aggregating multiple AV engines
for the detection of malicious files. Our results suggest that hash based analysis, on its own, is not an
effective defense, as during the experiments more than half of the downloaded files had unknown hashes. However, all of them were detected as malicious when uploaded to VirusTotal. Based on this
finding, the proposed proxy uploads for further analysis all downloaded files for which a hash-only
query returned no results. Such an analysis may introduce a delay (41 sec on average in our experi-
ments), which may not be acceptable for some users. However, the secure proxy can be configured to upload these files according a policy, e.g. by default deny access to these files, move the files in a
sandbox or ask the user to decide whether the files will be submitted for further analysis.
Similarly to what happens in other instances in the security domain, there is a tradeoff between security and usability. A combination of a whitelist and reputation based system will further limit the
number of files that have to be submitted for analysis - as only files that are not included in the whitelist
and are not blocked by the reputation system will have to be analyzed. In addition, the use of a progress bar, which shows the estimated time required for completion of the analysis and allows the user to
force the download to begin, at the user’s own risk, could be implemented. However, literature has
showed the unawareness of users that click through security messages [18], [19], [52], [53], as well as
the shortcomings of reputation systems [61]. The implementation of the secure proxy is based on the public version of VirusTotal’s API,
which introduces limitations. Firstly, VirusTotal is a service which was not designed to support semi-
real time queries, as the ones used by the proposed control. A dedicated service optimized for such use, such as CloudAV, might achieve far better performance [62] and as it can be hosted locally, it will
avoid any privacy implications. Also, the secure proxy, similarly to VirusTotal, does not weight
differently the responses from the various antivirus engines or URL blacklists. The proxy can be
extended to filter rogue sites based on a number of antivirus engines or blacklists that the user/administrator or the organization’s policy selects.
Finally, the results of our work are affected by the dynamic nature of the web ecosystem. This
is due to the dynamic nature of the threats and the new evasion techniques that attackers create. This is reflected on the comparison of the anti-phishing protection that is offered by the examined browser
in Q1 and Q2 (2014). Moreover, browsers add to the complexity their frequent updates, which might
include new security controls (e.g. analysis of downloaded files will be supported in the next version of Mozilla Firefox), and/or new weaknesses.
6.3 Conclusions
This paper provides an evaluation of the build-in protection mechanisms that are offered by web
browsers against rogue web sites, namely phishing sites and sites hosting malware (‘malicious sites’).
Our work focuses on the most popular desktop (Windows) browsers, namely Chrome, Firefox, Internet
Explorer and Opera, as well as their smartphone (Android and iOS) counterparts. The browsers were tested against a data set of 2800 rogue URLs (1400 phishing and 1400 malicious URLs) and our
analysis includes: (a) the number of the blocked URLs, (b) the number of the false negatives (i.e.
phishing or malicious URLs that were not blocked), and (c) the number of URLs that had been cleaned or were not accessible.
Our results uncover the shortcomings of the current security controls and highlighted the
substantial security gap, between desktop and mobile browsers. Mobile users are exposed significantly to rogue sites, as the default browser on iOS offers limited protection against rogue URLs, while the
default browser on Android offers no actual protection. The performance of Windows browsers also
differs significantly, especially between phishing and malicious sites. Furthermore, we highlighted
implementation problems of the Safe Browsing blacklist, which limit the level of protection offered by the browsers that use this blacklist.
In order to address these threats, we implemented - as a proof of concept - and evaluated a
countermeasure (i.e. secure proxy), which is based on the aggregation of multiple blacklists and AV engines. The secure proxy performs URL analysis using multiple blacklists and significantly increases
the level of protection against malicious downloads, by performing (a) hash based checks and (b)
content scanning, using multiple AV engines.
Our work has proved that the aggregation of multiple blacklists and AV engines can raise significantly the level of protection against rogue sites, regardless if users are using a smartphone or
desktop device. We regard that our work is beneficial both to the users and browser vendors. This holds
true as the former can be informed about the availability of protection against rogue web sites, which might help them choose a web browser based on an informed security decision. For the latter, this work
may stimulate browser vendors to adjust their current anti-phishing and anti-malware controls and/or
add missing controls, which would eliminate the privacy and performance limitations of our work and offer increased protection compared to their current deployed solutions. Finally, in the meantime we
envisage that the proposed security control can be used as the basis of a forward proxy, which protects
both smartphone and desktop users in an organization.
REFERENCES
[1] Nielsen, “The Digital Consumer”, The Nielsen Company. [Online]. Available at: http://www.nielsen.com/content/dam/corporate/us/en/reports-downloads/2014%20Reports/the-digital-consumer-report-feb-2014.pdf [Accessed: 2 Aug 2014].
[2] Gartner, “Top 10 Strategic Technology Trends For 2014”. [Online]. Available at: http://www.forbes.com/sites/peterhigh/2013/10/14/gartner-top-10-strategic-technology-trends-for-2014/ [Accessed: 2 Aug 2014].
[3] Mell, P., Kent, K., Nusbaum, J., “Guide to malware incident prevention and handling”, National Institute of Standards and Technology (NIST), 2005.
[4] Funk C., Garnaeva M. "Kaspersky Security Bulletin 2013. Overall Statistics for 2013". [Online]. Available: http://securelist.com/analysis/kaspersky-security-bulletin/58265/kaspersky-security-bulletin-2013-overall-statistics-for-2013/ [Accessed: 4 Aug 2014].
[5] CISCO "Cisco Annual Security Report". [Online]. Available at: http://www.cisco.com/c/en/us/products/security/annual_security_report.html [Accessed 10 Oct 2014]
24
[6] Virvilis N., Tsalis N., Mylonas A., Gritzalis D., "Mobile devices: A phisher's paradise", in Proc. of the 11th International Conference on Security and Cryptography (SECRYPT-2014), Austria, August 2014.
[7] Google, “Safe Browsing API”. [Online]. Available at: https://developers.google.com/safe-browsing/ [Accessed: 8 Mar 2014].
[8] Microsoft, “SmartScreen Filter”. [Online]. Available at: http://windows.microsoft.com/en-us/internet-explorer/products/ie-9/features/smartscreen-filter [Accessed: 8 Mar 2014].
[9] Netcraft, “Phishing Site Feed”. [Online]. Available at: http://www.netcraft.com/anti-phishing/phishing-site-feed/ [Accessed: 8 Mar 2014].
[10] Phishtank "Phishtank" [online]. Available: https://www.phishtank.com/ [Accessed: 6th Aug 2014].
[11] Abrams R., Pathak J.,Barrera O., Ghimire D. "Browser Security Comparative Analysis", NSS Labs, 2014. [Online]. Available: https://www.nsslabs.com/reports/browser-security-comparative-analysis-report-socially-engineered-malware [Accessed: 6th Aug 2014].
[12] Banu, M. Nazreen, S., Munawara Banu, “A Comprehensive Study of Phishing Attacks”, in Proc. of the International Journal of Computer Science and Information Technologies, vol. 4, issue 6, pp. 783-786, 2013.
[13] Rosiello, A. P., Kirda, E., Kruegel, C., Ferrandi, F., “A layout-similarity-based approach for detecting phishing pages”,
in Proc. of Security and Privacy in Communications Networks Workshops, pp. 454-463, 2007.
[14] Rani, S., Dubey, J., “A Survey on Phishing Attacks”, in Proc. of the International Journal of Computer Applications, vol. 88, issue 10, 2014.
[15] Jansson, K., Von Solms, R., “Phishing for phishing awareness”, in Proc. of Behavior & Information Technology Conference, vol. 32, issue 6, pp. 584-593, 2013.
[16] Bian R. M., “Alice in Battlefield: An Evaluation of the Effectiveness of Various UI Phishing Warnings”. [Online]. Available: https://www.cs.auckland.ac.nz/courses/compsci725s2c/archive/termpapers/725mbian13.pdf [Accessed 2 Feb 2014]
[17] Darwish A., Bataineh E., “Eye tracking analysis of browser security indicators”, in Proc. of Computer Systems and Industrial Informatics Conference, pp. 1–6, 2012.
[18] Akhawe D., Felt A. P., “Alice in Warningland: A large-scale field study of browser security warning effectiveness”, in Proc. of the 22nd USENIX Security Symposium, 2013.
[19] Egelman S., Schechter S., “The Importance of Being Earnest [In Security Warnings]”, in Proc. of Financial Crypto-graphy and Data Security, Springer, pp. 52–59, 2013.
[20] Sheng S., Wardman B., Warner G., Cranor L. Hong J., Zhang C., “An empirical analysis of phishing blacklists”, in Proc. of the 6th Conference on Email and Anti-Spam, 2009.
[21] Kirda E., Kruegel C., “Protecting users against phishing attacks with antiphish”, in Proc. of Computer Software and Applications Conference, vol. 1, pp. 517–524, 2005.
[22] Zhang J., Seifert C., Stokes J., and Lee W. "Arrow: Generating signatures to detect drive-by downloads", in Proc. of the 20th international conference on World wide web, WWW '11, 2011.
[23] AV “Anti-Phishing protection of popular web browsers,” AV Comparatives, Dec 2012. [Online]. Available: http://www.av-comparatives.org/images/docs/avc_phi_browser_201212_en.pdf [Accessed: 05 Jan 2014].
[24] Mazher N., Ashraf I., Altaf A., “Which web browser work best for detecting phishing”, in Proc. of Information & Communication Technologies Conference, pp. 1-5, 2013.
[25] Vidas T., Owusu E., Wang S., Zeng C., Cranor L., Christin N., “QRishing: The Susceptibility of Smartphone Users to QR Code Phishing Attacks”, in Proc. of Financial Cryptography and Data Security, pp. 52–69, 2013.
[26] Xu Z., Zhu S., “Abusing Notification Services on Smartphones for Phishing and Spamming”, in Proc. the 6th USENIX conference on Offensive Technologies, pp. 1–11, 2012.
[27] Mylonas A., Tsalis N., Gritzalis D., "Evaluating the manageability of web browsers controls", in Proc. of the 9th International Workshop on Security and Trust Management, pp. 82-98, Springer (LNCS 8203), UK, 2013.
[28] Google "Google Chrome Privacy Whitepaper" [online]. Available: https://www.google.com/intl/en/chrome/browser/privacy/whitepaper.html [Accessed: 6th Aug 2014].
[29] Rajab, M. A., Ballard, L., Lutz, N., Mavrommatis, P., AND Provos, N. "CAMP: Content-Agnostic Malware Protection", in Proc. of the Network and Distributed System Security Symposium (NDSS) (2013).
[30] Colvin R. " SmartScreen Application Reputation – Building Reputation". [Online]. Available: http://blogs.msdn.com/b/ie/archive/2011/03/22/smartscreen-174-application-reputation-building-reputation.aspx [Accessed: 4 Aug 2014].
[31] Antonakakis M., Perdisci R., Lee W., Vasiloglou II N. and Dagon D. "Detecting Malware Domains at the Upper DNS
Hierarchy", in Proc. of the 20th USENIX conference on Security (SEC'11), USENIX Association, Berkeley, CA, USA, p.16, 2011.
[32] Bilge, L., Kirda, E., Kruegel, C. and Balduzzi, M. "EXPOSURE: Finding Malicious Domains Using Passive DNS Analysis", ACM Transactions on Information and System Security, ACM, Vol. 16, No 4, USA, April 2014.
[33] Shahzad A., Hussain M. and Khan M. "Protecting from Zero-Day Malware Attacks", in Proc. of the Middle-East Journal of Scientific Research, Vol. 17, No, 4, pp.455--464, 2013
[34] Vadrevu P., Rahbarinia B., Perdisci R., Li, K. and Antonakakis, M. "Measuring and Detecting Malware Downloads in Live Network Traffic". Computer Security – ESORICS 2013, [online] pp.556-573. Available at: http://dx.doi.org/10.1007/978-3-642-40203-6_31 [Accessed 19 Jul. 2014].
[35] Curtsinger, C., Livshits, B., Zorn, B. and Seifert, C. "ZOZZLE: Fast and Precise In-Browser JavaScript Malware Detection". in Proc. of the 20th USENIX conference on Security (SEC'11), USENIX Association, Berkeley, CA, USA pp.33--48, 2011.
[36] Shin S. Xu Z. and Gu G. "EFFORT: Efficient and effective bot malware detection", Computer Networks, Volume 57, Issue 13, pp.2846--2850, 2012.
[37] Kolter J. and Maloof M. "Learning to detect and classify malicious executables in the wild", in Proc. of the Journal of Machine Learning Research, 7, pp.2721--2744, 2006.
[38] Perdisci R., Lanzi A., and Lee W. "Classification of packed executables for accurate computer virus detection". Pattern
Recognition Letters, 29(14):1941, October 2008.
[39] Perdisci R., Lanzi A., and Lee W. "Mcboost: Boosting scalability in malware collection and analysis using statistical classication of executables", in Proc. of the 2008 Annual Computer Security Applications Conference, ACSAC '08, pages 301{310, 2008.
[40] Caballero J., Grier C., Kreibich C., and Paxson V. "Measuring pay-per-install: the commoditization of malware distribution". in Proc. of the 20th USENIX conference on Security, SEC'11, 2011.
[41] Lu M., Leita C., Thonnard O., Keromytis A., and Dacier M.. "An analysis of rogue av campaigns", in Proc. of the 13th international conference on Recent advances in intrusion detection, RAID'10, 2010.
[42] Lu L., Yegneswaran V., Porras P., and Lee W. "Blade: an attack-agnostic approach for preventing drive-by malware infections", in Proc. of the 17th ACM conference on Computer and communications security, CCS '10, 2010.
[43] Provos N., McNamee D., Mavrommatis P., Wang K., and Modadugu N. "The ghost in the browser analysis of web-based malware", in Proc. of the 1st conference on First Workshop on Hot Topics in Understanding Botnets, HotBots'07, pages 4--4, Berkeley, CA, USA, 2007. USENIX Association.
[44] Provos N., Mavrommatis P., Rajab M., and Monrose F. "All your iframes point to us", in Proc. of the 17th conference on Security symposium, SS'08, 2008.
[45] Zhang, H., Liu, G., Chow, T. W., Liu, W., “Textual and visual content-based anti-phishing: A Bayesian approach”, in
Proc. IEEE Transactions on Neural Networks, vol. 22, issue 10, pp. 1532-1546, 2011.
[48] OpenDNS [online]. Available: http://www.opendns.com/ [Accessed: 18th Aug 2014].
[49] VirusTotal "VirusTotal" [online]. Available: https://www.virustotal.com/ [Accessed: 6th Aug 2014].
[50] Bradley, T., “Android Dominates Market Share, But Apple Makes All The Money”. [Online]. Available at: http://www.forbes.com/sites/tonybradley/2013/11/15/android-dominates-market-share-but-apple-makes-all-the-money/ [Accessed: 12 Apr 2014].
[52] Mylonas A., Kastania A., Gritzalis D., “Delegate the smartphone user? Security awareness in smartphone platforms”, Computers & Security, Vol. 34, pp. 47-66, May 2013.
[53] Mylonas A., Gritzalis D., Tsoumas B., Apostolopoulos T., “A qualitative metrics vector for the awareness of smartphone security users”, in Proc. of the 10th International Conference on Trust, Privacy & Security in Digital
Business (TRUSTBUS-2013), pp. 173-184, Springer (LNCS 8058) Czech Republic, August 2013.
[55] Zhou Y, Jiang X. "Dissecting Android malware: Characterization and evolution", in Proc. of the IEEE Symposium on Security and Privacy. IEEE, pp. 95-109, 2012.
[56] Zhou Y, Wang Z, Zhou W, Jiang X. “Hey, you, get off of my market: Detecting malicious apps in official and
alternative Android markets”, in Proc. of the 19th Network and Distributed System Security Symposium. USA; 2012.
[57] Chew M. "Download files more safely with Firefox 31" [online]. Available: http://monica-at-mozilla.blogspot.co.uk/2014/07/download-files-more-safely-with-firefox.html [Accessed: 6th Aug 2014].
[58] Gartner, “Gartner Says Annual Smartphone Sales Surpassed Sales of Feature Phones for the First Time in 2013”,
[Online]. Available: https://www.gartner.com/newsroom/id/2665715 [Accessed: 10th Aug 2014].
[59] Love D. " The Latest Jailbreak Statistics Are Jaw-Dropping". [Online]. Available: http://www.businessinsider.com/jailbreak-statistics-2013-3 [Accessed: 4 Aug 2014].
[60] Mármol, Félix Gómez, and Gregorio Martínez Pérez. "Security threats scenarios in trust and reputation models for
distributed systems." Computers & Security, Vol. 28, pp. 545-556, 2009.
[61] Oberheide J., Cooke E, and Jahanian F. "CloudAV: N-Version Antivirus in the Network Cloud." In USENIX Security Symposium, pp. 91-106. 2008.
