ROCCO Signalling Firewall Vendor Performance Report 2018 was created, designed and sponsored by ROCCO™
www.roamingconsulting.com
STRATEGIC ANALYSIS - POSITIVE TECHNOLOGIES EXTRACT VERSION
PLEASE NOTE: This report has been purchased by Positive Technologies for Unlimited Dis-tribution to its clients. The copyright of the report belongs to ROCCO™.
We hope you enjoy the report and if you wish to give feedback on our methodology or the results of the report please contact us at [email protected]
About the Authors Polina Hristova Jason BryanPolina is a research analyst and journalist at ROCCO. She is really into the details of what is happening in our industry these days. Po-lina is known for her articles on all kinds of interesting industry topics. Her analysis has really helped ROCCO succeed in developing the most relevant and insightful research.
Jason is Senior Consultant, Analyst and CEO of ROCCO based between Rome Italy and the ROCCO HQ in the UK. He has overseen the last 3 years of in-depth monthly research on Industry Analysis, Vendor Performance, Market Intelligence and Market Pricing in the Roaming and In-terconnect space. Jason was the founder of ROCCO in 2012, building on 20 years ex-perience of Roaming and Interconnect roles in Vodafone, Telefonica Groups and a num-ber of Vendors, as well as GSMA Chair-manship roles.
Choosing the right vendor to essentially become the backbone of your business can be a challenge when the options are many, especially when there is no clear indication as to how they perform. In this research, we identified 23 vendors who submitted their names for MNO evaluation, the maximum rating being 5.
ROCCO is a trusted source of original, non-sponsored and neutral data for the past 5 years, 500 MNOs and 180 Vendors participate in our research annually. Having reported extensively on Messaging, IPX, and Roaming, many MNOs rely on us to provide guidance in growing markets, such as the Signalling Firewall one. To have a better understanding of the current leading vendors, we gather the opinions of MNOs based on a variety of KPIs.
Investing in Vendors always takes a great effort, MNOs need to make RFP, RFQ and selection processes which take time and energy away from monetisation of Signalling. This report aims to offer advice from other MNOs which might help MNOs not yet decided on which Vendor to choose. If the MNO is not part of a group, or an independent MNO they hardly ever get the advice of other MNOs to help make decisions on the challenges they face, so the benefits of listening to a truly neutral 3rd party like ROCCO are clear.
Executive Summary
About the Executive Summary This part of the report is given for free to MNOs taking part in this research. The Full Strategic Analysis version of this report contains detailed feedback from MNOs on all the Performance and Leadership KPIs including the leaderboard, whereas this version of the report is providing a general summary.
The Leading Vendors The infographic below features the leading Vendors for 2018. Starting in alphabetical order we have two leaders in tier one for 2018. This means that all these Vendors scored overall between 4-5 points in their Overall Score.
The Signalling Firewall enables integra-tion of various machine learning, anti-fraud modules into the network of mo-bile operator ensuring for real-time de-tection and blocking of fraud attempts and security issues.
This is the second time that ROCCO has studied Signalling Firewall vendors. Fol-lowing the success of last year’s report, ROCCO has conducted another survey to review the leading vendors of this growing industry, reflective of the cur-rent climate in the market and among MNOs. Fraud prevention is slowly but steadily starting to put more value in preventing, rather than remediating; spotting trends and incoming threats has become the main priority, and hav-ing a reliable SigFire vendor can be an essential line of defence and overall effi-ciency.
Despite the wide range of innovation, new applications and handsets, the un-derlying industry technology can be of-ten overlooked and become the nest of vipers, and gaining access to mobile networks has always been a sweet temptation for attackers. The industry’s evolution towards an IP-based architec-ture has made it significantly easier for malicious 3rd parties to tinker with the possibility of sneaking in uninvited.
About the MNOs 49 respondents representing 71 MNOs (including perceptions from 6 Group MNO respondees) in 62 Countries took part in this research. Only respondees from Europe, North America, Far East Asia and Middle East continents responded.
Tier TableTiers Vendors
All ratings are out of 5 points. The Overall Score determines the Tier in which the Vendor is placed within our Table. The Overall Score is determined by the 4 sub-scores for the four rat-ing sections:• Performance KPIs, • Performance KPIS specific to this research,• Leadership • and their Overall Rating
All rating are combined to achieve an average Overall Score. E.g. 3.5 + 3 + 4 + 3.5 = 3.5 Over-all Score = Tier Two
Vendors are shown in Alphabetical Order with an overall ranking available in the Strategic Ana-lysis Version of this report.
Tier One Vendors: Whose Overall Score is between 4-5 out of 5 Cellusys
Positive Technologies
Tier Two Vendors: Whose Overall Score is between 3.5-4 out of 5 Adaptive
The top requirements MNOs have for Signalling Firewall
We asked MNOs what are the 3 most important criteria for them as an MNO when they consider the Signalling Firewall service. Some MNOs chose more than 3 criteria!
Question: What are the top requirements you have for Signalling Firewall Vendors?
MNOs indicated to us that for the second year, the “SS7 Attack Types” was the most important criteria for them. Followed by and the increasing importance of “Real-Time notification capabilities” and “Deep knowledge of signalling fraud market and trends”.
Compliance with GSMA Standards (FS and IR Docs)Deep knowledge of signalling fraud market and trendsFlexibility of deployment: TDM, SIGTRAN, IN-LINE, OPCODE, ANSI etcManaged Service offerings and capabilitiesOptions for Integration Models: Loopback, GatewayPattern recognition capabilities, contextual awareness (for types of fraudulent attack)Protection against extraction of authentication vectorsProtection against Interception and TMSI to IMSI TranslationReal-time notification capabilities associated with threat protectionScalabilitySolutions for Roaming Hubbing, Network Extensions and Outbound SolutionsSS7 Attack types (category 1, 2, 3 etc)Support for GEO Location tracking fraudTechnical support from Signalling experts
The Top Requirements 2018
9%4%
20%
6%4%
15%4%
4%
9%1%5%2%
10%
9%
! of !9 31Signalling Firewall
Vendor PerformanceReport 2018
POSITIVE TECHNOLOGIES EXTRACT �
Performance+ Overview ROCCO wants to know how Signalling Firewall providers are perceived also as far as the “Performance + criteria” are concerned. Those specific criteria which are fundamen-tal to the delivery of Signalling Firewall service. Below are some of the KPIS for Perfor-mance+ the position on all KPIs across all Vendors is shown in the Strategic Analysis ver-sion of this report. We can say while the overall performance is above average in scoring, it could improve.
Flexibility of deployment: TDM, SIGTRAN, IN-LINE, OPCODE, ANSI etc
Real-time notification capabilities associated with threat protection
Pattern recognition capabilities, contextual awareness (for types of fraudulent attack)
Deep knowledge of signalling fraud market and trends
Technical support for Signalling experts
Managed Service offerings and capabilities
SS7 Attack types (category 1, 2, 3 etc)
Support for GEO Location tracking fraud
Protection against Interception and TMSI to IMSI Translation
Protection against extraction of authentication vectors
Compliance with GSMA Standards (FS and IR Docs)
Solutions for Roaming Hubbing, Network Extensions and Outbound Solutions
Scalability
Options for Integration Models: Loopback, Gateway
Overall Score out of 5
3.5 3.7 3.9 4.1 4.3
4.04
3.98
3.75
3.86
3.94
3.76
3.65
3.97
3.9
4.21
3.99
3.79
4.06
4.01
! of !10 31Signalling Firewall
Vendor PerformanceReport 2018
POSITIVE TECHNOLOGIES EXTRACT �
Key Trends in Signalling Firewall
MNOs told us what were the key trends from their perspective.
“Very high amounts of fraud”
“Surpassing the amount we were expecting”
“IMSI / MSISDN harvesting /re-selling”
“Probably commercial model linking it to B2B or corporate mobile pack-age”
“Fake VLR”
“PSI, ATI, SRI for SM”
“PSI & ATI Messages”
“Clone Sims to attach in foreign countries to get TAN SMS”
“A2P fraud in addition to IMSI leak-age”
“Bots, we are mostly concerned in this area, this is going to be very painful for our Enterprise customers”
“eSIM is going to bring fresh issues here”
“We do not have the internal capabil-ities to deal with this, and they are rare to find, better to centralise intel-ligence in companies who specialise in Signalling Firewalls.”
“The security breaches we will see in the coming years will alienate us from Internet of Things, we must pro-tect our networks”
“We are worried about AI”
“We won’t be touching anything re-lated to Automotive, its too risky for our brand”
Preferred Pricing Models We asked operators which pricing model would they prefer if they could choose the pricing model of their Signalling Firewall Vendor: most respondees said that the “CAPEX” or “Managed Service” or CAPEX + Managed Service” pricing model is the one they prefer. Is the opposite approach that we saw for SMS Firewall and we have no explanation for why this could be.
Organisational Buy-In for Signalling Firewall We asked MNOs what was the best way for them to get Organisational buy-in or approval for Signalling Firewall services. These gave us several pieces of advice.
“Beauty Contest!”
“Mandated by regulatory bodies or fines for attacks”
“Security breach”
“Explain possible results of SS7 At-tacks”
“Signalling Firewall Analyser”.
“The SIG FW is highly necessary for any Operator to protect its network from attacking, hacking from foreign networks”
“To be in-line with all GSMA recom-mendations”
“Educate”
“It’s hard to get engagement when there’s so much fake news and pro-paganda in this space ”
“Our board are interested in current issues and are not into the details of what might happen later”
“A change in the pricing models”
“Strategically this is not on the radar”
“Get consultants to explain it”
“Our management are engaged on this but they still don’t see the need to spend money on it”
“We think the vendors in this space are scaremongering and we lost a lot of respect for them because of this approach "
“You have to explain it like it's their privacy which is effected”
We asked MNOs to tell us for how much time they had used Signalling Firewall ser-vices, the answers were very mixed but we had the clear view that the majority of MNOs had had these services for some time.
For how much time have MNOs had Signalling Firewall services
Num
ber o
f MN
Os
that
resp
onde
d
0
3.5
7
10.5
14
Time<6 Months 1 year 2 Years 3 Years 4 Years 5 Years 5+ Years
13
9
76
4
2
7
! of !14 31Signalling Firewall
Vendor PerformanceReport 2018
POSITIVE TECHNOLOGIES EXTRACT �
Opinions on MNOs paying fines for Signalling At-tacks
Question: Should MNOs pay fines for Signalling attacks?
We asked MNOs if they thought that MNOs should pay fines for Signalling Attacks. We were asked to raise this question by an MNO who felt that MNOs who do not take se-curity measures were actually responsible for other MNOs facing issues. 75% of MNOs felt that MNOs should pay fines.
Opinions on MNOs paying fines for Signalling Attacks
6%4%
15%
6%
69%
Yes, if its possible to prove which is the originating MNOYes, there needs to be a disincentive on MNOs offering access to signalling networksNo, because its hard to classify attacksNo, because MNOs are not yet fully invested in security measuresNo, because the onus should be on the target network to protect itself
! of !15 31Signalling Firewall
Vendor PerformanceReport 2018
POSITIVE TECHNOLOGIES EXTRACT �
Learning about Signalling Firewall Question: From the following list, which method was more useful for you to learn about Signalling Firewall?
There are several methods for learning about Signalling Firewall. According to our Survey’s outcome, we found out that a trusted relationship with the MNO’s Signalling Fire-wall partner is the preferred method of learning for MNOs. See the MNO rankings above and in the Strategic Analysis Report to know all the vendors better.
“We have tested a few of signalling FWs so far and from our experi-ence PT is the best in terms of customization, integration and has-sle free operation.”
Leaderboard Tier 1 The following charts show how the Vendors performed against all rating crite-ria and define the overall Vendor Ranking we call the Leaderboard.
How do we calculate the leaders? We take the following 4 criteria…
• Average of all Performance Category KPIs
• Average of Performance + Category KPIs (those KPIS specific to the Vendors ser-vice)
• Average of Leadership Category KPIs
• Average of General Rating Category
And we create an overall average.
We have concluded from our experience that this is the fairest way of understanding the overall Vendor Ranking.
Interesting to note, that while more MNO’s responded to our survey in 2018 than 2017, from the 23 Vendors in the market today, we only had feedback from MNOs on a handful of Vendors and in our final report only had relevant feedback on 5 vendors.
You have the feeling about Signalling Firewalls that many Vendors do not have a large amount of customers for these solutions. You could argue that the Network Manage-ment Team in any MNO might be questioning the need for a Signalling Firewall but as we heard from this report, one MNO put it very clearly “The SIG FW is highly necessary for any Operator to protect its network from attacking, hacking from foreign networks”.
Performance from all Vendors in Tier 1 and 2 2017, 2018, noted that Anam, Positive Technologies and Symsoft do not appear in both years
reports
Adaptive
Anam
Cellusys
Evolved Intelligence
Mobileum
Positive Technologies
Symsoft
3.5 3.7 3.9 4.1 4.3
4.1
3.71
3.8
4.21
3.68
4
3.76
4.07
4.06
3.9
3.7720172018
! of !24 31Signalling Firewall
Vendor PerformanceReport 2018
POSITIVE TECHNOLOGIES EXTRACT �
You could also argue that many MNOs have not yet been so severely impacted by Signalling Firewall issues to seek the support of an external vendor for help. In addition, probably for an MNO, the concept of having a Vendor support them in delivering a security solution for their network maybe a daunting concept, especially in terms of trust. But there is another factor to consider…
We would expect that the Enterprises the MNOs support would be looking for the Operators to have stronger Signalling Firewall protection, potentially as a value added ser-vice , especially in the case of Internet of Things related traffic. As we saw from this years IoT Security report, there are some major threats to MNOs on the IOT front which may lead to much more severe security situations that we have found with consumer devices. As we saw from the requirements comparison chart in the Executive Summary re-port, Vendors are performing especially well with Technical support. It would seem that consulting on Signalling Firewalls and the firewalls themselves go hand in hand. Clearly, technical support is also a growing requirement on these providers in 2018 and a potential value added service for vendors in this field alongside the real time notifications of impacts.
Looking at the leaderboard, it was interesting to note some clear leadership in re-cent years in this space. Congratulations to Cellusys for being the only Vendor who appears in tier one in both years and to Positive technologies who clearly have a lot of MNO sup-porters. It might also be recognised that Mobileum, Evolved Intelligence and Cellusys also all scored highly in our recent Innovators report showing a clear connection between lead-ing in Signalling Firewalls and having an innovative business environment. Exactly the right kind of confidence we need in these vendors to always be on top of the most recent threats to the signalling network.
Identifying Vendors There is often no formal list of Vendors available to MNOs. Even Associations don’t have all Vendors as members and often don’t provide full lists for people to view. We found there are a number of sources to create a definite list. The Vendors that were included in this survey appeared because of one of several reasons:
• They are easy to find with search engines
• They are active in social media
• They were recommended to be included by MNOs from personal experience
• They attend Industry Events and are known in the Industry, because they have Exhibition stands, produce standards in Industry associations, or simply throw promotional events or parties
• They are known Vendors to ROCCO
Even after checking all these sources, sometimes we miss a certain Vendor. There is no intention behind this. We are happy to include all Vendors however long this makes our list of Vendors that are included. However looking at the list below we believe it to be de-finitive.
Vendors who were included in this Research The 23 Vendors which are part of this report are listed below, some Vendors were removed from the research because we did not receive enough MNO ratings to provide meaningful results on their Performance and Leadership:
Vendors who were included in this years report Regretfully while 23 Vendors were featured in the report, not all Vendors are ranked in our rating table, this is either because there was no demand from MNOs to talk about them or the views were so minimal that we were not able to draw a significant view on how they perform. We sought at minimum 5 ratings on an MNO before including them in our rat-ing table . 1
However what the research does do, as with all of our reports, is provided that we have at least 5 responses from MNOs about a Vendor, is allow for any Vendor large or small to appear in our reports. We want to encourage competition in the market and in so doing allow a good choice of Vendors to MNOs, so we actively encourage even the smallest Ven-dor to grow its business and compete.
Overall, Vendors should be recognised for a job well done, or incase they didn’t per-form optimally, should have the ability to identify where they can improve. Even if they may not agree with the MNOs views here, they may have a second opinion into what's important to MNOs.
The chart below presents the percentage of times Vendors were chosen by respon-dees for further Performance and Leadership ratings.
A vendors position in the tiering system depends upon the Performance and Lead-ership ratings given for that Vendor.
There is no connection between how many people choose to give. Performance and Leadership ratings about a Vendor, as to how well that Vendor can perform. This is to en-sure fairness to all providers large and small.
However to get a reasonable perception of a Vendor (not based on a single possibly biased view), ROCCO requires that at least 5 MNOs select to give Performance and Lead-ership ratings to a Vendor before including that Vendor in our report.
Several Vendors were removed from our report since fewer than 5 MNOs chose to give Performance and Leadership ratings to them.
These Vendors being removed from this report has no reflection on their ability to perform Signalling Firewall services, just the amount of MNOs who wanted to talk about them within our research.
We encourage MNOs to talk about all Vendors in the market and to give fair and reasonable feedback. In our view all Vendors are equal and should be supported and encour-aged to develop their services.
Vendors who were in the Research but not in the final ReportVendors Reason for removal
AMD Telecom Not enough MNO ratings to provide meaningful results
ANAM Technologies Not enough MNO ratings to provide meaningful results
BICS Not enough MNO ratings to provide meaningful results
Computaris Not enough MNO ratings to provide meaningful results
Enghouse Networks Not enough MNO ratings to provide meaningful results
Group2000 Not enough MNO ratings to provide meaningful results
Haud Not enough MNO ratings to provide meaningful results
Jtendo Not enough MNO ratings to provide meaningful results
Which is the Signalling Firewall Vendor that you know best? (by % of MNOs who
rated that Vendor)
Positive Technologies16%
Mobileum18%
Evolved Intelligence18%
Cellusys33%
Adaptive14%
! of !28 31Signalling Firewall
Vendor PerformanceReport 2018
POSITIVE TECHNOLOGIES EXTRACT �
How the Ratings where Calculated The Overall Score has been created by taking all applicable MNO ratings for more than 40 KPIS, in the category of Performance, Performance+ (performance on important Signalling Firewall aspects) and Leadership:
Performance KPIs: • Customer Service
• Flexibility
• Offers Excellent Value Added Services (VAS)
• Quality of Service
• Reliability
• Reputation
Mavenir Not enough MNO ratings to provide meaningful results
Netnumber Not enough MNO ratings to provide meaningful results
Nokia Networks Not enough MNO ratings to provide meaningful results
Openmind Networks Not enough MNO ratings to provide meaningful results
Orange Not enough MNO ratings to provide meaningful results
Sonus Not enough MNO ratings to provide meaningful results
Squire Technologies Not enough MNO ratings to provide meaningful results
Svyazcom Not enough MNO ratings to provide meaningful results
Telecom Italia Sparkle Not enough MNO ratings to provide meaningful results
For every KPI above the MNO was invited to rate the Vendor out of 5. The Strategic Analysis version of this report contains the specific scores for each of the KPIs for each of the Vendors. For this Executive Summary we present the summary of the average ratings for categories using a stars system.
Eligibility for taking part in this report
Only MNOs were applicable to take part in this research. Feedback from Vendors, companies who were not MNOs was removed.
Your Feedback, Our Future Reports This represents ROCCO’s second Signalling Firewall Vendor Performance Report.
As we progress with our research we intend to refine questions and results much more finely but we are also aware that our being able to capture this data depends purely on the MNOs and their willingness to participate.
There are 700+ MNOs within the GSMA Membership. ROCCO made more than 3000 individual e-mails and calls to reach the feedback within this document and reach re-spondees at their most available.
It should be noted that it is important to identify respondents who know who the Providers are and can actively comment.
We would welcome your feedback on our questions, research, interpretation of the results and our presentation of the results. Please contact [email protected] to give your view.
We have taken a lot of feedback on this survey which we intend to use in the revi-sion of this survey for 2020.
In 2020 we intend to repeat several questions we have created here and then under-stand how perceptions are changing with regard to Signalling Firewall Vendors and how their performance has evolved.
