MWD Advisors White paper: Unlocking the potential of Big Data

MWD Advisors is a specialist IT advisory firm which provides practical, independent industry

insights that show how leaders create tangible business improvements from IT investments. We use

our significant industry experience, acknowledged expertise, and a flexible approach to advise

businesses on IT architecture, integration, management, organisation and culture.

www.mwdadvisors.com

© MWD Advisors 2012

Unlocking the potential of Big Data Helena Schwenk

A special report prepared for Actuate

March 2012

Big Data is one of the hottest trends in IT industry circles. Although overused as a buzzword it is

generally characterised by the large, varied and rapidly growing volume of information that often

remains untapped by existing BI and data warehousing systems. It’s data that comes in all shapes and

sizes emanating from sources as diverse as mobile phone, sensors, smart energy meters, e-commerce

and social media sites. Yet within all this data lies significant value – especially for those businesses

that successfully tap into it, exploit it and put it to work for better business effect. As an emerging

technology discipline it also brings its own set of challenges in terms of scarcity of skills and IT best

practices. But for those organisations that can overcome these obstacles there’s huge potential to

unlock its value as a way of enhancing productivity, driving efficiencies and growth, and creating a

sustainable competitive advantage.

This is a special report prepared independently for Actuate. For further information about MWD

Advisors’ research and advisory services please visit www.mwdadvisors.com.

a d v i s o r smwd

http://www.mwdadvisors.com/

Unlocking the potential of Big Data 2


Summary

Defining Big Data is not

straightforward

Pinning down a definition for Big Data is an ongoing challenge

especially since the industry and marketplace has yet to reach

consensus. Until there is some form of agreement it’s best to

consider Big Data by its core characteristics. To begin with,

Big Data is not just about data volume – it also needs to take

into account its shape, speed, complexity and variety.

Secondly, Big Data can often be characterised as data that’s

either too difficult or not economically viable to store and

process using traditional data warehousing systems and BI

tools.

Analytics brings Big Data to life

and unlocks its potential

While it’s easy to get hung up on the complexities of storing

and crunching Big Data, this activity on its own will not help

you unlock its true business value. Leveraging advanced

analytic capabilities such data mining and text analytics, on

the other hand, can provide the means to enable you to

answer new questions, discover hidden insights, or find

unknown relationships in data to drive real business

advantage. In turn this can enable you to keep ahead of the

curve, discover new revenue streams, reduce costs, enhance

the customer experience and build sustainable competitive

advantage.

Harnessing and exploiting Big Data

can bring significant rewards

The mining of Big Data has the potential to reveal actionable

and valuable insights across multiple industries, organisational

sizes and business functions. This is possible not least

because at the same time as the quantity and variety of data

continues to grow, the technology for capturing, managing

and analysing all of this data is steadily improving – and at an

increasingly affordable price. So although we’re at an early

stage of market maturity, the potential for Big Data

applications to create value, enhance competitiveness and

improve productivity are widespread – including those for

better fraud detection, deeper levels of customer

segmentation and more accurate consumer behaviour

predictions.

Success requires blending business

needs with investments in Big Data

technology, data integration

policies, and the right analytic

talent.

As you plan your Big Data initiative there are a range of

considerations that need to be taken into account to ensure

success. These include understanding the business need or

challenge; securing the right level of commitment and

investment from senior management; getting to grips with

the types and complexity of data sources at your disposal;

ensuring you navigate the technology landscape and choose

the right tools; and ensuring you invest in the right people

with the right skills to exploit your Big Data to its full effect.



Big Data makes its Big Impact

What’s the Big Deal with Big Data?

Unless you’ve been living in a vacuum it’s hard to avoid a conversation in today's business technology

circles without touching on the subject of Big Data. Similarly most press coverage of the topic centres

on Big Data as the new do-or-die technology that businesses need to leverage if they want to stay in

the game and remain one step ahead of the competition. It’s not surprising given these headlines

therefore that certain commentators have already written off Big Data as a bubble that is set to burst

and leave many IT organisations despondent in its wake.

While there is no shortage of hype – and there may very well be casualties along the way – Big Data’s

prominence and ascendency is driven by a very real business challenge, namely the unprecedented

growth of digital data across nearly every industry, region and size of organisation. It’s a challenge that

isn’t going to go away, as the figures demonstrate. According to a McKinsey Global Institute report1,

in 2010 enterprises globally stored more than 7 exabytes of new data on disk drives, while consumers

stored more than 6 exabytes of new data on devices such as PCs and notebooks. Likewise other

industry insiders point to the fact that that 90% of the data in the world today has been created in the

last two years alone. This tsunami of digital data is being generated by businesses and consumers alike

through social networks, sensors, online videos, e-commerce sites, GPS signals, printer streams and

Call Detail Records (CDR), to name but a few.

However, storing and managing this data is only one part of the challenge; the exponential growth in

information is also being matched by a strategic need and desire by businesses to find hidden nuggets

of information within this data. Extracting value and insight can help organisations keep ahead of the

curve in their quest to discover new revenue streams, reduce costs, enhance the customer

experience and build sustainable competitive advantage. Harnessing and extracting value from Big

Data is seen by many as a route to achieving these aims, where data is no longer purely seen as a ‘by

product’ of doing business, but is instead seen as an important asset that can be utilised to inform,

guide and improve the quality and speed of decision making.

In truth, any Big Data effort is likely to bring both opportunities and challenges for organisations. To

begin with, the management of Big Data is a difficult and complex undertaking. This is not only

because of the sheer volume of data that is being created, but also due to the variety of data types it

encompasses (such as unstructured and structured data), as well as the speed of its delivery, which in

some cases might be in real time. Similarly, once this data has been captured, stored and analysed,

organisations need to understand how those insights pertain to their business and how they can act

on them in a timely and effective manner. Yet in spite of this, the overriding fact remains that Big

Data, if used and harnessed successfully, can bring enormous benefit and value to organisations –

something that far outweighs the challenges and obstacles present in storing and processing it. In fact,

for some the benefits will only be limited by their ability to use data in more imaginative and valuable

ways.

What’s in a name?

Given its ubiquity as a term, coming up with a definition for Big Data is not as straightforward as you

might think, especially as the technology industry has yet to reach any kind of consensus. While

pinning down a definition can be akin to hitting a moving target, it’s helpful to consider Big Data by the

characteristics and traits it exhibits and in terms of how it differs from other more traditional data

management approaches.

1 Big data: The next frontier for innovation, competition, and productivity, May 2011, McKinsey Global Institute -

http://www.mckinsey.com/Insights/MGI/Research/Technology_and_Innovation/Big_data_The_next_frontier_for_innovation

http://www.mckinsey.com/Insights/MGI/Research/Technology_and_Innovation/Big_data_The_next_frontier_for_innovation



Our research suggests that to understand the full scope of Big Data management, it needs to be

framed in the following contexts:

Big Data is not just about data volume – it also needs to take into account its shape, complexity

and scope. In contrast to more traditional data management approaches, it encompasses semi-

and unstructured data as well as structured data, and includes data generated not only by humans

but machines too.

Similarly, the management of Big Data needs to takes into account data that is both ‘at rest’ –

where data is captured and analysed at a point in time – as well as ‘in motion’ – where data is

analysed as a continuous stream on the move.

Big Data can often be characterised as data that’s not economically viable to store and process

using traditional data warehousing systems and BI tools. In this sense it often requires a new

technology, analysis and architectural approach to data management to harness it effectively.

Don’t get distracted by size. Big Data is a subjective measure and can start from anything from

hundreds of terabytes to datasets that hit the petabyte range. What’s more important is the

context of its use in a traditional enterprise setting: Big Data projects typically apply to scenarios

where data has previously been too challenging to store and process or where data simply hasn’t

been accessible before.

Big Data can be sourced from both inside and outside the organisation, whether it’s in social

media data streams, sensor logs or transactional data stored behind the firewall.



Business opportunities associated with Big Data

Tapping into the gold mine of Big Data

While a lot of buzz has focused on the technicalities of storing and processing Big Data this view often

overlooks the most important question: why should you care? The answer lies in uncovering the Big

Data sources that hold potential treasure troves of information that can be explored, mined and

combined with existing data to unlock secrets, opportunities and potential successes that are aligned

with the needs of your particular business. This means that there’s no simple ‘one size fits all’ answer.

The effective management of Big Data promises deeper and richer insights based on the ability to

work with individual records, rather than basing insights on an aggregated data slice (typically found

within a data warehouse), or a sample of the information to hand. This is especially true in

exploratory data analysis where analysts don’t always have a clear understanding of the questions they

want to ask of data. Without the benefit of Big Data technologies and techniques, analysts have no

choice but to work with partial data, which can introduce errors or limit the scope of analysis,

whereas analysing a complete set of data allows organisation to get answers to questions that haven’t

been posed before. In this sense, taking advantage of a Big Data opportunity requires a more creative

and inquisitive approach to data analysis and problem solving – one that combines the ‘science’ of

analytics and data discovery with the ‘art’ of applying it to real-world scenarios and revenue models.

Likewise, since a lot of what commentators call Big Data emanates from embedded sensors found in

mobile phones, medical devices, automobiles or smart energy meters, the use cases for its analysis can

extend to areas outside of the traditional domain of BI and analytics, within industries such as

healthcare, oil and gas, and transportation. In these scenarios Big Data can enable organisations to use

advanced correlation techniques to identify potentially useful patterns that would otherwise remain

hidden in petabytes of data.

Analytics brings potential to Big Data

Given all this potential it’s worth underlining the fact that Big Data on its own cannot unlock business

value. Instead it’s the application of Big Data to real-world business scenarios that provides scope for

competitive advantage. As shown in figure 1, it’s about pulling data together, combining the right

technologies and tools, applying analytics and creating actionable insights that business managers can

use to make better, higher quality and quicker decisions.



Figure 1: The Big Data mix

Source: MWD Advisors

Getting the right mix enables organisations to sift through, find and exploit new patterns and

relationships in the data in order to, for example, identify risks, anticipate and respond to changes in

market conditions, and predict customer behaviour, conditions and events in ways that previously

haven’t been possible before. Similarly it can be used to add insight to existing analytics such as fine-

tuning customer segments for more targeted marketing campaigns, crafting better marketing

strategies, devising more profitable pricing strategies, offering more sophisticated product

recommendations and helping organisations discover new products and services.

It’s clear from some early use cases that the power of Big Data can yield some impressive business

results. However, the challenge for most organisations comes not only from how you process,

explore and mine Big Data, but also from understanding how those insights are relevant to your

business and how you can act on them in a timely and effective manner. In the next section we will

move on to talk about some of the most common use cases for leveraging Big Data in this business

context.

Actionable insights

Data integration

Analytic skills &

techniques

Big Data technologies

& architecture

Business need



Understanding the use cases for Big Data

The possibilities for tapping Big Data to reveal valuable insights seem almost limitless, particularly as

it’s a phenomenon that impacts multiple industry sectors, organisational sizes and business functions.

Just as the quantity and variety of data continues to expand, the technology for capturing, managing

and analysing all of this data is steadily improving at an increasingly affordable price, allowing more

businesses to leverage and exploit the potential of Big Data. Although it’s still early days in terms of

real-world use cases, there are signs that organisations are actively pursuing Big Data opportunities to

create value, enhance competitiveness and improve productivity. Today organisations are mining the

data they’re currently capturing and storing, although may not necessarily be exploiting to its full

potential. At present, our research suggests that the usage scenarios for Big Data fall into one of four

broad business opportunities and drivers, as shown in figure 2.

Figure 2: The business opportunity for Big Data

Source: MWD Advisors

Business Driver Opportunity Example

Improving operational efficiencies

Identifying and preventing customer churn

Telcos can analyse growing volumes of CDRs, together with interaction usage, network and transactional data to discover and predict new forms of churn in their network.

Fraud detection Insurance companies can identify patterns of fraudulent behaviour much much faster found in terabytes of online, mobile and transactional data for insurance claim fraud and anti-money laundering,

Pinpointing areas for cost efficiencies

Retailers can use data captured from loyalty reward programs, and in store, mobile and online transactions to optimise and improve margins for product inventory, and markdowns

Mitigating risk Financial service companies can monitor and analyse financial data streams in faster timescales to identify and minimise their credit and market risk exposure.

Enhancing the customer experience

Understanding customer sentiment

Consumer Package Goods companies can acquire and mine unstructured data from social networks to get an overall picture of their brand’s perception and conduct real-time market research.

Fine tuning customer segmentation

Financial institutions can segment customers by credit card behaviour at a finer level of granularity to target and tailor products more effectively to specific risk profiles

Gaining a 360-degree view of the customer

Organisations of all sizes can capture and accumulate a wider range of customer attributes to gain deeper and more accurate insight into customer behaviour and model it with greater precision.

Improving revenue generation

Identifying new sales opportunities

Web-based companies can get a fuller picture of visitor usage and purchase patterns to help optimise website design, content creation and develop product recommendations that boost traffic and sales.

More granular customer targeting

A retailer can collect and mine customer purchase data to micro-segment its customer base that is used to optimise its product mix, pricing, and promotions more accurately

Driving strategic change

Better planning and performance management

Utilities and energy companies can tap into vast volumes of smart meter data to accurately predict retail demand and control supply costs in ways that have not been possible before.

Understanding new markets

A credit card provider can create value from the wealth of data it is storing and analysing by selling consumer insights based on the data streams it generates from processing payments.

Discovering and developing new products or services

Healthcare providers can aggregate and analyse enormous volumes of clinical and claims data, to find the next big ‘super’ drug that will help .



Bigger, better and faster

While some of these application areas and use cases are familiar and well understood by BI and data

warehousing communities, what’s different now is the scale and scope of analysis that Big Data can

enable. In other words, if leveraged in meaningful and more accurate ways Big Data can help you

exploit information to do things bigger, better and faster. This in turn will place new requirements on

BI and analytic toolsets as they are called upon to support the volume, speed, variety and workload

demands of Big Data. It’s an effort that will require you to look seriously at the technologies used to

drive both your current and future data management strategies and information needs.

To begin with, your BI environment will need to extend its support for analytic techniques such as

data mining, predictive modelling, natural language processing, machine learning and advanced SQL, as

well as improving support for collaboration, data discovery and visualisation techniques to help

interpret the results of Big Data analysis.

At the same time this needs to be married from a data management and integration perspective with

capabilities for sourcing new forms of data, including unstructured and structured data, the ability to

support both high and low latency data demands, as well architectural support for scale-out and high

speed data processing.

Today these Big Data challenges cannot be solved by a single platform or engine but instead need to

employ a variety of technologies, components and architectures. These may include technologies such

as Hadoop, MapReduce and distributed NoSQL databases, but it could also include technologies such

as in-memory databases, columnar databases and massively parallel processing architectures. However

for some, the real potential value of Big Data will only come when it’s merged and integrated with

existing business processes and data assets, such as a data warehouse, to provide a fuller and more

complete picture of their business.

Finally, any Big Data effort will require you to think carefully about sourcing and investing in the right

people, analytic skills and experience to make sure you can take advantage of the huge opportunities

that Big Data presents.



Where to start on your Big Data journey

As you plan to embark on a Big Data initiative there are a range of considerations to take into

account and challenges to overcome if your initiative is to realise its full potential. You need to

develop a practice that involves assessing business priorities and needs and match these with

investments in Big Data technology and techniques, data integration policies and the right analytic

talent. To assist you on the path to Big Data success the following steps provide guidance about how

and where to start your Big Data journey.

Get buy-in and commitment. It’s true to say that all IT programmes benefit from having

senior-level sponsorship and buy in, but this is especially true in the case of Big Data projects. A

sponsor needs not only to invest time and money in any effort but also match this with a

compelling vision and understanding of how Big Data can unlock real business potential for your

organisation.

Choose your data sources. A large part of the Big Data effort involves assessing the type and

format of data sources you want to use. In many cases this could mean considering opportunities

for analysing new types of data such as log files, sensor data or video streams that were

previously not available or possible before.

Good data preparation reaps rewards. It doesn’t make sense to always subject Big Data to

the same rigorous data cleansing, scrubbing and matching routines required in an enterprise data

warehousing environment. However, in certain scenarios you will still need to transform the data

and apply hygiene routines to Big Data in order to maximise its potential, for example by ensuring

you have prepared the data for analysis and rectified any data quality issues in the source data.

Change the way you think about data. The ability to analyse all of your data rather than just

a subset or sample will require a subtle but different analytic mindset. Big Data environments are

often regarded as exploratory platforms where analysts can dig and play around in the data as

they attempt to uncover new and interesting insights. It’s a mindset that requires a more creative

and inquisitive approach to data analysis and problem solving, and one that combines traditional

analytic disciplines with the ability to apply these to real-world business scenarios.

Pick your tools. With such an array of technologies and architectures to choose from, expect a

considerable part of any Big Data effort to be spent on understanding and navigating the

technology landscape. You need to consider key capabilities such as the performance, scale, and

data delivery rates of each tool or platform alongside support and integration with BI and

advanced analytic tool and techniques.

Invest in skills, skills, skills. Finding the right talent to utilise Big Data technologies and

techniques will continue to be a challenge for most. Those of you who are new or have had

limited exposure to disciplines such as Hadoop, data mining or statistics will need invest time in

sourcing or training staff. However, this is only part of the story: there should also be an equally

concerted effort to employ and develop those skills for aligning the data with the business, so

insights derived from Big Data can be used to drive better decision-making and business

outcomes.

Date post:	13-Jan-2015
Category:	Technology
Upload:	actuate-corporation
View:	1,479 times
Download:	0 times