Serene Zawaydeh - Big Data -Investment -Wavelets

University of Toronto

School of Continuing Studies

SCS 2942 – 002

Foundations of Enterprise Data Analytics - Concepts and Controls

Instructors

Larry Simon, MBA, CMC Brad Brown, MBA, MSc. [email protected] [email protected]

Big Data, Investment Management

and Wavelets

Prepared by:

Serene Zawaydeh, MBA, B.Sc. EE

[email protected]

March 24, 2014

Serene Zawaydeh

Big Data, Investment Management and Wavelets March 2014

Page 2 of 24

Table of Contents

Big Data in Investment Management ........................................................................................................... 3

“Acute Big Data Challenges Facing Asset Managers” ................................................................................... 4

Capital Market Firms using Unstructured Data ............................................................................................ 4

Big Data Applications in Investment Industry and Big Data Technologies used by Capital Market Firms ... 5

Examples of Big Data Systems in Investment Management......................................................................... 6

Visual Representation of Stock Market Co-Movement using Wavelets ....................................................... 8

Building Wavelet Histograms on Large Data in MapReduce ........................................................................ 9

Data Mining ................................................................................................................................................. 10

K-Nearest Neighbor – Machine Learning .................................................................................................... 10

Visualization of Unstructured Text ............................................................................................................. 11

Sentiment Analysis ...................................................................................................................................... 11

Data Mining in Investment Management ................................................................................................... 12

Theoretical Big Data Model ........................................................................................................................ 13

Implementation of Big Data Projects .......................................................................................................... 15

Wavelets and Non Stationary Signals ......................................................................................................... 16

19 Level Filter Bank System ........................................................................................................................ 17

Bibliography ................................................................................................................................................ 20

Table of Figures

Figure 1: Big Data Applications in Investment Industry ................................................................................ 5

Figure 2: Big Data Technologies used by Capital Market Firms .................................................................... 5

Figure 3: Comovement between stock returns ............................................................................................ 9

Figure 4: MapReduce .................................................................................................................................... 9

Figure 5: Time Series Classification ............................................................................................................. 10

Figure 6: Theoretical Big Data Analytics Model incorporating Applications of Wavelets .......................... 14

Figure 7: Changing value of Beta by changing the period around Beta is calculated ................................. 17

Figure 8: 19 Level Filter Bank System for Analyzing ECG Signals using Wavelet Packets ........................... 18

Figure 9: Spectrum of the Successive Filters using Daubechies 4............................................................... 19

Figure 10: Wave Packet Tiling of the Time-Frequency Plane

Figure 11: Scaling Function (Low Pass Filter) and Wavelet Mother (Band Pass Filter)............................... 19

Serene Zawaydeh


Page 3 of 24

Big Data in Investment Management

Volume, velocity, and variety characterize big data in general, and these characteristics apply to data in

the investment industry. The fourth V according to McKinsey is Value: Big data has high commercial

value and will be a major source of competitive advantage for firms, allowing them to better understand

their customers and their own business. (Manyika & al, 2011)

Companies are heavily investing in big data. 15% of 1,217 surveyed companies by TCS (TCS) spent at

least $100 million each on Big Data initiatives in 2012, and 7% invested at least $500 million. 643

companies undertook Big Data initiatives in 2012. 24% spent less than $2.5 million each. Industries

spending the most are telecommunications, travel-related, high tech, and banking. The value of

implementing big data systems was proven in the healthcare industry, which has seen a 20% decrease in

patient mortality by analyzing streaming patient data, while the Telco industry has seen a 92% increase

in processing time by analyzing network and call data.

Diversifying investment portfolio across markets increases the amount of data that needs to be

processed when making investment decisions. Global markets are interdependent. Political changes in

one country affect neighboring markets, and a financial crisis starting in one economy can have a global

impact. Understanding interdependence between stock markets provides valuable insights to

diversification strategies. Negative news on one company can be positively reflected on a competitor’s

stock price. Organizational changes, layoffs, earnings and dividend announcements, bankruptcies, as

well as currency changes and inflation (GetSmarterAboutMoney.ca) affect stock markets. With the

efficient market hypothesis and according to the random walk, stocks’ past performance is not an

indicator of the future performance. Historical data, publicly available data, and confidential data are all

integrated into the stock prices. Emerging events affect stock markets, and therefore a system that

integrates real time market changes, extending to tweets about stocks and companies, would be of

value.

Big data solutions are being implemented in the investment industry, allowing processing of a large

volume of variables including real time changes. Traditional technical analysis that looks into the stock

price movement does not look at the company’s financial statements. Meanwhile, valuation models

such as the discounted cash flow models, including fundamental analysis of companies’ historical

financial statements and projecting future cash flows, do not have room for real time changes that

affect the market. Big data visualizations and integration of structured and unstructured data from

different sources, and the ability to integrate real time changes into the system make it a suitable

solution for investment management.

In addition to highlighting current applications of big data in the investment industry, this paper

identified applications of Wavelets in finance and Big Data. Academic studies proved the benefits of

using Wavelets for forecasting stock prices and data mining among other applications.

Serene Zawaydeh


Page 4 of 24

“Acute Big Data Challenges Facing Asset Managers”

A survey of 400 asset managers and owners conducted by Economic Intelligence Unit on behalf of State

Street (Chris, 2013) found out that Data accuracy; lack of data integration facilities; high pricing; and

lack of timeliness in external data are key challenges.

• Nine out of 10 institutional investors view data and analytics as a key strategic priority.

• Investments in data technologies and platforms increased, yet less than a third currently gain

any competitive advantage from their data and analytics capabilities.

• The ability to aggregate, analyze and transform data is key to institutional investors’ ability to

compete.

Investment in Big Data is rising, and targeted at tools to support decision making in the front office, and

solutions to manage risk and regulatory compliance more efficiently. Data and analytics are a primary

strategic priority, and are a source of competitive advantage.

Data leaders need to improve risk tools with multi-asset class capabilities; Develop better tools to

manage regulation in multiple jurisdictions; improve the ability to manage and extract insight from

multiple data sources; and optimize electronic trading platforms developing a scalable data architecture

that will grow with the business. (State Street, 2013)

Capital Market Firms using Unstructured Data

Risk analytics, regulation, and trading analytics are three of the five areas identified by Infosys in which

Capital Market firms are using big unstructured data and the State Street report (State Street, 2013).

The two other areas are financial data management and reference data management and data tagging.

•••• Financial data management and reference data mangement: Data storage for historical trading,

internal data management challenge, and overall control on reference data (on-demand data mining

to dig into meta-data to deconstruct/ reconstruct data models, etc.). It can be very tough in

maintaining (storing, handling, and processing) data from various asset classes coming from various

vendors.

•••• Regulation: Includes preparation for regulations like Dodd Frank, Solvenc II, EMIR, audits etc.

•••• Risk analytics: Includes fraud minitgation, anti-money laundering (AML), Know Your Customer (KYC),

rogue trading, on-demand enterprise risk management, etc.

•••• Trading analytics: Includes Analytics for High Frequency Trading, Predictive Analytics, Pre-trade

decision-support analytics, including sentiment measurement and temporal / bi-temporal analytics

etc.

•••• Data tagging: In enterprise-level monitoring and report, it is often hard to match and reconcile

trades from various systems built on different symbology standards – usually resulting in invalid,

duplicated and missed trades. Data tagging can easily identify trades and events such as corporate

actions and enable regulators to detect stress signs early.

Serene Zawaydeh


Page 5 of 24

Big Data Applications in Investment Industry and Big Data Technologies used by Capital Market Firms

Figure 1: Big Data Applications in Investment Industry

Investment Bank Big Data Use Cases Areas

Investment Bank

An investment firm with assets of over $1 trillion and operations in approximately 50

countries uses big data technology to deliver Reference Data to the Murex trading

platform and other downstream operations

Reference Data

Management

Investment Bank

An investment firm with assets of over $1 trillion and operations in approximately 50

countries uses big data to manage risk exposure through real-time communication

across bond, futures and credits trading. Risk Analytics

US Investment

Bank

An investment bank shifted to the risk management and P&L towards a real-time

environment. Big Data technologies were leveraged to help the firm to gather all

relevant data into one place. Regulation

European

Investment Bank

An investment bank used Big Data analytics to track performance monitoring, risk

analysis and reporting

Risk Analytics

and Regulation

Asian Investment

Bank

An investment bank used Big Data technologies to generate on-demand performance

metrics for risk measures across multiple global trading businesses.

Trading

Analytics

European

Investment Bank

An investment manager used Big Data technologies to gather relevant details so as to

respond as a witness to litigation action against a prime broker Compliance

US Investment

Manager

Investment manager used Big Data technology to centralize data and applications to

apply governance policies and mitigate risk of damages from litigation discovery

Risk Analytics

and Regulation

Global Exchange

A major global exchange used Big Data technology to provide global market

participants with on-demand access to data and data-mining tools for trading,

analytics and risk management in a cloud-based / hosted environment

Trading

Analytics & Risk

Analytics

US Regulator

A US regulator used Big Data technology to create a searchable library of research,

econometric and other information generated by the regulator's activities. Regulation

Buy Side firm

A major buy-side firm uses Big Data technologies for market surveillance, an activity

requiring processing of vast quantities of market information Regulation

Asset Manager

Fiduciary management - a new area of interest in which asset managers outsource

management of their portfolios to third-party administrators in order to benefit from

economies of scale

Fiduciary

Management-

Emerging Area

Regulatory

compliance and

advanced

analytics

An investment bank uses big-data techniques to handle and manage petabytes of

regulatory compliance and advanced analytics. The bank used technology from

Hadoop, open source framework that supports data-intensive distributed computing,

which allows data to be crunched over a distributed network of computers

Regulation &

Risk Analytics

http://www.infosys.com/industries/financial-services/white-papers/Documents/big-data-analytics.pdf

Figure 2: Big Data Technologies used by Capital Market Firms

Data Grids Use distributed catching to manage large volumes ofdata across a network ofservers.

Compute grids Offer a way of parellizing processes across multiplesservers, handling capacity / failure issues

and orchestrating tasks across the grid

Massively parallel processors Involves the coordinated processing of a programme between multiple independent

computers, each with its own operating system and memory

In-memory databases Databases that store data in the main memory rather than a disk, as is the case with traditional

databases

NoSQL Shell relational database management systems that don’t use Structured Query Language, are

more simple than traditional databases and whose tables are compatible with a wide range of

external platforms

Specialized databases Contain the necessary architecture to store the unstructured data. Ex: IBM Viper DB2 database,

EMC Greenplum, Greeplum

Hadoop A tool used to query the unstructured data which is a major part of big data analytics. Ex. EMC

Map Reduce, IBM Netezza

http://www.infosys.com/industries/financial-services/white-papers/Documents/big-data-analytics.pdf

Serene Zawaydeh


Page 6 of 24

Examples of Big Data Systems in Investment Management

Regulatory Compliance Systems

Regulations in the investment industry require safeguarding insider information, and ensuring that no

improper trading occurs. Privacy of investors and security of data needs to be maintained in traditional

trading and investment environment and also in Big Data.

Merwin et al’s Patent Application US 20130290218 A1 introduces a system and method for regulatory

compliance management (Merwin, Higgins, Hardy, & Marks, 2013). For an investment portfolio, the

method tags and searches regulatory and other documents using a query module. It includes receiving

the documents by an analysis module. The system determines trading errors; transactions in different

accounts in the investment portfolio; long and short positions occurring within a same security in the

investment portfolio; daily trading volume for a security in the investment portfolio; value of

transactions; and commissions.

AIMCO www.aimco.alberta.ca

Alberta Investment Management Corporation (AIMCo), established in 2008, manages approximately

C$70 billion on behalf of 60 pension, endowment, government reserve clients in the Alberta. There was

a need for clean and timely data not just through spreadsheets, a practice that is not uncommon even

for investment organization managing billions of dollars.

AIMCO introduced a new architecture, with a centralized data warehouse where information can be

shared across the firm. This reduced the redundant systems, reduced reconciliation efforts required to

keep all information in synch. It aimed at storing more detailed, granular data that would support in-

depth queries in real time, allowing investment professionals to select various views of portfolios to gain

unique insights. The data infrastructure supports strong internal audit and compliance processes for

data, while allowing AIMCo to be competitive through enriched analytics that cannot be purchased from

outside vendors.

State Street Global Exchange www.statestreetglobalexchange.com

State Street Global Exchange established a big data division devoted to portfolio modelling, investment

analytics, data management and data projections. The division aims at enabling institutional investors

to better analyse client data to identify risks and monitor the efficiency of portfolios. Insurance

companies are also working on projects to better understand their customers’ profiles. (McGrath, 2014).

Portfolio Analytics

StatPro North America www.statepro.com is a global provider of portfolio analytics for investment.

StatPro Revolution a sophisticated portfolio analysis platform based in the cloud. It provides instant

access to information on portfolio performance, risk, attribution and allocation analysis. Portfolio

analysis is shared with the clients. What If analysis allows viewing the effects of investment decisions.

(StatPro, 2012).

Serene Zawaydeh


Page 7 of 24

Angoss http://www.angoss.com/

Angoss provides a predictive analytics solution which is being implemented in the financial industry,

insurance, and mutual funds. It maintains data integrity, security. Angoss business intelligence software

and predictive analytics solutions provide more than 300 companies worldwide. Its clients include

Microsoft, Bank of America, and mutual funds such as Vanguard, Fidelity, Dynamic Funds.

Apama: Real Time Analytics

The Progress Apama platform implements a patented CEP architecture that can monitor market data

(both market feeds and related information, like electronic news). The sub-millisecond responsiveness

of Apama is sustainable when market data volumes reach the tens of thousands of events per second

and when concurrent strategies number in the thousands.

Apama strategies execute within the Apama Correlator, which employs a patented, multi-dimensional

filtering architecture to detect patterns and identify appropriate actions, in under a millisecond. The

Apama Correlator offers tremendous scalability with flexible configuration options that support various

designs for load balancing and fault tolerance. Unlike single-purposed CEP engines, the Apama

Correlator can support thousands of discrete strategies executing simultaneously with no performance

degradation. Additionally, the Apama high-availability (HA) architecture incorporates a “cluster” model

that enables recovery of failed nodes such that they can re-synchronize with redundant nodes and

commence operation with no impact on performance.

Wavelets in Finance and Big Data

Multidimentional filter banks and Discrete Wavelet Analysis were the topic of my digital signal

processing research. Hardoon et al used a tree structure for predictions (Hardoon & Shmueli, 2013).

Associating that to Wavelets which also have a tree structure, I investigated whether there are other

applications for Wavelets in Big Data concepts learned throughout the course.

Wavelets are suitable for the analysis of stock data and time series that have a non stationary

distribution. Several applications were identified for Wavelets in finance in addition to different

components of the Big Data system. Scientific papers on using Wavelets to calculate beta, forecast time

series, Map Reduce, conduct sentiment analysis, text visualization, financial data mining, studying

comovement of stock markets, and clustering techniques. Since several of these topics were discussed

during the course, following is a review of identified research papers.

Ramsey reviewed the contribution of Wavelets to the analysis of economic and financial data (Ramsey,

1996). The author provided suggestions about improving understanding and evaluation of forecasts

using a wavelet approach.

Wavelets were among the discussion points listed in Proceedings of the Fourth World Congress on

Engineering Asset Management (WCEAM) 2009. (Kiritsis, Emmanouilidis, Koronios, & Mathew, 2010).

Sun et al introduced “A new wavelet-based denoising algorithm for high-frequency financial data

Serene Zawaydeh


Page 8 of 24

mining” (Sun & Meinl, 2012). Meanwhile, Giovanis used MATLAB applications for Trading Rules using

Wavelets (Giovanis, 2009).

While investment managers diversify asset allocation across geographies and industries, the global

financial crisis in 2008 started in USA, and its effect spread across stock markets. Interdependence and

correlation between global markets was researched. Sahu et al studied the co-integration of stock

markets using Wavelets and Data mining (R.Sahu). The authors state that “real data for the stock

indices, using tick-by-tick observations obtained, are no longer accepted to be stationary.” Leonel et al

also used Wavelets, and observed the behavior of stock market with its links and correlations using

network and graph theory (Leonel & Yoneyama). Zhao et al developed simulation of a Wavelet neural

network that can forecast stock market returns (Zhao, Zhang, & Qi, 2008). Wavelets were also used to

predict oil prices (Shahriar Yousefi, 2004).

A Wavelet-Based beta estimation of China Stock market was used by Xiong et al (Xiong, Zhang, Zhang, &

Li, 2005) and was linked to behavioral finance. Empirical results showed that the predictions of the

CAPM model are more relevant at short time horizons as compared to long.

Visual Representation of Stock Market Co-Movement using Wavelets

Having a visual representation of the interdependence between the stock markets is useful when

determining the diversification strategy. Basdas investigated the integration of emerging stock markets

over different time horizons using daily data over 1992-2011 (Basdas, 2012). The links among major

Middle East and North African (MENA) stock exchange markets were considered by adopting wavelet

comovement analysis (Rua, 2010). The results indicated that MENA stock markets are partially

integrated and the degree of interdependence increased significantly after 2008 Crisis.

Serene Zawaydeh


Page 9 of 24

Figure 3: Comovement between stock returns

Source: (Basdas, Interaction between MENA Stock Markets: A Comovement Wavelet Analysis, 2012)

Building Wavelet Histograms on Large Data in MapReduce

MapReduce is becoming the de facto framework for storing and processing massive data, due to its

excellent scalability, reliability, and elasticity. In many MapReduce applications, obtaining a compact

accurate summary of data is essential. Among various data summarization tools, histograms have

proven to be particularly important and useful for summarizing data, and the wavelet histogram is one

of the most widely used histograms. Researchers investigated the problem of building wavelet

histograms efficiently on large datasets in MapReduce.

Figure 4: MapReduce

Two-level sampling at mapper Two-level sampling at reducer

A wavelet-based measure

through a contour plot,

representing comovement

across 7 stock exchanges.

The horizontal axis refers

to time (in years) while the

vertical axis refers to

frequency (time units, in

years). In the plots,

deepening red color

corresponds to an

increasing value of

interdependence and

mimics the height in a

surface plot. Changes in

comovement can be

followed over time and

different frequencies.

Red

Red

Red

Red

Red Red

Serene Zawaydeh


Page 10 of 24

Time Series Classification

Time series data are widely seen in analytics (Zhao Y. , 2011), including stock indexes/ prices. However,

classification and clustering of time series data are not readily supported by existing R functions or

packages. Time series classification builds a classification model based on labelled time series and then

uses the model to predict the label of unlabelled time series. The way for time series classification with R

is to extract and build features from time series data first, and then apply existing classification

techniques, such as SVM, k-NN, neural networks, regression and decision trees, to the feature set.

Discrete Wavelet Transform (DWT) provides a multi-resolution representation using wavelets. The

author provided the following example for time series classification using Wavelets.

Figure 5: Time Series Classification

http://i0.wp.com/rdatamining.files.wordpress.com/2011/08/image0131.png

Clustering

Ray and Mallick (Ray & Mallick, 2004) proposed a nonparametric Bayes wavelet model for clustering of

functional data. The wavelet-based methodology is aimed at the resolution of generic global and local

features during clustering and is suitable for clustering high dimensional data.

Data Mining

Pentaho Data Mining Tools and Techniques provide Discrete Wavelet Transform among unsupervised

attribute-based learning techniques used in Weka.

K-Nearest Neighbor – Machine Learning Knn is a non parametric method used for classification and regression. K-NN is a type of instance-based

learning, or lazy learning, where the function is only approximated locally and all computation is

deferred until classification. The K-NN algorithm is the simplest form of machine learning algorithm.

Serene Zawaydeh


Page 11 of 24

(Qiao, Lu, & Sun, 2006) reviewed two fast algorithms based on wavelet transform. References search the

k closest vectors in the wavelet domain. The motivation is that the time for performing the wavelet

transform is low and the energy of the vector is compacted on a few coefficients. The algorithm speeds

up searching k nearest neighbors, which is confirmed with the experimental results.

Knowledge Discovery in Financial Investment

Li & Kuo (Li & Kuo, 2008) used wavelet-based Self Organizing Map (SOM) networks for knowledge

discovery in financial investment for forecasting and trading strategy. The authors proposed a hybrid

approach on the basis of the knowledge discovery methodology by integrating K-chart technical analysis

for feature representation of stock price movements, discrete wavelet transform for feature extraction

to overcome the multi-resolution obstacle, and a novel two-level self-organizing map network for the

underlying forecasting model. A visual trajectory analysis was conducted to reveal the relationship of

movements between primary bull and bear markets and help determine appropriate trading strategies

for short-term investors and trend followers. The resultant intelligent investment model can help

investors, fund managers and investment decision-makers of national stabilization funds make

profitable decisions.

Visualization of Unstructured Text

Miller et al (Miller, Wong, Brewster, & Foote) developed TOPIC-O-GRAPHYTM Technology to provide a

visualization of unstructured text using Wavelets. The patent was granted in 2000.

Patent US 6070133 (Brewster & Miller, 2000) provides an information retrieval system using Wavelet

Transform. The method is for automatically partitioning an unstructured electronically formatted

natural language document into its sub-topic structure. The document is converted to an electronic

signal and a wavelet transform is then performed on the signal. The resultant signal may then be used to

graphically display and interact with the sub-topic structure of the document.

Sentiment Analysis

Patent Application number WO 2011123378 A1 uses Wavelets for sentiment analysis (O'neil, 2011). A

document can be processed to provide sentiment values for phrases in the document. The sequence of

sentiment values associated with the sequence of phrases in a document can be handled as if they were

a sampled discrete time signal. For phrases which have been identified as entities, a filtering operation

can be applied to the sequence of sentiment values around each entity to determine a sentiment value

for the entity.

Serene Zawaydeh


Page 12 of 24

Data Mining in Investment Management

Data Mining – Customer Behavior

Investors’ behaviour and investment strategies can be “learned”, to propose which stocks, and sectors

would be suitable to invest in. In fact, Mak et al developed an intelligent financial data mining model to

extract customer behavior in the financial industry, to increase customer satisfaction. The model

investigated customization of investment portfolio to the customers using clusters. The financial model

clustered the customers into several sectors, then found the correlation between the sectors. This

improves the workflow of a financial company, and deepens the understanding of investment

behaviour. The company can customize the most suitable products and services for customers on the

basis of the rules extracted. (Mak, Ho, & Ting, 2011)

Data Mining and Algorithmic Asset Management

Giovanni and Parrella used data mining for algorithmic asset management using an ensemble learning

approach. Algorithmic asset management refers to the use of expert systems that enter trading orders

without any intervention. Market-neutral systems aim at generating positive returns regardless of

underlying market conditions. The algorithm developed learns the fair price of the security under

management, using the most recent market information acquired by means of streaming financial data.

The difficult issue of learning in non-stationary environments was addressed by adopting an ensemble

learning strategy, where a meta-algorithm strategically combines the opinion of a pool of experts.

Experimental results based on nearly seven years of historical data for the iShare S&P500 ETF

demonstrate that satisfactory risk-adjusted returns can be achieved by the temporal data mining system

after transaction costs. (Giovanni & Parrella)

Serene Zawaydeh


Page 13 of 24

Theoretical Big Data Model

Big data solutions would provide insights to investment managers to quickly make decisions through an

easy to navigate dashboard and data visualization tools can be acceptable by the finance community.

The suggested model would be one that makes use of different applications of Wavelets in finance and

big data: Forecasting time series; Beta estimation; studying interdependence between stock markets to

help with diversification strategies; MapReduce algorithms with Wavelet histograms; data mining with

multi dimensional filtering method; and K-nearest neighbors Wavelet algorithm that speeds up the

search for K nearest neighbor; trading rules; unstructured text visualization; sentiment analysis.

Other components of the system would include: Twitter based trading strategies; and real time

Sentiment Analysis. Stock Sonar provides sentiment analysis for the stock market.

Data visualization technology providers: Some of the companies that provide data visualization tools

are: Streambase, SAS, IntraLinks, Panopticon Software, and Aqumin. (Rodier, 2012)

Compliance with the rules and regulations can be implemented using the patent pending compliance

system US 20130290218 A1. Data about the investors needs to be protected, and security of

transactions needs to be maintained.

Serene Zawaydeh


Page 14 of 24

Figure 6: Theoretical Big Data Analytics Model incorporating Applications of Wavelets

From Data to Dash

Time series forecasting

(Stock Prices, currencies,

economy)

Wavelets

Sentiment Analysis

Real time market data on individual

companies, local and regional stock

markets, local and international

economy, political and organizational

changes

Wavelets

HADOOP

Function of All

the Data

MapReduce

processor

Wavelet

Historgram

Wavelets to calculate Beta

Real time decision on suitability

of investment

Equip portfolio managers with

tools to help in the decision of

assets to buy or sell to which

investor based on investment

strategy

Correlation

Interdependence between

Stock Markets

Wavelets

Data Mining

Algorithmic asset management

Customize portfolio to clients

Data Visualization

Visualization of Unstructured text

Wavelets (Patented)

Trading Rules – Wavelets

Twitter –based trading strategy

Knowledge Discovery –

Wavelet SOM network

Regulatory Compliance

system (Patent Application)

Serene Zawaydeh


Page 15 of 24

Implementation of Big Data Projects

Numerous scientific papers provide results that can be of added value in the investment decision making

process, and can result in higher returns. Turning scientific research into commercialized products that

enable portfolio managers to make decisions based on real time data is needed. Wavelets can help with

forecasting stock prices, and have applications in different stages of the Big Data system, such as

MapReduce, financial data dining to increase customer satisfaction, sentiment analysis, unstructured

text visualization, and trading strategies, while the comovement of stock markets can be used in

diversification strategies.

The implementation of a big data system that incorporates findings from scientific research produced by

researchers across the globe could be the challenge. There is a need for collaborative research. As an

example, while one part of the big data system needs digital signal processing researchers, the final

product needs to provide visualization tools, which might need input from graphic designers. Such a

product will be used by portfolio managers, who come from a finance background. It needs to be on a

secure IT infrastructure that protects investors’ privacy, and therefore needs IT support. It needs to

adhere to the regulatory requirements in the investment industry, and therefore the product need to be

built on a legal framework. Developing a comprehensive system would require bridging the divide

between different domains. It can be costly, as some intellectual property could be patented or could be

patent pending. Developing a big data solution in house would be a challenge.

Statpro North America, a provider of Portfolio Analytics for the Investment community, emphasizes the

need for alliances in big data. Alliances and data aggregators are enablers for “Turning Big Data into a

Dashboard for Investment Managers”. Such alliances would enable portfolio managers and their clients

to get a complete view of a portfolio’s performance, and have total confidence in the completeness of

the information. (Peddar, 2012)

Zitek wonders how long it will be before securities analysts become data scientists (Zitek, 2014), and

describes primary fundamental research as fading away. Data discovery and visualization tools will

replace spreadsheets for identifying dependencies, patterns and trends, valuation analysis, and

investment decision making.

Serene Zawaydeh


Page 16 of 24

Wavelets and Non Stationary Signals

I had used Wavelets for the analysis of biomedical non stationary signals in my digital signal processing –

Electrical Engineering thesis in 1997. I used Discrete Wavelet Analysis for the analysis of non stationary

biomedical ECG (heart beats) signals and heart sounds (PCG). (Zawaydeh, Discrete Wavelet Analysis and

Applications to ECG and PCG Signals, 1997). There is a clear distinction between stationary and non

stationary signals which can be quickly captured when looking at a distribution in the time domain.

Stationary signals have one frequency, and look like sine waves. Meanwhile, non stationary signals, have

different ups and downs, and consequently have more than one frequency.

In 2008, I used Event Study methodology in my MBA thesis to investigate value creation for acquisitions

and divestiture operations for two telecom operators with an internationalization strategy (Zawaydeh,

Etude d’événements sur des operations financières et d'acquisitions et application aux cas de deux

opérateurs de telecom Arabes, 2008). This followed four years of professional research experience on

telecom markets in the Middle East and North Africa (between 2003 and 2004).

I investigated abnormal stock return. Positive abnormal return was an indicator of value creation, and

negative abnormal return indicated destruction of value. At least 40% of the acquisitions created value.

I used the Capital Asset Pricing Model to calculate Beta, the systemic risk coefficient. Beta is calculated

using the Covariance of the stock return, and the market return, divided by the Variance of the market

return.

Beta is used in portfolio management to determine the risk of the asset, and whether it has higher or

lower risk than the market. Negative Beta means lower risk than the market, and Beta higher than 1 is

higher risk than the market. Beta is one of the indicators used when providing information on equities

(stocks).

Changing the period around which Beta is calculated, changes the value of Beta. The theory of the

Capital Asset Pricing Model showed that it is based on the hypothesis of “stationary” returns. (Fama,

1976). Based on my previous research on the applications of Wavelets for the analysis of non stationary

signals, I concluded that the Wavelet Transform could be used to analyze stock data, which has non

uniform ups and downs. I was able to find papers on the applications of Wavelets in Finance.

Los et al used Wavelets in a “Multi-Fractal Spectral Analysis of the 1987 Stock Market Crash”. The

authors state that the most striking result, was that the multifractal spectra of stock market returns are

not stationary. (Los & Yamalova, 2004)

Serene Zawaydeh


Page 17 of 24

Ramazan Gencay (Ramazan Gençay, 2001) authored a book on using Filter Banks in Finance. NAG

highlights the applications of Wavelets in Finance (Tong), and the increasing demand for wavelet

analysis. Gençay et al proposed an alternative multiscale estimator for the systemic risk or beta of an

asset using Wavelets. (Gençay, Selçuk, & Whitcher, 2004)

Figure 7: Changing value of Beta by changing the period around Beta is calculated

Source: (Zawaydeh, Etude d’événements sur des operations financières et d'acquisitions et application

aux cas de deux opérateurs de telecom Arabes, 2008).

19 Level Filter Bank System

Using MATLAB, I designed the following 19 level Filter Bank system. The incoming signal was analyzed

into low frequencies, and high frequencies. The low frequencies were subsequently divided again into

low frequencies and high frequencies, and so on.

The Wavelet Transform can be seen as zooming into the signal to see the details that cannot be

captured by looking at the original non stationary signal in the time domain.

Serene Zawaydeh


Page 18 of 24

Figure 8: 19 Level Filter Bank System for Analyzing ECG Signals using Wavelet Packets

Source: “Discrete Wavelet Analysis and Applications to ECG and PCG Signals”, Serene Zawaydeh, 1997

Input

Signal

Serene Zawaydeh


Page 19 of 24

Figure 9: Spectrum of the Successive Filters using Daubechies 4


Figure 10: Wave Packet Tiling of the Time-Frequency Plane Figure 11: Scaling Function (Low Pass Filter) and Wavelet

Mother (Band Pass Filter)


Gain

Serene Zawaydeh


Page 20 of 24

Bibliography Basdas, U. (2012, August 1). Interaction between MENA Stock Markets: A Comovement Wavelet

Analysis. Retrieved from http://ssrn.com/abstract=2333774

Basdas, U. (2012, August 1). Interaction between MENA Stock Markets: A Comovement Wavelet

Analysis. Retrieved from SSRN: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2333774

Brewster, M., & Miller, N. (2000, May 30). Patent No. 6070133. US.

Buffett , M., & Clark , D. (2002). The New Buffettology: The Proven Techniques for Investing Successfully

in Changing Markets That Have Made Warren Buffett the World's Most Famous Investor. New

York: Rawson Associates.

Cavoukian, A., & Jonas, J. (2012, June 8). Privacy by Design in the Age of Big Data. Retrieved February 4,

2014, from Privacy by Design: http://privacybydesign.ca/content/uploads/2012/06/pbd-

big_data.pdf

Chris, F. (2013, November 13). Acute ‘big data’ challenges facing asset managers. Retrieved March 1,

2014, from http://www.ft.com/cms/s/0/5c6a8e34-4880-11e3-8237-

00144feabdc0.html#ixzz2ulV04ZFG

Daruvala, T. (2013, April). How advanced analytics are redefining banking. Retrieved February 21, 2014,

from McKinsey & Company - Insights and Publications:

http://www.mckinsey.com/insights/business_technology/how_advanced_analytics_are_redefin

ing_banking

de Bever, L., Bachher, J. S., Chuyan, R., & Monk, A. (2013, July 3). Towards the Next Generation of

Performance Attribution for Institutional Investment Management. Retrieved February 21, 2014,

from SSRN: http://ssrn.com/abstract=2289562 or http://dx.doi.org/10.2139/ssrn.2289562

Fama, E. (1976). The Behavior of Stock Market Returns.

Gençay, R., Selçuk, F., & Whitcher, B. (2004). Multiscale systematic risk. Retrieved March 23, 2014, from

http://www.sfu.ca/~rgencay/jarticles/jimf-capm.pdf

GetSmarterAboutMoney.ca. (n.d.). Factors that can affect stock prices . Retrieved March 12, 2014, from

GetSmarterAboutMoney.ca: http://www.getsmarteraboutmoney.ca/en/managing-your-

money/investing/stocks/Pages/Factors-that-can-affect-stock-prices.aspx

Giovanis, E. (2009). MATLAB Applications of Trading Rules and GARCH with Wavelets Analysis.

Giovanni , M., & Parrella, F. (n.d.). Data mining for algorithmic asset management:an ensemble learning

approach. Retrieved March 23, 2014, from Academia.edu:

Serene Zawaydeh


Page 21 of 24

http://www.academia.edu/1054897/Data_mining_for_algorithmic_asset_management_an_ens

emble_learning_approach

Hardoon, D. R., & Shmueli, G. (2013). Getting Started with Business Analytics. Insightful Decision-

Making. FL: CRC Press.

How long before Securities Analyst become Scientists. (n.d.). Retrieved March 22, 2014, from

http://disruptivetechnologyinvestments.com/big-data/how-long-before-securities-analysts-

become-scientists/

Kiritsis, D., Emmanouilidis, C., Koronios, A., & Mathew, J. (2010). Engineering Asset Management.

Retrieved 3 1, 2014, from

http://www.springer.com/engineering/production+engineering/book/978-0-85729-320-6

Kristoufek, L. (2013, October ). Fractal Markets Hypothesis and the Global Financial Crisis: Wavelet

Power Evidence. Retrieved January 29, 2014, from nature.com:

http://www.nature.com/srep/2013/131004/srep02857/full/srep02857.html

Lahmiri, S. (2012). Wavelet Transform, Neural Networks, and the prediction of S&P Price Index: A

comparative study of back propagation numerical algorithms. Retrieved March 23, 2014, from

saycocorporativo.com:

http://www.saycocorporativo.com/saycoUK/BIJ/journal/Vol5No2/Article_4.pdf

Leonel, M. A., & Yoneyama, T. (n.d.). Forecasting short term abrupt changes in the stock market with

wavelet decomposition and graph theory. Retrieved February 2, 2014, from Forecasters.com:

http://www.forecasters.org/proceedings12/CAETANOMARCOISF2012.pdf

Li, S.-T., & Kuo, S.-C. (2008, February). Knowledge discovery in financial investment for forecasting and

trading strategy through wavelet-based SOM networks. Retrieved March 23, 2014, from Science

Direct: http://www.sciencedirect.com/science/article/pii/S0957417406003551

Los, C., & Yamalova, R. (2004). Multi-Fractal Spectral Analysis of the 1987 Stock Market Crash. Retrieved

from http://ssrn.com/abstract=588823

Mak, M. K., Ho, G. T., & Ting, S. (2011, July 23). A Financial Data Mining Model for Extracting Customer

Behaviour. Retrieved March 23, 2014, from http://cdn.intechopen.com/pdfs-wm/17982.pdf

Manyika, J., & al, e. (2011, May). Big data: The next frontier for innovation, competition, and

productivity. Retrieved March 22, 2014, from McKinsey & Company:

http://www.mckinsey.com/insights/business_technology/big_data_the_next_frontier_for_inno

vation

Serene Zawaydeh


Page 22 of 24

Marathe, A., & Shawky, H. (n.d.). Categorizing Mutual Funds using Clusters. Retrieved March 23, 2014,

from citeseerx.ist.psu.edu:

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.202.9593&rep=rep1&type=pdf

McGrath, J. (2014, January 13). Why big data is big business for asset managers. Retrieved March 1,

2014, from http://www.efinancialnews.com/story/2014-01-13/why-big-data-is-big-business-for-

asset-managers?ea9c8a2de0ee111045601ab04d673622

Merwin, J., Higgins, J., Hardy, C., & Marks, K. (2013). Patent No. 13/840,191. US.

Miller, N., Wong, P. C., Brewster, M., & Foote, H. (n.d.). TOPIC ISLANDS – A Wavelet-Based Text

Visualization System. Retrieved March 23, 2014, from http://www-hagen.informatik.uni-

kl.de/vis98/archive/tp/papers/vis98paper.pdf

Millman, R. (2013, April 26). Google big data sets forecast stock market movements, researchers claim.

Retrieved from IT Pro. IT Analysis. Business Insight: http://www.itpro.co.uk/asset-

management/19710/google-big-data-sets-forecast-stock-market-movements-researchers-claim

O'neil, J. (2011, October 6). Patent No. 2011123378 A1. WO.

Peddar, A. (2012, April 19). Turning Big Data into a Dashboard for Investment Managers. Retrieved

February 21, 2014, from http://www.wallstreetandtech.com/data-management/turning-big-

data-into-a-dashboard-for-in/232900574?pgno=2

Peddar, A. (2012, April 19). Turning Big Data into a Dashboard for Investment Managers. Retrieved from

http://www.wallstreetandtech.com/data-management/turning-big-data-into-a-dashboard-for-

in/232900574?pgno=2

Prabakaran, S., Verma, S., & Sahu, R. (2006). A Clustering and Selection Method using Wavelet Power

Spectrum. Retrieved March 23, 2014, from

http://www.iaeng.org/IJCS/issues_v32/issue_4/IJCS_32_4_6.pdf

Qiao, Y.-L., Lu, Z.-M., & Sun, S.-H. (2006). Fast K Nearest Neighbors Search Algorithm Based on Wavelet

Transform. Retrieved March 09, 2014, from

http://www.google.ca/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CC4QFjAA&url=htt

p%3A%2F%2Fwww.paper.edu.cn%2Fselfs%2Fdownpaper%2Fqiaoyulong315734-self-200809-

8&ei=fxkdU6b-

AojmrQGYmoGIBA&usg=AFQjCNGTexkvIB7_zDlQYX2btcS3Xy7Ftg&sig2=PCz7B2RFUnh5wURreO

8R8Q

R.Sahu, P. (n.d.). Co-integration of Stock Markets using Wavelet Theory and Data Mining. Retrieved

February 2, 2014, from Finance Innovation: http://www.finance-

innovation.org/risk08/files/1588125.pdf

Serene Zawaydeh


Page 23 of 24

Ramazan Gençay, F. S. (2001). An Introduction to Wavelets and Other Filtering Methods in Finance and

Economics. Academic Press.

Ramsey, J. (1996). The Contribution of Wavelets to the Analysis of Economic and Financial Data. New

York University, Department of Economics. New York: Royal Society Transcript.

Rao, S. (2013, December 12). Ukraine tests Templeton star Hasenstab's contrarian style. Retrieved

March 1, 2014, from http://www.reuters.com/article/2013/12/12/ukraine-templeton-

hasenstab-idUSL6N0JR2JB20131212

Ray, S., & Mallick, B. (2004, April). Functional clustering by Bayesian wavelet methods. Retrieved March

23, 2014, from people.ee.duke.edu:

http://people.ee.duke.edu/~lcarin/BayesianWaveletsDP.pdf

Rodier, M. (2012). Capital Markets Outlook 2012: Data Visualization. Retrieved March 23, 2014, from

Wallstreet and Technology: http://www.wallstreetandtech.com/2012-outlook/data-

visualization

Shahriar Yousefi, I. W. (2004). Wavelet-based prediction of oil prices. Elsevier, 265-275.

State Street. (2013). Leader or Laggard? How data drives competitive advantage in the Investment

Community. Retrieved March 16, 2014, from

http://www.statestreetglobalexchange.com/downloads/DataFullReport.pdf

StatPro. (2012, August 14). StatPro Revolution. Retrieved March 22, 2014, from

https://www.youtube.com/watch?v=aBUo7oZ7Nzo

Stephan Schlüte, C. D. (2010). Using Wavelets for Time Series Forecasting –Does it Pay Off? Nuremberg:

Department of Statistics and Econometrics, University of Erlangen.

Sun, E., & Meinl, T. (2012, March 16). A new wavelet-based denoising algorithm for high-frequency

financial data mining. Retrieved from

http://www.sciencedirect.com/science/article/pii/S0377221711009027

Tan, C. (2009, May 31). Financial Time Series Forecasting Using Improved Wavelet Neural Network.

Retrieved from http://users-birc.au.dk/cstorm/students/Chong_Jul2009.pdf

TCS. (n.d.). The Emerging Big Returns on Big Data. Retrieved March 16, 2014, from

http://www.tcs.com/SiteCollectionDocuments/Trends_Study/TCS-Big-Data-Global-Trend-Study-

2013.pdf

The 3vs That Define Big Data. (n.d.). Retrieved from Data Science Central:

http://www.datasciencecentral.com/forum/topics/the-3vs-that-define-big-data

Serene Zawaydeh


Page 24 of 24

Tong, R. (n.d.). Wavelet Analysis for Financial Market Data. Retrieved February 2, 2014, from NAG:

http://www.nag.com/IndustryArticles/Wavelet_Analysis_for_Financial_Market_Data.asp

Wang , W., Lu, D., Zhou, X., Zhang, B., & Mu, J. (2013). Statistical wavelet-based anomaly detection in big

data with compressive sensing. Retrieved 2 10, 2014, from

http://jwcn.eurasipjournals.com/content/pdf/1687-1499-2013-269.pdf

Wang, X. (2013). Patent No. 8548574 B2. US.

Witten, I., Frank, E., & Hall, M. (2011). Data Mining. Practical Machine Learning Tools and Techniques.

3rd Ed. Elsevier Inc.

Wood, P. (n.d.). How to tackle big data from a security point of view. Retrieved February 4, 2014, from

Computer Weekly: http://www.computerweekly.com/feature/How-to-tackle-big-data-from-a-

security-point-of-view

Xiong, X., Zhang, X.-T., Zhang, W., & Li, C.-Y. (2005). Wavelet-based beta estimation of China stock

market. Machine Learning and Cybernetics. Proceedings of 2005 International Conference on .

Zawaydeh, S. (1997, December 31). Discrete Wavelet Analysis and Applications to ECG and PCG Signals.

Irbid, Jordan: Jordan University of Science and Technology.

Zawaydeh, S. (2008). Etude d’événements sur des operations financières et d'acquisitions et application

aux cas de deux opérateurs de telecom Arabes. Beirut, Lebanon: Ecole Supérieure des Affaires

(ESA).

Zhao, Y. (2011, 08 23). Time Series Analysis and Mining with R. Retrieved March 23, 2014, from R-

Bloggers: http://www.r-bloggers.com/time-series-analysis-and-mining-with-r/

Zhao, Y., Zhang, Y., & Qi, C. (2008). Prediction Model of Stock Market Returns Based on Wavelet Neural

Network.

Zitek, J. (2014, Jaunary 08). How long before Securities Analyst become Scientists. Retrieved March 22,

2014, from http://disruptivetechnologyinvestments.com/big-data/how-long-before-securities-

analysts-become-scientists/

Date post:	07-Aug-2015
Category:	Data & Analytics
Upload:	serene-zawaydeh
View:	174 times
Download:	0 times

Serene Zawaydeh - Big Data -Investment -Wavelets

Data & Analytics