Topic:

Data Mining Presented by: Basit

Rafiq

Process of sorting through large amounts of data and picking out relevant information

Process of analyzing data from different perspectives and summarizing it into useful information

Discovering hidden value in database

It is non-trivial process of identifying valid, novel, useful and understandable patterns in data

Extracting or mining knowledge from large amounts of data

April 15, 2023Data Mining 2

Data Mining

Why Data Mining?

Data, Data, Data Every where …

I can’t find data I need – data is scattered over network

I can’t get the data I need

I can’t understand the data I need

I can’t use the data I found

April 15, 2023Data Mining 3

Data Mining is the technique used to carry out KDD.

Data Mining turns data into information and then to knowledge

April 15, 2023Data Mining 4

Data Mining Process

Information

Data

Knowledge

Data

War

ehou

se Data M

ining

Why Data Mining? (Cont..)

An abundance of data Super Market Scanners, POS

data Credit cards transactions Call Center records ATM Machines Demographic data Sensor Networks Cameras Web server logs Customer web site trails Geographic Information System National Medical Records Weather Images

This data occupies

Terabytes - 10^12 bytes

Petabytes - 10^15 bytes

Exabytes - 10^18bytes

Zettabytes - 10^21bytes

Zottabytes -10^24bytes

Walmart - 24 Terabytes

April 15, 2023Data Mining 5

April 15, 2023Data Mining 6

History Notes – Many Names of Data Mining

YEAR Names USES

1960 Data Fishing, Data Dredging

Statisticians

1990 Data Mining DB Community, business

1989 Knowledge Discovery in databases

AI, Machine Learning community

Other Names

Data Archaeology, Information Harvesting, Information Discovery, Knowledge Extraction,

Data Warehousing provides the Enterprise with a

memory

April 15, 2023 Data Mining 7

Data Mining provides the Enterprise with intelligence

Data Mining works with Warehouse Data

Why Data Mining?(Cont..)

April 15, 2023Data Mining 8

Data Warehouse is single, complete and consistent store of data from variety of different sources available to end users

For example, AT and T handles billions of calls per day. Europe's Very Long Baseline Interferometer (VLBI) has 16 telescopes, each of which produces 1 Gigabit/second of astronomical data over a 25-day observation session

We need data mining for Transforming data into useful information to users Present data in useful format Provide data access to business analyst, Information

technology professionals

1.Data cleaning To remove noise and inconsistent data

2. Data integrationTo integrate (compile) multiple data

sources3. Data selection

Data relevant to analysis is selected4. Data transformation

Summary normalization aggregation operations are performed (convert data into two dimension form) and consolidate the data

Steps in Data Mining

April 15, 2023Data Mining 9

5. Data miningIntelligent methods are applied to the data to discover knowledge or patterns

6. Pattern evaluationEvaluation of the interesting patterns by thresholding

7. Knowledge DiscoveryVisualization and presentation methods are used to present the mined knowledge to the user.

April 15, 2023Data Mining 10

Steps in Data Mining(Cont..)

◦ Data mining: the core of knowledge discovery process.

Data Cleaning

Data Integration

Databases

Data Warehouse

Task-relevant Data

Selection

Data Mining

Pattern Evaluation

KDD Process

April 15, 2023 11Data Mining

1. Classification• Classification maps data into predefined groups or classes.• It may be represented by methods such as decision trees,

etc.

Decision tree Flow chart like tree structure Each node denotes test of

an attribute value Each branch represents

outcome of test Leaves represent classes

or class distribution.

April 15, 2023Data Mining 12

Data Mining Tasks

2. RegressionUsed to map a data item to a real valued

prediction variable.Example. A manager wants to reach a certain level of savings before

his retirement. Periodically he predicts his retirement savings by current value and several past values. He uses a simple linear regressive formula to predict the values of savings in future.

3. PredictionMany real world applications can be seen

predicting future data states based on

past and current data.Example - Predicting flooding is difficult problem

April 15, 2023Data Mining 13

Data Mining Tasks(Cont..)

4. ClusteringClustering is similar to classification

except that the groups are not predefined.

5. Association RuleAssociation refers to uncovering relationship among data. Used in retail sales community to identify the items (products) that are frequently purchased together.

April 15, 2023Data Mining 14

Data Mining Tasks(Cont..)

1998

Zzzz...

Bread and Jam sell

together!

6. SummarizationSummarization of general characteristics or features of

target class of data. Data characterization presented in various forms - pie charts,

bar charts, curves.Data discrimination comparison of general features of target

class of data objects with general features of objects from one or a set of contrasting classes.

7. Outlier Analysis Database may contain data objects that do not comply with

general behavior model of data. These data objects are called as outliers.

Data mining methods discard outliers as noise or exceptions. In applications such as fraud detection, rare events may be more

interesting than regularly occurring events.

April 15, 2023Data Mining 15

Data Mining Tasks(Cont..)

Relational data and transactional data

Text

Images, video

Mixtures of data

Data Mining: Types of Data

April 15, 2023 16Data Mining

17

Data Mind -- neurOagent Information Discovery SAS Institute -- SAS/ Neuronets

Data Mining Products

April 15, 2023Data Mining

RapidMiner and Weka – Defining data mining process

Top 8 data mining software in 2008

1. Angoss software2. Infor CRM Epiphany3. Portrait Software4. SAS5. SPSS6. ThinkAnalytics7. Unica8. Viscovery

April 15, 2023 18Data Mining

Data Mining Software

Industry ApplicationFinance Credit Card AnalysisInsurance Fraud AnalysisTelecommunication Call record analysis

Application Areas

April 15, 2023 19Data Mining

Data Mining 20

Financial Industry, Banks, Businesses, E-commerce◦ Stock and investment analysis◦ Identify loyal customers and risky customer◦ Predict customer spending

Database analysis and decision support◦ Market analysis and management

target marketing, customer relation management, market basket analysis.

◦ Risk analysis and management Forecasting, quality control, competitive analysis

◦ Fraud detection and management

Applications

April 15, 2023

1. Intelligent Miner

It is IBM data mining product

Distinct feature is include scalability of its mining algorithm and tight integration with IBM DB2 related data base system.

2. DB Miner

Developed by DBMiner Technologies Inc.

Distinct features of DBMiner are Data cube based Online Analytical Mining

Data Mining in Usage

April 15, 2023 21Data Mining

April 15, 2023Data Mining 22

India

Product

Sales Channel

Regio

ns

Retail Direct Special

Household

Telecomm

Video

AudioFar East

Europe

The Telecomm Slice

April 15, 2023Data Mining 23

Data mining: discovering interesting patterns from large amounts of data

A KDD process includes data cleaning, data integration, data selection, transformation, data mining, pattern evaluation, and knowledge presentation

Mining can be performed in a variety of information repositories

Data mining functionalities: characterization, discrimination, association, classification, clustering, outlier etc

Conclusion

April 15, 2023Data Mining 24

Thank you !!!

?????????????????? Any questions ????????????????????????????????????????