Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

Chapter 10

New Application

Slide 10- 2Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

Chapter Outline

Data Mining Data Warehousing Knowledge Discovery in Databases (KDD) Goals of Data Mining and Knowledge Discovery Purpose of Data Warehousing Introduction, Definitions, and Terminology Comparison with Traditional Databases Characteristics of Data Warehouses Classification of Data Warehouses Multimedia Databases

Slide 10- 3Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

Definitions of Data Mining

The discovery of new information in terms of patterns or rules from vast amounts of data.

The process of finding interesting structure in data.

The process of employing one or more computer learning techniques to automatically analyze and extract knowledge from data.

Slide 10- 4Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

Data Warehousing

The data warehouse is a historical database designed for decision support.

Data mining can be applied to the data in a warehouse to help with certain types of decisions.

Proper construction of a data warehouse is fundamental to the successful use of data mining.

Slide 10- 5Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

Knowledge Discovery in Databases (KDD)

Data mining is actually one step of a larger process known as knowledge discovery in databases (KDD).

The KDD process model comprises six phases Data selection Data cleansing Enrichment Data transformation or encoding Data mining Reporting and displaying discovered knowledge

Slide 10- 6Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

Goals of Data Mining and Knowledge Discovery (PICO)

Prediction: Determine how certain attributes will behave in the

future. Identification:

Identify the existence of an item, event, or activity. Classification:

Partition data into classes or categories. Optimization:

Optimize the use of limited resources.

Slide 10- 7Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

Types of Discovered Knowledge

Association Rules Classification Hierarchies Sequential Patterns Patterns Within Time Series Clustering

Slide 10- 8Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

Data Mining Applications

Marketing Marketing strategies and consumer behavior

Finance Fraud detection, creditworthiness and investment

analysis Manufacturing

Resource optimization Health

Image analysis, side effects of drug, and treatment effectiveness

Slide 10- 9Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

Purpose of Data Warehousing Traditional databases are not optimized for data access only they

have to balance the requirement of data access with the need to ensure integrity of data.

Most of the times the data warehouse users need only read access but, need the access to be fast over a large volume of data.

Most of the data required for data warehouse analysis comes from multiple databases and these analysis are recurrent and predictable to be able to design specific software to meet the requirements.

There is a great need for tools that provide decision makers with information to make decisions quickly and reliably based on historical data.

The above functionality is achieved by Data Warehousing and Online analytical processing (OLAP)

Slide 10- 10Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

Introduction, Definitions, and Terminology

W. H Inmon characterized a data warehouse as: “A subject-oriented, integrated, nonvolatile,

time-variant collection of data in support of management’s decisions.”

Slide 10- 11Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

Introduction, Definitions, and Terminology

Data warehouses have the distinguishing characteristic that they are mainly intended for decision support applications.

Traditional databases are transactional. Applications that data warehouse supports are:

OLAP (Online Analytical Processing) is a term used to describe the analysis of complex data from the data warehouse.

DSS (Decision Support Systems) also known as EIS (Executive Information Systems) supports organization’s leading decision makers for making complex and important decisions.

Data Mining is used for knowledge discovery, the process of searching data for unanticipated new knowledge.

Slide 10- 12Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

Conceptual Structure of Data Warehouse

Data Warehouse processing involves Cleaning and reformatting of data OLAP Data Mining

Databases

Data Warehouse

Cleaning Reformatting

Updates/New Data

Back Flushing

Other Data Inputs

OLAP

DataMining

Data

Metadata

DSSIEIS

Slide 10- 13Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

Comparison with Traditional Databases

Data Warehouses are mainly optimized for appropriate data access.

Traditional databases are transactional and are optimized for both access mechanisms and integrity assurance measures.

Data warehouses emphasize more on historical data as their main purpose is to support time-series and trend analysis.

Compared with transactional databases, data warehouses are nonvolatile.

In transactional databases transaction is the mechanism change to the database. By contrast information in data warehouse is relatively coarse grained and refresh policy is carefully chosen, usually incremental.

Slide 10- 14Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

Characteristics of Data Warehouses

Multidimensional conceptual view Generic dimensionality Unlimited dimensions and aggregation levels Unrestricted cross-dimensional operations Dynamic sparse matrix handling Client-server architecture Multi-user support Accessibility Transparency Intuitive data manipulation Consistent reporting performance Flexible reporting

Slide 10- 15Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

Classification of Data Warehouses

Generally, Data Warehouses are an order of magnitude larger than the source databases.

The sheer volume of data is an issue, based on which Data Warehouses could be classified as follows.

Enterprise-wide data warehouses They are huge projects requiring massive investment of time

and resources. Virtual data warehouses

They provide views of operational databases that are materialized for efficient access.

Data marts These are generally targeted to a subset of organization, such

as a department, and are more tightly focused.

Slide 10- 16Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

2 Multimedia Databases

In the years ahead multimedia information systems are expected to dominate our daily lives. Our houses will be wired for bandwidth to handle

interactive multimedia applications. Our high-definition TV/computer workstations will

have access to a large number of databases, including digital libraries, image and video databases that will distribute vast amounts of multisource multimedia content.

Slide 10- 17Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

2.1 Multimedia Databases

DBMSs have been constantly adding to the types of data they support.

Today many types of multimedia data are available in current systems.

Slide 10- 18Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

2.1 Multimedia Databases(2)

Types of multimedia data are available in current systems Text: May be formatted or unformatted. For ease

of parsing structured documents, standards like SGML and variations such as HTML are being used.

Graphics: Examples include drawings and illustrations that are encoded using some descriptive standards (e.g. CGM, PICT, postscript).

Slide 10- 19Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

2.1 Multimedia Databases(3)

Types of multimedia data are available in current systems (contd.) Images: Includes drawings, photographs, and so

forth, encoded in standard formats such as bitmap, JPEG, and MPEG. Compression is built into JPEG and MPEG.

These images are not subdivided into components. Hence querying them by content (e.g., find all images containing circles) is nontrivial.

Animations: Temporal sequences of image or graphic data.

Slide 10- 20Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

2.1 Multimedia Databases(4)

Types of multimedia data are available in current systems (contd.) Video: A set of temporally sequenced

photographic data for presentation at specified rates– for example, 30 frames per second.

Structured audio: A sequence of audio components comprising note, tone, duration, and so forth.

Slide 10- 21Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

2.1 Multimedia Databases(5)

Types of multimedia data are available in current systems (contd.) Audio: Sample data generated from aural

recordings in a string of bits in digitized form. Analog recordings are typically converted into digital form before storage.

Slide 10- 22Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

2.1 Multimedia Databases(6)

Types of multimedia data are available in current systems (contd.) Composite or mixed multimedia data: A

combination of multimedia data types such as audio and video which may be physically mixed to yield a new storage format or logically mixed while retaining original types and formats. Composite data also contains additional control information describing how the information should be rendered.

Slide 10- 23Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

2.1 Multimedia Databases(7)

Nature of Multimedia Applications: Multimedia data may be stored, delivered, and

utilized in many different ways. Applications may be categorized based on their

data management characteristics.

Slide 10- 24Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

2.1 Multimedia Databases(8)

Characterization of applications based on their data management characteristics: Repository applications: A large amount of

multimedia data as well as metadata is stored for retrieval purposes. Examples include repositories of satellite images, engineering drawings and designs, space photographs, and radiology scanned pictures.

Slide 10- 25Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

2.1 Multimedia Databases(9)

Characterization of applications based on their data management characteristics (contd.):

Presentation applications: A large amount of applications involve delivery of multimedia data subject to temporal constraints; simple multimedia viewing of video data, for example, requires a system to simulate VCR-like functionality. Complex and interactive multimedia presentations involve orchestration directions to control the retrieval order of components in a series or in parallel. Interactive environments must support capabilities such as real-time editing analysis or annotating of video and audio data.

Slide 10- 26Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

2.1 Multimedia Databases(10)

Characterization of applications based on their data management characteristics: Collaborative work using multimedia information:

This is a new category of applications in which engineers may execute a complex design task by merging drawings, fitting subjects to design constraints, and generating new documentation, change notifications, and so forth. Intelligent healthcare networks as well as telemedicine will involve doctors collaborating among themselves, analyzing multimedia patient data and information in real time as it is generated.

Slide 10- 27Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

2.2 Data Management Issues

Multimedia applications dealing with thousands of images, documents, audio and video segments, and free text data depend critically on Appropriate modeling of the structure and content

of data Designing appropriate database schemas for

storing and retrieving multimedia information.

Slide 10- 28Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

2.2 Data Management Issues(2)

Multimedia information systems are very complex and embrace a large set of issues: Modeling

Complex objects Design

Conceptual, logical, and physical design of multimedia has not been addressed fully.

Slide 10- 29Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

2.2 Data Management Issues(3)

Multimedia information systems are very complex and embrace a large set of issues (contd.): Storage

Multimedia data on standard disklike devices presents problems of representation, compression, mapping to device hierarchies, archiving, and buffering during the input/output operation.

Queries and retrieval “Database” way of retrieving information is based on

query languages and internal index structures.

Slide 10- 30Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

2.2 Data Management Issues(4)

Multimedia information systems are very complex and embrace a large set of issues (contd.): Performance

Multimedia applications involving only documents and text, performance constraints are subjectively determined by the user.

Applications involving video playback or audio-video synchronization, physical limitations dominate.

Slide 10- 31Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe

2.3 Multimedia Database Applications

Large-scale applications of multimedia databases can be expected encompasses a large number of disciplines and enhance existing capabilities. Documents and records management Knowledge dissemination Education and training Marketing, advertising, retailing, entertainment,

and travel Real-time control and monitoring