Modern database management jeffrey a. hoffer, mary b. prescott,

Chapter 1 © 2007 by Prentice Hall© 2007 by Prentice Hall

Grading SystemGrading System

Lecture GradeLecture Grade 11stst Exam Exam -- 10%10% Ch 1 – 2 Ch 1 – 2 22ndnd Exam Exam -- 10%10% Ch 3 – 5Ch 3 – 5 33rdrd Exam Exam -- 10%10% Ch 7 – 8 (SQL)Ch 7 – 8 (SQL) 44thth Exam Exam -- 15%15% OverallOverall ProjectProject -- 15%15% Q/A/EtcQ/A/Etc -- 40%40%

TOTALTOTAL -- 100%100% * * .75.75

11

Chapter 1 © 2007 by Prentice Hall© 2007 by Prentice Hall

Laboratory GradeLaboratory Grade Laboratory ExercisesLaboratory Exercises -- 10%10% Hands – on ExamHands – on Exam -- 15 %15 %

TOTALTOTAL -- 25%25%

GRADE = LEC + LAB = 75% + 25% = GRADE = LEC + LAB = 75% + 25% = 100%100%

22

© 2007 by Prentice Hall© 2007 by Prentice Hall 33

Chapter 1:Chapter 1:The Database The Database EnvironmentEnvironment

Modern Database Modern Database ManagementManagement

88thth Edition EditionJeffrey A. Hoffer, Mary B. Prescott, Jeffrey A. Hoffer, Mary B. Prescott,

Fred R. McFaddenFred R. McFadden

Chapter 1 © 2007 by Prentice Hall© 2007 by Prentice Hall 44

ObjectivesObjectives Definition of termsDefinition of terms Explain growth and importance of databasesExplain growth and importance of databases Name limitations of conventional file Name limitations of conventional file

processingprocessing Identify five categories of databasesIdentify five categories of databases Explain advantages of databasesExplain advantages of databases Identify costs and risks of databasesIdentify costs and risks of databases List components of database environmentList components of database environment Describe evolution of database systemsDescribe evolution of database systems


DefinitionsDefinitions Database: organized collection of logically Database: organized collection of logically

related datarelated data Data: stored representations of meaningful Data: stored representations of meaningful

objects and eventsobjects and events Structured: numbers, text, datesStructured: numbers, text, dates Unstructured: images, video, documentsUnstructured: images, video, documents

Information: data processed to increase Information: data processed to increase knowledge in the person using the dataknowledge in the person using the data

Metadata: data that describes the properties Metadata: data that describes the properties and context of user dataand context of user data


Figure 1-1a Data in context

Context helps users understand data


Graphical displays turn data into useful information that managers can use for decision making and

interpretation

Figure 1-1b Summarized data


Descriptions of the properties or characteristics of the data, including data types, field sizes, allowable values, and

data context


Disadvantages of File Disadvantages of File ProcessingProcessing

Program-Data DependenceProgram-Data Dependence All programs maintain metadata for each file they useAll programs maintain metadata for each file they use

Duplication of DataDuplication of Data Different systems/programs have separate copies of the Different systems/programs have separate copies of the

same datasame data

Limited Data SharingLimited Data Sharing No centralized control of dataNo centralized control of data

Lengthy Development TimesLengthy Development Times Programmers must design their own file formatsProgrammers must design their own file formats

Excessive Program MaintenanceExcessive Program Maintenance 80% of information systems budget80% of information systems budget


Problems with Data Problems with Data DependencyDependency

Each application programmer must Each application programmer must maintain his/her own datamaintain his/her own data

Each application program needs to Each application program needs to include code for the metadata of each include code for the metadata of each filefile

Each application program must have its Each application program must have its own processing routines for reading, own processing routines for reading, inserting, updating, and deleting datainserting, updating, and deleting data

Lack of coordination and central controlLack of coordination and central control Non-standard file formatsNon-standard file formats


Figure 1-3 Old file processing systems at Pine Valley Furniture Company

Duplicate Data


Problems with Data Problems with Data RedundancyRedundancy

Waste of space to have duplicate dataWaste of space to have duplicate data Causes more maintenance headachesCauses more maintenance headaches The biggest problem: The biggest problem:

Data changes in one file could cause Data changes in one file could cause inconsistenciesinconsistencies

Compromises in Compromises in data integritydata integrity


SOLUTION: SOLUTION: The DATABASE ApproachThe DATABASE Approach

Central repository of shared dataCentral repository of shared data Data is managed by a controlling Data is managed by a controlling

agentagent Stored in a standardized, Stored in a standardized,

convenient formconvenient form

Requires a Database Management System (DBMS)


Database Management Database Management SystemSystem

DBMS manages data resources like an operating system manages hardware resources

A software system that is used to create, maintain, and provide controlled access to user databases

Order Filing System

Invoicing System

Payroll System

DBMSCentral database

Contains employee,order, inventory,

pricing, and customer data


Advantages of the Database Advantages of the Database ApproachApproach

Program-data independenceProgram-data independence Planned data redundancyPlanned data redundancy Improved data consistencyImproved data consistency Improved data sharingImproved data sharing Increased application development Increased application development

productivityproductivity Enforcement of standardsEnforcement of standards Improved data qualityImproved data quality Improved data accessibility and Improved data accessibility and

responsivenessresponsiveness Reduced program maintenanceReduced program maintenance Improved decision supportImproved decision support


Costs and Risks of the Database Costs and Risks of the Database ApproachApproach

New, specialized personnelNew, specialized personnel Installation and management cost Installation and management cost

and complexityand complexity Conversion costsConversion costs Need for explicit backup and Need for explicit backup and

recoveryrecovery Organizational conflictOrganizational conflict


Elements of the Database Elements of the Database ApproachApproach

Data models Data models Graphical system capturing nature and relationship of dataGraphical system capturing nature and relationship of data Enterprise Data Model–high-level entities and relationships Enterprise Data Model–high-level entities and relationships

for the organizationfor the organization Project Data Model–more detailed view, matching data Project Data Model–more detailed view, matching data

structure in database or data warehouse structure in database or data warehouse Relational DatabasesRelational Databases

Database technology involving tables (relations) Database technology involving tables (relations) representing entities and primary/foreign keys representing entities and primary/foreign keys representing relationshipsrepresenting relationships

Use of Internet TechnologyUse of Internet Technology Networks and telecommunications, distributed databases, Networks and telecommunications, distributed databases,

client-server, and 3-tier architecturesclient-server, and 3-tier architectures Database ApplicationsDatabase Applications

Application programs used to perform database activities Application programs used to perform database activities (create, read, update, and delete) for database users(create, read, update, and delete) for database users


Segment of an Enterprise Data Model

Segment of a Project-Level Data Model


One customer may place many orders, but each order is placed by a single customer

One-to-many relationship


One order has many order lines; each order line is associated with a single order



One product can be in many order lines, each order line refers to a single product



Therefore, one order involves many products and one product is involved in many orders

Many-to-many relationship


Figure 1-4 Enterprise data model for Figure 1-3 segments


Figure 1-5 Components of the Database Environment


Components of the Components of the Database EnvironmentDatabase Environment

CASE ToolsCASE Tools––computer-aided software engineeringcomputer-aided software engineering RepositoryRepository––centralized storehouse of metadatacentralized storehouse of metadata Database Management System (DBMS) Database Management System (DBMS) ––

software for managing the databasesoftware for managing the database DatabaseDatabase––storehouse of the datastorehouse of the data Application ProgramsApplication Programs––software using the datasoftware using the data User InterfaceUser Interface––text and graphical displays to text and graphical displays to

usersusers Data/Database AdministratorsData/Database Administrators––personnel personnel

responsible for maintaining the databaseresponsible for maintaining the database System DevelopersSystem Developers––personnel responsible for personnel responsible for

designing databases and softwaredesigning databases and software End UsersEnd Users––people who use the applications and people who use the applications and

databasesdatabases


The Range of Database The Range of Database ApplicationsApplications

Personal databasesPersonal databases Workgroup databasesWorkgroup databases Departmental/divisional databasesDepartmental/divisional databases Enterprise databaseEnterprise database



Figure 1-6Typical data from a personal database


Figure 1-7 Workgroup database with wireless local area network


Enterprise Database Enterprise Database ApplicationsApplications

Enterprise Resource Planning (ERP)Enterprise Resource Planning (ERP) Integrate all enterprise functions Integrate all enterprise functions

(manufacturing, finance, sales, marketing, (manufacturing, finance, sales, marketing, inventory, accounting, human resources)inventory, accounting, human resources)

Data WarehouseData Warehouse Integrated decision support system derived Integrated decision support system derived

from various operational databasesfrom various operational databases


Figure 1-8 An enterprise data warehouse


Evolution of DB SystemsEvolution of DB Systems

© 2007 by Prentice Hall

3333

Chapter 2:Chapter 2: The Database The Database

Development Process Development Process





ObjectivesObjectives Definition of termsDefinition of terms Describe system development life cycleDescribe system development life cycle Explain prototyping approachExplain prototyping approach Explain roles of individualsExplain roles of individuals Explain three-schema approachExplain three-schema approach Explain role of packaged data modelsExplain role of packaged data models Explain three-tiered architecturesExplain three-tiered architectures Explain scope of database design projectsExplain scope of database design projects Draw simple data modelsDraw simple data models


Enterprise Data ModelEnterprise Data Model

First step in database developmentFirst step in database development Specifies scope and general contentSpecifies scope and general content Overall picture of organizational data at Overall picture of organizational data at

high level of abstractionhigh level of abstraction Entity-relationship diagramEntity-relationship diagram Descriptions of entity typesDescriptions of entity types Relationships between entitiesRelationships between entities Business rulesBusiness rules


Figure 2-1 Segment from enterprise data model

Enterprise data model describes the high-level entities in an

organization and the relationship between

these entities


Information Systems Information Systems ArchitectureArchitecture

(ISA)(ISA) Conceptual blueprint for organization’s Conceptual blueprint for organization’s

desired information systems structuredesired information systems structure Consists of:Consists of:

Data (e.g. Enterprise Data ModelData (e.g. Enterprise Data Model––simplified ER simplified ER Diagram)Diagram)

ProcessesProcesses––data flow diagrams, process data flow diagrams, process decomposition, etc.decomposition, etc.

Data NetworkData Network––topology diagram (like Fig 1-9)topology diagram (like Fig 1-9) PeoplePeople––people management using project people management using project

management tools (Gantt charts, etc.)management tools (Gantt charts, etc.) Events and points in time (when processes are Events and points in time (when processes are

performed)performed) Reasons for events and rules (e.g., decision Reasons for events and rules (e.g., decision

tables)tables)


Information EngineeringInformation Engineering A data-oriented methodology to create A data-oriented methodology to create

and maintain information systemsand maintain information systems Top-down planning–a generic IS Top-down planning–a generic IS

planning methodology for obtaining a planning methodology for obtaining a broad understanding of the IS needed broad understanding of the IS needed by the entire organizationby the entire organization

Four steps to Top-Down planning:Four steps to Top-Down planning: PlanningPlanning AnalysisAnalysis DesignDesign ImplementationImplementation


Information Systems Information Systems PlanningPlanning

(Table 2-1) (Table 2-1) PurposePurpose––align information align information

technology with organization’s technology with organization’s business strategiesbusiness strategies

Three steps:Three steps:1.1. Identify strategic planning factors Identify strategic planning factors

2.2. Identify corporate planning objectsIdentify corporate planning objects

3.3. Develop enterprise modelDevelop enterprise model


Identify Strategic Planning Identify Strategic Planning Factors (Table 2-2)Factors (Table 2-2)

Organization goals–what we hope to Organization goals–what we hope to accomplishaccomplish

Critical success factors–what MUST Critical success factors–what MUST work in order for us to survivework in order for us to survive

Problem areas–weaknesses we now Problem areas–weaknesses we now havehave


Identify Corporate Planning Identify Corporate Planning Objects (Table 2-3)Objects (Table 2-3)

Organizational units–departmentsOrganizational units–departments Organizational locationsOrganizational locations Business functions–groups of business Business functions–groups of business

processesprocesses Entity types–the things we are trying Entity types–the things we are trying

to model for the databaseto model for the database Information systems–application Information systems–application

programsprograms


Develop Enterprise ModelDevelop Enterprise Model

Functional decompositionFunctional decomposition Iterative process breaking system Iterative process breaking system

description into finer and finer detaildescription into finer and finer detail Enterprise data model Enterprise data model

Planning matrixes Planning matrixes Describe interrelationships Describe interrelationships

between planning objectsbetween planning objects


Figure 2-2 Example of process decomposition of an order fulfillment function (Pine Valley Furniture)

Decomposition = breaking large tasks into smaller tasks

in a hierarchical structure chart


Planning MatrixesPlanning Matrixes Describe relationships between Describe relationships between

planning objects in the organizationplanning objects in the organization Types of matrixes:Types of matrixes:

Function-to-data entityFunction-to-data entity Location-to-functionLocation-to-function Unit-to-functionUnit-to-function IS-to-data entityIS-to-data entity Supporting function-to-data entitySupporting function-to-data entity IS-to-business objectiveIS-to-business objective


Example business function-Example business function-to-data entity matrix (Fig. 2-to-data entity matrix (Fig. 2-

3)3)


Two Approaches to Two Approaches to Database and IS Database and IS

DevelopmentDevelopment SDLCSDLC System Development Life CycleSystem Development Life Cycle Detailed, well-planned development processDetailed, well-planned development process Time-consuming, but comprehensiveTime-consuming, but comprehensive Long development cycleLong development cycle

PrototypingPrototyping Rapid application development (RAD)Rapid application development (RAD) Cursory attempt at conceptual data modelingCursory attempt at conceptual data modeling Define database during development of initial prototypeDefine database during development of initial prototype Repeat implementation and maintenance activities with Repeat implementation and maintenance activities with

new prototype versionsnew prototype versions


Systems Development Life CycleSystems Development Life Cycle(see also Figures 2.4, 2.5) (see also Figures 2.4, 2.5)

Planning

Analysis

Physical Design

Implementation

Maintenance

Logical Design


Systems Development Life CycleSystems Development Life Cycle(see also Figures 2.4, 2.5) (see also Figures 2.4, 2.5)

(cont.)(cont.)Planning

Analysis

Physical Design

Implementation

Maintenance

Logical Design

Planning Purpose––preliminary understandingDeliverable––request for study

Database activity–– enterprise modeling

and early conceptual data modeling


Systems Development Life CycleSystems Development Life Cycle(see also Figures 2.4, 2.5) (cont.) (see also Figures 2.4, 2.5) (cont.)

Planning

Analysis

Physical Design

Implementation

Maintenance

Logical Design

Analysis

Purpose–thorough requirements analysis and structuring

Deliverable–functional system specifications

Database activity–Thorough and integrated conceptual

data modeling



Planning

Analysis

Physical Design

Implementation

Maintenance

Logical DesignLogical Design

Purpose–information requirements elicitation and structure

Deliverable–detailed design specifications

Database activity– logical database design

(transactions, forms, displays, views, data

integrity and security)



Planning

Analysis

Physical Design

Implementation

Maintenance

Logical Design

Physical Design

Purpose–develop technology and organizational specifications

Deliverable–program/data structures, technology purchases,

organization redesigns

Database activity– physical database design

(define database to DBMS, physical data organization,

database processing programs)



Planning

Analysis

Physical Design

Implementation

Maintenance

Logical Design

Implementation

Purpose–programming, testing, training, installation, documenting

Deliverable–operational programs, documentation, training materials

Database activity– database implementation,

including coded programs, documentation,

installation and conversion



Planning

Analysis

Physical Design

Implementation

Maintenance

Logical Design

Maintenance

Purpose–monitor, repair, enhanceDeliverable–periodic audits

Database activity– database maintenance,

performance analysis and tuning, error

corrections


Prototyping Database Prototyping Database MethodologyMethodology(Figure 2.6) (Figure 2.6)


Prototyping Database Prototyping Database MethodologyMethodology

(Figure 2.6) (Figure 2.6) (cont.)(cont.)



(Figure 2.6)(Figure 2.6) (cont.) (cont.)



(Figure 2.6)(Figure 2.6) (cont.) (cont.)



(Figure 2.6) (Figure 2.6) (cont.)(cont.)


CASECASE Computer-Aided Software Engineering Computer-Aided Software Engineering

(CASE)–software tools providing automated (CASE)–software tools providing automated support for systems developmentsupport for systems development

Three database features:Three database features: Data modeling–drawing entity-relationship Data modeling–drawing entity-relationship

diagramsdiagrams Code generation–SQL code for table creationCode generation–SQL code for table creation Repositories–knowledge base of enterprise Repositories–knowledge base of enterprise

informationinformation


Packaged Data ModelsPackaged Data Models

Model components that can be purchased, Model components that can be purchased, customized, and assembled into full-scale customized, and assembled into full-scale data modelsdata models

AdvantagesAdvantages Reduced development timeReduced development time Higher model quality and reliabilityHigher model quality and reliability

Two types:Two types: Universal data modelsUniversal data models Industry-specific data modelsIndustry-specific data models


Managing ProjectsManaging Projects Project–a planned undertaking of related Project–a planned undertaking of related

activities to reach an objective that has a activities to reach an objective that has a beginning and an endbeginning and an end

Involves use of review points for:Involves use of review points for: Validation of satisfactory progressValidation of satisfactory progress Step back from detail to overall viewStep back from detail to overall view Renew commitment of stakeholdersRenew commitment of stakeholders

Incremental commitment–review of systems Incremental commitment–review of systems development project after each development development project after each development phase with rejustification after each phasephase with rejustification after each phase


Managing Projects: People Managing Projects: People InvolvedInvolved

Business analystsBusiness analysts Systems analystsSystems analysts Database analysts and data modelersDatabase analysts and data modelers UsersUsers ProgrammersProgrammers Database architectsDatabase architects Data administratorsData administrators Project managersProject managers Other technical expertsOther technical experts


Database SchemaDatabase Schema Physical Schema Physical Schema

Physical structures–covered in Chapters 5 and 6Physical structures–covered in Chapters 5 and 6 Conceptual SchemaConceptual Schema

E-R models–covered in Chapters 3 and 4E-R models–covered in Chapters 3 and 4 External SchemaExternal Schema

User ViewsUser Views Subsets of Conceptual SchemaSubsets of Conceptual Schema Can be determined from business-function/data Can be determined from business-function/data

entity matricesentity matrices DBA determines schema for different usersDBA determines schema for different users


Different people have different

views of the database…these are the external

schema

The internal schema is the

underlying design and

implementation

Figure 2-7 Three-schema architecture


Figure 2-8 Developing the three-tiered architecture


Figure 2-9 Three-tiered client/server database architecture


Pine Valley FurniturePine Valley Furniture

Segment of project data model (Figure 2-11)


Figure 2-12 Four relations (Pine Valley Furniture)


Figure 2-12 Four relations (Pine Valley Furniture) (cont.)


Chapter 3:Chapter 3:Modeling Data in the Modeling Data in the

OrganizationOrganization

Modern Database Modern Database ManagementManagement88thth Edition Edition

Jeffrey A. Hoffer, Mary B. Prescott, Jeffrey A. Hoffer, Mary B. Prescott, Fred R. McFaddenFred R. McFadden


ObjectivesObjectives Definition of termsDefinition of terms Importance of data modelingImportance of data modeling Write good names and definitions for entities, Write good names and definitions for entities,

relationships, and attributesrelationships, and attributes Distinguish unary, binary, and ternary relationshipsDistinguish unary, binary, and ternary relationships Model different types of attributes, entities, Model different types of attributes, entities,

relationships, and cardinalitiesrelationships, and cardinalities Draw E-R diagrams for common business situationsDraw E-R diagrams for common business situations Convert many-to-many relationships to associative Convert many-to-many relationships to associative

entitiesentities Model time-dependent data using time stampsModel time-dependent data using time stamps


Business RulesBusiness Rules

Statements that define or constrain Statements that define or constrain some aspect of the businesssome aspect of the business

Assert business structureAssert business structure Control/influence business behaviorControl/influence business behavior Expressed in terms familiar to end Expressed in terms familiar to end

usersusers Automated through DBMS softwareAutomated through DBMS software


A Good Business Rule is:A Good Business Rule is:

Declarative–what, not howDeclarative–what, not how Precise–clear, agreed-upon meaningPrecise–clear, agreed-upon meaning Atomic–one statementAtomic–one statement Consistent–internally and externallyConsistent–internally and externally Expressible–structured, natural Expressible–structured, natural

languagelanguage Distinct–non-redundantDistinct–non-redundant Business-oriented–understood by Business-oriented–understood by

business peoplebusiness people


A Good Data Name is:A Good Data Name is:

Related to business, not technical, Related to business, not technical, characteristicscharacteristics

Meaningful and self-documentingMeaningful and self-documenting UniqueUnique ReadableReadable Composed of words from an approved Composed of words from an approved

listlist RepeatableRepeatable


Data DefinitionsData Definitions

Explanation of a term or factExplanation of a term or fact Term–word or phrase with specific meaningTerm–word or phrase with specific meaning Fact–association between two or more termsFact–association between two or more terms

Guidelines for good data definitionGuidelines for good data definition Gathered in conjunction with systems Gathered in conjunction with systems

requirementsrequirements Accompanied by diagramsAccompanied by diagrams Iteratively created and refinedIteratively created and refined Achieved by consensusAchieved by consensus


E-R Model ConstructsE-R Model Constructs Entities:Entities:

Entity instance–person, place, object, event, concept (often Entity instance–person, place, object, event, concept (often corresponds to a row in a table)corresponds to a row in a table)

Entity Type–collection of entities (often corresponds to a Entity Type–collection of entities (often corresponds to a table)table)

Relationships:Relationships: Relationship instance–link between entities (corresponds to Relationship instance–link between entities (corresponds to

primary key-foreign key equivalencies in related tables)primary key-foreign key equivalencies in related tables) Relationship type–category of relationship…link between Relationship type–category of relationship…link between

entity typesentity types

Attribute–Attribute–property or characteristic of an entity or property or characteristic of an entity or relationship type (often corresponds to a field in a table)relationship type (often corresponds to a field in a table)


Sample E-R Diagram (Figure 3-1)


Relationship degrees specify

number of entity types

involved

Entity symbols

A special entity that is also a relationship

Relationship symbols

Relationship cardinalities specify how

many of each entity type is

allowed

Attribute symbols

Basic E-R notation (Figure 3-2)


What Should an Entity Be?What Should an Entity Be? SHOULD BE:SHOULD BE:

An object that will have many An object that will have many instances in the databaseinstances in the database

An object that will be composed of An object that will be composed of multiple attributesmultiple attributes

An object that we are trying to modelAn object that we are trying to model SHOULD NOT BE:SHOULD NOT BE:

A user of the database system A user of the database system An output of the database system An output of the database system

(e.g., a report)(e.g., a report)


Inappropriate entities

System System useruser

System System outputoutput

Figure 3-4 Example of inappropriate entities

Appropriate entities


AttributesAttributes Attribute–property or characteristic Attribute–property or characteristic

of an entity or relationahip typeof an entity or relationahip type Classifications of attributes:Classifications of attributes:

Required versus Optional AttributesRequired versus Optional Attributes Simple versus Composite AttributeSimple versus Composite Attribute Single-Valued versus Multivalued Single-Valued versus Multivalued

AttributeAttribute Stored versus Derived AttributesStored versus Derived Attributes Identifier AttributesIdentifier Attributes


Identifiers (Keys)Identifiers (Keys)

Identifier (Key)–An attribute (or Identifier (Key)–An attribute (or combination of attributes) that combination of attributes) that uniquely identifies individual uniquely identifies individual instances of an entity typeinstances of an entity type

Simple versus Composite IdentifierSimple versus Composite Identifier Candidate Identifier–an attribute that Candidate Identifier–an attribute that

could be a key…satisfies the could be a key…satisfies the requirements for being an identifierrequirements for being an identifier


Characteristics of IdentifiersCharacteristics of Identifiers

Will not change in valueWill not change in value Will not be nullWill not be null No intelligent identifiers (e.g., No intelligent identifiers (e.g.,

containing locations or people that containing locations or people that might change)might change)

Substitute new, simple keys for long, Substitute new, simple keys for long, composite keyscomposite keys


Figure 3-7 A composite attribute

An attribute broken into

component parts

Figure 3-8 Entity with multivalued attribute (Skill) and derived attribute (Years_Employed)

Multivaluedan employee can have

more than one skillDerived

from date employed

and current date


Figure 3-9 Simple and composite identifier attributes

The identifier is boldfaced and underlined


Figure 3-19 Simple example of time-stamping

This attribute that is both multivalued

and composite


More on RelationshipsMore on Relationships Relationship Types vs. Relationship InstancesRelationship Types vs. Relationship Instances

The relationship type is modeled as lines between The relationship type is modeled as lines between entity types…the instance is between specific entity types…the instance is between specific entity instancesentity instances

Relationships can have attributesRelationships can have attributes These describe features pertaining to the association These describe features pertaining to the association

between the entities in the relationshipbetween the entities in the relationship

Two entities can have more than one type of Two entities can have more than one type of relationship between them (multiple relationship between them (multiple relationships)relationships)

Associative Entity–combination of Associative Entity–combination of relationship and entityrelationship and entity


Figure 3-10 Relationship types and instances

a) Relationship type

b) Relationship

instances


Degree of RelationshipsDegree of Relationships

Degree of a relationship is Degree of a relationship is the number of entity types the number of entity types that participate in itthat participate in itUnary RelationshipUnary RelationshipBinary RelationshipBinary RelationshipTernary RelationshipTernary Relationship


Degree of relationships – from Figure 3-2

Entities of two different types related to each other Entities of three

different types related to each

other

One entity related to

another of the same

entity type


Cardinality of RelationshipsCardinality of Relationships

One-to-OneOne-to-One Each entity in the relationship will have exactly Each entity in the relationship will have exactly

one related entityone related entity One-to-ManyOne-to-Many

An entity on one side of the relationship can An entity on one side of the relationship can have many related entities, but an entity on have many related entities, but an entity on the other side will have a maximum of one the other side will have a maximum of one related entityrelated entity

Many-to-ManyMany-to-Many Entities on both sides of the relationship can Entities on both sides of the relationship can

have many related entities on the other sidehave many related entities on the other side


Cardinality ConstraintsCardinality Constraints Cardinality Constraints - the number of Cardinality Constraints - the number of

instances of one entity that can or must instances of one entity that can or must be associated with each instance of be associated with each instance of another entityanother entity

Minimum CardinalityMinimum Cardinality If zero, then optionalIf zero, then optional If one or more, then mandatoryIf one or more, then mandatory

Maximum CardinalityMaximum Cardinality The maximum numberThe maximum number


Figure 3-12 Examples of relationships of different degrees

a) Unary relationships


Figure 3-12 Examples of relationships of different degrees (cont.)

b) Binary relationships


Figure 3-12 Examples of relationships of different degrees (cont.)

c) Ternary relationship

Note: a relationship can have attributes of its own


Figure 3-17 Examples of cardinality constraints

a) Mandatory cardinalities

A patient must have recorded at least one history, and can have

many

A patient history is recorded for one and

only one patient


Figure 3-17 Examples of cardinality constraints (cont.)

b) One optional, one mandatory

An employee can be assigned to any number

of projects, or may not be assigned to any at all

A project must be assigned to at least one employee, and may be assigned to

many


Figure 3-17 Examples of cardinality constraints (cont.)

a) Optional cardinalities

A person is is married to at

most one other person, or may not be married

at all


Entities can be related to one another in more than one way

Figure 3-21 Examples of multiple relationships

a) Employees and departments


Figure 3-21 Examples of multiple relationships (cont.)

b) Professors and courses (fixed lower limit constraint)

Here, min cardinality

constraint is 2


Figure 3-15a and 3-15b Multivalued attributes can be represented as relationships

simple

composite


Strong vs. Weak Entities, andStrong vs. Weak Entities, andIdentifying RelationshipsIdentifying Relationships

Strong entities Strong entities exist independently of other types of entitiesexist independently of other types of entities has its own unique identifierhas its own unique identifier identifier underlined with single-lineidentifier underlined with single-line

Weak entityWeak entity dependent on a strong entity (identifying owner)…cannot exist on its dependent on a strong entity (identifying owner)…cannot exist on its

ownown does not have a unique identifier (only a partial identifier)does not have a unique identifier (only a partial identifier) Partial identifier underlined with double-linePartial identifier underlined with double-line Entity box has double lineEntity box has double line

Identifying relationshipIdentifying relationship links strong entities to weak entitieslinks strong entities to weak entities


Strong entity Weak entity

Identifying relationship


Associative EntitiesAssociative Entities An An entityentity–has attributes–has attributes

A A relationshiprelationship–links entities together–links entities together

When should a When should a relationship with attributesrelationship with attributes instead be instead be an an associative entityassociative entity? ? All relationships for the associative entity should be manyAll relationships for the associative entity should be many The associative entity could have meaning independent of the The associative entity could have meaning independent of the

other entitiesother entities The associative entity preferably has a unique identifier, and The associative entity preferably has a unique identifier, and

should also have other attributesshould also have other attributes The associative entity may participate in other relationships The associative entity may participate in other relationships

other than the entities of the associated relationshipother than the entities of the associated relationship Ternary relationships should be converted to associative Ternary relationships should be converted to associative

entitiesentities


Figure 3-11a A binary relationship with an attribute

Here, the date completed attribute pertains specifically to the employee’s completion of a course…it is an attribute of the

relationship


Figure 3-11b An associative entity (CERTIFICATE)

Associative entity is like a relationship with an attribute, but it is also considered to be an entity in its own right.

Note that the many-to-many cardinality between entities in Figure 3-11a has been replaced by two one-to-many relationships

with the associative entity.


Figure 3-13c An associative entity – bill of materials structure

This could just be a relationship with attributes…it’s a judgment call


Figure 3-18 Ternary relationship as an associative entity


Microsoft Visio Example forE-R diagram

Different modeling software tools may

have different notation for the same

constructs


Chapter 4:Chapter 4:The Enhanced ER Model The Enhanced ER Model

and Business Rulesand Business Rules





ObjectivesObjectives Definition of termsDefinition of terms Use of supertype/subtype relationshipsUse of supertype/subtype relationships Use of generalization and specialization techniquesUse of generalization and specialization techniques Specification of completeness and disjointness Specification of completeness and disjointness

constraintsconstraints Develop supertype/subtype hierarchies for realistic Develop supertype/subtype hierarchies for realistic

business situationsbusiness situations Develop entity clustersDevelop entity clusters Explain universal data modelExplain universal data model Name categories of business rulesName categories of business rules Define operational constraints graphically and in Define operational constraints graphically and in

EnglishEnglish


Supertypes and SubtypesSupertypes and Subtypes

Subtype:Subtype: A subgrouping of the entities in an A subgrouping of the entities in an entity type that has attributes distinct from entity type that has attributes distinct from those in other subgroupingsthose in other subgroupings

Supertype:Supertype: A generic entity type that has a A generic entity type that has a relationship with one or more subtypesrelationship with one or more subtypes

Attribute Inheritance:Attribute Inheritance: Subtype entities inherit values of all Subtype entities inherit values of all

attributes of the supertypeattributes of the supertype An instance of a subtype is also an instance An instance of a subtype is also an instance

of the supertypeof the supertype


Figure 4-1 Basic notation for supertype/subtype notation

a) EER

notation


Different modeling tools may have different notation for the same modeling constructs

b) Microsoft

Visio Notation

Figure 4-1 Basic notation for supertype/subtype notation (cont.)


Figure 4-2 Employee supertype with three subtypes

All employee subtypes will have emp nbr, name,

address, and date-hired

Each employee subtype will also have its own

attributes


Relationships and SubtypesRelationships and Subtypes

Relationships at the Relationships at the supertypesupertype level level indicate that all subtypes will indicate that all subtypes will participate in the relationshipparticipate in the relationship

The instances of a The instances of a subtypesubtype may may participate in a relationship unique to participate in a relationship unique to that subtype. In this situation, the that subtype. In this situation, the relationship is shown at the subtype relationship is shown at the subtype levellevel


Figure 4-3 Supertype/subtype relationships in a hospital

Both outpatients

and resident patients are

cared for by a responsible

physician

Only resident patients are assigned to a bed


Generalization and Generalization and SpecializationSpecialization

Generalization:Generalization: The process of The process of defining a more general entity type from defining a more general entity type from a set of more specialized entity types. a set of more specialized entity types. BOTTOM-UPBOTTOM-UP

Specialization:Specialization: The process of The process of defining one or more subtypes of the defining one or more subtypes of the supertype and forming supertype and forming supertype/subtype relationships. TOP-supertype/subtype relationships. TOP-DOWNDOWN


Figure 4-4 Example of generalization

a) Three entity types: CAR, TRUCK, and MOTORCYCLE

All these types of vehicles have common attributes


Figure 4-4 Example of generalization (cont.)

So we put the shared

attributes in a supertype

Note: no subtype for motorcycle, since it has no unique attributes

b) Generalization to VEHICLE supertype


Figure 4-5 Example of specialization

a) Entity type PART

Only applies to manufactured parts

Applies only to purchased parts


b) Specialization to MANUFACTURED PART and PURCHASED PART

Note: multivalued attribute was replaced by an associative entity relationship to another entity

Created 2 subtypes

Figure 4-5 Example of specialization (cont.)


Constraints in Supertype/ Constraints in Supertype/ Completeness ConstraintCompleteness Constraint

Completeness ConstraintsCompleteness Constraints: Whether : Whether an instance of a supertype an instance of a supertype mustmust also be a also be a member of at least one subtypemember of at least one subtype Total Specialization Rule: Yes (double line)Total Specialization Rule: Yes (double line) Partial Specialization Rule: No (single line)Partial Specialization Rule: No (single line)


Figure 4-6 Examples of completeness constraints

a) Total specialization rule

A patient must be either an outpatient or a

resident patient


b) Partial specialization rule

A vehicle could be a

car, a truck, or

neither

Figure 4-6 Examples of completeness constraints (cont.)


Constraints in Supertype/ Constraints in Supertype/ Disjointness constraintDisjointness constraint

Disjointness ConstraintsDisjointness Constraints: : Whether an instance of a supertype may Whether an instance of a supertype may simultaneouslysimultaneously be a member of two (or be a member of two (or more) subtypesmore) subtypes Disjoint Rule: An instance of the supertype Disjoint Rule: An instance of the supertype

can be only ONE of the subtypescan be only ONE of the subtypes Overlap Rule: An instance of the supertype Overlap Rule: An instance of the supertype

could be more than one of the subtypescould be more than one of the subtypes


a) Disjoint rule

Figure 4-7 Examples of disjointness constraints

A patient can either be outpatient or resident, but not both


b) Overlap rule

A part may be both purchased

and manufactured

Figure 4-7 Examples of disjointness constraints (cont.)


Constraints in Supertype/ Constraints in Supertype/ Subtype DiscriminatorsSubtype Discriminators

Subtype DiscriminatorSubtype Discriminator: An attribute : An attribute of the supertype whose values determine of the supertype whose values determine the target subtype(s)the target subtype(s) DisjointDisjoint – a – a simplesimple attribute with alternative attribute with alternative

values to indicate the possible subtypesvalues to indicate the possible subtypes OverlappingOverlapping – a – a compositecomposite attribute whose attribute whose

subparts pertain to different subtypes. Each subparts pertain to different subtypes. Each subpart contains a boolean value to indicate subpart contains a boolean value to indicate whether or not the instance belongs to the whether or not the instance belongs to the associated subtypeassociated subtype


Figure 4-8 Introducing a subtype discriminator (disjoint rule)

A simple attribute with different possible values

indicating the subtype


Figure 4-9 Subtype discriminator (overlap rule)A composite

attribute with sub-attributes

indicating “yes” or “no” to determine

whether it is of each subtype


Figure 4-10 Example of supertype/subtype hierarchy


Entity ClustersEntity Clusters

EER diagrams are difficult to read when EER diagrams are difficult to read when there are too many entities and there are too many entities and relationshipsrelationships

Solution: Group entities and relationships Solution: Group entities and relationships into into entity clustersentity clusters

Entity clusterEntity cluster: Set of one or more entity : Set of one or more entity types and associated relationships types and associated relationships grouped into a single abstract entity typegrouped into a single abstract entity type


Figure 4-13a Possible entity

clusters for Pine Valley Furniture in

Microsoft Visio

Related groups of

entities could become clusters


Figure 4-13b EER diagram of PVF entity clusters

More readable, isn’t it?


Figure 4-14 Manufacturing entity cluster

Detail for a single cluster


137137

Chapter 4:Chapter 4:The Enhanced ER Model The Enhanced ER Model

and Business Rulesand Business Rules





ObjectivesObjectives Definition of termsDefinition of terms Use of supertype/subtype relationshipsUse of supertype/subtype relationships Use of generalization and specialization techniquesUse of generalization and specialization techniques Specification of completeness and disjointness Specification of completeness and disjointness

constraintsconstraints Develop supertype/subtype hierarchies for realistic Develop supertype/subtype hierarchies for realistic

business situationsbusiness situations Develop entity clustersDevelop entity clusters Explain universal data modelExplain universal data model Name categories of business rulesName categories of business rules Define operational constraints graphically and in Define operational constraints graphically and in

EnglishEnglish


Supertypes and SubtypesSupertypes and Subtypes

Subtype:Subtype: A subgrouping of the entities in an A subgrouping of the entities in an entity type that has attributes distinct from entity type that has attributes distinct from those in other subgroupingsthose in other subgroupings

Supertype:Supertype: A generic entity type that has a A generic entity type that has a relationship with one or more subtypesrelationship with one or more subtypes

Attribute Inheritance:Attribute Inheritance: Subtype entities inherit values of all Subtype entities inherit values of all

attributes of the supertypeattributes of the supertype An instance of a subtype is also an instance An instance of a subtype is also an instance

of the supertypeof the supertype


Figure 4-1 Basic notation for supertype/subtype notation

a) EER

notation


Different modeling tools may have different notation for the same modeling constructs

b) Microsoft

Visio Notation

Figure 4-1 Basic notation for supertype/subtype notation (cont.)


Figure 4-2 Employee supertype with three subtypes

All employee subtypes will have emp nbr, name,

address, and date-hired

Each employee subtype will also have its own

attributes


Relationships and SubtypesRelationships and Subtypes

Relationships at the Relationships at the supertypesupertype level level indicate that all subtypes will indicate that all subtypes will participate in the relationshipparticipate in the relationship

The instances of a The instances of a subtypesubtype may may participate in a relationship unique to participate in a relationship unique to that subtype. In this situation, the that subtype. In this situation, the relationship is shown at the subtype relationship is shown at the subtype levellevel


Figure 4-3 Supertype/subtype relationships in a hospital

Both outpatients

and resident patients are

cared for by a responsible

physician

Only resident patients are assigned to a bed


Generalization and Generalization and SpecializationSpecialization

Generalization:Generalization: The process of The process of defining a more general entity type from defining a more general entity type from a set of more specialized entity types. a set of more specialized entity types. BOTTOM-UPBOTTOM-UP

Specialization:Specialization: The process of The process of defining one or more subtypes of the defining one or more subtypes of the supertype and forming supertype and forming supertype/subtype relationships. TOP-supertype/subtype relationships. TOP-DOWNDOWN


Figure 4-4 Example of generalization

a) Three entity types: CAR, TRUCK, and MOTORCYCLE

All these types of vehicles have common attributes


Figure 4-4 Example of generalization (cont.)

So we put the shared

attributes in a supertype

Note: no subtype for motorcycle, since it has no unique attributes

b) Generalization to VEHICLE supertype


Figure 4-5 Example of specialization

a) Entity type PART

Only applies to manufactured parts

Applies only to purchased parts


b) Specialization to MANUFACTURED PART and PURCHASED PART

Note: multivalued attribute was replaced by an associative entity relationship to another entity

Created 2 subtypes

Figure 4-5 Example of specialization (cont.)


Constraints in Supertype/ Constraints in Supertype/ Completeness ConstraintCompleteness Constraint

Completeness ConstraintsCompleteness Constraints: Whether : Whether an instance of a supertype an instance of a supertype mustmust also be a also be a member of at least one subtypemember of at least one subtype Total Specialization Rule: Yes (double line)Total Specialization Rule: Yes (double line) Partial Specialization Rule: No (single line)Partial Specialization Rule: No (single line)


Figure 4-6 Examples of completeness constraints

a) Total specialization rule

A patient must be either an outpatient or a

resident patient


b) Partial specialization rule

A vehicle could be a

car, a truck, or

neither

Figure 4-6 Examples of completeness constraints (cont.)


Constraints in Supertype/ Constraints in Supertype/ Disjointness constraintDisjointness constraint

Disjointness ConstraintsDisjointness Constraints: : Whether an instance of a supertype may Whether an instance of a supertype may simultaneouslysimultaneously be a member of two (or be a member of two (or more) subtypesmore) subtypes Disjoint Rule: An instance of the supertype Disjoint Rule: An instance of the supertype

can be only ONE of the subtypescan be only ONE of the subtypes Overlap Rule: An instance of the supertype Overlap Rule: An instance of the supertype

could be more than one of the subtypescould be more than one of the subtypes


a) Disjoint rule

Figure 4-7 Examples of disjointness constraints

A patient can either be outpatient or resident, but not both


b) Overlap rule

A part may be both purchased

and manufactured

Figure 4-7 Examples of disjointness constraints (cont.)


Constraints in Supertype/ Constraints in Supertype/ Subtype DiscriminatorsSubtype Discriminators

Subtype DiscriminatorSubtype Discriminator: An attribute : An attribute of the supertype whose values determine of the supertype whose values determine the target subtype(s)the target subtype(s) DisjointDisjoint – a – a simplesimple attribute with alternative attribute with alternative

values to indicate the possible subtypesvalues to indicate the possible subtypes OverlappingOverlapping – a – a compositecomposite attribute whose attribute whose

subparts pertain to different subtypes. Each subparts pertain to different subtypes. Each subpart contains a boolean value to indicate subpart contains a boolean value to indicate whether or not the instance belongs to the whether or not the instance belongs to the associated subtypeassociated subtype


Figure 4-8 Introducing a subtype discriminator (disjoint rule)

A simple attribute with different possible values

indicating the subtype


Figure 4-9 Subtype discriminator (overlap rule)A composite

attribute with sub-attributes

indicating “yes” or “no” to determine

whether it is of each subtype


Figure 4-10 Example of supertype/subtype hierarchy


Entity ClustersEntity Clusters

EER diagrams are difficult to read when EER diagrams are difficult to read when there are too many entities and there are too many entities and relationshipsrelationships

Solution: Group entities and relationships Solution: Group entities and relationships into into entity clustersentity clusters

Entity clusterEntity cluster: Set of one or more entity : Set of one or more entity types and associated relationships types and associated relationships grouped into a single abstract entity typegrouped into a single abstract entity type


Figure 4-13a Possible entity

clusters for Pine Valley Furniture in

Microsoft Visio

Related groups of

entities could become clusters


Figure 4-13b EER diagram of PVF entity clusters

More readable, isn’t it?


Figure 4-14 Manufacturing entity cluster

Detail for a single cluster


Packaged data models provide generic models

that can be customized for a

particular organization’s business rules


Business rulesBusiness rules

Statements that Statements that definedefine or or constrainconstrain some some aspect of the businessaspect of the business

Classification of business rules:Classification of business rules: Derivation–rule derived from other knowledge, often Derivation–rule derived from other knowledge, often

in the form of a formula using attribute valuesin the form of a formula using attribute values Structural assertion–rule expressing static structure. Structural assertion–rule expressing static structure.

Includes attributes, relationships, and definitionsIncludes attributes, relationships, and definitions Action assertion–rule expressing constraints/control Action assertion–rule expressing constraints/control

of organizational actionsof organizational actions


Figure 4-18EER diagram

to describe business

rules


Types of Action AssertionsTypes of Action Assertions ResultResult

Condition–IF/THEN ruleCondition–IF/THEN rule Integrity constraint–must always be trueIntegrity constraint–must always be true Authorization–privilege statementAuthorization–privilege statement

FormForm Enabler–leads to creation of new objectEnabler–leads to creation of new object Timer–allows or disallows an actionTimer–allows or disallows an action Executive–executes one or more actionsExecutive–executes one or more actions

RigorRigor Controlling–something must or must not happenControlling–something must or must not happen Influencing–guideline for which a notification Influencing–guideline for which a notification

must occurmust occur


Stating an Action AssertionStating an Action Assertion Anchor Object–an object on which Anchor Object–an object on which

actions are limitedactions are limited Action–creation, deletion, update, or Action–creation, deletion, update, or

readread Corresponding Objects–an object Corresponding Objects–an object

influencing the ability to perform an influencing the ability to perform an action on another business ruleaction on another business rule

Action assertions identify corresponding objects that Action assertions identify corresponding objects that constrain the ability to perform actions on anchor objectsconstrain the ability to perform actions on anchor objects


Figure 4-19 Data model segment for class scheduling


Figure 4-20 Business Rule 1: For a faculty member to be assigned to teach a section of a course, the faculty member must be qualified to

teach the course for which that section is scheduled

Action assertion

Anchor object

Corresponding object


In this case, the action assertion

is a RRestriction


Figure 4-21 Business Rule 2: For a faculty member to be assigned to teach a section of a course, the faculty member must not be assigned to

teach a total of more than three course sections

Action assertionAnchor object


In this case, the action assertion

is an

UUpper LIMLIMit


172172

Chapter 5:Chapter 5:Logical Database Design Logical Database Design and the Relational Modeland the Relational Model





ObjectivesObjectives Definition of termsDefinition of terms List five properties of relationsList five properties of relations State two properties of candidate keysState two properties of candidate keys Define first, second, and third normal formDefine first, second, and third normal form Describe problems from merging relationsDescribe problems from merging relations Transform E-R and EER diagrams to Transform E-R and EER diagrams to

relationsrelations Create tables with entity and relational Create tables with entity and relational

integrity constraintsintegrity constraints Use normalization to convert anomalous Use normalization to convert anomalous

tables to well-structured relationstables to well-structured relations


RelationRelation Definition: A relation is a named, two-dimensional table Definition: A relation is a named, two-dimensional table

of data of data Table consists of rows (records) and columns (attribute Table consists of rows (records) and columns (attribute

or field)or field) Requirements for a table to qualify as a relation:Requirements for a table to qualify as a relation:

It must have a unique nameIt must have a unique name Every attribute value must be atomic (not multivalued, not Every attribute value must be atomic (not multivalued, not

composite)composite) Every row must be unique (can’t have two rows with exactly the Every row must be unique (can’t have two rows with exactly the

same values for all their fields)same values for all their fields) Attributes (columns) in tables must have unique namesAttributes (columns) in tables must have unique names The order of the columns must be irrelevantThe order of the columns must be irrelevant The order of the rows must be irrelevantThe order of the rows must be irrelevant

NOTE: all NOTE: all relationsrelations are in are in 11stst Normal form Normal form


Correspondence with E-R Correspondence with E-R ModelModel

Relations (tables) correspond with entity types Relations (tables) correspond with entity types and with many-to-many relationship typesand with many-to-many relationship types

Rows correspond with entity instances and Rows correspond with entity instances and with many-to-many relationship instanceswith many-to-many relationship instances

Columns correspond with attributesColumns correspond with attributes

NOTE: The word NOTE: The word relationrelation (in relational (in relational database) is NOT the same as the word database) is NOT the same as the word relationshiprelationship (in E-R model) (in E-R model)


Key FieldsKey Fields

Keys are special fields that serve two main Keys are special fields that serve two main purposes:purposes: Primary keysPrimary keys are are uniqueunique identifiers of the relation in identifiers of the relation in

question. Examples include employee numbers, social question. Examples include employee numbers, social security numbers, etc. security numbers, etc. This is how we can guarantee This is how we can guarantee that all rows are uniquethat all rows are unique

Foreign keysForeign keys are identifiers that enable a are identifiers that enable a dependentdependent relation (on the many side of a relationship) to refer to relation (on the many side of a relationship) to refer to its its parentparent relation (on the one side of the relationship) relation (on the one side of the relationship)

Keys can be Keys can be simplesimple (a single field) or (a single field) or compositecomposite (more than one field)(more than one field)

Keys usually are used as indexes to speed up the Keys usually are used as indexes to speed up the response to user queries (More on this in Ch. 6)response to user queries (More on this in Ch. 6)


Primary Key

Foreign Key (implements 1:N relationship between customer and order)

Combined, these are a composite primary key (uniquely identifies the

order line)…individually they are foreign keys (implement M:N

relationship between order and product)

Figure 5-3 Schema for four relations (Pine Valley Furniture Company)


Integrity ConstraintsIntegrity Constraints

Domain ConstraintsDomain Constraints Allowable values for an attribute. See Allowable values for an attribute. See

Table 5-1Table 5-1 Entity IntegrityEntity Integrity

No primary key attribute may be null. No primary key attribute may be null. All primary key fields All primary key fields MUSTMUST have data have data

Action AssertionsAction Assertions Business rules. Recall from Ch. 4Business rules. Recall from Ch. 4


Domain definitions enforce domain integrity constraints


Integrity ConstraintsIntegrity Constraints

Referential Integrity–rule states that any foreign key Referential Integrity–rule states that any foreign key value (on the relation of the many side) MUST match a value (on the relation of the many side) MUST match a primary key value in the relation of the one side. (Or the primary key value in the relation of the one side. (Or the foreign key can be null) foreign key can be null) For example: Delete RulesFor example: Delete Rules

Restrict–don’t allow delete of “parent” side if related rows Restrict–don’t allow delete of “parent” side if related rows exist in “dependent” sideexist in “dependent” side

Cascade–automatically delete “dependent” side rows that Cascade–automatically delete “dependent” side rows that correspond with the “parent” side row to be deletedcorrespond with the “parent” side row to be deleted

Set-to-Null–set the foreign key in the dependent side to null if Set-to-Null–set the foreign key in the dependent side to null if deleting from the parent side deleting from the parent side not allowed for weak entities not allowed for weak entities


Figure 5-5 Referential integrity constraints (Pine Valley Furniture)

Referential integrity

constraints are drawn via arrows from dependent to

parent table


Figure 5-6 SQL table definitions

Referential integrity

constraints are implemented with

foreign key to primary key references


Transforming EER Diagrams Transforming EER Diagrams into Relationsinto Relations

Mapping Regular Entities to Relations Mapping Regular Entities to Relations 1.1. Simple attributes: E-R attributes map Simple attributes: E-R attributes map

directly onto the relationdirectly onto the relation

2.2. Composite attributes: Use only their Composite attributes: Use only their simple, component attributes simple, component attributes

3.3. Multivalued Attribute–Becomes a Multivalued Attribute–Becomes a separate relation with a foreign key separate relation with a foreign key taken from the superior entitytaken from the superior entity


(a) CUSTOMER entity type with

simple attributes

Figure 5-8 Mapping a regular entity

(b) CUSTOMER relation


(a) CUSTOMER entity type with

composite attribute

Figure 5-9 Mapping a composite attribute

(b) CUSTOMER relation with address detail


Figure 5-10 Mapping an entity with a multivalued attribute

One–to–many relationship between original entity and new relation

(a)

Multivalued attribute becomes a separate relation with foreign key

(b)


Transforming EER Diagrams Transforming EER Diagrams into Relations (cont.)into Relations (cont.)

Mapping Weak EntitiesMapping Weak Entities Becomes a separate relation with a Becomes a separate relation with a

foreign key taken from the superior foreign key taken from the superior entityentity

Primary key composed of:Primary key composed of: Partial identifier of weak entityPartial identifier of weak entity Primary key of identifying relation Primary key of identifying relation

(strong entity)(strong entity)


Figure 5-11 Example of mapping a weak entity

a) Weak entity DEPENDENT


NOTE: the domain constraint for the foreign key should NOT allow null value

if DEPENDENT is a weak entity

Foreign key

Composite primary key

Figure 5-11 Example of mapping a weak entity (cont.)

b) Relations resulting from weak entity



Mapping Binary RelationshipsMapping Binary Relationships One-to-Many–Primary key on the one side One-to-Many–Primary key on the one side

becomes a foreign key on the many sidebecomes a foreign key on the many side Many-to-Many–Create a Many-to-Many–Create a new relationnew relation

with the primary keys of the two entities with the primary keys of the two entities as its primary keyas its primary key

One-to-One–Primary key on the mandatory One-to-One–Primary key on the mandatory side becomes a foreign key on the side becomes a foreign key on the optional sideoptional side


Figure 5-12 Example of mapping a 1:M relationship

a) Relationship between customers and orders

Note the mandatory one

b) Mapping the relationship

Again, no null value in the foreign key…this is because of the mandatory minimum

cardinalityForeign key


Figure 5-13 Example of mapping an M:N relationship

a) Completes relationship (M:N)

The Completes relationship will need to become a separate relation


New intersection

relation

Foreign key

Foreign key

Composite primary key

Figure 5-13 Example of mapping an M:N relationship (cont.)

b) Three resulting relations


Figure 5-14 Example of mapping a binary 1:1 relationship

a) In_charge relationship (1:1)

Often in 1:1 relationships, one direction is optional.


b) Resulting relations

Figure 5-14 Example of mapping a binary 1:1 relationship (cont.)

Foreign key goes in the relation on the optional side,Matching the primary key on the mandatory side



Mapping Associative EntitiesMapping Associative Entities Identifier Not Assigned Identifier Not Assigned

Default primary key for the Default primary key for the association relation is composed of association relation is composed of the primary keys of the two entities the primary keys of the two entities (as in M:N relationship)(as in M:N relationship)

Identifier Assigned Identifier Assigned It is natural and familiar to end-usersIt is natural and familiar to end-users Default identifier may not be uniqueDefault identifier may not be unique


Figure 5-15 Example of mapping an associative entity

a) An associative entity


Figure 5-15 Example of mapping an associative entity (cont.)


Composite primary key formed from the two foreign keys


Figure 5-16 Example of mapping an associative entity with an identifier

a) SHIPMENT associative entity


Figure 5-16 Example of mapping an associative entity with an identifier (cont.)


Primary key differs from foreign keys



Mapping Unary RelationshipsMapping Unary Relationships One-to-Many–Recursive foreign key in the One-to-Many–Recursive foreign key in the

same relationsame relation Many-to-Many–Two relations:Many-to-Many–Two relations:

One for the entity typeOne for the entity type One for an associative relation in which One for an associative relation in which

the primary key has two attributes, the primary key has two attributes, both taken from the primary key of the both taken from the primary key of the entityentity


Figure 5-17 Mapping a unary 1:N relationship

(a) EMPLOYEE entity with unary relationship

(b) EMPLOYEE relation with

recursive foreign key


Figure 5-18 Mapping a unary M:N relationship

(a) Bill-of-materials relationships (M:N)

(b) ITEM and COMPONENT

relations



Mapping Ternary (and n-ary) Mapping Ternary (and n-ary) RelationshipsRelationships One relation for each entity and One relation for each entity and

one for the associative entityone for the associative entity Associative entity has foreign keys Associative entity has foreign keys

to each entity in the relationshipto each entity in the relationship


Figure 5-19 Mapping a ternary relationship

a) PATIENT TREATMENT Ternary relationship with associative entity


b) Mapping the ternary relationship PATIENT TREATMENT

Remember that the

primary key MUST be

unique

Figure 5-19 Mapping a ternary relationship (cont.)

This is why treatment date and time are

included in the composite

primary key

But this makes a very

cumbersome key…

It would be better to create a

surrogate key like Treatment#


Transforming EER Transforming EER Diagrams into Relations Diagrams into Relations

(cont.)(cont.)Mapping Supertype/Subtype RelationshipsMapping Supertype/Subtype Relationships

One relation for supertype and for each subtypeOne relation for supertype and for each subtype Supertype attributes (including identifier and Supertype attributes (including identifier and

subtype discriminator) go into supertype relationsubtype discriminator) go into supertype relation Subtype attributes go into each subtype; primary Subtype attributes go into each subtype; primary

key of supertype relation also becomes primary key of supertype relation also becomes primary key of subtype relationkey of subtype relation

1:1 relationship established between supertype 1:1 relationship established between supertype and each subtype, with supertype as primary and each subtype, with supertype as primary tabletable


Figure 5-20 Supertype/subtype relationships


Figure 5-21 Mapping Supertype/subtype relationships to relations

These are implemented as one-to-one relationships


Data NormalizationData Normalization Primarily a tool to validate and Primarily a tool to validate and

improve a logical design so that it improve a logical design so that it satisfies certain constraints that satisfies certain constraints that avoid unnecessary avoid unnecessary duplication of dataduplication of data

The process of decomposing The process of decomposing relations with anomalies to produce relations with anomalies to produce smaller, smaller, well-structuredwell-structured relationsrelations


Well-Structured RelationsWell-Structured Relations A relation that contains minimal data redundancy A relation that contains minimal data redundancy

and allows users to insert, delete, and update and allows users to insert, delete, and update rows without causing data inconsistenciesrows without causing data inconsistencies

Goal is to avoid anomaliesGoal is to avoid anomalies Insertion AnomalyInsertion Anomaly–adding new rows forces user to –adding new rows forces user to

create duplicate datacreate duplicate data Deletion AnomalyDeletion Anomaly–deleting rows may cause a loss of –deleting rows may cause a loss of

data that would be needed for other future rowsdata that would be needed for other future rows Modification AnomalyModification Anomaly–changing data in a row forces –changing data in a row forces

changes to other rows because of duplicationchanges to other rows because of duplication

General rule of thumb: A table should not pertain to more than one entity type


Example–Figure 5-2bExample–Figure 5-2b

Question–Is this a relation? Answer–Yes: Unique rows and no multivalued attributes

Question–What’s the primary key? Answer–Composite: Emp_ID, Course_Title


Anomalies in this TableAnomalies in this Table InsertionInsertion–can’t enter a new employee without –can’t enter a new employee without

having the employee take a classhaving the employee take a class DeletionDeletion–if we remove employee 140, we lose –if we remove employee 140, we lose

information about the existence of a Tax Acc information about the existence of a Tax Acc classclass

ModificationModification–giving a salary increase to –giving a salary increase to employee 100 forces us to update multiple employee 100 forces us to update multiple recordsrecords Why do these anomalies exist?

Because there are two themes (entity types) in this one relation. This results in data duplication and an

unnecessary dependency between the entities


Functional Dependencies and Functional Dependencies and KeysKeys

Functional Dependency: The value of one Functional Dependency: The value of one attribute (the attribute (the determinantdeterminant) determines ) determines the value of another attributethe value of another attribute

Candidate Key:Candidate Key: A unique identifier. One of the candidate A unique identifier. One of the candidate

keys will become the primary keykeys will become the primary key E.g. perhaps there is both credit card number and E.g. perhaps there is both credit card number and

SS# in a table…in this case both are candidate SS# in a table…in this case both are candidate keyskeys

Each non-key field is functionally dependent Each non-key field is functionally dependent on every candidate keyon every candidate key


Figure 5.22 Steps in normalization


First Normal FormFirst Normal Form No multivalued attributesNo multivalued attributes Every attribute value is atomicEvery attribute value is atomic Fig. 5-25 Fig. 5-25 is notis not in 1 in 1stst Normal Form Normal Form

(multivalued attributes) (multivalued attributes) it is not a it is not a relationrelation

Fig. 5-26 Fig. 5-26 isis in 1 in 1stst Normal form Normal form All relationsAll relations are in 1 are in 1stst Normal Form Normal Form


Table with multivalued attributes, not in 1st normal form

Note: this is NOT a relation


Table with no multivalued attributes and unique rows, in 1st normal form

Note: this is relation, but not a well-structured one


Anomalies in this TableAnomalies in this Table InsertionInsertion–if new product is ordered for order –if new product is ordered for order

1007 of existing customer, customer data 1007 of existing customer, customer data must be re-entered, causing duplicationmust be re-entered, causing duplication

DeletionDeletion–if we delete the Dining Table from –if we delete the Dining Table from Order 1006, we lose information concerning Order 1006, we lose information concerning this item's finish and pricethis item's finish and price

UpdateUpdate–changing the price of product ID 4 –changing the price of product ID 4 requires update in several recordsrequires update in several records

Why do these anomalies exist? Because there are multiple themes (entity types) in

one relation. This results in duplication and an unnecessary dependency between the entities


Second Normal FormSecond Normal Form 1NF PLUS 1NF PLUS every non-key every non-key

attribute is fully functionally attribute is fully functionally dependent on the ENTIRE dependent on the ENTIRE primary keyprimary key Every non-key attribute must be Every non-key attribute must be

defined by the entire key, not by only defined by the entire key, not by only part of the keypart of the key

No partial functional dependenciesNo partial functional dependencies


Order_ID Order_Date, Customer_ID, Customer_Name, Customer_Address

Therefore, NOT in 2nd Normal Form

Customer_ID Customer_Name, Customer_Address

Product_ID Product_Description, Product_Finish, Unit_Price

Order_ID, Product_ID Order_Quantity

Figure 5-27 Functional dependency diagram for INVOICE


Partial dependencies are removed, but there are still transitive dependencies

Getting it into Getting it into Second Normal Second Normal

FormForm

Figure 5-28 Removing partial dependencies


Third Normal FormThird Normal Form 2NF PLUS 2NF PLUS no transitive dependenciesno transitive dependencies

(functional dependencies on non-primary-key (functional dependencies on non-primary-key attributes)attributes)

Note: This is called transitive, because the Note: This is called transitive, because the primary key is a determinant for another primary key is a determinant for another attribute, which in turn is a determinant for a attribute, which in turn is a determinant for a thirdthird

Solution: Non-key determinant with transitive Solution: Non-key determinant with transitive dependencies go into a new table; non-key dependencies go into a new table; non-key determinant becomes primary key in the determinant becomes primary key in the new table and stays as foreign key in the old new table and stays as foreign key in the old table table


Transitive dependencies are removed

Figure 5-28 Removing partial dependencies

Getting it into Getting it into Third Normal Third Normal

FormForm


Merging RelationsMerging Relations View Integration–Combining entities from View Integration–Combining entities from

multiple ER models into common relationsmultiple ER models into common relations Issues to watch out for when merging entities Issues to watch out for when merging entities

from different ER models:from different ER models: Synonyms–two or more attributes with different Synonyms–two or more attributes with different

names but same meaningnames but same meaning Homonyms–attributes with same name but different Homonyms–attributes with same name but different

meaningsmeanings Transitive dependencies–even if relations are in 3NF Transitive dependencies–even if relations are in 3NF

prior to merging, they may not be after mergingprior to merging, they may not be after merging Supertype/subtype relationships–may be hidden Supertype/subtype relationships–may be hidden

prior to mergingprior to merging


Enterprise KeysEnterprise Keys

Primary keys that are unique in the whole Primary keys that are unique in the whole database, not just within a single relationdatabase, not just within a single relation

Corresponds with the concept of an object Corresponds with the concept of an object ID in object-oriented systemsID in object-oriented systems


Figure 5-31 Enterprise keys

a) Relations with enterprise key

b) Sample data with enterprise key


228228

Chapter 6:Chapter 6:Physical Database Design Physical Database Design

and Performanceand Performance

Modern Database Modern Database ManagementManagement88thth Edition Edition

Jeffrey A. Hoffer, Mary B. Prescott, Jeffrey A. Hoffer, Mary B. Prescott, Fred R. McFaddenFred R. McFadden


ObjectivesObjectives Definition of termsDefinition of terms Describe the physical database design Describe the physical database design

processprocess Choose storage formats for attributesChoose storage formats for attributes Select appropriate file organizationsSelect appropriate file organizations Describe three types of file organizationDescribe three types of file organization Describe indexes and their appropriate useDescribe indexes and their appropriate use Translate a database model into efficient Translate a database model into efficient

structuresstructures Know when and how to use denormalizationKnow when and how to use denormalization


Physical Database DesignPhysical Database Design

Purpose–translate the logical Purpose–translate the logical description of data into the description of data into the technical technical specificationsspecifications for storing and for storing and retrieving dataretrieving data

Goal–create a design for storing data Goal–create a design for storing data that will provide that will provide adequate adequate performanceperformance and insure and insure database database integrityintegrity, , securitysecurity, and , and recoverabilityrecoverability


Physical Design ProcessPhysical Design Process

Normalized relations

Volume estimates

Attribute definitions

Response time expectations

Data security needs

Backup/recovery needs

Integrity expectations

DBMS technology used

Inputs

Attribute data types

Physical record descriptions (doesn’t always match

logical design)

File organizations

Indexes and database architectures

Query optimization

Leads to

Decisions


Figure 6-1 Composite usage map (Pine Valley Furniture Company)


Figure 6-1 Composite usage map (Pine Valley Furniture Company) (cont.)

Data volumes



Access Frequencies (per hour)



Usage analysis:140 purchased parts accessed

per hour 80 quotations accessed from

these 140 purchased part accesses

70 suppliers accessed from these 80 quotation accesses



Usage analysis:75 suppliers accessed per hour

40 quotations accessed from

these 75 supplier accesses 40 purchased parts accessed

from these 40 quotation accesses


Designing FieldsDesigning Fields

Field: smallest unit of data in Field: smallest unit of data in databasedatabase

Field design Field design Choosing data typeChoosing data type Coding, compression, encryptionCoding, compression, encryption Controlling data integrityControlling data integrity


Choosing Data TypesChoosing Data Types

CHAR–fixed-length characterCHAR–fixed-length character VARCHAR2–variable-length character VARCHAR2–variable-length character

(memo)(memo) LONG–large numberLONG–large number NUMBER–positive/negative numberNUMBER–positive/negative number INEGER–positive/negative whole numberINEGER–positive/negative whole number DATE–actual dateDATE–actual date BLOB–binary large object (good for graphics, BLOB–binary large object (good for graphics,

sound clips, etc.)sound clips, etc.)


Figure 6-2 Example code look-up table(Pine Valley Furniture Company)

Code saves space, but costs an additional lookup

to obtain actual value


Field Data IntegrityField Data Integrity Default value–assumed value if no Default value–assumed value if no

explicit valueexplicit value Range control–allowable value limitations Range control–allowable value limitations

(constraints or validation rules)(constraints or validation rules) Null value control–allowing or prohibiting Null value control–allowing or prohibiting

empty fieldsempty fields Referential integrity–range control (and Referential integrity–range control (and

null value allowances) for foreign-key to null value allowances) for foreign-key to primary-key match-upsprimary-key match-ups

Sarbanes-Oxley Act (SOX) legislates importance of financial data integrity


Handling Missing DataHandling Missing Data

Substitute an estimate of the missing Substitute an estimate of the missing value (e.g., using a formula)value (e.g., using a formula)

Construct a report listing missing valuesConstruct a report listing missing values In programs, ignore missing data unless In programs, ignore missing data unless

the value is significant (sensitivity the value is significant (sensitivity testing)testing)

Triggers can be used to perform these operations


Physical RecordsPhysical Records Physical Record: A group of fields Physical Record: A group of fields

stored in adjacent memory locations stored in adjacent memory locations and retrieved together as a unitand retrieved together as a unit

Page: The amount of data read or Page: The amount of data read or written in one I/O operationwritten in one I/O operation

Blocking Factor: The number of Blocking Factor: The number of physical records per pagephysical records per page


DenormalizationDenormalization Transforming Transforming normalizednormalized relations into relations into

unnormalizedunnormalized physical record specifications physical record specifications Benefits:Benefits:

Can improve performance (speed) by reducing number of Can improve performance (speed) by reducing number of table lookups (i.e. table lookups (i.e. reduce number of necessary join queriesreduce number of necessary join queries))

Costs (due to data duplication)Costs (due to data duplication) Wasted storage spaceWasted storage space Data integrity/consistency threatsData integrity/consistency threats

Common denormalization opportunitiesCommon denormalization opportunities One-to-one relationship (Fig. 6-3)One-to-one relationship (Fig. 6-3) Many-to-many relationship with attributes (Fig. 6-4)Many-to-many relationship with attributes (Fig. 6-4) Reference data (1:N relationship where 1-side has data not Reference data (1:N relationship where 1-side has data not

used in any other relationship) (Fig. 6-5)used in any other relationship) (Fig. 6-5)


Figure 6-3 A possible denormalization situation: two entities with one-to-one relationship


Figure 6-4 A possible denormalization situation: a many-to-many relationship with nonkey attributes

Extra table access

required

Null description possible


Figure 6-5A possible

denormalization situation:

reference data

Extra table access

required

Data duplication


PartitioningPartitioning Horizontal Partitioning: Distributing the rows of Horizontal Partitioning: Distributing the rows of

a table into several separate filesa table into several separate files Useful for situations where different users need Useful for situations where different users need

access to different rowsaccess to different rows Three types: Key Range Partitioning, Hash Three types: Key Range Partitioning, Hash

Partitioning, or Composite PartitioningPartitioning, or Composite Partitioning Vertical Partitioning: Distributing the columns Vertical Partitioning: Distributing the columns

of a table into several separate relationsof a table into several separate relations Useful for situations where different users need Useful for situations where different users need

access to different columnsaccess to different columns The primary key must be repeated in each fileThe primary key must be repeated in each file

Combinations of Horizontal and VerticalCombinations of Horizontal and Vertical

Partitions often correspond with User Schemas (user views)


Partitioning (cont.)Partitioning (cont.) Advantages of Partitioning:Advantages of Partitioning:

Efficiency: Records used together are grouped togetherEfficiency: Records used together are grouped together Local optimization: Each partition can be optimized for Local optimization: Each partition can be optimized for

performanceperformance Security, recoverySecurity, recovery Load balancing: Partitions stored on different disks, Load balancing: Partitions stored on different disks,

reduces contentionreduces contention Take advantage of parallel processing capabilityTake advantage of parallel processing capability

Disadvantages of Partitioning:Disadvantages of Partitioning: Inconsistent access speed: Slow retrievals across Inconsistent access speed: Slow retrievals across

partitionspartitions Complexity: Non-transparent partitioningComplexity: Non-transparent partitioning Extra space or update time: Duplicate data; access from Extra space or update time: Duplicate data; access from

multiple partitionsmultiple partitions


Data ReplicationData Replication

Purposely storing the same data in Purposely storing the same data in multiple locations of the databasemultiple locations of the database

Improves performance by allowing Improves performance by allowing multiple users to access the same data multiple users to access the same data at the same time with minimum at the same time with minimum contentioncontention

Sacrifices data integrity due to data Sacrifices data integrity due to data duplicationduplication

Best for data that is not updated oftenBest for data that is not updated often


Designing Physical FilesDesigning Physical Files Physical File: Physical File:

A named portion of secondary memory allocated A named portion of secondary memory allocated for the purpose of storing physical recordsfor the purpose of storing physical records

Tablespace–named set of disk storage elements Tablespace–named set of disk storage elements in which physical files for database tables can be in which physical files for database tables can be storedstored

Extent–contiguous section of disk spaceExtent–contiguous section of disk space Constructs to link two pieces of data:Constructs to link two pieces of data:

Sequential storageSequential storage Pointers–field of data that can be used to locate Pointers–field of data that can be used to locate

related fields or recordsrelated fields or records


Figure 6-4 Physical file terminology in an Oracle environment


File OrganizationsFile Organizations Technique for physically arranging records of a file Technique for physically arranging records of a file

on secondary storageon secondary storage Factors for selecting file organization:Factors for selecting file organization:

Fast data retrieval and throughputFast data retrieval and throughput Efficient storage space utilizationEfficient storage space utilization Protection from failure and data lossProtection from failure and data loss Minimizing need for reorganizationMinimizing need for reorganization Accommodating growthAccommodating growth Security from unauthorized useSecurity from unauthorized use

Types of file organizationsTypes of file organizations SequentialSequential IndexedIndexed HashedHashed


Figure 6-7a Sequential file

organization

If not sortedAverage time to

find desired record = n/2

1

2

n

Records of the file are stored

in sequence by the primary key

field values

If sorted – every insert or delete requires

resort


Indexed File OrganizationsIndexed File Organizations Index–a separate table that contains Index–a separate table that contains

organization of records for quick retrievalorganization of records for quick retrieval Primary keys are automatically indexedPrimary keys are automatically indexed Oracle has a CREATE INDEX operation, and Oracle has a CREATE INDEX operation, and

MS ACCESS allows indexes to be created for MS ACCESS allows indexes to be created for most field typesmost field types

Indexing approaches:Indexing approaches: B-tree index, Fig. 6-7bB-tree index, Fig. 6-7b Bitmap index, Fig. 6-8Bitmap index, Fig. 6-8 Hash Index, Fig. 6-7cHash Index, Fig. 6-7c Join Index, Fig 6-9Join Index, Fig 6-9


Figure 6-7b B-tree index

uses a tree searchAverage time to find desired

record = depth of the tree

Leaves of the tree are all at same

level consistent access

time


Figure 6-7cHashed file or

index organization

Hash algorithmUsually uses division-

remainder to determine record position. Records

with same position are grouped in lists


Figure 6-8Bitmap index

index organization

Bitmap saves on space requirementsRows - possible values of the attribute

Columns - table rows

Bit indicates whether the attribute of a row has the values


Figure 6-9 Join Indexes–speeds up join operations



Clustering FilesClustering Files

In some relational DBMSs, related records In some relational DBMSs, related records from different tables can be stored from different tables can be stored together in the same disk areatogether in the same disk area

Useful for improving performance of join Useful for improving performance of join operationsoperations

Primary key records of the main table are Primary key records of the main table are stored adjacent to associated foreign key stored adjacent to associated foreign key records of the dependent tablerecords of the dependent table

e.g. Oracle has a CREATE CLUSTER e.g. Oracle has a CREATE CLUSTER commandcommand


Rules for Using IndexesRules for Using Indexes

1.1. Use on larger tablesUse on larger tables2.2. Index the primary key of each tableIndex the primary key of each table3.3. Index search fields (fields frequently Index search fields (fields frequently

in WHERE clause)in WHERE clause)4.4. Fields in SQL ORDER BY and GROUP Fields in SQL ORDER BY and GROUP

BY commandsBY commands5.5. When there are >100 values but When there are >100 values but

not when there are <30 valuesnot when there are <30 values


Rules for Using Indexes Rules for Using Indexes (cont.)(cont.)

6.6. Avoid use of indexes for fields with long Avoid use of indexes for fields with long values; perhaps compress values firstvalues; perhaps compress values first

7.7. DBMS may have limit on number of indexes DBMS may have limit on number of indexes per table and number of bytes per indexed per table and number of bytes per indexed field(s)field(s)

8.8. Null values will not be referenced from an Null values will not be referenced from an indexindex

9.9. Use indexes heavily for non-volatile Use indexes heavily for non-volatile databases; limit the use of indexes for databases; limit the use of indexes for volatile databasesvolatile databases

Why? Because modifications (e.g. inserts, deletes) require Why? Because modifications (e.g. inserts, deletes) require updates to occur in index filesupdates to occur in index files


RAIDRAID

Redundant Array of Inexpensive Redundant Array of Inexpensive DisksDisks

A set of disk drives that appear to A set of disk drives that appear to the user to be a single disk drivethe user to be a single disk drive

Allows parallel access to data Allows parallel access to data (improves access speed)(improves access speed)

Pages are arranged in Pages are arranged in stripesstripes


Figure 6-10RAID with four disks

and striping

Here, pages 1-4 can be

read/written simultaneously


Raid Types (Figure 6-10)Raid Types (Figure 6-10) Raid 0

Maximized parallelism No redundancy No error correction no fault-tolerance

Raid 1 Redundant data – fault tolerant Most common form

Raid 2 No redundancy One record spans across data

disks Error correction in multiple

disks– reconstruct damaged data

Raid 3 Error correction in one disk Record spans multiple data

disks (more than RAID2) Not good for multi-user

environments, Raid 4

Error correction in one disk Multiple records per stripe

Parallelism, but slow updates due to error correction

contention Raid 5

Rotating parity array Error correction takes place in

same disks as data storage Parallelism, better performance

than Raid4


Data

base

D

ata

base

A

rch

itect

ure

s

Arc

hit

ect

ure

s (

Figure

(F

igure

6

-11

)6

-11

)

Legacy

Systems

Current

Technology

Data

Warehouses


267267

Chapter 7:Chapter 7:Introduction to SQLIntroduction to SQL





ObjectivesObjectives Definition of termsDefinition of terms Interpret history and role of SQL Interpret history and role of SQL Define a database using SQL data Define a database using SQL data

definition languagedefinition language Write single table queries using SQLWrite single table queries using SQL Establish referential integrity using SQLEstablish referential integrity using SQL Discuss SQL:1999 and SQL:2003 Discuss SQL:1999 and SQL:2003

standardsstandards


SQL OverviewSQL Overview Structured Query LanguageStructured Query Language

The standard for relational database The standard for relational database management systems (RDBMS) management systems (RDBMS)

RDBMS: A database management RDBMS: A database management system that manages data as a system that manages data as a collection of tables in which all collection of tables in which all relationships are represented by relationships are represented by common values in related tablescommon values in related tables


History of SQLHistory of SQL 1970–E. Codd develops relational database 1970–E. Codd develops relational database

conceptconcept 1974-1979–System R with Sequel (later SQL) 1974-1979–System R with Sequel (later SQL)

created at IBM Research Labcreated at IBM Research Lab 1979–Oracle markets first relational DB with SQL1979–Oracle markets first relational DB with SQL 1986–ANSI SQL standard released1986–ANSI SQL standard released 1989, 1992, 1999, 2003–Major ANSI standard 1989, 1992, 1999, 2003–Major ANSI standard

updatesupdates Current–SQL is supported by most major Current–SQL is supported by most major

database vendorsdatabase vendors


Purpose of SQL StandardPurpose of SQL Standard Specify syntax/semantics for data Specify syntax/semantics for data

definition and manipulationdefinition and manipulation Define data structuresDefine data structures Enable portabilityEnable portability Specify minimal (level 1) and complete Specify minimal (level 1) and complete

(level 2) standards(level 2) standards Allow for later growth/enhancement to Allow for later growth/enhancement to

standardstandard


Benefits of a Standardized Benefits of a Standardized Relational LanguageRelational Language

Reduced training costsReduced training costs ProductivityProductivity Application portabilityApplication portability Application longevityApplication longevity Reduced dependence on a single Reduced dependence on a single

vendorvendor Cross-system communicationCross-system communication


SQL EnvironmentSQL Environment CatalogCatalog

A set of schemas that constitute the description of a databaseA set of schemas that constitute the description of a database SchemaSchema

The structure that contains descriptions of objects created by a The structure that contains descriptions of objects created by a user (base tables, views, constraints)user (base tables, views, constraints)

Data Definition Language (DDL)Data Definition Language (DDL) Commands that define a database, including creating, altering, Commands that define a database, including creating, altering,

and dropping tables and establishing constraintsand dropping tables and establishing constraints Data Manipulation Language (DML)Data Manipulation Language (DML)

Commands that maintain and query a databaseCommands that maintain and query a database Data Control Language (DCL)Data Control Language (DCL)

Commands that control a database, including administering Commands that control a database, including administering privileges and committing dataprivileges and committing data


Figure 7-1A simplified schematic of a typical SQL environment, as

described by the SQL-2003 standard


Some SQL Data types Some SQL Data types


Figure 7-4 DDL, DML, DCL, and the database development process


SQL Database DefinitionSQL Database Definition Data Definition Language (DDL)Data Definition Language (DDL) Major CREATE statements:Major CREATE statements:

CREATE SCHEMA–defines a portion of the CREATE SCHEMA–defines a portion of the database owned by a particular userdatabase owned by a particular user

CREATE TABLE–defines a table and its CREATE TABLE–defines a table and its columnscolumns

CREATE VIEW–defines a logical table from CREATE VIEW–defines a logical table from one or more viewsone or more views

Other CREATE statements: CHARACTER Other CREATE statements: CHARACTER SET, COLLATION, TRANSLATION, SET, COLLATION, TRANSLATION, ASSERTION, DOMAINASSERTION, DOMAIN


Table CreationTable CreationFigure 7-5 General syntax for CREATE TABLE

Steps in table creation:

1. Identify data types for attributes

2. Identify columns that can and cannot be

null

3. Identify columns that must be unique

(candidate keys)

4. Identify primary key–foreign key mates

5. Determine default values

6. Identify constraints on columns (domain

specifications)

7. Create the table and associated indexes


The following slides create tables The following slides create tables for this enterprise data modelfor this enterprise data model


Figure 7-6 SQL database definition commands for Pine Valley Furniture

Overall table definitions


Defining attributes and their data types


Non-nullable specification

Identifying primary key

Primary keys can never have

NULL values


Non-nullable specifications

Primary key

Some primary keys are composite– composed of multiple attributes


Default value

Domain constraint

Controlling the values in attributes


Primary key of parent table

Identifying foreign keys and establishing relationships

Foreign key of dependent table


Data Integrity ControlsData Integrity Controls Referential integrity–constraint Referential integrity–constraint

that ensures that foreign key that ensures that foreign key values of a table must match values of a table must match primary key values of a related primary key values of a related table in 1:M relationshipstable in 1:M relationships

Restricting:Restricting: Deletes of primary recordsDeletes of primary records Updates of primary recordsUpdates of primary records Inserts of dependent recordsInserts of dependent records


Relational integrity is

enforced via the primary-

key to foreign-key match

Figure 7-7 Ensuring data integrity through updates


Changing and Removing Changing and Removing TablesTables

ALTER TABLE statement allows you ALTER TABLE statement allows you to change column specifications:to change column specifications: ALTER TABLE CUSTOMER_T ADD (TYPE ALTER TABLE CUSTOMER_T ADD (TYPE

VARCHAR(2))VARCHAR(2)) DROP TABLE statement allows you to DROP TABLE statement allows you to

remove tables from your schema:remove tables from your schema: DROP TABLE CUSTOMER_TDROP TABLE CUSTOMER_T


Schema DefinitionSchema Definition Control processing/storage efficiency:Control processing/storage efficiency:

Choice of indexesChoice of indexes File organizations for base tablesFile organizations for base tables File organizations for indexesFile organizations for indexes Data clusteringData clustering Statistics maintenanceStatistics maintenance

Creating indexesCreating indexes Speed up random/sequential access to base table dataSpeed up random/sequential access to base table data ExampleExample

CREATE INDEX NAME_IDX ON CUSTOMER_T(CUSTOMER_NAME)CREATE INDEX NAME_IDX ON CUSTOMER_T(CUSTOMER_NAME) This makes an index for the CUSTOMER_NAME field of the This makes an index for the CUSTOMER_NAME field of the

CUSTOMER_T tableCUSTOMER_T table


Insert StatementInsert Statement Adds data to a tableAdds data to a table Inserting into a tableInserting into a table

INSERT INTO CUSTOMER_T VALUES (001, ‘Contemporary INSERT INTO CUSTOMER_T VALUES (001, ‘Contemporary Casuals’, ‘1355 S. Himes Blvd.’, ‘Gainesville’, ‘FL’, Casuals’, ‘1355 S. Himes Blvd.’, ‘Gainesville’, ‘FL’, 32601);32601);

Inserting a record that has some null attributes Inserting a record that has some null attributes requires identifying the fields that actually get requires identifying the fields that actually get datadata INSERT INTO PRODUCT_T (PRODUCT_ID, INSERT INTO PRODUCT_T (PRODUCT_ID,

PRODUCT_DESCRIPTION,PRODUCT_FINISH, STANDARD_PRICE, PRODUCT_DESCRIPTION,PRODUCT_FINISH, STANDARD_PRICE, PRODUCT_ON_HAND) VALUES (1, ‘End Table’, ‘Cherry’, 175, 8);PRODUCT_ON_HAND) VALUES (1, ‘End Table’, ‘Cherry’, 175, 8);

Inserting from another tableInserting from another table INSERT INTO CA_CUSTOMER_T SELECT * FROM CUSTOMER_T INSERT INTO CA_CUSTOMER_T SELECT * FROM CUSTOMER_T

WHERE STATE = ‘CA’;WHERE STATE = ‘CA’;


Creating Tables with Identity Creating Tables with Identity ColumnsColumns

Inserting into a table does not require explicit customer ID Inserting into a table does not require explicit customer ID entry or field listentry or field list

INSERT INTO CUSTOMER_T VALUES ( ‘Contemporary INSERT INTO CUSTOMER_T VALUES ( ‘Contemporary Casuals’, ‘1355 S. Himes Blvd.’, ‘Gainesville’, ‘FL’, 32601);Casuals’, ‘1355 S. Himes Blvd.’, ‘Gainesville’, ‘FL’, 32601);

New with SQL:2003


Delete StatementDelete Statement

Removes rows from a tableRemoves rows from a table Delete certain rowsDelete certain rows

DELETE FROM CUSTOMER_T WHERE DELETE FROM CUSTOMER_T WHERE STATE = ‘HI’;STATE = ‘HI’;

Delete all rowsDelete all rows DELETE FROM CUSTOMER_T;DELETE FROM CUSTOMER_T;


Update StatementUpdate Statement

Modifies data in existing rowsModifies data in existing rows

UPDATE PRODUCT_T SET UNIT_PRICE = UPDATE PRODUCT_T SET UNIT_PRICE = 775 WHERE PRODUCT_ID = 7;775 WHERE PRODUCT_ID = 7;


Merge StatementMerge Statement

Makes it easier to update a table…allows combination of Insert and Update in one statement

Useful for updating master tables with new data


SELECT StatementSELECT Statement Used for queries on single or multiple tablesUsed for queries on single or multiple tables Clauses of the SELECT statement:Clauses of the SELECT statement:

SELECTSELECT List the columns (and expressions) that should be returned from the queryList the columns (and expressions) that should be returned from the query

FROMFROM Indicate the table(s) or view(s) from which data will be obtainedIndicate the table(s) or view(s) from which data will be obtained

WHEREWHERE Indicate the conditions under which a row will be included in the resultIndicate the conditions under which a row will be included in the result

GROUP BYGROUP BY Indicate categorization of results Indicate categorization of results

HAVINGHAVING Indicate the conditions under which a category (group) will be includedIndicate the conditions under which a category (group) will be included

ORDER BYORDER BY Sorts the result according to specified criteriaSorts the result according to specified criteria


Figure 7-10 SQL statement

processing order (adapted

from van der Lans, p.100)


SELECT ExampleSELECT Example

Find products with standard price less Find products with standard price less than $275than $275

SELECTSELECT PRODUCT_NAME, STANDARD_PRICE PRODUCT_NAME, STANDARD_PRICE

FROMFROM PRODUCT_V PRODUCT_V

WHEREWHERE STANDARD_PRICE < 275; STANDARD_PRICE < 275;

Table 7-3: Comparison Operators in SQL


SELECT Example Using AliasSELECT Example Using Alias

Alias is an alternative column or table Alias is an alternative column or table namename

SELECT SELECT CUSTCUST.CUSTOMER AS .CUSTOMER AS NAMENAME, , CUST.CUSTOMER_ADDRESS CUST.CUSTOMER_ADDRESS

FROM CUSTOMER_V FROM CUSTOMER_V CUSTCUST

WHERE WHERE NAMENAME = ‘Home = ‘Home Furnishings’;Furnishings’;


SELECT Example SELECT Example Using a FunctionUsing a Function

Using the COUNT Using the COUNT aggregate functionaggregate function to find totalsto find totals

SELECT SELECT COUNT(*)COUNT(*) FROM ORDER_LINE_V FROM ORDER_LINE_VWHERE ORDER_ID = 1004;WHERE ORDER_ID = 1004;

Note: with aggregate functions you can’t Note: with aggregate functions you can’t have single-valued columns included in the have single-valued columns included in the SELECT clauseSELECT clause


SELECT Example–Boolean SELECT Example–Boolean OperatorsOperators

ANDAND, , OROR, and , and NOTNOT Operators for customizing Operators for customizing conditions in WHERE clauseconditions in WHERE clause

SELECT PRODUCT_DESCRIPTION, PRODUCT_FINISH, SELECT PRODUCT_DESCRIPTION, PRODUCT_FINISH, STANDARD_PRICESTANDARD_PRICE

FROM PRODUCT_VFROM PRODUCT_V

WHERE (PRODUCT_DESCRIPTION WHERE (PRODUCT_DESCRIPTION LIKELIKE ‘ ‘%%Desk’Desk’

OROR PRODUCT_DESCRIPTION PRODUCT_DESCRIPTION LIKELIKE ‘ ‘%%Table’) Table’)

ANDAND UNIT_PRICE > 300; UNIT_PRICE > 300;

Note: the LIKE operator allows you to compare strings using wildcards. For example, the % wildcard in ‘%Desk’ indicates

that all strings that have any number of characters preceding the word “Desk” will be allowed


Venn Diagram from Previous Venn Diagram from Previous QueryQuery


SELECT Example – SELECT Example – Sorting Results with the ORDER BY Sorting Results with the ORDER BY

ClauseClause Sort the results first by STATE, and Sort the results first by STATE, and within a state by CUSTOMER_NAMEwithin a state by CUSTOMER_NAME

SELECT CUSTOMER_NAME, CITY, STATESELECT CUSTOMER_NAME, CITY, STATE

FROM CUSTOMER_VFROM CUSTOMER_V

WHERE STATE WHERE STATE ININ (‘FL’, ‘TX’, ‘CA’, ‘HI’) (‘FL’, ‘TX’, ‘CA’, ‘HI’)

ORDER BYORDER BY STATE, CUSTOMER_NAME; STATE, CUSTOMER_NAME;

Note: the IN operator in this example allows you to include rows whose STATE value is either FL, TX, CA, or HI. It is more efficient than separate

OR conditions


SELECT Example– SELECT Example– Categorizing Results Using the GROUP BY Categorizing Results Using the GROUP BY

ClauseClause For use with aggregate functionsFor use with aggregate functions

Scalar aggregateScalar aggregate: single value returned from SQL query with : single value returned from SQL query with aggregate functionaggregate function

Vector aggregateVector aggregate: multiple values returned from SQL query : multiple values returned from SQL query with aggregate function (via GROUP BY)with aggregate function (via GROUP BY)

SELECT CUSTOMER_STATE, COUNT(CUSTOMER_STATE) SELECT CUSTOMER_STATE, COUNT(CUSTOMER_STATE)


GROUP BYGROUP BY CUSTOMER_STATE; CUSTOMER_STATE;

Note: you can use single-value fields with aggregate Note: you can use single-value fields with aggregate functions if they are included in the GROUP BY clausefunctions if they are included in the GROUP BY clause


SELECT Example– SELECT Example– Qualifying Results by Categories Qualifying Results by Categories

Using the HAVING ClauseUsing the HAVING Clause For use with GROUP BYFor use with GROUP BY

SELECT CUSTOMER_STATE, SELECT CUSTOMER_STATE, COUNT(CUSTOMER_STATE) COUNT(CUSTOMER_STATE)


GROUP BY CUSTOMER_STATEGROUP BY CUSTOMER_STATE

HAVINGHAVING COUNT(CUSTOMER_STATE) > 1; COUNT(CUSTOMER_STATE) > 1;

Like a WHERE clause, but it operates on groups (categories), not Like a WHERE clause, but it operates on groups (categories), not on individual rows. Here, only those groups with total numbers on individual rows. Here, only those groups with total numbers greater than 1 will be included in final resultgreater than 1 will be included in final result


Using and Defining ViewsUsing and Defining Views Views provide users controlled access to tablesViews provide users controlled access to tables Base Table–table containing the raw dataBase Table–table containing the raw data Dynamic ViewDynamic View

A “virtual table” created dynamically upon request by a user A “virtual table” created dynamically upon request by a user No data actually stored; instead data from base table made No data actually stored; instead data from base table made

available to useravailable to user Based on SQL SELECT statement on base tables or other viewsBased on SQL SELECT statement on base tables or other views

Materialized ViewMaterialized View Copy or replication of dataCopy or replication of data Data actually storedData actually stored Must be refreshed periodically to match the corresponding Must be refreshed periodically to match the corresponding

base tablesbase tables


Sample CREATE VIEWSample CREATE VIEWCREATE VIEW EXPENSIVE_STUFF_V ASCREATE VIEW EXPENSIVE_STUFF_V AS

SELECT PRODUCT_ID, PRODUCT_NAME, UNIT_PRICESELECT PRODUCT_ID, PRODUCT_NAME, UNIT_PRICE

FROM PRODUCT_TFROM PRODUCT_T

WHERE UNIT_PRICE >300WHERE UNIT_PRICE >300

WITH CHECK_OPTION;WITH CHECK_OPTION;

View has a nameView is based on a SELECT statementCHECK_OPTION works only for

updateable views and prevents updates that would create rows not included in

the view


Advantages of ViewsAdvantages of Views Simplify query commandsSimplify query commands Assist with data security (but don't rely on Assist with data security (but don't rely on

views for security, there are more views for security, there are more important security measures)important security measures)

Enhance programming productivityEnhance programming productivity Contain most current base table dataContain most current base table data Use little storage spaceUse little storage space Provide customized view for userProvide customized view for user Establish physical data independenceEstablish physical data independence


Disadvantages of ViewsDisadvantages of Views Use processing time each time view Use processing time each time view

is referencedis referenced May or may not be directly May or may not be directly

updateableupdateable


309309

Chapter 8:Chapter 8:Advanced SQLAdvanced SQL





ObjectivesObjectives Definition of termsDefinition of terms Write multiple table SQL queriesWrite multiple table SQL queries Define and use three types of joinsDefine and use three types of joins Write correlated and noncorrelated Write correlated and noncorrelated

subqueriessubqueries Establish referential integrity in SQLEstablish referential integrity in SQL Understand triggers and stored proceduresUnderstand triggers and stored procedures Discuss SQL:1999 standard and its Discuss SQL:1999 standard and its

extension of SQL-92extension of SQL-92


Processing Multiple Tables–Processing Multiple Tables–JoinsJoins

JoinJoin––a relational operation that causes two or more tables a relational operation that causes two or more tables with a common domain to be combined into a single table with a common domain to be combined into a single table or viewor view

Equi-joinEqui-join––a join in which the joining condition is based a join in which the joining condition is based on equality between values in the common columns; on equality between values in the common columns; common columns appear redundantly in the result tablecommon columns appear redundantly in the result table

Natural joinNatural join––an equi-join in which one of the duplicate an equi-join in which one of the duplicate columns is eliminated in the result tablecolumns is eliminated in the result table

Outer joinOuter join––a join in which rows that do not have a join in which rows that do not have matching values in common columns are nonetheless matching values in common columns are nonetheless included in the result table (as opposed to included in the result table (as opposed to innerinner join, in join, in which rows must have matching values in order to appear which rows must have matching values in order to appear in the result table)in the result table)

Union joinUnion join––includes all columns from each table in the includes all columns from each table in the join, and an instance for each row of each tablejoin, and an instance for each row of each table

The common columns in joined tables are usually the primary key of the dominant table and the foreign key of the dependent table in 1:M relationships


The following slides create tables The following slides create tables for this enterprise data modelfor this enterprise data model


These tables are used in queries that follow

Figure 8-1 Pine Valley Furniture Company Customer and Order tables with pointers from customers to their orders


For each customer who placed an order, what is For each customer who placed an order, what is the customer’s name and order number?the customer’s name and order number?

SELECT CUSTOMER_T.CUSTOMER_ID, CUSTOMER_NAME, ORDER_IDSELECT CUSTOMER_T.CUSTOMER_ID, CUSTOMER_NAME, ORDER_IDFROM CUSTOMER_T NATURAL JOIN ORDER_T ON FROM CUSTOMER_T NATURAL JOIN ORDER_T ON

CUSTOMER_T.CUSTOMER_ID = ORDER_T.CUSTOMER_ID;CUSTOMER_T.CUSTOMER_ID = ORDER_T.CUSTOMER_ID;

Join involves multiple tables in FROM clause

Natural Join ExampleNatural Join Example

ON clause performs the equality check for common columns of the

two tables

Note: from Fig. 1, you see that only 10

Customers have links with orders.

Only 10 rows will be returned from this

INNER join.


List the customer name, ID number, and order List the customer name, ID number, and order number for all customers. Include customer number for all customers. Include customer information even for customers that do have an orderinformation even for customers that do have an order

SELECT CUSTOMER_T.CUSTOMER_ID, CUSTOMER_NAME, ORDER_IDSELECT CUSTOMER_T.CUSTOMER_ID, CUSTOMER_NAME, ORDER_IDFROM CUSTOMER_T, LEFT OUTER JOIN ORDER_TFROM CUSTOMER_T, LEFT OUTER JOIN ORDER_TON CUSTOMER_T.CUSTOMER_ID = ORDER_T.CUSTOMER_ID;ON CUSTOMER_T.CUSTOMER_ID = ORDER_T.CUSTOMER_ID;

Outer Join Example Outer Join Example (Microsoft Syntax)(Microsoft Syntax)

LEFT OUTER JOIN syntax with ON causes customer

data to appear even if there is no corresponding

order data

Unlike INNER join, this will include customer

rows with no matching order rows


Results

Unlike INNER

join, this will

include customer rows with

no matching

order rows


Assemble all information necessary to create an invoice for order Assemble all information necessary to create an invoice for order number 1006number 1006

SELECT CUSTOMER_T.CUSTOMER_ID, CUSTOMER_NAME, CUSTOMER_ADDRESS, CITY, SATE, SELECT CUSTOMER_T.CUSTOMER_ID, CUSTOMER_NAME, CUSTOMER_ADDRESS, CITY, SATE, POSTAL_CODE, ORDER_T.ORDER_ID, ORDER_DATE, QUANTITY, PRODUCT_DESCRIPTION, POSTAL_CODE, ORDER_T.ORDER_ID, ORDER_DATE, QUANTITY, PRODUCT_DESCRIPTION, STANDARD_PRICE, (QUANTITY * UNIT_PRICE)STANDARD_PRICE, (QUANTITY * UNIT_PRICE)

FROM CUSTOMER_T, ORDER_T, ORDER_LINE_T, PRODUCT_TFROM CUSTOMER_T, ORDER_T, ORDER_LINE_T, PRODUCT_T

WHERE CUSTOMER_T.CUSTOMER_ID = ORDER_LINE.CUSTOMER_ID WHERE CUSTOMER_T.CUSTOMER_ID = ORDER_LINE.CUSTOMER_ID AND AND ORDER_T.ORDER_ID = ORDER_LINE_T.ORDER_ID ORDER_T.ORDER_ID = ORDER_LINE_T.ORDER_ID

AND ORDER_LINE_T.PRODUCT_ID = PRODUCT_PRODUCT_IDAND ORDER_LINE_T.PRODUCT_ID = PRODUCT_PRODUCT_IDAND ORDER_T.ORDER_ID = 1006;AND ORDER_T.ORDER_ID = 1006;

Four tables involved in this join

Multiple Table Join Multiple Table Join ExampleExample

Each pair of tables requires an equality-check condition in the WHERE clause, matching primary keys against foreign keys


Figure 8-2 Results from a four-table join

From CUSTOMER_T table

From ORDER_T table From PRODUCT_T table


Processing Multiple Tables Processing Multiple Tables Using SubqueriesUsing Subqueries

Subquery–placing an inner query (SELECT Subquery–placing an inner query (SELECT statement) inside an outer querystatement) inside an outer query

Options:Options: In a condition of the WHERE clauseIn a condition of the WHERE clause As a “table” of the FROM clauseAs a “table” of the FROM clause Within the HAVING clauseWithin the HAVING clause

Subqueries can be:Subqueries can be: Noncorrelated–executed once for the entire Noncorrelated–executed once for the entire

outer queryouter query Correlated–executed once for each row returned Correlated–executed once for each row returned

by the outer queryby the outer query


Show all customers who have placed an Show all customers who have placed an orderorder

SELECT CUSTOMER_NAME FROM CUSTOMER_TSELECT CUSTOMER_NAME FROM CUSTOMER_TWHERE CUSTOMER_ID INWHERE CUSTOMER_ID IN

(SELECT DISTINCT CUSTOMER_ID FROM ORDER_T);(SELECT DISTINCT CUSTOMER_ID FROM ORDER_T);

Subquery ExampleSubquery Example

Subquery is embedded in parentheses. In this case it

returns a list that will be used in the WHERE clause of the

outer query

The IN operator will test to see if the CUSTOMER_ID value of a row is

included in the list returned from the subquery


Correlated vs. Correlated vs. Noncorrelated SubqueriesNoncorrelated Subqueries

Noncorrelated subqueries:Noncorrelated subqueries: Do not depend on data from the outer queryDo not depend on data from the outer query Execute once for the entire outer queryExecute once for the entire outer query

Correlated subqueries:Correlated subqueries: Make use of data from the outer queryMake use of data from the outer query Execute once for each row of the outer queryExecute once for each row of the outer query Can use the EXISTS operatorCan use the EXISTS operator


Figure 8-3a Processing a

noncorrelated subquery

No reference to data in outer query, so

subquery executes once only

These are the only customers that have IDs in the ORDER_T table

1. The subquery executes and

returns the customer IDs

from the ORDER_T table

2. The outer query on the results of

the subquery


Show all orders that include furniture finished in natural ashShow all orders that include furniture finished in natural ash

SELECT DISTINCT ORDER_ID FROM ORDER_LINE_TSELECT DISTINCT ORDER_ID FROM ORDER_LINE_TWHERE EXISTSWHERE EXISTS

(SELECT * FROM PRODUCT_T (SELECT * FROM PRODUCT_T WHERE PRODUCT_ID = ORDER_LINE_T.PRODUCT_ID WHERE PRODUCT_ID = ORDER_LINE_T.PRODUCT_ID AND PRODUCT_FINISH = ‘Natural ash’);AND PRODUCT_FINISH = ‘Natural ash’);

Correlated Subquery Correlated Subquery ExampleExample

The subquery is testing for a value that comes from the outer query

The EXISTS operator will return a TRUE value if the subquery resulted

in a non-empty set, otherwise it returns a FALSE


Figure 8-3b Processing a

correlated subquery

Subquery refers to outer-query data, so executes once for each row of outer query

Note: only the orders that

involve products with

Natural Ash will be

included in the final

results


Show all products whose standard price is higher than the average Show all products whose standard price is higher than the average priceprice

SELECT PRODUCT_DESCRIPTION, STANDARD_PRICE, AVGPRICESELECT PRODUCT_DESCRIPTION, STANDARD_PRICE, AVGPRICEFROMFROM

(SELECT AVG(STANDARD_PRICE) AVGPRICE FROM PRODUCT_T),(SELECT AVG(STANDARD_PRICE) AVGPRICE FROM PRODUCT_T),PRODUCT_TPRODUCT_TWHERE STANDARD_PRICE > AVG_PRICE;WHERE STANDARD_PRICE > AVG_PRICE;

Another Subquery ExampleAnother Subquery Example

The WHERE clause normally cannot include aggregate functions, but because the aggregate is performed in the subquery its result can be used in the outer query’s

WHERE clause

One column of the subquery is an aggregate function that has

an alias name. That alias can then be referred to in the outer

query

Subquery forms the derived table used in the FROM clause of the

outer query


Union QueriesUnion Queries Combine the output (union of multiple Combine the output (union of multiple

queries) together into a single result tablequeries) together into a single result table

First query

Second query

Combine


Conditional Expressions Using Conditional Expressions Using Case SyntaxCase Syntax

This is available with This is available with newer versions of newer versions of SQL, previously not SQL, previously not part of the part of the standardstandard


Ensuring Transaction Ensuring Transaction IntegrityIntegrity

Transaction = A discrete unit of work Transaction = A discrete unit of work that must be completely processed or that must be completely processed or not processed at allnot processed at all May involve multiple updatesMay involve multiple updates If any update fails, then all other updates If any update fails, then all other updates

must be cancelledmust be cancelled SQL commands for transactionsSQL commands for transactions

BEGIN TRANSACTION/END TRANSACTIONBEGIN TRANSACTION/END TRANSACTION Marks boundaries of a transactionMarks boundaries of a transaction

COMMITCOMMIT Makes all updates permanentMakes all updates permanent

ROLLBACKROLLBACK Cancels updates since the last COMMITCancels updates since the last COMMIT


Figure 8-5 An SQL Transaction sequence (in pseudocode)


Data Dictionary FacilitiesData Dictionary Facilities

System tables that store metadataSystem tables that store metadata Users usually can view some of these tablesUsers usually can view some of these tables Users are restricted from updating themUsers are restricted from updating them Some examples in Oracle 10gSome examples in Oracle 10g

DBA_TABLES–descriptions of tablesDBA_TABLES–descriptions of tables DBA_CONSTRAINTS–description of constraintsDBA_CONSTRAINTS–description of constraints DBA_USERS–information about the users of the systemDBA_USERS–information about the users of the system

Examples in Microsoft SQL Server 2000Examples in Microsoft SQL Server 2000 SYSCOLUMNS–table and column definitionsSYSCOLUMNS–table and column definitions SYSDEPENDS–object dependencies based on foreign SYSDEPENDS–object dependencies based on foreign

keyskeys SYSPERMISSIONS–access permissions granted to usersSYSPERMISSIONS–access permissions granted to users


SQL:1999 and SQL:2003 SQL:1999 and SQL:2003 Enhancements/ExtensionsEnhancements/Extensions

User-defined data types (UDT)User-defined data types (UDT) Subclasses of standard types or an object typeSubclasses of standard types or an object type

Analytical functions (for OLAP)Analytical functions (for OLAP) CEILING, FLOOR, SQRT, RANK, DENSE_RANKCEILING, FLOOR, SQRT, RANK, DENSE_RANK WINDOW–improved numerical analysis WINDOW–improved numerical analysis

capabilitiescapabilities New Data TypesNew Data Types

BIGINT, MULTISET (collection), XMLBIGINT, MULTISET (collection), XML CREATE TABLE LIKE–create a new table CREATE TABLE LIKE–create a new table

similar to an existing onesimilar to an existing one MERGEMERGE


Persistent Stored Modules (SQL/PSM)Persistent Stored Modules (SQL/PSM) Capability to create and drop code Capability to create and drop code

modulesmodules New statements:New statements:

CASE, IF, LOOP, FOR, WHILE, etc.CASE, IF, LOOP, FOR, WHILE, etc. Makes SQL into a procedural languageMakes SQL into a procedural language

Oracle has propriety version called Oracle has propriety version called PL/SQL, and Microsoft SQL Server PL/SQL, and Microsoft SQL Server has Transact/SQLhas Transact/SQL

SQL:1999 and SQL:2003 SQL:1999 and SQL:2003 Enhancements/Extensions Enhancements/Extensions

(cont.)(cont.)


Routines and TriggersRoutines and Triggers RoutinesRoutines

Program modules that execute on demandProgram modules that execute on demand FunctionsFunctions–routines that return values and –routines that return values and

take input parameterstake input parameters ProceduresProcedures–routines that do not return –routines that do not return

values and can take input or output values and can take input or output parametersparameters

TriggersTriggers Routines that execute in response to a Routines that execute in response to a

database event (INSERT, UPDATE, or database event (INSERT, UPDATE, or DELETE)DELETE)


Figure 8-6 Triggers contrasted with stored procedures

Procedures are called explicitly

Triggers are event-drivenSource: adapted from Mullins, 1995.


Figure 8-7 Simplified trigger syntax, SQL:2003

Figure 8-8 Create routine syntax, SQL:2003


Embedded and Dynamic Embedded and Dynamic SQLSQL

Embedded SQLEmbedded SQL Including hard-coded SQL statements in Including hard-coded SQL statements in

a program written in another language a program written in another language such as C or Javasuch as C or Java

Dynamic SQLDynamic SQL Ability for an application program to Ability for an application program to

generate SQL code on the fly, as the generate SQL code on the fly, as the application is runningapplication is running


337337

Chapter 9:Chapter 9: The Client/Server The Client/Server

Database EnvironmentDatabase Environment





ObjectivesObjectives Definition of termsDefinition of terms List advantages of client/server architectureList advantages of client/server architecture Explain three application components: Explain three application components:

presentation, processing, and storagepresentation, processing, and storage Suggest partitioning possibilitiesSuggest partitioning possibilities Distinguish between file server, database Distinguish between file server, database

server, 3-tier, and n-tier approachesserver, 3-tier, and n-tier approaches Describe and discuss middlewareDescribe and discuss middleware Explain database linking via ODBC and JDBCExplain database linking via ODBC and JDBC


Client/Server SystemsClient/Server Systems

Networked computing modelNetworked computing model Processes distributed between clients Processes distributed between clients

and serversand servers Client–Workstation (usually a PC) that Client–Workstation (usually a PC) that

requests and uses a servicerequests and uses a service Server–Computer (PC/mini/mainframe) Server–Computer (PC/mini/mainframe)

that provides a servicethat provides a service For DBMS, server is a database serverFor DBMS, server is a database server


Application Logic in C/S Application Logic in C/S SystemsSystems

GUI Interface

Procedures, functions,programs

DBMS activities

Processing LogicProcessing Logic I/O processingI/O processing Business rulesBusiness rules

Data managementData management

Storage LogicStorage Logic Data storage/retrievalData storage/retrieval

Presentation LogicPresentation Logic Input–keyboard/mouseInput–keyboard/mouse Output–monitor/printerOutput–monitor/printer


Client/Server ArchitecturesClient/Server Architectures

File Server ArchitectureFile Server Architecture

Database Server ArchitectureDatabase Server Architecture

Three-tier ArchitectureThree-tier Architecture

Client does extensive processing

Client does little processing


File Server ArchitectureFile Server Architecture

All processing is done at the PC that All processing is done at the PC that requested the data requested the data

Entire files are transferred from the server Entire files are transferred from the server to the client for processingto the client for processing

Problems:Problems: Huge amount of data transfer on the networkHuge amount of data transfer on the network Each client must contain full DBMS Each client must contain full DBMS

Heavy resource demand on clientsHeavy resource demand on clients Client DBMSs must recognize shared locks, integrity Client DBMSs must recognize shared locks, integrity

checks, etc.checks, etc.

FAT FAT CLIENTCLIENT


Figure 9-2 File Server Architecture

FAT FAT CLIENTCLIENT


Two-Tier Database Server Two-Tier Database Server ArchitecturesArchitectures

Client is responsible for Client is responsible for I/O processing logic I/O processing logic Some business rules logicSome business rules logic

Server performs all data storage Server performs all data storage and access processing and access processing DBMS is only on serverDBMS is only on server


Advantages of Two-Tier Advantages of Two-Tier ApproachApproach

Clients do not have to be as powerfulClients do not have to be as powerful Greatly reduces data traffic on the Greatly reduces data traffic on the

networknetwork Improved data integrity since it is all Improved data integrity since it is all

processed centrallyprocessed centrally Stored proceduresStored procedures DBMS code that DBMS code that

performs some business rules done on performs some business rules done on serverserver


Advantages of Advantages of Stored ProceduresStored Procedures

Compiled SQL statementsCompiled SQL statements Reduced network trafficReduced network traffic Improved securityImproved security Improved data integrityImproved data integrity Thinner clientsThinner clients


Figure 9-3 Two-tier database server architecture

ThinneThinner r

clientsclients

DBMS DBMS only on only on serverserver


Three-Tier ArchitecturesThree-Tier Architectures

Thin Client PC just for user interface and a little application

processing. Limited or no data storage (sometimes no hard drive)

GUI interface (I/O processing)

Browser

Business rules Web Server

Data storage DBMS

ClientClient

Application serverApplication server

Database serverDatabase server


Figure 9-4 Three-tier architecture

Thinnest Thinnest clientsclients

Business rules Business rules on separate on separate

serverserver

DBMS only DBMS only on DB serveron DB server


Advantages of Three-Tier Advantages of Three-Tier ArchitecturesArchitectures

ScalabilityScalability Technological flexibilityTechnological flexibility Long-term cost reductionLong-term cost reduction Better match of systems to business Better match of systems to business

needsneeds Improved customer serviceImproved customer service Competitive advantageCompetitive advantage Reduced riskReduced risk


Application PartitioningApplication Partitioning

Placing portions of the application Placing portions of the application code in different locations (client vs. code in different locations (client vs. server) AFTER it is writtenserver) AFTER it is written

AdvantagesAdvantages Improved performanceImproved performance Improved interoperabilityImproved interoperability Balanced workloadsBalanced workloads


Common Logic Common Logic DistributionsDistributions

Figure 9-5a Two-tier client-server environment

Figure 9-5b n-tier client-server environment

Processing logic could be at client, server, or both

Processing logic will be at

application server or Web server


Role of the MainframeRole of the Mainframe Mission-critical legacy systems have tended to Mission-critical legacy systems have tended to

remain on mainframes remain on mainframes Distributed client/server systems tend to be used for Distributed client/server systems tend to be used for

smaller, workgroup systemssmaller, workgroup systems Difficulties in moving mission critical systems from Difficulties in moving mission critical systems from

mainframe to distributedmainframe to distributed Determining which code belongs on server vs. clientDetermining which code belongs on server vs. client Identifying potential conflicts with code from other Identifying potential conflicts with code from other

applicationsapplications Ensuring sufficient resources exist for anticipated loadEnsuring sufficient resources exist for anticipated load

Rule of thumbRule of thumb Mainframe for centralized data that does not need to be Mainframe for centralized data that does not need to be

movedmoved Client for data requiring frequent user access, complex Client for data requiring frequent user access, complex

graphics, and user interfacegraphics, and user interface


MiddlewareMiddleware

Software that allows an application Software that allows an application to to interoperateinteroperate with other software with other software

No need for programmer/user to No need for programmer/user to understand internal processingunderstand internal processing

Accomplished via Accomplished via Application Application Program InterfaceProgram Interface (API)(API)

The “glue”“glue” that holds client/server applications together


Types of MiddlewareTypes of Middleware Remote Procedure Calls (RPC) Remote Procedure Calls (RPC)

client makes calls to procedures running on remote computersclient makes calls to procedures running on remote computers synchronous and asynchronoussynchronous and asynchronous

Message-Oriented Middleware (MOM) Message-Oriented Middleware (MOM) asynchronous calls between the client via message queuesasynchronous calls between the client via message queues

Publish/SubscribePublish/Subscribe push technology push technology server sends information to client when available server sends information to client when available

Object Request Broker (ORB)Object Request Broker (ORB) object-oriented management of communications between clients object-oriented management of communications between clients

and serversand servers SQL-oriented Data AccessSQL-oriented Data Access

middleware between applications and database serversmiddleware between applications and database servers


Database MiddlewareDatabase Middleware ODBCODBC–Open Database Connectivity–Open Database Connectivity

Most DB vendors support thisMost DB vendors support this

OLE-DBOLE-DB Microsoft enhancement of ODBCMicrosoft enhancement of ODBC

JDBCJDBC–Java Database Connectivity–Java Database Connectivity Special Java classes that allow Java Special Java classes that allow Java

applications/applets to connect to applications/applets to connect to databasesdatabases


Client/Server SecurityClient/Server Security Network environment Network environment complex complex

security issuessecurity issues Security levels:Security levels:

System-level password securitySystem-level password security for allowing access to the systemfor allowing access to the system

Database-level password securityDatabase-level password security for determining access privileges to tables; for determining access privileges to tables;

read/update/insert/delete privilegesread/update/insert/delete privileges Secure client/server communication Secure client/server communication

via encryptionvia encryption


Keys to Successful Client-Keys to Successful Client-Server ImplementationServer Implementation

Accurate business problem analysisAccurate business problem analysis Detailed architecture analysisDetailed architecture analysis Architecture analysis Architecture analysis beforebefore choosing tools choosing tools Appropriate scalabilityAppropriate scalability Appropriate placement of servicesAppropriate placement of services Network analysisNetwork analysis Awareness of hidden costsAwareness of hidden costs Establish client/server securityEstablish client/server security


Benefits of Moving to Benefits of Moving to Client/Server ArchitectureClient/Server Architecture

Staged delivery of functionality speeds Staged delivery of functionality speeds deploymentdeployment

GUI interfaces ease application useGUI interfaces ease application use Flexibility and scalability facilitates Flexibility and scalability facilitates

business process reengineeringbusiness process reengineering Reduced network traffic due to Reduced network traffic due to

increased processing at data sourceincreased processing at data source Facilitation of Web-enabled Facilitation of Web-enabled

applicationsapplications


Using ODBC to Link External Using ODBC to Link External Databases Stored on a Database Databases Stored on a Database

ServerServer Open Database Connectivity (ODBC)Open Database Connectivity (ODBC) API provides a common language for application programs to API provides a common language for application programs to

access and process SQL databases independent of the access and process SQL databases independent of the particular RDBMS that is accessedparticular RDBMS that is accessed

Required parameters:Required parameters: ODBC driver ODBC driver Back-end server nameBack-end server name Database nameDatabase name User id and passwordUser id and password

Additional information:Additional information: Data source name (DSN)Data source name (DSN) Windows client computer nameWindows client computer name Client application program’s executable nameClient application program’s executable name

Java Database Connectivity (JDBC) is similar to ODBC–built specifically for Java applications


ODBC Architecture ODBC Architecture (Figure 9-6)(Figure 9-6)

Each DBMS has its own ODBC-compliant driver

Client does not need to know anything about

the DBMS

Application Program Interface (API)

provides common interface to all DBMSs


362362

Chapter 10:Chapter 10: The Internet Database The Internet Database

EnvironmentEnvironment





ObjectivesObjectives Definition of termsDefinition of terms Explain the importance of attaching a database to Explain the importance of attaching a database to

a Web pagea Web page Describe necessary environment for Internet and Describe necessary environment for Internet and

Intranet database connectivityIntranet database connectivity Use Internet terminology appropriatelyUse Internet terminology appropriately Explain the purpose of WWW ConsortiumExplain the purpose of WWW Consortium Explain the purpose of server-side extensionsExplain the purpose of server-side extensions Describe Web servicesDescribe Web services Compare Web server interfaces (CGI, API, Java Compare Web server interfaces (CGI, API, Java

servlets)servlets) Decribe Web load balancing methodsDecribe Web load balancing methods Explain plug-insExplain plug-ins Explain the purpose of XML as a standardExplain the purpose of XML as a standard


Web Characterstics that Web Characterstics that Support Web-Based Database Support Web-Based Database

ApplicationsApplications Web browsers are simple to useWeb browsers are simple to use Information transfer can take place across Information transfer can take place across

different platformsdifferent platforms Development time and cost have been Development time and cost have been

reducedreduced Sites can be static (no database) or Sites can be static (no database) or

dynamic/interactive (with database)dynamic/interactive (with database) Potential e-business advantages Potential e-business advantages

(improved customer service, faster market (improved customer service, faster market time, better supply chain management)time, better supply chain management)


Figure 10-1 Database-enabled intranet/internet environment


Internet and Intranet Internet and Intranet ServicesServices

Web serverWeb server Database-enabled servicesDatabase-enabled services Directory, security, authenticationDirectory, security, authentication E-mailE-mail File Transfer Protocol (FTP)File Transfer Protocol (FTP) Firewalls and proxy serversFirewalls and proxy servers News or discussion groupsNews or discussion groups Document searchDocument search Load balancing and cachingLoad balancing and caching


World Wide Web Consortium World Wide Web Consortium (W3C)(W3C)

An international consortium of An international consortium of companies working to develop open companies working to develop open standards that foster the standards that foster the development of Web conventions so development of Web conventions so that Web documents can be that Web documents can be consistently displayed on all consistently displayed on all platforms platforms

See See www.w3c.orgwww.w3c.org


Web-Related TermsWeb-Related Terms World Wide Web (WWW)World Wide Web (WWW)

The total set of interlinked hypertext documents residing on The total set of interlinked hypertext documents residing on Web servers worldwideWeb servers worldwide

BrowserBrowser Software that displays HTML documents and allows users to Software that displays HTML documents and allows users to

access files and software related to HTML documentsaccess files and software related to HTML documents Web ServerWeb Server

Software that responds to requests from browsers and Software that responds to requests from browsers and transmits HTML documents to browserstransmits HTML documents to browsers

Web pages–HTML documentsWeb pages–HTML documents Static Web pages–content established at development time Static Web pages–content established at development time Dynamic Web pages–content dynamically generated, usually by Dynamic Web pages–content dynamically generated, usually by

obtaining data from databaseobtaining data from database


Communications Communications TechnologyTechnology

IP AddressIP Address Four numbers that identify a node on the InternetFour numbers that identify a node on the Internet e.g. 131.247.152.18e.g. 131.247.152.18

Hypertext Transfer Protocol (HTTP)Hypertext Transfer Protocol (HTTP) Communication protocol used to transfer pages from Web server to browserCommunication protocol used to transfer pages from Web server to browser HTTPS is a more secure versionHTTPS is a more secure version

Uniform Resource Locator (URL)Uniform Resource Locator (URL) Mnemonic Web address corresponding with IP addressMnemonic Web address corresponding with IP address Also includes folder location and html file nameAlso includes folder location and html file name

Typical URL


Internet-Related Internet-Related LanguagesLanguages Hypertext Markup Language (HTML)Hypertext Markup Language (HTML)

Markup language specifically for Web pagesMarkup language specifically for Web pages Standard Generalized Markup Language (SGML)Standard Generalized Markup Language (SGML)

Markup language standardMarkup language standard Extensible Markup Language (XML)Extensible Markup Language (XML)

Markup language allowing customized tagsMarkup language allowing customized tags XHTMLXHTML

XML-compliant extension of HTMLXML-compliant extension of HTML JavaJava

Object-oriented programming language for appletsObject-oriented programming language for applets JavaScript/VBScriptJavaScript/VBScript

Scripting languages that enable interactivity in HTML documentsScripting languages that enable interactivity in HTML documents Cascading Style Sheets (CSS)Cascading Style Sheets (CSS)

Control appearance of Web elements in an HML documentControl appearance of Web elements in an HML document XSL and XSLTXSL and XSLT

XMS style sheet and transformation to HTMLXMS style sheet and transformation to HTML

Standards and Web conventions established

byWorld Wide Web World Wide Web

Consortium (W3C)Consortium (W3C)


XML OverviewXML Overview Becoming the standard for E-Commerce data Becoming the standard for E-Commerce data

exchangeexchange A markup language (like HTML)A markup language (like HTML)

Uses elements, tags, attributesUses elements, tags, attributes Includes document type declarations (DTDs), XML Includes document type declarations (DTDs), XML

schemas, comments, and entity referencesschemas, comments, and entity references XML Schema (XSD) replacing DTDsXML Schema (XSD) replacing DTDs Relax NG–ISO standard XML database Relax NG–ISO standard XML database

definitiondefinition Document Structure Description (DSD)– Document Structure Description (DSD)–

expressive, easy to use XML database expressive, easy to use XML database definitiondefinition


Sample XML SchemaSample XML Schema

Schema is a record

definition, analogous to

the Create SQL statement, and

therefore provides

metadata


Sample XML Document Sample XML Document DataData

XML data involves elements and attributes defined in the schema, and is analogous to inserting a record into

a database.


Server-Side ExtensionsServer-Side Extensions Programs that interact directly with Web Programs that interact directly with Web

servers to handle requestsservers to handle requests e.g. database-request handling middlewaree.g. database-request handling middleware

Figure 10-2 Web-to-database middleware


Web Server InterfacesWeb Server Interfaces Common Gateway Interface (CGI)Common Gateway Interface (CGI)

Specify transfer of information between Web server and Specify transfer of information between Web server and CGI programCGI program

Performance not very goodPerformance not very good Security risksSecurity risks

Application Program Interface (API)Application Program Interface (API) More efficient than CGIMore efficient than CGI Shared as dynamic link libraries (DLLs)Shared as dynamic link libraries (DLLs)

Java ServletsJava Servlets Like applets, but stored at serverLike applets, but stored at server Cross-platform compatibleCross-platform compatible More efficient than CGIMore efficient than CGI


Web ServersWeb Servers Provide HTTP serviceProvide HTTP service Passing plain text via TCP connectionPassing plain text via TCP connection Serve many clients at onceServe many clients at once

Therefore, multithreaded and multiprocessedTherefore, multithreaded and multiprocessed Load balancing approaches:Load balancing approaches:

Domain Name Server (DNS) balancingDomain Name Server (DNS) balancing One DNS = multiple IP addressesOne DNS = multiple IP addresses

Software/hardware balancingSoftware/hardware balancing Request at one IP address is distributed to multiple Request at one IP address is distributed to multiple

serversservers Reverse proxyReverse proxy

Intercept client request and cache responseIntercept client request and cache response


Client-Side ExtensionsClient-Side Extensions Add functionality to the browserAdd functionality to the browser Plug-insPlug-ins

Hardware/software modules that extend Hardware/software modules that extend browser capabilities by adding features (e.g. browser capabilities by adding features (e.g. encryption, animation, wireless access)encryption, animation, wireless access)

ActiveXActiveX Microsoft COM/OLE components that allow Microsoft COM/OLE components that allow

data manipulation inside the browserdata manipulation inside the browser CookiesCookies

Block of data stored at client by Web server Block of data stored at client by Web server for later usefor later use


Components for Dynamic Web Components for Dynamic Web SitesSites

DBMS–Oracle, Microsoft SQL Server, DBMS–Oracle, Microsoft SQL Server, Informix, Sybase, DB2, Microsoft Access, Informix, Sybase, DB2, Microsoft Access, MySQLMySQL

Web server–Apache, Microsoft IISWeb server–Apache, Microsoft IIS Programming languages/development Programming languages/development

technologies–ASP .NET, PHP, ColdFusion, technologies–ASP .NET, PHP, ColdFusion, Coral Web Builder, Macromedia’s Coral Web Builder, Macromedia’s DreamweaverDreamweaver

Web browser–Microsoft Internet Explorer, Web browser–Microsoft Internet Explorer, Netscape Navigator, Mozilla Firefox, Apple’s Netscape Navigator, Mozilla Firefox, Apple’s Safari, OperaSafari, Opera

Text editor–Notepad, BBEdit, vi, or an IDEText editor–Notepad, BBEdit, vi, or an IDE FTP capabilities–SmartFTP, WS_FTPFTP capabilities–SmartFTP, WS_FTP


Figure 10-3 Dynamic Web development environment


Figure 10-4 Sample PHP script that accepts user registration inputa) PHP script initiation and input validation

(Ullman, PHP and MySql for Dynamic Web Sites, 2003, Script 6.6)


Figure 10-4a (cont.)


Figure 10-4 Sample PHP script that accepts user registration inputb) Adding user information to the database


Figure 10-4 Sample PHP script that accepts user registration inputc) Close PHP script and display HTML form


Web ServicesWeb Services XML-based standards that define protocols for automatic XML-based standards that define protocols for automatic

communication between applications over the Web. communication between applications over the Web. Web Service Components:Web Service Components:

Universal Description, Discovery, and Integration (UDDI)Universal Description, Discovery, and Integration (UDDI) Technical specification for distributed registries of Web services and Technical specification for distributed registries of Web services and

businesses open to communication on these servicesbusinesses open to communication on these services Web Services Description Language (WSDL)Web Services Description Language (WSDL)

XML-based grammar for describing Web services and providing public XML-based grammar for describing Web services and providing public interfaces for these servicesinterfaces for these services

Simple Object Access Protocol (SOAP)Simple Object Access Protocol (SOAP) XML-based communication protocol for sending messages between XML-based communication protocol for sending messages between

applications via the Internetapplications via the Internet Challenges for Web ServicesChallenges for Web Services

Lack of mature standardsLack of mature standards Lack of securityLack of security


Figure 10-5 A typical order entry system that uses Web services

(adapted from Newcomer 2002, Figure 1-3) Figure 10-6 Web services protocol stack


Figure 10-7 Web services deployment (adapted from Newcomer, 2002)


Service Oriented Service Oriented ArchitecturesArchitectures

Collection of services that communicate Collection of services that communicate with each other by passing datawith each other by passing data

Web services, CORBA, Java, XML, SOAP, Web services, CORBA, Java, XML, SOAP, WSDLWSDL

Loosely coupledLoosely coupled InteroperableInteroperable Using SOA results in increased software Using SOA results in increased software

development efficiency (up to 40%)development efficiency (up to 40%)


Semantic WebSemantic Web W3C project using Web metadata to W3C project using Web metadata to

automate collection of knowledge automate collection of knowledge and storing in easily understood and storing in easily understood formatformat

Structuring based on:Structuring based on: XMLXML Resource Description Framewok (RDF)Resource Description Framewok (RDF) Web Ontology Language (OWL)Web Ontology Language (OWL)


Rapidly Accelerating Internet Rapidly Accelerating Internet ChangesChanges

Integrated database environmentsIntegrated database environments Use of cell phones and PDAsUse of cell phones and PDAs Changes in organizational relationshipsChanges in organizational relationships GlobalizationGlobalization Challenges to IT personnel require:Challenges to IT personnel require:

Business and technology infrastructure Business and technology infrastructure understandingunderstanding

Leadership and communication skillsLeadership and communication skills Upward influence techniquesUpward influence techniques Employee management techniquesEmployee management techniques


390390

Chapter 11:Chapter 11: Data Warehousing Data Warehousing





ObjectivesObjectives Definition of termsDefinition of terms Reasons for information gap between Reasons for information gap between

information needs and availabilityinformation needs and availability Reasons for need of data warehousingReasons for need of data warehousing Describe three levels of data warehouse Describe three levels of data warehouse

architecturesarchitectures List four steps of data reconciliationList four steps of data reconciliation Describe two components of star schemaDescribe two components of star schema Estimate fact table sizeEstimate fact table size Design a data martDesign a data mart


DefinitionDefinition Data WarehouseData Warehouse: :

A subject-oriented, integrated, time-variant, non-A subject-oriented, integrated, time-variant, non-updatable collection of data used in support of updatable collection of data used in support of management decision-making processesmanagement decision-making processes

Subject-oriented:Subject-oriented: e.g. customers, patients, e.g. customers, patients, students, productsstudents, products

Integrated: Integrated: Consistent naming conventions, Consistent naming conventions, formats, encoding structures; from multiple data formats, encoding structures; from multiple data sourcessources

Time-variant: Time-variant: Can study trends and changesCan study trends and changes Nonupdatable: Nonupdatable: Read-only, periodically refreshedRead-only, periodically refreshed

Data MartData Mart:: A data warehouse that is limited in scopeA data warehouse that is limited in scope


Need for Data WarehousingNeed for Data Warehousing Integrated, company-wide view of high-quality Integrated, company-wide view of high-quality

information (from disparate databases)information (from disparate databases) Separation of Separation of operationaloperational and and informationalinformational

systems and data (for improved performance)systems and data (for improved performance)


Source: adapted from Strange (1997).


Data Warehouse Data Warehouse ArchitecturesArchitectures

Generic Two-Level ArchitectureGeneric Two-Level Architecture Independent Data MartIndependent Data Mart Dependent Data Mart and Dependent Data Mart and

Operational Data StoreOperational Data Store Logical Data Mart and Real-Time Data Logical Data Mart and Real-Time Data

WarehouseWarehouse Three-Layer architectureThree-Layer architecture

All involve some form of extraction, transformation and loading (ETLETL)


Figure 11-2: Generic two-level data warehousing architecture

E

T

LOne,

company-wide

warehouse

Periodic extraction data is not completely current in warehouse


Figure 11-3 Independent data mart data warehousing architecture

Data marts:Data marts:Mini-warehouses, limited in scope

E

T

L

Separate ETL for each independent data mart

Data access complexity due to multiple data marts


Figure 11-4 Dependent data mart with operational data store: a three-level architecture

ET

L

Single ETL for enterprise data warehouse

(EDW)(EDW)

Simpler data access

ODS ODS provides option for obtaining current data

Dependent data marts loaded from EDW


ET

L

Near real-time ETL for Data WarehouseData Warehouse

ODS ODS and data warehousedata warehouse are one and the same

Data marts are NOT separate databases, but logical views of the data warehouse

Easier to create new data marts

Figure 11-5 Logical data mart and real time warehouse architecture


Figure 11-6 Three-layer data architecture for a data warehouse


Data CharacteristicsData CharacteristicsStatus vs. Event DataStatus vs. Event Data

Status

Status

Event = a database action

(create/update/delete) that results from a

transaction

Figure 11-7 Example of DBMS

log entry


Data CharacteristicsData CharacteristicsTransient vs. Periodic DataTransient vs. Periodic Data

With transient

data, changes

to existing records

are written

over previous records,

thus destroyi

ng the previous

data content

Figure 11-8 Transient

operational data


Periodic data are

never physicall

y altered

or deleted

once they have been

added to the store

Data CharacteristicsData CharacteristicsTransient vs. Periodic DataTransient vs. Periodic Data

Figure 11-9: Periodic

warehouse data


Other Data Warehouse Other Data Warehouse ChangesChanges

New descriptive attributesNew descriptive attributes New business activity attributesNew business activity attributes New classes of descriptive attributesNew classes of descriptive attributes Descriptive attributes become more Descriptive attributes become more

refinedrefined Descriptive data are related to one Descriptive data are related to one

anotheranother New source of dataNew source of data


The Reconciled Data LayerThe Reconciled Data Layer Typical operational data is:Typical operational data is:

Transient–not historicalTransient–not historical Not normalized (perhaps due to denormalization for Not normalized (perhaps due to denormalization for

performance)performance) Restricted in scope–not comprehensiveRestricted in scope–not comprehensive Sometimes poor quality–inconsistencies and errorsSometimes poor quality–inconsistencies and errors

After ETL, data should be:After ETL, data should be: Detailed–not summarized yetDetailed–not summarized yet Historical–periodicHistorical–periodic Normalized–3Normalized–3rdrd normal form or higher normal form or higher Comprehensive–enterprise-wide perspectiveComprehensive–enterprise-wide perspective Timely–data should be current enough to assist decision-Timely–data should be current enough to assist decision-

makingmaking Quality controlled–accurate with full integrityQuality controlled–accurate with full integrity


The ETL ProcessThe ETL Process

Capture/ExtractCapture/Extract Scrub or data cleansingScrub or data cleansing TransformTransform Load and IndexLoad and Index

ETL = Extract, transform, and load


Static extractStatic extract = capturing a snapshot of

the source data at a point in time

Incremental extractIncremental extract = capturing changes that

have occurred since the last static extract

Capture/Extract…obtaining a snapshot of a chosen subset of the source data for loading into the data

warehouseFigure 11-10: Steps in data reconciliation


Scrub/Cleanse…uses pattern recognition and AI techniques to upgrade data quality

Fixing errors:Fixing errors: misspellings, erroneous

dates, incorrect field usage, mismatched addresses,

missing data, duplicate data, inconsistencies

Also:Also: decoding, reformatting, time stamping, conversion, key generation,

merging, error detection/logging, locating

missing data

Figure 11-10: Steps in data reconciliation

(cont.)


Transform = convert data from format of operational system to format of data warehouse

Record-level:Record-level:Selection–data partitioning

Joining–data combiningAggregation–data

summarization

Field-level:Field-level: single-field–from one field to one

fieldmulti-field–from many fields to

one, or one field to many


(cont.)


Load/Index= place transformed data into the warehouse and

create indexes

Refresh mode:Refresh mode: bulk rewriting of target data at

periodic intervals

Update mode:Update mode: only changes in source data are written to data warehouse


(cont.)


Figure 11-11: Single-field transformation

In general–some transformation function

translates data from old form to new form

Algorithmic transformation uses a formula or logical

expression

Table lookup–another approach, uses a

separate table keyed by source record code


Figure 11-12: Multifield transformation

M:1–from many source fields to one target

field

1:M–from one source field to

many target fields


Derived DataDerived Data ObjectivesObjectives

Ease of use for decision support applicationsEase of use for decision support applications Fast response to predefined user queriesFast response to predefined user queries Customized data for particular target audiencesCustomized data for particular target audiences Ad-hoc query supportAd-hoc query support Data mining capabilitiesData mining capabilities

CharacteristicsCharacteristics Detailed (mostly periodic) dataDetailed (mostly periodic) data Aggregate (for summary)Aggregate (for summary) Distributed (to departmental servers)Distributed (to departmental servers)

Most common data model = star schemastar schema(also called “dimensional model”)


Figure 11-13 Components of a star schemastar schemaFact tables contain

factual or quantitative data

Dimension tables contain descriptions about the subjects of

the business

1:N relationship between dimension tables and fact

tables

Excellent for ad-hoc queries, but bad for online transaction processing

Dimension tables are denormalized to maximize performance


Figure 11-14 Star schema example

Fact table provides statistics for sales broken down by product,

period and store dimensions


Figure 11-15 Star schema with sample data


Issues Regarding Star Issues Regarding Star SchemaSchema

Dimension table keys must be Dimension table keys must be surrogatesurrogate (non- (non-intelligent and non-business related), because:intelligent and non-business related), because: Keys may change over timeKeys may change over time Length/format consistencyLength/format consistency

Granularity of Fact Table–what level of detail do Granularity of Fact Table–what level of detail do you want? you want? Transactional grain–finest levelTransactional grain–finest level Aggregated grain–more summarizedAggregated grain–more summarized Finer grains Finer grains better better market basket analysismarket basket analysis capability capability Finer grain Finer grain more dimension tables, more rows in fact table more dimension tables, more rows in fact table

Duration of the database–how much history Duration of the database–how much history should be kept?should be kept? Natural duration–13 months or 5 quartersNatural duration–13 months or 5 quarters Financial institutions may need longer durationFinancial institutions may need longer duration Older data is more difficult to source and cleanseOlder data is more difficult to source and cleanse


Figure 11-16: Modeling dates

Fact tables contain time-period data Date dimensions are important


The User InterfaceThe User InterfaceMetadata (data catalog)Metadata (data catalog)

Identify subjects of the data martIdentify subjects of the data mart Identify dimensions and factsIdentify dimensions and facts Indicate how data is derived from enterprise Indicate how data is derived from enterprise

data warehouses, including derivation rulesdata warehouses, including derivation rules Indicate how data is derived from operational Indicate how data is derived from operational

data store, including derivation rulesdata store, including derivation rules Identify available reports and predefined queriesIdentify available reports and predefined queries Identify data analysis techniques (e.g. drill-down)Identify data analysis techniques (e.g. drill-down) Identify responsible peopleIdentify responsible people


On-Line Analytical Processing (OLAP) On-Line Analytical Processing (OLAP) ToolsTools

The use of a set of graphical tools that provides The use of a set of graphical tools that provides users with multidimensional views of their data users with multidimensional views of their data and allows them to analyze the data using and allows them to analyze the data using simple windowing techniquessimple windowing techniques

Relational OLAP (ROLAP)Relational OLAP (ROLAP) Traditional relational representationTraditional relational representation

Multidimensional OLAP (MOLAP)Multidimensional OLAP (MOLAP) CubeCube structure structure

OLAP OperationsOLAP Operations Cube slicingCube slicing–come up with 2-D view of data–come up with 2-D view of data Drill-downDrill-down–going from summary to more detailed –going from summary to more detailed

viewsviews


Figure 11-23 Slicing a data cube


Figure 11-24 Example of drill-down

Summary report

Drill-down with color added

Starting with summary data,

users can obtain details for particular

cells


Data Mining and Data Mining and VisualizationVisualization

Knowledge discovery using a blend of statistical, AI, and computer Knowledge discovery using a blend of statistical, AI, and computer graphics techniquesgraphics techniques

Goals:Goals: Explain observed events or conditionsExplain observed events or conditions Confirm hypothesesConfirm hypotheses Explore data for new or unexpected relationshipsExplore data for new or unexpected relationships

TechniquesTechniques Statistical regressionStatistical regression Decision tree inductionDecision tree induction Clustering and signal processingClustering and signal processing AffinityAffinity Sequence associationSequence association Case-based reasoningCase-based reasoning Rule discoveryRule discovery Neural netsNeural nets FractalsFractals

Data visualization–representing data in graphical/multimedia formats Data visualization–representing data in graphical/multimedia formats for analysisfor analysis


424424

Chapter 12:Chapter 12: Data and Database Data and Database

AdministrationAdministration





ObjectivesObjectives Definition of termsDefinition of terms List functions and roles of data/database administrationList functions and roles of data/database administration Describe role of data dictionaries and information Describe role of data dictionaries and information

repositoriesrepositories Compare optimistic and pessimistic concurrency controlCompare optimistic and pessimistic concurrency control Describe problems and techniques for data securityDescribe problems and techniques for data security Describe problems and techniques for data recoveryDescribe problems and techniques for data recovery Describe database tuning issues and list areas where Describe database tuning issues and list areas where

changes can be done to tune the databasechanges can be done to tune the database Describe importance and measures of data qualityDescribe importance and measures of data quality Describe importance and measures of data availabilityDescribe importance and measures of data availability


Traditional Administration Traditional Administration DefinitionsDefinitions

Data AdministrationData Administration:: A high-level function A high-level function that is responsible for the overall that is responsible for the overall management of data resources in an management of data resources in an organization, including maintaining organization, including maintaining corporate-wide definitions and standardscorporate-wide definitions and standards

Database AdministrationDatabase Administration:: A technical A technical function that is responsible for physical function that is responsible for physical database design and for dealing with database design and for dealing with technical issues such as security technical issues such as security enforcement, database performance, and enforcement, database performance, and backup and recoverybackup and recovery


Traditional Data Administration Traditional Data Administration FunctionsFunctions

Data policies, procedures, standardsData policies, procedures, standards PlanningPlanning Data conflict (ownership) resolutionData conflict (ownership) resolution Managing the information repositoryManaging the information repository Internal marketing of DA conceptsInternal marketing of DA concepts


Traditional Database Traditional Database Administration FunctionsAdministration Functions

Selection of DBMS and software toolsSelection of DBMS and software tools Installing/upgrading DBMSInstalling/upgrading DBMS Tuning database performanceTuning database performance Improving query processing Improving query processing

performanceperformance Managing data security, privacy, and Managing data security, privacy, and

integrityintegrity Data backup and recoveryData backup and recovery


Evolving Approaches to Evolving Approaches to Data AdministrationData Administration

Blend data and database administration into one Blend data and database administration into one rolerole

Fast-track development – monitoring development Fast-track development – monitoring development process (analysis, design, implementation, process (analysis, design, implementation, maintenance)maintenance)

Procedural DBAs–managing quality of triggers and Procedural DBAs–managing quality of triggers and stored proceduresstored procedures

eDBA–managing Internet-enabled database eDBA–managing Internet-enabled database applicationsapplications

PDA DBA–data synchronization and personal PDA DBA–data synchronization and personal database managementdatabase management

Data warehouse administrationData warehouse administration


Data Warehouse Data Warehouse AdministrationAdministration

New role, coming with the growth in data New role, coming with the growth in data warehouseswarehouses

Similar to DA/DBA rolesSimilar to DA/DBA roles Emphasis on integration and coordination of Emphasis on integration and coordination of

metadata/data across many data sourcesmetadata/data across many data sources Specific roles:Specific roles:

Support DSS applicationsSupport DSS applications Manage data warehouse growthManage data warehouse growth Establish service level agreements regarding Establish service level agreements regarding

data warehouses and data martsdata warehouses and data marts


Open Source DBMSsOpen Source DBMSs

An alternative to proprietary packages An alternative to proprietary packages such as Oracle, Microsoft SQL Server, such as Oracle, Microsoft SQL Server, or Microsoft Accessor Microsoft Access

mySQL is an example of open-source mySQL is an example of open-source DBMSDBMS

Less expensive than proprietary Less expensive than proprietary packagespackages

Source code available, for modificationSource code available, for modification


Figure 12-2 Data modeling responsibilities


Database SecurityDatabase Security

Database Security:Database Security: Protection Protection of the data against accidental or of the data against accidental or intentional loss, destruction, or intentional loss, destruction, or misusemisuse

Increased difficulty due to Increased difficulty due to Internet access and client/server Internet access and client/server technologiestechnologies


Figure 12-3 Possible locations of data security threats


Threats to Data SecurityThreats to Data Security Accidental losses attributable to:Accidental losses attributable to:

Human errorHuman error Software failureSoftware failure Hardware failureHardware failure

Theft and fraudTheft and fraud Improper data access:Improper data access:

Loss of privacy (personal data)Loss of privacy (personal data) Loss of confidentiality (corporate data)Loss of confidentiality (corporate data)

Loss of data integrityLoss of data integrity Loss of availability (through, e.g. sabotage)Loss of availability (through, e.g. sabotage)


Figure 12-4 Establishing Internet Security


Web SecurityWeb Security

Static HTML files are easy to secureStatic HTML files are easy to secure Standard database access controlsStandard database access controls Place Web files in protected directories on Place Web files in protected directories on

serverserver Dynamic pages are harderDynamic pages are harder

Control of CGI scriptsControl of CGI scripts User authenticationUser authentication Session securitySession security SSL for encryptionSSL for encryption Restrict number of users and open portsRestrict number of users and open ports Remove unnecessary programs Remove unnecessary programs


W3C Web Privacy StandardW3C Web Privacy Standard Platform for Privacy Protection (P3P) Platform for Privacy Protection (P3P) Addresses the following:Addresses the following:

Who collects dataWho collects data What data is collected and for what purposeWhat data is collected and for what purpose Who is data shared withWho is data shared with Can users control access to their dataCan users control access to their data How are disputes resolvedHow are disputes resolved Policies for retaining dataPolicies for retaining data Where are policies kept and how can they be Where are policies kept and how can they be

accessedaccessed


Database Software Database Software Security FeaturesSecurity Features

• Views or subschemasViews or subschemas• Integrity controlsIntegrity controls• Authorization rulesAuthorization rules• User-defined proceduresUser-defined procedures• EncryptionEncryption• Authentication schemesAuthentication schemes• Backup, journalizing, and Backup, journalizing, and

checkpointingcheckpointing


Views and Integrity ControlsViews and Integrity Controls

ViewsViews Subset of the database that is presented to one Subset of the database that is presented to one

or more usersor more users User can be given access privilege to view User can be given access privilege to view

without allowing access privilege to underlying without allowing access privilege to underlying tablestables

Integrity ControlsIntegrity Controls Protect data from unauthorized useProtect data from unauthorized use Domains–set allowable valuesDomains–set allowable values Assertions–enforce database conditionsAssertions–enforce database conditions


Authorization RulesAuthorization Rules Controls incorporated in the data Controls incorporated in the data

management systemmanagement system Restrict: Restrict:

access to dataaccess to data actions that people can take on dataactions that people can take on data

Authorization matrix for:Authorization matrix for: SubjectsSubjects ObjectsObjects ActionsActions ConstraintsConstraints


Figure 12-5 Authorization matrix


Some DBMSs also provide capabilities for

user-defined procedures to

customize the authorization process

Figure 12-6a Authorization table for subjects (salespeople)

Figure 12-6b Authorization table for objects (orders)

Figure 12-7 Oracle privileges

Implementing authorization

rules


Encryption – the coding or scrambling of

data so that humans cannot read them

Secure Sockets Layer (SSL) is a popular

encryption scheme for TCP/IP connections

Figure 12-8 Basic two-key encryption


Authentication SchemesAuthentication Schemes Goal – obtain a Goal – obtain a positivepositive

identification of the useridentification of the user Passwords: First line of defensePasswords: First line of defense

Should be at least 8 characters longShould be at least 8 characters long Should combine alphabetic and Should combine alphabetic and

numeric datanumeric data Should not be complete words or Should not be complete words or

personal informationpersonal information Should be changed frequentlyShould be changed frequently


Authentication Schemes Authentication Schemes (cont.)(cont.) Strong AuthenticationStrong Authentication

Passwords are flawed:Passwords are flawed: Users share them with each otherUsers share them with each other They get written down, could be copiedThey get written down, could be copied Automatic logon scripts remove need to explicitly type them inAutomatic logon scripts remove need to explicitly type them in Unencrypted passwords travel the InternetUnencrypted passwords travel the Internet

Possible solutions:Possible solutions: Two factor–e.g. smart card plus PINTwo factor–e.g. smart card plus PIN Three factor–e.g. smart card, biometric, PINThree factor–e.g. smart card, biometric, PIN Biometric devices–use of fingerprints, retinal scans, Biometric devices–use of fingerprints, retinal scans,

etc. for positive IDetc. for positive ID Third-party mediated authentication–using secret keys, Third-party mediated authentication–using secret keys,

digital certificatesdigital certificates


Security Policies and Security Policies and ProceduresProcedures

Personnel controlsPersonnel controls Hiring practices, employee monitoring, security Hiring practices, employee monitoring, security

trainingtraining Physical access controlsPhysical access controls

Equipment locking, check-out procedures, screen Equipment locking, check-out procedures, screen placementplacement

Maintenance controlsMaintenance controls Maintenance agreements, access to source code, Maintenance agreements, access to source code,

quality and availability standardsquality and availability standards Data privacy controlsData privacy controls

Adherence to privacy legislation, access rulesAdherence to privacy legislation, access rules


Database RecoveryDatabase Recovery

Mechanism for restoring a Mechanism for restoring a database quickly and accurately database quickly and accurately after loss or damageafter loss or damage

Recovery facilities:Recovery facilities:• Backup FacilitiesBackup Facilities• Journalizing FacilitiesJournalizing Facilities• Checkpoint FacilityCheckpoint Facility• Recovery ManagerRecovery Manager


Back-up FacilitiesBack-up Facilities Automatic dump facility that produces Automatic dump facility that produces

backup copy of the entire databasebackup copy of the entire database Periodic backup (e.g. nightly, weekly)Periodic backup (e.g. nightly, weekly) Cold backup–database is shut down Cold backup–database is shut down

during backupduring backup Hot backup–selected portion is shut Hot backup–selected portion is shut

down and backed up at a given timedown and backed up at a given time Backups stored in secure, off-site Backups stored in secure, off-site

locationlocation


Journalizing FacilitiesJournalizing Facilities Audit trail of transactions and Audit trail of transactions and

database updatesdatabase updates Transaction log–record of essential Transaction log–record of essential

data for each transaction processed data for each transaction processed against the databaseagainst the database

Database change log–images of Database change log–images of updated dataupdated data Before-image–copy before modificationBefore-image–copy before modification After-image–copy after modificationAfter-image–copy after modification

Produces an audit trailaudit trail


Figure 12-9 Database audit trail

From the backup and logs, databases

can be restored in case of damage or

loss


Checkpoint FacilitiesCheckpoint Facilities

DBMS periodically refuses to accept DBMS periodically refuses to accept new transactionsnew transactions

system is in a system is in a quietquiet state state Database and transaction logs are Database and transaction logs are

synchronizedsynchronized

This allows recovery manager to resume processing from short period, instead of

repeating entire day


Recovery and Restart Recovery and Restart ProceduresProcedures

Disk Mirroring–switch between identical Disk Mirroring–switch between identical copies of databasescopies of databases

Restore/Rerun–reprocess transactions Restore/Rerun–reprocess transactions against the backupagainst the backup

Transaction Integrity–commit or abort Transaction Integrity–commit or abort all transaction changesall transaction changes

Backward Recovery (Rollback)–apply Backward Recovery (Rollback)–apply before imagesbefore images

Forward Recovery (Roll Forward)–apply Forward Recovery (Roll Forward)–apply after images (preferable to after images (preferable to restore/rerun)restore/rerun)


Transaction ACID PropertiesTransaction ACID Properties

AtomicAtomic Transaction cannot be subdividedTransaction cannot be subdivided

ConsistentConsistent Constraints don’t change from before Constraints don’t change from before

transaction to after transactiontransaction to after transaction IsolatedIsolated

Database changes not revealed to users until Database changes not revealed to users until after transaction has completedafter transaction has completed

DurableDurable Database changes are permanentDatabase changes are permanent


Figure 12-10 Basic recovery techniques a) Rollback


Figure 12-10 Basic recovery techniques (cont.)b) Rollforward


Database Failure Database Failure ResponsesResponses

Aborted transactionsAborted transactions Preferred recovery: rollbackPreferred recovery: rollback Alternative: Rollforward to state just prior to abortAlternative: Rollforward to state just prior to abort

Incorrect dataIncorrect data Preferred recovery: rollbackPreferred recovery: rollback Alternative 1: rerun transactions not including inaccurate data updatesAlternative 1: rerun transactions not including inaccurate data updates Alternative 2: compensating transactionsAlternative 2: compensating transactions

System failure (database intact)System failure (database intact) Preferred recovery: switch to duplicate databasePreferred recovery: switch to duplicate database Alternative 1: rollbackAlternative 1: rollback Alternative 2: restart from checkpointAlternative 2: restart from checkpoint

Database destructionDatabase destruction Preferred recovery: switch to duplicate databasePreferred recovery: switch to duplicate database Alternative 1: rollforwardAlternative 1: rollforward Alternative 2: reprocess transactionsAlternative 2: reprocess transactions


Concurrency ControlConcurrency Control ProblemProblem–in a multiuser environment, –in a multiuser environment,

simultaneous access to data can simultaneous access to data can result in interference and data lossresult in interference and data loss

SolutionSolution––Concurrency ControlConcurrency Control The process of managing simultaneous The process of managing simultaneous

operations against a database so that operations against a database so that data integrity is maintained and the data integrity is maintained and the operations do not interfere with each operations do not interfere with each other in a multi-user environmentother in a multi-user environment


Figure 12-11 Lost update (no concurrency control in effect)

Simultaneous access causes updates to cancel each otherA similar problem is the inconsistent readinconsistent read problem


Concurrency Control Concurrency Control TechniquesTechniques

SerializabilitySerializability Finish one transaction before starting Finish one transaction before starting

anotheranother Locking Mechanisms Locking Mechanisms

The most common way of achieving The most common way of achieving serializationserialization

Data that is retrieved for the purpose of Data that is retrieved for the purpose of updating is locked for the updaterupdating is locked for the updater

No other user can perform update until No other user can perform update until unlockedunlocked


Figure 12-12: Updates with locking (concurrency control)

This prevents the lost update problem


Locking MechanismsLocking Mechanisms Locking level:Locking level:

Database–used during database updatesDatabase–used during database updates Table–used for bulk updatesTable–used for bulk updates Block or page–very commonly usedBlock or page–very commonly used Record–only requested row; fairly commonly Record–only requested row; fairly commonly

usedused Field–requires significant overhead; impracticalField–requires significant overhead; impractical

Types of locks:Types of locks: Shared lock–Read but no update permitted. Shared lock–Read but no update permitted.

Used when just reading to prevent another user Used when just reading to prevent another user from placing an exclusive lock on the recordfrom placing an exclusive lock on the record

Exclusive lock–No access permitted. Used Exclusive lock–No access permitted. Used when preparing to updatewhen preparing to update


DeadlockDeadlock An impasse that results when two or more An impasse that results when two or more

transactions have locked common resources, and transactions have locked common resources, and each waits for the other to unlock their resourceseach waits for the other to unlock their resources

Figure 12-13The problem of deadlock

John and Marsha will wait John and Marsha will wait forever for each other to forever for each other to

release their locked release their locked resources!resources!


Managing DeadlockManaging Deadlock Deadlock prevention:Deadlock prevention:

Lock all records required at the beginning of a Lock all records required at the beginning of a transactiontransaction

Two-phase locking protocolTwo-phase locking protocol Growing phaseGrowing phase Shrinking phaseShrinking phase

May be difficult to determine all needed May be difficult to determine all needed resources in advanceresources in advance

Deadlock Resolution:Deadlock Resolution: Allow deadlocks to occurAllow deadlocks to occur Mechanisms for detecting and breaking themMechanisms for detecting and breaking them

Resource usage matrixResource usage matrix


VersioningVersioning

Optimistic approach to concurrency Optimistic approach to concurrency controlcontrol

Instead of lockingInstead of locking Assumption is that simultaneous updates Assumption is that simultaneous updates

will be infrequentwill be infrequent Each transaction can attempt an update Each transaction can attempt an update

as it wishesas it wishes The system will reject an update when it The system will reject an update when it

senses a conflictsenses a conflict Use of rollback and commit for thisUse of rollback and commit for this


Figure 12-15 The use of versioning

Better performance than locking


Managing Data QualityManaging Data Quality Causes of poor data qualityCauses of poor data quality

External data sourcesExternal data sources Redundant data storageRedundant data storage Lack of organizational commitmentLack of organizational commitment

Data quality improvementData quality improvement Perform data quality auditPerform data quality audit Establish data stewardship program (data Establish data stewardship program (data

stewardsteward is a liaison between IT and business is a liaison between IT and business units)units)

Apply total quality management (TQM) practicesApply total quality management (TQM) practices Overcome organizational barriersOvercome organizational barriers Apply modern DBMS technologyApply modern DBMS technology Estimate return on investmentEstimate return on investment


Data Dictionaries and Data Dictionaries and RepositoriesRepositories

Data dictionaryData dictionary Documents data elements of a databaseDocuments data elements of a database

System catalogSystem catalog System-created database that describes all database System-created database that describes all database

objectsobjects Information RepositoryInformation Repository

Stores metadata describing data and data processing Stores metadata describing data and data processing resourcesresources

Information Repository Dictionary System Information Repository Dictionary System (IRDS)(IRDS) Software tool managing/controlling access to Software tool managing/controlling access to

information repositoryinformation repository


Figure 12-16 Three components of the repository system architecture

A schema of the repository

information

Software that

manages the

repository objects

Where repository objects are stored

Source: adapted from Bernstein, 1996.


Database Performance Database Performance TuningTuning DBMS InstallationDBMS Installation

Setting installation parametersSetting installation parameters Memory Usage Memory Usage

Set cache levelsSet cache levels Choose background processesChoose background processes

Input/Output (I/O) ContentionInput/Output (I/O) Contention Use stripingUse striping Distribution of heavily accessed filesDistribution of heavily accessed files

CPU UsageCPU Usage Monitor CPU loadMonitor CPU load

Application tuningApplication tuning Modification of SQL code in applicationsModification of SQL code in applications


Data AvailabilityData Availability Downtime is expensiveDowntime is expensive How to ensure availabilityHow to ensure availability

Hardware failures–provide redundancy Hardware failures–provide redundancy for fault tolerancefor fault tolerance

Loss of data–database mirroringLoss of data–database mirroring Maintenance downtime–automated and Maintenance downtime–automated and

nondisruptive maintenance utilitiesnondisruptive maintenance utilities Network problems–careful traffic Network problems–careful traffic

monitoring, firewalls, and routersmonitoring, firewalls, and routers

Date post:	27-Jan-2015
Category:	Education
Upload:	blackice86
View:	143 times
Download:	8 times

Modern database management jeffrey a. hoffer, mary b. prescott,

Education