Lecture 5 SQL - GitHub PagesSQL Introduction •SQL is a standard language for querying and...

CS639:DataManagementfor

DataScienceLecture4:SQLforDataScience

TheodorosRekatsinas1

2

Announcements• Assignment1isduetomorrow(endofday)• Anyquestions?

• PA2isout.Itisdueonthe19th• StartearlyJ• AskquestionsonPiazza• Gooveractivitiesandreadingbeforeattempting

• Outoftownforthenexttwolectures.• WewillresumeonFeb13th.

Today’sLecture

1. FinishRelationalAlgebra(slidesinpreviouslecture)

2. IntroductiontoSQL

3. Single-tablequeries

4. Multi-tablequeries

5. AdvancedSQL

3

1.IntroductiontoSQL

4

SQLMotivation

• ButwhyuseSQL?• Therelationalmodelofdata isthemostwidelyusedmodeltoday

• MainConcept:therelation- essentially,atable

Logicaldataindependence:protectionfromchangesinthelogicalstructureofthedata

SQLisalogical,declarativequerylanguage.WeuseSQLbecausewehappentousetherelationalmodel.

Remember: Thereasonforusingtherelationalmodelisdataindependence!

6

BasicSQL

SQLIntroduction

• SQLisastandardlanguageforqueryingandmanipulatingdata

• SQLisaveryhigh-levelprogramminglanguage• Thisworksbecauseitisoptimizedwell!

• Manystandardsoutthere:• ANSISQL,SQL92(a.k.a.SQL2),SQL99(a.k.a.SQL3),….• Vendorssupportvarioussubsets

Probablytheworld’smostsuccessfulparallelprogramminglanguage(multicore?)

SQL standsforStructuredQueryLanguage

8

SQLisa…

• DataDefinitionLanguage(DDL)• Definerelationalschemata• Create/alter/deletetablesandtheirattributes

• DataManipulationLanguage(DML)• Insert/delete/modifytuplesintables• Queryoneormoretables– discussednext!

9

TablesinSQL

PName Price Manufacturer

Gizmo $19.99 GizmoWorks

Powergizmo $29.99 GizmoWorks

SingleTouch $149.99 Canon

MultiTouch $203.99 Hitachi

ProductArelation ortable isamultiset oftupleshavingtheattributesspecifiedbytheschema

Let’sbreakthisdefinitiondown

10

TablesinSQL






Product

Amultiset isanunorderedlist(or:asetwithmultipleduplicateinstancesallowed)

List:[1,1,2,3]Set:{1,2,3}Multiset:{1,1,2,3}

i.e.nonext(),etc.methods!

11

TablesinSQL






Product Anattribute (orcolumn)isatypeddataentrypresentineachtupleintherelation

AttributesmusthaveanatomictypeinstandardSQL,i.e.notalist,set,etc.

12

TablesinSQL






Product

Atuple orrow isasingleentryinthetablehavingtheattributesspecifiedbytheschemaAlsoreferredtosometimesasarecord

13

TablesinSQL






Product

Thenumberoftuplesisthecardinality oftherelation

Thenumberofattributesisthearity oftherelation

14

DataTypesinSQL

• Atomictypes:• Characters:CHAR(20),VARCHAR(50)• Numbers:INT,BIGINT,SMALLINT,FLOAT• Others:MONEY,DATETIME,…

• Everyattributemusthaveanatomictype• Hencetablesareflat

15

TableSchemas

• Theschema ofatableisthetablename,itsattributes,andtheirtypes:

• Akey isanattributewhosevaluesareunique;weunderlineakey

Product(Pname: string, Price: float, Category: string, Manufacturer: string)

Product(Pname: string, Price: float, Category: string, Manufacturer: string)

Keyconstraints

• Akeyisanimplicitconstraintonwhichtuplescanbeintherelation

• i.e.iftwotuplesagreeonthevaluesofthekey,thentheymustbethesametuple!

1.Whichwouldyouselectasakey?2.Isakeyalwaysguaranteedtoexist?3.Canwehavemorethanonekey?

Akey isaminimalsubsetofattributes thatactsasauniqueidentifierfortuplesinarelation

Students(sid:string, name:string, gpa: float)

NULLandNOTNULL

• Tosay“don’tknowthevalue”weuseNULL• NULLhas(sometimespainful)semantics,moredetailslater

sid name gpa123 Bob 3.9143 Jim NULL Say,Jimjustenrolledinhisfirstclass.

InSQL,wemayconstrainacolumntobeNOTNULL,e.g.,“name”inthistable

Students(sid:string, name:string, gpa: float)

GeneralConstraints

• Wecanactuallyspecifyarbitraryassertions• E.g.“Therecannotbe25peopleintheDBclass”

• Inpractice,wedon’tspecifymanysuchconstraints.Why?• Performance!

Wheneverwedosomethingugly(oravoiddoingsomethingconvenient)it’sforthesakeofperformance

GooverActivity2-1

19

2.Single-tablequeries

20

21

SQLQuery

• Basicform(therearemanymanymorebellsandwhistles)

CallthisaSFW query.

SELECT <attributes>FROM <one or more relations>WHERE <conditions>

22

SimpleSQLQuery:SelectionPName Price Category ManufacturerGizmo $19.99 Gadgets GizmoWorks

Powergizmo $29.99 Gadgets GizmoWorksSingleTouch $149.99 Photography CanonMultiTouch $203.99 Household Hitachi

PName Price Category ManufacturerGizmo $19.99 Gadgets GizmoWorks

Powergizmo $29.99 Gadgets GizmoWorks

SELECT *FROM ProductWHERE Category = ‘Gadgets’

Selection istheoperationoffilteringarelation’stuplesonsomecondition

23

SimpleSQLQuery:ProjectionPName Price Category ManufacturerGizmo $19.99 Gadgets GizmoWorks


PName Price ManufacturerGizmo $19.99 GizmoWorks


SELECT Pname, Price, ManufacturerFROM ProductWHERE Category = ‘Gadgets’

Projection istheoperationofproducinganoutputtablewithtuplesthathaveasubsetoftheirpriorattributes

24

Notation

SELECT Pname, Price, ManufacturerFROM ProductWHERE Category = ‘Gadgets’

Product(PName, Price, Category, Manfacturer)

Answer(PName, Price, Manfacturer)

Inputschema

Outputschema

25

AFewDetails

• SQLcommands arecaseinsensitive:• Same:SELECT,Select,select• Same:Product,product

• Values arenot:• Different: ‘Seattle’,‘seattle’

• Usesinglequotesforconstants:• ‘abc’- yes• “abc”- no

26

LIKE:SimpleStringPatternMatching

• sLIKE p:patternmatchingonstrings• pmaycontaintwospecialsymbols:• %=anysequenceofcharacters• _=anysinglecharacter

SELECT *FROM ProductsWHERE PName LIKE ‘%gizmo%’

27

DISTINCT:EliminatingDuplicates

SELECT DISTINCT CategoryFROM Product

Versus

SELECT CategoryFROM Product

CategoryGadgetsGadgets

PhotographyHousehold

CategoryGadgets

PhotographyHousehold

28

ORDERBY:SortingtheResults

SELECT PName, Price, ManufacturerFROM ProductWHERE Category=‘gizmo’ AND Price > 50ORDER BY Price, PName

TiesarebrokenbythesecondattributeontheORDERBYlist,etc.

Orderingisascending,unlessyouspecifytheDESCkeyword.

GooverActivity2-2

29

3.Multi-tablequeries

30

ForeignKeyconstraints

student_id aloneisnotakey- whatis?

sid name gpa101 Bob 3.2123 Mary 3.8

student_id cid grade

123 564 A123 537 A+

Students Enrolled

Wesaythatstudent_id isaforeignkey thatreferstoStudents

Students(sid: string, name: string, gpa: float)

Enrolled(student_id: string, cid: string, grade: string)

• Supposewehavethefollowingschema:

• Andwewanttoimposethefollowingconstraint:• ‘Onlybonafidestudentsmayenrollincourses’ i.e.astudentmustappearintheStudentstabletoenrollinaclass

DeclaringForeignKeys

Students(sid: string, name: string, gpa: float)Enrolled(student_id: string, cid: string, grade: string)

CREATE TABLE Enrolled(student_id CHAR(20),cid CHAR(20),grade CHAR(10),PRIMARY KEY (student_id, cid),FOREIGN KEY (student_id) REFERENCES Students(sid)

)

ForeignKeysandupdateoperations

DBAchooses(syntaxinthebook)

Students(sid: string, name: string, gpa: float)

Enrolled(student_id: string, cid: string, grade: string)

• WhatifweinsertatupleintoEnrolled,butnocorrespondingstudent?• INSERTisrejected(foreignkeysareconstraints)!

• Whatifwedeleteastudent?1. Disallowthedelete2. Removeallofthecoursesforthatstudent3. SQLallowsathirdviaNULL(notyetcovered)

34

KeysandForeignKeys

PName Price Category ManufacturerGizmo $19.99 Gadgets GizmoWorks


Product

CompanyCName StockPrice Country

GizmoWorks 25 USACanon 65 JapanHitachi 15 Japan

Whatisaforeignkeyvs.akeyhere?

35

Joins

Ex: Findallproductsunder$200manufacturedinJapan;returntheirnamesandprices.

SELECT PName, PriceFROM Product, CompanyWHERE Manufacturer = CName

AND Country=‘Japan’AND Price <= 200

Product(PName, Price, Category, Manufacturer)

Company(CName, StockPrice, Country) Note:wewilloftenomitattributetypesinschemadefinitionsforbrevity,butassumeattributesarealwaysatomictypes

36

Joins

Ex: Findallproductsunder$200manufacturedinJapan;returntheirnamesandprices.



Ajoin betweentablesreturnsalluniquecombinationsoftheirtupleswhichmeetsomespecifiedjoincondition


Company(CName, StockPrice, Country)

37

Joins

SeveralequivalentwaystowriteabasicjoininSQL:



SELECT PName, PriceFROM ProductJOIN Company ON Manufacturer = Cname

AND Country=‘Japan’WHERE Price <= 200


Company(CName, StockPrice, Country)

38

Joins

PName Price Category ManufGizmo $19 Gadgets GWorks

Powergizmo $29 Gadgets GWorks

SingleTouch $149 Photography Canon

MultiTouch $203 Household Hitachi

ProductCompany

Cname Stock CountryGWorks 25 USACanon 65 JapanHitachi 15 Japan

PName PriceSingleTouch $149.99



39

TupleVariableAmbiguityinMulti-Table

SELECT DISTINCT name, addressFROM Person, CompanyWHERE worksfor = name

Person(name, address, worksfor)

Company(name, address)

Which“address”doesthisreferto?

Which“name”s??

40

Person(name, address, worksfor)

Company(name, address)

SELECT DISTINCT Person.name, Person.addressFROM Person, CompanyWHERE Person.worksfor = Company.name

SELECT DISTINCT p.name, p.addressFROM Person p, Company cWHERE p.worksfor = c.name

Bothequivalentwaystoresolvevariableambiguity

TupleVariableAmbiguityinMulti-Table

41

Meaning(Semantics)ofSQLQueries

SELECT x1.a1, x1.a2, …, xn.akFROM R1 AS x1, R2 AS x2, …, Rn AS xnWHERE Conditions(x1,…, xn)

Answer={}for x1 in R1 do

for x2 in R2 do…..

for xn in Rn doif Conditions(x1,…,xn)

then Answer=AnswerÈ {(x1.a1,x1.a2,…,xn.ak)}return Answer

Almostneverthefastest waytocomputeit!

Note:this isamultiset union

AnexampleofSQLsemantics

42

SELECT R.AFROM R, SWHERE R.A = S.B

A13

B C2 33 43 5

A B C1 2 31 3 41 3 53 2 33 3 43 3 5

CrossProduct

A B C3 3 43 3 5

A33

ApplyProjectionApply

Selections/Conditions

Output

Notethesemantics ofajoin

43

SELECT R.AFROM R, SWHERE R.A = S.B

Recall:Crossproduct(AXB)isthesetofalluniquetuplesinA,B

Ex:{a,b,c}X{1,2}={(a,1),(a,2),(b,1),(b,2),(c,1),(c,2)}

=Filtering!

=Returningonlysome attributes

Rememberingthisorderiscriticaltounderstandingtheoutputofcertainqueries(seelateron…)

1. Takecrossproduct:𝑋 = 𝑅×𝑆

2. Applyselections/conditions:𝑌 = 𝑟, 𝑠 ∈ 𝑋 𝑟. 𝐴 == 𝑟. 𝐵}

3. Applyprojections togetfinaloutput:𝑍 = (𝑦. 𝐴, )𝑓𝑜𝑟𝑦 ∈ 𝑌

Note:wesay“semantics”not“executionorder”

• Theprecedingslidesshowwhatajoinmeans

• NotactuallyhowtheDBMSexecutesitunderthecovers

GooverActivity2-3

45

4.AdvancedSQL

46

47

SetOperatorsandNestedQueries

48

SELECT DISTINCT R.AFROM R, S, TWHERE R.A=S.A OR R.A=T.A

AnUnintuitiveQuery

ComputesRÇ (SÈ T)

ButwhatifS=f?

S T

R

Gobacktothesemantics!

Whatdoesitcompute?

49


AnUnintuitiveQuery

• Recallthesemantics!1. Takecross-product2. Applyselections /conditions3. Applyprojection

• IfS={},thenthecrossproductofR,S,T={},andthequeryresult={}!

Mustconsidersemanticshere.Aretheremoreexplicitwaytodosetoperationslikethis?

50


WhatdoesthislooklikeinPython?

• Semantics:1. Takecross-product

2. Applyselections /conditions

3. Applyprojection

Joins/cross-products arejustnestedforloops (insimplestimplementation)!

If-thenstatements!

RÇ (SÈ T)

S T

R

51


WhatdoesthislooklikeinPython?

RÇ (SÈ T)

S T

R

output = {}

for r in R:for s in S:

for t in T:if r[‘A’] == s[‘A’] or r[‘A’] == t[‘A’]:

output.add(r[‘A’])return list(output)

CanyouseenowwhathappensifS=[]?

52

Multisetoperations

RecallMultisets

53

Tuple

(1,a)

(1,a)

(1, b)

(2,c)

(2,c)

(2,c)

(1,d)

(1,d)

Tuple 𝝀(𝑿)

(1,a) 2

(1,b) 1

(2,c) 3

(1, d) 2EquivalentRepresentationsofaMultiset

Multiset X

Multiset X

Note:Inasetallcountsare{0,1}.

𝝀 𝑿 =“CountoftupleinX”(Itemsnotlistedhaveimplicitcount0)

GeneralizingSetOperationstoMultisetOperations

54

Tuple 𝝀(𝑿)

(1,a) 2

(1,b) 0

(2,c) 3

(1, d) 0

Multiset X

Tuple 𝝀(𝒀)

(1,a) 5

(1,b) 1

(2,c) 2

(1, d) 2

Multiset Y

Tuple 𝝀(𝒁)

(1,a) 2

(1,b) 0

(2,c) 2

(1, d) 0

Multiset Z

∩ =

𝝀 𝒁 = 𝒎𝒊𝒏(𝝀 𝑿 , 𝝀 𝒀 )Forsets,thisisintersection

55

Tuple 𝝀(𝑿)

(1,a) 2

(1,b) 0

(2,c) 3

(1, d) 0

Multiset X

Tuple 𝝀(𝒀)

(1,a) 5

(1,b) 1

(2,c) 2

(1, d) 2

Multiset Y

Tuple 𝝀(𝒁)

(1,a) 5

(1,b) 1

(2,c) 3

(1, d) 2

Multiset Z

∪ =

𝝀 𝒁 = 𝒎𝒂𝒙(𝝀 𝑿 , 𝝀 𝒀 )Forsets,

thisisunion


Multiset OperationsinSQL

56

s

ExplicitSetOperators:INTERSECT

57

SELECT R.AFROM R, SWHERE R.A=S.AINTERSECTSELECT R.AFROM R, TWHERE R.A=T.A Q1 Q2

𝑟. 𝐴 𝑟. 𝐴 = 𝑠. 𝐴 ∩ 𝑟. 𝐴 𝑟. 𝐴 = 𝑡. 𝐴}

UNION

58

SELECT R.AFROM R, SWHERE R.A=S.AUNIONSELECT R.AFROM R, TWHERE R.A=T.A Q1 Q2

𝑟. 𝐴 𝑟. 𝐴 = 𝑠. 𝐴 ∪ 𝑟. 𝐴 𝑟. 𝐴 = 𝑡. 𝐴}

Whyaren’tthereduplicates?

Whatifwewantduplicates?

UNIONALL

59

SELECT R.AFROM R, SWHERE R.A=S.AUNION ALLSELECT R.AFROM R, TWHERE R.A=T.A Q1 Q2

𝑟. 𝐴 𝑟. 𝐴 = 𝑠. 𝐴 ∪ 𝑟. 𝐴 𝑟. 𝐴 = 𝑡. 𝐴}

ALLindicatestheMultisetdisjointunionoperation

s

60

Tuple 𝝀(𝑿)

(1,a) 2

(1,b) 0

(2,c) 3

(1, d) 0

Multiset X

Tuple 𝝀(𝒀)

(1,a) 5

(1,b) 1

(2,c) 2

(1, d) 2

Multiset Y

Tuple 𝝀(𝒁)

(1,a) 7

(1,b) 1

(2,c) 5

(1, d) 2

Multiset Z

=

𝝀 𝒁 = 𝝀 𝑿 + 𝝀 𝒀Forsets,

thisisdisjointunion


t

EXCEPT

61

SELECT R.AFROM R, SWHERE R.A=S.AEXCEPTSELECT R.AFROM R, TWHERE R.A=T.A Q1 Q2

𝑟. 𝐴 𝑟. 𝐴 = 𝑠. 𝐴 \{𝑟. 𝐴|𝑟. 𝐴 = 𝑡. 𝐴}

Whatisthemultiset version?

𝝀 𝒁 = 𝝀 𝑿 − 𝝀 𝒀ForelementsthatareinX

INTERSECT:Stillsomesubtleproblems…

62

Company(name, hq_city)Product(pname, maker, factory_loc)

SELECT hq_cityFROM Company, ProductWHERE maker = name

AND factory_loc = ‘US’INTERSECTSELECT hq_cityFROM Company, ProductWHERE maker = name

AND factory_loc = ‘China’

WhatiftwocompanieshaveHQinUS:BUTonehasfactoryinChina(butnotUS)andviceversa? Whatgoeswrong?

“HeadquartersofcompanieswhichmakegizmosinUSAND China”

INTERSECT:Rememberthesemantics!

63

Company(name, hq_city) AS CProduct(pname, maker, factory_loc) AS P


AND factory_loc=‘US’INTERSECTSELECT hq_cityFROM Company, ProductWHERE maker = nameAND factory_loc=‘China’

Example:CJOINPonmaker=nameC.name C.hq_city P.pname P.maker P.factory_loc

XCo. Seattle X XCo. U.S.

YInc. Seattle X Y Inc. China

s

INTERSECT:Rememberthesemantics!

64

Company(name, hq_city) AS CProduct(pname, maker, factory_loc) AS P


AND factory_loc=‘US’INTERSECTSELECT hq_cityFROM Company, ProductWHERE maker = nameAND factory_loc=‘China’

Example:CJOINPonmaker=nameC.name C.hq_city P.pname P.maker P.factory_loc

XCo. Seattle X XCo. U.S.

YInc. Seattle X Y Inc. China

XCohasafactoryintheUS(butnotChina)YInc.hasafactorinChina(butnotUS)

ButSeattleisreturnedbythequery!

WedidtheINTERSECTonthewrongattributes!

OneSolution:NestedQueries

65

Company(name, hq_city)Product(pname, maker, factory_loc)

SELECT DISTINCT hq_cityFROM Company, ProductWHERE maker = name

AND name IN (SELECT makerFROM ProductWHERE factory_loc = ‘US’)

AND name IN (SELECT makerFROM ProductWHERE factory_loc = ‘China’)

“HeadquartersofcompanieswhichmakegizmosinUSAND China”

Note:Ifwehadn’tusedDISTINCThere,howmanycopiesofeachhq_city wouldhavebeenreturned?

s

High-levelnoteonnestedqueries

• WecandonestedqueriesbecauseSQLiscompositional:

• Everything(inputs/outputs)isrepresentedasmultisets- theoutputofonequerycanthusbeusedastheinputtoanother(nesting)!

• Thisisextremely powerful!

67

Nestedqueries:Sub-queriesReturningRelations

SELECT c.cityFROM Company cWHERE c.name IN (

SELECT pr.makerFROM Purchase p, Product prWHERE p.product = pr.nameAND p.buyer = ‘Joe Blow‘)

“CitieswhereonecanfindcompaniesthatmanufactureproductsboughtbyJoeBlow”

Company(name, city)Product(name, maker)Purchase(id, product, buyer)

Anotherexample:

68

NestedQueries

SELECT c.cityFROM Company c,

Product pr, Purchase p

WHERE c.name = pr.makerAND pr.name = p.productAND p.buyer = ‘Joe Blow’

Isthisqueryequivalent?

Bewareofduplicates!

69

NestedQueries

SELECT DISTINCT c.cityFROM Company c,

Product pr, Purchase p

WHERE c.name = pr.makerAND pr.name = p.productAND p.buyer = ‘Joe Blow’

Nowtheyareequivalent

SELECT DISTINCT c.cityFROM Company cWHERE c.name IN (SELECT pr.makerFROM Purchase p, Product prWHERE p.product = pr.name

AND p.buyer = ‘Joe Blow‘)

70

Subqueries ReturningRelations

SELECT nameFROM ProductWHERE price > ALL(

SELECT priceFROM ProductWHERE maker = ‘Gizmo-Works’)

Product(name, price, category, maker)

Youcanalsouseoperationsoftheform:• s>ALLR• s<ANYR• EXISTSR

Findproductsthataremoreexpensivethanallthoseproducedby“Gizmo-Works”

Ex:

ANYandALLnotsupportedbySQLite.

71

SubqueriesReturningRelations

SELECT p1.nameFROM Product p1WHERE p1.maker = ‘Gizmo-Works’

AND EXISTS(SELECT p2.nameFROM Product p2WHERE p2.maker <> ‘Gizmo-Works’

AND p1.name = p2.name)

Product(name, price, category, maker)

Youcanalsouseoperationsoftheform:• s>ALLR• s<ANYR• EXISTSR

Find‘copycat’products,i.e.productsmadebycompetitorswiththesamenamesasproductsmadeby“Gizmo-Works”

Ex:

<>means!=

72

NestedqueriesasalternativestoINTERSECTandEXCEPT

(SELECT R.A, R.BFROM R)

INTERSECT(SELECT S.A, S.BFROM S)

SELECT R.A, R.BFROM RWHERE EXISTS(

SELECT *FROM SWHERE R.A=S.A AND R.B=S.B)

SELECT R.A, R.BFROM RWHERE NOT EXISTS(

SELECT *FROM SWHERE R.A=S.A AND R.B=S.B)

INTERSECTandEXCEPTnotinsomeDBMSs!

IfR,Shavenoduplicates,thencanwritewithoutsub-queries(HOW?)(SELECT R.A, R.B

FROM R)EXCEPT(SELECT S.A, S.BFROM S)

73

CorrelatedQueries

SELECT DISTINCT titleFROM Movie AS mWHERE year <> ANY(

SELECT yearFROM MovieWHERE title = m.title)

Movie(title, year, director, length)

Notealso:thiscanstillbeexpressedassingleSFWquery…

Findmovieswhosetitleappearsmorethanonce.

Notethescopingofthevariables!

74

ComplexCorrelatedQuery

SELECT DISTINCT x.name, x.makerFROM Product AS xWHERE x.price > ALL(

SELECT y.priceFROM Product AS yWHERE x.maker = y.maker

AND y.year < 1972)

Findproducts(andtheirmanufacturers)thataremoreexpensivethanallproductsmadebythesamemanufacturerbefore1972

Product(name, price, category, maker, year)

Canbeverypowerful(alsomuchhardertooptimize)

GooverActivity3-1

75

BasicSQLSummary

• SQLprovidesahigh-leveldeclarativelanguageformanipulatingdata(DML)

• TheworkhorseistheSFWblock

• Setoperatorsarepowerfulbuthavesomesubtleties

• Powerful,nestedqueriesalsoallowed.

76

Date post:	25-May-2020
Category:	Documents
Upload:	others
View:	20 times
Download:	0 times

Lecture 5 SQL - GitHub PagesSQL Introduction •SQL is a standard language for querying and...

Documents