Jacome Cunha Flatten from/to Relational – 1 / 21

Flatten from/to Relational

Jacome Cunha∗ Joao Saraiva∗ Joost Visser†

CIC 2007

22-23 October

∗Universidade do Minho†Software Improvement Group

Overview

Introduction

The Transformation

Example

Conclusions

Jacome Cunha Flatten from/to Relational – 2 / 21

Introduction

The Transformation

Example

Conclusions

Introduction

Introduction

• Motivation• Bidirectionaltransformation

• What is a spreadsheet?

• Relational database

• Advantages of DB

The Transformation

Example

Conclusions

Jacome Cunha Flatten from/to Relational – 3 / 21

Motivation

Introduction

• Motivation• Bidirectionaltransformation

• What is a spreadsheet?

• Relational database

• Advantages of DB

The Transformation

Example

Conclusions

Jacome Cunha Flatten from/to Relational – 4 / 21

Spreadsheets are considered one of the largest pro-gramming languages in the world

Their languages/systems lack structured programmingfeatures

Programming in a spreadsheet environment is an errorprone task

Data manipulation is not so supported as in otherparadigms

Bidirectional transformation

Introduction

• Motivation• Bidirectionaltransformation

• What is a spreadsheet?

• Relational database

• Advantages of DB

The Transformation

Example

Conclusions

Jacome Cunha Flatten from/to Relational – 5 / 21

DBSS

The flatten model of a spreadsheet is mapped into arelational database model

The other way around is also shown

What is a spreadsheet?

Introduction

• Motivation• Bidirectionaltransformation

• What is a spreadsheet?

• Relational database

• Advantages of DB

The Transformation

Example

Conclusions

Jacome Cunha Flatten from/to Relational – 6 / 21

celltext

formulasgraphics

incrementalreferences to other cells

Relational database

Introduction

• Motivation• Bidirectionaltransformation

• What is a spreadsheet?

• Relational database

• Advantages of DB

The Transformation

Example

Conclusions

Jacome Cunha Flatten from/to Relational – 7 / 21

It is a structured collection of records distributed overtables

Some concepts:

Primary key it is a set of attributes in the record thatuniquely determine all the others

Functional dependency a set of attributes X function-ally determines a set of attributes Y iff each X valueis associated with only one Y value

Foreign key it is a reference to another entry in othertable

Advantages of DB

Introduction

• Motivation• Bidirectionaltransformation

• What is a spreadsheet?

• Relational database

• Advantages of DB

The Transformation

Example

Conclusions

Jacome Cunha Flatten from/to Relational – 8 / 21

The existence of database management system(DBMS) allows:

• insert, remove and update data in the DB• chance the format of the DB• ensures the integrity of the DB• backup and replication• give permissions to users or deny them

Examples:

• Oracle, Microsoft Access, MySQL, PostgreSQL

The Transformation

Introduction

The Transformation

• The big picture

• Inferring FDs

• Normalisation

• Data migration

Example

Conclusions

Jacome Cunha Flatten from/to Relational – 9 / 21

The big picture

Introduction

The Transformation

• The big picture

• Inferring FDs

• Normalisation

• Data migration

Example

Conclusions

Jacome Cunha Flatten from/to Relational – 10 / 21

a11 . . . a1n...

...am1 . . . amn

:: (A1 × . . . × An)∗

ց↓ extract FDs

↓ 2LT Π(Ai . . . Ak ⇀ Aj . . . Al)

↓ normalisation

(am . . . av ⇀ an . . . ay)∗ :: Π(Am . . . Av ⇀ An . . . At)

Inferring FDs

Introduction

The Transformation

• The big picture

• Inferring FDs

• Normalisation

• Data migration

Example

Conclusions

Jacome Cunha Flatten from/to Relational – 11 / 21

We assume that the table has only data, not code orformulas or other things

Information about relation between columns is neededotherwise there will be too many FDs and will becomeuseless

The FUN algorithm (Noel Novelli and Rosine Cicchetti)is used to infer the FDs

Normalisation

Introduction

The Transformation

• The big picture

• Inferring FDs

• Normalisation

• Data migration

Example

Conclusions

Jacome Cunha Flatten from/to Relational – 12 / 21

For each association of columns several FDs areyielded by the algorithm

One is chosen and a table is created according to it

The columns that are not referred are collected into atable where the primary key is a set constituted by theprimary keys from other tables

Data migration

Introduction

The Transformation

• The big picture

• Inferring FDs

• Normalisation

• Data migration

Example

Conclusions

Jacome Cunha Flatten from/to Relational – 13 / 21

Performed by implementing refinement laws into 2LT

kA ⇀ B

to

((

∼= [kA × B]

from

hh

Abstraction (from) and representation (to) functionsare needed to witness the refinement

Example

Introduction

The Transformation

Example

• Internships

• Company’s table

• Student’s tables

• DB model

Conclusions

Jacome Cunha Flatten from/to Relational – 14 / 21

Internships

Introduction

The Transformation

Example

• Internships

• Company’s table

• Student’s tables

• DB model

Conclusions

Jacome Cunha Flatten from/to Relational – 15 / 21

Internships

Introduction

The Transformation

Example

• Internships

• Company’s table

• Student’s tables

• DB model

Conclusions

Jacome Cunha Flatten from/to Relational – 15 / 21

There should be “enough” cases to represent the reality

Company’s table

Introduction

The Transformation

Example

• Internships

• Company’s table

• Student’s tables

• DB model

Conclusions

Jacome Cunha Flatten from/to Relational – 16 / 21

company, contacts and location are related

company -> contacts, location

contacts -> company, location

Company’s table

Introduction

The Transformation

Example

• Internships

• Company’s table

• Student’s tables

• DB model

Conclusions

Jacome Cunha Flatten from/to Relational – 16 / 21

company, contacts and location are related

company -> contacts, location

contacts -> company, location

companycontactslocation

Company

Student’s tables

Introduction

The Transformation

Example

• Internships

• Company’s table

• Student’s tables

• DB model

Conclusions

Jacome Cunha Flatten from/to Relational – 17 / 21

st.number, st.name and graduation are alsorelated

st.number -> st.name, graduation

Student’s tables

Introduction

The Transformation

Example

• Internships

• Company’s table

• Student’s tables

• DB model

Conclusions

Jacome Cunha Flatten from/to Relational – 17 / 21

st.number, st.name and graduation are alsorelated

st.number -> st.name, graduation

number

namegraduation

Student

DB model

Introduction

The Transformation

Example

• Internships

• Company’s table

• Student’s tables

• DB model

Conclusions

Jacome Cunha Flatten from/to Relational – 18 / 21

companycontactslocation

companynumber

salarystatus

descriptiontime

number

namegraduation

Company Internship Student

Company (_company, contacts, location)

Student (_st.number, st.name, graduation)

Internship (_company, _st.number, salary, status, description, time)

foreign key (company)

references Company(company)

foreign key (st.number)

references Student(st.number)

Conclusions

Introduction

The Transformation

Example

Conclusions

• Conclusions

• Future work

Jacome Cunha Flatten from/to Relational – 19 / 21

Conclusions

Introduction

The Transformation

Example

Conclusions

• Conclusions

• Future work

Jacome Cunha Flatten from/to Relational – 20 / 21

Start with a flatten data table, a spreadsheet

Determine FDs (automatically) and the relational DBschema (not so automatically)

Allows the migration to a well known and supportedparadigm

Future work

Introduction

The Transformation

Example

Conclusions

• Conclusions

• Future work

Jacome Cunha Flatten from/to Relational – 21 / 21

Import a spreadsheet (and other formats) to HASKELL

Tune the FDs inference/normalisation

Integration into 2LT framework

Create ways of export the new model to SQL or tospreadsheet