MobAppDev

Structured Data Persistence on Android Devices with SQLite

Vladimir Kulyukin

www.vkedco.blogspot.com

Outline

● Persisting of Structured Data with SQL Databases

● Lite Intro to SQLite● SQLite Database Inspection● SQLite by Example● Creation & Population of SQLite Databases

Persistence of Structured Data with

SQL Databases

SharedPreferences: Pros & Cons

● Pros Lightweight Easy to code and debug

● Cons Slow with large data volumes Not helpful with highly structured data when key/value

pairs are insufficient Entire file needs to be read and parsed to access data Takes up more space because XML is used for storage

SQL Databases: Pros & Cons● Pros

Flexible with structured data Scalable: can handle large data volumes Robust: can handle incremental changes Responsive: can handle flexible queries

● Cons Heavyweight: more code & more memory Maintenance: harder to code & debug

Three-Tier Data Management Model

XML

Java Objects SQLite DB

GUI

Object Inflation

Objects to SQLite Records

Object Visualization

SQLite Records to Objects

Database Design

● Standard database design practices also apply on Android● A key design principle is to manage space well: reduction

redundancy via database normalization● The objective of database normalization is to decompose relations

with anomalies to produce smaller, well structured relations● Database design is more an art than a science: takes experience &

practice● Helpful resource: en.wikipedia.org/Database_normalization

Tables & Data Types

● Database is a persistent collection of structured data● Data are structured because they are organized in tables● A Table is similar to an Excel spreadsheet except that its

columns have standard SQL data types (e.g., TEXT, INTEGER, FLOAT, etc.)

● In terms of Object-Oriented Modeling, each table represents an Class (e.g., Book, Author) and each column represents an Attribute

● Each table row represents an Object of a specific Class

Primary Keys● In many tables, the ID column represents the

primary key● The primary key is the key that uniquely

identifies the row entry object● When design an SQL table, it is typical (not

necessary!) to specify the primary key as an integer and mark it as auto-incrementable: this way every inserted row is guaranteed to have a unique primary key

Lite Intro to SQLite

What is SQLite?● SQLite is a relational database management system

(RDBMS)● SQLite is a compact C library that is part of the Android

software stack● Since it is a library, it can be integrated into each

application without external dependencies: this simplifies transactions and synchronizations, minimizes latency

● If you know or have worked with SQL, you will have no problem with SQLite

Where to Look for Application-Specific SQLite Databases

● Every application can create its own database over which it has complete control

● SQLite databases should be considered only when you need to manage complex, structured data

● Databases are stored in /data/data/<package_name>/databases/● For example, if the package is org.vkedco.mobappdev.simple_db_app_01,

then look for the databases in

/data/data/org.vkedco.mobappdev.simple_db_app_01/databases/

SQL vs SQLite Data Typing

● SQL engines typically use static typing: datatype of a value is determined by the column in which the value is stored

● SQLite engines use dynamic typing: datatype of a value is associated with the value not the column in which it is stored

● Basic SQL vs. SQLite tradeoff: security and safety (SQL) vs. flexibility (SQLite)

● SQLite rule of thumb: do not expose your SQLite API to third parties

SQLite Datatypes

● NULL – the value is NULL● INTEGER – signed integers; signed integer can be stored as

1, 2, 3, 4, 5, 6, or 8 bytes; the storage size depends on the value magnitude

● REAL – float-point numbers; stored as 8 bytes● TEXT – text string; stored according to the database

encoding (e.g., UTF-8)● BLOB – data blob, typically stored as array of bytes

Sample Book Database

SQLite Database

● Our sample book database on Sufi poetry will contain two tables

● The first table consists of five book records● The second table consists of two book authors● The exact tables are given on the next two

slides

Book Table

ID Title Author Translator ISBN Price

1 The Essential Rumi Jalal al-Din Rumi

C. Barks, J. Moyne

9780062509581 17.51

2 The Illuminated Rumi Jalal al-Din Rumi

C. Barks 9780767900027 25.04

3 A Year with Rumi: Daily Readings

Jalal al-Din Rumi

C. Barks 9780060845971 14.92

4 A Year with Hafiz: Daily Contemplations

Hafiz D. Ladinsky 9780143117544 12.24

5 The Gift Hafiz D. Ladinsky 9780140195811 12.24

- Book is a Class - ID, Title, Author, Translator, ISBN, Price are Attributes - Rows 1 – 5 are Book Objects

Author Table

ID Name BirthYear DeathYear Country

1 Jalal al-Din Rumi 1207 1273 Persia

2 Hafiz 1325 1389 Persia

- Author is a Class - ID, Name, BirthYear, DeathYear, Country, are Attributes - Row 1 is an Author Object - Row 2 is an Author Object

Command Line DB Access

Suppose that you have created a database such as the book database shown in the previous slides (we will learn how to do this shortly) and want to interact with it. Here is how:

1. Open a command line

2. CD into platform-tools directory

3. Connect to the device (e.g., adb -e shell)

4. CD into the database directory: cd /data/data/<package>databases>

5. Execute: sqlite3 <database_name> (e.g. sqlite3 book_info.db)

Now you can interct with book_info.db via sqlite shell

6. Execute .quit to quit

Connecting to ADB

You can use adb -e shell to connect to the emulator;

Then you can use standard Linux commands such as ls

CD Into Data/Data/<APP PACKAGE>

1. CD Into data/data/ 2. LS 3. Look for APP PACKAGE 4. CD into it 5. LS to find database names

Discovering TABLE Schemas

● A Table Schema is the specification of column values and types for a given table

● To access table schemas for a DB:

1. Get in to the sqlite shell

2. Execute: .schema

You should see something like:

CREATE TABLE android_metadata (locale TEXT);

CREATE TABLE author (ID integer primary key autoincrement, Name text not null, ...);

CREATE TABLE book (ID integer primary key autoincrement, Title text not null, ...);

Using SQLite3 Interpreter

SQLite by Example

Displaying Tables

Here is how you can display the entire table:

SELECT * FROM <TABLE>;

Examples: SELECT * FROM BOOK; SELECT * FROM AUTHOR;

Column Projections

To project the values of specific columns for each record:

SELECT COLUM_NAME_1, …, COLUMN_NAME_n FROM <TABLE>;

Examples:

SELECT Author, Title FROM BOOK;

SELECT Title, Author FROM BOOK;

Retrieval of Records with Specific Values

To retrieve records with specific values, use the WHERE clause:

SELECT COLUM_NAME_1, …, COLUMN_NAME_n FROM <TABLE> WHERE <COL_VAL_SPEC>;

Examples:

SELECT Title FROM BOOK WHERE ID = 3;

SELECT Title, Author FROM BOOK WHERE ID < 3;

Record Ordering

To order records by attributes, use the ORDER BY clause:

SELECT COLUM_NAME_1, …, COLUMN_NAME_n FROM <TABLE> ORDER BY COL_NAME Asc/Desc;

Examples:

SELECT Title, Author FROM BOOK ORDER BY Price Asc;

SELECT Title, Author FROM BOOK ORDER BY Price Desc;

Creation & Population of

SQLite Databases

Book DB Application

Write an Android application that creates and populates an SQLite database on Sufi poetry from an XML specification of the book title and book author tables. The book titles are displayed in a ListView. When the user clicks on an book title, its database record is displayed in a Toast. The application does OO modeling of SQLite tables as Java classes.

Book DB Application

source code is here

Sample Screenshots

Book Table as Book Class

public class Book {

protected String mTitle;

protected String mAuthor;

protected String mTranslator;

protected String mISBN;

protected float mPrice;

// rest of code

}

public class Book {

protected String mTitle;

protected String mAuthor;

protected String mTranslator;

protected String mISBN;

protected float mPrice;

// rest of code

}

source code is in Book.java

Author Table as Author Class

public class Author {

protected String mName;

protected int mBirthYear;

protected int mDeathYear;

protected String mCountry;

// rest of code

}

source code is in Author.java

XML DB Table Specs

● XML DB table specs can come in a variety of formats and from a variety of resources

● Each XML table spec must describe inflatable row records; table specs inflated into table Java objects

● Table specs give you the flexibility to persist (save) Java into SQLite databases or Object databases

● Source of table specs can vary from local XML files, remote XML files, RSS feeds, etc

XML Specs of Book Title Table

<string-array name="book_title_table">

<item>The Essential Rumi;Jalal adDin Rumi;C. Barks, J. Moyne;9780062509581;17.51</item>

<item>The Illuminated Rumi;Jalal adDin Rumi;C. Barks;9780767900027;25.04</item>

<item>A Year with Rumi: Daily Readings;Jalal adDin Rumi;C. Barks;9780060845971;14.92</item>

<item>A Year with Hafiz: Daily Contemplations;Hafiz;D. Ladinsky;9780143117544;12.24</item>

<item>The Gift;Hafiz;Daniel Ladinsky;9780140195811;12.24</item>

</string-array>

XML source is in strings.xml

XML Specs of Book Author Table

<string-array name="book_author_table">

<item>Jalal adDin Rumi;1207;1273;Persia</item>

<item>Hafiz;1325;1389;Persia</item>

</string-array>

XML source is in strings.xml

Design & Implementation of

Database Adapters

DB XML Specs Table Inflation

● Typically there is a class that creates objects from XML table specs

● In our case, we simply inflate XML string-arrays and create objects from XML strings

● Database adapter classes are intermediaries b/w data and SQLite databases; they typically extend SQLiteOpenHelper

● Each object is then given to the database adapter for SQLite persistence

Population of SQLite DB

● Once we have an adapter class, we can use it to populate the SQLite DB

● This is done in SufiPoetryBooksDBMainAct.java by the method populateBookList()

● This method parses XML, creates Book and Author objects and uses the methods insertUniqueBook() and insertUniqueAuthor() defined in SufiPoetryBooksDBHelper.java

Population of SQLite DB: XML Inflation

// 1. open db

mDBHelper.open();

// 2. Read XML arrays or use an XML parser to parse XML

String[] book_table = mRes.getStringArray(R.array.book_table);

String[] author_table =

mRes.getStringArray(R.array.author_table);

Population of SQLite DB: Population of Book Table

// 3. Inflate book table xml specs into Book objects and place them into SQLite DB

String[] book_entry_parts;

for(String book_entry: book_table) {

book_entry_parts = book_entry.trim().split(XML_ENTRY_SEPARATOR);

mDBHelper.insertUniqueBook(new Book(book_entry_parts[0], // title

book_entry_parts[1], // author

book_entry_parts[2], // translator

book_entry_parts[3], // isbn

Float.parseFloat(book_entry_parts[4]) // price

));

}

Population of SQLite DB: Population of Author Table// 4. Inflate author table xml specs into Author objects and insert them into DB

String[] author_entry_parts;

for(String author_entry: author_table) {

author_entry_parts = author_entry.trim().split(XML_ENTRY_SEPARATOR);

mDBHelper.insertUniqueAuthor(

new Author(author_entry_parts[0], // author's name

Integer.parseInt(author_entry_parts[1]), // author's birth year

Integer.parseInt(author_entry_parts[2]), // author's death year

author_entry_parts[3]) // author's country

);

}

// 5. Close db

mDBHelper.close();

Storing Images in SQLite Databases

Modification of Book DB Application

Let us extend our book db application so that it has an SQLite table that stores book cover images and maps them into book ISBNs.

source is here

BOOK_COVER_IMAGE Table

Data Rows

9780062509581|BLOB_01

9780767900027|BLOB_02

9780060845971|BLOB_03

9780143117544|BLOB_04

9780140195811|BLOB_05

Database Schema

ISBN | Image (BLOB)

Image Blobs from Amazon

BLOB_01 BLOB_02 BLOB_03 BLOB_04 BLOB_05

OO Modeling of Book Cover Image Table

public class BookCoverImage {

protected String mISBN;

protected byte[ ] mCoverRef;

public BookCoverImage(String isbn, byte[] coverRef) {

mISBN = isbn; mCoverRef = coverRef;

}

public String getISBN() { return mISBN; }

public byte[] getCoverRef() { return mCoverRef; }

}

Images Into Byte Arrays Bitmap bmp = null; Resources mRes = cntxt.getResources();

// 1. Construct a bitmap from a source image

bmp = BitmapFactory.decodeResource(mRes, R.drawable.essential_rumi_cover);

// 2. Create a ByteArrayOutputStream

ByteArrayOutputStream bos = new ByteArrayOutputStream();

// 3. Compress the bitmap into the ByteArrayOutputStream

byte[] bitmapbytes = null; bmp.compress(CompressFormat.PNG, 0, bos);

// 4. Get the bytes the bitmap's bytes from the ByteArrayOutputStream

bitmapbytes = bos.toByteArray();

// 5. Recycle the bitmap so that it can be used again

bmp.recycle();

// 6. Close the ByteArrayOutputStream

try { bos.close(); } catch (IOException e) { e.printStackTrace(); }

Images Into Byte Arrays Bitmap bmp = null; Resources mRes = cntxt.getResources();

// 1. Construct a bitmap from a source image

bmp = BitmapFactory.decodeResource(mRes, R.drawable.essential_rumi_cover);

// 2. Create a ByteArrayOutputStream

ByteArrayOutputStream bos = new ByteArrayOutputStream();

// 3. Compress the bitmap into the ByteArrayOutputStream

byte[] bitmapbytes = null; bmp.compress(CompressFormat.PNG, 0, bos);

// 4. Get the bytes the bitmap's bytes from the ByteArrayOutputStream

bitmapbytes = bos.toByteArray();

// 5. Recycle the bitmap so that it can be used again

bmp.recycle();

// 6. Close the ByteArrayOutputStream

try { bos.close(); } catch (IOException e) { e.printStackTrace(); }

Inserting Images into SQLite Databases // Assume that bci is a BookCoverImage object.

byte[] bitmapbytes = bci.getCoverRef();

// we assume that bitmapbytes is not null

ContentValues newBookCoverRef = new ContentValues();

// COVER_IMAGE_ISBN_COL_NAME is a string constant

newBookCoverRef.put(COVER_IMAGE_ISBN_COL_NAME, bci.getISBN());

newBookCoverRef.put(COVER_IMAGE_IMG_COL_NAME, bitmapbytes);

// mDb is a database reference of SQLiteDatabase class.

return mDb.insertWithOnConflict(BOOK_COVER_IMAGE_TABLE,

null, newBookCoverRef,

SQLiteDatabase.CONFLICT_REPLACE);

References● en.wikipedia.org/Database_normalization● http://developer.android.com/tools/help/adb.html● http://developer.android.com/reference/android/database/sqlite/SQLiteDatabase.html