VNR SQLite Carver - RUSOLUT · 2019-05-20 · SQL table definition SQLite storage classes INTEGER...

transcript

VNR SQLite CarverVeronika - Rusolut

PСLaptopsMobile devices

Flight softwareCar multimedia systemsDrones

Most applications that people use every day

Well-knownoperatingsystems

WEB browsersFile hosting services

SQLITE IS EVERYWHERE

WHAT’S INSIDE SQLITE DATABASE

id name surname created_at updated_at

1 John Jones 2019-03-15 08:08:52 2019-03-24 08:06:10

2 Maria Mendis 2019-03-16 06:10:00 2019-03-26 02:51:12

N Joe Black 2019-03-16 08:30:03 2019-04-11 11:47:38

Database

Record(row)

Field(column)Primary key

DATABASE TRANSACTION

Initial state

End state

COMMIT

ROLLBACK

transaction

Completedtransaction

Failedtransaction

DATA RECOVERY LOOK INTO SQLITE

FS metadata

Allocated spaceUnallocated

spaceAllocated

spaceUnallocated

space …

clusters

Logical image/File System level

SQLiteFile level

SQLitePage level

Header

Old page 2

Old page N

Header

New page 2

New page N

The newest page 2

Page N

Unallocted space

Page 2Page N

MAIN SQLITE DATABASE FILELock-byte page

Freelist

Payload overflow page

B-tree page

Pointer map page

B-tree page No.1

(*.db, *.sqlite, *.sqlite3 etc.)

THE ROLLBACK JOURNAL

Page N

Data change

Commit

Old page 2

Old page 3

Old page N

New page 2

New page 3

New page N

Initial state

Main DB file Main DB file

Rollback journal file

Rollback journalfile header

Rollbackjournal

Size = 0

Zeroedheader

(“-journal” file)

THE WRITE-AHEAD LOG

Page N

Data change

Checkpoint

New page 2

New page 3

New page N

New page 2

New page 3

New page NCOMMIT

The newest page 3

The newest page NCOMMIT

Old page 2

Old page 3

Old page N

Initial state

Main DB file Main DB file Main DB file

WAL file

WAL file header New page 2

New page 3

New page NCOMMIT

The newest page 3

The newest page NCOMMIT

WAL file header

(“-wal” file)

SQLITE DATABASE PAGES

Unused page

Database file header

B-tree page header

Reserved region

Cell content/Data area

Unallocated space/Deleted cells

Cell pointer array

Freelist- Deleted data

B-tree page No.1

Record

Recordlength, B

ROWIDRecord header

length, BType of Field1

…Type of Field N

Data of Field 1

…Data of Field K

Record header

CELL STRUCTURE

0 0 NULL

1 ≤ N ≤ 4 N Signed integer

5 6 Signed integer

6 8 Signed integer

7 8 IEEE float

8 0 Integer 0

9 0 Integer 1

N ≥ 12 even (N-12)/2 BLOB

N ≥ 13 odd (N-13)/2 TEXT

Type of Field Data size, B Data type

DELETED CELL FROM UNALLOCATED SPACE

Сhanged data of deleted cell

Recordlength, B

ROWIDRecord header

length, BType of Field1

…Type of Field N

Data of Field 1

…Data of Field K

SQL table definition SQLite storage classes

INTEGER

NUMERIC

CREATE TABLE my_table (

id INTEGER PRIMARY KEY,

productID BIGINT,

address TEXT,

name VARCHAR(255),

date DATETIME,

adds NUMERIC,

image BLOB,

longitude REAL,

latitude REAL)

INTEGERType 1-6,8,9

TEXTType ≥ 13 odd

NULLType 0

Type ≥ 12 even

REALType 7

REPRESENTATION OF SQL TABLES

SQLite data types

- Is that all?

- Yes. No. It’s complicated!

Any column in an SQLite database may store a value of any storage class …

VNR SQLite Carver

… and so we decided to create

Manual AutomaticSemi-automatic

VNR SQLite Carver

• PROS• Detailed data analysis

• CONS• Extremely time consuming• You need to convert raw

data manually• Requires very deep

knowledge of databasetheory

HEX Viewer Well-known mobileforensic solutions

• PROS• Universal flexible platform• Get results in a few simple

steps• Adjustable results• More data than other

solutions• CONS

• You may need a minimal SQLite knowledges

• PROS• One-click solutions• Easy to use interface• No SQLite knowledges

required• CONS

• Lack of flexibility• Need to wait for updates

from vendor support• Limited application support

EXISTING SOLUTIONS FOR IN-DEPTH SQLITE DATA CARVING

MORE ADVANTAGES OF VNR SQLITE CARVER

• It automatically creates carving template which can retrieve allpossible data from the dump (unallocated and allocated space )

• It works on database files, partitions and the whole dump

• It recognizes various data types and use the appropriate convertersto transform data to a readable form

• It removes all duplicates to clean your results from unnecessarygarbage

• It is a universal tool – you shouldn’t wait while developers of anyother tool will release the update to add new apps or make changesaccording to the new app version

STEP 1. SELECT SOURCE AND PRESS BUTTON

STEP 2. SELECT THE MODE TO GET CARVING TEMPLATE

STEP 3. SELECT TABLE YOU ARE INTERESTED IN AND GET CARVING TEMPLATE

STEP 4. RUN SQLITE DATA CARVING PROCESS

STEP 5. GET DATA

Processing results

GroupingSortingFiltering data

VNR SQLITE CARVER TEMPLATE ELEMENTS

CARVER TEMPLATE ELEMENT FEATURES

Set size

Set range

Text encoding

Use REGEX

Date & Time converters

Reversed byteorder

Set count

✓ ✓ ✓

✓ ✓

✓ ✓ ✓ ✓

INTEGER

NUMERIC

CARVER TEMPLETE ADJUSTMENT

• Set size

• Set range

• Use REGEX

Use GAP element

Сombinetemplates

into a universal template

Get more clean data

Run data carving again

Specify the range of values

Step 1 Step 2 Step 3

UNIVERSAL TEMPLATE

1st template 2nd template Universal template

VALUE RANGE ADJUSTMENT

Specify field types

Set range

MORE CLEAN DATA

1st template results 2nd template resultsUniversal template results

Now let’s check how it works in real world

VNR SQLite Carver - RUSOLUT · 2019-05-20 · SQL table definition SQLite storage classes INTEGER...

Documents