COMP228 [& 327] App Development Session: 2019-2020Lecture Set 3 - Data Persistence
[ last updated: 23 Oct 2019 ] !1Illustration credit: vecteezy.com
!2
In these Slides...
• We will cover...• An introduction to Local Data Storage
• The iPhone filesystem
• Property lists
• UserDefaults
• Data Modelling using Core Data
Storing Data These slides will allow you to understand how data can be modelled and stored/cached locally. This may be
for a local database, or simply saving state or preferences for an
application.
Local Data Storage
Data Persistence and Core Data
!4
Intro to Data and Persistence
• There may be a number of reasons for wanting to read or write data to some form of persistent store• Storing preferences - (why have preferences?)• Storing simple data in general• Storing state• Simple values or container classes• Serialising custom object data
• Managing data• SQLite• Core Data
!5
iPhone File System• Each application sandboxes it’s file system• Ensures security and privacy (how?)• When apps are deleted, all the associated files are deleted too
• Each app maintains its own set of directories• somewhat reminiscent of a UNIX filesystem• Files stored in Caches are not backed up during iCloud backup
• Apps cannot write into their own bundles• This violates the code-signing mechanism• If apps want to include data in the bundle that can later be modified• it must copy the data into your documents directory• then the data can be modified!• mark such data as isExcludedFromBackup• only copy the data on first use
The sandbox file directory for a simple Core Data app
Inside a simple Core Data app
!6
File Paths in your Application• Getting to the Application’s (writeable) home directory• NSHomeDirectory
• Finding directories in the file system• NSSearchPathForDirectoriesInDomains• Creates an array of path strings for the specified directories in the specified domains• Good for discovering where a “known” directory is in the file system
// Documents directorylet documentsPath = NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true)[0]
// <Application Home>/Documents/myData.plistlet myDataPath = documentsPath + “myData.plist" //it's actually a stringprint(myDataPath)
Why No Parameter names in the method call?
public func NSSearchPathForDirectoriesInDomains(_ directory: FileManager.SearchPathDirectory, _ domainMask: FileManager.SearchPathDomainMask, _ expandTilde: Bool) -> [String]
!7
• Applications can expose files in their Documents directory to iTunes (when syncing)• Allows users to add and delete files directly to an application
• Capability is set by setting the Application supports iTunes file sharing Property in the App’s info.plist to YES
iTunes File sharing
!8
iTunes File sharing
!9
iTunes File sharing
@IBAction func saveAction() { // Documents directory (writeable by the app) let documentsPath = NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true)[0]
// <Application Home>/Documents/myData.plist let myDataPath = documentsPath + “/myData.plist"
let myArray = [NSDate.distantFuture, 5, "Hello World! 👾"] as NSArray //Note - an NSArray myArray.write(toFile: myDataPath, atomically: true) }
<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"><plist version="1.0"><array>
<date>4001-01-01T00:00:00Z</date><integer>5</integer><string>Hello World! 👾</string>
</array></plist>
!10
Property Lists• Property lists are structured data used by Cocoa and Core Foundation• Used extensively within iOS, iPadOS and MacOS etc.• Typically XML-based data format
• Provides support for• Primitive data types
• strings• numbers - integers• numbers - floating point • binary data - (NSData)• dates - (NSDate)• boolean values
• Collections - which can recursively contain other collections• arrays - (NSArray)• dictionaries - (NSDictionary)
• Root-level object is almost always either an array or dictionary• Could be a single element plist containing a primitive data type
!11
Property Lists
• A really good way to store small, structured persistent data fragments• Good for:• less than a few hundred kilobytes of data• well defined data elements, that fit the XML data
serialisation
• Bad for:• Large data volumes• Loading is “one-shot”
• Complex objects• Blocks of binary data
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>CFBundleDevelopmentRegion</key> <string>en</string> <key>CFBundleExecutable</key> <string>$(EXECUTABLE_NAME)</string> <key>CFBundleIdentifier</key> <string>$(PRODUCT_BUNDLE_IDENTIFIER)</string> <key>CFBundleInfoDictionaryVersion</key> <string>6.0</string> <key>CFBundleName</key> <string>$(PRODUCT_NAME)</string> <key>CFBundlePackageType</key> <string>APPL</string> <key>CFBundleShortVersionString</key> <string>1.0</string> <key>CFBundleVersion</key> <string>1</string> <key>LSRequiresIPhoneOS</key> <true/> <key>UILaunchStoryboardName</key> <string>LaunchScreen</string> <key>UIMainStoryboardFile</key> <string>Main</string> <key>UIRequiredDeviceCapabilities</key> <array> <string>armv7</string> </array> <key>UISupportedInterfaceOrientations</key> <array> <string>UIInterfaceOrientationPortrait</string> <string>UIInterfaceOrientationLandscapeLeft</string> <string>UIInterfaceOrientationLandscapeRight</string> </array> </dict> </plist>
!12
Reading and Writing Property Lists
• Both NSArray and NSDictionary have recursive convenience methods• Reading from a file or URL:
• let tempDict2 = NSDictionary(contentsOf: URL) // uses the NSDictionary class initialiser
• Writing to a file or URL• func write(toFile path: String, atomically useAuxiliaryFile: Bool) -> Bool • func write(to url: URL, atomically: Bool) -> Bool
• Property lists can also be serialised from objects into a static format• Can be stored in the file system (in different formats) and read back later• Uses PropertyListSerialization class
// Reading Data.plist from the resource bundle let filePath = Bundle.main.path(forResource: "Data", ofType: “plist") let tempDict = NSDictionary(contentsOfFile: filePath!)
// Reading WrittenArray.plist from the application’s file directory let tempArray = NSArray(contentsOfFile: “WrittenArray.plist")
Note this example is accessing a file from the App Bundle - an actual part of the App that cannot be modified. You can read from but can’t write to the Bundle.
!13
Example: Writing an Array to Disk
let myArray = [NSDate.distantFuture, 5, "Hello World! 👾"] as NSArray myArray.write(toFile: “writtenArray.plist”, atomically: true)
<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"><plist version="1.0"><array>
<date>4001-01-01T00:00:00Z</date><integer>5</integer><string>Hello World! 👾</string>
</array></plist>
Create an array of three items - a date, a value and a string
Call the array’s write toFile method (courtesy of the NSArray type)
The resulting XML file is written in the application’s file space, to be read later
!14
JSON - JavaScript Object Notation• Open standard file format that uses human-readable text to transmit
data objects• Language independent - originally from Javascript• Supports:• Collections of name/value pairs - objects { string : value , string : value , ... }• Ordered list of values - arrays [ value , value , ... ]• Values can be strings, numbers, objects, arrays, “true”, “false”, “null”
• Elements can be nested
• JSONSerialization Class• Converts JSON data into Foundation objects• Converts Foundation objects into JSON data• see isValidJSONObject() for supported types
See http://json.org for full definition
!15
What does it look like?• The JSON structure naturally matches that of the Swift types:
• Dictionaries that contain key-value pairs
• Arrays or Sets that contain lists of values
• (and their bridged Foundation equivalents, NS…)
{ "title": "University of Liverpool Computer Science Programme List", "groups": [
{ "grouptitle": "Agents ART Group", "grouplabel": "agentArt", "members": [ "Katie", "Trevor", "Frans", "Paul D.", "Floriana", "Wiebe", "Dave J.", "Terry", "Valli"]},{ "grouptitle": "Complexity Theory and Algorithmics Group", "grouplabel": "ctag", "members": ["Leszek", "Irina", "Igor", "Prudence", "Michele"]},{ "grouptitle": "Logic and Computation Group", "grouplabel": "loco", "members": ["Michael", "Frank", "Clare", "Ullrich", "Boris", "Alexei", "Sven"]},{ "grouptitle": "Economics and Computation Group", "grouplabel": "ecco", "members": ["Piotr", "Rahul", "Martin"]}
]}
!16
Example using JSON{ "id": "questionnaire-1", "title": "COMP327 Questionnaire", "description": "This is a sample questionnaire", "questions": [ { "name": "question-1", "type": "single-option", "title": "Q 1", "question": "How much have you enjoyed COMP327 practicals so far?", "choices": [ { "label": "not at all", "value": 1 }, { "label": "a little", "value": 2 }, { "label": "somewhat", "value": 3 }, { "label": "quite a lot", "value": 4 }, { "label": "Very much", "value": 5 } ] }, { "name": "question-2", "type": "multi-option", "title": "Q 2", "question": "Which of these COMP327 lectures did you attend?", "choices": [ { "label": "Introduction to COMP327", "value": 1 }, { "label": "Introduction to Core Data", "value": 2 } ] } ]}
Questionnaire:{“id”: String,"title": String,"description": String,”questions": [Array]}
Question:{“name”: String,”type": String,"title": String,"question": String, “choices”:[Array] }
Choices:{“label”: String,”value": Int}
!17
Example using JSONvar theQuestionaire: NSDictionary? = nillet url = URL(string: "https://cgi.csc.liv.ac.uk/~phil/COMP327/questionnaire.json")!let task = URLSession.shared.dataTask(with: url) { (data, response, error) in if error != nil { print(error) } else { if let urlContent = data { do { let jsonResult = try JSONSerialization.jsonObject(with: urlContent, options: JSONSerialization.ReadingOptions.mutableContainers) as AnyObject theQuestionaire = (jsonResult as? NSDictionary) let questionnaireTitle = theQuestionaire!["title"] let questionsArray = theQuestionaire!["questions"] as! NSArray let currentQuestion = questionsArray[0] as! NSDictionary print(currentQuestion["question"] as! String) //print the first question "How much have you enjoyed COMP327 practicals so far?” } catch { print("====\nJSON processing Failed\n=====") } } }}task.resume()
!18
Questionnaire
ID, title, description, questions[ ]
Questions [ ]
[0] [1] [2] …
Questions[0]
name, type, title, question, choices[ ]
choices[ ]
[0] [1] …
choices[0]
label, value
!19
Encoding and Decoding in Swift 4
{"name": "Monalisa Octocat","email": "[email protected]","date": "2011-04-14T16:00:49Z"}
struct Author { let name: String let email: String let date: Date}
struct Author: Codable { let name: String let email: String let date: Date}
!20
Encoding and Decoding in Swift 4
let jsonData = """{"name": "Monalisa Octocat","email": "[email protected]","date": "2011-04-14T16:00:49Z"}""".data(using: .utf8)!
struct Author: Codable { let name: String let email: String let date: Date}
let decoder = JSONDecoder()decoder.dateDecodingStrategy = .iso8601let author = try decoder.decode(Author.self, from: jsonData)
print(author.name)
!21
Encoding and Decoding in Swift 4
{ "url": "https://api.github.com/.../6dcb09", "author": { "name": "Monalisa Octocat", "email": "[email protected]", "date": "2011-04-14T16:00:49Z" }, "message": "Fix all the bugs", “comment_count": 0,}
struct Commit: Codable { let url: URL struct Author: Codable { let name: String let email: String let date: Date } let author: Author let message: String let comment_count: Int}
let decoder = JSONDecoder()decoder.dateDecodingStrategy = .iso8601let commit = try decoder.decode(Commit.self, from: jsonData)
print(commit.author.name)print(commit.url)
let jsonData = """{ "url": "https://api.github.com/.../6dcb09", "author": { "name": "Monalisa Octocat", "email": "[email protected]", "date": "2011-04-14T16:00:49Z" }, "message": "Fix all the bugs", "comment_count": 0,}""".data(using: .utf8)!
!22
Example using JSON// Decodable structures for JSON convertingstruct Questionnaire : Decodable { let id : String? let title : String? let description : String? let questions : [Questions]?}
struct Questions : Decodable { let name : String? let type : String? let title : String? let question : String? let choices: [Choices]?}
struct Choices : Decodable { let label : String? let value : Int?} if let url = Bundle.main.url(forResource: fileName, withExtension: "json") { do { let data = try Data(contentsOf: url) let decoder = JSONDecoder() let questionnaire = try decoder.decode(Questionnaire.self, from: jsonData) print(questionnaire.questions![0].question!) //prints “How much have you enjoyed …” } catch { print("error:\(error)") } }
in Swift 4
let jsonResult = try JSONSerialization.jsonObject(with: urlContent, options: JSONSerialization.ReadingOptions.mutableContainers) as AnyObject theQuestionaire = (jsonResult as? NSDictionary) let questionnaireTitle = theQuestionaire!["title"] let questionsArray = theQuestionaire!["questions"] as! NSArray let currentQuestion = questionsArray[0] as! NSDictionary print(currentQuestion["question"] as! String)
!23
Archiving Objects (Serialisation)
• A serialisation of some object graph that can be saved to disk
• and then be loaded later
• This is what is used by nibs/xibs/Storyboard
• Use the Encodable and Decodable protocols
• Declares two methods that a class must implement so that instances can be encoded or decoded
• Encode an object for an archiver: encode(to:)
• Decode an object from an archive: init(from:)
in Swift 4
!24
Using Web Services
• A lot of data is “in the cloud”• data is available from some web server• data can then be sent back to a web server• A number of APIs available• Google, Flickr, Ebay, etc...• Typically exposed via RESTful services• Uses XML or JSON data formats
• Parsing XML• iOS provides some XML support - e.g. PropertyListDecoder handles a specific format of XML used in
Apple’s plist files.• Several open-source XML parsers available
• JSON is also used by Apple Push Notifications
!25
User Defaults and Settings Bundles• Often applications need to save a small number of settings• e.g. preferences or last used settings
• UserDefaults provides a “registry” for storing values• Generated on a per user / per application basis• Register the different settings• Determines the default values and types of the settings for first use, or if reset
UserDefaults.standard.set("Phil", forKey: "name") //save let nameObject = UserDefaults.standard.object(forKey: "name") //retrieve if let name = nameObject as? String { print(name) } let arr = [1,2,3,4] UserDefaults.standard.set(arr, forKey: "array") //save let arrayObject = UserDefaults.standard.object(forKey: "array") //retrieve if let array = arrayObject as? NSArray { print(array) }
!26
User Defaults and Settings Bundles
• UserDefaults can be set and retrieved at any point in the application• Access the defaults object, and treat as a dictionary
• Values are cached in the application and the OS periodically stores values to the defaults database• (Legacy code may refer to the synchronize() method - you should no longer use this. Let the OS do it)
var lastCell = UserDefaults.standard.object(forKey: "lastSelectedCell")
… //program makes some changes to lastCell here
UserDefaults.standard.set(lastCell, forKey: "lastSelectedCell")
Local Data Storage
Core Data
!28
Core Data• A schema-driven object graph management and persistence framework• Model objects can be saved to the file store...• .. and then retrieved later
• Specifically, Core Data• provides an infrastructure for managing all the changes to your model objects• Provides automatic support for undo and redo
• supports the management of a subset of data in memory at any time• Good for managing memory and keeping the footprint low
• uses a diagrammatic schema to describe your model objects and relationships• Supports default values and value-validation
• maintains disjoint sets of edits on your objects• Can allow the user to edit some of your objects, whilst displaying them unchanged elsewhere
!29
The Core Data Stack
• A collection of Core Data framework objects that access a persistent store• This could be a database, but doesn’t have to be.
• Core Data allows the developer to manage data at the top of this stack• abstracts away the storage mechanism• Allows the developer to focus on• Managed Objects (i.e. records from tables in databases)• Managed Object Context (i.e. work area which manages objects from a Core Data store
Managed Object Model
A collection entity descriptions
Managed Object Store
A collection of managed Objects
Persistent Object Store
A collection of object data
Persistent Store Coordinator
A collection of stores
Store File
!30
Managed Objects
• An object representation of a record from a table in a database• The model object (from the MVC pattern) managed by Core Data• The data used within the application• shapes, lines, groups of elements in a drawing program• artists, tracks, albums in a music database• people and departments in a HR application
• An instance of the NSManagedObject • or a NSManagedObject subclass
!31
Managed Object Contexts
• A context represents a single object space in an application• Responsible for managing a collection of Managed Objects• Managed objects form a group of related model objects that represent an internally consistent view of a data
store• e.g. records and their relationships
• Also responsible for:• the complete life-cycle management• validation of data• relationship maintenance• Undo and Redo of actions
• Instance of an NSManagedObjectContext• or an NSManagedObjectContext subclass
!32
Managed Object Contexts• Managed Objects exist within a Managed Object
Context• New managed objects are inserted into context• Existing records in a database are fetched into a context as
managed objects• Changes to records are kept in memory until they are
committed to the store by saving the context• Includes insertion or deletion of complete objects• Includes manipulation of property values
This managed object context contains two managed objects corresponding to two records in an external database.
Note that Nigel’s salary has been increased, but that the change has not been committed to the database.
Other records exist in the database, but there are no corresponding managed objects.
Managed Object Context
EmployeeName FredSalary 90 000
EmployeeName NigelSalary 60 000
EmployeeName SalaryFred 90 000Julie 97 000Nigel 50 000Tanya 56 000
Unsaved Data
Current Data
!33
Managed Object Model• A Managed Object model represents a schema that describes the
data (and hence the database)• And hence the managed objects in the application
• A model is a collection of entity description objects• Instances of NSEntityDescription• Describes an entity or table in a database, in terms of:• Its name• Name of the class used to describe the entity in the app• Its properties and attributes
Core Data It uses the model to
map between managed objects in your app to records
in the database
Managed ObjectName FredSalary 90 000entityDescription
Entity DescriptionName “Employee”
Managed Object Class NSManagedObjectAttribute name
Attribute salary
Database Table
EmployeeName SalaryFred 90 000Julie 97 000Nigel 50 000Tanya 56 000
!34
Using Core Data: The Basics• Core Data support can be included in several of the project templates• Select “Use Core Data” when creating the project
• Includes additional code within the app delegate implementation file• Manages files in the applications Documents Directory• Provides a persistent store coordinator for the app• Returns the managed object model for the app• Also provides the managed object context for the app
• Also includes a Core Data Model (.xcdatamodeld) file• Known as the managed object model
!35
Getting started with Core Data
• Any use of core data will need access to the Managed Object Context
• Get this from the app delegate
let appDelegate = UIApplication.shared.delegate as! AppDelegate
let context = appDelegate.persistentContainer.viewContext
let request = NSFetchRequest<NSFetchRequestResult>(entityName: “Users")
request.returnsObjectsAsFaults = false
do { let results = try context.fetch(request) if results.count > 0 { for result in results as! [NSManagedObject] { if let username = result.value(forKey: "username") as? String { print(username)
!36
Modelling your data• A data model is a collection of entity and property
description objects that describe the Managed Objects• This model can be described programmatically• Alternatively, the graphical modelling tool can be used
• Different editor styles can be selected• Entity and Attribute properties can be viewed by opening the right-most
pane (the Data Model Inspector)
• Entities can be added to the model• This should be an object of type NSManagedObject
• (see later)
• The entity name doesn’t have to be the same as class that will represent instances of this object
• Attributes can then be defined• complete with name and type
!37
Modelling your data• Here we have a simple data
model - with one entity “Users” and three attributes.
• We can add attributes and define their types. Types available are:• undefined, integer 16, integer
32, integer 64, Decimal, Double, Float, String, Boolean, Date, Binary Data, UUID, URI and Transformable.
• We can also view and edit / interact with the data model graphically
!38
Modelling your data
!39
Fetching objects from the store
Database Table
EmployeeName SalaryFred 90 000Julie 97 000Nigel 50 000Tanya 56 000
Resulting Array
Managed ObjectName JulieSalary 97 000entityDescription
Managed ObjectName FredSalary 90 000entityDescription
Managed Object Store
A collection of managed Objects
Persistent Object Store
A collection of object data
Persistent Store Coordinator
A collection of stores
Fetch Request
Entity (table name) Employee
Predicate (optional) salary > 60 000
SortOrderings (optional) name:ascending alphabetical
To fetch objects you need a managed object context and a fetch request. Often, objects not directly fetched, but that are relevant to it (such as related objects) will also be retrieved.
!40
Creating and Executing a Request
// Create the request let request = NSFetchRequest<NSFetchRequestResult>(entityName: “Events”) request.returnsObjectsAsFaults = false
// Set the sort descriptor request.sortDescriptors?.append(NSSortDescriptor(key: "creationDate", ascending: true))
do { var results = try context.fetch(request) // do stuff here
} catch { print("Couldn't fetch results") }
Create the fetch request, and identify the entity. Provide the name of the entity (“Events”) to the managed context.
Set the sort descriptor; otherwise the order the objects are returned will be undefined. As multiple sort orderings may be specified, these are given in an array.
Execute the request - in this case the results are placed in a mutable array, as the results may be modified within our application. Each element of the array is an NSManagedObject
!41
Deleting Managed Objects
override func tableView(_ tableView: UITableView, commit editingStyle: UITableViewCellEditingStyle, forRowAt indexPath: IndexPath) {
if editingStyle == UITableViewCellEditingStyle.delete { // Delete the managed object at the given index path let itemToRemove = eventsArray[indexPath.row] context.delete(itemToRemove)
// Update the array and table view eventsArray.remove(at: indexPath.row) tableView.deleteRows(at: [indexPath], with: UITableViewRowAnimation.automatic) table.reloadData() //should not be needed - OS should update table
// Commit the change do { try context.save() } catch { print(“changes failed") }
....
In this case, we have received an edit request from the UITableView to delete an entry in the table, that corresponds to an entry we want to delete from our store. Identify the NSManagedObject which is to be deleted. In this case, it is held within the eventsArray, which is also used to fill in entries in the table.
In this app, we also need to delete the entry from our array, and from the table view
Save the context, to push the changes down to the persistent store!
!42
let toInsert = NSEntityDescription.insertNewObject(forEntityName: "ArtWork", into: context!) as! ArtWorktoInsert.artist = artInfo.artisttoInsert.fileName = artInfo.fileNametoInsert.id = Int32(artInfo.id!)!toInsert.info = artInfo.InformationtoInsert.lastModified = stringToDate(artInfo.lastModified!)toInsert.latitude = Double(artInfo.lat!)!toInsert.longtitude = Double(artInfo.long!)!toInsert.location = artInfo.locationtoInsert.locationNotes = artInfo.locationNotestoInsert.title = artInfo.titletoInsert.yearOfWork = artInfo.yearOfWorktoInsert.enabled = Int(artInfo.enabled!)! == 1
!43
import Foundationimport CoreData
@objc(CoreArtwork)public class CoreArtwork: NSManagedObject {
}
import Foundationimport CoreDataextension CoreArtwork { @nonobjc public class func fetchRequest() -> NSFetchRequest<CoreArtwork> { return NSFetchRequest<CoreArtwork>(entityName: "CoreArtwork") } @NSManaged public var artist: String? @NSManaged public var enabled: Bool? @NSManaged public var fileName: String? @NSManaged public var id: Int32? @NSManaged public var info: String? @NSManaged public var lastModified: String? @NSManaged public var latitude: Double @NSManaged public var location: String? @NSManaged public var locationNotes: String? @NSManaged public var longitude: Double? @NSManaged public var title: String? @NSManaged public var yearOfWork: String? }
CoreArtwork+CoreDataClass.swift
CoreArtwork+CoreDataProperties.swift
!44
Modelling your data
!45
Modelling your data
Questions?
Data Persistence and Core Data