+ All Categories
Home > Technology > Resumable File Upload API using GridFS and TUS

Resumable File Upload API using GridFS and TUS

Date post: 25-May-2015
Category:
Upload: khangtoh
View: 1,803 times
Download: 4 times
Share this document with a friend
Description:
TUS is a resumable file upload protocol and with MongoDB GridFS, we build an API for uploading files through a REST API and show how to scale this API horizontally using MongoDB as the storage for these files. Singapore MongoDB User Group March Meetup
Popular Tags:
29
MONGO DB FOR CAT LOVERS
Transcript
Page 1: Resumable File Upload API using GridFS and TUS

MONGO DB FOR CAT LOVERS

Page 2: Resumable File Upload API using GridFS and TUS

About me• Developer / Entrepreneur

• Co-Founder / CTO at PicoCandy Singapore and MogiMe Inc, San Francisco :)

Page 3: Resumable File Upload API using GridFS and TUS

What do you do as an entrepreneur?

Page 4: Resumable File Upload API using GridFS and TUS

You think of ideas

Page 5: Resumable File Upload API using GridFS and TUS

You see funny cats pics at work

Sometimes you get bored and google for funny cats at

work

Page 6: Resumable File Upload API using GridFS and TUS

And when you suddenly thought of this awesome idea,

you can’t sleep

Page 7: Resumable File Upload API using GridFS and TUS

What if !!??!!User Generated Funny Cat

Pics and Videos!

Shot using their mobile phone camera!

Using the Catstagram app!

Page 8: Resumable File Upload API using GridFS and TUS

MVP Development 101

• Wrote ruby server code

• Wrote iOS code

• Wrote ruby server code

• Wrote iOS code

Page 9: Resumable File Upload API using GridFS and TUS

Problem

How to do file upload on mobile?

Page 10: Resumable File Upload API using GridFS and TUS

Why File Upload for mobile sucks*

Page 11: Resumable File Upload API using GridFS and TUS

Why File Upload for mobile sucks*

Page 12: Resumable File Upload API using GridFS and TUS

Why File Upload for mobile sucks*

SUCKY SG MOBILE NETWORK

Page 13: Resumable File Upload API using GridFS and TUS

How to make file upload RESUMABLE for mobile?

Page 14: Resumable File Upload API using GridFS and TUS

Then you google like every developer should*

Page 15: Resumable File Upload API using GridFS and TUS

TUS.IO a resumable file upload protocol on top of HTTP

Page 16: Resumable File Upload API using GridFS and TUS

What is the TUS protocol?

• Simple, open and free

• HTTP-based ( POST, HEAD, PATCH )

• Split file into chunks

• Send smaller pieces, server keeps track of chunks / offset

• Web and mobile clients library

Page 17: Resumable File Upload API using GridFS and TUS

What is the TUS protocol?

POST "/files" HTTP/1.1Host: tus.example.orgContent-Length: 0Final-Length: 100

RESPONSE:

HTTP/1.1 201 CreatedLocation: http://tus.example.org/files/1

Page 18: Resumable File Upload API using GridFS and TUS

What is the TUS protocol?

PATCH "/files/1" HTTP/1.1Host: tus.example.orgContent-Length: 70Offset: 0[file data]

RESPONSE:

HTTP/1.1 200 Ok

Page 19: Resumable File Upload API using GridFS and TUS

What is the TUS protocol?

HEAD "/files/1" HTTP/1.1Host: tus.example.org

RESPONSE:

HTTP/1.1 200 OkOffset: 70

Page 20: Resumable File Upload API using GridFS and TUS

What is the TUS protocol?

PATCH "/files/1" HTTP/1.1Host: tus.example.orgContent-Length: 30Offset: 70

[remaining file data]

RESPONSE:

HTTP/1.1 200 Ok

Page 21: Resumable File Upload API using GridFS and TUS

Awesome! What about the servers and the

clients?• tusd - reference server implementation

using Go

• tus-ios-client, native objective-C client library

• tus-jquery, javascript client library

• rubytus, ruby gemhttps://github.com/picocandy/rubytus

Page 22: Resumable File Upload API using GridFS and TUS

Where are the chunks?

• tusd - reference server implementation using Go

• tus-ios-client, native objective-C client library

• tus-jquery, javascript client library

• rubytus, ruby gemhttps://github.com/picocandy/rubytus

Page 23: Resumable File Upload API using GridFS and TUS

Where are the chunks? Take 2.

Page 24: Resumable File Upload API using GridFS and TUS

What is GridFS?

• GridFS is a specification for storing and retrieving files that exceed the BSON-document size limit of 16MB.

• Just like the other collections

• By default, GridFS uses two collections with names prefixed by fs bucket:

• fs.files and fs.chunks

• Use a different bucket name? Yes!Create multiple buckets in a single database? Oh yes!

Page 25: Resumable File Upload API using GridFS and TUS

What is GridFS?

{

"_id" : <ObjectId>,

"length" : <num>,

"chunkSize" : <num>

"uploadDate" : <timestamp>

"md5" : <hash>

"filename" : <string>,

"contentType" : <string>,

"aliases" : <string array>,

"metadata" : <dataObject>,

}

The files Collection - fs.files

Page 26: Resumable File Upload API using GridFS and TUS

What is GridFS?

{

"_id" : <ObjectId>,

"files_id" : <ObjectId>,

"n" : <num>,

"data" : <binary>

}

The chunks Collection - fs.chunks

Page 27: Resumable File Upload API using GridFS and TUS

Mongo Ruby Driverrequire 'mongo'include Mongo

@db = MongoClient.new('localhost').db('picotusd')@grid = Grid.new(@db)

Saving Fileimage = File.open(“cat.jpg")

id_for_cat = @grid.put(image, :filename => “cat.jpg")

Retrieving Filethe_first_cat_image = @grid.get(id_for_cat)

Page 28: Resumable File Upload API using GridFS and TUS

Demo / Code

Page 29: Resumable File Upload API using GridFS and TUS

Questions?


Recommended