Learning a Node stream

2012-01-28kumatch / Yosuke Kumakura

• Yosuke Kumakura (kumatch)

• @kumatch

• Feedtailor inc.

• Video game fun

Agenda

• Summary

• Usage

• Case study

• Stream classes

Stream summary

• An abstract interface Node's I/O

• Controls data stream

• Readable / Writable stream

Case study

How to copy files on Node ?

var fs = require('fs');

var input = './oriainal.jpg';

var output = './copy.jpg';

var data = fs.readFileSync(input);

fs.writeFileSync(output, data);

console.log('copied.');

var fs = require('fs');

var input = './original.jpg';var output = './copy.jpg';

fs.readFile(input, function (err, data) { if (err) throw err;

fs.writeFile(output, data, function (err) { if (err) throw err;

console.log('copied.'); });});

var fs = require('fs');

var readStream = fs.createReadStream('./original.jpg');var writeStream = fs.createWriteStream('./copy.jpg');

readStream.resume();

readStream.on('data', function (buffer) { writeStream.write(buffer);});

readStream.on('end', function () { writeStream.end();});

writeStream.on('close', function () { console.log('copied');});

var fs = require('fs');

var readStream = fs.createReadStream('./original.jpg');var writeStream = fs.createWriteStream('./copy.jpg');

readStream.pipe(writeStream);

writeStream.on('close', function () { console.log('copied');});

Stream usage

• Stream is EventEmmiter

• and has some methods.

Stream usage / Readable stream

• Methods

• resume

• pause

• destroy

Stream usage / Readable stream

• Events

• data

• end

• close

• error

Stream usage /Writable stream

• Methods

• write

• end

• destroy

Stream usage /Writable stream

• Events

• drain

• close

• error

var fs = require('fs');

var readStream = fs.createReadStream('./original.jpg');var writeStream = fs.createWriteStream('./copy.jpg');

readStream.resume();

readStream.on('data', function (buffer) { writeStream.write(buffer);});

readStream.on('end', function () { writeStream.end();});

writeStream.on('close', function () { console.log('copied');});

Stream usage /Stream pipe()

Source stream(readable)

pipeDestination stream

(writable)

Stream usage /Stream pipe()

• destination.write() if source on ‘data’.

• source.pause() if destination buffer is full.

• source.resume() if destination on ‘drain’.

Stream usage /Stream pipe()

• [optional]Keeps the destination stream open.

• Do not destination.end().

var fs = require('fs');

var readStream = fs.createReadStream('./original.jpg');var writeStream = fs.createWriteStream('./copy.jpg');

readStream.pipe(writeStream);

writeStream.on('close', function () { console.log('copied');});

Case study 2

Digest SHA1 hash of a file

Stream classes

• Filesystem (fs)

• readStream (Readable)

• writeStream (Writable)

Stream classes

• Net

• net.Socket (Readable/Writable)

Stream classes

• HTTP

• http.ServerRequest (Readable)

• http.ServerResponse (Writable)

Stream classes

• Zlib

• all classes (Readable/Writable)

• Gzip/Gunzip

• Deflate/Inflate

• DeflateRaw/InflateRaw

Stream classes

fs.Readable

zlib.Gzip

http.response

pipe

pipe

References

• Node manual & documentation

• by Jxck

• http://d.hatena.ne.jp/Jxck/20111204

• A future in stream / Streams2 (Github issue)

• https://github.com/joyent/node/pull/1681