NOTES ON NETTY PART 2

RICK HIGHTOWER’S

TRANSPORTS AND BUFFERS

ABOUT RICK HIGHTOWERABOUT RICK

• Implemented Microservices, Vert.x/Netty at massive scale

• Author of QBit, microservices lib and Boon, Json parser and utility lib

• Founder of Mammatus Technology

• Rick’s Twitter, Rick’s LinkedIn, Rick’s Blog, Rick’s Slideshare

Great book on Netty!

https://www.manning.com/books/netty-in-action

Great talk about Netty Best Practices given at Facebook,then Twitter University

https://goo.gl/LbXheq

http://www.infoq.com/presentations/apple-netty

Great talk about Netty at scale

PREVIOUS SLIDE DECKPREVIOUS SLIDE DECK

• Notes on Netty Basics Part 1

• Slideshare

• http://www.slideshare.net/richardhightower/notes-on-netty-baics

• Notes on Netty Basics Part 1

• Google Slides

• https://goo.gl/aUGm2N

CHANNELS AND TRANSPORTS

NETTY TRANSPORTSTRANSPORTS

• OIO - old school Java networking IO lib pre NIO

• (IMO Deprecated)

• NIO - Uses Java Async API

• Epoll - Uses Linux Network libs via Java Native Interface (JNI)

• Local Transport - Async communication same JVM

• Embedded Transport - for unit testing your channel handlers

• UDS - Uses Unix Domain Sockets for same Server Interprocess communication (IPC)

IN THE END IT IS ALL STREAMS OF OCTETSSTREAMS OF BYTES

• Old school blocking API for Java is easy to use but nothing like NIO

• Hard to switch from one to the other

• Netty provides common interface of

• blocking API (Why?),

• Java NIO (works on OS X, Windows, BSD),

• faster Linux epoll NIO,

• UDS, internal JVM, embedded for unit tests

• Same interface easy to switch between implementations

FASTER WHEN YOU DON’T HAVE TO COPY BYTESZERO COPY

• Only available with NIO and epoll

• Moves data from file system to network without copying buffers from user space (your program) and kernel space (Linux, OS X, etc.)

• Works great for HTTP, FTP, anytime you can move files

HIGHLY SCALABLE IO EVENT NOTIFICATIONEPOLL

• Part of Linux Kernel since 2002 (other OS have similar event systems)

• Faster than select/poll system calls

• Java NIO uses epoll and equiv.

• Netty now has highly tuned Linux epoll implementation

• uses corking,

• send file,

• edge limit,

• much less synchronization than Java NIO

WHEN SHOULD YOU USE OIO?OIO

• Probably never

• If doing blocking IO operations (like JDBC) maybe, but…

• You should just use your own thread pools (JDK concurrent) instead or something like Vert.x or QBit

• Vert.x allows pools of workers called worker verticles

• QBit allows pools of service queues

• Both use Netty

TRANSPORTSTRANSPORTS

• NIO

• TCP, UDP, SCTP, UDT (Interprocess communication)

• Epoll Linux only

• TCP, UDP

• OIO (IMO deprecated)

• TCP, UDP, SCTP, UDT

• SCTP is stream message oriented

• When to use SCTP over TCP? Who uses SCTP? 4G LTE, telephony

NETTY BUFFERS

BIG PERFORMANCE GAINS TO BE HADBYTEBUF

• It is all about the bytes

• Performance is reducing byte copies

• Performance is not zero-ing out buffers when not needed

• Performance is reducing the amount of direct buffer creation by pooling buffers

BYTEBUF, BYTEBUFHOLDERBYTE BUF API

• Easier to use API than JDK ByteBuffer

• no flip (index for read, index for write)

• Capacity is expanded on demand like ArrayList, StringBuilder

• Fluent API (method-chaining)

• Performance improvements

• Zero copy by using built-in composite buffers

• Pooling is supported

BYTEBUF, BYTEBUFHOLDERBYTE BUF API

• read…(), skip…() operation increase read index

• write…() operation increase write index

• get…() operations to not update the indexes

• markReaderIndex(), markWriterIndex(), resetReaderIndex(), resetWriterIndex(), writerIndex(index), readerIndex(index) to move indexes around, reset to 0, etc.

• forEachByte(predicate) to find a location in the buffer.

• If you try to read beyond what is written you get an IndexOutOfBoundException

• You can specify max capacity, defaults to MAX_VALUE on Integer

THERE ARE TWO MAIN TYPES OF BYTE BUFFERSTWO TYPES OF BYTE BUFS

• Array backed - backed by a byte[] (byte array)

• Direct buffer

• memory gets allocated off JVM heap

• GOOD: Allows native IO calls that do not have to copy buffers to IO lib

• BAD: More expensive to allocate and release

• You need to copy data into heap to use it, can’t just ask for the array (hasArray() == false, byteBuf.array() throws exception)

AGGREGATED VIEW OF MULTIPLE BYTEBUFSCOMPOSITE BUFFERS

• ByteBuf, CompositeByteBuf

• includes N number of on-heap (array backed) and direct byte buffers

• hasArray() only true if only backed by one on heap instance

• Use addComponent(byteBuf) to combine buffers and then no need to copy buffer contents

• Two areas responsible for different parts of the message but the message sent back to client

• e.g., one area creates header response, one area creates response body.

• Instead of copying buffers use CompositeByteBuf (add headers and then add response body buf then send composite)

VIEW OR SLICE OF AN EXISTING BUFFERDERIVED BUFFERS

• duplicate(), slice(), slice(start, stop), unmodifiableBuffer(…), order(ByteOrder), readSlice(index)

• Return new buffer view with their own indexes and marker indexes

• Changes also change underlying buffer (use copy(), copy(start, stop) if this is not the behavior that you want)

HOLDS A BYTE BUFFER, USED FOR BUFFER REF COUNTINGBYTE BUF HOLDER

• holds a ByteBuf

• content() returns held ByteBuf

• copy() gives a copy of the held ByteBuf

• duplicate() gives a view with unique read/write/mark indexes to the same ByteBuf buffer data as held ByteBuf

ALLOCATING BUFFERS FROM POOLSBYTE BUF ALLOCATORS

• ByteBufAllocator (you get it from the channel.alloc())

• Used to allocate a buffer, buffer(…), heapBuffer(…), directBuffer(…), compositeBuffer(…), ioBuffer(…)

• May create a new buffer or use a pooled buffer

• Netty ByteBuf Allocators use Java version of jemalloc

• see talk cited in into area of this slide deck to learn more about jemalloc

• efficient pools of buffers by allowing threads to have their own cache of buffers and then fallback to larger pool if buffer not available in thread local cache

• reduces overhead, fast, Fast, FAST

• First rule of using pools, if you allocated it, you must free it

• buffer.release()

THANKS AND BUY NORMAN’S BOOK

NETTY IN ACTIONIDEAS FOR SLIDES

• Many ideas for slides are directly derived from Netty in Action book by Norman et al and/or his talks on Netty

• BUY THE BOOK Netty in Action!

• The slides are a study aid for myself so I can better learn Netty

• I’ve worked with ByteBuffer, NIO, Vert.x, and have often wanted to use parts of Netty on projects but lacked the knowledge of Netty internals so used NIO or ByteBuffer or OIO when I really wanted to use Netty instead

PREVIOUS SLIDE DECKPREVIOUS SLIDE DECK

• Notes on Netty Basics Part 1

• Slideshare

• http://www.slideshare.net/richardhightower/notes-on-netty-baics

• Notes on Netty Basics Part 1

• Google Slides

• https://goo.gl/aUGm2N

ABOUT RICK HIGHTOWERABOUT RICK

• Implemented Microservices, Vert.x/Netty at massive scale

• Author of QBit, microservices lib and Boon, Json parser and utility lib

• Founder of Mammatus Technology

• Rick’s Twitter, Rick’s LinkedIn, Rick’s Blog, Rick’s Slideshare

The end… sleepless dev.