+ All Categories
Home > Career > Playback data systems

Playback data systems

Date post: 16-Apr-2017
Category:
Upload: ketan-duvedi
View: 760 times
Download: 1 times
Share this document with a friend
20
Distributed Systems Engineer Playback Data Systems https://jobs.netflix.com/job s/861178 Ketan Duvedi
Transcript
Page 1: Playback data systems

Distributed Systems EngineerPlayback Data Systems

https://jobs.netflix.com/jobs/861178Ketan Duvedi

Page 2: Playback data systems

NetflixThe world’s leading Internet TV network

190+ countries

86+ millionmembers

1500+ devicessupported

125 millionhours / day

Page 3: Playback data systems

High scale

Philip Fisher-Ogden
Sandvine publishes the data every half year, so you should be able to find a 2016 version if you wanted to update this.
Page 4: Playback data systems

Big Global OpportunityGlobal broadband households continue to grow

Page 5: Playback data systems

Innovative CultureValues are what we value

High Performance

Freedom & Responsibility

Context, not Control

Highly Aligned, Loosely Coupled

Pay top of market

Promotions & Development

http://www.slideshare.net/reed2001/culture-1798664

Page 6: Playback data systems

Team Focus

Build and Operate server side infrastructure to collect, process, and provide high quality data about Netflix playback experiences.

Page 7: Playback data systems

Data CollectionViewing Data

Playback events Player QoE Errors

Ingest > 500K events/sec

Philip Fisher-Ogden
minor format suggestion - the 500K/sec might work better at the bottom of the slide, or at least visually set apart from the viewing data. Currently there isn't a clear place your eye is supposed to focus, and the mix of viewing data concept versus ingestion rate is off - it's hard to tell them apart.
Ketan Duvedi
Is this better?
Page 8: Playback data systems

Data CollectionDevice Attributes

Firmware, SDK, UI State changes

Ingest ~ 50K events/sec

Philip Fisher-Ogden
Similar suggestion here as the last slide - consider setting the ingestion rate apart, perhaps below the data definition, maybe in a box or in a different font color.
Ketan Duvedi
[email protected] [email protected] What is the rate of ingestion for DHS?_Assigned to Dhruv Garg_
Dhruv Garg
We get an peak ingestion rate of about a peak of 125k device attributes/sec per second (about 45k peak events per second) to DHS as client traffic. We write about 120K device attribute changes to cassandra (including global replication) - 25k/sec archive and 90k/sec to live cluster
Dhruv Garg
[email protected] - I did include the CL traffic in the above rps. can you post it?_Reassigned to Jinhua Li_
Jinhua Li
The initial rps for CL is less than 1k/s. It could grow to 600k/s for all CL events. I'm planning to cap it at 10k/s with sliding window sampling.
Page 9: Playback data systems

Data Processing

SessionizeSanitize

SynthesizeSummarize

Page 10: Playback data systems

Data Processing - High Volume @ Low Latency

PDS owns 4 of Top 5 Cassandra Clusters at Netflix

Page 11: Playback data systems

Data Providing

What you’ve watchedWhere you’re at

Real time Operational Insights

Page 12: Playback data systems

Data ProvidingData feeds● Events and summaries● Large number of consumers

Personalized Recommendations

Page 13: Playback data systems

DevOps CultureInsights into our systems enables

Operational ExcellenceYou Build It, You Run It

Page 14: Playback data systems

Technology Stack

Page 15: Playback data systems

Big Opportunities

Page 16: Playback data systems

Global Scale

Improve the microservices architecture to ensure linear scalability at sublinear cost.

Page 17: Playback data systems

Data quality

Deliver high quality data despite diversity of quality in input events.

Use data to deliver business functionality

Page 18: Playback data systems

Flexible Processing

Build platform to enable partners to quickly explore new data

transformations and views.

Page 19: Playback data systems

For more info...Details of the team’s work and approaches:● How Netflix knows where you are in House of

Cards● Viewing History System Evolution

Page 20: Playback data systems

Interested?https://jobs.netflix.com/jobs/861178

Ketan Duvedi


Recommended