Nokta Digital Content Solutions

Technical Competency Presentation

Hakan KocakulakCTO of Noktacom Medya İnternet

Teknolojileri

Nokta in Nutshell

In 2013, according to comScore,

we have reached to %69 of

Turkish Internet Users

Nokta in Nutshell

Source; , Media Metrix, Jan-June 2013, Total Number of Internet Users 23.9 Million

Turkey’s leader local video

site176M + monthly video

views1.8M + content

Source; , Video Metrix, July 2013

Turkey’s leader movies site

32M monthly pageviews

More than

40.000 trailers

200920102011201220131.

Source; , Media Metrix, July 2013

Nostalgic video website

13M videos watched per month

22M monthly

pageviews

comScoreEntertainment-Humor

%42Reach and ranks

1st

Source; , Media Metrix, July 2013

Turkey’s leader blogging

website6M monthly unique visitor

37M monthly pageviews

Source; , Media Metrix, July 2013

60M monthly pageviews

comScoreBusiness/Financial Information category

%46Reach and ranks

1st

Source; , Media Metrix, July 2013

1.7M photos

285.000 photographers2M monthly pageviews

Source; , Media Metrix, July 2013

1.7M photos

248.000 photographers7M monthly pageviews

3M monthly pageviews50.000 recipes

Source; , Media Metrix, July 2013

Proven Track Record in Live Digital Events

• Vodafone high school music contest was broadcast live on Izlesene and Twitter at the same time through Izlesene portal. (Izlesene the first player in Turkey to broadcast live on Twittter)

• Can Bonomo concert was broadcast live on Izlesene channel and got in Worldwide & Turkish TT list on Twitter on a soccer night.

NOKTA MOBILE ECOSYSTEM / EMPOWER – GROWING MOBILE AUDIENCES

1.4 M+ 12 M+10 M+ 25:00 min

LOREM IPSUM

15 M+ Monthly Visitors

350M+ Monthly

Video Views

60M+ Monthly Visits

Time Spent 16:00 min

NOKTA WEB ECOSYSTEM

CDN & Private CDN• Why Content Delivery Network? (CDN)

• CDN caches contents to the nearest POP Server• Pulls popular content from origin, caches temporarily & saves

bandwidth, saves cost• Reduces network latency & improves user experience

• Highly Available, when the nearest pop is dead, streams from other pops

• Why Adaptive Stream?• Stream quality is selected dynamically by player• When the bandwidth to the client on any network segment is

saturated, the client reduces the quality and vice versa• Less buffer emptying occurs and improves user experience• Segmented, Less VOD (video on demand) start & seek time• Less storage area for mpeg dash packing

CDN Services & Features• Video & Audio Content Streaming (Both VOD and Live)

• Adaptive Stream, Mpeg Dash (Desktop, Android)• Adaptive Stream, HLS (IOS)• Progressive Stream, mp4 (Legacy Platforms)• RTMP Streaming

• Static Content Caching• Images, css, js and any static content• Real time image resize

• Dynamic web site accelerator• Traffic Management

• DC & Geography Awareness• Carrier sensitivity• Rule based

• Low cost• Secure Links• Transfer Rate Limits• Monitoring• Highly Available• Player

• Flash player, html5 player, ios app player, android player• Can integrate with big data analysis service

Video Transcoding Services (VTS)

• Fast parallel content download from source• Parallel mpeg dash, hls and mp4 transcoding• Configurable resolutions and bitrates, H264 and AAC encodings• Up to 4k videos transcoded• Image capture from video content within configurable intervals• Storyboard image creation• Optional three replica, highly available object storage service for

transcoded videos and images• Optional audio fingerprint service• Fully integrated with CDN service

Big Data Analyzer (BDA)

• Real Time and Batch Analysis• Video views with several breakdowns (platform, category, ownership, etc)• Video completion rate• Player buffer rate• Player connection error rate• Last mile analysis• Page view, visit, ctr, etc.• Any definable metric• Real time comparison with previous day and previous week day• A/B Testing• Data driven solutions (Recommendations, Personalization, Search)

Some Highlights

• 65 Gb/s peak traffic• 150k visit/s at whole Nokta products• 20k video start/min• 1+ billion video start/month• 150k ad view/s• image load time < 100 ms , ad load time < 20 ms• 25 MB/s log, 150 GB/day log, 10k msgs/s • 25 k req/s at load balancer• 5000 Video/day transcoded to 6 different formats• 300+ VM on private cloud• 150+ server• Layer 1, 2, 3 network administration, monitoring and maintenance• 200+ metrics on BDA• 400+ hosts, 10000+ items monitored with 2000+ triggers• Tubitak Research Projects

• Kullanıcı Video İzleme Davranışlarından Profil Tanıma Sistemi• Eşle İndirge ile Bulut Üzerinde Yığın Video İşleme Sistemi

Behind the Scenes, Storage

• Open Source Distributed Object & Block Storage Ceph• Highly available, infinitely scalable, DC & rack aware, data redundancy with

replication• Current infra works on 45 servers• Each server has 12 disk slot and each slot has 3.5 inch 4 TB NL sas disk and

2x2.5 inch 300 GB OS disk• No RAID configuration for data disks, RAID 1 for OS disks• Each server has 1x10 Gb/s, 2x 1 Gb/s, 1x 100 Mb/s interface• Each data has three replica• S3/Swift API for Object Storage, qemu end kernel driver for block storage• Works as object storage for videos, images, statics etc• Works as networked block storage for os & data volumes of virtual servers

working on Private Cloud

Behind the Scenes, Private Cloud

• Based on Open Stack• Current infra works on 20 servers• 2-slot 16-core Xeon servers 64-128G RAM• 1x10 Gb/s, 2x 1 Gb/s, 1x 100 Mb/s network interface• Multi-tenant networking• KVM based OS virtualization• Fast VM provisioning• Cobbler and Puppet for bare-metal provisioning and Configuration

Management

Behind the Scenes, CDN

• 12-14 Gb/s attainable throughput per server for high-cardinality static content. More for viral content or live streams

• Based on nginx with lua (openresty suite), managed in-house• Fast SAS disks for caching• 2x 10Gb/s, 2x1 Gb/s, 2x100 Mb/s network interface• Highly Available • Traffic Management

• DC & Geography Awareness• Carrier sensitivity

Behind the Scenes, Transcoding

• Resource scheduling with Apache Zookeeper• Services have been developed in Python• Whole extensive video & audio formats supported• Content is downloaded and transcoded in parallel.• Deployed on 15 nodes• Configurable resolutions & bitrates & formats, H264 and AAC encodings• Up to 4k videos are transcoded• Images are extracted from video at configurable interval

Behind the Scenes, Big Data Analysis

• 2x 20 server cluster• One cluster for sensitive ads data• One cluster for analytics purpose• HDFS, Hadoop, Hbase, zookeeper, flume, storm, spark, impala, kafka are

used• Services are coded using Scala and Java

Behind the Scenes, Monitoring

• BDA product is used for business metrics• Grafana is used for Visualizing Time Series data with back end Hbase• Zabbix is used for Network and Hosts• OMSA is used for physical host monitoring

Q & A

Hakan Kocakulak, CTO of Noktacom Medya İnternet Teknolojilerihakankocakulak@nokta.com

http://tr.linkedin.com/in/hakankocakulak