Post on 01-Jun-2020
transcript
Redis in Kenshoo microservicesJesque based delayed priority queue system
Presented by Dima Vizelman
A Redis Use-Case
Performance Aggregation Microservice
So what I am going to speak about?
❏ Brief intro to Analysis App❏ Aggregation server requirements❏ Proposed solution❏ Solution Overview❏ Key features of the provided service
Brief intro to Analysis App
1. Analysis app
Orchestrates advertising across search platforms
Brief intro to Analysis App
2. Performance aggregation microservice
A micro service which performs aggregations of statistics\performance data to be used for analysis.
Aggregation request
Brief intro to Analysis App
Performance aggregation microservice
Aggregation types:
● On demand\user initiated or scheduled ● Lightweight or heavy ● Real time or delayed
Brief intro to Analysis App
❏ Brief intro to Analysis Apps❏ Aggregation server requirements❏ Proposed solution❏ Solution Overview❏ Key features of the provided service
● 500 Analysis Apps
● 2500 scheduled aggregations per App
● 200 user requests every 5 seconds
Aggregation server requirements
● UX
● Failure Handling
● Work in a cluster
Aggregation Server Requirements
❏ Brief Intro to Analysis Apps❏ Aggregation Server Requirements❏ Proposed Solution❏ Solution Overview❏ Key Features of the Provided Service
Proposed Solution
● Priority Queues
● Delayed Jobs
● Distributed workers
Proposed Solution
● Simple
● Fast
● Robust
Jesque - why?
❏ Brief Intro to Analysis Apps❏ Aggregation Server Requirements❏ Proposed Solution❏ Solution Overview❏ Key Features of the Provided Service
Solution Overview
❏ Brief Intro to Analysis Apps❏ Aggregation Server Requirements❏ Proposed Solution❏ Solution Overview❏ Key Features of the Provided Service
Priority Queue
● No Priority Jobs
● High/Low Priority Queues
● Workers vs Queues
Priority Queue
Why delayed jobs ?
Delayed Jobs
Delayed Jobs
❏ Brief Intro to Analysis Apps❏ Aggregation Server Requirements❏ Proposed Solution❏ Solution Overview❏ Key Features of the Provided Service❏ Bonus features
Disaster Recovery
1. Distributed cache
2. Persistency for disaster recovery information
Q&A
Thank You!