Microservice and Persistent VolumesModel of PV Usage Glenn West

gwest@redhat.com

Microservice Architecture IntroLarge Applications are broken into many microservices.

A microservice is designed to be able to be developed by a very small team

Rapid Development

Self-contained

Well Defined API

Local DB

No Central DB - Dependency issues

Application is intended to degrade slowly with loss of any single microservice

Data and Code should be separated, easy to upgrade restart and replace

Better to duplicate code/data than wait on other

Concepts for large applications (Financial Focus)To Make microservices small and focused, we create a microservice per tenant/data set.

Example, in uThought, while we are working across a whole market (NASDDAQ), a microservice typically will only deal with a single share, or even a single POV of a share.

Genetic Algorithms/Big Data Considerations - Multiple algorithms may be used even on a single POV, futher increases number of microservices.

Dataset size of a single microservice can be quite small. (1 Gigabyte)

Due to the numbers of PV, each PV should be thin-provisioned.

While stateless is best, vast majority of services need some minimal data kept.

Terms and Concepts - POVPOV - Point Of View

Time of Day

End of Quarter

Tax Seasons

First Monday

Second Tuesday

End Of Day

Intended to look at time/spatial legal concepts and causality with stock price

Genetic AlgorithmWe are solving a fuzzy problem, in the real-world you have different people that are domain experts, and you often consult a few in important decisions.

In Deep Learning, there are multiple Deep Learn Algorithms, the best one is highly variable depending on the Data

In NodeJS, there is current 15 of these available.

Best practice, run all, and determine dynamically what is the best for a POV/Equity

Types of DataMariaDB

MongoDB

RockdB

SQLite

Small record size

Fast Access

Generally need fast access - SSD

Time = Money Lost

uThought - Types of ContainersStock Dicovery Service - One Per Exchange / Data ProviderStock History Retrieval Service - One Per EquityStock History Rest - One Per EquityPov Manager Service - One Per EquityPov Spliter Service - One Per EquitySystem KV Store - One Per SystemEquity KV Store - Three Per EquityPov Service Manager - One Per SystemDeep Neural Net - POV x EquityAlorithmProfiler - One Per EquityEquityRanker - One Per SystemEquityUI - One Per EquityBuyEVAL - One Per SystemGlobalUserInterface - One Per Equity

uThought Sizing Base

One Stock Exchange

Small POV Count

Algorithms based on publically available DLL

Contains per Host / Max

PV Analysis

SummaryIts useful to have small PV Volume - to support easy update, without loss of previous state

99% of containers find small PV Storage Useful

In this app, while indidual apps could re-learn, based on replay, startup times get problematic

Storage needs to be close to container cluster

May not need multi-region, depending on how app degrades

Backup of PV to object store needs to be thought thru

Thin Provisioning is important

Automation Automation Automation

ResourcesHigh Level Overview Slides http://www.slideshare.net/glennswest/uthought-executive-overview

LinkedIn Intro to uThought https://www.linkedin.com/pulse/using-containers-docker-change-world-glenn-west?trk=mp-reader-card

uThought Physical Sizing - https://www.linkedin.com/pulse/lots-containers-kubernetes-red-hat-openstack-platform-glenn-west?trk=mp-reader-card