Scaling a backend for big dataand blockchain environment

1. Introduction (The companies, project and me)

2. Backend challenge

3. Big Data Integration

4. Blockchain Integration

·············· P. 3

··································································· P. 7

································································· P. 17

··························································· P. 21

@ganchix

Head of Blockchain / Backend

Me

ganchix

The companies

Fecha

The project

• Financial platform• Marketplace

• Tokenization• Operate with cryptocurrency

• Liquidity predictive models• Credit scoring• Product recommendation

Backend challenge - The evolution

Dapp

Smart Contract

Ethereum

Backend challenge - The evolution

Backend challenge - The evolution

Dapps problem:

• Noncryptocurrency users• Problem with some integrations• Legal problem

Backend challenge - The evolution

Backend challenge - The evolution

Fecha

Backend challenge - Why microservices?

• Migration of Dapp easier

• Easy to scale

• Polyglot Database and Languages

Why don't use exclusively blockchainwith a database?

1. Spring CloudNetflix and Kubernetes• Easy to learn.• Nice integrations• Spring 5 reactive

2. Docker• Most adopted vendor technology for containers• Well supported

3. Kubernetes• Multi-cloud provider and on-premises data centers• Self-repair and health check capabilities• Auto-scale

Backend challenge - Microservice Architecture Stack

Backend challenge - Microservice Architecture Stack

Backend challenge - Deployment

PFM values generation from user data. Apache Spark + Cassandra

Forecast prediction and regeneration of this modelsApache Spark + Cassandra

Product recommendations based on the economic profile of the user and his real needs. Apache Spark + Cassandra + Neo4j

Credit scoring calculationApache Spark + Cassandra

Big Data Integration - Tasks

• Tasks are hard, needs:• Time• Resources

• Not Real Time is needed.

• Event Driven Architecture.

Big Data Integration - Events

Big Data Integration - How?

Big Data Integration - RabbitMQ vs Kafka

KafkaRabbitMq• RabbitMQ is designed as a general

purpose message broker

• Support existing protocols like AMQP, STOMP, MQTT.

• Finer-grained consistency control/guarantees on a peer-message.

• Complex routing.

• Apache Kafka is designed for high volume publish-subscribe messages and streams, meant to be durable, fast, and scalable.

• Event Sourcing

• Your application needs access to stream history.

• No complex routing.

https://content.pivotal.io/blog/understanding-when-to-use-rabbitmq-or-apache-kafka

• Deployed in Kubernetes.

• Only accessible by NodeJS API.

• All keys are stored in secrets vaults.

• Used for:• Tokenization• Transactions of users

Blockchain Integration - Private Ethereum

Blockchain Integration - Private Ethereum

Blockchain Integration - Ethereum Main Net

Blockchain Integration - Ethereum Main Net

Blockchain Integration - Ethereum Main Net

• We are the owner of the wallets

• We use Infura to connect blockchain

• Used for:• Payment• Transfers

@ganchix

@2getherbank

@bigeeksoftwa

re

Thanks!

ICO is coming!