Title of the Presentation Goes Here © 2018 Carnegie Mellon University

QCon 2018 SÃO PAULO, BRASIL

1

Marvin AI - An Open Source Platform to Deploy and Manage Machine Learning ModelsDaniel Takabayashi

QCON 2018

2

About me…

Daniel Takabayashi • Technology Manager and Software Architect @ B2W Digital (São Paulo - Brazil) • Startups Mentor @ Founder Institute (San Francisco - USA) • Startups Mentor @ Aceleradora Cotidiano (Brasilia - Brazil) • Co-Founder of Boolabs, an Brazilian artificial intelligence startup acquired by B2W in 2016 • MSc in Computer Engineering (IPT - Brazil)

and my contacts… daniel.takabayashi@b2wdigital.com @DanTakabayashi linkedin.com/danieltakabayashi github.com/takabayashi

QCON 2018

3

B2W Digital is the leading e-commerce company in Latin America.

QCON 2018

4

Agenda

1.Some Problems in Machine Learning Projects (8 slides)

2.Marvin AI Platform (10 slides)

2.1.Main Components

2.2.Architecture Views

3.Hands on…

5

Some Problems in ML Projects

QCON 2018

6

The knowledge domains to understand, research, build and deploy ML projects are huge and distinct.

Building a “team" with complementary profiles makes the project more expensive.

Some Problems in Machine Learning Projects (1 of 5)

QCON 2018

7

Almost all data scientists (Type A) do not have the necessary software engineer skills to build a production grade solution.

And good Type B professionals are unicorns!

Some Problems in Machine Learning Projects (2 of 5)

QCON 2018

8

The faster the creation process ends, the sooner the improvement process starts.

Rapidly establishing a baseline MVP (around few weeks) is strategic to the project success!

Some Problems in Machine Learning Projects (3 of 5)

QCON 2018

9

ML Teams must save the hypothesis, data, code and metrics for each new iteration of the project.

Reproducibility is always a requirement !!!

Some Problems in Machine Learning Projects (4 of 5)

QCON 2018

10

Code is prototyped locally or in Jupyter notebooks (interactive IDE) in any language.

Models run over test datasets, but are not scalable for production.

Some Problems in Machine Learning Projects (5 of 5)

How to simplify the process of exploring, building, testing and

deploying machine learning projects in a reproducible way?

ABSTRACTION +

STANDARDIZATION

github.com/marvin-ai

14

Marvin AI Platform

QCON 2018

15

Marvin AI Platform: General Infos

• Started at B2W Digital in 2016 to solve internal problems • Released as open source on 09/2017 with Apache 2 licence • First Paper published in Papis.io conference (Boston) on 09/2017 • Three versions released since 09/2017 • Meetup Group created in 01/2018 - https://www.meetup.com/pt-BR/

marvin-ai/ • Community is growing…

QCON 2018

16

Marvin AI Platform: Quality Attributes

For Data Scientists: • Interoperability - to support different programmer languages • Usability - to accelerate and simplify the model creation process

For Administrators: • Manageability - to simplify the distributed deploy/management process • Scalability - to support from tiny to intensive loads

For Marvin Developers: • Modifiability - to improve and release new versions constantly • Maintainability - to allow all type of programmer (from beginners to experts) to

contribute

QCON 2018

17

Marvin AI Platform: Main components (1 of 3)

DASFE* Data Acquisition, Serving, Feedback and Evaluation

QCON 2018

18

Marvin AI Platform: Main components (2 of 3)

Engine - Specific language project that contains source code related to the model.

Implementation of DASFE pattern.

Artefacts - Persistent and versioned binaries (initial dataset, dataset, model, and metrics).

Engine Executor - Architectural abstraction implementation around the Engine such as

parallelism, distribution, versioning, rest apis, availability and so on.

Toolbox - Set of CLI's, utilities, classes and libraries, specific per programming language,

that supports the whole process of exploring, developing, testing and deploying an engine

(Eg. python-toolbox, scala-toolbox, r-toolbox etc).

QCON 2018

19

Marvin AI Platform: Main components (3 of 3)

QCON 2018

20

Marvin AI Platform: Some Architectural TacticsQuality Attributes Main Tactics

InteroperabilityUsing gRPC connections between the EngineExecutor and the UserCode code and a DSL to describe the interfaces.

UsabilityCLI’s with default parameters and Generic Rest APIs to manage and request everything in the system. Marvin defines external and coherent concepts (Eg. Executor, Engine, Action and Toolbox).

ManageabilityA Manager actor to control (locally or remotely) everything in the system and a cluster concept to help in distributed installations.

ScalabilityActor model architecture to increase parallelism and distribution throughout the system. Containerisation as deployment solution.

ModifiabilityEncapsulation through Actor model and base classes, minimum responsibility of each actor and a lot of abstraction.

MaintainabilityScala as implementation language, encapsulation, unit tests and continuous delivery. Virtualized development environment (vagrant and docker).

QCON 2018

21

Context Diagram

Marvin AI Platform: Architecture Views (1 of 4)

QCON 2018

22

Components Diagram 1

Marvin AI Platform: Architecture Views (2 of 4)

QCON 2018

23

Components Diagram 2

Marvin AI Platform: Architecture Views (3 of 4)

QCON 2018

24

Deployment Diagram (draft)

Marvin AI Platform: Architecture Views (4 of 4)

Hands On…

Fork me on Github.com/marvin-ai and fell free to contribute.

Thank you!

twitter.com/_marvin_aigitter.im/marvin-ai