AWS re:Invent 2016: Relational and NoSQL Databases on AWS: NBC, MarkLogic, and FileMaker...

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

11/29/16

Jorge Garzon, Software Development Manager, AWS Marketplace

Aaron Rosenbaum, Vice President, Product Strategy, MarkLogic

Anand Vaghela, Senior Product Manager, FileMaker

November 2016

DAT205

Relational & NoSQL Databases on AWSNBC, MarkLogic, and FileMaker Perspectives

on Data Management for Enterprise & Consumer Apps

Architecture and

configuration decisions

that drove success with

a large automotive

company

What to Expect from the Session


01Database and data

management from

AWS and Marketplace

Scaling, architecture and

operational decisions that

drove NBC success 02

03 0

AWS MarketplaceDiscover, procure, deploy, and manage software in the cloud

• Easily find +300 data management solutions

• Ready-to-run on AWS

• Quick deployment

• AWS Marketplace cluster & AWS resources*

• Pay for what you use

• Software costs integrated on AWS bill

• AWS Cloud compliance, shared security

responsibilities with software vendors

https://aws.amazon.com/mp/database/

*Selected vendors


Operate your own Database on AWS

IF YOU NEED CONSIDER USING

A relational database services

with minimal administration

Amazon RDS, a managed relational database service that offers a choice of

MySQL, Oracle, SQL Server, PostgresSQL, or Amazon Aurora database

engines, scale compute & storage. Multi-AZ availability, Read, Replace, and

more.

A fast highly scalable

nonrelational database services

Amazon DynamoDB, a managed NoSQL database that offers extremely fast

performance, seamless scalability and reliability, low cost and more.

An easy-to-operate

in-memory cache

Amazon ElastiCache, a managed in-memory cache cloud service that offers a

choice of two in-memory caching engines, Memcached and Redis.

A fast, petabyte-scale

data warehouse

Amazon Redshift, a managed data warehousing service that makes it simple

and cost-effective to efficiently analyze all of your data using your existing

business intelligence tools.

A database you can

manage on your own

Your choice of relational AMIs on Amazon EC2 and EBS that provide scale

compute & storage, complete control over instances, and more.


AWS MarketplaceDatabase sub-categories and data related categories

A1Operational

Data Stories

A2Data

Integration

A3Data

Warehousing

A4Database

performance

optimization

tools

A6Data lake and

analytics

environment

B1Business Intelligence

& Big Data

• Data Analysis & Visualization

• Advanced analytics and machine

learning

B2Application

Development


A5Application and

reporting data

stories

NBC/MarkLogicAaron Rosenbaum

Vice President, Product Strategy

MarkLogic

SWAP 7 + 8

• Semantic search to

power an intelligent

recommendation engine

• Massive scalability and

elasticity to support 40th

anniversary of SNL

• Lower cost and ease-of-

delivery via MarkLogic

and AWS Marketplace


NBCUniversal – SNL40 App40 Years of Comedy Delivered on Saturday Night Live App


NBCUniversal – SNL40 App

Smart Content: Enriched and targeted

content constantly tuned recommendations

Season

34

Episode 4Anne Hathaway

& Killers

Date10/4/08

CharacterMaharelle Sister

EraTalent

Kristen

Wiig

SegmentThe Lawrence

Welk Show

CharacteristicTiny hands

IncludesPart of

Part of

Aired

on

Acted

in

Played

Has

Shows

Acted

in

SegmentThe Lawrence Welk

Show

TalentKristen Wiig

MULTI-SCREEN

INTERACTION

SINGLE DATA LAYER

FOR MANY DEVICES

REAL-TIME UPDATES

AND REPLAY

Lexis Nexis

BBC

Disney

MarkLogic + AWSEntertainment & Media


MarkLogic on AWS Marketplace

LICENCE TYPE SINGLE-INSTANCE CLUSTER

Developer HVM Yes Yes

Essential

EnterpriseHVM Yes Yes

• http://developer.marklogic.com/products/aws#clusters

• http://www.marklogic.com/resources/marklogic-aws/


All Regions

Annual or Per Hour

Traditional Modern

DATABASE, SEARCH, APP SERVICES

HTML / CSS / JAVASCRIPT

VIEW

GENERATION

WEB

SERVICES

DATABASE SEARCH

HTML / CSS / JAVASCRIPT

VIEW

GENERATION

WEB

SERVICES

BUSINESS LOGIC

OBJECT-RELATIONAL MAPPING

Database

Middle Tier

Front End

MarkLogic Scale Out Architecture

• Non-disruptive scale operations

• Rebalancing of content to keep

performance optimal

• REST APIs to expand and

contract clusters

• On-premises or in the cloud


ADD NODES TO ACCOMMODATE GROWTH

Rebalances data

Scale vertically

• Use different instance

types (more CPUs,

more memory)

Scale horizontally

• Add evaluator nodes

• Add data nodes

• Add replica clusters

Scaling Patterns for MarkLogic Clusters


• If your application uses CPU cycles

for “eval” activities

• Snippetting/highlighting

• Semantic queries

• They are simpler to add

• There is no data

• Join the cluster, ready to go

• Consider a separate MarkLogic group

• Suitable for dynamic scaling

Adding E-nodes


• Add “new rings”

• Example: 3 nodes at a time

• Spread replicas within them

• Allow re-balancer to populate new

forests

• Consider tiered storage APIs

• Retire forests when scaling back

Adding D-nodes



Adding Foreign Clusters


HA

Proxy

MarkLogic SNL App Architecture

r3.2xlarge

GP2 EBS

Elastic

Load

Balancer

AWS Availability Zone 1 AWS Availability Zone 2 AWS Availability Zone 3

Elastic

Load

Balancer

VPC

Router

Node.js

Region-wide

Cluster

DNS CDN

Rights Management

ABR Steams

User

Devices

HA

ProxyNode.js

HA

ProxyNode.js

NBC Universal Scaling Strategy


Latency > 4 seconds? Add E-Nodes

Writes > 10 seconds? Route to SQS

CPU over 70%? Add E-Nodes

CPU at 100%? Add D-Nodes

AZ Unavailable, MarkLogic Failover

Results

We've done a

number of popular

products in the

past, but this one is

without any doubt

and by any metric

a breakaway

success.”

Michael Martin, SVP,

NBC Entertainment

Digital -

More than

2 million

downloads2M+

200M200 million

streamed

videos


“

Focus of Planning

Architecture:

• Fenced scope – internal contracts

• Scalable services

• Metrics + monitoring

Run Book:

• Team roles + responsibilities

• Scaling procedures

• Failure responses

• Escalation paths

Best Practices

Lessons Learned

• Scale services, don’t flatten experience

• No failure “cliffs”, degrade smoothly

• Contain unknown user behavior traffic

risks through scope control.

• Scale-out much easier to orchestrate than

scale-up.

• Single region sufficient resiliency/latency

for US-only app.

• 3X your upper traffic prediction – a

successful launch will generate more

traffic


Delivering high-scale dealer facing applicationsAnand Vaghela

Sr. Product Manager

FileMaker

Global Automotive Company/FileMaker

Automaker Journey to the Cloud

• Participated in FileMaker’s “Early Testing” program

• 2-weeks after launch of FileMaker Cloud, successfully deployed 1st

mission critical custom app running on EC2 instance

• Majority of current install-base is on-premises server, now opportunity to

use more FileMaker Cloud based on business needs

• New project underway utilizing FileMaker Cloud running on AWS, based

on success of previous project


Challenge: FileMaker Server (Windows) on AWS

FLMK

Database

Engine

xDBC

ESS

SASE

Admin Server

WebDirect

FileMaker

Server

(Windows)

Server

ConfigurationFLMK database & files &

remote container data

FLMK WebDirect

xDBC Client

FLMK Pro

FLMK Go

Universal Admin

Console

FileMaker Server

Running Windows FileMaker Server on AWS:

Deployment:

• Windows EC2 compute

• Set up security group settings

• Install & configure FileMaker Server

Manageability:

• Software updates

• EC2 upgrade/downgrade

• EBS upgrade


Solution: FileMaker Cloud

Deployment:

• Simplified using AWS

CloudFormation

• Few clicks and fully

deployed and configured

• Simple Email Service (SES)

Separate OS, FileMaker,

Config, & data

• Allows easier scalability of

EC2 and storage

FileMaker CloudFLMK WebDirect

xDBC Client

FLMK Pro

FLMK Go

Cloud Admin

Console

Admin Server

FileMaker

Cloud Server

(Linux)

FLMK Cloud

Robot

Customer database, files &

remote container dataOS & FileMaker

Configuration

AWS Cloud StorageAWS Cloud Storage

xDBC

ESS

SASE

WebDirect

FLMK

Database

Engine


Overcoming Challenges


• Problem: 90-days to deploy a

Server

• Problem: FileMaker WebDirect

is resource hungry, requires

upgrading existing hardware

• Problem: integration with

FileMaker platform

• Now: deployed in 20-mins

running on AWS resulting to

30000% productivity gain

• Now: ability with single-click to

upgrade/downgrade EC2 to

meet business needs

• Now: BYOL allowed use of

existing FileMaker platform and

FileMaker Cloud

• Using AWS CloudFormation • Using Amazon EC2 scalability • Using AWS Marketplace

Demo – FileMaker Cloud on

AWS Marketplace

• Store data in volumes of a provisioned size,

similar to a local disk drive on your physical

server

• Leverage Amazon SES instead of system

administrator setting up independent email

service

• Future to use Amazon S3 for performance,

price, and flexibility

Lessons Learned & Best Practices


Amazon SESAmazon Elastic

Block Storage (EBS)

Summary

• Use AWS Marketplace to accelerate the

evaluation, deployment and launch of

data management and analytics

projects.

• Understand your storage, read/write

throughput requirements

• Follow AWS best practices to architect

and develop your services

• Test your application at scale

• Review and implement a Disaster

Recovery plan

• Enable monitoring using Amazon

CloudWatch

https://aws.amazon.com/mp/database/


Q&A


Thank you!


Remember to complete

your evaluations!


Date post:	06-Jan-2017
Category:	Technology
Upload:	amazon-web-services
View:	169 times
Download:	0 times

AWS re:Invent 2016: Relational and NoSQL Databases on AWS: NBC, MarkLogic, and FileMaker...

Technology