External Objects for External Big Data

Sumit SarkarChief Data EvangelistProgress

sumit.sarkar@progress.com@SAsInSumit

Agenda

1. Intro to External Objects for Big Data• What is Salesforce Connect?• What is Big Data?• What is OData?

2. How to access Big Data from Salesforce3. Live Demo (fingers crossed)4. Lessons Learned

External Objects for External Big Data

What is Salesforce Connect?

Salesforce Connect maps Salesforce external objects to data tables in external systems. Instead of copying the data into your organization, Salesforce Connect accesses the data on demand and in real time. The data is never stale, and we access only what you need.

Recommended when:• You have a large amount of data that you don’t

want to copy into your Salesforce organization.• You need small amounts of data at any one

time.• You want real-time access to the latest data.

What is Salesforce Connect?

What is Big Data?

Salesforce uses Big Data technologies within their platform (HBase, Pig, Phoenix, etc) to support technologies such as Einstein or exposed through different APIs such as BigObjects.

However most organizations have their own Big Data technologies to analyze and expose data sets similar in scale or variety…

Don’t ETL Big Data sets into the platform!

What is Big Data?

How my team uses Big Data TechnologyEnhance the Customer Journey

What support resources were consumed for each evaluation?

What other product or solution pages were visited to create smarter targeted campaigns?

What is the success score for leads and contacts evaluating products?

What is OData?

An open protocol to allow the creation and consumption of queryable and interoperable RESTful APIs in a simple and standard way.

OASIS Standard REST API (“SQL for the web”)

Ratified as an OASIS standard February, 2014Operations built on REST principlesUniform URL conventionsSurface metadata in standard way

What is OData?

First member to join OData Technical Committee

OData URL Query Conventions$SELECT /Products?$select=*,Category/Name

/Suppliers?$filter=Address/City eq 'Redmond’

/Products?$orderby=ReleaseDate asc, Rating desc

/Products?$select=*,Category/Name&$expand=Category

$FILTER

$ORDERBY

$EXPAND

/Products?$top=5&$skip=2$TOP $SKIP

http://services.odata.org/OData/OData.svc/Category(1)/Products?$top=2&$orderby=name\__________________________________/ \________________/ \__________________/service root URI resource path query options

How to access Big Data from Salesforce

How to produce OData from Big Data?

Use Salesforce Connect External Objects (intro’d Winter ‘15)

Open Source TechnologiesDataDirect Cloud Apex Connector Framework

Connect Salesforce to Big Data

Success ScoringPersonalizationArchived Insight360 Reporting

Corporate Firewall

Demo

Demo [Lookups, Search, Write, Reporting]

Progress Corporate Firewall

Data Lake

OData

D2C On-premises Connector

HTTPS and UDP firewall traversal

Lessons Learned

Lessons Learned

1. Mapping OData entities to Big Data objects2. Primary keys for Big Data entities3. HiveServer1 vs HiverServer2 for concurrency4. External Objects have limits and 2 minute max timeout5. Native Reporting support being added in Winter ‘176. Search considerations7. Need agile OData service with Data Lake8. Data Governance and Masking9. CRM User Experience (strategies to improve performance)

Accessing external Big Data objects

Design Patterns for external objects Enable Separate Loading of Related Lists of External Objects

Performance Tuning Tips for Related Lists in Account000148978

Decrease latency accessing Big Data over Hive Stuff Salesforce devs don’t care about

1. Use Apache Tez as execution engine for Hive2. Use ORCfile, new storage format3. Use vectorization query execution (Hive 0.13)4. Performance Tuning (Partitions, Indexes, Buckets, Block Sizes, etc)5. Consider another query interface (i.e. Apache Hawq)

ResourcesSubtitle placeholder

Learn More

Salesforce Connecthttps://help.salesforce.com/HTViewHelpDoc?id=platform_connect_about.htmTutorial for SFDevshttp://bit.ly/getxdataDataDirect Cloud OData Service (Hadoop, Oracle, SQL Server, etc)https://www.datadirectcloud.com

Sumit.sarkar@progress.com | www.linkedin.com/in/meetsumit | @SASInSumit

thank y u