Koop: Using 3rd Party Services in ArcGISDaniel Fenton

@dmfenton

dfenton@esri.com

https://github.com/dmfenton

Koop is a bridge

http://arcg.is/2kI8efh

https://www.yelp.com/search

https://washingtondc.craigslist.org/search/apa

Background

• Node.js- JavaScript on the server- Asynchronous i/o- https://nodejs.org

• NPM- Package manager for JavaScript- Npm install- https://npmjs.org- Module.exports = {}

• Geoservices- Open API Specification used by ArcGIS- https://geoservices.github.io

New in Koop 3.0

• Simplified Provider- Route and Controller are Optional- FeatureServer Routes and Handler built in

• Simplified Cache API- Koop-Cache-Memory

• New Plugin Type: Outputs- Handle a set of routes automatically- Can call functions on Koop and Provider model

Output

Core

Cache

Provider

Koop is extensible

• Provider: Translate remote APIs- https://github.com/koopjs/koop-provider-yelp

• Output: Translate GeoJSON into an API standard e.g. Geoservices, SODA, WFS- https://github.com/koopjs/koop-output-geoservices

• Cache: Store and Process GeoJSON- https://github.com/koopjs/koop-cache-memory

• FileSystem: Write files to disk or cloud storage- https://github.com/koopjs/koop-filesystem-s3

• General: extend the Koop API in any way- https://github.com/koopjs/koop-plugin-reverse-geocoder

Providers

• Translate between any remote API and GeoServices• Pass-Through

- Communicate with remote API on every request- Data is too big to gather all at once- And/Or API supports filters and geometry queries- E.g. https://github.com/koopjs/koop-provider-yelp

• Cached- Gather all data at once- Data is small or doesn’t change often- E.g. https://github.com/dmfenton/koop-provider-craigslist

Output

Core

Cache

Provider

Start the server locally

• npm install => install dependencies from npmjs• npm start => activate the server• View on command line with: `curl --silent

http://localhost:8080/sample/FeatureServer/0

Deploy a server

1. Download and install the Now application from Zeit- https://zeit.co/now

2. From the root directory run `now --npm -e KOOP_SERVER_PORT=80`3. That’s it!

Sample ProviderTriMet Bus API to Feature Service

GeoServices Output

Core

Memory Cache

Portland TriMet Bus API

`https://developer.trimet.org/ws/v2/vehicles/onRouteOnly/false/appid/${key}`=>`http://apapters.koopernetes.com/trimet/FeatureServer

How to write a provider

1. Find API docs or Reverse Engineer API via network tab2. Implement `getData` function3. Configure settings4. Launch Server

From: https://washingtondc.craigslist.org/jsonsearch/apa/

To: http://adapters.koopernetes.com/craigslist/washingtondc/apartments/FeatureServer/0

Output

Core

Cache

Provider

getData in detail

• Purpose is to talk to remote API and return geojson• Function takes in a `req` and `callback`

- Req contains parameters `id` and `host- Callback is called with an error or geojson

• Cache time is managed by this function• Example: https://

github.com/dmfenton/koop-craigslist/blob/master/craigslist.js#L7-L20

Output

Core

Cache

Provider

Index.jsDialing in the specifics of your provider

https://gist.github.com/dmfenton/93379d71f4412716b3e508e1c8612cfa Output

Core

Cache

Provider

Advanced ProvidersExtending the routes and functionality of Koop

- Routes- Maps http verbs and paths to handler functions- E.g. https://github.com/koopjs/koop-provider-agol/blob/master/routes.js

- Controller- Handles actions specified in routes- E.g. https://github.com/koopjs/koop-provider-agol/blob/master/controllers/index.js

Output

Core

Cache

Provider

Outputs

• Adds routes to every provider• Works like Express middleware• Can call functions on the provider

- Or Koop• Leverage the cache automatically

Teaching Koop New languages

Output

Core

Cache

Provider

Cache

• Used automatically by Koop- If a TTL is set (Time To Live)

• Avoid rate limiting• Improve response time• Add capabilities not supported by remote API• Use when:

- Data does not change often- Data is too large to request each time

Output

Core

Cache

Provider

Plugins

•Extend Koop’s Core•Can be used by any part of stack•E.g. Queue, Tile Generator, File Exporter

Output

Core

Cache

Provider

Lego Blocks

https://gist.github.com/dmfenton/93379d71f4412716b3e508e1c8612cfa

• Winnow: query geojson without a database- https://github.com/featureserver/winnow

• FeatureServer: given geojson and a route, respond like a feature server would- https://github.com/featureserver/featureserver

Resources

• https://github.com/koopjs/koop• https://github.com/koopjs• https://koopjs.github.io• https://github.com/koopjs/koop-sample-provider

Questions?@dmfentondfenton@esri.comhttps://github.com/dmfenton

Please Take Our Survey on the Esri Events App!

Select the session you attended

Scroll down to find the survey

Complete Answers and Select “Submit”

Download the Esri Events app and find your event