Post on 19-Mar-2017
transcript
Koop: Using 3rd Party Services in ArcGISDaniel Fenton
@dmfenton
dfenton@esri.com
https://github.com/dmfenton
Koop is a bridge
http://arcg.is/2kI8efh
https://www.yelp.com/search
https://washingtondc.craigslist.org/search/apa
Background
• Node.js- JavaScript on the server- Asynchronous i/o- https://nodejs.org
• NPM- Package manager for JavaScript- Npm install- https://npmjs.org- Module.exports = {}
• Geoservices- Open API Specification used by ArcGIS- https://geoservices.github.io
New in Koop 3.0
• Simplified Provider- Route and Controller are Optional- FeatureServer Routes and Handler built in
• Simplified Cache API- Koop-Cache-Memory
• New Plugin Type: Outputs- Handle a set of routes automatically- Can call functions on Koop and Provider model
Output
Core
Cache
Provider
Koop is extensible
• Provider: Translate remote APIs- https://github.com/koopjs/koop-provider-yelp
• Output: Translate GeoJSON into an API standard e.g. Geoservices, SODA, WFS- https://github.com/koopjs/koop-output-geoservices
• Cache: Store and Process GeoJSON- https://github.com/koopjs/koop-cache-memory
• FileSystem: Write files to disk or cloud storage- https://github.com/koopjs/koop-filesystem-s3
• General: extend the Koop API in any way- https://github.com/koopjs/koop-plugin-reverse-geocoder
Providers
• Translate between any remote API and GeoServices• Pass-Through
- Communicate with remote API on every request- Data is too big to gather all at once- And/Or API supports filters and geometry queries- E.g. https://github.com/koopjs/koop-provider-yelp
• Cached- Gather all data at once- Data is small or doesn’t change often- E.g. https://github.com/dmfenton/koop-provider-craigslist
Output
Core
Cache
Provider
Start the server locally
• npm install => install dependencies from npmjs• npm start => activate the server• View on command line with: `curl --silent
http://localhost:8080/sample/FeatureServer/0
Deploy a server
1. Download and install the Now application from Zeit- https://zeit.co/now
2. From the root directory run `now --npm -e KOOP_SERVER_PORT=80`3. That’s it!
Sample ProviderTriMet Bus API to Feature Service
GeoServices Output
Core
Memory Cache
Portland TriMet Bus API
`https://developer.trimet.org/ws/v2/vehicles/onRouteOnly/false/appid/${key}`=>`http://apapters.koopernetes.com/trimet/FeatureServer
How to write a provider
1. Find API docs or Reverse Engineer API via network tab2. Implement `getData` function3. Configure settings4. Launch Server
From: https://washingtondc.craigslist.org/jsonsearch/apa/
To: http://adapters.koopernetes.com/craigslist/washingtondc/apartments/FeatureServer/0
Output
Core
Cache
Provider
getData in detail
• Purpose is to talk to remote API and return geojson• Function takes in a `req` and `callback`
- Req contains parameters `id` and `host- Callback is called with an error or geojson
• Cache time is managed by this function• Example: https://
github.com/dmfenton/koop-craigslist/blob/master/craigslist.js#L7-L20
Output
Core
Cache
Provider
Index.jsDialing in the specifics of your provider
https://gist.github.com/dmfenton/93379d71f4412716b3e508e1c8612cfa Output
Core
Cache
Provider
Advanced ProvidersExtending the routes and functionality of Koop
- Routes- Maps http verbs and paths to handler functions- E.g. https://github.com/koopjs/koop-provider-agol/blob/master/routes.js
- Controller- Handles actions specified in routes- E.g. https://github.com/koopjs/koop-provider-agol/blob/master/controllers/index.js
Output
Core
Cache
Provider
Outputs
• Adds routes to every provider• Works like Express middleware• Can call functions on the provider
- Or Koop• Leverage the cache automatically
Teaching Koop New languages
Output
Core
Cache
Provider
Cache
• Used automatically by Koop- If a TTL is set (Time To Live)
• Avoid rate limiting• Improve response time• Add capabilities not supported by remote API• Use when:
- Data does not change often- Data is too large to request each time
Output
Core
Cache
Provider
Plugins
•Extend Koop’s Core•Can be used by any part of stack•E.g. Queue, Tile Generator, File Exporter
Output
Core
Cache
Provider
Lego Blocks
https://gist.github.com/dmfenton/93379d71f4412716b3e508e1c8612cfa
• Winnow: query geojson without a database- https://github.com/featureserver/winnow
• FeatureServer: given geojson and a route, respond like a feature server would- https://github.com/featureserver/featureserver
Resources
• https://github.com/koopjs/koop• https://github.com/koopjs• https://koopjs.github.io• https://github.com/koopjs/koop-sample-provider
Please Take Our Survey on the Esri Events App!
Select the session you attended
Scroll down to find the survey
Complete Answers and Select “Submit”
Download the Esri Events app and find your event