Westy Tracking

#javaland

8 March ‘16

Agenda● About

○ Me○ JDriven

● The case : Westy Tracking○ Background○ Architecture○ Implementation○ Demo

About me● Proud dad of two kids and a '82 VW Westy T3 Joker● Founder and managing partner of JDriven● Love to code, solder and surf● Twitter: @r088

About JDriven● 30+ highly skilled colleagues who love to code● Specialized in all aspects of enterprise development on the JVM and within the

browser● Key players in projects of top 100 organization in the Netherlands as developer,

architect, advisor and coach● Organization focused on optimizing personal growth● Twitter: @jdriven_nl

Before we start

We only have 35 minutes left

Westy Tracking

+ =Westy Tracking Platform● Show actual details● Show trip details● Plot actual position &

trip on map

Westy Tracking : Tracker Specifications● GSM Quad-band Frequency● GPS chipset● Realtime tracking by SMS/GPRS● Overspeed Alarm, ACC anti-theft alarm, SOS alarm● Real-time voice monitoring● Two-Way radio calling● Built-in backup battery to realize power failure alarm● Device lights turn into sleep mode after 5 minutes● Cut vehicle oil or circuit using a relay● GPRS Protocol Specification available (sort of) $25

Tracker Protocol : Westy TrackingLogin:78 78 0D 01 03 53 41 35 32 15 03 62 00 02 2D 06 0D 0A

Login reply:78 78 05 01 00 02 EB 47 0D 0A

Location:78 78 1F 12 0B 08 1D 11 2E 10 CF 02 7A C7 EB 0C 46 58 49 00 14 8F 01 CC 00 28 7D 00 1F B8 00 03 80 81 0D 0A

Location reply:78 78 05 12 00 03 90 3F 0D 0A

Not that well documented ;)22º32.7658’=(22X60+32.7658)X3000=40582974

Reactive

“Classic architecture”

Data ReceiverReceives GPS Data,

DatabasePostGis

Trip DetectorPeriodically Detects

trips

GeocoderPeriodically reverse geocodes locations

Sends tracker data[ TCP ]

Stores data[ JDBC ]

Requests page[ HTTPS ]

Tracker WebsiteShows devices,

locations, trips and map

Requests data[ JDBC ]

Requests and stores data[ JDBC ]

Requests and stores data[ JDBC ]

Westy Tracker Platform system boundary

Requests Geocoding information[ HTTP ]

“Modern architecture”

Data ReceiverReceives GPS Data

StoreRedis

Trip DetectorDetects trips

GeocoderPeriodically reverse geocodes locations

Sends tracker data[ TCP ]

Location[ Event ]

[ HTTPS ]

Tracker WebsiteShows devices,

locations, trips and map

Location[ Event ]

Enrich Location Request[ Event + Response ]

Westy Tracker Platform system boundary

Location[ Event ]

Trip[ Event ]

Enriched Location[ Event ]

Requests Geocoding information[ HTTP ]

Location Enrich.Enriches locations

[ WebSockets (Trip & Location Events) ]

Technologies applied

Vert.x A lightweight, high performance application platform for the JVM

PolyglotJava, JavaScript, Ruby, Python, Groovy, Clojure, Codox, Scala

Simple Small core with APIs for async: HTTP/HTTPS, WebSockets, DNS, FileSystem, Scheduling

ExtensibleLots of additional features available your application platform

Vert.x : Verticles

import io.vertx.core.AbstractVerticle;

public class Server extends AbstractVerticle {

public void start() { vertx.createHttpServer().requestHandler(req -> { req.response() .putHeader("content-type", "text/plain") .end("Hello from Vert.x!"); }).listen(8080); }}

● Vert.x executable: Single file or class● Communicates with other Verticles using the EventBus● Multiple Instances of Verticles can be deployed● Single-threaded, don’t block the event loop (use Worker Verticles instead)

Vert.x : Core● UDP, TCP & HTTP clients and servers● Shared data - local maps and clustered distributed maps● Periodic and delayed actions● Handling Verticle deployment● DNS client● File system access● The Event bus● High availability & Clustering

// Receive the messageeb.consumer("javaland.news", message -> { System.out.println("Update: " + message.body());});

EventBus eb = vertx.eventBus();

// Send a messageeventBus.send("javaland.news", "It’s awsome!");

Vert.x : Core : Eventbus● Async Message Bus● Message payload:

○ Simple, String, Buffers or JSON○ Custom using codecs

● Message patterns:○ Publish / subscribe○ Point to point○ Timeout

● JavaScript Bridge● Distributed● No guarantees

Vert.x : Other extensions● Web

○ Routing, SockJS (event bus bridge), templating● Data access

○ MongoDB, Redis, JDBC, SQL, ...● Integration

○ Mail, Stromp, JCA, ...● Authentication and Authorisation

○ JWT, Shiro, OAuth2, ...● Services● ...

Vert.x : Upgraded to Vert.x 3● Easier to run, deploy and test● Less a container, more embeddable● Introduction of Vert.x Web and other extensions● Java 8 instead of Groovy ● It rules!

Location Enriching Verticle

GeoHash Verticle

Location Persistence

Verticle

Location Verticles

Tracker Protocol Verticle

TrackerPersistence

Verticle

Tracker Verticles

Reverse Geocoding

Verticle

Vert.x : Westy : Verticle Overview

Main Verticle

Web Verticle

Trip DetectionVerticle

TripPersistence

Verticle

Trip Verticles

Tracker Tcp Connection

Verticle

Location Enriching Verticle

GeoHash Verticle

Location Persistence

Verticle

Location Verticles

Tracker Protocol Verticle

TrackerPersistence

Verticle

Tracker Verticles

Reverse Geocoding

Verticle

Vert.x : Westy : Tracker Tcp Connection Verticle

Main Verticle

Web Verticle

Trip DetectionVerticle

TripPersistence

Verticle

Trip Verticles

Tracker Tcp Connection

Verticle

Vert.x : Westy : Tracker Tcp Connection Verticlepublic class TrackerTcpConnectionVerticle extends AbstractVerticle {

public void start() throws Exception { vertx.createNetServer().connectHandler(connection -> { String socketId = connection.writeHandlerID(); connection.handler(buffer -> { JsonObject messageBody = createMessageBody(socketId, buffer); vertx.eventBus().publish("tracker.tcp.in", messageBody); }); }).listen(port, host); }}

Location Enriching Verticle

GeoHash Verticle

Location Persistence

Verticle

Location Verticles

Tracker Protocol Verticle

TrackerPersistence

Verticle

Tracker Verticles

Reverse Geocoding

Verticle

Vert.x : Westy : Location Enriching Verticle

Main Verticle

Web Verticle

Trip DetectionVerticle

TripPersistence

Verticle

Trip Verticles

Tracker Tcp Connection

Verticle

vertx.eventBus().consumer("tracker.location", message -> { ... Future<JsonObject> geoHashFuture = Future.future(); Future<JsonObject> reverseGeoCodingFuture = Future.future();

vertx.eventBus().send("tracker.location.geohash", message.body(), geoHashResult -> { if (geoHashResult.succeeded()) { geoHashFuture.complete((JsonObject) geoHashResult.result().body()); } else { geoHashFuture.fail(geoHashResult.cause()); } });

DeliveryOptions deliveryOptions = new DeliveryOptions().setSendTimeout(TIMEOUT); vertx.eventBus().send("tracker.location.reversegeocode", message.body(), options, rgResult -> { if (rgResult.succeeded()) { reverseGeoCodingFuture.complete((JsonObject) rgResult.result().body()); } else { reverseGeoCodingFuture.fail(rgResult.cause()); } });

});

Vert.x : Westy : Location Enriching Verticle

CompositeFuture.all(geoHashFuture, reverseGeoCodingFuture).setHandler(combinedResult -> { JsonObject enrichedPayload = ((JsonObject)message.body()).copy(); if (geoHashFuture.succeeded()) { enrichedPayload.put("geoHash", geoHashFuture.result().getString("geoHash")); if (reverseGeoCodingFuture.succeeded()) { enrichedPayload.put("location", reverseGeoCodingFuture.result()); } else { logger.warn("ReverseGeocode failed, continue without", reverseGeoCodingFuture.cause()); } vertx.eventBus().publish("tracker.location.enriched", enrichedPayload); } });

Vert.x : Westy : Location Enriching Verticle (2)

Redis : Key Features● Key-value cache and store

○ Optional persistence to disk● More a data structure server, the value can contain different types ● It also supports transactions, publish / subscribe and Time-to-Live ● Clients available for virtually every language

Redis : Data Types● String● List

Collections of string elements sorted according to the order of insertion.● Set

Collections of unique, unsorted string elements.● Sorted Set

Similar to Sets but where every string element is associated to a floating number value, called score.

● HashMaps composed of fields associated with values. Both the field and the value are strings.

Redis : Data Types : String

> SET hello westy

OK

> GET hello

"westy"

Redis : Data Types : List

> LPUSH mylist T # Push at head

(integer) 1

> LPUSH mylist S E W # Push multiple at head

(integer) 4

> RPUSH mylist Y # Push at tail

(integer) 5

> LRANGE mylist 0 -1

1) "W"

2) "E"

3) "S"

4) "T"

5) "Y"

Redis : Data Types : Set

> SADD myset M Y W E S T Y

(integer) 6

> SMEMBERS myset

1) "M"

2) "Y"

3) "E"

4) "T"

5) "S"

6) "W"

Redis : Data Types : Sorted Set

> ZADD transporter:types 1990 "VW T4"

(integer) 1

> ZADD transporter:types 1950 "VW T1"

(integer) 1

> ZADD transporter:types 1979 "VM T3"

(integer) 1

> ZADD transporter:types 1967 "VW T2"

(integer) 1

> ZRANGE transporter:types 0 -1

1) "VW T1"

2) "VW T2"

3) "VM T3"

4) "VW T4"

Redis : Data Types : Hash

> HMSET vw:1000 license aa-bb-cc buildyear 1977 verified 1

OK

> HGET vw:1000 license

"aa-bb-cc"

> HGET vw:1000 buildyear

"1977"

> HGETALL vw:1000

1) "license"

2) "aa-bb-cc"

3) "buildyear"

4) "1977"

5) "verified"

6) "1"

Dependency (Gradle)

compile "io.vertx:vertx-redis-client:${vertx.version}"

Initialization (Verticle)

RedisOptions redisOptions = new RedisOptions(config().getJsonObject("redis"));redisClient = RedisClient.create(vertx, redisOptions);

Configuration (platform.json){ "redis": { "address": "spotwatch.redis", "host": "redis", "port": 6379, "encoding": "UTF-8" } }

Redis : Westy : Include Redis

Redis : Westy : Store location details vertx.eventBus().consumer("tracker.location.enriched", message -> {

JsonObject messageBody = (JsonObject) message.body(); Long receivedAt = messageBody.getLong("receivedAt"); String deviceId = messageBody.getString("deviceId"); // Store location information at tracker:<deviceId>:location:receivedAt redisClient.zadd(getTrackerLocationsKey(deviceId), receivedAt, messageBody.encode(), result -> { message.reply(result.result()); });

// Store received at tracker:<deviceId>:location:geoHash using the geoHash as key redisClient.hset( getTrackerLocationsGeoHashKey(deviceId),

messageBody.getString("geoHash"),String.valueOf(receivedAt), result -> {

message.reply(result.result()); }

); });

AngularJS : Key Features● Declarative HTML approach● Two way Data Binding● MVC/MVVM Design Pattern● Dependency Injection● Routing● Reusable Components: Modules, Services, Directives, …● Built-in end to end Integration Testing / Unit Testing

AngularJS : Westy : Web Verticlepublic class WebVerticle extends AbstractVerticle {

public void start() throws Exception { Router router = Router.router(vertx); router.route().handler(StaticHandler.create(webConfig.getString("webroot")));

SockJSHandler sockJSHandler = SockJSHandler.create(vertx, new SockJSHandlerOptions());

PermittedOptions permitted = new PermittedOptions().setAddressRegex("web\\..+"); sockJSHandler.bridge(new BridgeOptions() .addInboundPermitted(permitted) .addOutboundPermitted(permitted) ); router.route("/eventbus/*").handler(sockJSHandler); HttpServer server = vertx.createHttpServer(); server.requestHandler(router::accept).listen(port, host); }}

AngularJS : Westy : AngularJSangular.module('frontendApp').controller('MainCtrl', function (..., vertxEventBusService) { // Register eventbus system handlers $rootScope.$on('vertx-eventbus.system.connected', function () { console.log('Eventbus connected') $scope.connected = true; requestInitialData(); }); $rootScope.$on('vertx-eventbus.system.disconnected', function () { console.log('Eventbus disconnected'); $scope.connected = false; }); var requestInitialData = function () { vertxEventBusService.send('web.devices.request').then(function (devicesData) { _.each(devicesData, function (device) { $scope.devices[device.id] = device; }); }); };

// Register functionality related handler(s) vertxEventBusService.on('web.device.location', function (location) { $scope.deviceLocations[location.deviceId] = location; });

...

AngularJS : Westy : AngularJS : D3 & Leaflet

D3<nvd3-multi-bar-horizontal-chart data="activeSpeedGraph" showValues="true" objectEquality="true"> <svg></svg></nvd3-multi-bar-horizontal-chart>

Leaflet<leaflet height="600" center="mapCenter" markers="deviceMarkers" paths="tripPaths"></leaflet>

Westy : Last remarks● Focus on dev and ops experience using specialized

tools but fully integrated Gradle build● Vert.x is simple if it fit’s your needs● Vert.x 3 rules, try it!● Not everything is reactive, watch the arrows

Demo

Q & A

Thank you for your attention

You really should experience it!

@jdriven_nl