© 2015 CY Lin, Columbia UniversityE6895 Advanced Big Data Analytics – Final Project

Presentation

1

E6895 Advanced Big Data Analytics:

Safest Route Prediction in New York City

May 12, 2016

Team Members: Gabriel Thomas (gtm2122), Anubha

Bhargava (ab3955)

© 2015 CY Lin, Columbia UniversityE6895 Advanced Big Data Analytics – Final Project

Presentation

Motivation

● Develop a useful, problem solving tool that displays the safest walking

route in a city

● The application will be designed for those unfamiliar with a city or

uncomfortable walking at night.

The application will…

● Display the safest route on a map interface

● Provide walking directions to the user

© 2015 CY Lin, Columbia UniversityE6895 Advanced Big Data Analytics – Final Project

Presentation

Datasets, Software Languages and APIs Used

Datasets: NYPD Major Felony Incidents Crime Dataset, spotcrime.com

Software Languages and packages: Python, JavaScript, HTML, Spark

APIs: Yelp, imaplib, GoogleMaps, MapQuest

© 2015 CY Lin, Columbia UniversityE6895 Advanced Big Data Analytics – Final Project

Presentation

Front-End Interface

© 2015 CY Lin, Columbia UniversityE6895 Advanced Big Data Analytics – Final Project

Presentation

Visualization Map

© 2015 CY Lin, Columbia UniversityE6895 Advanced Big Data Analytics – Final Project

Presentation

Algorithm

1. Get 24 hour shops from Yelp API

2. Load historical crime datasets and Spotcrime.com data

3. Get circle around the origin and destination

4. Get all the coordinates for 24 hour shops within this area

5. Use Gaussian Mixture Modelling to fit the crime locations

6. Use the mixed Gaussian Multivariate Distribution on the locations of the

24 hour shops to check safety

7. Use these locations as waypoints to plot the route in Google Maps and

provide walking directions

8. Plot the crime data on using a visualization plot on Google Maps

© 2015 CY Lin, Columbia UniversityE6895 Advanced Big Data Analytics – Final Project

Presentation

Safest Walking Route Prediction

© 2015 CY Lin, Columbia UniversityE6895 Advanced Big Data Analytics – Final Project

Presentation

Safest Walking Route Prediction

© 2015 CY Lin, Columbia UniversityE6895 Advanced Big Data Analytics – Final Project

Presentation

Safest Walking Route Prediction

Google Maps Original Route:

© 2015 CY Lin, Columbia UniversityE6895 Advanced Big Data Analytics – Final Project

Presentation

Safest Walking Route Prediction

© 2015 CY Lin, Columbia UniversityE6895 Advanced Big Data Analytics – Final Project

Presentation

Safest Walking Route Prediction

Google Maps Original Route:

© 2015 CY Lin, Columbia UniversityE6895 Advanced Big Data Analytics – Final Project

Presentation

Opportunities for Future Development

● Instead of only using 24 hour shops, using additional waypoints would

aid in determining safety.

○ A more precise technique would be determining the safety at the

coordinates of each leg.

● The current algorithm holds each crime at the same weight, but a

more robust algorithm would hold more dangerous crimes with more

weight.

● Each crime is currently modeled as a independent and identically

distributed Gaussian mixture model.

○ It would be useful to experiment with different mixture models.

© 2015 CY Lin, Columbia UniversityE6895 Advanced Big Data Analytics – Final Project

Presentation

Questions?