Date post: | 18-Jan-2018 |
Category: |
Documents |
Upload: | sarah-goodman |
View: | 219 times |
Download: | 0 times |
Wojtowicz Tomasz
Big Data Project
Far Cry’s Ranking System
Big Data ProjectOverview
• General Informations and Target
• Data Input Model
• Tasks Execution
• Comparison of Performances between RDBMS and NoSQL
• Conclusions
Big Data ProjectGeneral Informations and Target
• What is Far Cry’s Ranking System?
• Why Far Cry’s Ranking System?
• Target
VSVSVS
Big Data ProjectData Input Model
• First Data Modellog.stats_date_mapName.txt4000 matches registered
Playername : SenatorrTime : 2392.84228515625Team : blueTeam : blueClass : Support================================================================================Kills : 34Deaths : 47Teamkills : 1Selfkills : 1Flag_activated_kills : 8Headshots :3. . . . .
Big Data ProjectData Input Model
• Second Data ModelGlobalID.txt47000 records registered
[16.01.2015 19:40:28] ea3e5b9daae54b5a8c44d1ab4e841172 81.20.205.136 Senatorr[16.01.2015 19:46:32] 46771fc133ad42098bd0222d51eff5b5 94.226.196.112 Fair Cry[16.01.2015 19:48:50] 77343357167540e8bcea21d96790412c 2.54.19.131 yosf[16.01.2015 19:48:56] 4749f2a60e1d4dbc9f51eccf9f47dd7a 217.249.58.153 bic[16.01.2015 19:56:42] e3fc4f89434443e69eb01a352c82ae17 84.115.154.149 r0ny……….[16.01.2015 20:04:18] e3fc4f89434443e69eb01a352c82ae17 84.115.154.149 r0ny[16.01.2015 20:04:20] d889c6ff9ee04ff49a50537c4104d223 83.30.128.174 xxrewepe[16.01.2015 20:04:23] ea3e5b9daae54b5a8c44d1ab4e841172 81.20.205.136 Senatorr
Big Data ProjectData Input Model
• Second Data Model
GlobalID.txt ----> GlobalIDr.txt47000 records ---> +/- 9000 records
[16.01.2015 19:40:28] ea3e5b9daae54b5a8c44d1ab4e841172 81.20.205.136 Senatorr[16.01.2015 19:46:32] 46771fc133ad42098bd0222d51eff5b5 94.226.196.112 Fair Cry[16.01.2015 19:48:50] 77343357167540e8bcea21d96790412c 2.54.19.131 yosf[16.01.2015 19:48:56] 4749f2a60e1d4dbc9f51eccf9f47dd7a 217.249.58.153 bic[16.01.2015 19:56:42] e3fc4f89434443e69eb01a352c82ae17 84.115.154.149 r0ny……….[16.01.2015 20:04:18] e3fc4f89434443e69eb01a352c82ae17 84.115.154.149 r0ny[16.01.2015 20:04:20] d889c6ff9ee04ff49a50537c4104d223 83.30.128.174 xxrewepe[16.01.2015 20:04:23] ea3e5b9daae54b5a8c44d1ab4e841172 81.20.205.136 Senatorr
Big Data ProjectTasks ExecutionRankingSystem
BigDataNoSQLwhile(true)
Write Log
BigDataSQL
Write Log
1
3
2
while(true)
log.statslog_date_mapName.txtGlobalID.txt / GlobalIDr.txt Elaborate
Update RDBMS/NoSQL double score
int matchint killsint deathsecc
Big Data ProjectTasks Execution
• Updates Visualisation
Big Data ProjectComparison of Performances between RDBMS and NoSQL
• Logs of execution:
OrientDB 2015-07-13T21:16:24.023+02:00Players in database: 0Inserted players :10 in 3326 msInserted players :100 in 6526 msInserted players :500 in 12620 msInserted players :1000 in 25909 msInserted players :2000 in 62460 msProcessed log files: 497Inserted/Updated Players: 2001Ended on: 2015-07-13T21:17:26.498+02:00Time in (ms) / (s): 62475ms / 62sRetrieving Time in (ms) / (s): 141ms / 0s
PostgreSQL 2015-07-13T21:20:24.734+02:00Players in database: 0Inserted players :10 in 8777 msInserted players :100 in 53723 msInserted players :500 in 224043 msInserted players :1000 in 419842 msInserted players :2000 in 750921 msProcessed log files: 497Inserted/Updated Players: 2001Ended on: 2015-07-13T21:32:55.842+02:00Time in (ms) / (s): 751108ms / 751sRetrieving Time in (ms) / (s): 281ms / 0s
Big Data ProjectComparison of Performances between RDBMS and NoSQL
PostgreSQL vs OrientDBwithout MapReduce
Big Data ProjectComparison of Performances between RDBMS and NoSQL
PostgreSQL vs OrientDBwith MapReduce
Big Data ProjectConclusions
Input : 10 players -----> +/- 2 time fasterInput : 100 players ----> +/- 8 time faster
Input : 500 players -----> +/- 17 time fasterRetrieval Time ----------> 1.99 time faster
+/- 75 % less code to write
VS
Big Data Project
Thank You