Event Logging System for Collaborative Communities...Django REST framework is a powerful and...

Indian Institute of Technology,Bombay

Summer Internship 2018Ekalavya

Event Logging System forCollaborative Communities

Principal InvestigatorProf. D. B. Phatak

Project In-ChargeMr. Nagesh Karmali

Authors:

Mr. Divyansh SharmaMr. Kartik VermaMr. Rahul RajMr. Vishrut Jetly

Project Mentor:

Ms. Minali Upreti

July 4, 2018

Summer Internship 2018 ProjectApproval Certificate

Department of Computer Science and EngineeringIndian Institute of Technology, Bombay

Certificate

The project entitled Event Logging System for Collaborative Communitiessubmitted by Mr. Divyansh Sharma, Mr. Kartik Verma, Mr. Rahul Raj andMr. Vishrut Jetly is approved for Summer Internship 2018 programme from15th May 2018 to 6th July 2018, at Department of Computer Science and En-gineering, IIT Bombay

Prof. Deepak B. PhatakDept of CSE, IIT-BPrincipal Investigator

Mr. Nagesh KarmaliDept of CSE, IIT-B

Project In-charge

Date: July 4, 2018Place: IIT-Bombay, Powai

Declaration

I declare that this written submission represents my ideas in my own wordsand where others’ ideas or words have been included, I have adequately citedand referenced the original sources. I also declare that I have adhered to allprinciples of academic honesty and integrity and have not misrepresented orfabricated or falsified any idea/data/fact/source in my submission. I under-stand that any violation of the above will be cause for disciplinary action by theinstitute and can also evoke penal action from the sources which have thus notbeen properly cited or from whom proper permission has not been taken whenneeded.

Divyansh SharmaNIT-Hamirpur

Kartik VermaNIT-Hamirpur

Rahul RajNIT-Patna

Vishrut JetlyDA-IICT

Date:

Acknowledgement

We, the summer interns of the Fundamental Research Group - Event LoggingSystem for Collaborative Communities are overwhelmed in all humbleness andgratefulness to acknowledge our deep gratitude to all those who have helpedus put our ideas to perfection and have assigned tasks well above the level ofsimplicity and into something concrete and unique. We wholeheartedly thankProf. D.B. Phatak for having faith in us, selecting us to be a part of his valuableproject and for constantly motivating us to do better. We thank Mr. NageshKarmali for providing us the opportunity to work on this project. We are alsovery thankful to our mentor Ms. Minali Upreti for her valuable suggestions. Shewas and is always there to show us the right track when needed help. With helpof her brilliant guidance and encouragement, we all were able to complete ourtasks properly and were up to the mark in all the tasks assigned. During theprocess, we got a chance to see the stronger side of our technical and nontechnicalaspects and also strengthen our concepts. Last but not the least, we sincerelythank all our other colleagues working in different projects under Prof. D.B.Phatak for helping us evolve better with their advice.

Abstract

Event Logging System for Collaborative Communities is a project underFundamental Research Group aimed to log the events of different activitiesdone by the user in the Collaborative Communities System. There are differentactivities such as when a user (a) views a community, (b) creates a resource, (c)changes the state of a resource, (d) views a resource, etc. For these activities,events are generated that are needed to be logged in some database, which areagain used as a dataset by the Recommendation System.

The Django Middleware is used to catch the data from the event triggered bythe user. This emitted event sends a http request to the server. The middlewarehelps in collecting the meta data, user specific data, etc and create the usefuldata that is needed to be stored. This project makes use of ELK stack (Elastic-search, Logstash and Kibana) to log and store the events. The Logstash filtersthe data and sends this data to the Elasticsearch. Elasticsearch then storesthe data in its database that can be queried later by using REST APIs. TheKibana is used to give better understanding of the event logs by enabling theuser to create custom visualizations. These visualizations can be in the form ofbar graph, pie chart, line chart, JSON format, etc. Token authorization is usedto secure the data stored in Elasticsearch.

The logs which are stored in the ELK stack, can be used for analytics. Theseare then queried to obtain the meaningful data required in the visualization fordifferent users. The graphs are made by using C3.js framework, a library injavascript. It is used to show the view timelines of article and communities, toparticles in the communities and trending articles in community. User specificdashboard includes the status of articles, most viewed articles and recentlyviewed articles.

CONTENTS Event Logging System

Contents1 Introduction 4

1.1 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.2 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 Technologies Used 52.1 ELK Stack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.1.1 Elasticsearch . . . . . . . . . . . . . . . . . . . . . . . . . 52.1.2 Logstash . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.1.3 Kibana . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.2 C3.js . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.3 Django REST Framework . . . . . . . . . . . . . . . . . . . . . . 62.4 Django Middleware . . . . . . . . . . . . . . . . . . . . . . . . . . 6

3 Installation 73.1 Installing Event Logging module . . . . . . . . . . . . . . . . . . 7

4 Event Logging Module 94.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94.2 Event Logging Module Architecture . . . . . . . . . . . . . . . . 104.3 Naming Convention for Events . . . . . . . . . . . . . . . . . . . 114.4 Log Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124.5 Adding New Events . . . . . . . . . . . . . . . . . . . . . . . . . 174.6 Editing Configuration Variables in .env . . . . . . . . . . . . . . . 18

5 ELK Stack 205.1 Logstash . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

5.1.1 Configuration of Logstash . . . . . . . . . . . . . . . . . . 205.1.2 Add new plugin to Logstash . . . . . . . . . . . . . . . . . 20

5.2 Kibana . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205.2.1 Configuration of Kibana . . . . . . . . . . . . . . . . . . . 215.2.2 Defining Index Patterns: . . . . . . . . . . . . . . . . . . . 215.2.3 Creating Visualizations . . . . . . . . . . . . . . . . . . . 225.2.4 Creating Queries . . . . . . . . . . . . . . . . . . . . . . . 22

6 Event Logging REST API 246.1 Working . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246.2 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266.3 Token Authentication . . . . . . . . . . . . . . . . . . . . . . . . 27

6.3.1 Working . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276.3.2 Generating Token . . . . . . . . . . . . . . . . . . . . . . 27

6.4 Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276.5 REST API Endpoints . . . . . . . . . . . . . . . . . . . . . . . . 286.6 GET Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . 316.7 Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

6.7.1 Fixed Fields. . . . . . . . . . . . . . . . . . . . . . . . . . 34

1

CONTENTS Event Logging System

6.7.2 Optional Fields . . . . . . . . . . . . . . . . . . . . . . . . 356.7.3 Error Output . . . . . . . . . . . . . . . . . . . . . . . . . 356.7.4 Sample Call . . . . . . . . . . . . . . . . . . . . . . . . . . 356.7.5 Error Codes . . . . . . . . . . . . . . . . . . . . . . . . . . 40

7 Analytics 417.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417.2 Working . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417.3 Dashboards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

8 Selenium Testing 448.1 Test Case 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 448.2 Test Case 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

9 Future Scope 45

10 Bibliography 46

2

LIST OF TABLES Event Logging System

List of Figures1 Middleware Working . . . . . . . . . . . . . . . . . . . . . . . . . 62 Module Flow Diagram . . . . . . . . . . . . . . . . . . . . . . . . 93 Event Logging Module Work Flow Diagram . . . . . . . . . . . . 104 Creating Index patterns . . . . . . . . . . . . . . . . . . . . . . . 215 Configuring Settings . . . . . . . . . . . . . . . . . . . . . . . . . 216 Sample Visualization . . . . . . . . . . . . . . . . . . . . . . . . . 227 Flow Diagram of REST API . . . . . . . . . . . . . . . . . . . . . 248 Article View Timeline . . . . . . . . . . . . . . . . . . . . . . . . 419 Community View Timeline . . . . . . . . . . . . . . . . . . . . . 4210 Most Viewed Articles in a Community . . . . . . . . . . . . . . 4211 Current State of articles created by the user . . . . . . . . . . . . 4312 Most Viewed articles created by the user . . . . . . . . . . . . . . 43

List of Tables1 Events Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 Parameter Endpoint 1 . . . . . . . . . . . . . . . . . . . . . . . . 283 Parameter Endpoint 2 . . . . . . . . . . . . . . . . . . . . . . . . 284 Parameter Endpoint 3 . . . . . . . . . . . . . . . . . . . . . . . . 295 Parameter Endpoint 4 . . . . . . . . . . . . . . . . . . . . . . . . 296 Parameter Endpoint 5 . . . . . . . . . . . . . . . . . . . . . . . . 307 Paging Parameter Table . . . . . . . . . . . . . . . . . . . . . . . 318 Time Range Parameter Table . . . . . . . . . . . . . . . . . . . . 319 Filter Parameter Table . . . . . . . . . . . . . . . . . . . . . . . . 3210 Field Parameter Table . . . . . . . . . . . . . . . . . . . . . . . . 3211 Sort Parameter Table . . . . . . . . . . . . . . . . . . . . . . . . 3312 Aggregation Parameter Table . . . . . . . . . . . . . . . . . . . . 3413 Error Code of API . . . . . . . . . . . . . . . . . . . . . . . . . . 40

3

1 INTRODUCTION Event Logging System

1 Introduction

1.1 PurposeThe purpose of the event logging system is to log all the activities done by theusers of the Collaborative Communities system. Different types of events aregenerated in the system based on the user activities e.g. when a user views acommunity/group, creates/edits an article, writes a comment, and so on. Theseevents will be logged and used to create a view history for recommendationsystem and to display analytics to the users. Events could be emitted by theserver or the browser and to capture the information about interaction of theuser with the Collaborative Communities system this system is needed.

The logs which are created is then used for analysing and then visualizing inthe forms of graphs. It shows the results in the graphical way which is moreappealing than the descriptive way. The different graphs like bar graph, pie-chart, timeline graphs, etc could be used for the detail analysis. The recentactivities of the users could also be listed. It also helps the user to reviewthe recent activities done by him/her so that it could be useful in planningthe future work properly. The top activities like top articles in the communitycould also be shown. The idea behind showing the trending articles is that itincreases the interest of the authors for creating new articles. The visualizationhelps in finding the top communities, trending articles and communities, etc.It gives the overall idea of popularity of the communities. It could also help ingrouping of alike views and behaviour together. The alike minded users couldcome together and work collaboratively to achieve the common goal.

1.2 ScopeThe event logging system is designed in a way that provides easy integration withthe Collaborative Communities’ main source code. The event logging systemis designed as a separate module and mainly focuses on the logging of the userevents for the Collaborative Communities system. So, for now the scope of theevent logging module is limited to the Collaborative Communities system butwith some modification it could easily be expanded to any other Django basedproject.

The analytics part is designed in such a way that it could get easily integratewith the Collaborative Communities’ main source code providing high cohesionand less coupling and thus following the software principles. The graphs anddashboard are the separate module from the main system and focuses on visu-alizing and analysing the logs in the dashboard of the user. The scope of thispart is that it may be used in other Django project in providing analytics likefeatures. The only dependency is that it uses ELK Stack to capture and storethe logs.

4

2 TECHNOLOGIES USED Event Logging System

2 Technologies Used

2.1 ELK Stack"ELK" is the acronym for three open source projects: Elasticsearch, Logstash,

and Kibana. Elasticsearch is a search and analytics engine. Logstash is aserver-side data processing pipeline that ingests data from multiple sources si-multaneously, transforms it, and then sends it to a "stash" like Elasticsearch.Kibana lets users visualize data stored in Elasticsearch in the form of charts andgraphs.

2.1.1 Elasticsearch

Elasticsearch [3] is a highly scalable open-source full-text search and analyticsengine. It allows you to store, search, and analyse big volumes of data quicklyand in near real time. It is generally used as the underlying engine/technologythat powers applications that have complex search features and requirements. InCollaboration communities we are using Elasticsearch to store the logs and Elas-ticsearch’s DSL query system to query the Elasticsearch to extract the data.

2.1.2 Logstash

Logstash [5] is an open source data collection engine with real-time pipeliningcapabilities. Logstash can dynamically unify data from disparate sources andnormalize the data into destinations of your choice. Cleanse and democratize allyour data for diverse advanced downstream analytics and visualization use cases.

In Collaboration Communities we are using Logstash’s http-plugin. The logsare sent as http requests to the Logstash and using Logstash pipeline they arepassed to the Elasticsearch for storage.

2.1.3 Kibana

Kibana [4] is an open source analytics and visualization platform designed towork with Elasticsearch. User can use Kibana to search, view, and interactwith data stored in Elasticsearch indices. User can easily perform advanceddata analysis and visualize data in a variety of charts, tables, and maps.

Kibana makes it easy to understand large volumes of data. Its simple, browser-based interface enables you to quickly create and share dynamic dashboardsthat display changes to Elasticsearch queries in real time.

2.2 C3.jsC3 [1] is a JavaScript library which builds on top of famous D3 JavaScriptlibrary. C3 makes it easy to generate D3-based charts by wrapping the coderequired to construct the entire chart. It gives classes to each element when

5

2 TECHNOLOGIES USED Event Logging System

generating the charts, allowing the users to create custom styles. It providesvariety of APIs, using which the charts can be updated even after it’s rendered.

2.3 Django REST FrameworkDjango REST framework is a powerful and flexible toolkit for building WebAPIs. It’s a way to expose parts of the application to third-parties (externalapplications and Websites).

In our system it is data-oriented, in a sense that the Web service (the RESTfulAPI), simply make available the information that is stored in the Elasticsearchusing a common format, JSON. This way, an external application can interactwith the data stored in Elasticsearch, without having to connect directly intothe Elasticsearch.

2.4 Django Middleware[6] Middleware is a framework of hooks into Django’s request/response process-ing. It’s a light, low-level “plugin” system for globally altering Django’s inputor output. Each middleware component is responsible for doing some specificfunction. For example, Django includes a middleware component, Authentica-tionMiddleware, that associates users with requests using sessions.

Figure 1: Middleware Working

6

3 INSTALLATION Event Logging System

3 Installation

3.1 Installing Event Logging module1. Open a new Terminal by pressing Ctrl + Alt + T

2. Installing Docker and docker-compose Follow the instructions in thebelow link to install docker and docker-compose (see error handling sectionfor alternative installation)

• Docker Installation: https://docs.docker.com/install/linux/docker-ce/ubuntu/#set-up-the-repository

• Docker-compose Installation: https://docs.docker.com/compose/install/

3. Installing Event log module.Change the present working directory tohome by typing following command

$ cd

4. Make a new directory by typing the following command

$ mkdir work

5. Write the following command to change to work directory

$ cd work

6. To clone the Collaboration System repository write following command

1 $ git clonehttps://github.com/convosyn/Collaboration-System.git↪→

7. Move to the Collaboration System folder by typing

1 $ cd Collaboration-System

8. Open .env and add/replace the following line

LOG_TYPE=STORE

9. Follow steps in README.md of Collaboration System to make sure ev-erything is working fine

10. Once everything is fine, clone the event logging module by typing thefollowing command

1 $ git clonehttps://github.com/convosyn/Collaboration-System-Event-Logs.git↪→

11. Type the following command in terminal

1 $ cd Collaboration-System-Event-logs/evelog-Docker

7

https://docs.docker.com/install/linux/docker-ce/ubuntu/#set-up-the-repository

https://docs.docker.com/install/linux/docker-ce/ubuntu/#set-up-the-repository

https://docs.docker.com/compose/install/

3 INSTALLATION Event Logging System

12. Run the following command to build the docker file

1 $ sudo docker-compose build

13. Run the docker-compose file by typing following command

1 $ sudo docker-compose up

14. Change to root folder of the Collaboration System

15. Open .env file and do the following changes:

(a) To store the logs in Elasticsearch add/replace the following line

1 LOG_TYPE=TOSERVER

(b) Add/Replace the following line to add the ip-address of logstash

LOG_ADDRESS=<ip-address of logstash>

Note: use address of Logstash docker-container in case of docker

16. Follow the steps in section 6.2 on REST API Configuration to configureREST API.

17. Run the following command to start the system

$ python3 manage.py runserver

8

4 EVENT LOGGING MODULE Event Logging System

4 Event Logging Module

4.1 OverviewThis project of event logging system for collaboration communities can bebroadly classified into three main parts as follow.

• Events Capturing and logging in ELK Stack.

• An API to get data from the ELK Stack.

• Provide visualizations to the users using the logged events.

The event logging module uses the middleware of the Django to process theHTTP requests and develops the log’s structure using its internal functionality.The captured event logs are sent to Elasticsearch for storage using the Logstashpipeline. An API has been developed to get the logs from the Elasticsearch.This API uses Elasticsearch’s DSL Query mechanism to query the Elasticsearchand sends the response back to the client.

Figure 2: Module Flow Diagram

Using these logs the trending articles of a community, recent activities of a user,the popularity of a community or an article is provided to the users using theC3.js visualizations.

9


4.2 Event Logging Module Architecture

Figure 3: Event Logging Module Work Flow Diagram

10


The Event logging module architecture is developed keeping in mind thatthere should be minimum coupling with the Collaboration-System’s main webapp. The events are captured using the middleware which hooks into Django’srequest/response processing. Event logging module has its own customized mid-dleware which processes only the specified requests and builds the log’s structureaccording to the format specified in the module. This module processes theserequests on the separate threads in order to optimize the log processing.

After capturing the requests, the events are identified by matching the urlswith the regular expression specified in urls.py in the eventlog module. In thisway the associated event is identified from the corresponding HTTP requests.

The next stage is of the log structure formation. The logs are formed as apython dictionary. Every event log has some common fields and some eventspecific fields which are nested in the main structure. The common fields arethe fields which are same for every event and consists of information like times-tamp, ip-address, http-agent, session etc. On the other hand, event specificfields change according to the type of the event. The event logging module uti-lizes the power of the various functions defined in the logprocess.py to populatethe event specific fields. The logprocess.py is a collection of various functionswhich are used to populate the various fields. This module can be easily ex-tended to add new functionality according to the future needs.

The next stage is passing these logs to the permanent storage. This moduleprovides the various options to store the logs. The logs can be stored in logfiles, in ELK Stack, in the databases and even in the cloud. At present themodule provides the functionality to store the logs in the log files and in theELK Stack but this functionality could be extended to store these logs in thedatabases, clouds etc. In order to specify the storage type make the change tothe storage type variable defined in the .env file. At present logs are send to theLogstash via the HTTP requests.

The Logstash takes the input from the HTTP request sent by the event loggingmodule. This is done by the http input plugin of the Logstash. The Logstashwill specify the index name. Index is like a database in a relational database. Ithas a mapping which defines multiple types. These logs are then stored in theElasticsearch, which is a full-text, distributed NoSQL database.

4.3 Naming Convention for Events[2] The module follows a very general naming convention for the events as definedbelow:

event.param1.param2

param1: specifies, with whom the event is associated with. e.g. whether it isrelated to community, article, user or to something else.

11


param2: specifies usually the type of the events e.g. view, edit, update, loginetc. To understand this convention consider a situation where a user views aparticular community then that event will be named as ‘event.community.view’.Similarly when an article is being edited then that event will be named as‘event.article.edited’.

4.4 Log StructureThe logs follows the JSON nested document structure. There are some commonfields which are common to all the events and some event specific fields whichare nested in the main structure and changes according to the type of the events.

Common fieldsThe fields which are common to all the events are as follows

1. User Agent: Browser agent string of the user who triggered the event.

2. ip-address IP address of the user who triggered the event.

3. Server-host: The site visited by the user.

4. referer:The URI from the HTTP Referer request-header.

5. accept-language:The value from the HTTP Accept-Language request-header field.

6. session-id:This 32-character value is a key that identifies the user’s ses-sion.

7. path-info: The URL path of the page visited by user.

8. time-stamp:Gives the UTC time at which the event was emitted in‘YYYY-MM-DD hh:mm:ss’ format.

9. event-source: Specifies the source of the interaction that triggered theevent.

10. user-id: This value is a key that identifies the user

11. event name: This specifies the name of the event

12. event: This is a dictionary(in python), or a nested JSON object thatcontains some event specific details

Event Specific FieldsThe following tables shows the fields are are specific to the specific events.

12


Table 1: Events Table

Events List

SrNo. Event Name Specific field

1event.article.create

Event triggered when a usercreates an article

• article-title: This value is aname that identifies the arti-cle.

• community-id: This value isa key that identifies the com-munity.

• group-id: This value is a keythat identifies the group.

2event.article.view

Event triggered when a userviews an article

• article-id: This value is a keythat identifies the article.


• group-id: This value is a keythat identifies the group.

• article-status This value is aname that identifies the cur-rent article state

3event.article.edit

Event triggered when a useredits an article


13


4event.article.statusChangedEvent triggered when a userchanges the state of an article


• status: This value is the newstate of the article

5event.article.publishedEvent triggered when a user

publishes an article


• publisher-id: This value isa key that identifies the pub-lisher.

6event.community.viewEvent triggered when a user

views a community


7event.community.subscribeEvent triggered when a usersubscribes a community


8event.community.unsubscribe

Event triggered when a userunsubscribes a community


9event.user.login

Event triggered when a user logsinto the account

• Username: This value isa name that identifies thelogged in user

14


10event.user.logout

Event triggered when a user logsout of the account

• user-id: This value is a keythat identifies the user whologged out

11event.profile.view

Event triggered when a userviews the profile

• user-visited: This value isa key that identifies the userwho visited other’s profile

12event.group.view

Event triggered when a userviews a group

• community-name: Thisvalue is a name that identifiesthe community.

13event.group.unsubscribeEvent triggered when a user

unsubscribes a group• group-id: This value is a key

that identifies the group.

14event.community.createEvent triggered when a user

creates a community

• community-name: Thisvalue is a name that identifiesthe community.

• admin-username: Thisvalue is the username ofadmin of the newly formedcommunity.

15event.group.create

Event triggered when a usercreates a group


• group-name: This value is aname that identifies the newlyformed group.

15


16event.group.manageEvent triggered when a user

manages a group

• invitation-username: Thisvalue is a username that iden-tifies the user to whom invita-tion link was sent.

• role: This value is the namethat identifies the role in thegroup in the invitation link

• status: This is a value thatidentifies whether the userwas added or removed

17event.content.view

Event triggered when a userviews the community’s content


18event.course.create

Event triggered when a usercreates a course


• course-name: This value isthe name that identifies thenewly created course.

19event.course.view

Event triggered when a userviews a course

• course-id: This value is a keythat identifies the course

20event.course.edit

Event triggered when a useredits a course

• user-id: This value is a keythat identifies the user

16


21event.course.manage Event

triggered when a user manages acourse

• user-id: This value is a keythat identifies the user

22event.course.update Event

triggered when a user updates acourse

• course-name: This value isa name that identifies thecourse

23event.comment.post Eventtriggered when a user posts a

comment

• reply-to-user-id: This valueis a key that identifies the userto whom reply was sent.

• object-pk: This value is akey that identifies the com-ment

4.5 Adding New EventsAdding new events to this module is very simple. Its a three step process asinstructed below. One just has to follow these instructions to register new eventsin the module.

Steps• Do the url mapping with event name in urls.py. Specify the type of therequest e.g. whether it is POST, GET request.

• Specify the event specific fields in the settings.py by defining a dictio-nary where key is event name as defined in url.py and value would be adictionary consisting of event specific fields.

• Make the use of functions defined in logprocess.py to populate these fields.If one needs some new functionality which is not currently provided bylogprocess.py that could be easily added to logprocess.py by just defininga new function.

ExampleConsider a situation where we want to log the event for a situation when a userview a community for that we first defined the event name corresponding to the

17


request’s url and the type of the requests in the urls.py as shown below

r'^community-view/(?P<pk>\d+)/$' : {'GET' : {

'event_name' : 'event.community.view'}

},

Now specify the this event’s specific fields in settings.py as shown below. Makethe use of the functions defined in the logprocess.py to populate these fields.logprocess.py is a rich collections of functions providing most of the neededfunctions. Even if, one need more functionality one can easily add new functionin logprocess.py

"event.community.view": {"community-id": logprocess.process_community_info,

},

where process_community.info is a function defined in logprocess.py module.Following these simple steps one can easily add new events to the module

4.6 Editing Configuration Variables in .env1. LOG_TYPE Specifies whether to store the logs in a file or in elasticsearch.

Possible values are:

(a) STORE Stores the logs in a file

(b) TOSERVER Stores the logs in elasticsearch

2. LOG_PROTOCOL Specifies whether the protocol used is http, https, or ftp.Default is http.

3. LOG_ADDRESS Specifies the IP address of logstash.

4. LOG_PORT Specifies the port on which logstash is running.

5. LOG_STORE if LOG_TYPE is set to STORE, then it specifies the name of filewhere logs will be stored.

6. LOG_USE_PROXY Specifies whether the proxy is to be used or not. If set totrue following variables must be added in environment file

• Proxy Configuration. Set the following three variables

(a) http_proxy. Format is http://address:port/(b) https_proxy. Format is https://address:port/(c) ftp_proxy. Format is ftp://address:port/

18


• Proxy Authentication. Set the following variables

(a) PROXY_USER. username for proxy(b) PROXY_PASS. password for proxy

7. URL_BASIC. Specifies the url on which the system is running. For instance,http://localhost:9000/

8. PAGE_SIZE. Specifies the number of results returned per request to Eventlogging system

9. MAX_PAGE_SIZE. Specifies the max number of results that can be returnedper request to Event logging system

10. EVENTAPI_TOKEN_USERNAME. Specifies the username used to generate theauthentication token for use with eventlog api.

11. EVENTAPI_TOKEN_PASSWORD. Specifies the password used to generate theauthentication token for use with eventlog api.

12. EVENT_API_TOKEN. Specifies the token used for authentication with event-log api.

19

5 ELK STACK Event Logging System

5 ELK StackThe event log system makes use of the ELK stack to store the logs. To configurethe ELK follow the following steps.

5.1 LogstashAfter generation of logs, they are sent to the Logstash. We are making the use ofLogstash’s http input plugin to get the logs via the http requests. The logs arethen passed through the Logstash’s pipeline and then sent to the Elasticsearchfor the storage.

5.1.1 Configuration of Logstash

To configure the Logstash make the required changes in the logstash.conf file inthe pipeline folder

input {http {

port => 5000}

}## Add the filtersoutput {

elasticsearch {hosts => "elasticsearch:9200"index => "logs"

}}

• To change the Logstash listening port, change the port defined in http ofinput block. New input method can also be specified in the input block.

• One can add new filters in the pipeline.

• To change the index name and address of Elasticsearch, make changes atrespective positions in output block. New output methods can also beadded here.

5.1.2 Add new plugin to Logstash

One can add new plugins in Dockerfile of Logstash as instructed below

# Add your logstash plugins setup hereRUN logstash-plugin install logstash-input-http

5.2 KibanaKibana is used to display the visualizations to the admin.

20


5.2.1 Configuration of Kibana

One can change the ip-address of the Elasticsearch by specifying the same inthe kibana.yml file in config folder of Kibana.

elasticsearch.url: http://elasticsearch:9200

5.2.2 Defining Index Patterns:

Index patterns tell Kibana which Elasticsearch indices the user want to explore.

1. In Kibana, open Management, and then click Index Patterns.

2. If this is your first index pattern, the Create index pattern page opensautomatically. Otherwise, click Create index pattern in the upper left.

3. Enter logs* in the Index pattern field as shown in Figure 4.

Figure 4: Creating Index patterns

4. Click Next step.

Figure 5: Configuring Settings

21


5. In Configure settings, select "I don’t want to use the time filter" from thedrop-down menu as shown in Figure 5.

6. Click Create index pattern.

5.2.3 Creating Visualizations

To create visualizations

1. Click on Visualize in the side navigation.

2. Click the Create new visualization button or the + button.

3. Choose the visualization type.

4. Select the index pattern.

5. Select the split slices option and aggregate based on Terms.

Figure 6: Sample Visualization

6. Choose the desired field and generate the visualization.

7. Save this visualization and add it to the dashboard from the Dashboardtab.

5.2.4 Creating Queries

To create a query to obtain data from the elasticsearch.

1. Click on Dev Tools in the side navigation.

2. In the console create the desired query according to the syntax. An ex-ample to obtain all the article view events is shown below

22


GET _search/{

"query": {"bool": {

"must": [{"match": {"event_name":"event.article.view"}}

]}

}}

23

6 EVENT LOGGING REST API Event Logging System

6 Event Logging REST APIThe Rest API for event logging system is provided to retrieve the logs stored inthe Elasticsearch. The Rest API for the event logging provides a wrapper overthe Elasticsearch search API. The API has been designed to provide a rich set offeatures and allows customizations so that client can get the exact informationrequired.

6.1 Working

Figure 7: Flow Diagram of REST API

As specified previously, the REST API provides a wrapper over the Elastic-search search API for retrieving data. The Elasticsearch uses Apache Lucence[8] syntax parser for searching which allows Elasticsearch to provide an easy touse syntax and a rich set of features for searching.

Our API uses the python bindings for Elasticsearch to query Elasticsearchusing the DSL Query language (which uses Apache Lucence [8]). It basicallyuses the Elasticsearch module (referred as Elasticsearch Client in figure). Itcan be installed using the pip package manager as follows:

$ pip3 install elasticsearch

Our API, on receiving the properly autheticated requests from client pro-cesses the request and extracts any GET parameters present in the request.The functions present in eventlog/api/helpers.py does this job of extractingvarious parameters and forms a properly structured body for this request.

24


Body of features extracted The structure is primarily a python dictio-nary which contains nested dictionaries as specified below:

1. request_keys It is a dictionary of parameters passed in url which definesthe url called. These don’t include the GET parameters. These are alwayspresent in the body. The fields in this dictionary vary depending upon url.

2. time_range It is a dictionary of GET parameters which defines the rangeof time over which the logs are required. These are always present in thebody. The two fields in this are:

• before

• after

3. paging It is a dictionary of GET parameters passed in url which definesthe page settings. These are always present in the body. The fields presentare:

• size

• from

4. fields It is a list of GET parameters passed in url which defines the fieldsto return. These are always present in the body.

5. filter_keys It is a dictionary of GET parameters in the form of key-valuepair passed in url which defines the filters to apply while searching. Thesemay not be present in the body. The fields may vary depending upon theneeds.

6. sort_keys It is a dictionary of GET parameters passed in url which de-fines the fields on which sorting is to be applied. These may not be presentin the body. The fields present are varying.

7. agg_keys It is a dictionary of GET parameters passed in url which definesthe aggregations/groupings to apply and fields on which these aggregationsshould be applied. These may not be present in the body. The fieldspresent are varying.

Afterwards, the body is passed to eventlog/api/essearch.py module whichactually interacts with the Elasticsearch and retrieves the logs. Before, we canuse the essearch.py we have to define the ip and port(default is 9200) ofElasticsearch.

The results returned are then passed back to the user in JSON Format.(Seesection on RestAPI Output to see the format of output returned).

25


6.2 ConfigurationBefore Rest API can be used, some configuration is required as described

below:

1. Authentication. The Rest API uses Token Authentication for authen-ticating the API requests. For now, a fixed token is generated and isstored as an environment variable. The graphs and Dashboard moduleuse this token to send request to eventlog api and will also be used by therecommendation System. To generate and use token do the following:

(a) change to the root folder of Collaboration-System

(b) run following command to add a user

$ vim .env

Add/Replace following lines in this file

EVENTAPI_TOKEN_USERNAME=eventloguserEVENTAPI_TOKEN_PASSWORD=eventlogpass

(c) Run the following command from root folder of the CollaborationSystem to generate a new Token

$ python3 manage.py generateToken --n

(d) Open .env as specified in step (b) and add/replace the following line

EVENT_API_TOKEN=<token-generated>

put the token generated in step (c) in place of <token-generated>

(e) Add this token in Headers in the following format:

'Authorization': 'Token <token-value>'

Note.Perform these changes in .env.docker file, if you are using dockerand run the corresponding docker commands.

2. Connecting to elasticsearch. Make sure elasticsearch python mod-ule is installed on your system before using REST API(as mentioned insection 6.1). For the api to work, it has to talk to Elasticsearch. So, theAPI needs to know the ip address and port, where the Elasticsearch isrunning. To specify the address and port open the .env(.env.dockerusing docker) file and add/replace the following line:

ELASTICSEARCH_ADDRESS=<address-of-elasticsearch>

or, if you want to explicitly specify the port, you can type as follows:

ELASTICSEARCH_ADDRESS=<address:port>

Note. Specify the address of the Elasticsearch container if you are usingdocker.

26


6.3 Token AuthenticationEvent Logging Rest API uses Token Authentication. The Token generated isa fixed token, which is based on the Django User model. The logic for thegenerating the token is stored in eventlog/management/commands/.

6.3.1 Working

When a new token is to be generated, a Django User is created, which uses thesame user model as used by the Collaboration Communities System. The nameand password of the user are specified in .env as explained in the section 5.2.The token is then generated for this user and should be stored in .env so thatcan be accessed.

6.3.2 Generating Token

• To generate a new token run the following command from root folder ofCollaboration System

$ python3 manage.py generateToken --n

• To renew token for the given user run the following command from rootfolder of Collaboration System

$ python3 manage.py generateToken --r

• To get the current token run the following command from root folder ofCollaboration System

$ python3 manage.py generateToken --g

In case of docker, use the corresponding docker commands.

6.4 FormatThe restapi has two parts:

1. RestAPI end points e.g

/logapi/event/community/view/1

2. GET Parameters. These parameters provide the clients a more bettercontrol over the results e.g. you can specify the specific fields that arerequired or the filters that you may want to apply.Example call:

/logapi/event/community/view/1/?fields=event_name&fields=user-id

27


6.5 REST API Endpoints1. GET /logapi/user/{id}/

Parameters.

Table 2: Parameter Endpoint 1

Parameter Description type

idRepresents the id of the particularuser for which the data is required int

Output. It fetches all the logs for a given user.Example. /logapi/user/1/ fetches all the records of user with id 1.

2. GET /logapi/event/{param1}/{param2}/

Parameters.



param1Represents either community, article,group as specified in the format of an

event namestring

param2can be one of the CRUD events e.g.

view, edit etc. string

Output. It fetches all the logs for the event name event.param1.param2and for all users.Example. /logapi/event/community/view/ fetches all the community-view events for all users.

28


3. GET /logapi/event/{param1}/{param2}/{eid}/

Parameters.




event namestring



eidspecifies a community-id, article-id,group-id specific to param1 etc. int

Output. returns the logs of type event.param1.param2 for a particularresource, community, etc. specified by eid for all users.Example. /logapi/event/community/view/1/ fetches all the community-view events for community with id 1 and for all users.

4. GET /logapi/user/{id}/event/{param1}/{param2}/

Parameters.





event namestring



Output. It returns logs of all the events of type event.param1.param2

29


for user idExample. /logapi/user/1/event/community/view/ fetches all the community-view events by user with id 1.

5. GET /logapi/user/{id}/event/{param1}/{param2}/{eid}/

Parameters.





event namestring



eidspecifies a community-id, article-id,group-id specific to param1 etc. int

Output. returns the logs of type event.param1.param2 for a particularresource, community, etc. specified by eid for all users.Example. /logapi/user/1/event/community/view/1/ fetches the community-view events for user id 1 and community id 1.

Note some events like event.user.login won’t have a id field so for those eventsyou can query them using queries of type 2 and then filter the result as required.

30


6.6 GET Parameters1. Paging Parameters

Table 7: Paging Parameter Table

Parameter Required/optional

DefaultValue Description

start required 0 Specifies the offset for list inresult

limit required 1000 Specifies the number ofresults to return per request

Note Maximum limit for page size is 10000 which can be changed in .envfile

Example

http://localhost:7000/logapi/user/1/?start=0&limit=5

It returns the list of five events starting from index 0.

2. Time range Parameters

Table 8: Time Range Parameter Table

Parameter Required/optional


before required currentTime

Specifies that the eventtimestamp must be less thanor equal to the time in before

after required

10 daysbeforethe timein before

Specifies that the eventtimestamp must be greaterthan or equal to the time in

after

Note Format for time is: YYYY-MM-DDThh:mm:ss

Example

31


http://localhost:7000/logapi/user/1/?before=2018-07-15T10:10:10

It returns all the events that have a timestamp before 2018-07-15 10:10:10and after is automatically set to 2018-07-05 10:10:10, You can set it(after)to a value similar as for before

3. Filter Parameters

Table 9: Filter Parameter Table

ParameterFormat

Required/optional


{nameOfFi-eld}={va-

lue}required current

Time

SSpecifies that the eventtimestamp must be less thanor equal to the time in before

Note nameOfField is one of all the possible fields in eventlog data struc-ture,value is the specific value that is to filtered.Format for time is: YYYY-MM-DDThh:mm:ss

Example

http://localhost:7000/logapi/user/1/?article-id=1&event_name=event.article.edit

It returns all the events of user 1 within the 10 days from now where theuser has edited article with id 1

4. Fields Parameters

Table 10: Field Parameter Table

ParameterFormat

Required/optional


fields optional None

Specifies fields that arerequired in the search result.It is a multivalued list type

attribute

32


Note nameOfField is one of all the possible fields in eventlog data struc-ture,

Example

http://localhost:7000/logapi/user/1/?fields=event_name&fields=event&fields=user-id

It returns all the events of user 1 within the 10 days from now where thefields returned are event_name, event structure as specified in section 9.1and user-id

5. Sort Parameters

Table 11: Sort Parameter Table

ParameterFormat

Required/optional


sorts Required -times-tamp

Specifies fields on which tosort, It is a multivalued list

parameter

Note You can use the following with sort:

• Descending order To specify descending order use ‘-’ as prefix tothe field name(see the example)

• Ascending order To specify ascending order just use the name ofthe field

Example

http://localhost:7000/logapi/user/1/?sort=-event_name&sort=ipaddress

It returns number of events of user 1 within the 10 days from now wherethe sort is in ascending order for ipaddress and descending order forevent_name.

33


6. Aggregation parameters

Table 12: Aggregation Parameter Table

ParameterFormat

Required/optional


agg_type optional None Specifies type of aggregation

agg_field optional None

Specifies name of field(fromeventlog data structure) onwhich aggregation is to be

applied

Note agg_type is one of the following aggregation types:

• cardinality. use it to return distinct count i.e. number of uniqueentries for that field

• value_count. returns the total number of values present in thatfield

• terms. use it to get the distinct values for the given field(See Outputsection for a sample output)

Example

http://localhost:7000/logapi/user/1/?agg_type=value_count&agg_field=event_name

It returns number of events of user 1 within the 10 days from now.

NOTE

• you must follow the format agg_type={aggregation_name}&agg_field=aggregation_field otherwise the query may return Index-Error

• The multiple and nested aggregations are not supported yet.(See sectionon status codes for the errors)

6.7 Output6.7.1 Fixed Fields.

If everything goes well you will see the following three fields always:

1. status code: should be 200

34


2. total hits: total number of hits

3. result: contains a list of json objects which contain actual logs

6.7.2 Optional Fields

1. next_link: if all the records don’t fit on the same page you may get a linkto fetch the next set of records

2. prev_link: if there are records before the currently fetched records youmay see this link. But the limit will be same as the one sent by user incurrent link. So there may be some overlapping

6.7.3 Error Output

1. error: It is a dictionary that contains the status code and error message.The fields present are:

(a) status code: provides the status code returned

(b) error message: provide information about the error

6.7.4 Sample Call

1. A sample api call on a local host.

$ curl -X GET http://localhost:7000/logapi/user/2/

1 {2 "status code": 200,3 "total hits": 1,4 "result": [5 {6 "time-stamp": "2018-06-22 17:24:47",7 "referer":

"http://localhost:8000/login/?next=/community-view/1",↪→

8 "headers": {9 "http_accept": "*/*",

10 "content_type": "application/json",11 "http_accept_encoding": "gzip, deflate",12 "http_version": "HTTP/1.1",13 "request_path": "/",14 "http_connection": "keep-alive",15 "request_method": "PUT",16 "http_host": "logstash:5000",17 "request_uri": "/",18 "content_length": "516",

35


19 "http_user_agent": "Mozilla/5.0 (X11; Linuxx86_64) AppleWebKit/537.36 (KHTML, likeGecko) Chrome/66.0.3359.181Safari/537.36"

↪→

↪→

↪→

20 },21 "accept-language": "en-US,en;q=0.9",22 "user-id": 2,23 "path-info": "/community-view/1/",24 "server-host": "98db49baef58",25 "ip-address": "172.18.0.1",26 "@timestamp": "2018-06-22T17:24:47.532Z",27 "host": "172.18.0.6",28 "@version": "1",29 "event_name": "event.community.view",30 "event": {31 "community-id": "1"32 },33 "event-source": "server",34 "user-agent": "Mozilla/5.0 (X11; Linux x86_64)

AppleWebKit/537.36 (KHTML, like Gecko)Chrome/67.0.3396.62 Safari/537.36",

↪→

↪→

35 "session-id": "epotylin9n4k7xkhj4k8mmvxtr37hiph"36 }37 ]38 }

2. A sample api call on a local host using GET parameters.

1 $ curl -X GET http://localhost:7000/logapi/user/3/?sort=-event_name&limit=2↪→

1 {2 "next_link": "http://localhost:8000/logapi/user/3/?

sort=-event_name&limit=2&start=2",↪→

3 "status code": 200,4 "total hits": 5,5 "result": [6 {7 "time-stamp": "2018-06-22 17:40:42",8 "referer":

"http://localhost:8000/login/?next=/community-view/2/",↪→

9 "headers": {10 "http_accept": "*/*",11 "content_type": "application/json",12 "http_accept_encoding": "gzip, deflate",13 "http_version": "HTTP/1.1",

36


14 "request_path": "/",15 "http_connection": "keep-alive",16 "request_method": "PUT",17 "http_host": "logstash:5000",18 "request_uri": "/",19 "content_length": "517",20 "http_user_agent": "Mozilla/5.0 (X11; Linux

x86_64) AppleWebKit/537.36 (KHTML, likeGecko) Chrome/66.0.3359.181Safari/537.36"

↪→

↪→

↪→



↪→

↪→

36 "session-id": "os6x0mzgxd1z3nmkcf2lbmhngzoseztp"37 },38 {39 "time-stamp": "2018-06-22 17:40:47",40 "referer":

"http://localhost:8000/community-view/2/",↪→

41 "headers": {42 "http_accept": "*/*",43 "content_type": "application/json",44 "http_accept_encoding": "gzip, deflate",45 "http_version": "HTTP/1.1",46 "request_path": "/",47 "http_connection": "keep-alive",48 "request_method": "PUT",49 "http_host": "logstash:5000",50 "request_uri": "/",51 "content_length": "504",

37


52 "http_user_agent": "Mozilla/5.0 (X11; Linuxx86_64) AppleWebKit/537.36 (KHTML, likeGecko) Chrome/66.0.3359.181Safari/537.36"

↪→

↪→

↪→



↪→

↪→

68 "session-id": "os6x0mzgxd1z3nmkcf2lbmhngzoseztp"69 }70 ]71 }

3. Aggregations

(a) terms aggregation Getting the total article edits for all articles bydifferent users in the order of decreasing order

1 $ curl -X GEThttp://localhost:8000/logapi/event/article/edit/?agg_type=terms&agg_field=user-id

↪→

↪→

{"status code": 200,"total hits": 4,"result": [

{"key": "5","doc_count": 3

},{

"key": "3","doc_count": 2

},{

38



},{


}]

}

(b) cardinality aggregation Getting the count of distinct events by auser with id 2

1 curl -X GET http://localhost:8000/logapi/user/2/?agg_type=cardinality&agg_field=event_name↪→

1 {2 "status code": 200,3 "total hits": 1,4 "result": [5 {6 "value": 77 }8 ]9 }

(c) value_count aggregation Getting total count of events by userwith id 2

1 curl -X GET http://localhost:8000/logapi/user/2/?agg_type=value_count&agg_field=event_name↪→

1 {2 "status code": 200,3 "total hits": 1,4 "result": [5 {6 "value": 127 }8 ]9 }

39


6.7.5 Error Codes

Table 13: Error Code of API

StatusCode Description

200 Everything is OK

400 time not in format yyyy-mm-ddThh:mm:ss

400 aggregate type and aggregate fields are of different lengths

400 Improper Configuration Error

400 Connection Error

400 Error Raised by Elasticsearch

404 Not Found Error

409 Conflict Error

500 Database Error

500 Request Error

501 multiple aggregation and nested aggregation not imple-mented

40

7 ANALYTICS Event Logging System

7 Analytics

7.1 OverviewOne of the main aims of having an event logging system is to provide the mean-ingful analytics to the user based on his/her past activities. Since these are tobe viewed and understood by all, representing the data in a graphical form ismost appropriate.

7.2 WorkingThe data necessary to plot the visualisations[7] for the user is queried fromelasticsearch using the APIs. The graphs module is designed to use these APIsto obtain the data stored in the elasticsearch and also performs queries to obtainthe data stored in the database. From this data the required information isparsed and fed to C3.js in appropriate format. C3.js then builds the chartaccording to the data and the structure provided.

7.3 DashboardsThere are three types of Dashboards:

1. Article Dashboard

2. Community Dashboard

3. User Insight Dashboard

Article Dashboard: The article dashboard displays the Article view timelineof a particular article (i.e. it shows the views of that article over a period oftime) in a form of a line chart. This helps the user to analyse the variation inpopularity of the article over a period of time.

Figure 8: Article View Timeline

41


Community Dashboard: The community dashboard displays the commu-nity view timeline, trending articles of a community and the top five most viewedarticle of the community.

Community view timeline displays the views of that community over a periodof time in a form of a line chart. This is useful in analysing the variations inthe popularity of the community over a period of time.

Figure 9: Community View Timeline

Trending Articles displays the list of the articles which have been viewed mostin the community in the past 10 days.

Most viewed articles in the community are shown in the form of a bar graph,with the bar height displaying the total number of views

Figure 10: Most Viewed Articles in a Community

42


User Insights Dashboard: The user insight dashboard displays the articlestatus, recently viewed articles and the top five most viewed articles.

Article status displays the current state of the article created by that user inform of a piechart. This helps the user identify the number of articles which heneeds to work upon.

Figure 11: Current State of articles created by the user

Most viewed articles displays the top five most viewed articles which werecreated by that user in form of a bar graph. This helps the user identify his/hermost popular articles.

Figure 12: Most Viewed articles created by the user

43

8 SELENIUM TESTING Event Logging System

8 Selenium TestingOur goal is to test the Recommendation System. The required selenium testcases are given below.

8.1 Test Case 1Input Train the system so that it can recommend. Log in into the Collab-orative Communities by any user and go to the articles page. Select any onearticle.

Output Observe the article’s page and check whether the recommendationlinks are present. The recommendation links which are present must open prop-erly without any error.

8.2 Test Case 2Input Create 2 communities and add 6 published articles in each communities.Now add 3 users to the system.All this will be done by django admin. The firstuser will join community 1 and the second user will join community 2. Nowcreate some views for the articles in each community. The user in community 1will give views to some articles of community 1 and same for community 2. Somearticles will get more views than the others in each community. For instance,article 1 from community 1 will get 10 views, article 2 will get 8 views, article 3will get 2 views and the left wouldn’t get any new views by user1 and similarlyin community 2 by user 2. Now user 3 comes into picture and views article 2,3and 4 from community 1. Now the model is trained.

Output Log in with user 3 and view any of the published articles. Observe therecommendations. The percentage of articles present in the recommendationslist should be more from the community 1 than the community 2 as the user 3has viewed artices from community 1 only.

44

9 FUTURE SCOPE Event Logging System

9 Future ScopeThe future scope for the event logging system includes a wide variety of taskssome of which are:

1. Adding browser events or events that are generated in front end and do notsend any request to the backend e.g. events like resizing window, scrollingup and down which may provide some useful information regarding humaninteraction with the website.

2. Adding events for video related activities like play, pause, skip etc.

3. Using event logs to provide more analytics

4. Storing logs in cloud.

5. Create dynamic dashboards that display changes to Elasticsearch queriesin real time.

45

REFERENCES Event Logging System

10 Bibliography

References[1] c3js. c3js github. url: https://github.com/c3js/c3 (visited on 06/12/2018).

[2] edx. edx event tracker. url: https://github.com/edx/edx-platform/tree/master/common/djangoapps/track (visited on 05/25/2018).

[3] elastic. Elasticsearch Reference and Docs. url: https://www.elastic.co/guide/en/elasticsearch/reference/6.3/index.html (visited on06/02/2018).

[4] elastic. Kibana Reference. url: https://www.elastic.co/guide/en/kibana/current/getting-started.html (visited on 06/04/2018).

[5] elastic. Logstash Reference. url: https://www.elastic.co/guide/en/logstash/6.2/index.html (visited on 06/03/2018).

[6] Vitor Freitas. How to Create a Custom Django Middleware. url: https://simpleisbetterthancomplex.com/tutorial/2016/07/18/how-to-create-a-custom-django-middleware.html (visited on 05/20/2018).

[7] Pawel Kuna. Analytics CSS. url: https://github.com/tabler/tabler/blob/dev/dist/assets/plugins/charts-c3/plugin.css (visited on06/10/2018).

[8] Apache Lucence. Apache Lucene - Query Parser Syntax. url: https://lucene.apache.org/core/2_9_4/queryparsersyntax.html (visited on05/10/2018).

46

https://github.com/c3js/c3

https://github.com/edx/edx-platform/tree/master/common/djangoapps/track

https://github.com/edx/edx-platform/tree/master/common/djangoapps/track

https://www.elastic.co/guide/en/elasticsearch/reference/6.3/index.html

https://www.elastic.co/guide/en/elasticsearch/reference/6.3/index.html

https://www.elastic.co/guide/en/kibana/current/getting-started.html

https://www.elastic.co/guide/en/kibana/current/getting-started.html

https://www.elastic.co/guide/en/logstash/6.2/index.html

https://www.elastic.co/guide/en/logstash/6.2/index.html

https://simpleisbetterthancomplex.com/tutorial/2016/07/18/how-to-create-a-custom-django-middleware.html



https://github.com/tabler/tabler/blob/dev/dist/assets/plugins/charts-c3/plugin.css

https://github.com/tabler/tabler/blob/dev/dist/assets/plugins/charts-c3/plugin.css

https://lucene.apache.org/core/2_9_4/queryparsersyntax.html

https://lucene.apache.org/core/2_9_4/queryparsersyntax.html

Date post:	29-May-2020
Category:	Documents
Upload:	others
View:	12 times
Download:	0 times

Event Logging System for Collaborative Communities...Django REST framework is a powerful and...

Documents