Generic Database Generic Database ExplorerExplorer

Xunyang ShenXunyang ShenDemetrios DimatosDemetrios Dimatos

Madhujith HapuarachchiMadhujith Hapuarachchi

Project OverviewProject Overview

►Original version of BDAPOriginal version of BDAP HomepageHomepage with description with description Generic Database Explorer Generic Database Explorer

►Enhancements Enhancements Pending QueuePending Queue User Cached ResultsUser Cached Results Baby DatabasesBaby Databases

►Team workTeam work Do work individually with Do work individually with

interdependencyinterdependency SVN, Weekly meeting, testing after SVN, Weekly meeting, testing after

mergencemergence

Control StructureControl Structure

Baby DatabasesBaby Databases

►Basic idea: to generate a smaller database Basic idea: to generate a smaller database characterizing the full database so that queries can characterizing the full database so that queries can be run on it very fast.be run on it very fast.►Approach:Approach:

Load/configure a databaseLoad/configure a databaseGenerate histogram infoGenerate histogram infoGenerate baby databaseGenerate baby database

►Statistic InformationStatistic Information►Latest InterfaceLatest Interface

Baby Database Baby Database Development ProcessDevelopment Process

►Design documentDesign document►DifficultiesDifficulties

Algorithm to generate baby databases;Algorithm to generate baby databases;The major logic path of query.php;The major logic path of query.php;The pending queue has some The pending queue has some dependencies on baby databases;dependencies on baby databases;Easy to trig conflicts in SVN, since Easy to trig conflicts in SVN, since everybody needs to revise query.php & everybody needs to revise query.php & function.lib.php;function.lib.php;Testing is slow for big databases.Testing is slow for big databases.

Baby Database CodeBaby Database Code►Estimate total code lines: Estimate total code lines: 850850►Files created or changed:Files created or changed:

configDB.phpconfigDB.phpsetDB.phpsetDB.phploadDB.phploadDB.phpviewStats.phpviewStats.phpgenerateBDB.phpgenerateBDB.phpbabyDB.phpbabyDB.phpquery.phpquery.phpfunction.lib.phpfunction.lib.php

Pending QueuePending QueuePrevious State : In the previous version Previous State : In the previous version of software, once a query was of software, once a query was submitted to the database, it was run to submitted to the database, it was run to completion and no other queries can completion and no other queries can even be submitted from the page.even be submitted from the page.

Enhanced State: Each submitted query Enhanced State: Each submitted query is stored in a data structure that acts as is stored in a data structure that acts as a FIFO queue. The users no longer a FIFO queue. The users no longer have to wait for a query to end before have to wait for a query to end before they can submit another.they can submit another.

Pending Queue Pending Queue Development ProcessDevelopment Process

► Attempting the run the queries on the Attempting the run the queries on the database while collecting the submitted database while collecting the submitted queries tp a queue using the single control queries tp a queue using the single control point was difficult. This was solved by point was difficult. This was solved by extracting the query execution part in to extracting the query execution part in to independent PHP script that infinitely ran as a independent PHP script that infinitely ran as a background process, initiated by the background process, initiated by the administrator. administrator.

. . ► Estimating the time for each query that would Estimating the time for each query that would

be run on the full database was obtained by be run on the full database was obtained by manipulating the sizes of the full database, manipulating the sizes of the full database, the baby database and the time consumed for the baby database and the time consumed for querying the baby database. querying the baby database.

► Coding while learning PHP and MySql, Coding while learning PHP and MySql, presented difficulties but COSI books, google presented difficulties but COSI books, google and friends were always a savior. and friends were always a savior.

Pending Queue CodePending Queue Code

►query.phpquery.php

►function.lib.phpfunction.lib.php (All functions of BDAP) (All functions of BDAP)

►Execute_query.php (Independent PHP query)Execute_query.php (Independent PHP query)

►display_pending_queue.php (Global Pending display_pending_queue.php (Global Pending Queue)Queue) ►display_personal_queue.php (User Pending display_personal_queue.php (User Pending Queue)Queue)

Implemented Features and Implemented Features and RestrictionsRestrictions

► Once a query has been selected to be sent Once a query has been selected to be sent to the full database, it will arrive at the to the full database, it will arrive at the pending queue. Unless the exact query is pending queue. Unless the exact query is not already present in the queue table, it not already present in the queue table, it will wait to be executed.will wait to be executed.

► Links in the navigation bar will display the Links in the navigation bar will display the pending queue for all users as well as all pending queue for all users as well as all the queue for the current logged in user.the queue for the current logged in user.

► The users may delete their queries from The users may delete their queries from their pending queue.their pending queue.

User Cached QueriesUser Cached Queries

►Store all user queriesStore all user queries Every submitted query after Every submitted query after

running on the baby running on the baby database is stored in the database is stored in the user cache.user cache.

Allows for users to share Allows for users to share cached queriescached queries

►Fast retrievalFast retrieval Queries stored as serialized Queries stored as serialized

datadata Recreate the table from the Recreate the table from the

serialized dataserialized data

User Cached QueriesUser Cached Queries

►Manageable cache sizeManageable cache size Administrator can specify each user's Administrator can specify each user's

maximum cache size.maximum cache size. Automated garbage collection occurs Automated garbage collection occurs

when a user has used more then their when a user has used more then their space allowance. Deletion of the space allowance. Deletion of the oldest viewed query.oldest viewed query.

►View and delete cached queriesView and delete cached queries Users can view only their own cached Users can view only their own cached

queries as well as delete them.queries as well as delete them.

User Cache User Cache Development ProcessDevelopment Process

Using Version ControlUsing Version Control

►Managing files Managing files Occasional conflictsOccasional conflicts

SVN merging of source codeSVN merging of source code

RollbackRollback

SubversionSubversion

The End.