Local Query Processor Manager for the Composite ...web.mit.edu/smadnick/www/wp2/1990-18.pdf ·...

Local Query Processor Manager for theComposite Information

System/Tool Kit

Min-Yuan Tung

May 1990 WP # CIS-90-18

Local Query Processor Managerfor the

Composite Information System/Tool Kit

Min-yuan Tung

Bachelor of Science Thesis in Computer Science and EngineeringMassachusetts Institute of Technology

Cambridge, MA 02139

ABSTRACT

The Local Query Processors Manager (LOP Manager) provides back-up support forinformation retrieval in the Composite Information System / Tool Kit (CIS/TK). The LQPManager reroutes queries to alternate databases when the currently selected databasebecomes inaccessible. The mechanism used to achieve this rerouting groups together all thedatabases in the system with the same local schema. Alternate databases are selected fromthe same group as the that of the current database. Status tables as used to help keep trackof the status of all the LOPs in the system. The LOP Manager also provides a downloadingfacility to incrementally download data from remote databases into local databases.

ACKNOWLEDGEMENTS:

Thesis advisor: Stuart Madnick; with assistance from the CIS/TK Research Group.Supported in part by the MIT International Financial Services Research Center, Reuters, AT&T.

Table of Contents

1 Introduction...................................................................................................................1

1.1 Background - CIS/TK Project ...................................................................... 1

1.2 Back-up support and local data storage presently in CIS/TK ........................... 1

1.3 Goals of Thesis ........................................................................................... 1

1.4 Overview of Thesis ...................................................................................... 2

2 Overview of the Local Query Processor Manager ......................................................... 2

2.1 Overview of CIS/TK.................................................................................... 2

2.2 Local Query Processor................................................................................ 2

2.3 The Global Query Processor....................................................................... 2

2.4 Pseudo-databases.................................................................................... 6

2.5 Error handling capabilities of CIS/TK (problems need to solve) ....................... 6

2.6 LQP Manager ............................................................................................. 6

2.7 Download facility in the LOP Manager........................................................... 7

2.8 Implementation Environment ....................................................................... 9

3 The Local Query Processor Manager........................................................................... 9

3.1 Overview of the LOP Manager Architecture .................................................. 9

3.2 LQ P status tables ........................................................................................ 10

3.3 Query Router ............................................................................................. 1 2

3.4 Rechecking Mechanism ............................................................................. 13

3.5 LOP SETS ................................................................................................. 14

4 Downloading Facility.................................................................................................... 14

4.1 Global View .................................................................................................. 14

4.2 Details of the downloading facility................................................................ 16

5 Menu System ................................................................................................................ 19

5.1 LQP Configuration Menu............................................................................ 19

5.2 CIS/TK Configuration Menu......................................................................... 19

6 Conclusion....................................................................................................................20

6.1 Future W ork ............................................................................................... 20

REFERENCES ................................................................................................................... 22

APPENDIX A: Sample of LQP Manager and Downloading facility ...................................... 23

APPENDIX B : Menu System ........................................................................................... 31

APPENDIX d ....................................................................................................... 38

LOP Manager in the Composite Information Systems/Tool Kit

1 Introduction

The Composite Information System / Tool Kit (CIS/TK) is a prototype being developed at the MIT SloanSchool of Management that explores connectivity issues in the field of information systems. [WON 89]The Local Query Processor Manager is the regulation unit in CIS/TK that coordinates informationretrieval in terms of automatically providing alternate databases to query when the selected databasesare not accessible.

The Local Query Processor Manager (LQP Manager) provides an interface for the user to choose aparticular database he wants to query. The LQP Manager also has a mechanism for rerouting queries toalternative databases if the present database is unavailable. This is achieved with the development ofLocal Query Processor Sets (LOP SETs) and two tables (LQP-STATUS and ACTIVE-LOP).

The CIS/TK system presently has copies of remote databases at the local site. A downloading facility willbe implemented as an extension of the LOP Manager to facilitate the automatic maintenance of theselocal databases.

This thesis describes the design and implementation of the LOP Manager.

1.1 Background - CIS/TK Project

With the revolutionary way managers are now conducting business, their information needs and usageis also changing rapidly. The CIS/TK project addresses the needs of the businesses for more and fasterinformation manipulation among heterogeneous databases.

The Local Query Processor Manager provides a more robust information system in terms of informationretrieval. The system can now automatically retrieve data from an alternate source if the current databaseis inaccessible. The query is not aborted as before. In the LOP Manager design, the alternate databasehas to have the same local schema as the remote database. Presently, only the local databases havethe same local schema as their remote databases.

Another issue that CIS/TK has not yet addressed is that of local storage of data and downloading ofdata. Having remote data at the local site will enable information retrieval when the remote databasecannot be reached. Keeping the information in the local storage current and complete is an issue thatthe downloading mechanism will discuss.

1.2 Back-up support and local data storage presently In CIS/TK

CIS/TK has partial downloads of tables from the remote database used for debugging purposes. TheLOP Manager will use these partial downloaded databases for data when the current database isunavailable or when the local databases are selected specifically for debugging purposes. The benefitsof the local databases is discussed in Chapter 2. The information in these partial downloaded databasesare not current and may be outdated. Presently, the upkeep of these databases is the responsibility ofthe database administrators.

1.3 Goals of Thesis

To achieve back-up functionality, the LOP Manager must attain the following goals:

- Provide a structure for coherently organizing the LOPs- Provide facilities for rerouting queries- Provide downloading facilities- Provide an interface into the present CIS/TK

Page 1


1.4 OvervIew of Thesis

Chapter 1 : Introduction of thesisChapter 2 : Overview of the Local Query Processor Manager, downloading facility, and design

decisionsChapter 3 : Details of the implementation of the LQP ManagerChapter 4: Details of downloading facilityChapter 5: Description of the Menu systemChapter 6 : Conclusion and future work

2 Overview of the Local Query Processor Manager

This chapter describes the architecture of the Local Query Processor Manager and discusses some ofthe major design considerations. One of the restrictions in designing the LOP Manager is that it shouldnot require large amounts of modification of the present code in CIS/TK. The LOP Manager is also astep toward integrating local schemas into CIS/TK. It provides a foundation for some issues that localschemas should address.

This chapter will present the overall architecture of CIS/TK and its relation to the LOP Manager. We willthen discuss some of the problems faced in providing a uniform yet flexible view of the underlyingdatabases.

2.1 Overview of CIS/TK

Figure 2.1 describes all the components in the CIS/TK and how they interact. The Applications QueryProcessor is the interface with the user. This component has not been completely developed. TheGlobal Query Processor processes the user queries and directs them to the correct Local QueryProcessor. The Local Query Processor links CIS/TK with the remote databases.

2.2 Local Query Processor

The Local Query Processor (LOP) is the translator between the abstract query language used by theprototype and the database language (whether it be SQL, menu-base, etc.) used by the remotedatabase.

Presently, there are four main LOPs in the system. They are for the IPSharp Disclosure, Finsbury'sDataline, Recruit Database, and the Alumni Database on the AT&T 3b2.

2.3 The Global Query Processor

The Global Query Processor (GOP) is the "brains" of the prototype. It decides how to subdivide androute the queries according to some set rules and the global schema. [WON 89]

The global schema can be thought of as an organized "view" of the various underlying databases. Itfollows the Entity-Relation model. It presently maps the attributes of the entity directly to the specificdatabase, table, and column which contains the data.

Page 2


CIS/TK

MERGEDATA L

MODEL

LocalAbstractQuery

LocalDBMSQuery

Figure 2.1 : Overview of CIS/TK architecture before LOP Manager

GOP

LocalkbstracQueryFormatted

Data

RawData

LocalDBMSQuery

Page 3


The LOP Manager requires a slight modification to the global schema because it changed the view thatthe global schema has of the underlying databases. Instead of seeing individual databases, the globalsees a group of databases (LOPSETS) that will provide it with a certain type of information. Themodification changes the mapping of the entity attributes from specific databases to LQP_SETs.

Since the LOP Manager specifications does not involve joins, subdividing queries, and providingsynonyms (all the functionalities of the GOP), it is important that the LOP Manager *ill not require largemodifications of the GOP code. The hashing mechanism, query routing are all unaffected by theaddition of the LOP Manager. However, the LOP Manager interfaces directly with the GOP andtherefore some modifications will be necessary. As mentioned above, the global schema will bemodified. Also, the synonyms and translation tables will be updated with the names of the LOP_SETsinstead of the names of the LOPs themselves. This one-to-one replacement is possible because of thedesign of the LOP_SETs and its requirement that the databases in it has to have the same table namesand column names

Figure 2.2 shows the LOP Manager as the link between the GOP and the LOPs. All the queries sent bythe GOP are directed to the LOP Manager instead of directly to the LQPs. The LOP Manager finds thecurrently active LOP for the LQP_SET indicated by the query and sends the query to the LOP. Shouldthe database be inaccessible due to failure of the satellite link or the remote database itself, then theLOP Manager proceeds to locate another database for the query.

The present global schema contains only the databases that it can query. In other words, it contains allthe selected database. The global schema contains only the remote or "real" LOPs. When a problemarises with a remote database, the database administrator has to manually modify the global schema. Hewould replace the failed database with the local database. Therefore, usually, the remote and thecorresponding local database are not both in the same global schema at the same time.

With the addition of the LOPSETS, the CIS/TK system will be able to access the local databaseimmediately and automatically when a remote or selected database becomes inaccessible. TheLQP_SETS contains the name of all the databases that has the same local schema. Right now, only thelocal databases has the same local schema as its respective remote database. But through thisadditional abstraction, the LOP Manager will be able to access altemate databases.

A shortened version of the global schema before LOP_SETS.

Company Entity:* name -- > recruit-ibmrt, "companytbl", company_name

-- > disclosuredb, static, co-- > pseudo-dataline-mit2e, data, code

* address-1 -- > disclosuredb, static, adl* country -- > pseudo-dataline-mit2e, data, country* exchange -- > disclosuredb, static, ex* position -- > recruit-ibmrtr "companytbl"r position

Note that the pseudo-dataline-mit2e LOP is used in the global schema instead of the remote datalineLOP. This is probably because the remote dataline database is down and the database administratorhas manually changed the global schema to send queries to the local dataline database.

Page 4

LQP Manager in the Composite Information Systems/Tool Kit

Figure 2.2 : Overview of the architecture of the main components in the CIS/TK system with the

addition of the LOP Manager and the Downloading facility.

A shortened version of the global schema after the implementation with LQP_SETS:

Company Entity:* name -- > oracleset, "companytbl", company_name

-- > disclosureset, static, co-- > dataline_set, data, code

* address-1 -- > disclosureset, static, ad1* country -- > datalineset, data, country* exchange -- > disclosure_set, static, ex* position -- > oracle set, "companytbl", position

ACTIVE-LOPLOP-STATUS

QP SET Selected LOP

* Updates status* Retrieves status

data

query

Retrieves datafrom Local Databasewhen Remote DBis inaccessible

Page 5


Note that in the new global schema that only the name of the LOPSET is used in the global schema.The dataline set could be selected as either the remote dataline LOP or the local dataline LOP. Thequery can then be routed according to the current selection.

2.4 Pseudo-databases

As mentioned before, there exists on the CIS/TK system pseudo databases. These contain partialdownloaded data from remote databases. The databases do not contain complete or current data.However, these pseudo databases are very useful to the CIS/TK system. They are used in debuggingand testing code which saves time and money. The pseudo databases are also faster in retuming dataand are independent of communications link.

The pseudo databases can be more useful than for debugging purposes. For one thing, they can beused as caches for the remote databases. To be used as caches, the pseudo databases have tocontain current data. There has to be a mechanism to regulate the data in the caches. In addition, theGOP will have to be modified to make use of the pseudo databases as caches.

An altemative use for the pseudo databases is as a back-up source for data. The pseudo databases areused only when the remote database can not be reached. Since they are used as a back-up sources ofdata, the data in the databases do not have to be complete or current. This use is a more appropriateuse for the pseudo databases in the current system.

The use of the pseudo databases as a back-up source of data fits into the design of the LOPManager to use them as the alternative database when the remote database cannot be reached.The LOP Manager does not make any assumptions about the correctness of the data in thesepseudo databases. It just retrieves it. The user will need to specify whether he wants the datafrom these pseudo databases.

As part of this thesis, a downloading facility will be implemented to incrementally fill and update thepseudo databases. The pseudo databases are easier to maintain as back-up databases. Theconstraints on the type and currency of the data are not as severe as the constraints on the databases ifthey were used as caches.

2.5 Error handling capabilities of CIS/TK (problems need to solve)

The current CIS/TK system aborts queries that cannot be sent. The present communication serverdoes not distinguish the difference between a database login error, a communications link error, or thatall lines are busy (this feature is a future work in Boniface Makatiani's thesis). Boniface Makatiani isdeveloping a more robust communications server that can distinguish among all these types of errors.[MAK 90] The LOP Manager can use this information to decide whether rerouting of the query isnecessary. For example, if all the lines are busy, it will not necessitate rerouting the query.

2.6 LOP Manager

The LOP Manager is designed to reroute queries from unavailable databases. One possible design ofthe LOP Manager reroutes the queries to another remote database that contains the neededinformation. The advantage of this design is that the back-up data retrieved is also current. Thedisadvantage is that the alternative databases may not contain all the information required by the query.Some data may only be found in the original database, and therefore rerouting a query may involvesubdividing the query again among the available databases. This method would require reprocessingthe query from the GOP with the knowledge that one of the databases is unavailable. This methodwould require modification of the present GOP, which we want to avoid.

Page 6


The second design of the rerouting mechanism in the LOP Manager, the one implemented, uses thelocal downloaded databases as the back-up database. The LQP Manager groups the downloadeddatabases and the remote databases into LOP_SETs. The disadvantage of this design is that thesystem is limited and not robust. It cannot access other databases that might contain part but not all ofthe needed information. The advantage is that it utilizes the present resource of the system (thedownloaded databases). This design is a basis for the integration of local schemas into the system. TheLQP_SETs provides an abstraction between the global schema and the underlying databases. Thedatabases in the same LQP_SET have the same local schema. This design also does not require thedynamic modification of the global schema to reroute a query.

Since the databases in the same LQP_SET have the same local schema, the databases in theLOP_SETs has to contain the same type of information and has to have the same table names andcolumn names. Presently, the only databases that can be grouped together are the pseudodownloaded databases (or duplicated database) with their respective remote database. (i.e. pseudo-disclosure-mit2e with IPSharp Disclosure, pseudo-dataline-mit2e with Finsbury's Dataline, alumni-mit2dwith alumni-mit2e)

The system runs into trouble with the present IPSharp database. The remote database has multipletables, some for static data, and other for time-series data. However, the implementation of the IPSharpLOP [GAN89] shows the remote database as having one table containing both time-series and staticdata. Presently, in the downloaded IPSharp database, there are three tables instead of one. Havingthree tables eliminate the redundancy of data storage. However, the LOP for the downloaded databaseshows three tables to the global schema. The remote and local databases, therefore, cannot have thesame local schema. The final design showed both databases as having two tables, static andtimeseries. The need for two tables is due to the constraint placed on the system by the remoteIPSharp database. It can only receive queries concerning static or timeseries data, not both in thesame query.

2.7 Download facility In the LOP Manager

The purpose of designing and implementing a facility to maintain a back-up database for the realdatabases on the CIS/TK is for debugging purposes and more importantly, to try to have locally ascomplete and updated information as possible to be used when the real database is inaccessible.

The downloading facility will update the back-up databases (or pseudo-database) with data retrievedfrom a query sent to the real database. This design enables the pseudo-database to be filled with dataincrementally as the system is used. This means that the pseudo database is empty at its time ofcreation, and as more queries are sent to-its real database, the pseudo database will grow with theretrieved information. One of the problems encountered in storing the data retrieved from a query isthat data may not exist for all the attributes in the table. However, we chose this design because first, it isbetter than no back-up data. Secondly, the data most often requested will be in the back-up database,so therefore, there is a good chance that needed data will be in the back-up database.

The size of the back-up database has presently been limited to 1000 rows. This value can be modifiedas needed. The decision to limit the size of the back-up database is because CIS/TK presently does nothave the memory to maintain a complete back-up database for each of the real databases on the system.

The downloading facility uses two procedures to download data: insert and update. Insert is used fornew rows, and update is used for rows that already exist in the back-up database. In limiting the size ofthe back-up database, we kept the database to a manageable size, but we also want our data to be ascurrent as possible. Therefore, the download facility will stop inserting new rows into the back-updatabase when the predetermined size of the back-up database has been reached. However, updateswill still be performed.

Page 7


A local schema is devised for each LQP. A local schema should describe as completely as necessarythe properties of the underlying database. The devised local schema (attributes catalogue - figure 2.3)lists the attributes of the tables in the database and it also lists the primary key(s) for the tables in thedatabase. In addition, it also specifies the type of format the data in the database is in. For example, itspecifies the scale of any numbers, the currency the data is in, the format of the date, etc. [RIG 90].

The attributes catalogues are Korel objects [LEV87]. Korel is an object-oriented Language written foruse in the CIS/TK system.

The attributes catalogues are located within each LOP. This catalogue is shared between the LOPManager and the LOPs. The LQP Manager uses it to find the primary keys for the tables in thedatabase. The LOP uses it to translate data into the form that the GOP specifies in the global attributescatalogue. [Rig 90] [WON 89]

In the design of the LOP Manager, the pseudo databases were assumed to have the same tables andprimary keys as their real databases. Therefore, the real and pseudo database share the same localschema.

A future work is to use the local schema to capture some of the properties of the remote database,thereby maximizing the functionalities of the remote database. An example is for SOL databases to dojoins at the remote site instead of locally by the GOP.

Attributes Catalogue for Disclosure:

;; List of attributes in the disclosure database* attributes -- > co

* attributes -- > PC

* attributes -- > sc

* attributes -- > year* attributes -- > cf

* attributes -- > ni

* attributes -- > ns

* attributes -- > in

...etc.

;; Match attribute to its data typeco -->name

* PC -- > name*sc -->name* year -- > year

*cf -->date* ni -- > figure* ns -- > figure

*in -->nameetc...

;; KEYS for the tables in disclosure* static -- > co

* timeseries -- > co, Cf

Figure 2.3: Sample attributes catalogue for the IPSharp Disclosure database. It is used as the localschema for both the remote and local Disclosure databases.

Page 8


2.8 Implementation Environment

CIS/TK is developed on the UNIX platform but most of the kernel of CIS/TK is developed using KOREL[LEV 87], an object-oriented programming language developed in the Common LISP environment.The LQP Manager utilizes the object-oriented paradigm to provide the abstraction between the globalschema and the databases themselves.

3 The Local Query Processor Manager

The main function of the LOP Manager is to organize and coordinate all the LQPs currently in thesystem.

The LOP Manager is designed with the following functions:

- Maintaining the status of all the LOPs in the system. (i.e. whether the satellite link or remote database isoperational. )- Allowing users to select specific LOPs (i.e. database) to send a query.- Error handing for cases where the LOP, the communication link,the remote database, or the query is faulty.- Downloading of information from remote databases into local databases.

3.1 Overview of the LOP Manager Architecture

The LOP Manager architecture is comprised of two components: status tables and query routing. Thepartitioning reflects the two main tasks that occur in query processing. The LOP Manager retrieves theselected LOP from the status tables and send the query to the LOP. If an error occurs with the remotedatabase, the query is rerouted to another database as specified by the user or to a default databaseand the status tables are updated accordingly. Figure 3.1 outlines the details of the LOP Manager.

The Status Tables

The LOP Manager maintains the status of each LOP through the help of two INFORMIX tables, LOP-STATUS and ACTIVE-LQP. The LQP-STATUS table contains information on the members of eachLQP_SET, the priority of the LOPs in the LOPSET, the status of each LOP (available or unavailable),and the machine each LOP is located. The ACTIVE-LOP table maps the name of the currently selectedLQP to the LQPSET.

The design uses two tables for purposes of extendibility. When CIS/TK becomes multi-user, each userthen has his own ACTIVE-LOP table that maps the databases that he wants to query without interferingin other users database selection. A common LOP-STATUS table is shared by all users on the systembecause the status of each LOP is the same for every user.

The Query Router

As mentioned earlier, the databases in the system are grouped into LOP_SETs. These LOP_SETsgroup "equivalent" databases. Equivalent means that the databases contain the same tables with thesame type of information. This is necessary as discussed in Chapter 2 on design considerations.

The query router sends the query to the selected LOP as indicated in the active-lqp table. If thedatabase is unavailable, then the query is reroute to an alternative database selected from the sameLOPSET as the initial database. Should there be no more alternative databases, then the query isautomatically aborted.

Page 9


The automatic selection of the alternate database is accomplished with the help of a priority assignmenton every LOP in the LOPSET by the database administrator when the system is first setup. Thepriority number indicates which database in the LOP_SET should be tried first. The priority number isused to select the default database when the remote database is unavailable.

Figure 3.1 : Details of the LQP Manager architecture.

3.2 LOP status tables

The status tables LOP-STATUS and ACTIVE-LOP are where the information concerning all the LOPs inthe system are kept. The following tables show the state of the status tables at initialization of theCIS/TK.

ACTIVE-LOP table:

LOP SET SELECTED LOPDataline set Datalinedb

Disclosure set DisclosuredbOracle set Recruit-ibmrt

Portfolio set PortfoliodbAlumni set Alumni-mit2e

Page 10


LOP-STATUS table:

LQP SET LQP S P F M-nameDataline set Datalinedb ok 1 FinsburyDataline set Pseudo-dataline-mit2e ok 2 Mit2&Disclosure set Disclosuredb ok 1 IPSharpDisclosure set Pseudo-disclosure-mit2e ok 2 Mit2eOracle set Recruit-ibmrt ok 1 DonnerOracle set Pseudo-recruit-mit2e ok 2 Mit2ePortfolio set Portfoliodb ok 1 Mit2eAlumni set Alumni-mit2e ok 1 Mit2e

S = StatusP = PriorityM-name = Machine-name

All the information and setup are predetermined by the database administrator who is responsible forproviding the correct data and view of the underlying database.

When CIS/TK is first loaded on a system, the status tables are initializedAny updates made during query processing are made on the copy.permanent changes to the status tables. It cannot be done during queryto do the update through the menu system.

from a set of initialization tables.Only the DBA can make any

processing. The DBA will have

The LOP Manager needs to be able to manipulate the data in these two tables as the status of thesystem changes. The present LOP provides four methods for information retrieval (self-info, get-tables,get-columns, get-data). The LQP for the status tables also need the capabilities to update, delete andinsert data.

The update function is needed by the LOP Manager to change status of a LOP should it becomeunavailable. The delete and insert functions are used only by the DBA to add to or to remove a LOPfrom the system. The status tables are maintained in the Informix SQL database accessed through theInformix LQP.

A lexical parser is written for the update, delete and insert functions for the Informix LQP. The format ofthe update query should look like the following:

(send-message lqpset :update-data(table-name((= column-name update)

(= column-name2 update2) ... )

(conditions)))

The method update-data will parse the query into SQL format andcorrect update will retum a response of NIL.

send the query to the status table. A

An example of an update query is:

(send-message 'lqp-status :update-data('lqp-status

(= 'status "ok")((= 'lqp "alumni-mit2e")

(= 'lqp set "alumni-set"))))

Page 11


This query updates the status of the LOP "alumni-mit2e" in the LOP_SET "alumniset" to "ok".

The format for the insert function is similar to the syntax of its corresponding SQL format.

(send-message 'lqp-status :insert-data(table-name

(v1 v2 v3 v4 .... )))

The values v1 v2 v3 v4 ... are in the order of the columns in the table. The parser does not check for thecorrect order. It is the responsibility of the DBA to make sure the data is correctly entered in the query.Should there be a blatant misfit of data type in the query and the data type required by the databasetable, then an error will occur and the changes made up to the point of error will remain in the database.

Note: The downloading facility do not use the insert method to insert data into the pseudodatabases,but it does use the update method to update data in the pseudo databases. Chapter 4.

The present method assumes that the DBA will ensure the stability of the status tables when inserting ordeleting LOPs. As a future work, this function need to be made more robust.

Lastly, the delete function. It takes the following format:

(send-message 'lqp-status :delete-data(table-name conditions))

The query takes the table name and deletes all the rows that satisfy the given conditions.

The update function is accessible by the LOP Manager and the user for updating the status column ofthe lqp-status table and the selected-lqp column of the active-lqp table. Only the DBA has access to theother columns in the table. The DBA has the ability to modify the priority of the LOPs in the LQP_SETs,add and delete new LOPs and LOPSETs.

3.3 Query Router

With the given set of tools for manipulating the status tables, the LOP Manager is able to reroute queries

Every database in the system is placed into a LOPSET. An LOP_SET is a group of LOPs whosedatabases contain equivalent information. The LOP_SETs vary depending on the global schema of thesystem.

The LQP_SETs have one method: get-data. The get-data is the driver of the query router. It reroutesthe queries to an alternative database, either selected by the user or by default, when the remotedatabase is unavailable. Presently, there is no need for the other methods such as get-columns, andget-tables. Theses are not needed by the LOP Manager or the GOP. An issue that is not specifiedclearly is that whether :get-tables methods of the remote LQPs retum all the tables in the database orjust the ones that are needed by the global schema. In other words, should the GOP be able to accesstables other than what is defined by the global schema. This is important because the local databasesare only subsets of the remote databases. They do not contain all the tables in the remote database.According to the specifications, the get-tables method of the remote database should only return thetables that are used by the CIS/TK system. This will provide uniformity in the local schema. Then, thetable names for each LOP method can be retrieved from the local schema. Presently, some LOP get-table methods contain hardcoded data, ie dataline, while others, Alumni-mit2e, are not.

Figure 3.1 shows how the LOP Manager handles the queries received from the GOP. The query fromthe GOP is sent to one of the LOPSET objects in the LOP Manager as specified in the global schema.

Page 12


The query router proceeds to locate the currently selected database in this LOP_SET and sends thequery to it. If the query is successful, the data is returned to the GOP. If the query is not successful,then an alternate database is located.

Boniface Makatiani is designing a new communications server that can distinguish between problemswith the satellite link and remote database errors. The new communications server will return an errormessage in the file where data was suppose to be. Therefore, the LOP Manager kpows when an erroroccurs that requires the rerouting of the query. The errors that the communications server will retum arecommunications error (problem with the communications link), remote database error (problem with thelogin or changes in the remote database system), lines busy (when all the modem lines are busy).

The query router also accounts for the case where the remote database is accessible but no data isavailable. In this case, the LOP Manager just returns the null data to the GOP and no rerouting is done.This is because of the way the LOP_SETs are organized. All the databases in the same LOP_SETshould contain the same information, if the data cannot be found in one, then it should not exist inanother database in the same LOP_SET.

If the query is unsuccessful the first time, then the query router first updates the lqp-status table with theinformation that the present LOP is unavailable due to either a communications problem, or a remotedatabase problem. Then it proceeds to locate a default database to reroute the query. Should one notbe found, then the user is notified and the query aborted. If one is found, then the user has the optionof selecting an alternate LOP manually, or using the default. In either case, the ACTIVE-LOP table isupdated with the newly selected lqp and the query is resent.

The query router also has a recheck mechanism that double checks the status of the LOPs by sending aquery to the LOP. The LOPs' status is rechecked only when there are no available LOPs left in theLQP_SET. If after the recheck, all LOPs are still down, then the query is aborted.

As a future feature, the recheck mechanism can be implemented with the clock on the system so that atpre-specified intervals, the recheck mechanism will activate. Presently, a menu selection is provided sothat rechecking of the status of LOPs can be activated when seen appropriate by the databaseadministrator.

An alternative design for the recheck mechanism is to recheck all the LOPs status before a query to besure that the database of highest priority is being queried. This method is tedious and costly inoverhead.

3.4 Rechecking Mechanism

Rechecking the status of the LQPs is done by sending a canned query to the LOP. If the LOP returnsdata and not an error message from the communications server, then the status of the LOP is "ok". Thisworks in a similar manner as the LOP Manager. A query is sent to an LOP; if an error message returnsfrom the communications server ,such as ,***UNABLETOMAKECONNECTION*** or***REMOTEDBERROR***, then the status of the LOP is updated to signify the error, otherwise, thestatus of the LOP is termed as "ok". This recheck mechanism is limited in that it does not signal errorwhen the remote database itself has changed, ie the table names, column names, table structure,location of database on the system. The communications server will check the login procedure andguarantees that it succeeds, otherwise the error message ***REMOTEDBERROR*** will be sentback to the LOP.

The canned queries used by the recheck mechanism are stored in a hash table (APPENDIX C) underthe index of the name of the LOP. This hash table is created from a file every time CIS/TK is loaded.The file that holds the commands for the creation of the hash table and this file is updated every time aLOP is added permanently to the CIS/TK system.

Page 13


It might be possible to use the :get-tables method in each LQP to check the status of each LOP insteadof sending a canned query. This would eliminate the need for the hash table of canned queries. Theproblem is that in LOPs such as Dataline, the table name is hard coded. The method does not query theremote database. Therefore, the option of using the :get-tables method as a way to test the status ofthe remote database is not available currently but might be possible in the future when the LOPmethods are more consistent.

The hash table is chosen as the means of storing queries over database tables betause it gives fasterretrieval of information. However, since access to the remote database is quite slow, it would not havematter much if the canned queries were also stored in database tables. The hash table implementationis used because it is a different method of data storage from database tables.

3.5 LOP SETS

The following is how the present CIS/TK system is grouped into LOP_SETs.

Alumniset : alumni-mit2e, alumni-mit2dDisclosureset: disclosuredb, pseudo-disclosure-mit2eDataline_set: datalinedb, pseudo-dataline-mit2eOracleset: recruit-ibmrt, pseudo-recruit-mit2ePortfolio_set: portfoliodb

4 Downloading Facility

There are several important issues to be addressed during the design of the downloading facility. Theyinclude:

- How to update the pseudo database when the data retrieved from the real database does not includethe primary keys.- What does the local schema look like and what information it should hold.- Method of installation of data into the back-up databases.

4.1 Global View

Figure 2.1 showed the present CIS/TK internal workings. The GOP is the driver of the system. It parsesand routes queries directly to the LOPs. Figure 4.1 shows the downloading feature with the LOPManager. The LOP Manager acts like an abstraction between the Global Schema and the underlyingdatabases. It allows the user to manually change the database he wants to query. Also, when a realdatabase is inaccessible, the LOP Manager will locate an alternative database if one exists.

Look again at Figure 4.1. In step one, the GOP sends a query to the LOP Manager. The LOP Managertakes the query and sends it to the selected LOP for the particular LQP_SET. Before sending thequery, the LOP Manager may modify the query so that downloading the retrieved data is possible.

For data to be downloaded into the back-up databases, the primary key(s) of the queried table must bepart of the retrieved data. (Note: the GOP and the global schema subdivides all queries into subqueriesthat query only a single table. This is important to the design that is chosen.) Every table has primarykeys that uniquely distinguishes each row of data. There are two alternative designs of the downloading

Page 14


abs-lo

(table-name (colu(con

Insert primkeys

(table-name(column-names with key)(conditions))

03,

cal-query

nn-names)ditions))

ary

0DATA

return controlto LOP Manager

0DATA

IPSharpDisclosure

compnoI cf ni ns

A Akeys

DATA

PseudoDisclosure

0

compno cf ni ns

keys

Figure 4.1 : Detailed view of the downloading facility and of the steps needed in downloading data.

facility. One is to download only the data that contains the primary key(s). The second is to insert primarykey attribute(s) into the query before it is sent to the real database. The implementation uses the laterdesign. It allows all data returned to be updated into the back-up database without major complications.

K )

Page 15

i


Since the GOP already subdivides all the queries into ones that query a particular table, all the dataretrieved from a LOP would be from one table. The downloading facility would not have to distinguishbetween data from different tables. It needs only to distinguish data between rows in a table.

This would be a problem if in the future LOPs were capable of doing joins. It would then be necessaryas part of the downloading facility to distinguish the columns from the various tables. This should not bedifficult because the :get-columns method in each LOP should return the columns for the specifiedtable. A procedure is needed to parse the list of columns in the query to the respective tables and todownload the data to each table. This would be a future work when LOPs are more robust.

In the LOP Manager, the primary keys are inserted in the query before it is sent to the selected LQP(Step 3 in figure 4.1). In step 4, the data is returned from the LOP in the form of charater strings in a list.The data is sent to the downloading facility which updates the back-up database with the data (step 5 infigure 4.1). The name and location of the back-up database is located in each of the LOP objects in theslot :pseudo-db-name.

After knowing the name of the pseudo-database and the list of data to update the pseudo databasewith, the downloading facility proceeds using a record-by-record download. (Note: presently, all thepseudo-databases are using Informix SOL. Informix SOL can only do updates row by row. But insertscan be batched.) Figure 4.1 shows the algorithm used. For each row of data, the downloading facilitydetermines whether this row already exists in the pseudo-database by sending a select query on theprimary key(s) to the pseudo-database. If the row exists already, then an update function is performed.If the row does not exist, then the data list is inserted into a list. After all the data have been processed,then the list of data to be inserted is written to a file and loaded into the table through the LOAD functionof the Informix SOL. The control now returns to the LOP Manager. (step 6 in figure 4.1) The LOPManager then sends the formatted data to the GOP. (step 7 in figure 4.1)

4.2 DetaIls of the downloading facility

Figure 4.2 is a detailed algorithm used by the LOP Manager to process each query. The function of theLOP Manager is to be able to reroute queries from one database to another and to control thedownloading facility for the back-up databases. (Refer to Figure 4.2 and 4.3 throughout thisdiscussion)

The LOP Manager is sent a query by the GOP in a form called the abstract local language. The form ofthis query is just a list of the table name, column names, and the conditions of the query. This query issent to the LOP by the LOP Manager.

The LOP Manager first determines which database to query for the given LOP_SET. After a database islocated, then it is checked for the type of database it is, whether it is a remote database (real database) orthe pseudo database (back-up database). If the database is remote then the primary keys are inserted inthe column list. Primary keys are the only ways of uniquely identifying between the rows in the back-updatabase.

The LOP Manager is able to identify the type of database using the new slot (:pseudo-db-name) that isadded to each LOP object that indicates whether this LOP has a back-up database, and if so, its name.If this slot has a value, then it means it is a remote database; otherwise, it means that the LOP is for a localdatabase.

The primary key values are kept in the attribute catalogue [RIG 90] for each LOP. The attributecatalogue contains the attributes that the LOP supports and the type of data-type each attribute is. Italso contains the tablenames of the tables in the LOP and matches them to their primary key(s). Atypical attribute catalogue is shown in Chapter 2.6.

Page 16


MANAGER (abs-local-query):

(selected-lqp = GETLQP get the currently selected lqpfor the chosen LQPSET)

(Call REMOTE-DB? to check that the selected-lqp is a remotedatabase)

(IF the selected-lqp is for a remote database)(THEN call KEY-CHECK: check that primary key(s) for

the queried table is in the projection list.If not, put it in. It returns the new query.)

(formatted data, unformatted data = SEND query toselected-lqp)

COND:((formatted data = nil)(return error))

((formatted = link-down or remote-db-error)(FIND-ALTERNATE-LQP: locates an alternate database

based on the LQPSET))

(t(IF (Does the selected-lqp have a pseudo-database?)

(DOWNLOAD-DATA: downloads data from remotedatabase)))

(return formatted data to GQP))))

GET_LQP (lqp-set)(Look up in the Active-Lqp table the selected lqp for thegiven lqp-set)

KEY-CHECK (selected-lqp abs-local-query)(Retrieve the primary key for the selected-lqp from theslot with the table's name-in the Attribute Catalogue. Theattribute catalogue can be reached through thedata-type-catalogue in the lqp object.)

(IF the key(s) are not in the column list in the abs-local-query)(THEN append the key attributes to the list of column

attributes in the abs-local-query))

(return the abs-local-query))

Figure 4.2: Algorithm for the LQP Manager

Page 17


DOWNLOAD-DATA (data-list abs-local-query lqp)

(FOR each row of data in the data-list )(Check to see if the row already exist from a previous

download by using the primary keys.)(IF the row does not exist and the table is not full

(THEN insert the row in the insert-list)(ELSE update the pseudo-database with the data in

the data-list)))(IF the insert-list is not empty(THEN write the insert-list into a file and

load the table with data from the file))

Figure 4.3

Figure 4.3: Algorithm for the Downloading facility

The present LOP returns a list of the results in the form of character strings. [CHA 88] This is the datathat is downloaded into the pseudo-database. If the database is a remote database then the datareturned will be downloaded. This second check of the selected-lqp is to determine LQP whether todownload the data. The first check is to determine whether to insert the primary keys into the columnslist.

DOWNLOAD-DATA determines whether the data is to be updated or inserted. Since the pseudodatabases use the Informix SQL, there is a distinction between updating and inserting data. Aconstraint of the Informix SQL is that it only allows a row by row update. Therefore, the design isconstrained to record-by-record update.

To determine whether a record is to be inserted or updated, a query on the primary key(s) is sent to thepseudo database. If the query finds a row with the same key(s) then the row exists already and anupdate function is used to place the data into the pseudo database. If the query does not find the row,then the data is placed into a list. This list is later converted into a file that can be unloaded into aninformix table. Data is added to this list only if the pseudo-database is not full. The size is predeterminedbut changeable. A global variable in the code specifies the size. This value is modifiable by the DBA.

The update function uses the Informix SQL update function to update the columns retrieved by thequery. The update is based on the primary keys. Updates are processed even if the table is full.

After all the data have been processed, the control is returned to the LQP Manager. The LOP Managerthen returns the data to the GOP and the LOP Managers job is complete. (step 7 in figure 4.1)

NOTE: After the installation of Bertrand Rigaldies' thesis, the LQP's output will be different.[RIG 90]The LOP will return the same data in two forms: formatted and unformatted. Formatted means that thedata has been changed to the data type specified by the global attribute catalogue. Several of thechanges are the format of the date, type of currency the data is in, and the scale of any money amount.The unformatted data is the data that was originally retumed by the LOP. The downloaded data shouldhave the same data type as its remote database data. This is necessary because of the local schemaspecifications. Databases with the same local schema should have data in the same data type.

Page 18


5 Menu System

A menu system is devised so that the DBA has the ability to modify the status tables as necessary andalso to retrieve status information on the LOPs in the system.

5.1 LOP Configuration Menu

The DBA is provided with the following abilities:

- To see the status, priority, and machine name of the LQPs in the system and which LQPs are selected.- To add and delete a LOP into the system. The LOP will be installed in the requested LOP_SETs withthe new priorities.- To change the priority and machine name of the LOPs in any LQPSET.- To select a new LOP for any LOPSET to query.- To recheck the status of the unavailable LOPs in the system by sending a dummy query to the LOPsto see if they are operational again.

The first menu under LOP Configuration ask the user which LOP he wants to update. The two LOPsare the "lqp-status" and the "init-lqp". Both of these LQPs contain the LOP-STATUS table and theACTIVE-LQP table. When CIS/Tk is first loaded, the tables in "lqp-status" is replaced by a copy of thetables in "init-lqp". This is so because the LOP Manager used the status tables in the "lqp-status" LOP.Any changes made to the status of LQPs during query processing is made to the tables in the "lqp-status" LOP. Therefore, when CIS/TK is loaded up again, any changes in status from before would notbe shown unless the DBA specifically adds the changes to the "init-lqp" tables. To repeat, updating the"init-lqp" table makes the update permanent in the sense that the next time CIS/TK is loaded, thechange will be there. If the "lqp-status" is updated instead, then the update will last only as long as thissession of CIS/TK is running. The changes are not saved for longer term.

The decision to use two sets of tables is because every time CIS/TK loads up, the contents of the tablesshould be consistent. If any changes were made that were for debugging or testing purposes, it wouldbe made on the temporary copy of the status tables, and therefore the changes would not be reflectedthe next time CIS/TK loads up. Any permanent changes, such as adding a new LOP to the system,should be performed on the "init-lqp" tables so that the next time CIS/TK loads up, the change will bethere. Therefore, the DBA has to be careful of which set of tables he is updating, whether it is thepermanent or the temporary.

To add or delete an LOP from the system requires that the LOP is already somewhere on the system.Meaning that the LOP object code are in the correct directories. The add and delete functions of theLOP Manager just enable the LOP Manager to know about the new LOP and to send queries to it. Inadding an LOP to the system, the menu will guide the user through all the necessary tables andstructures that needs to be updated.

Selecting a LOP to be the one that the LOP_SET will send the query to is the function of this menu.Selecting a LOP does not change its priority in the LOPSET.

In updating the priority of the LOPs in the LOP_SET, the selected LOP for the LOP_SET isunchanged. Only the new priorities are assigned. If the selected LOP need to be changed, then usethe "select LOP" menu.

5.2 CIS/TK Configuration Menu

Two new selections are added to the CIS/TK Configuration Menu. Selection number 5 and 6.Selection 5 turns off the user menu that the LOP Manager brings up when a database goes down. The

Page 19


menu asks the user to select an alternate database. If the menu is turned off, the default LOP is used asthe alternate LOP.

Selection 6 turns off the downloading facility in the system. Since downloading takes a long time, thisfeature can prove useful for debugging, testing, and demo purposes.

6 ConclusionIn this thesis, the design of a back-up system for the CIS/TK for data retrieval and incrementaldownloading was presented. This was motivated by a need to retrieve data even when the remotedatabase is unavailable. Also, it was a first step in installing local schemas into the system. In thischapter, we present some possible future work for extending local schemas.

6.1 Future Work

In researching the local schema, we discovered that local schemas can incorporate information aboutthe remote database so that the functionalities of the remote database can be utilized. An example is ifthe remote database is SQL, then joins can be performed remotely rather than locally by the GOP. Thiswould require modifying the global schema so that not all queries would be divided into subqueries bytables.

There are several other issues that the LOP Manager could address. One is a procedure thatautomatically creates pseudo-databases given its local schema. In this implementation of the LOPManager, a local schema is provided in the form of attribute catalogues. It contains the neededinformation to create pseudo-databases: table names, columns names in the correct order, primarykeys. This should be enough information to create a pseudo-database.

In Chapter 2, caching the pseudo-databases was briefly mentioned. It is an ambitious goal to try to usethe pseudo-databases as caches for the remote databases. One of the major problems is where to putthe caching mechanism. The GOP would still subdivide queries to the correct LOP_SETs. How wouldthe caching mechanism know that all the data for the query is in the cache? Queries like "Get thecompany names that has profits over 1 billion US dollars" if sent to the cache would probably returnsome of the company names, but not all. A solution would be to send all such queries directly to theremote database and avoid the cache altogether. A requirement for the information in a cache is that ithas to contain complete information on a pre-specified column, like company name. Therefore, allqueries involving information on just one company can be looked up first in the cache.

There are still the other issues of replacement scheme in the cache: first-in-first-out, least-recently-used.Also, the setup of the cache has to be decided: size, how many tables, what the selection cached on,etc. Notice that the GOP, global schema, and the LOP Manager might all be affected to varyingdegrees depending on the type of cache implemented. The cache design would have to take thesefactors into account.

The recheck mechanism in the LOP Manager can be improved upon. Right now, the LOP Manager onlyrechecks the status of the LOPs when there are no LQPs with an "ok" status in the LOP SET. Afuture work would be to use the recheck mechanism to make sure the status of the LOPs are currentbefore sending the query. This means that make sure that the LQPs that have "ok" status are really"ok", and that the LOP that are down are really down. Therefore, the query sent would be successfulthe first time. An efficiency problem will occur depending on where this auto-recheck is placed in theLOP Manager because the recheck mechanism is slow. It has to query several remote databases. If theauto-recheck activates every time a query is sent, one query can take an extremely long time. Asolution would be to do a random recheck over certain intervals of time.

Page 20


Further research could be performed on the trade-off of speed verses share/multi-user capabilities ofthe ACTIVE-LOP table. A suggestion is to implement ACTIVE-LOP table as an object. It would befaster to access as an object, and at the same time users can have their own copy of the the object.

A better device is needed to answer the performance issue in downloading data from remotedatabases. Downloading data takes time. There is no getting around that performance issue. Devisinga method of downloading, ie. when to download data and when not to, can greatly improve the queryingprocess. Presently, the LOP Manager would download every time it gets data from-a remote database.The downloading can be tumed off manually in the configuration menu. This means that the databaseadministrator has to know what he wants to download and then turn on the downloading mechanism.This process should be more automated in the future.

One last issue that the LOP Manager should address is that of parallel processing query requests. Thisissue has not been touched in this implementation, but should be looked into in the future. CIS/TKsystem is a single-user system at present. For CIS/TK to become multi-user, at some level, whether itbe at the AOP, GOP, LOP Manager level, parallel processing is needed. First, the level at which toparallel process must be decided. Some of the problems to be encountered in parallel processing atthe LOP Manager level are that the LOP Manager would need to know the user that sent the query.The LOP Manager would need to keep the queries from the various users separate. A method ofparallel processing must also be decided, whether to use a queue to simulate parallel processing, or totime-share. Another question is how would time-sharing affect the retrieval of information from remotedatabases?

Page 21


REFERENCES

[CHA 88] Champlin, A., "Interfacing Multiple Remote Databases in an Object-Oriented Framework",Bachelor's Thesis, MIT, May 1988. Z,

[DEV 88] Devlin, B.A., Murphy, P.T.,"An Architecture for a Business and Information System", IBM

Systems Journal, VOL 27, NO 1. , 1988

[GAN 88] Gan, F., "An Architecture Design and Implementation of a Communication Server forDisparate Databases.", B.S. Thesis, MIT, 1989.

[GER 89] Gerber, H., "Optimizing Information Retrieval For Disparate Menu Driven Database Systems.",B.S. Thesis, MIT, 1989.

[LEV 87] Levine S., "Interfacing Objects and Databases", M.S. Thesis, MIT, 1987.

[MAK 90] Makatiani, B., "Enhancement of the Communication Server in the Composite InformationSystem/Tool Kit (CIS/TK) Prototype", B.S. Thesis, MIT, 1990.

[RIG 90] Rigaldies, B., "The Design of a Data Representation Conversion Facility in the CompositeInformation System/Tool Kit Prototype", M.S. Thesis, MIT, 1990.

[WON 89] Wong, T.K., "Data Connectivity for the Composite Information system/Tool Kit", B.S. Thesis,MIT, 1989.

Page 22

LOP Manager in the Composite Information Systems/Tool Kit Page 23

APPENDIX A Sample of LOP Manager and Downloading facility

LOP MANAGER

Here is a trace of what happens when a query is sent to a remote database and a communications linkerror occurs and how the LQP Manager recovers. This is a sample run of a quejy from the CIS/TKsystem.

Note: explanation of the dribble is in italics and the user choice in the menu is in bold.

* WELCOME TO THE MIT PLACEMENT ASSISTANT SYSTEM ***

1. Alumni Information2. Company Recruiting Information3. Financial Information4. Alumni-Recruiting-Financial Information5. Private Portfolio Information6. Exit To CIS/TK Menu7. Quit

CHOOSE: 5

* Private Portfolio Information *

1. View Portfolio2. Find Alumni-Recruiting-Financial Info.3. Find Company's Headquartered In Same State As Client4. Exit to CIS/TK Menu5. Return to PAS Menu

CHOOSE: 3

Company's Headquartered In Same State As Client

Input Client's Name or ? (eg. S. Madnick, TK) : S. Madnick

Sending Query to CIS/TK ...(JOIN (SELECT PORTFOLIO (CSTATE) WHERE (= CLIENT-NAME "S. Madnick"))

(SELECT COMPANY (NAME TELEPHONE EXCHANGE)) ON SAMESTATE)

The GQP parses the query into two subqueries, one to the portfolio database to look up the state of theclient S. Madnick. The other query takes the result of the first and queries for the company name,telephone number and exchange of the companies in that state.

So the first query is to the portfolio database.

PARSING QUERY...

ROUTING INTERMEDIATE QUERY...

ENTERING LQP MANAGER ...



The LOP Manager looks in the ACTIVE_LOP table for the currently selected LOP for the portfolio set.It is the local database portfoliodb.

The portfoliodb returns the state name of the client S. Madnick.

The GOP now sends the second query to the LOP Manager with the data from the first query. Thelqpset is disclosureset, and the selected-lqp is disclosuredb, the remote database.


LOP Manager sends the query to the remote database.

CANNOT REACH LQP: disclosuredb

ERROR: connect-diff

The LOP sends back an error ***UNABLE_TO_MAKE_CONNECTION*** in the datalist. This meansthat the remote database is inaccessible due to communications link error. The LOP Manager updatesthe status of the datalinedb LOP from "ok" to "connect-diff".

LOOKING FOR ALTERNATE LQP ....

Then the LOP Manager proceeds to locate an alternate database. It queries for all the LOPs that are"ok" in the LOP SET disclosure set. Once it has them, it selects the one with the highest priorityamong them. This is the default LOP.

The user is shown a menu at this time so that he may either select a LOP from the ones that are up oruse the default LOP selected. The query is then sent to the alternate database. This menu can beturned off using the CIS/TK configuration menu before querying begins. (Appendix B: Menu System)

The default LOP is the LOP with the highest priority that has a status of "ok". Since there is only oneLOP whose status is "ok", it has to be the default LOP.

LQP STATUS PRIORITY SELECTION

disclosuredb connect-diff 1 selectedpseudo-disclosure-mit2e ok 2 default

*** CIS/TK SYSTEM LQP SELECTION MENU ***

1. Select another LQP2. Use Default3. Abort query

CHOOSE: 2

Page 24


CSTATE NAME TELEPHONE

AMERICAN SCIENCE + ENGINEERING INC 617-868-1600BOLT BERANEK + NEWMAN INC 617-873-2000DEC 508-493-5111ELECTRO POWERPACS CORP 617-876-9220INTERMETRICS INC 617-661-1840POLAROID CORP 617-577-2000IMMUNOGEN INC 617-661-9312DYNAGEN INC 617-491-2527RISE TECHNOLOGY INC 617-547-7272T CELL SCIENCES INC 617-621-1400

Enter y to continue and q to quit -- > y

CSTATE NAME TELEPHONE

SPINNAKER SOFTWARE CORPMOLECULON INCADVANCED MAGNETICS INCSPRINGBOARD SOFTWARE INCREPLIGEN CORPNORTHEASTERN MORTGAGE CO INCHOLOMETRIX INCBOSTON TECHNOLOGY INC

617-494-1200617-577-9900617-497-2070617-494-1200617-225-6000617-864-5500617-868-8050617-225-0500

Downloading facility

Here is a trace of what happens to a query to a remote database when the downloading facility is on.The query is in the form of the abstract local query from the GOP. The following query will download datafrom the IPSharp database. Some data will be inserted, some will be updated.

* WELCOME TO THE MIT PLACEMENT ASSISTANT SYSTEM ***

1. Alumni Information2. Company Recruiting Information3. Financial Information4. Alumni-Recruiting-Financial Information5. Private Portfolio Information6. Exit To CIS/TK Menu7. Quit

CHOOSE: 3

* FINANCIAL INFORMATION FROM I.P.SHARP & REUTERS ***

1. Find by Company Name (North American)2. Find by Company Name (EEC)3. Find by Company Name (International)

MAMAMAMAMAMAMAMAMAMA

EXCHANGE

AMSNYSNYSOTHNMSNYSNDQOTHNDQNMS

MAMAMAMAMAMAMAMA

EXCHANGE

NMSNDQNMSOTHNMSAMSOTHNDQ

Page 25


4. Find by Zip Code (North American)5. Exit to CIS/TK Menu6. Return to PAS Menu

CHOOSE: 1

North American Company Financial Information

Input Company Name or ? (eg. FORD MOTOR CO, CITICORP) : CITICORP

Sending Query to CIS/TK ...(SELECT FINANCE (COMPANY-NAME EXCHANGE REVENUE PROFIT PERIODENDING)

WHERE (= COMPANY-NAME "CITICORP"))

PARSING QUERY...



The LQP Manager would first look up the selected lqp for the disclosureset. It is the remote IPSharpdatabase. The primary key for the static table in the IPSharp database can be found in the attributecatalogue for disclosure under the slot for timeseries.

primary key for the static table = (get-object'disc-attributes-catalogue'static)

= (compno co) ; company number andcompany name

Once the keys are found, then the column list (co ex) in the abstract local query is checked to see if itcontains the key(s). The columns list is missing the compno attribute. Therefore, it is appended to theend of the columns list and the query is sent to the remote database.

Call to 'lqpdriver.c':lqpdriver I.P.SHARP - N. American Financial Database SLOAN: ... ... h19IRRELEVANT isql disclosure - SELECT CO, EX, COMPNO FROM STATIC WHERE CO -

'CITICORP'I.P. Sharp LQP - Query Received: SELECT CO, EX, COMPNO FROM STATIC WHERE CO= 'CITICORP'Parse display columns; check for static and time-series data.CO SEX SCOMPNO SConvert SQL condition into I.P. Sharp syntax condition.Condition received: CO = 'CITICORP'Condition transformed: (CO EQ CITICORP)Build a query for each static fact and one query for all the time-seriesfacts...IPSharp LQP - Queries generated:DISPLAY '(CO EQ CITICORP)' ADISCLOSURE 'CO'DISPLAY '(CO EQ CITICORP)' ADISCLOSURE 'EX'DISPLAY '(CO EQ CITICORP)' ADISCLOSURE 'COMPNO'

Page 26


The communication script is created in /usr/cistk/mtung/lqpman/tmp/ipsharpThe old communication script is removed.Years string:Old ipresults file is removed.rm: /usr/cistk/mtung/lqpman/tmp/ipresults non-existentThe communication server 'thesisl' is invoked.Connected

DISPLAY '(CO EQ CITICORP)' ADISCLOSURE 'CO'

CITICORPCITICORP ACCEPTANCE COMPANY INC

DISPLAY '(CO EQ CITICORP)' ADISCLOSURE 'EX'NYSOTH

DISPLAY '(CO EQ CITICORP)' ADISCLOSURE 'COMPNO'158020683

DisconnectedReading the file ipresults.'lqpdriver.c' of IP Sharp LQP: Started reading result file...Results from I.P.Sharp are being filtered.Filtering is done.

The data that comes back has two formats: one for the GOP, the other for the back-up database.

Note the difference in the format of the date between the two sets of data. The formatted version iswhat the GQP requests while the unformatted data is the format of the data at the remote database.

The LOP Manager checks that the selected-lqp is for a remote database. The first time that the LOPManager checks to see if the selected-lqp is for a remote database is to decide whether to insert primarykeys or not. If the data is from a remote database, then the pseudo database is looked up in the LOP'sobject slot under :pseudo-db-name and the data would be downloaded. In this case, the pseudodatabase is pseudo-disclosure-mit2e.

DOWNLOADING REMOTE DATA INTO LOCAL DATABASE...

The unformatted data list is passed to the DOWNLOAD-DATA module. First, the size of the pseudodatabase must not exceed the predetermined size. In CISITK, the limit for the pseudo databases hasbeen set at 1000 rows. Therefore, a query is sent to check the size of the pseudo database.

number of rows = (send-message 'pseudo-disclosure-mit2e:get-data (static ("count (*) "))

= 100

The size variable is important only when an insert function is performed.Now, for each row in the list, the key values are sent to the pseudo database to test for existence.

(send-message 'pseudo-disclosure-mit2e :get-data(static (compno co)

((= compno "1580")(= co "CITICORP"))))

Page 27


No data is returned; therefore, the row does not exist in the pseudo database; therefore, the row isreordered into the correct sequence and inserted into the insert-list and the next row is processed. Butfirst, the size of the pseudo database must not exceed the predetermined size. Therefore, if thenumber of rows (100) is not greater than 1000, then the insert function is processed. Also to do aninsert function, the values of the columns must be in the correct order that the table needs.

At the beginning of the downloading procedure, the get-columns method is csiled to retrieve thecolumns of the table in order. Then the data in the data list is sorted accordingly before inserting thedata into the insert-list.

Reordering (CO EX COMPNO) into (CO COMPNO CURR AD1 AD2 AD3 CY ST ZP TEINC TS EX EM COUNTRY SC PC DEl DE2DE3) ...

Reordering (CO EX COMPNO) into (CO COMPNO CURR AD1 AD2 AD3 CY ST ZP TEINC TS EX EM COUNTRY SC PC DEl DE2DE3) ...

After processing all the rows, the insert-list is checked to see if it is empty. If it is not, then the data in theinsert-list is written to a file and LOADED into the STATIC table in the pseudo database.

INSERTING ROWS INTO PSEUDO DATABASE2 row(s) loaded.

The second part of the query queries the time-series table in the disclosuredb. The columns are ns andni. The keys are found for the time-series table. They are compno and cf. These names are insertedinto the projection list of the query so that data for these two attributes are present for downloading.


ENTERING LQP MANAGER ...Call to 'lqpdriver.c':lqpdriver I.P.SHARP - N. American Financial Database SLOAN:.... ... h19IRRELEVANT isql disclosure - SELECT NS, NI, CF, COMPNO FROM TIMESERIESWHERE (COMPNO = '1580') OR (COMPNO = '20683')I.P. Sharp LQP - Query Received: SELECT NS, NI, CF, COMPNO FROM TIME SERIESWHERE (COMPNO = '1580') OR (COMPNO = '20683')Parse display columns; check for static and time-series data.NS TNI TCF TCOMPNO SConvert SQL condition into I.P. Sharp syntax condition.Condition received: (COMPNO = '1580') ORCondition transformed: ((COMPNO EQ 1580)) ORCondition received: (COMPNO = '20683')Condition transformed: ((COMPNO EQ 1580)) OR ((COMPNO EQ 20683))Build a query for each static fact and one query for all the time-seriesfacts...IPSharp LQP - Queries generated:DISPLAY '((COMPNO EQ 1580)) OR ((COMPNO EQ 20683))' ADISCLOSURE 'COMPNO'DISPLAY '((COMPNO EQ 1580)) OR ((COMPNO EQ 20683))' ADISCLOSURE 'NSNICF'The communication script is created in /usr/cistk/mtung/lqpman/tmp/ipsharpThe old communication script is removed.rm: /usr/cistk/mtung/lqpman/tmp/ipsharp non-existentYears string:Old ipresults file is removed.

Page 28


rm: /usr/cistk/mtung/lqpman/tmp/ipresults non-existentThe communication server 'thesis1' is invoked.rm: /usr/cistk/mtung/lqpman/tmp/tocomm non-existentrm: /usr/cistk/mtung/lqpman/tmp/fromcomm non-existentConnected

1985 1986 1987 1988LINE 1 22,504,000 23,386,000 27,881,000 32,02;4,000LINE 2 998,000 1,028,000 -1,182,000 1,858,000LINE 3 19,851,231 19,861,231 19,871,231 19,881,231LINE 4LINE 5LINE 6 19,880,915

[mit2e]

DisconnectedReading the file ipresults.rm: /usr/cistk/mtung/lqpman/tmp/iptemp non-existent'lqpdriver.c' of IP Sharp LQP: Started reading result file ...Results from I.P.Sharp are being filtered.Filtering is done.

DOWNLOADING REMOTE DATA INTO LOCAL DATABASE ...

UPDATING ROWS IN PSEUDO DATABASE

The same procedure occurs as in the first query to the static table. Thisquery will demonstrate the update function of the Informix SQL. The row ischecked to see if it already exists in the table.

(send-message 'pseudo-disclosure-mit2e :get-data(time-series (compno cf)

((= compno "1580 ")(= cf "19,581,231"))))

Data is returned; therefore, the row exists already in the pseudo database and an update function isused to download the new data.

(send-message 'pseudo-disclosure-mit2e :update-data(time series

((= ns "22,504,000" )(= ni "998,000"))

(and (= compno "1580")(= cf "19,851,231"))))

For the next row in the data list, the same procedure is used.

The results of the query:

COMPANY-NAME CITICORPEXCHANGE NYSREVENUE 22,504,000PROFIT 998,000PERIODENDING 19,851,231

Page 29


COMPANY-NAMEEXCHANGEREVENUEPROFITPERIODENDING





CITICORPNYS23, 386, 0001,028,00019, 861,231

CITICORPNYS27,881,000-1,182,00019, 871,231

CITICORPNYS32,024,0001,858,00019, 881, 231

CITICORP ACCEPTANCE COMPANY INCOTHNILNIL19, 880, 915

CITICORP ACCEPTANCE COMPANY INCOTHNILNILNIL

Page 30


APPENDIX B : Menu System

The following is sample of some of the menu functions added for the LOP Manager.This first menu shows two new selections (Numbers 5 and 6).

Number 5 is a toggle switch that turns the user menu in the LOP Manager OFF. It is the menu thatprompts the user for an alternate LOP. When this menu is turned off, then the defgult LOP is selectedautomatically.

Number 6 turns ON/OFF the downloading facility in CIS/TK. When this is turned off, no data isdownloaded from remote databases. This would speed up the query process.

*** CIS/TK SYSTEM CONFIGURATION MENU ***

1. Summary of Current Configuration2. Switch Synonym Facility (ON/OFF)3. Switch Translation Facility (ON/OFF)4. Switch User Database Selection (ON/OFF)5. Manual Selection of alternate LQP When

Selected Database Unavailable (ON/OFF)6. Download data from queries (YES/NO)7. LQP Configuration8. Trace Mode9. Exit

CHOOSE: 7

This next menu asked for the set of tables you want to update or query. The LQP-INIT tables are theinitialization tables. The LOP-STATUS is a copy of the LQP-INIT tables. LQP-STATUS is the set oftables that is used during query processing. LOP-INIT is only used for initialization values.

A change in one set of tables will NOT be reflected in the other set. The purpose of having two tables isso that there is always a consistent set of values to initialize the status tables with.

*** CIS/TK SYSTEM LQP CONFIGURATION MENU ***

WHICH OF THE FOLLOWING DATABASES DO YOUWANT TO WORK WITH?

LQP-STATUS IS THE WORKING DATABASELQP-INIT IS THE STABLE DATABASE

1. LQP-STATUS2. LQP-INIT3. Exit

CHOOSE: 1


MAIN LOP CONFIGURATION MENU

*** CIS/TK SYSTEM LQP CONFIGURATION MENU **

1. Show status of LQPS2. Update priority, machine-name of LQPs3. Add a LQP4. Delete a LQP5. Select a LQP6. Update status of LQPs7. Exit

CHOOSE: 1

CHOICE 1: MENU THAT DISPLAYS STATUS OF LOPS

*** DISPLAY STATUS OF LQPS ***

1. Show status of all the LQPs2. Show status of all LQPs in one LQPSET3. Show status of one LQP4. Display all LQPs that are currently up5. Display all LQPs that are currently down6. Display selected LQPs7. Exit

CHOOSE: 1

LQP STATUS -MACHINE

alumni-mit2e ok mit2ealumni-mit2a ok mit2apseudo-dataline-mit2e ok mit2edatalinedb ok finsburydisclosuredb ok IPSharppseudo-disclosure-mit2e ok mit2eportfoliodb ok mit2erecruit-ibmrt ok donnerpseudo-recruit-mit2e ok mit2e

HIT RETURN TO CONTINUE

LQP Manager in the Composite Information Systems/Tool Kit Page 33

CHOICE 6: SUB-MENU TO DISPLAY SELECTED LOP

*** DISPLAY STATUS OF LQPS ***

1. Show status of all the LQPs2. Show status of all LQPs in one LQPSET3. Show status of one LQP4. Display all LQPs that are currently up5. Display all LQPs that are currently down6. Display selected LQPs7. Exit

CHOOSE: 6

LQPSET SELECTEDLQP

alumni set alumni-mit2eoracle set recruit-ibmrtdisclosure set disclosuredbdataline set datalinedbportfolio-set portfoliodb


CHOICE 2: UPDATING VALUES IN THE STATUS TABLES

* CIS/TK SYSTEM LQP CONFIGURATION MENU *


CHOOSE: 2

The following menu updates priority, status, machine name of LQPs.


*** ATTRIBUTE OF LQP TO BE UPDATED *

1. Priority of a particular LQPSET2. Status3. Machine name4. Exit

CHOOSE: 1

LQPSET

alumni-setoracle-setdisclosure setdataline setportfolioset

NAME OF LQP SET YOU WANT TO MODIFY (with double quotes):"oracle-set"

LQP PRIORITY

recruit-ibmrt 1pseudo-recruit-mit2e 2

ENTER THE CORRESPONDING PRIORITY FOR THE LQP

LQP

recruit-ibmrtpseudo-recruit-mit2e

PRIORITY

-- > 2-- > 1

INSTALL UPDATES? (Y or N): y

If you don't want to change the priorities, then just type 'n' when asked if you want to install the updates.

ADDING A LOP

*** CIS/TK SYSTEM LQP CONFIGURATION MENU *

1. Show status of LQPS2. Update priority, machine-name of LQPs3. Add a LQP

CHOICE 3:

Page 34


4. Delete a LQP5. Select a LQP6. Update status of LQPs7. Exit

CHOOSE: 3

*** ADDING A LQP ***

1. Help Message (what you need before adding a lqp)2. Add a LQP within an existing LQPSET3. Add a LQPSET4. Exit

CHOOSE: 1

YOU NEED TO KNOW THE FOLLOWING BEFORE ADDING A LQP

* NAME OF NEW LQP* NAME OF ALL THE LQPSETS THAT THE LQP WILL BE A PART OF* A CORRECTLY WRITTEN DUMMY QUERY IN THE FORM OF THE

ABSTRACT LOCAL QUERY* THE PRIORITY LIST FOR THE LQP SETS MODIFIED* KNOW WHETHER YOU WANT TO CHANGE THE SELECTED LQP

OF THE LQPSETS THAT YOU MODIFIED


The dummy query is needed by the LOP Manager to be put into the hash table for the recheckmechanism. This query is assumed by the LOP Manager to be correct. It is the database administrator'sresponsibility to make sure that this query is in the form needed by the new LOP.

*** ADDING A LQP ***

1. Help Message (what you need before adding a lqp)2. Add a LQP within an existing LQPSET3. Add a LQPSET4. Exit

CHOOSE: 2

NAME OF NEW LQP (with double quotes) : "alumni-mit2e"

LQP ALREADY EXISTS, TRY AGAIN.

NAME OF NEW LQP (with double quotes): "dummy-mit2e"

MACHINE WHICH NEW DB IS ON (with double quotes): "mit2e"

STATUS OF NEW LQP (i.e. "ok" "connect-diff" "db-diff"): "ok"

WHICH LQPSET DOES IT BELONG(use double quotes) -- > "alumniset"



ENTER A SIMPLE VALID QUERY TO BE SENTTO THE NEW LQP. THIS WILL BE USED TO TESTTHE LQP BY THE LQP MANAGER.

QUERY (in abstract query format)ex. (tablename (coll col2) (and (= col3 "min") (= col4 "shu")))

QUERY: (dummyl (name address) (name = "sam"))

ARE YOU SURE? (Y or N): Y

UPDATING PRIORITY FOR LQPSET alumni-set

The above is the dummy query needed by the recheck mechanism. This query is placed into the hashtable under the index of the new LOP.

LQP PRIORITY

alumni-mit2e 1alumni-mit2a 2dummy-mit2e 0

ENTER THE CORRESPONDING PRIORITY FOR THE LQP

LQP PRIORITY

alumni-mit2e -- > 1alumni-mit2a -- > 2dumrmy-mit2e -- > 3

TRY AGAIN (T) or UPDATE (U) or QUIT (Q): U


alumni-mit2e ok 1 selectedalumni-mit2a ok 2dummy-mit2e ok 3

WOULD YOU LIKE TO MODIFY THE CURRENT SELECTION OF THE LQP (Y OR N)? y

NAME OF LQP TO SELECT (with double quotes) : "dumy-mit2e"



CHOICE 5: MENU TO SELECT A LOP

*** CIS/TK SYSTEM LQP CONFIGURATION MENU *


CHOOSE: 5

LQPSET

alumni setoracle-setdisclosure setdataline setportfolioset

NAME OF LQPSET TO SELECT (with double quotes): "alumni-set"


alumnni-mit2e ok 1alumni-mit2a ok 2

dummy-mit2e ok 3 selected

NAME OF LQP TO SELECT (with double quotes): "alumni-mit2a"


APPENDIX C Code

LOP MANAGER

The following procedures are what the :get-data;; method of each LQPSET calls. This method is the

main procedure that drives the LQP Manager.

.****** * *** * *****

;; MAIN PROCEDURE THAT DRIVES THE LQP MANAGER FOR ALUMNISET

(defun get-LQPM-ALUMNI-data (abs-local-query)(let ((data))

;; initalize variables(init-set-var "alumni set")(erase-new-lqp-var)(set-size)(format t "-3%ENTERING LQP MANAGER ... ~%")(setf data (LQPMrouter abs-local-query))(erase-set-var)(erase-new-lqp-var)data))

;; MAIN PROCEDURE THAT DRIVES THE LQP MANAGER FOR DISCLOSURESET.. ******* ** **** ***

(defun get-LQPM-DISCLOSURE-data (abs-local-query)(let ((data))

(init-set-var "disclosure-set")(erase-new-lqp-var)(setf new-query abs-local-query)(set-size)(format t "-3%ENTERING LQP MANAGER ... ~%")(setf data (LQPMrouter abs-local-query))(erase-set-var)(erase-new-lqp-var)data))

;; MAIN PROCEDURE THAT DRIVES THE LQP MANAGER FOR DATALINESET

(defun get-LQPM-DATALINE-data (abs-local-query)(let ((data))(init-set-var "dataline set")(erase-new-lqp-var)(setf new-query abs-local-query)(set-size)(format t "-3%ENTERING LQP MANAGER ...

(setf data (LQPMrouter abs-local-query))(erase-set-var)(erase-new-lqp-var)data))

Page 38


;; MAIN PROCEDURE THAT DRIVES THE LQP MANAGER FOR ORACLE SET

(defun get-LQPM-ORACLE-data (abs-local-query)(let ((data))(init-set-var "oracle set")(erase-new-lqp-var)(setf new-query abs-local-query)(set-size)(format t "-3%ENTERING LQP MANAGER ...

(setf data (LQPMrouter abs-local-query))(erase-set-var)(erase-new-lqp-var)data))

;; MAIN PROCEDURE THAT DRIVES THE LQP MANAGER FOR PORTFOLIO SET

(defun get-LQPM-PORTFOLIO-data (abs-local-query)(let ((data))

(init-set-var "portfolio_set")(erase-new-lqp-var)(setf new-query abs-local-query)(set-size)(format t "-3%ENTERING LQP MANAGER ...

(setf data (LQPMrouter abs-local-query))(format t "portfolio data = -A" data)(erase-set-var)(erase-new-lqp-var)data))

(defun LQPMrouter (abs-local-query)(let ((lqp)

(pseudo-db)(data)(new-query))

(setf new-query abs-local-query)

(loop

;; get selected lqp(setf lqp (get-lqp)) lqp is in symbol format

;; check to see if data should be downloaded(if (remote-db? lqp)(setf new-query (key-check lqp abs-local-query)))

;; send message to lqp(setf data (send-message lqp :get-data new-query))(format t "after send-message. data = -A" data)

;; test for errors(cond

((lqp-error? data)(progn

(format t "LQP reported an error")(return-from get-LQPM-ALUMNI-data)))

Page 39


((null (rest data))(return data))

((link-busy? (first (second data)))(progn (link-busy-menu)

(if (equal "abort" *new lqp_var*)(return nil))))

((link-down? (first (second data)))(progn(format t "Link down, data = -A" data)(find-alternate-lqp (string-downcase lqp) "connect-diff")(if (equal "abort" *new_lqp_var*)

(return nil))));return something to indicate abort??((remote-db-down? (first (second data)))(progn(find-alternate-lqp (string-downcase lqp) "db-diff")(if (equal "abort" *new_lqpvar*)

(return nil))))(t (progn

(format t "OK , data = -A" data)(if *download-lqp*

(progn(if (remote-db? lqp)

(progn(setf pseudo-db (get-object lqp 'pseudo-db-name))(download-pseudo-data data new-query lqp)))))

(return data)))))))

LOP Manager : Find Alternate and Recheck procedures

*************** ******** * ******** ***************

** These are the procedures that support the LQP Manager.. **

** April 5, 1990

Functions to manipulate global variables

The variable *set var* contains the current LQPSETThe variable *newlqpvar* contains the currently selected-lqpThe variable *query_userdefault* is the LQP Manager menu flagThe variable *download-lqp* is the flag for the downloading facility

(defun init-set-var (var)(setf *set var* var))

(defun erase-set-var ()(setf *set var* nil))

(defun init-new-lqp-var (var)(setf *newlqp-var* var))

(defun erase-new-lqp-var ()(setf *new-lqpvar* nil))

(defun set-size ()(setf *pseudo-db-size* 1000))

Page 40


(defun abort-query ()(setf *newlqpvar* "abort"))

(defun error-query-user ()(setf *query-user-default* t))

(defun error-dont-query-user ()(setf *query-user-default* nil))

(defun download-lqpman-data ()(setf *download-lqp* t))

(defun no-download-lqpman-data ()(setf *download-lqp* nil))

;; TESTS FOR ERROR IN REMOTE DB AND CONNECTION

(defun link-down? (var)(if (equal var "***UNABLETOMAKECONNECTION***")

tnil))

(defun link-busy? (var)(if (equal var "***ALLCONNECTIONSBUSY***")

tnil))

(defun remote-db-down? (var)(if (equal var "***REMOTEDBERROR***")

tnil))

;; TEST TO SEE IF LQP IS FOR REMOTE DB

(defun remote-db? (lqp)(let ((pseudo-db (get-object lqp 'pseudo-db-name)))

(if (null pseudo-db)nilpseudo-db)))

;; GETTING THE SELECTED LQP IN THE LQPSET

;; Returns the LQP in symbol format

(defun get-lqp ()(let ((data (send-message 'lqp-status :get-data

*(active lqp selectedlqp(= lqpset *setvar*)))))

(cond((lqp-error? data)

(format t "LQP returned an error")) ;should not occur((null (rest data))


(format t "NO selected LQP for this LOPSET-%"))(t (read-from-string (first (second data)))))))

* **** ***** ****** *** **

;; DRIVER FOR LOCATING THE ALTERNATE LOP

Update the status of the LOP that is unavailable. Update its status;; for all occurences in thelqp- status table, not just the occurence;; in this particular LQP_SET

it updates the active-lqp table and returns nothing

;; RETURNS either "abort", "update-error", or nil

(defun find-alternate-lqp (lqpvar stat)(let* ((lqp_var2 (string-downcase lqpvar))

(data (send-message 'lqp-status :update-data(lqp status (= status ,stat)

(= lqp ,lqpvar2))))(datal (send-message 'lqp-status :get-data

(lqpstatus (lqp status)(= lqp set *setvar*))))

data2)(format t "-5%CANNOT REACH LOP: -A " lqp-var2)(format t "-2%ERROR: -A " stat)(format t "-2%LOOKING FOR ALTERNATE LQP...-%")(if (not (null data))(progn

(format t "error-no-update-%") ; CHANGE NOT UPDATED !!(abort-query)(return-from find-alternate-lqp)))

;; Check to see if there is at least one available lqp

(init-new-lqp-var (default-lqp))(if (null *newlqp_var*)(progn

;; Recheck status of unavailable LQPs(recheck-status-lqp-set lqpvar2)(init-new-lqp-var (default-lqp))(if (null *newlqpvar*)

(progn(format t "All LOPS are unavailable, query aborted -%")(abort-query))

(if *query-user-default*(lqpman-menu))))

(if *query-user-default*(lqpman-menu)))

(if (not (equal "abort" *new lqp-var*))(update-active-lqp))))

Else ask User for selection or use default

;; RECHECK THE STATUS OF THE LOPS

;; Rechecks the status of all lqps in the given lqpset

Page 42


(defun recheck-status-lqp-set (lqpvar)(let* ((datal (send-message 'lqp-status :get-data

'(lqpstatuslqp(= lqp set *setvar*))))

(data2 (remove '(,lqpvar) datal :test #' equal))result test lqp query)

(setf result nil)(dolist (x (rest data2) result)

(progn(setf lqp (read-from-string (first x)))(setf query (gethash lqp query-table))(setf test (send-message lqp :get-data query))(setf test (first (second test)))(cond ((remote-db-down? test)(update-status (first x) "db-diff"))

((link-down? test)(update-status (first x) "connect-diff"))

((lqp-error? data)(format t "LQP returned an error")) ; should not occur

(t (update-status (first x) "ok") nil))))))

Update all occurences of the LQP

(defun update-status (lqpvar status)(let ((data (send-message 'lqp-status

:update-data'(lqp status (= status ,status)

(= lqp ,lqpvar)))))(if (not (null data))(format t "ERROR: Status for LQP -A not updated -%" lqp_var))))

; CHANGE NOT UPDATED !!; PRINT ERROR MESSAGE BUT CAN IGNOR

;; FIND DEFAULT LQP

(defun default-lqp ()(let ((data (send-message 'lqp-status

:get-data(lqp_status(lqp priority)(and (= lqp_set *setvar*)

(= status "ok")))))lqplist result lqpl lqp2)

(setf lqplist (rest data))(setf result nil)(cond ((null lqp list)

(format t "NO default-lqp-%"))((equal 1 (length lqp list))(setf result (caar lqplist)))(t (dotimes (x (- (length lqplist) 1) result)

(setf lqpl (first lqp_list))(setf lqp2 (second lqplist))(if (string< (second lqp1) (second lqp2))

(progn (setf lqplist (cons lqpl (cddr lqplist)))(setf result (first lqpl)))

(progn (setf lqplist (cons lqp2 (cddr lqp list)))(setf result (first lqp2)))))))

(values result)))

Page 43


Additional methods written for the maintenance of the status tables

* ** ******** ** **** *** *** *** ** **

** FILE: lqp-tables.lsp **

By Min Tung (March 1990)As part of MIT Undergrad. Thesis

Addition to INFORMIX QUERY PROCESSOR FOR MACHINE MIT2E (AT MIT SLOAN SCHOOL)

' THREE NEW METHODS ARE ADDED TO THE QUERY PROCESSORS FOR THE UPKEEPING OF

THE TWO STATUS TABLES NEEDED BY THE LQP MANAGER.

The LQP-STATUS and the ACTIVE-LQP objects responds to the following; messages:

=> :self-info=> :get-tables=> :get-columns <table-name>=> :get-data <cis/tk-single-query>

; NEW METHODS:=> :update-data <table-name, update, conds>=> :insert-data <table-name, insert-data>=> :delete-data <table-name, conds>

;; This update procedure is also used by the downloading facility to update data in;; the pseudo-databases.

(defun update-INFORMIX2E-LQPM-data (abs_local_query)(let ((SQL (form-SQL-update abs localquery)))

(lqp-print 'normal "SQL query to be sent to DBMS .... -%-A-%" SQL)(write-byte 7 *standard-output*)(let ((result (connect (get-current-object) SQL))

(commondir (get-self 'common-dev-lqp-directory)))(lqp-print 'normal "Result file after conversion to 'standard' -

form ...

(lqp-print-file 'normal result)(read-standard-table result commondir))))

(defun insert-INFORMIX2E-LQPM-data (abs_localquery)(let ((SQL (form-SQL-insert abs localquery)))

(lqp-print 'normal "SQL query to be sent to DBMS .... -%-A-%" SQL)(write-byte 7 *standard-output*)(let ((result (connect (get-current-object) SQL))

(commondir (get-self 'common-dev-lqp-directory)))(lqp-print 'normal "Result file after.conversion to 'standard' ~

form ...


(defun delete-INFORMIX2E-LQPM-data (abs_localquery)(let ((SQL (form-SQL-delete abs localquery)))(lqp-print 'normal "SQL query to be sent to DBMS .... ~%-A-%" SQL)(write-byte 7 *standard-output*)(let ((result (connect (get-current-object) SQL))

Page 44


(commondir (get-self 'common-dev-lqp-directory)))(lqp-print 'normal "Result file after conversion to 'standard' ~

form ... ~%")


SQL PARSING PROCEDURES FOR THE STATUS LOPS

S** FILE: SQL.LSP **

By Min TungAs part of MIT Undergrad. Thesis

;; MODIFICATIONS:

I added a form-sql-update command for an sql update command

CIS/TK STANDARD LOCAL QUERY TO SQL QUERY STRING TRANSLATION ROUTINES

; - - - - - - - - - - - - - - - - - - - - - - - - - - - ---------- - --- ;

This procedure parses the query passed to the LQP to make sure the;; format is correct before generating the SQL form of the update

query to send to the database

The update format of the query in abstract query language is:(table (= column value) conds)

It can modify more than one column at a time.(table ((= colnl vl) (= coln2 v2) (= coln3 v3) ... ) conds)

(defun form-SQL-update (query)(let ((table (parse-SQL-tname (car query)))

(conds (if (not (null (caddr query)))(parse-SQL-conds (caddr query))))

(updatecond (parse-SQL-update-cond (cadr query))))(cond ((or (equal table 'ERROR)

(equal conds 'ERROR)(equal updatecond 'ERROR))

(lqp-print 'verbose"FORM-SQL -- Error detected. No query returned.-%")

(values nil))((null conds)(values (format nil "UPDATE -A SET -A" table update cond)))

(t (values (format nil "UPDATE -A SET -A WHERE -A"table update cond conds))))))

(defun parse-SQL-update-cond (conds)(cond ((null conds) (lqp-print 'verbose

"PARSE-SQL-UPDATE -- No update data.-%"))((atom conds) (lqp-print 'verbose

"PARSE-SQL-UPDATE -- Search condition -A in -

improper form. ABORTING -%" conds)

'ERROR)((listp conds)

Page 45


(cond ((and (equal (length conds) 3) (eq (car conds) '=))(format nil "-A -A -A" (second conds)

(first conds)(parse-SQL-col-or-lit (third conds))))

((equal 1 (length conds))

(parse-SQL-update-cond (car conds)))((listp (car conds))

(let ((carx (parse-SQL-update-cond (car conds)))(cdrx (parse-SQL-update-cond (cdr conds))))

(values (format nil "-A, -A" carx cdrx))))(t (lqp-print 'verbose

"PARSE-SQL-UPDATE -- Incorrect form: -

-A ABORTING.-%" conds)'ERROR)))

(t (lqp-print 'verbose"PARSE-SQL-UPDATE -- Couldn't interpret statements ~A. ~

ABORTING.-%" conds)

'ERROR)))

Insert the values into the tableFormat: (table name (vl v2 v3 v4 ... ))

any value that are characters should be surroundedby quotation marks.

(defun form-SQL-insert (query)(let ((table (parse-SQL-tname (car query)))

(vallistl (cadr query))val list2)

(cond ((null val listl)(lqp-print 'verbose

"SQL-insert -- No values to insert into table.-%"))((listp vallistl)(setq vallist2 (parse-SQL-insert vallistl))(values (format nil "INSERT INTO -A VALUES (-A)" table val list2)))

(t (lqp-print 'verbose"PARSE-SQL-insert -- Can't interpret insert list -A. -

ABORTING.-%" val listl)'ERROR))))

(defun parse-SQL-insert (val list)(cond ((null vallist) (lpq-print 'verbose

"No argument in list-%"))((atom val list)(let ((vallist2 (parse-SQL-col-or-lit vallist)))

(format nil "-A" val list2)))((listp val list)

(if (equal 1 (length val list))(parse-SQL-insert (car val list))

(let ((carx (parse-SQL-insert (car val list)))(cdrx (parse-SQL-insert (cdr val list))))

(format nil "-A, -A" carx cdrx))))(t (lqp-print 'verbose

"PARSE-SQL-INSERT -- Can't interpret list -A. ~ABORTING.-%" val list)

'ERROR)))

Page 46


;; Delete row(s) from the table;; Format: (table-name conds)

(defun form-SQL-delete (query)(let ((table (parse-SQL-tname (car query)))

(conds (if (not (null (cdr query)))(parse-SQL-conds (cadr query)))))

(cond ((or (equal table 'ERROR)

(equal conds 'ERROR))(lqp-print 'verbose

"FORM-SQL-DELETE -- Error detected.-%")

(values nil))((null conds)(lqp-print 'verbose

"FORM-SQL-DELETE -- Error detected.-%"))(t (values (format nil "DELETE FROM -A WHERE -A"

table conds))))))

LOP MANAGER MENU

The Menu inside the LQP Manager that prompts;; the user for an alternate LQP when the remote

LQP is not accessible

;; MAIN-MENU: ASK USER TO SELECT A LQP OR CHOOSE DEFAULT

(defun lqpman-menu ()(display-lqp-status-priority-default *setvar* *new lqpvar*)(print-menu " *** CIS/TK SYSTEM LQP SELECTION MENU

'(("Select another LQP" select-lqp)("Use Default" dummy)("Abort query" abort-query)

(defun dummy ()(return-from dummy))

gets all the lqps in *setvar* and all its valid lqps and calls;; lqp-select-menu (which should put a new value into *newlqpvar*)

(defun select-lqp ()(let* ((data (send-message 'lqp-status

:get-data '(lqpstatus(lqp status)(= lqpset *set-var*))))

(lqps (rest data))(head (first data))(valid-lqps (send-message 'lqp-status

:get-data(lqp statuslqp(and (= status "ok")

(= lqp set *set var*))))))

Page 47


(lqp-select-menu lqps valid-lqps head)))

Updates the active-lqp table with *newlqp_var* and *set var*

(defun update-active-lqp ()(let ((data (send-message 'lqp-status

:update-data(activelqp (= selectedlqp ,*newlqp_var*)

(= lqp_set ,*setvar*)))))(if (not (null data))(format t "ERROR: Status for LQP -A not updated -%" *new lqpvar*))))

CHANGE NOT UPDATED !!PRINT ERROR MESSAGE BUT CAN IGNOR

Gets the selected lqp and puts the value in *new_lqp_var*

(defun lqp-select-menu (lqps valid-lqps header)(let (result)

(clear-input)(format t "-18,0T * CIS/TK SYSTEM LQP MENU(format t "-16,0T CHOOSE A LQP FROM THE LQP SET -A -%-%" *set var*)(format t "-22,0T -A -0,48T -B -%" (first header) (second header))(format t "-18,0T * -0,48T ******~%%")(dolist (x lqps)

(format t "-18,OT -A -0,50T -B -%" (first x) (second x))

(format t "-%-%-10,0T SELECTED LQP (i.e. \"alumni-mit2e\"): ")(setf result (read *terminal-io*))

(cond ((null (find result valid-lqps :test #'equal :key #'car))(progn (format t "-%-%-10,0T NOT A VALID LQP SELECTION -%-%")(format t "-10,OT HIT ANY KEY TO TRY AGAIN -%")(wait-for)(lqp-select-menu lqps valid-lqps header)))

Check that selected lqp is valid(t (init-new-lqp-var result)))))

(defun link-busy-menu ()(print-menu " *** CIS/TK SYSTEM: ALL LINES ARE BUSY ***"

'(("Try query again" dummy)("Abort query" abort-query)

(defun display-lqp-status-priority-default (set var default-lqp)(let* ((display (send-message 'lqp-status :get-data

(lqpstatus (lqp status priority)(= lqp set ,set-var))))

(displayl (rest display))(data (send-message 'lqp-status :get-data

(active lqp (selectedlqp)(= lqpset setvar))))

(selected-lqp (first (second data)))(listl)(result))

(setf list1 nil)(setf result nil)

Page 48


(dolist (lqp-list displayl result)(cond ((equalp selected-lqp (first lqp-list))(setf list1 (append lqp-list (list "selected"))))

((equalp default-lqp (first lqp-list))(setf listl (append lqp-list (list "default"))))

(t (setf listl (append lqp-list (list "")))))(setf result (append result (list listl))))

(format t "-4%")(new-print (append

'(((X LQP) (X STATUS) (X PRIORITY) (X SELECTION)))result))))

HASH-TABLE

This file initalizes the hash table with the correct;; format of the dummy query'

(setf query-table (make-hash-table :rehash-size 5:rehash-threshold 2 :size 20))

(setf (gethash 'alumni-mit2e query-table)'(alumnitb (zipcode) (= sequencenum"1958873946")))

(setf (gethash 'alumni-mit2c query-table)'(alumnitb (sequencenum) (= siccode "371")))

(setf (gethash 'recruit-ibmrt query-table)'("companytbl" (companyname) (= companyname "Valeo SA")))

(setf (gethash 'pseudo-recruit-mit2e query-table)'("companytbl" (companyname) (= companyname "Chemical Bank")))

(setf (gethash 'disclosuredb query-table) '(static (co) (= zp "1754")))

(setf (gethash 'pseudo-disclosure-mit2e query-table)'(static (co) (= zp "1754")))

(setf (gethash 'datalinedb query-table)'(data (code yr) (and (= yr "88") (= code "reut"))))

(setf (gethash 'pseudo-dataline-mit2e query-table)'(data (code yr) (and (= yr "88") (= code "reut"))))

(setf (gethash 'portfoliodb query-table)'(cistkportf (companyname) (= clientname "TK")))

DOWNLOADING FACILITY

The following procedures run the downloading facility

;; COUNT THE NUMBER OF ROWS IN THE TABLE

Page 49


(defun table-size (lqp table)(let* ((data (send-message lqp :get-data '(,table "count (*)")))

(size (first (second data))))(read-from-string size)))

KEY-CHECK: CHECK TO SEE IF THE PRIMARY KEY FOR THE TABLETHAT THE QUERY IS SENT TO IS IN THE PROJECTION LIST.IF NOT, PUT THEM IN

(defun key-check (selected-lqp abs-local-query)(let* ((table-name (first abs-local-query))

(primary-key (get-object(get-object(get-object selected-lqp 'data-type-catalogue)'attributes-catalogue)table-name))

(conds (third abs-local-query))(columns (second abs-local-query))resultkeydatanew-query)

(setf result columns)(setf data nil)(dolist (key primary-key)

(setf data (find key columns :test #'equal)) find not right.(if (null data)

(setf result (append result (list key)))))(setf new-query '(,table-name ,result ,conds))new-query))

DOWNLOAD-PSEUDO-DATA: DOWNLOAD THE DATALISTINTO THE SPECIFIEDPSEUDO-DB

(defun download-pseudo-data (datalist new-query selected-lqp)(let* ((data (rest datalist))

(table (first new-query))(table-name (string-downcase (format nil "-A" table)))(columns (second new-query))(pseudo-db (get-object selected-lqp 'pseudo-db-name))(primary-key (get-object

(get-object(get-object pseudo-db 'data-type-catalogue)'attributes-catalogue)table))

(all-columns-in-order (send-message pseudo-db :get-columns table))(all-columns-in-order (mapcar #'(lambda (x)

(read-from-string (first x)))(rest all-columns-in-order)))

(t-size (table-size pseudo-db table))(script-file (get-object 'downloadfacility 'script-file))(database-directory (get-object pseudo-db 'database-directory))(database-name (get-object pseudo-db 'database))(tmp-files-directory (get-object pseudo-db 'tmp-files-directory))(unique-file-name (get-object 'download-facility 'tmp-file-name))

Page 50


(data-file-name (concatenate 'string tmp-files-directory "/"unique-file-name))

(insert-list)(conds)(tmp)

(p)(result)(key-conds)(count-conds))

(format t "-5%DOWNLOADING REMOTE DATA INTO LOCAL DATABASE...(format t "-%-% UPDATING ROWS IN PSEUDO DATABASE")(dolist (row data result)

(setf conds nil)(setf count-conds 0)(dolist (key primary-key)(setf count-conds (+ count-conds 1))(setf p (position key columns))(setf key-conds (nth p row))(if (> count-conds 1)

(setf conds (append '(and) (list conds)(list (= ,key ,key-conds))))

(setf conds (append conds (= ,key ,key-conds)))))(setf data (send-message pseudo-db :get-data

'(,table ,primary-key ,conds)))(if (and (null (rest data))

(< t-size *pseudo-db-size*))(progn

(setf tmp (reorder-data-list (append (list columns)(list row))

all-columns-in-order))(setf insert-list (append insert-list tmp))(setf t-size (+ t-size 1)))

(update-data-pseudo-db row columns pseudo-db table conds)))(if (not (null insert-list))(progn(write-into-file data-file-name insert-list)(format t "-% INSERTING ROWS INTO PSEUDO DATABASE ")(system (unix-format "-A -A -A -A -A" script-file

data-file-name database-directorydatabase-name table-name))))))

(defun update-data-pseudo-db (row header lqp table key-conds)(let ((update-conds)

(data))(setf update-conds (mapcar #'(lambda (x y) '(= ,x ,y))

header row))(setf data (send-message lqp :update-data

'(,table ,update-conds ,key-conds)))(if (not (null data))(format t "-%-% ERROR: NO UPDATE! "))))

SCRIPT FILE USED TO INSERT DOWNLOADED DATA

# This is the script file that will load data from the# specified file into the specified table and database.

# Min Tung -- for use in the downloading facility to insert# data.

Page 51


# May 9, 1990

# Script to download data# $1 = data file name (with directory path name)# $2 = database directory# $3 = database name# $4 = table name

cd $2

echo LOAD FROM $1 INSERT INTO $4 | isql $3

GLOBAL SCHEMA

Placement Assistance System

;;;; MIT PLACEMENT OFFICE GLOBAL SCHEMA5 entities: alumni, company, finance, portfolio, and position5 relationships: worksfor, worksas, financeinfo, recruiting_for,

;;; contains.

Updated on June 29, 1989.The entity FINANCE is mapped to the I.P. Sharp LQP and Finsbury Dataline

Updated on May 5, 1990 for the installation of LQPSETs;;; IPSharp LQP now has two tables: static, time-series

BEGIN SCHEMA DIFINITION(create-schema mit-placement)

;;; ENTITIES

;;; ALUMNI(create-entity alumni

:attributes ((sequence-num (alumni-set alumnitbsequencenum)

(alumni-set sicnumtbsequencenum))

(first-name (alumni set alumnitb first name))(last-name (alumni-set alumnitb last-name))(degree (alumniset alumnitb degree))(address-1 (alumni s^bt alumnitb address 11))(address-2 (alumni set alumnitb address 12))(address-3 (alumni set alumnitb address_13))(zip-code (alumniset alumnitb zipcode))(position-code (alumni-set alumnitb

positioncode)(alumniset positiontb

positioncode))(position (alumni-set positiontb

positionname))(company (alumniset alumnitb companyname))(country-code (alumniset alumnitb

country_code)(alumni-set countrytb

country_code))(country-name (alumniset countrytb

country_name))(sic (alumni set sicnumtb sic-code)))

:table-relations ((merge (alumniset alumnitb)(alumniset positiontb) on

Page 52


(= (alumniset alumnitbpositioncode)

(alumniset positiontbpositioncode)))

(merge (alumniset countrytb)(alumni set alumnitb) on( (alumniset countrytb

country_code)(alumni set alumnitb

country_code)))(merge (alumniset sicnumtb)

(alumni set alumnitb) on(= (alumni-set sicnumtb

sequencenum)(alumni set alumnitb

sequencenum))))

;;; COMPANY

(create-entity company:attributes ((name (oracleset "companytbl"

companyname)(disclosure set static co)(dataline set data code))

(address-1 (disclosure set static AD1))(address-2 (disclosure set static AD2))(address-3 (disclosure set static AD3))(country (datalineset data country))(telephone (disclosure-set static te))(exchange (disclosureset static ex))(employees (disclosureset static em))(zip-code (disclosureset static ZP))(ts (disclosure set static TS))(position (oracleset "companytbl" position))(date (oracle-set "companytbl" visitday)))

:table-relations ((merge (disclosureset static)(oracleset "companytbl")on (= (disclosure-set static

co)(oracle set"companytbl"companyname)))

(merge (disclosureset static)(dataline set data)on (= (disclosure set static co)

(dataline set data code))))

disclosuredb is an LQP instance of the I.P. Sharp LQPdatabase: disclosure.datalinedb is an LQP instance of the Finsbury Dataline LQPdatabase: data

Note:There is no currency fact in I.P. Sharp disclosure; it is implicitlyUS $There is no scale fact in I.P. Sharp. There is a command that can beused to set a scale: SCALE n. The default value is 1.The currency and scale in Dataline database appears in the table output.

The currency is consistent with each company's country of incorporation.

Page 53


NOTATION:I.P. Sharp:

;; ns : net salesni : net incomecf : financial reporting date (yyyy)lafd: lastest annual financial date (yyyymmdd)

Dataline:;; sales : sales;; efo : earned for ordinary

periodending: period ending (dd-mm-yy);; year ; year

(create-entity finance:attributes ((company-name (disclosureset static co)

(dataline set data code))(ts (disclosure set static TS))(zip-code (disclosureset static ZP))(exchange (disclosureset static ex))(country (datalineset data country))(revenue (disclosure set time series ns)

(dataline set data sales))(profit (disclosureset timeseries ni)

(dataline set data efo))(year (disclosureset timeseries enda)

(datalineset data yr))(periodending (disclosureset timeseries cf)

(datalineset data periodending))):table-relations ((merge (disclosure set static)

(dataline set data)on (= (disclosure set static co)

(dataline set data code)))(merge (disclosureset static)

(disclosure-set time-series)on(= (disclosureset static compno)(disclosureset timeseries compno))))

;;; PORTFOLIOWe voluntarily use a limited portfolio database (5 rows) for

;; debuggage purpose.;; the table is called "cistkportf" in the database portfdb,

(MIT2E, /usr/cistk/global/findb).portfolio is an LQP instance of the informix-2e LQP

(create-entity portfolio:attributes((companyname (portfolioset cistkportf companyname))

(primary_ind (portfolio set cistkportf primary_ind))(secondary ind (portfolioset cistkportf secondind))(client-name (portfolio set cistkportf clientname))(cstate (portfolioset cistkportf clientstate))))

;; POSITION(create-entity position

:attributes((title (oracleset "companytbl" position))

(status (oracleset "companytbl" status))(num_position (oracleset "companytbl" schedule)))

Page 54


;;; INDUSTRY(create-entity industry

:attributes((industry-name (alumniset siccodetb industry))

(sic (alumni-set siccodetb sic code))))

;;; RELATIONS

;;; WORKS FOR

(create-relation works for:entity-from alumni:entity-to company:join (= (alumni company)

(company name)))

;;; FINANCE INFO(create-relation finance-info

:entity-from company:entity-to finance:join (= (company name)

(finance company-name)))

;;; RECRUITING FOR(create-relation recruitingfor

:entity-from company:entity-to position:join (= (company position)

(position title)))

;;; CONTAINS

(create-relation contains:entity-from portfolio:entity-to company:join (= (portfolio companyname)

(company name)));;; worksin

(create-relation works in:entity-from alumni:entity-to industry:join (= (alumni sic)

(industry sic)))

;;; same state

(create-relation same state:entity-from portfolio:entity-to company:join (= (portfolio cstate)

(company zip-code)))

Page 55


STATUS TABLE OBJECTS AND LOP SET OBJECTS

Min, 02/15/90instance of Informix2e: lqp status table

;; See if you can create a subclass so that you won't have to removeall the methods from informix-2e

Presently, using put-object in a methods slot for instances will disable;; the methods from its SUPERIOR to be carried into this instance. Therefore,;; I have re-put all the necessary methods for these two instances.

(create-instance 'informix-2e 'lqp-status)(put-object(put-object(put-object(put-object(put-object(put-object(put-object(put-object(put-object(put-object(put-object(put-object(put-slottable.lsp")

'lqp-status'lqp-status'lqp-status'lqp-status'lqp-status'lqp-status'lqp-status'lqp-status'lqp-status'lqp-status'lqp-status'lqp-status'lqp-status

'database "status")'database-directory "/usr/cistk/mtung/db")'comm-server-directory "/usr/cistk/mtung/lqpman/informix")'common-dev-lqp-directory "/usr/cistk/mtung/lqpman/common")'tmp-files-directory "/usr/cistk/mtung/lqpman/tmp")'methods '(:update-data update-INFORMIX2E-LQPM-data))'methods '(:insert-data insert-INFORMIX2E-LQPM-data))'methods '(:delete-data delete-INFORMIX2E-LQPM-data))'methods '(:self-info display-INFORMIX2E-self-info))'methods '(:get-tables get-INFORMIX2E-tables))'methods '(:get-columns get-INFORMIX2E-columns))'methods '(:get-data get-INFORMIX2E-data))'hash-table-directory "/usr/cistk/mtung/Setup/query-hash-

(create-instance 'informix-2e 'lqp-init)'database "init")'database-directory "/usr/cistk/mtung/db")'comm-server-directory "/usr/cistk/mtung/lqpman/informix")'common-dev-lqp-directory "/usr/cistk/mtung/lqpman/common")'tmp-files-directory "/usr/cistk/mtung/lqpman/tmp")'methods'methods'methods'methods'methods'methods'methods

'(:update-data update-INFORMIX2E-LQPM-data))'(:insert-data insert-INFORMIX2E-LQPM-data))'(:delete-data delete-INFORMIX2E-LQPM-data))'(:self-info display-INFORMIX2E-self-info))'(:get-tables get-INFORMIX2E-tables))'(:get-columns get-INFORMIX2E-columns))'(:get-data get-INFORMIX2E-data))

(make-object 'alumni set('methods '(:get-data

(make-object 'oracleset('methods '(:get-data

(make-object 'disclosure set('methods '(:get-data

(make-object 'dataline set('methods '(:get-data

(make-object 'portfolio set('methods '(:get-data

(make-object 'download-facility

get-LQPM-ALUMNI-data)))

get-LQPM-ORACLE-data)))

get-LQPM-DISCLOSURE-data)))

get-LQPM-DATALINE-data)))

get-LQPM-PORTFOLIO-data)))

('script-file "/usr/cistk/mtung/lqpman/d-script")('tmp-file-name "lqpman-download"))

Page 56

(put-object(put-object(put-object(put-object(put-object(put-object(put-object(put-object(put-object(put-object(put-object(put-object

'lqp-init'lqp-init'lqp-init'lqp-init'lqp-init'lqp-init'lqp-init'lqp-init'lqp-init'lqp-init'lqp-init'lqp-init


LOP CONFIGURATION MENU PROCEDURES

MODIFIED CIS/TK CONFIGURATION MENU

;;;; +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++;;;;

;;;; CIS/TK V3.0 KERNEL;;;; CONFIGURATION MENU;;;; JULY 14, 89 - tk

;;;; +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ;;;;

;;;; This fil'e implements the system configuration menu for selecting;;;; various system switches. The menu has the following capabilities:;;;; 1. SWITCH SYNONYM FACILITY ON/OFF;;; 2. SWITCH TRANSLATION FACILTIY ON/OFF;;; 3. SWITCH USER DATABASE SELECTION ON/OFF;;; 4. SEITCH USER DATABASE SELECTION ON/OFF;;; 5. MANUAL SELECTION OF ALTERNATE LQP WHEN SELECTED

DATABASE UNAVAILABE ON/OFF;;; 6. DOWNLOAD DATA FROM QUERIES YES/NO

7. LQP CONFIGURATION;;; 8. TRACE MODE

;;; 9. EXIT

;;; CONFIGMENU

(defun configmenu ()(print-menu " *** CIS/TK SYSTEM CONFIGURATION MENU ***"

'(("Summary of Current Configuration" current-config)("Switch Synonym Facility (ON/OFF)" switch-synonym)("Switch Translation Facility (ON/OFF)"switch-translation)("Switch User Database Selection (ON/OFF)"switch-selection)("Manual Selection of alternate LQP When

Selected Database Unavailable (ON/OFF)"switch-lqpman-default)("Download data from queries (YES/NO)" switch-download)("LQP Configuration" lqp-configure-menu)("Trace Mode" trace-menu)("Exit" main-menu)

CURRENT-CONFIG;;; shows a summary of everything(defun current-config ()

(print-submenu "Summary Of Current Configuration")(format t "=========== =======%

(format t " Synonym Switch On : -A-%" (getsynswitch))(format t " Translation Switch On : -A-%" (gettransswitch))(format t " User Selection Switch On : -A~%" *user-switch*)(format t " Current Trace Mode : -A-%" *current-trace-mode*)(format t " User Option in LQP Error : -A-%" (get_lqpmanswitch))(format t " Download data from query : -A-%" (getdownloadlqpmanswitch))(format t "-% HIT RETURN TO CONTINUE ")

(read-char)(configmenu)

Page 57


;;; SWITCH-SYNONYM(defun switch-synonym ()

(switch-syn)(configmenu))

SWITCH-TRANSLATION(defun switch-translation ()

(switch-trans)(configmenu))

;;; SWITCH-SELECTION(defun switch-selection ()

(switch-select)(config_menu))

;;; SWITCH-LQPMAN-DEFAULT(defun switch-lqpman-default ()

(switch-lqpman-def)(config_menu))

LQP-CONFIGURE-MENU;;; located in lqp-config.lsp

;;; trace mode

;;;; locate in trace.lsp

(defvar *query-user-default* t)(defvar *download-lqp* t)

(defun getlqpmanswitch ()(if *query-user-default*

"ON"

"OFF"))

(defun switch-lqpman-def ()(if *query-user-default*

(error-dont-query-user)(error-query-user)))

;; SWITCH-DOWNLOAD

(defun switch-download ()(switch-down)(configmenu))

(defun switch-down ()(if *download-lqp*

(no-download-lqpman-data)(download-lqpman-data)))

(defun getdownload lqpman_switch ()(if *download-lqp*

"ON"

"OFF"))

Page 58


LOP CONFIGURATION MENU

;;; lqp_config.lsp;;;; this file handles the lqp configuration menu

;;;; This contains the main menu of the LQP Configuration;;;; and any necessary support procedures and global variables

modified by Min Tung --- April 12, 1990

(defun init-db-name (var)(setf *mod-db* var))

(defun init-db-lqp-status ()(setf *mod-db* 'lqp-status))

(defun init-db-init ()(setf *mod-db* 'lqp-init))

(defun reset-db-name ()(setf *mod-db* nil))

(defun lqp-configure-menu ()(format t "-9%")(print-menu " *** CIS/TK SYSTEM LQP CONFIGURATION MENU

WHICH OF THE FOLLOWING DATABASES DO YOUWANT TO WORK WITH?

LQP-STATUS IS THE WORKING DATABASELQP-INIT IS THE STABLE DATABASE "

'(("LQP-STATUS" init-db-lqp-status)("LQP-INIT" init-db-init)("Exit" configmenu)

(lqp-configure-menul))

;;;; prints out the configuration menu;; LQP-CONFIGURE-MENU

(defun lqp-configure-menul ()(print-menu " *** CIS/TK SYSTEM LQP CONFIGURATION MENU

'(("Show status of LQPS" lqp-view)("Update priority, machine-name of LQPs" lqp-update-2)("Add a LQP" lqp-add)("Delete a LQP" lqp-delete)("Select a LQP" lqp-select)("Update status of LQPs" lqp-recheck)("Exit" configmenu)

(defun display-status-selection (lqp lqp set)(let* ((display (send-message lqp :get-data

(lqpstatus (lqp status)(= lqp_set ,lqpset))))

(displayl (rest display))(data (send-message lqp :get-data

(active lqp (selectedlqp)(= lqp set ,lqp-set))))

(selected-lqp (first (second data)))(listl)

Page 59


(result))(setf result nil)(setf listl nil)(dolist (lqp-list displayl result)

(if (equalp selected-lqp (first lqp-list))(setf listl (append lqp-list '("selected" ,lqp_set)))

(setf listl (append lqp-list '("" ,lqpset))))(setf result (append result (list listl))))

(new-print (append'(((X LQP) (X STATUS) (X SELECTION) (X LQPSET)))result))))

(defun display-lqp-status-priority (set var)(let* ((display (send-message 'lqp-status :get-data

'(lqp_status (lqp status priority)(= lqp_set ,setvar))))

(displayl (rest display))(data (send-message 'lqp-status :get-data

(active lqp (selected lqp)(= lqpset ,setvar))))

(selected-lqp (first (second data)))(listl)(result))

(setf listl nil)(setf result nil)(dolist (lqp-list displayl result)

(if (equalp selected-lqp (first lqp-list))(setf list1 (append lqp-list (list "selected")))

(setf listl (append lqp-list (list ""))))(setf result (append result (list listl))))


'(((X LQP) (X STATUS) (X PRIORITY) (X SELECTION)))result))))

(defun display-all-lqp-set (lqp)(let ((display (send-message lqp :get-data

'(active lqp lqpset))))(format t "-4%")(new-print (append

'(((X LQPSET)))(rest display)))))

(defun display-all-lqps (lqp)(let* ((display (send-message lqp :get-data

'(lqpstatus (lqp status machine))))(display2 (remove-duplicates (rest display) :test #'equal)))


'(((X LQP_SET) (X STATUS) (X MACHINE)))display2))))

(defun display-lqp-machine (lqp)(let* ((display (send-message lqp :get-data

' (lqpstatus (lqp machine))))(display2 (remove-duplicates (rest display))))


'(((X LQP) (X MACHINE)))

Page 60


(rest display2)))))

(defun get-all-lqp-set (lqp)(let* ((display (send-message lqp :get-data

'(active_lqp lqp set)))(display2 (rest display)))

display2))

(defun update? ()(let ((choice))

(format t "~%~% INSTALL UPDATES? (Y or N):(setf choice (read *terminal-io*))(equalp 'y choice)))

(defun pause-here ()(format t "-5,OT HIT RETURN TO CONTINUE ")(read-char)(read-char)

(defun get-all-lqps (lqp)(let* ((data (send-message lqp :get-data

'(lqpstatus lqp)))(datal (remove-duplicates (rest data) :test #'equal)))

datal))

(defun display-lqp-priority (setvar)(let ((display (send-message *mod-db* :get-data

(lqpstatus (lqp priority)(= lqp set ,setvar))))

result)(setf result nil)(format t "-4%")(new-print (append

'(((X LQP) (X PRIORITY)))(rest display)))

(rest display)))

SHOW STATUS MENU

;; SHOW STATUS

(defun lqp-view ()(print-menu " *** DISPLAY STATUS OF LQPS ***"

'(("Show status of all the LQPs" status-all)("Show status of all LQPs in one LQPSET" status-set)

("Show status of one LQP" status_lqp)("Display all LQPs that are currently up" statusup)("Display all LQPs that are currently down" statusdown)("Display selected LQPs" currently-selected-lqp)("Exit" lqp-configure-menul)


(defun status-all ()(let* ((data (send-message *mod-db* :get-data

'(lqp-status (lqp status machine))))(data2 (remove-duplicates (rest data) :test #'equal)))


'(((X LQP) (X STATUS) (X MACHINE)))data2))

(pause-here)(lqp-view)))

(defun currently-selected-lqp ()(let ((data (send-message *mod-db* :get-data

'(active lqp (lqpset selected lqp)))))(format t "-4%")(new-print (append

'(((X LQPSET) (X SELECTEDLQP)))(rest data)))


(defun status set ()(let ((choice)

(reply)(lqpset (get-all-lqp-set *mod-db*)))

(display-all-lqp-set *mod-db*)(loop(format t "-5,OT INPUT NAME OF LQPSET (with double quotes):(setf choice (read *terminal-io*))(if (null (find choice lqpset :test #'equal :key #'car))(format t "-%-5,0T NOT A VALID LQP SET. -%")(return)))

(setf reply (send-message *mod-db* :get-data(lqpstatus (lqp status priority machine)

(= lqpset ,choice))))(format t "~4%-5,OT LQPS IN THE LQPSET -A " choice)(new-print (append

'(((X LQP_NAME) (X STATUS) (X PRIORITY) (X MACHINE-NAME)))(rest reply)))


(defun statuslqp ()(let ((choice)

(reply)(lqps (get-all-lqps *mod-db*)))

(display-all-lqps *mod-db*)(loop(format t "-5,OT INPUT NAME OF LQP (with double quotes): ")

(setf choice (read *terminal-io*))(if (null (find choice lqps :test #'equal :key #'car))(progn (format t "-%-5,0T NOT A VALID LQP. -%")(pause-here))

(return)))(setf reply (send-message *mod-db* :get-data

(lqpstatus (lqp status machine)(= lqp ,choice))))


Page 62


'(((X LQP NAME) (X STATUS) (X MACHINE-NAME)))(rest reply)))


(defun statusup ()(let* ((display (send-message *mod-db* :get-data

'(lqp status (lqp status machine)

(= status "ok"))))(display2 (remove-duplicates (rest display) :test #'equal)))


'(((X LQPSET) (X STATUS) (X MACHINE-NAME)))display2))


(defun status-down ()(let* ((display (send-message *mod-db* :get-data

'(lqpstatus (lqp status machine)(not (= status "ok")))))

(display2 (remove-duplicates (rest display) :test #'equal)))(format t "-4%")(new-print (append

'(((X LOP SET) (X STATUS) (X MACHINE-NAME)))display2))


UPDATE PRIORITY. STATUS. MACHINE NAME MENU

;; UPDATE LOPe **** ** *

(defun lqp-update C)(let ((choice))

(format t "-4%-5,OT WHICH SET OF TABLES WOULD YOU LIKE TO UPDATE (LQP-STATUS or INIT)-2%")

(format t "-5,OT Note : LQP-STATUS is the current and temporary status tables -%")(format t "-5,OT while INIT is the stable initialization tables. ~%~%")(format t "-5,OT Update will be reflected only in the specified table-%")(loop(format t "-%-5,0T INPUT NAME OF TABLE (with double quotes): ")(setf choice (read *terminal-io*))(if (null (find choice '("INIT" "LQP-STATUS") :test #'equalp))(format t "-%-5,0T NOT A VALID TABLE CHOICE.-%")(return)))

(init-db-name (read-from-string choice) Y(lqp-update-2)))

(defun lqp-update-2 C)(print-menu " *** ATTRIBUTE OF LOP TO BE UPDATED ***"

'C

("Priority of a particular LQP_SET" lqp-update-p)("Status" lqp-update-s)("Machine name" lqp-update-m)("Exit" lqp-configure-menul)


(defun lqp-update-s ()(let ((choicel)

(choice2)(reply)(lqps (get-all-lqps *mod-db*)))

(display-all-lqps *mod-db*)(loop(format t "-5,OT NAME OF LQP WHOSE STATUS YOU WANT -%")(format t "-5,OT TO UPDATE (with double quotes): ")(setf choicel (read *terminal-io*))(if (null (find choicel lqps :test #'equal :key #'car))(format t "-%-5,0T NOT A VALID LQP.-%")(return)))

(loop(format t "-2%-5,0T UPDATE STATUS (i.e. \"ok\" \"connect-diff\" \"db-diff\"): ")(setf choice2 (read *terminal-io*))(if (null (find choice2 '("ok" "connect-diff" "db-diff") :test #'equal))(format t "-%-5,0T NOT A VALID STATUS. -%")

(return)))(if (update?)(send-message *mod-db* :update-data

'(lqp_status (= status ,choice2)(= lqp ,choicel))))

(lqp-update-2)))

(defun lqp-update-m ()(let ((choicel)

(choice2)(reply)(lqps (get-all-lqps *mod-db*)))

(display-lqp-machine *mod-db*)(loop(format t "-5,OT NAME OF LQP WHOSE MACHINE YOU WANT -%")(format t "-5,OT TO UPDATE (with double quotes): ")(setf choicel (read *terminal-io*))(if (null (find choicel lqps :test #'equal :key #'car))(format t "-%-5,0T NOT A VALID LQP. -%")(return)))

(loop(format t "-%-% NEW MACHINE NAME (with double quotes):(setf choice2 (read *terminal-io*))(if (stringp choice2)(return)(format t "-%-5,0T ERROR - DOUBLE QUOTES NEEDED")))

(if (update?)(send-message *mod-db* :update-data

'(lqpstatus (= machine ,choice2)(= lqp ,choicel))))

(lqp-update-2)))

(defun lqp-update-p ()(let ((choice)

(reply)(replyl)(lqps-priority)


(result)(d)(lqpset (get-all-lqp-set *mod-db*)))

(display-all-lqp-set *mod-db*)(loop(format t "~%-5,OT NAME OF LQP SET YOU WANT TO MODIFY (with double quotes):(setf choice (read *terminal-io*))(if (null (find choice lqpset :test #'equal :key #'car))(format t "-%-5,0T NOT A VALID LQP. -%")(return)))

(setf lqps-priority (display-lqp-priority choice))(format t "-% ENTER THE CORRESPONDING PRIORITY FOR THE LQP-%")(format t "-% -15,OT LOP ~0,48T PRIORITY -%")(dolist (x lqps-priority result)

(format t "-% -11,OT -A" (first x))(format t "-0,48T--> ")(setf reply (read *terminal-io*))(setf reply (prinl-to-string reply))(setf result (append result (list reply))))

(if (update?)(progn

(dolist (x lqps-priority)(setf lqps (first x))(setf y (first result))(setf result (rest result))(send-message *mod-db* :update-data

(lqp status (= priority ,y)(and (= lqp ,lqps)

(= lqpset ,choice)))))))(lqp-update-2)))

ADD LOP MENU

ADD LQP

(defun lqp-add ()(print-menu " * ADDING A LQP ***"'

("Help Message (what you need before adding a lqp)" lqp-info)("Add a LQP within an existing LQPSET" lqp-add-lqp)("Add a LQP_SET" lqp-add-set)("Exit" lqp-configure-menul)

(defun lqp-info ()(format t "-5% YOU NEED TO KNOW THE FOLLOWING BEFORE ADDING A LQP")(format t "-2% * NAME OF NEW LQP")(format t "~-% * NAME OF ALL THE LQP SETS THAT THE LQP WILL BE A PART OF")(format t "~-% * A CORRECTLY WRITTEN DUMMY QUERY IN THE FORM OF THE")(format t "-% ABSTRACT LOCAL QUERY")(format t "% * THE PRIORITY LIST FOR THE LQP SETS MODIFIED")(format t "~% * KNOW WHETHER YOU WANT TO CHANGE THE SELECTED LQP")(format t "-% OF THE LQPSETS THAT YOU MODIFIED -2%")(pause-here)(lqp-add))

Page 65


(defun lqp-add-set ()(let ((lqp-set)

(test))(loop(format t "-14%-5,OT NAME OF NEW LQPSET (in double quotes): ")(setf lqp-set (read *terminal-io*))(if (not (stringp lqp-set))(format t "-%-5,0T ERROR - DOUBLE QUOTES NEEDED. ")(return)))

(loop(format t "-%-5,0T IS THE LQP TO BE ADDED NEW OR EXISTING (N or E or \"done\"): ")(setf test (read *terminal-io*))(cond ((equalp "done" test)

(return))((equalp 'n test)(lqp-new-add lqp-set))

((equalp 'e test)(lqp-existing-add lqp-set))

(t nil nil)))(lqp-update-p-add lqp-set)(lqp-set-select-add lqp-set)(lqp-add)))

;; Adds new row to the ACTIVE-LQP table

(defun lqp-set-select-add (lqp-set)(let ((lqp-name)

(lqps))(setf lqps (send-message 'lqp-status :get-data

(lqpstatus lqp(and (= lqp set ,lqp-set)

(= status "ok")))))(setf lqps (rest lqps))(display-lqp-status-priority lqp-set)(loop(format t "-%-%-5,0T NAME OF LQP TO SELECT (with double quotes):(setf lqp-name (read *terminal-io*))(if (null (find lqp-name lqps :test #'equal :key #'car))(format t "-%-5,0T NOT A GOOD LQP. -%")(return)))

(if (update?)(send-message *mod-db* :insert-data

' (activelqp (,lqp-set ,lqp-name))))))

(defun lqp-existing-add (lqp-set)(let ((lqp-name)

(data)(lqps (get-all-lqps *mod-db*)))

(display-all-lqps *mod-db*)(loop(format t "-5,OT NAME OF LQP YOU WANT TO ADD -%")(format t "-5,OT (with double quotes): ")(setf lqp-name (read *terminal-io*))(if (null (find lqp-name lqps :test #'equal :key #'car))(format t "-%-5,0T NOT A VALID LQP.-%")(return)))

(setf data (send-message *mod-db* :get-data(lqp_status (status machine)

(= lqp ,lqp-name))))(setf datal (second data))

Page 66


(setf status (first datal))(setf machine (second datal))

;; Add LOP to the LQPSTATUS table(if (update?)(send-message *mod-db* :insert-data

(lqpstatus(,lqp-set ,lqp-name ,status ,machine 0))))))

(defun lqp-dummy-query (lqp)(let ((query)

(test))(format t "-%-5,0T ENTER A SIMPLE VALID QUERY TO BE SENT")(format t "-%-5,0T TO THE NEW LQP. THIS WILL BE USED TO TEST")(format t "-%-5,0T THE LQP BY THE LQP MANAGER.")

(loop(format t "-%-%-5,0T QUERY (in abstract query format)")(format t "-%-5,0T ex. (tablename (coll col2) (and (= col3 \"min\") (- col4

\"shu\")))")(format t "-%-%-5,0T QUERY: ")(setf query (read *terminal-io*))(format t "-%-%-5,0T ARE YOU SURE? (Y or N): ")

(setf test (read *terminal-io*))(if (equalp 'y test)(progn

(if (equal *mod-db* 'lqp-init)(update-hash-table-perm query lqp)

(setf (gethash lqp query-table) query))(return))))))

(defun update-hash-table-perm (query lqp)(let ((output)

(hfile))(setf output '(setf (gethash ',lqp query-table) ',query))(setf hfile (get-object 'lqp-status 'hash-table-directory))(with-open-file (min-stream hfile :direction :output :if-exists :append)

(when (streamp min-stream)(write output :stream min-stream))))

(defun lqp-new-add (lqp-set)(let ((lqps)

(lqp-list (get-all-lqps *mod-db*))(machine-name)(status))(loop(format t "-%-5,0T NAME OF NEW LQP (with double quotes): ")(setf lqps (read *terminal-io*))(cond ((not (stringp lqps))

(format t "-%-5,0T ERROR - DOUBLE QUOTES NEEDED. "))((not (null (find lqps lqp-list :test #'equal :key #'car)))(format t "-%-5,0T LQP ALREADY EXISTS, TRY AGAIN. -%"))

(t (return))))(loop(format t "-%-5,0T MACHINE WHICH NEW DB IS ON (with double quotes):(setf machine-name (read *terminal-io*))(if (not (stringp machine-name))(format t "-%-5,0T ERROR - DOUBLE QUOTES NEEDED. ")

(return)))(loop(format t "-2%-5,OT STATUS OF NEW LQP (i.e. \"ok\" \"connect-diff\" \"db-diff\"):


(setf status (read *terminal-io*))(if (null (find status '("ok" "connect-diff" "db-diff") :test #'equal))(format t "-%-5,0T NOT A VALID STATUS. ~%")(return)))

(if (update?)(progn

(lqp-dummy-query (read-from-string lqps))(send-message *mod-db* :insert-data

'(lqp_status(,lqp-set ,lqps ,status ,machine-name 0)))))))

(defun lqp-add-lqp ()(let ((lqp-name)

(lqp-set)(lqp-set-list (get-all-lqp-set *mod-db*) )(lqp-list (get -all-lqps *mod-db*) )(choice)(machine-name)(status))

(loop(format t "-%-5,OT NAME OF NEW LQP (with double quotes): ")

(setf lqp-name (read *terminal-io*))(cond ((not (stringp lqp-name))

(format t "-%-5,0T ERROR - DOUBLE QUOTES NEEDED. "))((not (null (find lqp-name lqp-list :test #'equal :key #'car)))(format t "-%-5,0T LQP ALREADY EXISTS, TRY AGAIN. -%"))

(t (return))))(loop(format t "-%-5,OT MACHINE WHICH NEW DB IS ON (with double quotes) :(setf machine-name (read *terminal-io*))(if (not (stringp machine-name))(format t "-%-5,0T ERROR - DOUBLE QUOTES NEEDED. ")

(return)))(loop(format t "-2%-5,OT STATUS OF NEW LQP (i.e. \"ok\" \"connect-diff\" \"db-diff\"):(setf status (read *terminal-io*))(if (null (find status '("ok" "connect-diff" "db-diff") :test #'equal))(format t "-%-5,0T NOT A VALID STATUS.")(return)))

(loop(format t "-%-5,OT WHICH LQPSET DOES IT BELONG")(format t "-%-5,OT (use double quotes) -- > ")(setf lqp-set (read *terminal-io*))(if (null (find lqp-set lqp-set-list :test #'equal :key #'car))(format t "-%-5,0T NOT A VALID LQPSET. -%')(return)))

(if (update?)(progn

(lqp-dummy-query (read-from-string lqp-name))

;; Update LQP_STATUS table with new LQP(send-message *mod-db* :insert-data

(lqp_status(,lqp-set ,lqp-name ,status ,machine-name 0)))

;; Update priority(lqp-update-p-add lqp-set)

(display-lqp-status-priority lqp-set)


(format t "WOULD YOU LIKE TO MODIFY THE CURRENT SELECTION OF THE LQP (Y OR N)? ")(setf choice (read *terminal-io*))(if (equalp 'y choice)

(lqp-select-add lqp-set))))(lqp-add)))

Changes the selected-lqp in ACTIVE-LQP table

(defun lqp-select-add (lqp-set)(let ((choice2)

(lqps))(setf lqps (send-message 'lqp-status :get-data

(lqp_status lqp(and (= lqp_set ,lqp-set)

(= status "ok")))))(setf lqps (rest lqps))(loop(format t "-%-%-5,0T NAME OF LQP TO SELECT (with double quotes):(setf choice2 (read *terminal-io*))(if (null (find choice2 lqps :test #'equal :key #'car))(format t "-%-5,0T NOT A GOOD LQP. -%")(return)))

(if (update?)(send-message *mod-db* :update-data

(active_lqp (= selectedlqp ,choice2)(= lqpset ,lqp-set))))))

(defun lqp-update-p-add (lqp-set)(let ((choice)

(reply)(replyl)(lqps-priority)(result))

(format t "-%-5,0T UPDATING PRIORITY FOR LQPSET -A" lqp-set)(setf lqps-priority (display-lqp-priority lqp-set))(loop(format t "-% ENTER THE CORRESPONDING PRIORITY FOR THE LQP-%")(format t "-% -15,OT LQP -0,48T PRIORITY -%")

(dolist (x lqps-priority result)(format t "-% -11,OT -A" (first x))

(format t "-0,48T--> ")(setf reply (read *terminal-io*))(setf reply (prinl-to-string reply))(setf result (append result (list reply))))

(format t "-%-5,0T TRY AGAIN (T) or UPDATE (U) or QUIT (Q)(setf choice (read *terminal-io*))(cond ((equalp 't choice)

nil)((equalp 'q choice)(return))

((equalp 'u choice)(progn(dolist (x lqps-priority)

(setf lqps (first x))(setf y (first result))(setf result (rest result))(send-message *mod-db* :update-data

(lqpstatus

(= priority ,y)(and (= lqp ,lqps)

Page 70LOP Manager in the Composite Information Systems/Tool Kit

(= lqp set ,lqp-set)))))(return)))

(t nil nil)))))

(defun procede? ()(let ((choice))(format t "-%-5,0T TRY AGAIN (T) or UPDATE (U) or QUIT (Q): ")

(setf choice (read *terminal-io*))(cond ((equalp 't choice)

(return))((equalp 'u choice)t)((equalp 'q choice)(return-from lqp-update-p-add))

(t procede?))))

DELETE LOP MENU

;; DELETE A LQP

(defun lqp-delete ()(print-menu " * DELETE A LQP

("DELETE A LQP" lqp-delete-lqp)("DELETE A LQP_SET" lqp-delete-set)("Exit" lqp-configure-menul)

(defun lqp-delete-lqp ()(let ((lqps)

(lqp-list.(get-all-lqps *mod-db*)))(loop(format t "-%-5,0T NAME OF LQP TO BE DELETED ")(format t "-%-5,OT (with double quotes) or \"done\" TO EXIT: ")(setf lqps (read *terminal-io*))(cond ((equalp "done" lqps)

(return))((null (find lqps lqp-list :test #'equal :key #'car))(format t "-%-5,0T LQP DOES NOT EXISTS, TRY AGAIN. -%"))

(t (send-message *mod-db* :delete-data'(lqp_status (= lqp ,lqps))))))

(lqp-delete)))

(defun lqp-delete-set ()(let ((lqp-set)

(lqp-set-list (get-all-lqp-set *mod-db*)))(loop(format t "-%-5,0T NAME OF LQP SET TO BE DELETED ")(format t "-%-5,OT (with double quotes) or \"done\" TO EXIT: ")(setf lqp-set (read *terminal-io*))(cond ((equalp "done" lqp-set)

(return))((null (find lqp-set lqp-set-list :test #'equal :key #'car))(format t "-%-5,0T LQP_SET DOES NOT EXISTS, TRY AGAIN. -%"))

(t (progn(send-message *mod-db* :delete-data


'(lqp status (= lqp_set ,lqp-set)))(send-message *mod-db* :delete-data

(active_lqp (= lqp set ,lqp-set)))))))(lqp-delete)))

SELECT LOP MENU

;; SELECT A LQP

(defun lqp-select ()(let ((choicel)

(choice2)(reply)(lpqset)(lqps))

(setf lqpset (get-all-lqp-set 'lqp-status))(display-all-lqp-set 'lqp-status)(loop(format t "-%-%-3,0T NAME OF LQPSET TO SELECT (with double quotes): ")(setf choicel (read *terminal-io*))(if (null (find choicel lqpset :test #'equal :key #'car))(format t "-%-5,0T NOT A VALID LQP SET. -%")

(return)))(setf lqps (send-message 'lqp-status :get-data

(lqp_status lqp(and (= lqp_set ,choicel)

(= status "ok")))))(setf lqps (rest lqps))(display-lqp-status-priority choicel)(loop(format t "-%-%-5,0T NAME OF LQP TO SELECT (with double quotes):(setf choice2 (read *terminal-io*))(if (null (find choice2 lqps :test #'equal :key #'car))(format t "-%-5,0T NOT A GOOD LQP. -%")

(return)))(send-message 'lqp-status :update-data

(active_lqp (= selectedlqp ,choice2)(= lqp set ,choicel)))

(lqp-configure-menul)))

RECHECK LOP MENU

;; RECHECK STATUS OF LQPS

;; recheck only updates the status of the LQPs in the system.;; Its doesn't change the selection on the LQPSETs.;; The User must do that manually.

(defun lqp-recheck ()(print-menu " *** RECHECK THE STATUS OF UNAVAILABLE LQPS *

'(("By LQPs" recheck-lqp)("By LQPSETs" recheck-set)("All" recheck-all)

Page 71


("Exit" lqp-configure-menul)

(defun recheck-set ()(let ((lqp-set)

(lqpset (get-all-lqp-set *mod-db*)))(display-all-lqp-set *mod-db*)(loop(format t "-5,OT INPUT NAME OF LQP_SET (with double quotes):(setf lqp-set (read *terminal-io*))(if (null (find lqp-set lqpset :test #'equal :key #car))(format t "~%-5,OT NOT A VALID LQP SET. ~%")(return)))

(init-set-var lqp-set)(recheck-status-lqp-set lqp-set)(erase-set-var)(lqp-recheck)))

(defun recheck-all ()(let ((lqpset (get-all-lqp-set *mod-db*)))

(dolist (x lqpset)(init-set-var (first x))(recheck-status-lqp-set "i"l))

(erase-set-var)(lqp-recheck)))

(defun recheck-lqp ()(let ((lqps)

(lqp-list (get-all-lqps *mod-db*)))(display-all-lqps *mod-db*)(loop(format t "-%-5,0T NAME OF LQP TO BE RECHECKED (with double quotes):(setf lqps (read *terminal-io*))(if (null (find lqps lqp-list :test #'equal :key #'car))(format t "-%-5,0T NOT A VALID LQP SET. -%")(return)))

(recheck-status-lqp lqps)(lqp-recheck)))

(defun recheck-status-lqp (lqpvar) ; lqpvar has to be a string(let ((result)

(test)(query)(lqp (read-from-string lqp_var)))

(setf result nil)(setf query (gethash lqp query-table))(setf test (send-message lqp :get-data query))(cond ((remote-db-down? (first test))

(update-status lqp-var "db-diff"))((link-down? (first test))(update-status lqpvar "connect-diff"))

((lqp-error? data)(format t "LQP returned an error")) ; should not occur

(t (update-status lqpvar "ok") nil))))

Page 72

Date post:	14-Aug-2018
Category:	Documents
Upload:	dinhbao
View:	220 times
Download:	0 times

Local Query Processor Manager for the Composite ...web.mit.edu/smadnick/www/wp2/1990-18.pdf ·...

Documents