PL/SQL PL/SQL (Procedural Language/Structured Query Language) is Oracle Corporation's procedural extension language for SQL and the Oracle relational database. PL/SQL's general syntax resembles that of Ada. Introduction PL/SQL supports variables, conditions, loops and exceptions. Arrays are also supported, though in a somewhat unusual way, involving the use of PL/SQL collections. PL/SQL collections are a slightly advanced topic. Implementations from version 8 of Oracle Database onwards have included features associated with object-orientation. PL/SQL program units (essentially code containers) can be compiled into the Oracle database. Programmers can thus embed PL/SQL units of functionality into the database directly. They also can write scripts containing PL/SQL program units that can be read into the database using the Oracle SQL*Plus tool. Once the program units have been stored into the database, they become available for execution at a later time. While programmers can readily embed Data Manipulation Language (DML) statements directly into their PL/SQL code using straight forward SQL statements, Data Definition Language (DDL) requires more complex "Dynamic SQL" statements to be written in the PL/SQL code. However, DML statements underpin the majority of PL/SQL code in typical software applications. In the case of PL/SQL dynamic SQL, early versions of the Oracle Database required the use of a complicated Oracle DBMS_SQL package library. More recent versions have however introduced a simpler "Native Dynamic SQL", along with an associated EXECUTE IMMEDIATE syntax. 1
Transcript
PL/SQLPL/SQL (Procedural Language/Structured Query Language) is Oracle Corporation's procedural extension language for SQL and the Oracle relational database. PL/SQL's general syntax resembles that of Ada.
IntroductionPL/SQL supports variables, conditions, loops and exceptions. Arrays are also supported, though in a somewhat unusual way, involving the use of PL/SQL collections. PL/SQL collections are a slightly advanced topic.
Implementations from version 8 of Oracle Database onwards have included features associated with object-orientation.
PL/SQL program units (essentially code containers) can be compiled into the Oracle database. Programmers can thus embed PL/SQL units of functionality into the database directly. They also can write scripts containing PL/SQL program units that can be read into the database using the Oracle SQL*Plus tool.
Once the program units have been stored into the database, they become available for execution at a later time.
While programmers can readily embed Data Manipulation Language (DML) statements directly into their PL/SQL code using straight forward SQL statements, Data Definition Language (DDL) requires more complex "Dynamic SQL" statements to be written in the PL/SQL code. However, DML statements underpin the majority of PL/SQL code in typical software applications.
In the case of PL/SQL dynamic SQL, early versions of the Oracle Database required the use of a complicated Oracle DBMS_SQL package library. More recent versions have however introduced a simpler "Native Dynamic SQL", along with an associated EXECUTE IMMEDIATE syntax.
Oracle Corporation customarily extends package functionality with each successive release of the Oracle Database.
PL/SQL program units
Anonymous Blocks
Anonymous blocks form the basis of the simplest PL/SQL code, and have the following structure:
<<label>>DECLARE
TYPE / item / FUNCTION / PROCEDURE declarations
1
BEGIN StatementsEXCEPTION
EXCEPTION handlersEND label;
The <<label>> and the DECLARE and EXCEPTION sections are optional.
Exceptions, errors which arise during the execution of the code, have one of two types:1. Predefined exceptions2. User-defined exceptions.
User-defined exceptions are always raised explicitly by the programmers, using the RAISE or RAISE_APPLICATION_ERROR commands, in any situation where they have determined that it is impossible for normal execution to continue. RAISE command has the syntax:RAISE <exception name>;
Oracle Corporation has pre-defined several exceptions like NO_DATA_FOUND, TOO_MANY_ROWS, etc. Each exception has a SQL Error Number and SQL Error Message associated with it. Programmers can access these by using the SQLCODE and SQLERRM functions.
The DECLARE section defines and (optionally) initialises variables. If not initialised specifically, they default to NULL.
For example:
DECLARE number1 NUMBER(2); number2 NUMBER(2) := 17; -- value default text1 VARCHAR2(12) := 'Hello world'; text2 DATE := SYSDATE; -- current date and timeBEGIN SELECT street_number INTO number1 FROM address WHERE name = 'INU';END;
The symbol := functions as an assignment operator to store a value in a variable.
The major datatypes in PL/SQL include NUMBER, INTEGER, CHAR, VARCHAR2, DATE, TIMESTAMP, TEXT etc.
Functions
Functions in PL/SQL are a collection of SQL and PL/SQL statements that perform a task and should return a value to the calling environment. User defined functions are used to supplement the many hundreds of functions built in by Oracle.
2
There are two different types of functions in PL/SQL. The traditional function is written in the form:
CREATE OR REPLACE FUNCTION <function_name> [(input/output variable declarations)] RETURN return_type[AUTHID <CURRENT USER | DEFINER>] <IS|AS>
[declaration block]BEGIN
<PL/SQL block WITH RETURN statement> RETURN <return_value>;[EXCEPTION
EXCEPTION block] RETURN <return_value>;END;
Pipelined Table Functions are written in the form:
CREATE OR REPLACE FUNCTION <function_name> [(input/output variable declarations)] RETURN return_type[AUTHID <CURRENT USER | DEFINER>] [<AGGREGATE | PIPELINED>] <IS|USING>
[declaration block]BEGIN
<PL/SQL block WITH RETURN statement> PIPE ROW <return type>; RETURN;[EXCEPTION
There are three types of parameter: IN, OUT and IN OUT. An IN parameter is used an input only. An IN parameter is passed by reference and thus cannot be changed by the called program. An OUT parameter is initially NULL. The program assigns the parameter a value and that value is returned to the calling program. An IN OUT parameter may or may not have an initial value. That initial value may or may not be modified by the called program. Any changes made to the parameter are returned to the calling program by default by copying but, with the NOCOPY hint may be passed by reference.
Procedures
Procedures are similar to Functions, in that they can be executed to perform work. The primary difference is that procedures cannot be used in a SQL statement and although they can have multiple out parameters they do not "RETURN" a value.
Procedures are traditionally the workhorse of the coding world and functions are traditionally the smaller, more specific pieces of code. PL/SQL maintains many of the distinctions between functions and procedures found in many general-purpose programming languages, but in addition, functions can be called from SQL, while procedures cannot.
3
Packages
Packages are groups of conceptually linked Functions, Procedures,Variable,Constants & Cursors etc. The use of packages promotes re-use of code. Packages usually have two parts, a specification and a body, although sometimes the body is unnecessary. The specification (spec for short) is the interface to your applications; it declares the types, variables, constants, exceptions, cursors, and subprograms available for use. The body fully defines cursors and subprograms, and so implements the spec.
Variables
Numeric variables
variable_name NUMBER(P[,S]) := VALUE;
To define a numeric variable, the programmer appends the variable type NUMBER to the name definition. To specify the (optional) precision(P) and the (optional) scale (S), one can further append these in round brackets, separated by a comma. ("Precision" in this context refers to the number of digits which the variable can hold, "scale" refers to the number of digits which can follow the decimal point.)
A selection of other datatypes for numeric variables would include: binary_float, binary_double, dec, decimal, double precision, float, integer, int, numeric, real, smallint, binary_integer
Character variables
variable_name VARCHAR2(L) := 'Text';
To define a character variable, the programmer normally appends the variable type VARCHAR2 to the name definition. There follows in brackets the maximum number of characters which the variable can store.
Other datatypes for character variables include: varchar, char, long, raw, long raw, nchar, nchar2, clob, blob, bfile
Date variables
variable_name DATE := '01-Jan-2005';
Oracle provides a number of data types that can store dates (DATE, DATETIME, TIMESTAMP etc.), however DATE is most commonly used.
Programmers define date variables by appending the datatype code "DATE" to a variable name. The TO_DATE function can be used to convert strings to date values. The function converts the first quoted string into a date, using as a definition the second quoted string, for example:
To convert the dates to strings one uses the function TO_CHAR (date_string, format_string).
PL/SQL also supports the use of ANSI date and interval literals.[3] The following clause gives an 18-month range:
WHERE dateField BETWEEN DATE '2004-12-31' - INTERVAL '1-6' YEAR TO MONTH AND DATE '2004-12-31'
Datatypes for specific columns
Variable_name Table_name.Column_name%type;
This syntax defines a variable of the type of the referenced column on the referenced tables.
Programmers specify user-defined datatypes with the syntax:
type data_type is record (field_1 type_1 :=xyz, field_2 type_2 :=xyz, ..., field_n type_n :=xyz);
For example:
DECLARE TYPE t_address IS RECORD ( name address.name%TYPE, street address.street%TYPE, street_number address.street_number%TYPE, postcode address.postcode%TYPE); v_address t_address;BEGIN SELECT name, street, street_number, postcode INTO v_address FROM address WHERE ROWNUM = 1;END;
This sample program defines its own datatype, called t_address, which contains the fields name, street, street_number and postcode.
So according to the example, we are able to copy the data from the database to the fields in the program.
Using this datatype the programmer has defined a variable called v_address and loaded it with data from the ADDRESS table.
5
Programmers can address individual attributes in such a structure by means of the dot-notation, thus: "v_address.street := 'High Street';"
Conditional Statements
The following code segment shows the IF-THEN-ELSIF construct. The ELSIF and ELSE parts are optional so it is possible to create simpler IF-THEN or, IF-THEN-ELSE constructs.
IF x = 1 THEN sequence_of_statements_1;ELSIF x = 2 THEN sequence_of_statements_2;ELSIF x = 3 THEN sequence_of_statements_3;ELSIF x = 4 THEN sequence_of_statements_4;ELSIF x = 5 THEN sequence_of_statements_5;ELSE sequence_of_statements_N;END IF;
The CASE statement simplifies some large IF-THEN-ELSE structures.
CASE WHEN x = 1 THEN sequence_of_statements_1; WHEN x = 2 THEN sequence_of_statements_2; WHEN x = 3 THEN sequence_of_statements_3; WHEN x = 4 THEN sequence_of_statements_4; WHEN x = 5 THEN sequence_of_statements_5; ELSE sequence_of_statements_N;END CASE;
CASE statement can be used with predefined selector:
CASE x WHEN 1 THEN sequence_of_statements_1; WHEN 2 THEN sequence_of_statements_2; WHEN 3 THEN sequence_of_statements_3; WHEN 4 THEN sequence_of_statements_4; WHEN 5 THEN sequence_of_statements_5; ELSE sequence_of_statements_N;END CASE;
Array handling
PL/SQL refers to arrays as "collections". The language offers three types of collections:
1. Index-by tables (associative arrays)
6
2. Nested tables
3. Varrays (variable-size arrays)
Programmers must specify an upper limit for varrays, but need not for index-by tables or for nested tables. The language includes several collection methods used to manipulate collection elements: for example FIRST, LAST, NEXT, PRIOR, EXTEND, TRIM, DELETE, etc. Index-by tables can be used to simulate associative arrays, as in this example of a memo function for Ackermann's function in PL/SQL.
Looping
As a procedural language by definition, PL/SQL provides several iteration constructs, including basic LOOP statements, WHILE loops, FOR loops, and Cursor FOR loops.
LOOP statements
Syntax:
<<parent_loop>>LOOP
statements
<<child_loop>>LOOP
statementsEXIT parent_loop WHEN <condition>; -- Terminates both loopsEXIT WHEN <condition>; -- Returns control to parent_loop
END LOOP;
EXIT WHEN <condition>;END LOOP parent_loop;
Loops can be terminated by using the EXIT keyword, or by raising an exception.
FOR loops
Cursor FOR loops
FOR RecordIndex IN (SELECT person_code FROM people_table)LOOP DBMS_OUTPUT.PUT_LINE(RecordIndex.person_code);END LOOP;
Cursor-for loops automatically open a cursor, read in their data and close the cursor again
7
As an alternative, the PL/SQL programmer can pre-define the cursor's SELECT-statement in advance in order (for example) to allow re-use or to make the code more understandable (especially useful in the case of long or complex queries).
DECLARE CURSOR cursor_person IS SELECT person_code FROM people_table;BEGIN FOR RecordIndex IN cursor_person LOOP DBMS_OUTPUT.PUT_LINE(RecordIndex.person_code); END LOOP;END;
The concept of the person_code within the FOR-loop gets expressed with dot-notation ("."):
RecordIndex.person_code
Example
DECLARE var NUMBER; BEGIN /*N.B. for loop variables in pl/sql are new declarations, with scope only inside the loop */ FOR var IN 0 .. 10 LOOP DBMS_OUTPUT.put_line(var); END LOOP; IF (var IS NULL) THEN DBMS_OUTPUT.put_line('var is null'); ELSE DBMS_OUTPUT.put_line('var is not null'); END IF; END;
Output:
0 1 2 3 4 5 6 7 8 9 10 var is null
8
Indexes
A database index is a data structure that improves the speed of data retrieval operations on a database table at the cost of slower writes and increased storage space. Indexes can be created using one or more columns of a database table, providing the basis for both rapid random look ups and efficient access of ordered records. The disk space required to store the index is typically less than that required by the table (since indexes usually contain only the key-fields according to which the table is to be arranged, and excludes all the other details in the table), yielding the possibility to store indexes in memory for a table whose data is too large to store in memory.
In a relational database, an index is a copy of one part of a table. Some databases extend the power of indexing by allowing indexes to be created on functions or expressions. For example, an index could be created on upper(last_name), which would only store the upper case versions of the last_name field in the index. Another option sometimes supported is the use of "filtered" indexes, where index entries are created only for those records that satisfy some conditional expression. A further aspect of flexibility is to permit indexing on user-defined functions, as well as expressions formed from an assortment of built-in functions.
Indexes may be defined as unique or non-unique. A unique index acts as a constraint on the table by preventing duplicate entries in the index and thus, the backing table. Microsoft SQL Server allows a unique index to contain a single NULL value.
Index Architecture
Index architectures can be classified clustered or unclustered.
Unclustered
An unclustered index is structured in a way that doesn't correspond to the order of the actual data records. It resembles the words index at the back of a book. For this reason, it will perform worse than clustered indexes on ranged searches where the result set is large, since each result could cost an additional I/O-operation to get the actual data record. One can have any number of these kinds of indexes—all that is required is the space to store the index itself; one does not copy the actual data to create a new index.
Clustered
Clustering alters the data block into a certain distinct order to match the index, hence it is also an operation on the data storage blocks as well as on the index. An address book ordered by first name resembles a clustered index in its structure and purpose. The exact operation of database systems vary, but because storing data is very redundant the row data can only be stored in one order. Therefore, only one clustered index can be created on a given database table. Clustered indexes can greatly increase overall speed of retrieval, but usually only where the data is
9
accessed sequentially in the same or reverse order of the clustered index, or when a range of items are selected.
Since the physical records are in this sort order on disk, the next row item in the sequence is immediately before or after the last one, and so fewer data block reads are required. The primary feature of a clustered index is therefore the ordering of the physical data rows in accordance with the index blocks that point to them. Some databases separate the data and index blocks into separate files, others put two completely different data blocks within the same physical file(s).
Index implementations
Indexes can be implemented using a variety of data structures. Popular indexes include balanced trees, B+ trees and hashes.[1]
In Microsoft SQL Server, the leaf node of the clustered index corresponds to the actual data, not simply a pointer to data that resides elsewhere, as is the case with a non-clustered index. [2] Each relation can have a single clustered index and many unclustered indexes.[3]
Column order
The order in which columns are listed in the index definition is important. It is possible to retrieve a set of row identifiers using only the first indexed column. However, it is not possible or efficient (on most databases) to retrieve the set of row identifiers using only the second or greater indexed column.
For example, imagine a phone book that is organized by city first, then by last name, and then by first name. If you are given the city, you can easily extract the list of all phone numbers for that city. However, in this phone book it would be very tedious to find all the phone numbers for a given last name. You would have to look within each city's section for the entries with that last name. Some databases can do this, others just won’t use the index.
Applications and limitations
Indexes are useful for many applications but come with some limitations. Consider the following SQL statement: SELECT first_name FROM people WHERE last_name = 'Smith';. To process this statement without an index the database software must look at the last_name column on every row in the table (this is known as a full table scan). With an index the database simply follows the b-tree data structure until the Smith entry has been found; this is much less computationally expensive than a full table scan.
Consider this SQL statement: SELECT email_address FROM customers WHERE
email_address LIKE '%@yahoo.com';. This query would yield an email address for every customer whose email address ends with "@yahoo.com", but even if the email_address column has been indexed the database still must perform a full table scan. This is because the index is
10
built with the assumption that words go from left to right. With a wildcard at the beginning of the search-term, the database software is unable to use the underlying b-tree data structure (in other words, the WHERE-clause is not sargable). This problem can be solved through the addition of another index created on reverse(email_address) and a SQL query like this: SELECT email_address FROM customers WHERE reverse(email_address) LIKE
reverse('%@yahoo.com');. This puts the wild-card at the right-most part of the query (now moc.oohay@%) which the index on reverse(email_address) can satisfy.
Types
Bitmap index
Main article: Bitmap index
A bitmap index is a special kind of index that stores the bulk of its data as bit arrays (bitmaps) and answers most queries by performing bitwise logical operations on these bitmaps. The most commonly used index, such as B+trees, are most efficient if the values it indexes do not repeat or repeat a smaller number of times. In contrast, the bitmap index is designed for cases where the values of a variable repeat very frequently. For example, the gender field in a customer database usually contains two distinct values: male or female. For such variables, the bitmap index can have a significant performance advantage over the commonly used trees.
Dense index
A dense index in databases is a file with pairs of keys and pointers for every record in the data file. Every key in this file is associated with a particular pointer to a record in the sorted data file. In clustered indexes with duplicate keys, the dense index points to the first record with that key.[4]
Sparse index
A sparse index in databases is a file with pairs of keys and pointers for every block in the data file. Every key in this file is associated with a particular pointer to the block in the sorted data file. In clustered indexes with duplicate keys, the sparse index points to the lowest search key in each block.
Reverse index
Main article: Reverse index
A reverse key index reverses the key value before entering it in the index. E.g., the value 24538 becomes 83542 in the index. Reversing the key value is particularly useful for indexing data such as sequence numbers, where new key values monotonically increase.
11
Covering Index
In most cases, an index is used to quickly locate the data record(s) from which the required data is read. In other words, the index is only used to locate data records in the table and not to return data.
A covering index is a special case where the index itself contains the required data field(s) and can return the data.
Consider the following table (other fields omitted):
ID Name Other Fields
12 Plug ...
13 Lamp ...
14 Fuse ...
To find the Name for ID 13, an index on (ID) will be useful, but the record must still be read to get the Name. However, an index on (ID, Name) contains the required data field and eliminates the need to look up the record.
A covering index can dramatically speed up data retrieval but may itself be large due to the additional keys, which slows down data insertion & update. To reduce such index size, some systems allow non-key fields to be included in the index. Non-key fields are not themselves part of the index ordering but only included at the leaf level, allowing for a covering index with less overall index size.
12
Stored procedures
A stored procedure is a subroutine available to applications accessing a relational database system. Stored procedures (sometimes called a proc, sproc, StoPro, or SP) are actually stored in the database data dictionary.
Typical uses for stored procedures include data validation (integrated into the database) or access control mechanisms. Furthermore, stored procedures are used to consolidate and centralize logic that was originally implemented in applications. Large or complex processing that might require the execution of several SQL statements is moved into stored procedures, and all applications call the procedures only.
Stored procedures are similar to user-defined functions (UDFs). The major difference is that UDFs can be used like any other expression within SQL statements, whereas stored procedures must be invoked using the CALL statement[citation needed]
CALL procedure(…)
or
EXECUTE procedure(…)
Stored procedures can return result sets, i.e. the results of a SELECT statement. Such result sets can be processed using cursors by other stored procedures by associating a result set locator, or by applications. Stored procedures may also contain declared variables for processing data and cursors that allow it to loop through multiple rows in a table. The standard Structured Query Language provides IF, WHILE, LOOP, REPEAT, CASE statements, and more. Stored procedures can receive variables, return results or modify variables and return them, depending on how and where the variable is declared.
Implementation
The exact and correct implementation of stored procedure varies from one database system to another. Most major database vendors support them in some form. Depending on the database system, stored procedures can be implemented in a variety of programming languages, for example SQL, Java, C, or C++. Stored procedures written in non-SQL programming languages may or may not execute SQL statements themselves.
The increasing adoption of stored procedures led to the introduction of procedural elements to the SQL language in the SQL:1999 and SQL:2003 standards in the part SQL/PSM. That made SQL an imperative programming language. Most database systems offer proprietary and vendor-specific extensions, exceeding SQL/PSM.
13
Other uses
In some systems, stored procedures can be used to control transaction management; in others, stored procedures run inside a transaction such that transactions are effectively transparent to them. Stored procedures can also be invoked from a database trigger or a condition handler. For example, a stored procedure may be triggered by an insert on a specific table, or update of a specific field in a table, and the code inside the stored procedure would be executed. Writing stored procedures as condition handlers also allow DBAs to track errors in the system with greater detail by using stored procedures to catch the errors and record some audit information in the database or an external resource like a file.
Comparison with dynamic SQL
Overhead: Because stored procedure statements are stored directly in the database, this may remove all or part of the compilation overhead that is typically required in situations where software applications send inline (dynamic) SQL queries to a database. (However, most database systems implement "statement caches" and other mechanisms to avoid repetitive compilation of dynamic SQL statements.) In addition, pre-compiled SQL statements, while avoiding some overhead, add to the complexity of creating an optimal execution plan because not all arguments of the SQL statement are supplied at compile time. Depending on the specific database implementation and configuration, mixed performance results will be seen from stored procedures versus generic queries or user defined functions.
Avoidance of network traffic: A major advantage with stored procedures is that they can run directly within the database engine. In a production system, this typically means that the procedures run entirely on a specialized database server, which has direct access to the data being accessed. The benefit here is that network communication costs can be avoided completely. This becomes particularly important for complex series of SQL statements.
Encapsulation of business logic: Stored procedures allow for business logic to be embedded as an API in the database, which can simplify data management and reduce the need to encode the logic elsewhere in client programs. This may result in a lesser likelihood of data becoming corrupted through the use of faulty client programs. Thus, the database system can ensure data integrity and consistency with the help of stored procedures.
Delegation of access-rights: In many systems, stored-procedures can be granted access rights to the database which the users who will execute those procedures do not directly have. Thus, the stored procedure becomes the only way that these users have, to do whatever the stored procedure does.
14
Comparison with functions
A function is a subprogram written to perform certain computations and return a single value.
Functions must return a value (using the RETURN keyword), but for stored procedures this is not compulsory.
Stored procedures can use RETURN keyword but without any value being passed.
Functions could be used in SELECT statements, provided they don’t do any data manipulation. However, procedures cannot be included in SELECT statements.
A function can have only IN parameters, while stored procedures may have OUT or INOUT parameters.
A stored procedure can return multiple values using the OUT parameter or return no value at all.
Disadvantages
Stored procedures are "defined once, used many times." If any changes are necessary, the (one and only one) definition of the stored procedure must be replaced. Dynamic SQL, of course, allows any SQL query to be issued at any time.
Any change to a stored procedure instantly impacts every other piece of software, report, etc. (inside or outside of the DBMS) which directly or indirectly refers to it. It is not always possible to determine with certainty exactly what those impacts will be, nor what changes can safely be made without adversely impacting something else.
For various reasons, many organizations strictly limit who is allowed to define and issue a query against the database. Programmers and other users may therefore find themselves having no choice but to implement inefficient solutions to their problems using what stored procedures are available to them ... whether or not the procedures are appropriate for this particular ancillary task.
Furthermore, some modern DBMS systems (notably from Microsoft SQL Server 2000 onwards) don't offer any performance benefits of using stored procedures: they are compiled and cached in the same manner as dynamic SQL.
15
Cursors
In database packages, a cursor comprises a control structure for the successive traversal (and potential processing) of records in a result set.
Cursors provide a mechanism by which a database client iterates over the records in a database. Using cursors, the client can get, put, and delete database records. Database programmers use cursors for processing individual rows returned by the database system for a query. Cursors address the problem of impedance mismatch, an issue that occurs in many programming languages[citation needed]. Most procedural programming languages do not offer any mechanism for manipulating whole result-sets at once. In this scenario, the application must process rows in a result-set sequentially. Thus one can think of a database cursor as an iterator over the collection of rows in the result set.
Several SQL statements do not require the use of cursors. That includes the INSERT statement, for example, as well as most forms of the DELETE and UPDATE statements. Even a SELECT statement may not involve a cursor if it is used in the variation of SELECT INTO. A SELECT INTO retrieves at most a single row directly into the application.
Working with cursors
This section introduces the ways the SQL:2003 standard defines how to use cursors in applications in embedded SQL. Not all application bindings for relational database systems adhere to that standard, and some (such as CLI or JDBC) use a different interface.
A programmer makes a cursor known to the DBMS by using a DECLARE ... CURSOR statement and assigning the cursor a (compulsory) name:
DECLARE cursor_name CURSOR FOR SELECT ... FROM ...
Before code can access the data, it must open the cursor with the OPEN statement. Directly following a successful opening, the cursor is positioned before the first row in the result set.
OPEN cursor_name
Programs position cursors on a specific row in the result set with the FETCH statement. A fetch operation transfers the data of the row into the application.
FETCH cursor_name INTO ...
Once an application has processed all available rows or the fetch operation is to be positioned on a non-existing row (compare scrollable cursors below), the DBMS returns a SQLSTATE '02000' (usually accompanied by an SQLCODE +100) to indicate the end of the result set.
16
The final step involves closing the cursor using the CLOSE statement:
CLOSE cursor_name
After closing a cursor, a program can open it again, which implies that the DBMS re-evaluates the same query or a different query and builds a new result-set.
Scrollable cursors
Programmers may declare cursors as scrollable or not scrollable. The scrollability indicates the direction in which a cursor can move.
With a non-scrollable cursor, also known as forward-only, one can FETCH each row at most once, and the cursor automatically moves to the immediately following row. A fetch operation after the last row has been retrieved positions the cursor after the last row and returns SQLSTATE 02000 (SQLCODE +100).
A program may position a scrollable cursor anywhere in the result set using the FETCH SQL statement. The keyword SCROLL must be specified when declaring the cursor. The default is NO SCROLL, although different language bindings like JDBC may apply different default.
DECLARE cursor_name sensitivity SCROLL CURSOR FOR SELECT ... FROM ...
The target position for a scrollable cursor can be specified relative to the current cursor position or absolute from the beginning of the result set.
FETCH [ NEXT | PRIOR | FIRST | LAST ] FROM cursor_name FETCH ABSOLUTE n FROM cursor_name FETCH RELATIVE n FROM cursor_name
Scrollable cursors can potentially access the same row in the result set multiple times. Thus, data modifications (insert, update, delete operations) from other transactions could have an impact on the result set. A cursor can be SENSITIVE or INSENSITIVE to such data modifications. A sensitive cursor picks up data modifications impacting the result set of the cursor, and an insensitive cursor does not. Additionally, a cursor may be ASENSITIVE, in which case the DBMS tries to apply sensitivity as much as possible.
"WITH HOLD"
Cursors are usually closed automatically at the end of a transaction, i.e when a COMMIT or ROLLBACK (or an implicit termination of the transaction) occurs. That behavior can be changed if the cursor is declared using the WITH HOLD clause. (The default is WITHOUT HOLD.) A holdable cursor is kept open over COMMIT and closed upon ROLLBACK. (Some DBMS deviate from this standard behavior and also keep holdable cursors open over ROLLBACK.)
17
DECLARE cursor_name CURSOR WITH HOLD FOR SELECT ... FROM ...
When a COMMIT occurs, a holdable cursor is positioned before the next row. Thus, a positioned UPDATE or positioned DELETE statement will only succeed after a FETCH operation occurred first in the transaction.
Note that JDBC defines cursors as holdable per default. This is done because JDBC also activates auto-commit per default. Due to the usual overhead associated with auto-commit and holdable cursors, both features should be explicitly deactivated at the connection level.
Positioned update/delete statements
Cursors can not only be used to fetch data from the DBMS into an application but also to identify a row in a table to be updated or deleted. The SQL:2003 standard defines positioned update and positioned delete SQL statements for that purpose. Such statements do not use a regular WHERE clause with predicates. Instead, a cursor identifies the row. The cursor must be opened and positioned on a row already using the FETCH statement.
UPDATE table_name SET ... WHERE CURRENT OF cursor_name DELETE FROM table_name WHERE CURRENT OF cursor_name
The cursor must operate on an updatable result set in order to successfully execute a positioned update or delete statement. Otherwise, the DBMS would not know how to apply the data changes to the underlying tables referred to in the cursor.
Cursors in distributed transactions
Using cursors in distributed transactions (X/Open XA Environments), which are controlled using a transaction monitor, is no different than cursors in non-distributed transactions.
One has to pay attention when using holdable cursors, however. Connections can be used by different applications. Thus, once a transaction has been ended and committed, a subsequent transaction (running in a different application) could inherit existing holdable cursors. Therefore, an application developer has to be aware of that situation.
Cursors in XQuery
The XQuery language allows cursors to be created using the subsequence() function.
The format is:
let $displayed-sequence := subsequence($result, $start, $item-count)
18
Where $result is the result of the initial XQuery, $start is the item number to start and $item-count is the number of items to return.
Equivalently this can also be done using a predicate:
let $displayed-sequence := $result[$start to $end]
Where $end is the end sequence.
For complete examples see the XQuery Wikibook.
Disadvantages of cursors
The following information may vary from database system to database system.
Fetching a row from the cursor may result in a network round trip each time. This uses much more network bandwidth than would ordinarily be needed for the execution of a single SQL statement like DELETE. Repeated network round trips can severely impact the speed of the operation using the cursor. Some DBMSs try to reduce this impact by using block fetch. Block fetch implies that multiple rows are sent together from the server to the client. The client stores a whole block of rows in a local buffer and retrieves the rows from there until that buffer is exhausted.
Cursors allocate resources on the server, for instance locks, packages, processes, temporary storage, etc. For example, Microsoft SQL Server implements cursors by creating a temporary table and populating it with the query's result-set. If a cursor is not properly closed (deallocated), the resources will not be freed until the SQL session (connection) itself is closed. This wasting of resources on the server can not only lead to performance degradations but also to failures.
19
Triggers
A database trigger is procedural code that is automatically executed in response to certain events on a particular table or view in a database. The trigger is mostly used for keeping the integrity of the information on the database. For example, when a new record (representing a new worker) is added to the employees table, new records should be created also in the tables of the taxes, vacations, and salaries.
The need and the usage
Triggers are commonly used to:
prevent changes (e.g. prevent an invoice from being changed after it's been mailed out) log changes (e.g. keep a copy of the old data)
audit changes (e.g. keep a log of the users and roles involved in changes)
enhance changes (e.g. ensure that every change to a record is time-stamped by the server's clock, not the client's)
enforce business rules (e.g. require that every invoice have at least one line item)
execute business rules (e.g. notify a manager every time an employee's bank account number changes)
replicate data (e.g. store a record of every change, to be shipped to another database later)
enhance performance (e.g. update the account balance after every detail transaction, for faster queries)
Some systems also support non-data triggers, which fire in response to Data Definition Language (DDL) events such as creating tables, or runtime events such as logon, commit, and rollback,be used for auditing purposes.
The major features of database triggers, and their effects, are:
do not accept parameters or arguments (but may store affected-data in temporary tables) cannot perform commit or rollback operations because they are part of the triggering
SQL statement (only through autonomous transactions)
can cancel a requested operation
can cause mutating table errors,
DML Triggers
20
There are typically three triggering events that cause data triggers to 'fire':
INSERT event (as a new record is being inserted into the database). UPDATE event (as a record is being changed).
DELETE event (as a record is being deleted).
Structurally, triggers are either "row triggers" or "statement triggers". Row triggers define an action for every row of a table, while statement triggers occur only once per INSERT, UPDATE, or DELETE statement. DML triggers cannot be used to audit data retrieval via SELECT statements, because SELECT is not a triggering event.
Furthermore, there are "BEFORE triggers" and "AFTER triggers" which run in addition to any changes already being made to the database, and "INSTEAD OF trigger" which fully replace the database's normal activity.
Triggers do not accept parameters, but they do receive information in the form of implicit variables. For row-level triggers, these are generally OLD and NEW variables, each of which have fields corresponding to the columns of the affected table or view; for statement-level triggers, something like SQL Server's Inserted and Deleted tables may be provided so the trigger can see all the changes being made.
For data triggers, the general order of operations will be as follows:
1. a statement requests changes on a row: OLD represents the row as it was before the change (or is all-null for inserted rows), NEW represents the row after the changes (or is all-null for deleted rows)
2. each statement-level BEFORE trigger is fired
3. each row-level BEFORE trigger fires, and can modify NEW (but not OLD); each trigger can see NEW as modified by its predecessor, they are chained together
4. if an INSTEAD OF trigger is defined, it is run using OLD and NEW as available at this point
5. if no INSTEAD OF trigger is defined, the database modifies the row according to its normal logic; for updatable views, this may involve modifying one or more other tables to achieve the desired effect; if a view is not updatable, and no INSTEAD OF trigger is provided, an error is raised
6. each row-level AFTER trigger fires, and is given NEW and OLD, but its changes to NEW are either disallowed or disregarded
7. each statement-level AFTER trigger is fired
8. implied triggers are fired, such as referential actions in support of foreign key constraints: on-update or on-delete CASCADE, SET NULL, and SET DEFAULT rules
21
In ACID databases, an exception raised in a trigger will cause the entire stack of operations to be rolled back, including the original statement.
Triggers in Oracle
In addition to triggers that fire when data is modified, Oracle 9i supports triggers that fire when schema objects (that is, tables) are modified and when user logon or logoff events occur. These trigger types are referred to as "Schema-level triggers".
Schema-level triggers
After Creation Before Alter
After Alter
Before Drop
After Drop
Before Logoff
After Logon
The two main types of triggers are:
1. Row Level Trigger2. Statement Level Trigger
Based on the 2 types of classifications, we could have 12 types of triggers.
Mutating tables
When a single SQL statement modifies several rows of a table at once, the order of the operations is not well-defined; there is no "order by" clause on "update" statements, for example. Row-level triggers are executed as each row is modified, so the order in which trigger code is run is also not well-defined. Oracle protects the programmer from this uncertainty by preventing row-level triggers from modifying other rows in the same table – this is the "mutating table" in the error message. Side-effects on other tables are allowed, however.
One solution is to have row-level triggers place information into a temporary table indicating what further changes need to be made, and then have a statement-level trigger fire just once, at the end, to perform the requested changes and clean up the temporary table.
Because a foreign key's referential actions are implemented via implied triggers, they are similarly restricted. This may become a problem when defining a self-referential foreign key, or
22
a cyclical set of such constraints, or some other combination of triggers and CASCADE rules (e.g. user deletes a record from table A, CASCADE rule on table A deletes a record from table B, trigger on table B attempts to SELECT from table A, error occurs.)
Triggers in Microsoft SQL Server
Microsoft SQL Server supports triggers either after or instead of an insert, update, or delete operation. They can be set on tables and views with the constraint that a view can be referenced only by an INSTEAD OF trigger.
Microsoft SQL Server 2005 introduced support for Data Definition Language (DDL) triggers, which can fire in reaction to a very wide range of events, including:
Drop table Create table
Alter table
Login events
Triggers in MySQL
MySQL 5.0.2 introduced support for triggers. Some of the triggers MYSQL supports are
Insert Trigger Update Trigger
Delete Trigger
The SQL:2003 standard mandates that triggers give programmers access to record variables by means of a syntax such as REFERENCING NEW AS n. For example, if a trigger is monitoring for changes to a salary column one could write a trigger like the following:
CREATE TRIGGER salary_trigger BEFORE UPDATE ON employee_table REFERENCING NEW ROW AS n, OLD ROW AS o FOR EACH ROW IF n.salary <> o.salary THEN END IF;;
23
Join (SQL)
An SQL JOIN clause combines records from two or more tables in a database. [1] It creates a set that can be saved as a table or used as is. A JOIN is a means for combining fields from two tables by using values common to each. ANSI standard SQL specifies four types of JOINs: INNER, OUTER, LEFT, and RIGHT. In special cases, a table (base table, view, or joined table) can JOIN to itself in a self-join.
A programmer writes a JOIN predicate to identify the records for joining. If the evaluated predicate is true, the combined record is then produced in the expected format, a record set or a temporary table, for example.
Novices should beware that the default JOIN in many systems is the inner join (this can be confusing because join ... on can seem to imply an ordering, but it is actually the join that applies to the table names, and the on that applies to the criteria; join is itself ambiguous between order-independence and order-dependence).
Sample tables
all subsequent explanations on join types in this article make use of the following two tables. The rows in these tables serve to illustrate the effect of different types of joins and join-predicates. In the following tables, Department.DepartmentID is the primary key, while Employee.DepartmentID is a foreign key.
Employee Table
LastName DepartmentID
Rafferty 31
Jones 33
Steinberg 33
Robinson 34
Smith 34
John NULL
Department Table
DepartmentID DepartmentName
31 Sales
33 Engineering
34 Clerical
35 Marketing
Note: The "Marketing" Department currently has no listed employees. Also, employee "John" has not been assigned to any Department yet.
24
Inner join
An inner join is the most common join operation used in applications, and represents the default join-type. Inner join creates a new result table by combining column values of two tables (A and B) based upon the join-predicate. The query compares each row of A with each row of B to find all pairs of rows which satisfy the join-predicate. When the join-predicate is satisfied, column values for each matched pair of rows of A and B are combined into a result row. The result of the join can be defined as the outcome of first taking the Cartesian product (or cross-join) of all records in the tables (combining every record in table A with every record in table B) - then return all records which satisfy the join predicate. Actual SQL implementations normally use other approaches like a Hash join or a Sort-merge join where possible, since computing the Cartesian product is very inefficient.
SQL specifies two different syntactical ways to express joins: "explicit join notation" and "implicit join notation".
The "explicit join notation" uses the JOIN keyword to specify the table to join, and the ON keyword to specify the predicates for the join, as in the following example:
SELECT *FROM employee INNER JOIN department ON employee.DepartmentID = department.DepartmentID
The "implicit join notation" simply lists the tables for joining (in the FROM clause of the SELECT statement), using commas to separate them. Thus, it specifies a cross-join, and the WHERE clause may apply additional filter-predicates (which function comparably to the join-predicates in the explicit notation).
The following example shows a query which is equivalent to the one from the previous example, but this time written using the implicit join notation:
SELECT * FROM employee, department WHERE employee.DepartmentID = department.DepartmentID
The queries given in the examples above will join the Employee and Department tables using the DepartmentID column of both tables. Where the DepartmentID of these tables match (i.e. the join-predicate is satisfied), the query will combine the LastName, DepartmentID and DepartmentName columns from the two tables into a result row. Where the DepartmentID does not match, no result row is generated.
Thus the result of the execution of either of the two queries above will be:
25
Employee.LastName
Employee.DepartmentID
Department.DepartmentName
Department.DepartmentID
Robinson 34 Clerical 34
Jones 33 Engineering 33
Smith 34 Clerical 34
Steinberg 33 Engineering 33
Rafferty 31 Sales 31
Note: Programmers should take special care when joining tables on columns that can contain NULL values, since NULL will never match any other value (or even NULL itself), unless the join condition explicitly uses the IS NULL or IS NOT NULL predicates.
Notice that the employee "John" and the department "Marketing" do not appear in the query execution results. Neither of these has any matching records in the respective other table: "John" has no associated department, and no employee has the department ID 35. Thus, no information on John or on Marketing appears in the joined table. Depending on the desired results, this behavior may be a subtle bug. Outer joins may be used to avoid it.
One can further classify inner joins as equi-joins, as natural joins, or as cross-joins (see below).
Equi-join
An equi-join, also known as an equijoin, is a specific type of comparator-based join, or theta join, that uses only equality comparisons in the join-predicate. Using other comparison operators (such as <) disqualifies a join as an equi-join. The query shown above has already provided an example of an equi-join:
SELECT *FROM employee INNER JOIN department ON employee.DepartmentID = department.DepartmentID
SQL provides an optional shorthand notation for expressing equi-joins, by way of the USING construct (Feature ID F402):
SELECT *FROM employee INNER JOIN department USING (DepartmentID)
The USING construct is more than mere syntactic sugar, however, since the result set differs from the result set of the version with the explicit predicate. Specifically, any columns mentioned in the USING list will appear only once, with an unqualified name, rather than once for each table in
26
the join. In the above case, there will be a single DepartmentID column and no employee.DepartmentID or department.DepartmentID.
The USING clause is supported by MySQL, Oracle, PostgreSQL, SQLite, DB2/400 and Firebird in version 2.1 or higher.
Natural join
A natural join offers a further specialization of equi-joins. The join predicate arises implicitly by comparing all columns in both tables that have the same column-name in the joined tables. The resulting joined table contains only one column for each pair of equally-named columns....
The above sample query for inner joins can be expressed as a natural join in the following way:
SELECT *FROM employee NATURAL JOIN department
As with the explicit USING clause, only one DepartmentID column occurs in the joined table, with no qualifier:
With either a JOIN USING or NATURAL JOIN, the Oracle database implementation of SQL will report a compile-time error if one of the equijoined columns is specified with a table name qualifier: "ORA-25154: column part of USING clause cannot have qualifier" or "ORA-25155: column used in NATURAL join cannot have qualifier", respectively.
Cross join
A cross join, cartesian join or product provides the foundation upon which all types of inner joins operate. A cross join returns the cartesian product of the sets of records from the two joined tables. Thus, it equates to an inner join where the join-condition always evaluates to True or where the join-condition is absent from the statement. In other words, a cross join combines every row in B with every row in A. The number of rows in the result set will be the number of rows in A times the number of rows in B.
Thus, if A and B are two sets, then the cross join is written as A × B.
27
The SQL code for a cross join lists the tables for joining (FROM), but does not include any filtering join-predicate.
Example of an explicit cross join:
SELECT *FROM employee CROSS JOIN department
Example of an implicit cross join:
SELECT *FROM employee, department;
Employee.LastName
Employee.DepartmentID
Department.DepartmentName
Department.DepartmentID
Rafferty 31 Sales 31
Jones 33 Sales 31
Steinberg 33 Sales 31
Smith 34 Sales 31
Robinson 34 Sales 31
John NULL Sales 31
Rafferty 31 Engineering 33
Jones 33 Engineering 33
Steinberg 33 Engineering 33
Smith 34 Engineering 33
Robinson 34 Engineering 33
John NULL Engineering 33
Rafferty 31 Clerical 34
Jones 33 Clerical 34
Steinberg 33 Clerical 34
Smith 34 Clerical 34
Robinson 34 Clerical 34
John NULL Clerical 34
Rafferty 31 Marketing 35
Jones 33 Marketing 35
Steinberg 33 Marketing 35
Smith 34 Marketing 35
Robinson 34 Marketing 35
John NULL Marketing 35
28
The cross join does not apply any predicate to filter records from the joined table. Programmers can further filter the results of a cross join by using a WHERE clause.
Outer joins
An outer join does not require each record in the two joined tables to have a matching record. The joined table retains each record—even if no other matching record exists. Outer joins subdivide further into left outer joins, right outer joins, and full outer joins, depending on which table(s) one retains the rows from (left, right, or both).
(In this case left and right refer to the two sides of the JOIN keyword.)
No implicit join-notation for outer joins exists in standard SQL.
Left outer join
The result of a left outer join (or simply left join) for table A and B always contains all records of the "left" table (A), even if the join-condition does not find any matching record in the "right" table (B). This means that if the ON clause matches 0 (zero) records in B, the join will still return a row in the result—but with NULL in each column from B. This means that a left outer join returns all the values from the left table, plus matched values from the right table (or NULL in case of no matching join predicate). If the left table returns one row and the right table returns more than one matching row for it, the values in the left table will be repeated for each distinct row on the right table.
For example, this allows us to find an employee's department, but still shows the employee(s) even when their department does not exist (contrary to the inner-join example above, where employees in non-existent departments are excluded from the result).
Example of a left outer join, with the additional result row italicized:
SELECT * FROM employee LEFT OUTER JOIN department ON employee.DepartmentID = department.DepartmentID
Employee.LastName
Employee.DepartmentID
Department.DepartmentName
Department.DepartmentID
Jones 33 Engineering 33
Rafferty 31 Sales 31
Robinson 34 Clerical 34
Smith 34 Clerical 34
John NULL NULL NULL
Steinberg 33 Engineering 33
29
Right outer joins
A right outer join (or right join) closely resembles a left outer join, except with the treatment of the tables reversed. Every row from the "right" table (B) will appear in the joined table at least once. If no matching row from the "left" table (A) exists, NULL will appear in columns from A for those records that have no match in B.
A right outer join returns all the values from the right table and matched values from the left table (NULL in case of no matching join predicate).
For example, this allows us to find each employee and his or her department, but still show departments that have no employees.
Example right outer join, with the additional result row italicized:
SELECT * FROM employee RIGHT OUTER JOIN department ON employee.DepartmentID = department.DepartmentID
Employee.LastName
Employee.DepartmentID
Department.DepartmentName
Department.DepartmentID
Smith 34 Clerical 34
Jones 33 Engineering 33
Robinson 34 Clerical 34
Steinberg 33 Engineering 33
Rafferty 31 Sales 31NULL NULL Marketing 35
In practice, explicit right outer joins are rarely used, since they can always be replaced with left outer joins (with the table order switched) and provide no additional functionality. The result above is produced also with a left outer join:SELECT * FROM department LEFT OUTER JOIN employee ON employee.DepartmentID = department.DepartmentID
Full outer join
A full outer join combines the results of both left and right outer joins. The joined table will contain all records from both tables, and fill in NULLs for missing matches on either side.
For example, this allows us to see each employee who is in a department and each department that has an employee, but also see each employee who is not part of a department and each department which doesn't have an employee.
30
Example full outer join:
SELECT * FROM employee FULL OUTER JOIN department ON employee.DepartmentID = department.DepartmentID
Employee.LastName
Employee.DepartmentID
Department.DepartmentName
Department.DepartmentID
Smith 34 Clerical 34
Jones 33 Engineering 33
Robinson 34 Clerical 34
John NULL NULL NULL
Steinberg 33 Engineering 33
Rafferty 31 Sales 31NULL NULL Marketing 35
Some database systems (like MySQL) do not support this functionality directly, but they can emulate it through the use of left and right outer joins and unions. The same example can appear as follows:SELECT *FROM employee LEFT JOIN department ON employee.DepartmentID = department.DepartmentIDUNIONSELECT *FROM employee RIGHT JOIN department ON employee.DepartmentID = department.DepartmentIDWHERE employee.DepartmentID IS NULL
SQLite does not support right join, so outer join can be emulated as follows:
SELECT employee.*, department.*FROM employee LEFT JOIN department ON employee.DepartmentID = department.DepartmentIDUNION ALLSELECT employee.*, department.*FROM department LEFT JOIN employee ON employee.DepartmentID = department.DepartmentIDWHERE employee.DepartmentID IS NULL
Self-join
A self-join is joining a table to itself.[2] This is best illustrated by the following example.
31
Example
A query to find all pairings of two employees in the same country is desired. If you had two separate tables for employees and a query which requested employees in the first table having the same country as employees in the second table, you could use a normal join operation to find the answer table. However, all the employee information is contained within a single large table. [3]
Considering a modified Employee table such as the following:
Which results in the following table being generated.
Employee Table after Self-join by Country
EmployeeID LastName EmployeeID LastName Country
123 Rafferty 124 Jones Australia
123 Rafferty 145 Steinberg Australia
124 Jones 145 Steinberg Australia
305 Smith 306 John Germany
For this example, note that:
32
F and S are aliases for the first and second copies of the employee table. The condition F.Country = S.Country excludes pairings between employees in different
countries. The example question only wanted pairs of employees in the same country.
The condition F.EmployeeID < S.EmployeeID excludes pairings where the EmployeeIDs are the same.
F.EmployeeID < S.EmployeeID also excludes duplicate pairings. Without it only the following less useful part of the table would be generated (for Germany only shown):
EmployeeID LastName EmployeeID LastName Country
305 Smith 305 Smith Germany
305 Smith 306 John Germany
306 John 305 Smith Germany
306 John 306 John Germany
Only one of the two middle pairings is needed to satisfy the original question, and the topmost and bottommost are of no interest at all in this example.
Alternatives
The effect of outer joins can also be obtained using correlated subqueries. For example
SELECT employee.LastName, employee.DepartmentID, department.DepartmentName FROM employee LEFT OUTER JOIN department ON employee.DepartmentID = department.DepartmentID
can also be written as
SELECT employee.LastName, employee.DepartmentID, (SELECT department.DepartmentName FROM department WHERE employee.DepartmentID = department.DepartmentID )FROM employee
Implementation
Much work in database-systems has aimed at efficient implementation of joins, because relational systems commonly call for joins, yet face difficulties in optimising their efficient execution. The problem arises because (inner) joins operate both commutatively and associatively. In practice, this means that the user merely supplies the list of tables for joining and the join conditions to use, and the database system has the task of determining the most efficient way to perform the operation. A query optimizer determines how to execute a query containing joins. A query optimizer has two basic freedoms:
33
1. Join order: Because joins function commutatively and associatively, the order in which the system joins tables does not change the final result-set of the query. However, join-order does have an enormous impact on the cost of the join operation, so choosing the best join order becomes very important.
2. Join method: Given two tables and a join condition, multiple algorithms can produce the result-set of the join. Which algorithm runs most efficiently depends on the sizes of the input tables, the number of rows from each table that match the join condition, and the operations required by the rest of the query.
Many join-algorithms treat their inputs differently. One can refer to the inputs to a join as the "outer" and "inner" join operands, or "left" and "right", respectively. In the case of nested loops, for example, the database system will scan the entire inner relation for each row of the outer relation.
One can classify query-plans involving joins as follows:[4]
left-deep
using a base table (rather than another join) as the inner operand of each join in the plan
right-deep
using a base table as the outer operand of each join in the plan
bushy
neither left-deep nor right-deep; both inputs to a join may themselves result from joins
These names derive from the appearance of the query plan if drawn as a tree, with the outer join relation on the left and the inner relation on the right (as convention dictates).
Join algorithms
Three fundamental algorithms exist for performing a join operation.
Nested loopsMain articles: Nested loop join and block nested loop
Use of nested loops produces the simplest join-algorithm. For each tuple in the outer join relation, the system scans the entire inner-join relation and appends any tuples that match the join-condition to the result set. Naturally, this algorithm performs poorly with large join-relations: inner or outer or both. An index on columns in the inner relation in the join-predicate can enhance performance.
34
The block nested loops (BNL) approach offers a refinement to this technique: for every block in the outer relation, the system scans the entire inner relation. For each match between the current inner tuple and one of the tuples in the current block of the outer relation, the system adds a tuple to the join result-set. This variant means doing more computation for each tuple of the inner relation, but far fewer scans of the inner relation.
Merge join
If both join relations come in order, sorted by the join attribute(s), the system can perform the join trivially, thus:
1. Consider the current "group" of tuples from the inner relation; a group consists of a set of contiguous tuples in the inner relation with the same value in the join attribute.
2. For each matching tuple in the current inner group, add a tuple to the join result. Once the inner group has been exhausted, advance both the inner and outer scans to the next group.
Merge joins offer one reason why many optimizers keep track of the sort order produced by query plan operators—if one or both input relations to a merge join arrives already sorted on the join attribute, the system need not perform an additional sort. Otherwise, the DBMS will need to perform the sort, usually using an external sort to avoid consuming too much memory.
Hash join
A hash join algorithm can only produce equi-joins. The database system pre-forms access to the tables concerned by building hash tables on the join-attributes. The lookup in hash tables operates much faster than through index trees. However, one can compare hashed values only for equality (or inequality), not for other relationships.
35
AJAX
The XMLHttpRequest
All new browsers support a new built-in JavaScript XMLHttpRequest object (IE5 and IE6 uses an ActiveXObject).
This object can be used to request information (data) from a server.
Let's update our HTML file with a JavaScript in the <head> section:
function loadXMLDoc(url){if (window.XMLHttpRequest){// code for IE7+, Firefox, Chrome, Opera, Safarixmlhttp=new XMLHttpRequest();}else{// code for IE6, IE5xmlhttp=new ActiveXObject("Microsoft.XMLHTTP");}xmlhttp.open("GET",url,false);xmlhttp.send(null);document.getElementById('test').innerHTML=xmlhttp.responseText;}
Example explained
Try to create a XMLHttpRequest object:
xmlhttp=new XMLHttpRequest()
If not (if IE5 or IE6) create an ActiveXObject:
xmlhttp=new ActiveXObject("Microsoft.XMLHTTP")
Open the request object:
xmlhttp.open("GET",url,false)
Send your request to your server:
xmlhttp.send(null)
36
Update your page with the response from the server:
Note: The code above can be used every time you need to create an XMLHttpRequest object, so just copy and paste it whenever you need it.
In the next chapter you will learn more about the XMLHttpRequest.
All Together Now
<html><head><script type="text/javascript">function loadXMLDoc(url){if (window.XMLHttpRequest){// code for IE7+, Firefox, Chrome, Opera, Safarixmlhttp=new XMLHttpRequest();}else{// code for IE6, IE5xmlhttp=new ActiveXObject("Microsoft.XMLHTTP");}xmlhttp.open("GET",url,false);xmlhttp.send(null);document.getElementById('test').innerHTML=xmlhttp.responseText;}</script></head>
<body>
<div id="test"><h2>Click to let AJAX change this text</h2></div><button type="button" onclick="loadXMLDoc('test1.txt')">Click Me</button><button type="button" onclick="loadXMLDoc('test2.txt')">Click Me</button>
</body></html>
Sending an AJAX Request to a Server37
To send a request to a web server, we use the open() and send() methods.
The open() method takes three arguments. The first argument defines which method to use (GET or POST). The second argument specifies the name of the server resource (URL). The third argument specifies if the request should be handled asynchronously.
The send() method sends the request off to the server. If we assume that requested a file called "time.asp", the code would be:
The third parameter in the open call is "false". This tells the XMLHttpRequest object to wait until the server request is completed before next statement is executed.
For small applications and simple server request, this might be OK. But if the request takes a long time or cannot be served, this might cause your web application to hang or stop.
XMLHttpRequest Open - Using True
By changing the third parameter in the open call to "true", you tell the XMLHttpRequest object to continue the execution after the request to the server has been sent.
Because you cannot simply start using the response from the server request before you are sure the request has been completed, you need to set the onreadystatechange property of the XMLHttpRequest, to a function (or name of a function) to be executed after completion.
In this onreadystatechange function you must test the readyState property before you can use the result of the server call.
The Common Language Runtime (CLR) is a core component of Microsoft's .NET initiative. It is Microsoft's implementation of the Common Language Infrastructure (CLI) standard, which defines an execution environment for program code. In the CLR, code is expressed in a form of bytecode called the Common Intermediate Language (CIL, previously known as MSIL—Microsoft Intermediate Language).
Developers using the CLR write code in a language such as C# or VB.NET. At compile time, a .NET compiler converts such code into CIL code. At runtime, the CLR's just-in-time compiler converts the CIL code into code native to the operating system. Alternatively, the CIL code can be compiled to native code in a separate step prior to runtime by using the Native Image Generator (NGEN). This speeds up all later runs of the software as the CIL-to-native compilation is no longer necessary.
Although some other implementations of the Common Language Infrastructure run on non-Windows operating systems, Microsoft's implementation runs only on Microsoft Windows operating systems.
41
The CLR allows programmers to ignore many details of the specific CPU that will execute the program. It also provides other important services, including the following:
Memory management Thread management
Exception handling
Garbage collection
Security
Memory management
Memory management is the act of managing computer memory. In its simpler forms, this involves providing ways to allocate portions of memory to programs at their request, and freeing it for reuse when no longer needed. The management of main memory is critical to the computer system.
Virtual memory systems separate the memory addresses used by a process from actual physical addresses, allowing separation of processes and increasing the effectively available amount of RAM using disk swapping. The quality of the virtual memory manager can have a big impact on overall system performance.
Garbage collection is the automated allocation, and deallocation of computer memory resources for a program. This is generally implemented at the programming language level and is in opposition to manual memory management, the explicit allocation and deallocation of computer memory resources.
Thread
In computer science, a thread of execution results from a fork of a computer program into two or more concurrently running tasks. The implementation of threads and processes differs from one operating system to another, but in most cases, a thread is contained inside a process. Multiple threads can exist within the same process and share resources such as memory, while different processes do not share these resources.
On a single processor, multithreading generally occurs by time-division multiplexing (as in multitasking): the processor switches between different threads. This context switching generally happens frequently enough that the user perceives the threads or tasks as running at the same time. On a multiprocessor or multi-core system, the threads or tasks will generally run at the same time, with each processor or core running a particular thread or task.
42
Many modern operating systems directly support both time-sliced and multiprocessor threading with a process scheduler. The kernel of an operating system allows programmers to manipulate threads via the system call interface. Some implementations are called a kernel thread, whereas a lightweight process (LWP) is a specific type of kernel thread that shares the same state and information.
Programs can have user-space threads when threading with timers, signals, or other methods to interrupt their own execution, performing a sort of ad-hoc time-slicing.
Threads compared with processes
Threads differ from traditional multitasking operating system processes in that:
processes are typically independent, while threads exist as subsets of a process processes carry considerable state information, whereas multiple threads within a process
share state as well as memory and other resources
processes have separate address spaces, whereas threads share their address space
processes interact only through system-provided inter-process communication mechanisms.
Context switching between threads in the same process is typically faster than context switching between processes.
Systems like Windows NT and OS/2 are said to have "cheap" threads and "expensive" processes; in other operating systems there is not so great a difference except the cost of address space switch which implies a TLB flush.
Multithreading: Advantages/Uses
Multithreading as a widespread programming and execution model allows multiple threads to exist within the context of a single process. These threads share the process' resources but are able to execute independently. The threaded programming model provides developers with a useful abstraction of concurrent execution. However, perhaps the most interesting application of the technology is when it is applied to a single process to enable parallel execution on a multiprocessor system.
This advantage of a multithreaded program allows it to operate faster on computer systems that have multiple CPUs, CPUs with multiple cores, or across a cluster of machines — because the threads of the program naturally lend themselves to truly concurrent execution. In such a case, the programmer needs to be careful to avoid race conditions, and other non-intuitive behaviors. In order for data to be correctly manipulated, threads will often need to rendezvous in time in order to process the data in the correct order. Threads may also require mutually-exclusive operations (often implemented using semaphores) in order to prevent common data from being
43
simultaneously modified, or read while in the process of being modified. Careless use of such primitives can lead to deadlocks.
Another advantage of multithreading, even for single-CPU systems, is the ability for an application to remain responsive to input. In a single threaded program, if the main execution thread blocks on a long running task, the entire application can appear to freeze. By moving such long running tasks to a worker thread that runs concurrently with the main execution thread, it is possible for the application to remain responsive to user input while executing tasks in the background.
Operating systems schedule threads in one of two ways:
1. Preemptive multithreading is generally considered the superior approach, as it allows the operating system to determine when a context switch should occur. The disadvantage to preemptive multithreading is that the system may make a context switch at an inappropriate time, causing priority inversion or other negative effects which may be avoided by cooperative multithreading.
2. Cooperative multithreading, on the other hand, relies on the threads themselves to relinquish control once they are at a stopping point. This can create problems if a thread is waiting for a resource to become available.
Traditional mainstream computing hardware did not have much support for multithreading, because switching between threads was generally already quicker than full process context switches.[clarification needed] Processors in embedded systems, which have higher requirements for real-time behaviors, might support multithreading by decreasing the thread-switch time, perhaps by allocating a dedicated register file for each thread instead of saving/restoring a common register file. In the late 1990s, the idea of executing instructions from multiple threads simultaneously has become known as simultaneous multithreading. This feature was introduced in Intel's Pentium 4 processor, with the name hyper threading.
Exception handlingException handling is a programming language construct or computer hardware mechanism designed to handle the occurrence of exceptions, special conditions that change the normal flow of program execution.
General characteristics
Unlike signals and event handlers that are part of the normal program flow, exceptions are typically used to signal that something went wrong (e.g. a division by zero occurred or a required file was not found). Exceptions are raised or thrown (initiated) by either the hardware or the program itself by using a special command.
Programming languages differ considerably in their support for exception handling as distinct from error checking[clarification needed (differentiating definition for "error checking" required)]. In some programming
44
languages there are functions which cannot be safely called on invalid input data ... or functions which return values which cannot be distinguished from exceptions. For example in C, the atoi (ASCII to integer conversion) function may return 0 (zero) for any input that cannot be parsed into a valid value. In such languages the programmer must either perform error checking (possibly through some auxiliary global variable such as C's errno) or input validation (perhaps using regular expressions).
The degree to which such explicit validation and error checking is necessary is in contrast to exception handling support provided by any given programming environment. Hardware exception handling differs somewhat from the support provided by software tools, but similar concepts and terminology are prevalent.
In general, an exception is handled (resolved) by saving the current state of execution in a predefined place and switching the execution to a specific subroutine known as an exception handler. Depending on the situation, the handler may later resume the execution at the original location using the saved information. For example, a page fault will usually allow the program to be resumed, while a division by zero might not be resolvable transparently.
From the processing point of view, hardware interrupts are similar to resume-able exceptions, though they are typically unrelated to the user's program flow.
From the point of view of the author of a routine, raising an exception is a useful way to signal that a routine could not execute normally. For example, when an input argument is invalid (e.g. a zero denominator in division) or when a resource it relies on is unavailable (like a missing file, or a hard disk error). In systems without exceptions, routines would need to return some special error code. However, this is sometimes complicated by the semipredicate problem, in which users of the routine need to write extra code to distinguish normal return values from erroneous ones.
In runtime engine environments such as Java or .NET, there exist tools that attach to the runtime engine and every time that an exception of interest occurs, they record debugging information that existed in memory at the time the exception was thrown (call stack and heap values). These tools are called Automated Exception Handling or Error Interception tools and provide 'root-cause' information for exceptions.
Contemporary applications face many design challenges when considering exception handling strategies. Particularly in modern enterprise level applications, exceptions must often cross process boundaries and machine boundaries. Part of designing a solid exception handling strategy is recognizing when a process has failed to the point where it cannot be economically handled by the software portion of the process. [1]
Exception safety
A piece of code is said to be exception-safe, if run-time failures within the code will not produce ill effects, such as memory leaks, garbled stored data, or invalid output. Exception-safe code
45
must satisfy invariants placed on the code even if exceptions occur. There are several levels of exception safety:
1. Failure transparency, also known as the no throw guarantee: Operations are guaranteed to succeed and satisfy all requirements even in presence of exceptional situations. If an exception occurs, it will not throw the exception further up. (Best level of exception safety.)
2. Commit or rollback semantics, also known as strong exception safety or no-change guarantee: Operations can fail, but failed operations are guaranteed to have no side effects so all data retain original values.[2]
3. Basic exception safety: Partial execution of failed operations can cause side effects, but invariants on the state are preserved. Any stored data will contain valid values even if data has different values now from before the exception.
4. Minimal exception safety also known as no-leak guarantee: Partial execution of failed operations may store invalid data but will not cause a crash, and no resources get leaked.
5. No exception safety: No guarantees are made. (Worst level of exception safety)
For instance, consider a smart vector type, such as C++'s std::vector or Java's ArrayList. When an item x is added to a vector v, the vector must actually add x to the internal list of objects and also update a count field that says how many objects are in v. It may also need to allocate new memory if the existing capacity isn't large enough. This memory allocation may fail and throw an exception. Because of this, a vector that provides failure transparency would be very difficult or impossible to write. However, the vector may be able to offer the strong exception guarantee fairly easily; in this case, either the insertion of x into v will succeed, or v will remain unchanged. If the vector provides only the basic exception safety guarantee, if the insertion fails, v may or may not contain x, but at least it will be in a consistent state. However, if the vector makes only the minimal guarantee, it's possible that the vector may be invalid. For instance, perhaps the size field of v was incremented but x wasn't actually inserted, making the state inconsistent. Of course, with no guarantee, the program may crash; perhaps the vector needed to expand but couldn't allocate the memory and blindly ploughs ahead as if the allocation succeeded, touching memory at an invalid address.
Usually at least basic exception safety is required. Failure transparency is difficult to implement, and is usually not possible in libraries where complete knowledge of the application is not available.
Verification of Exception Handling
The point of exception handling routines is to ensure that the code can handle error conditions. In order to establish that exception handling routines are sufficiently robust, it is necessary to present the code with a wide spectrum of invalid or unexpected inputs, such as can be created via software fault injection and mutation testing (which is also sometimes referred to as fuzz testing). One of the most difficult types of software for which to write exception handling
46
routines is protocol software, since a robust protocol implementation must be prepared to receive input that does not comply with the relevant specification(s).
In order to ensure that meaningful regression analysis can be conducted throughout a software development lifecycle process, any exception handling verification should be highly automated, and the test cases must be generated in a scientific, repeatable fashion. Several commercially available systems exist that perform such testing.
Garbage collection
In computer science, garbage collection (GC) is a form of automatic memory management. It is a special case of resource management, in which the limited resource being managed is memory. The garbage collector, or just collector, attempts to reclaim garbage, or memory occupied by objects that are no longer in use by the program. Garbage collection was invented by John McCarthy around 1959 to solve problems in Lisp.[1][2]
Garbage collection is often portrayed as the opposite of manual memory management, which requires the programmer to specify which objects to deallocate and return to the memory system. However, many systems use a combination of the two approaches, and other techniques such as stack allocation and region inference can carve off parts of the problem.[3]
Garbage collection does not traditionally manage limited resources other than memory that typical programs use, such as network sockets, database handles, user interaction windows, and file and device descriptors. Methods used to manage such resources, particularly destructors, may suffice as well to manage memory, leaving no need for GC. Some GC systems allow such other resources to be associated with a region of memory that, when collected, causes the other resource to be reclaimed; this is called finalization. Finalization may introduce complications limiting its usability, such as intolerable latency between disuse and reclaim of especially limited resources.
The basic principles of garbage collection are:
1. Find data objects in a program that cannot be accessed in the future2. Reclaim the resources used by those objects
By making manual memory deallocation unnecessary (and often forbidding it), garbage collection frees the programmer from having to worry about releasing objects that are no longer needed, which can otherwise consume a significant amount of design effort. It also aids programmers in their efforts to make programs more stable, because it prevents several classes of runtime errors. For example, it prevents dangling pointer errors, where a reference to a deallocated object is used. (The pointer still points to the location in memory where the object or data was, even though the object or data has since been deleted and the memory may now be used for other purposes. This can, and often does, lead to storage violation errors that are extremely difficult to resolve.)
47
Many computer languages require garbage collection, either as part of the language specification (e.g., Java, C#, and most scripting languages) or effectively for practical implementation (e.g., formal languages like lambda calculus); these are said to be garbage collected languages. Other languages were designed for use with manual memory management, but have garbage collected implementations available (e.g., C, C++). Some languages, like Ada, Modula-3, and C++/CLI allow both garbage collection and manual memory management to co-exist in the same application by using separate heaps for collected and manually managed objects; others, like D, are garbage collected but allow the user to manually delete objects and also entirely disable garbage collection when speed is required. In any case, it is far easier to implement garbage collection as part of the language's compiler and runtime system,[citation needed] but post hoc GC systems exist, including ones that do not require recompilation. The garbage collector will almost always be closely integrated with the memory allocator.
Benefits
Garbage collection frees the programmer from manually dealing with memory allocation and deallocation. As a result, certain categories of bugs are eliminated or substantially reduced:
Dangling pointer bugs, which occur when a piece of memory is freed while there are still pointers to it, and one of those pointers is then used.
Double free bugs, which occur when the program attempts to free a region of memory that is already free.
Certain kinds of memory leaks, in which a program fails to free memory occupied by objects that will not be used again, leading, over time, to memory exhaustion.
Disadvantages
Typically, garbage collection has certain disadvantages.
Garbage collection is a process that consumes limited computing resources in deciding what memory is to be freed and when, reconstructing facts that may have been known to the programmer. The penalty for the convenience of not annotating memory usage manually in the code is overhead leading, potentially, to decreased performance.
The point when the garbage is actually collected can be unpredictable, resulting in delays scattered throughout a session. Unpredictable delays can be unacceptable in real-time environments such as device drivers, or in transaction processing. Recursive algorithms that take advantage of automatic storage management often delay automatic release of stack objects until after the final call has completed, causing increased memory requirements.
Memory may leak despite the presence of a garbage collector if references to unused objects are not themselves manually disposed of. Researchers draw a distinction between "physical" and "logical" memory leaks. In a physical memory leak, the last pointer to a region of allocated memory is removed, but the memory is not freed. In a logical memory
48
leak, a region of memory is still referenced by a pointer, but is never actually used. [4]
Garbage collectors generally can do nothing about logical memory leaks.
Perhaps the most significant problem is that programs that rely on garbage collectors often exhibit poor locality (interacting badly with cache and virtual memory systems), occupying more address space than the program actually uses at any one time, and touching otherwise idle pages. These may combine in a phenomenon called thrashing, in which a program spends more time copying data between various grades of storage than performing useful work. They may make it impossible for a programmer to reason about the performance effects of design choices, making performance tuning difficult. They can lead garbage-collecting programs to interfere with other programs competing for resources.
Tracing garbage collectors
Tracing garbage collectors are the most common type of garbage collector. They first determine which objects are reachable (or potentially reachable), and then discard all remaining objects.
Reachability of an object
Informally, a reachable object can be defined as an object for which there exists some variable in the program environment that leads to it, either directly or through references from other reachable objects. More precisely, objects can be reachable in only two ways:
1. A distinguished set of objects are assumed to be reachable—these are known as the roots. Typically, these include all the objects referenced from anywhere in the call stack (that is, all local variables and parameters in the functions currently being invoked), and any global variables.
2. Anything referenced from a reachable object is itself reachable; more formally, reachability is a transitive closure.
The reachability definition of "garbage" is not optimal, insofar as the last time a program uses an object could be long before that object falls out of the environment scope. A distinction is sometimes drawn between syntactic garbage, those objects the program cannot possibly reach, and semantic garbage, those objects the program will in fact never again use. For example:
Object x = new Foo();Object y = new Bar();x = new Quux();/* at this point, we know that the Foo object will * never be accessed - it is syntactic garbage */ if(x.check_something()) { x.do_something(y);}System.exit(0);
49
/* in the above block, y *could* be semantic garbage, * but we won't know until x.check_something() returns * some value - if it returns at all */
The problem of precisely identifying semantic garbage can easily be shown to be partially decidable: a program that allocates an object X, runs an arbitrary input program P, and uses X if and only if P finishes would require a semantic garbage collector to solve the halting problem. Although conservative heuristic methods for semantic garbage detection remain an active research area, essentially all practical garbage collectors focus on syntactic garbage.[citation needed]
Another complication with this approach is that, in languages with both reference types and unboxed value types, the garbage collector needs to somehow be able to distinguish which variables on the stack or fields in an object are regular values and which are references; since in memory, an integer and a reference might look alike. This often runs into problems with generic programming, where you may want to have a generic container class be able to hold various types of elements (e.g. the programmers want to have a list that can hold integers or references, or a mix of them). The garbage collector then needs to know whether to treat the element as a reference and follow it, or whether it is a primitive value. One common solution is the use of Tagged pointers.
50
Just-in-time compilation
In computing, just-in-time compilation (JIT), also known as dynamic translation, is a technique for improving the runtime performance of a computer program. JIT builds upon two earlier ideas in run-time environments: bytecode compilation and dynamic compilation. It converts code at runtime prior to executing it natively, for example bytecode into native machine code. The performance improvement over interpreters originates from caching the results of translating blocks of code, and not simply reevaluating each line or operand each time it is met (see Interpreted language). It also has advantages over statically compiling the code at development time, as it can recompile the code if this is found to be advantageous, and may be able to enforce security guarantees. Thus JIT can combine some of the advantages of interpretation and static (ahead-of-time) compilation.
Several modern runtime environments, such as Microsoft's .NET Framework and most implementations of Java, rely on JIT compilation for high-speed code execution.
Overview
In a bytecode-compiled system, source code is translated to an intermediate representation known as bytecode. Bytecode is not the machine code for any particular computer, and may be portable among computer architectures. The bytecode may then be interpreted by, or run on, a virtual machine. A just-in-time compiler can be used as a way to speed up execution of bytecode. At the time the bytecode is run, the just-in-time compiler will compile some or all of it to native machine code for better performance. This can be done per-file, per-function or even on any arbitrary code fragment; the code can be compiled when it is about to be executed (hence the name "just-in-time").
In contrast, a traditional interpreted virtual machine will simply interpret the bytecode, generally with much lower performance. Some interpreters even interpret source code, without the step of first compiling to bytecode, with even worse performance. Statically compiled code or native code is compiled prior to deployment. A dynamic compilation environment is one in which the compiler can be used during execution. For instance, most Common Lisp systems have a compile function which can compile new functions created during the run. This provides many of the advantages of JIT, but the programmer, rather than the runtime, is in control of what parts of the code are compiled. This can also compile dynamically generated code, which can, in many scenarios, provide substantial performance advantages over statically compiled code, as well as over most JIT systems.
A common goal of using JIT techniques is to reach or surpass the performance of static compilation, while maintaining the advantages of bytecode interpretation: Much of the "heavy lifting" of parsing the original source code and performing basic optimization is often handled at compile time, prior to deployment: compilation from bytecode to machine code is much faster than compiling from source. The deployed bytecode is portable, unlike native code. Since the
51
runtime has control over the compilation, like interpreted bytecode, it can run in a secure sandbox. Compilers from bytecode to machine code are easier to write, because the portable bytecode compiler has already done much of the work.
JIT code generally offers far better performance than interpreters. In addition, it can in some or many cases offer better performance than static compilation, as many optimizations are only feasible at run-time:
1. The compilation can be optimized to the targeted CPU and the operating system model where the application runs. For example JIT can choose SSE2 CPU instructions when it detects that the CPU supports them. To obtain this level of optimization specificity with a static compiler, one must either compile a binary for each intended platform/architecture, or else include multiple versions of portions of the code within a single binary.
2. The system is able to collect statistics about how the program is actually running in the environment it is in, and it can rearrange and recompile for optimum performance. However, some static compilers can also take profile information as input.
3. The system can do global code optimizations (e.g. inlining of library functions) without losing the advantages of dynamic linking and without the overheads inherent to static compilers and linkers. Specifically, when doing global inline substitutions, a static compiler must insert run-time checks and ensure that a virtual call would occur if the actual class of the object overrides the inlined method (however, this need not be the case for languages employing a static type discipline).
4. Although this is possible with statically compiled garbage collected languages, a bytecode system can more easily rearrange memory for better cache utilization.
Startup delay and optimizations
JIT typically causes a slight delay in initial execution of an application, due to the time taken to load and compile the bytecode. Sometimes this delay is called "startup time delay". In general, the more optimization JIT performs, the better code it will generate, but the initial delay will also increase. A JIT compiler therefore has to make a trade-off between the compilation time and the quality of the code it hopes to generate. However, it seems that much of the startup time is sometimes due to IO-bound operations rather than JIT compilation (for example, the rt.jar class data file for the Java Virtual Machine is 40 MB and the JVM must seek a lot of data in this huge file).[1]
One possible optimization, used by Sun's HotSpot Java Virtual Machine, is to combine interpretation and JIT compilation. The application code is initially interpreted, but the JVM monitors which sequences of bytecode are frequently executed and translates them to machine code for direct execution on the hardware. For bytecode which is executed only a few times, this saves the compilation time and reduces the initial latency; for frequently executed bytecode, JIT compilation is used to run at high speed, after an initial phase of slow interpretation. Additionally, since a program spends most time executing a minority of its code, the saved
52
compilation time is big. Finally, during the initial code interpretation, execution statistics can be collected before compilation, which helps to perform better optimization.[2]
The correct tradeoff can vary due to circumstances. For example, Sun's Java Virtual Machine has two major modes—client and server. In client mode, minimal compilation and optimization is performed, to reduce startup time. In server mode, extensive compilation and optimization is performed, to maximize performance once the application is running by sacrificing startup time. Other Java just-in-time compilers have used a runtime measurement of the number of times a method has executed combined with the bytecode size of a method as a heuristic to decide when to compile.[3] Still another uses the number of times executed combined with the detection of loops.[4] In general, it is much harder to accurately predict which methods to optimize in short-running applications than in long-running ones.[5]
Native Image Generator (Ngen) by Microsoft is another approach at reducing the initial delay.[6]
Ngen pre-compiles (or "pre-jits") bytecode in a Common Intermediate Language image into machine native code. As a result, no runtime compilation is needed. .NET framework 2.0 shipped with Visual Studio 2005 runs Ngen on all of the Microsoft library DLLs right after the installation. Pre-jitting provides a way to improve the startup time. However, the quality of code it generates might not be as good as the one that is jitted, for many of the same reasons why statically compiled code cannot be as good as JIT compiled code in the extreme case.
There also exist Java implementations that combine an AOT (ahead-of-time) compiler with either a JIT compiler (Excelsior JET) or interpreter (GNU Compiler for Java.)
.NET metadata
.NET metadata, in the Microsoft .NET framework, refers to certain data structures embedded within the Common Intermediate Language code that describes the high-level structure of the code. Metadata describes all classes and class members that are defined in the assembly, and the classes and class members that the current assembly will call from another assembly. The metadata for a method contains the complete description of the method, including the class (and the assembly that contains the class), the return type and all of the method parameters.
A .NET language compiler will generate the metadata and store this in the assembly containing the CIL. When the CLR executes CIL it will check to make sure that the metadata of the called method is the same as the metadata that is stored in the calling method. This ensures that a method can only be called with exactly the right number of parameters and exactly the right parameter types.
53
Database Constraints
Functional dependency
A functional dependency (FD) is a constraint between two sets of attributes in a relation from a database.
Given a relation R, a set of attributes X in R is said to functionally determine another attribute Y, also in R, (written X → Y) if and only if each X value is associated with precisely one Y value. Customarily we call X the determinant set and Y the dependent attribute. Thus, given a tuple and the values of the attributes in X, one can determine the corresponding value of the Y attribute. For the purposes of simplicity, given that X and Y are sets of attributes in R, X → Y denotes that X functionally determines each of the members of Y - in this case Y is known as the dependent set. Thus, a candidate key is a minimal set of attributes that functionally determine all of the attributes in a relation.
(Note: the "function" being discussed in "functional dependency" is the function of identification.)
A functional dependency FD: X → Y is called trivial if Y is a subset of X.
The determination of functional dependencies is an important part of designing databases in the relational model, and in database normalization and denormalization. The functional dependencies, along with the attribute domains, are selected so as to generate constraints that would exclude as much data inappropriate to the user domain from the system as possible.
For example, suppose one is designing a system to track vehicles and the capacity of their engines. Each vehicle has a unique vehicle identification number (VIN). One would write VIN → EngineCapacity because it would be inappropriate for a vehicle's engine to have more than one capacity. (Assuming, in this case, that vehicles only have one engine.) However, EngineCapacity → VIN, is incorrect because there could be many vehicles with the same engine capacity.
This functional dependency may suggest that the attribute EngineCapacity be placed in a relation with candidate key VIN. However, that may not always be appropriate. For example, if that functional dependency occurs as a result of the transitive functional dependencies VIN → VehicleModel and VehicleModel → EngineCapacity then that would not result in a normalized relation.
Given that X, Y, and Z are sets of attributes in a relation R, one can derive several properties of functional dependencies. Among the most important are Armstrong's axioms, which are used in database normalization:
Subset Property (Axiom of Reflexivity): If Y is a subset of X, then X → Y Augmentation (Axiom of Augmentation): If X → Y, then XZ → YZ
Transitivity (Axiom of Transitivity): If X → Y and Y → Z, then X → Z
From these rules, we can derive these secondary rules:
Union: If X → Y and X → Z, then X → YZ Decomposition: If X → YZ, then X → Y and X → Z
Pseudotransitivity: If X → Y and YZ → W, then XZ → W
Augmentation (Axiom of Augmentation): If X → Y and W is subset of Z, then XZ → YW
Multivalued dependencyIn database theory, multivalued dependency is a full constraint between two sets of attributes in a relation.In contrast to the functional dependency, the multivalued dependency requires that certain tuples be present in a relation. Therefore, a multivalued dependency is a special case of tuple-generating dependency. The multivalued dependency plays a role in the 4NF database normalization.
Definitions
full constraintA constraint which expresses something about all attributes in a database. (In contrast to an embedded constraint.) That a multivalued dependency is a full constraint follows from its definition,as where it says something about the attributes R − β.
tuple-generating dependencyA dependency which explicitly requires certain tuples to be present in the relation.
trivial multivalued dependency
A multivalued dependency which involves all the attributes of a relation i.e. . A trivial multivalued dependency implies, for tuples t1 and t2, tuples t3 and t4 which are equal to t1 and t2.
In mathematics, a transitive dependency is a functional dependency which holds by virtue of transitivity. A transitive dependency can occur only in a relation that has three or more attributes. Let A, B, and C designate three distinct attributes (or distinct collections of attributes) in the relation. Suppose all three of the following conditions hold:
1. A → B2. It is not the case that B → A
3. B → C
Then the functional dependency A → C (which follows from 1 and 3 by the axiom of transitivity) is a transitive dependency.
Another way to look at it is a bit like a stepping stone across a river. If we consider a relation to be our river with a primary key A to be the far bank of the river and our non-key attribute C to be our current location, in order to get to A, our primary key, we need to step on a stepping stone B, another non-key attribute, to help us get there. Of course we could jump directly from C to A, but it is easier, and we are less likely to fall in, if we use our stepping stone B. Therefore current location C is transitively dependent on A through our stepping stone B.
Query stringWhen a web page is requested via the Hypertext Transfer Protocol, the server locates a file in its file system based on the requested URL. This file may be a regular file or a program. In the second case, the server may (depending on its configuration) run the program, sending its output as the required page. The query string is a part of the URL which is passed to the program. Its use permits data to be passed from the HTTP client (often a web browser) to the program which generates the web page.
.NET pages, known officially as "web forms", are the main building block for application development.[8] Web forms are contained in files with an ".aspx" extension; these files typically contain static (X)HTML markup, as well as markup defining server-side Web Controls and User Controls where the developers place all the required static and dynamic content for the web page. Additionally, dynamic code which runs on the server can be placed in a page within a block <% -- dynamic code -- %> which is similar to other web development technologies such as PHP, JSP, and ASP, but this practice is generally discouraged except for the purposes of data binding since it requires more calls when rendering the page.[citation needed]
Note that this sample uses code "inline", as opposed to code-behind.
<%@ Page Language="C#" %> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><script runat="server"> protected void Page_Load(object sender, EventArgs e) { Label1.Text = DateTime.Now.ToLongTimeString(); } </script> <html xmlns="http://www.w3.org/1999/xhtml"><head runat="server"> <title>Sample page</title></head><body> <form id="form1" runat="server"> <div> The current time is: <asp:Label runat="server" id="Label1" /> </div> </form> </body></html>
Code-behind model
Microsoft recommends dealing with dynamic program code by using the code-behind model, which places this code in a separate file or in a specially designated script tag. Code-behind files typically have names like MyPage.aspx.cs or MyPage.aspx.vb while the page file is MyPage.aspx (same filename as the page file (ASPX), but with the final extension denoting the
57
page language). This practice is automatic in Microsoft Visual Studio and other IDEs. When using this style of programming, the developer writes code to respond to different events, like the page being loaded, or a control being clicked, rather than a procedural walk through the document.
ASP.NET's code-behind model marks a departure from Classic ASP in that it encourages developers to build applications with separation of presentation and content in mind. In theory, this would allow a web designer, for example, to focus on the design markup with less potential for disturbing the programming code that drives it. This is similar to the separation of the controller from the view in model-view-controller frameworks.
The above tag is placed at the beginning of the ASPX file. The CodeFile property of the @ Page directive specifies the file (.cs or .vb) acting as the code-behind while the Inherits property specifies the Class the Page derives from. In this example, the @ Page directive is included in SampleCodeBehind.aspx, then SampleCodeBehind.aspx.cs acts as the code-behind for this page:
using System; namespace Website{
public partial class SampleCodeBehind : System.Web.UI.Page{
In this case, the Page_Load() method is called every time the ASPX page is requested. The programmer can implement event handlers at several stages of the page execution process to perform processing.
User controls
An event bubbling mechanism provides the ability to pass an event fired by a user control up to its containing page.
Custom Controls
Programmers can also build Custom controls for ASP.NET applications. Unlike User controls, these controls don't have an ASCX markup-file, having all their code compiled into a DLL-file.
58
Such Custom controls can be used across multiple web-applications and Visual Studio projects (which is not allowed with User controls). By using a Register directive, the control is loaded from the DLL.
Rendering technique
ASP.NET uses a visited composites rendering technique. During compilation, the template (.aspx) file is compiled into initialization code which builds a control tree (the composite) representing the original template. Literal text goes into instances of the Literal control class, and server controls are represented by instances of a specific control class. The initialization code is combined with user-written code (usually by the assembly of multiple partial classes) and results in a class specific for the page. The page doubles as the root of the control tree.
Actual requests for the page are processed through a number of steps. First, during the initialization steps, an instance of the page class is created and the initialization code is executed. This produces the initial control tree which is now typically manipulated by the methods of the page in the following steps. As each node in the tree is a control represented as an instance of a class, the code may change the tree structure as well as manipulate the properties/methods of the individual nodes. Finally, during the rendering step a visitor is used to visit every node in the tree, asking each node to render itself using the methods of the visitor. The resulting HTML output is sent to the client.
After the request has been processed, the instance of the page class is discarded and with it the entire control tree. This is usually a source of confusion among novice ASP.NET programmers that rely on class instance members that are lost with every page request/response cycle..
State management
ASP.NET applications are hosted by a web server and are accessed using the stateless HTTP protocol. As such, if an application uses stateful interaction, it has to implement state management on its own. ASP.NET provides various functions for state management. Conceptually, Microsoft treats "state" as GUI state; problems may arise if an application needs to keep track of "data state", for example, a finite state machine which may be in a transient state between requests (lazy evaluation) or which takes a long time to initialize.
Application state
Application state is held by a collection of shared user-defined variables. These are set and initialized when the Application_OnStart event fires on the loading of the first instance of the applications and are available till the last instance exits. Application state variables are accessed using the Applications collection, which provides a wrapper for the application state variables. Application state variables are identified by name.[9]
59
Session state
Server-side session state is held by a collection of user-defined session variables, which are persisted during a user session. These variables, accessed using the Session collection, are unique to each session instance. Then variables can be set to be automatically destroyed after a defined time of inactivity, even if the session does not end. Client-side user session is maintained by either an cookie or by encoding the session ID in the URL itself.[9]
ASP.NET supports three modes of persistence for session variables:[9]
In Process Mode
When using In Process Mode, the session variables are maintained within the ASP.NET process. This is the fastest way; however, in this mode the variables are destroyed when the ASP.NET process is recycled or shut down.
ASPState Mode
In this mode, ASP.NET runs a separate Windows service that maintains the state variables. As state management happens outside the ASP.NET process and .NET Remoting must be utilized by the ASP.NET engine to access the data there is a negative impact on performance compared to the In Process mode. This mode allows an ASP.NET application to be load-balanced and scaled across multiple servers and because the state management service runs independently of ASP.NET, the session variables can persist across ASP.NET process shutdowns. However, since session state server runs as a single instance it is still a single point of failure so far as session state is concerned. The session-state service cannot be load balanced and there are restrictions on types that can be stored in a session variable.
SqlServer Mode
In this mode, the state variables are stored in a database allowing session variables to be persisted across ASP.NET process shutdowns. The main advantage of this mode is that it allows the application to balance load on a server cluster, sharing sessions between servers. This is the slowest method of session state management in ASP.NET.
View state
View state refers to the page-level state management mechanism, utilized by the HTML pages emitted by ASP.NET applications to maintain the state of the web form controls and widgets. The state of the controls is encoded and sent to the server at every form submission in a hidden field known as __VIEWSTATE. The server sends back the variable so that when the page is re-rendered, the controls render at their last state. At the server side, the application may change the viewstate, if the processing requires a change of state of any control. The states of individual
60
controls are decoded at the server, and are available for use in ASP.NET pages using the ViewState collection.[10] [11]
The main use for this is to preserve form information across postbacks. View state is turned on by default and normally serializes the data in every control on the page regardless of whether it is actually used during a postback. This behavior can (and should be) modified, however, as View state can be disabled on a per-control, per-page, or server-wide basis.
Developers need to be wary of storing sensitive or private information in the View state of a page or control, as the base64 string containing the view state data can easily be de-serialized. By default, View state does not encrypt the __VIEWSTATE value. Encryption can be enabled on a server-wide (and server-specific) basis, allowing for a certain level of security to be maintained.[12]
Server-side caching
ASP.NET offers a "Cache" object that is shared across the application and can also be used to store various objects. The "Cache" object holds the data only for a specified amount of time and is automatically cleaned after the session time-limit elapses.
Other
Other means of state management that are supported by ASP.NET are cookies, caching, and using the query string.
Template engine
When first released, ASP.NET lacked a template engine. Because the .NET framework is object-oriented and allows for inheritance, many developers would define a new base class that inherits from "System.Web.UI.Page", write methods here that render HTML, and then make the pages in their application inherit from this new class. While this allows for common elements to be reused across a site, it adds complexity and mixes source code with markup. Furthermore, this method can only be visually tested by running the application - not while designing it. Other developers have used include files and other tricks to avoid having to implement the same navigation and other elements in every page.
ASP.NET 2.0 introduced the concept of "master pages", which allow for template-based page development. A web application can have one or more master pages, which, beginning with ASP.NET 3.5, can be nested.[13] Master templates have place-holder controls, called ContentPlaceHolders to denote where the dynamic content goes, as well as HTML and JavaScript shared across child pages.
Child pages use those ContentPlaceHolder controls, which must be mapped to the place-holder of the master page that the content page is populating. The rest of the page is defined by the
61
shared parts of the master page, much like a mail merge in a word processor. All markup and server controls in the content page must be placed within the ContentPlaceHolder control.
When a request is made for a content page, ASP.NET merges the output of the content page with the output of the master page, and sends the output to the user.
The master page remains fully accessible to the content page. This means that the content page may still manipulate headers, change title, configure caching etc. If the master page exposes public properties or methods (e.g. for setting copyright notices) the content page can use these as well.
Cache
In computer science, a cache (pronounced /ˈkæʃ/ kash) is a component that improves performance by transparently storing data such that future requests for that data can be served faster. The data that is stored within a cache might be values that have been computed earlier or duplicates of original values that are stored elsewhere. If requested data is contained in the cache (cache hit), this request can be served by simply reading the cache, which is comparably faster. Otherwise (cache miss), the data has to be recomputed or fetched from its original storage location, which is comparably slower. Hence, the more requests can be served from the cache the better the overall system performance is.
As opposed to a buffer, which is managed explicitly by a client, a cache stores data transparently: This means that a client who is requesting data from a system is not aware that the cache exists. Therefore the name cache (from French "cacher", to conceal).
To be cost efficient and to enable an efficient lookup of data, caches are comparably small. Nevertheless, caches have proven extremely effective in many areas of computing because access patterns in typical computer applications have locality of reference. References exhibit temporal locality if data is requested again that has been recently requested already. References exhibit spatial locality if data is requested that is physically stored close to data that has been requested already.
Hardware implements cache as a block of memory for temporary storage of data likely to be used again. CPUs and hard drives frequently use a cache, as do web browsers and web servers.
A cache is made up of a pool of entries. Each entry has a datum (a nugget of data) - a copy of the same datum in some backing store. Each entry also has a tag, which specifies the identity of the datum in the backing store of which the entry is a copy.
When the cache client (a CPU, web browser, operating system) needs to access a datum presumed to exist in the backing store, it first checks the cache. If an entry can be found with a tag matching that of the desired datum, the datum in the entry is used instead. This situation is known as a cache hit. So, for example, a web browser program might check its local cache on disk to see if it has a local copy of the contents of a web page at a particular URL. In this example, the URL is the tag, and the contents of the web page is the datum. The percentage of accesses that result in cache hits is known as the hit rate or hit ratio of the cache.
The alternative situation, when the cache is consulted and found not to contain a datum with the desired tag, has become known as a cache miss. The previously uncached datum fetched from the backing store during miss handling is usually copied into the cache, ready for the next access.
During a cache miss, the CPU usually ejects some other entry in order to make room for the previously uncached datum. The heuristic used to select the entry to eject is known as the replacement policy. One popular replacement policy, "least recently used" (LRU), replaces the least recently used entry (see cache algorithms). More efficient caches compute use frequency against the size of the stored contents, as well as the latencies and throughputs for both the cache and the backing store. While this works well for larger amounts of data, long latencies and slow throughputs, such as experienced with a hard drive and the Internet, it is not efficient for use with a CPU cache.[citation needed]
When a system writes a datum to the cache, it must at some point write that datum to the backing store as well. The timing of this write is controlled by what is known as the write policy.
In a write-through cache, every write to the cache causes a synchronous write to the backing store.
Alternatively, in a write-back (or write-behind) cache, writes are not immediately mirrored to the store. Instead, the cache tracks which of its locations have been written over and marks these locations as dirty. The data in these locations is written back to the backing store when those data are evicted from the cache, an effect referred to as a lazy write. For this reason, a read miss in a write-back cache (which requires a block to be replaced by another) will often require two memory accesses to service: one to retrieve the needed datum, and one to write replaced data from the cache to the store.
Other policies may also trigger data write-back. The client may make many changes to a datum in the cache, and then explicitly notify the cache to write back the datum.
63
No-write allocation is a cache policy which caches only processor reads, thus avoiding the need for write-back or write-through when the old value of the datum was absent from the cache prior to the write.
Entities other than the cache may change the data in the backing store, in which case the copy in the cache may become out-of-date or stale. Alternatively, when the client updates the data in the cache, copies of that data in other caches will become stale. Communication protocols between the cache managers which keep the data consistent are known as coherency protocols.
Applications
CPU cache
Small memories on or close to the CPU can operate faster than the much larger main memory. Most CPUs since the 1980s have used one or more caches, and modern high-end embedded, desktop and server microprocessors may have as many as half a dozen, each specialized for a specific function.
Disk cache
While CPU caches are generally managed entirely by hardware, a variety of software manages other caches. The page cache in main memory, which is an example of disk cache, is managed by the operating system kernel.
While the hard drive's hardware disk buffer is sometimes misleadingly referred to as "disk cache", its main functions are write sequencing and read prefetching. Repeated cache hits are relatively rare, due to the small size of the buffer in comparison to HDD's capacity.
In turn, fast local hard disk can cache information held on even slower data storage devices, such as remote servers (web cache) or local tape drives or optical jukeboxes. Such a scheme is the main concept of hierarchical storage management.
Web cache
Web browsers and web proxy servers employ web caches to store previous responses from web servers, such as web pages. Web caches reduce the amount of information that needs to be transmitted across the network, as information previously stored in the cache can often be re-used. This reduces bandwidth and processing requirements of the web server, and helps to improve responsiveness for users of the web.
As of 2009, modern web browsers employ a built-in web cache, but some internet service providers or organizations also use a caching proxy server, which is a web cache that is shared between all users of that network.
64
Another form of cache is P2P caching, where the files most sought for by peer-to-peer applications are stored in an ISP cache to accelerate P2P transfers.
Other caches
The BIND DNS daemon caches a mapping of domain names to IP addresses, as does a resolver library.
Write-through operation is common when operating over unreliable networks (like an Ethernet LAN), because of the enormous complexity of the coherency protocol required between multiple write-back caches when communication is unreliable. For instance, web page caches and client-side network file system caches (like those in NFS or SMB) are typically read-only or write-through specifically to keep the network protocol simple and reliable.
Search engines also frequently make web pages they have indexed available from their cache. For example, Google provides a "Cached" link next to each search result. This can prove useful when web pages from a web server are temporarily or permanently inaccessible.
Another type of caching is storing computed results that will likely be needed again, or memoization. ccache, a program that caches the output of the compilation to speed up the second-time compilation, exemplifies this type.
Database caching can substantially improve the throughput of database applications, for example in the processing of indexes, data dictionaries, and frequently used subsets of data.
Distributed caching [1] uses caches spread across different networked hosts.
The difference between buffer and cache
The terms "buffer" and "cache" are not mutually exclusive and the functions are frequently combined; however, there is a difference in intent. A buffer is a temporary memory location, that is traditionally used because CPU instructions cannot directly address data stored in peripheral devices. Thus, addressable memory is used as intermediate stage.
Additionally such a buffer may be feasible when a large block of data is assembled or disassembled (as required by a storage device), or when data may be delivered in a different order than that in which it is produced. Also a whole buffer of data is usually transferred sequentially (for example to hard disk), so buffering itself sometimes increases transfer performance. These benefits are present even if the buffered data are written to the buffer once and read from the buffer once.
A cache also increases transfer performance. A part of the increase similarly comes from the possibility that multiple small transfers will combine into one large block. But the main performance-gain occurs because there is a good chance that the same datum will be read from cache multiple times, or that written data will soon be read. A cache's sole purpose is to reduce
65
accesses to the underlying slower storage. Cache is also usually an abstraction layer that is designed to be invisible from the perspective of neighbouring layers.
Sessions
In computer science, in particular networking, a session is a semi-permanent interactive information interchange, also known as a dialogue, a conversation or a meeting, between two or more communicating devices, or between a computer and user (see Login session). A session is set up or established at a certain point in time, and torn down at a later point in time. An established communication session may involve more than one message in each direction. A session is typically, but not always, stateful, meaning that at least one of the communicating parts needs to save information about the session history in order to be able to communicate, as opposed to stateless communication, where the communication consists of independent requests with responses.
Communication sessions may be implemented as part of protocols and services at the application layer, at the session layer or at the transport layer in the OSI model.
Application layer examples: o HTTP sessions, which allow associating information with individual visitors
o A telnet remote login session
Session layer example:
o A Session Initiation Protocol (SIP) based Internet phone call
Transport layer example:
o A TCP session, which is synonymous to a TCP virtual circuit, a TCP connection, or an established TCP socket.
In the case of transport protocols that do not implement a formal session layer (e.g., UDP) or where sessions at the session layer are generally very short-lived (e.g., HTTP), sessions are maintained by a higher level program using a method defined in the data being exchanged. For example, an HTTP exchange between a browser and a remote host may include an HTTP cookie which identifies state, such as a unique session ID, information about the user's preferences or authorization level.
Protocol version HTTP/1.1 makes it possible to reuse the same TCP session for a sequence of service requests and responses (a sequence of file transfers) in view to reduce the session establishment time, while HTTP/1.0 only allows a single request and response during one TCP session. However, this transport layer session mechanism should not be confused with a so-called HTTP session, since it does not last a sufficiently long time, and does not provide application level interactive services such as dynamic web pages.
66
Software implementation
TCP sessions are typically implemented in software using child processes and/or multithreading, where a new process or thread is created when the computer establishes or joins a session. HTTP sessions are typically not implemented using one thread per session, but by means of a database with information about the state of each session. The advantage with multiple processes or threads is relaxed complexity of the software, since each thread is an instance with its own history and encapsulated variables. The disadvantage is large overhead in terms of system resources, and that the session may be interrupted if the system is restarted.
When a client may connect to any in a cluster of servers, a special problem is encountered in maintaining consistency when the servers must maintain session state. The client must either be directed to the same server for the duration of the session, or the servers must transmit server-side session information via a shared file system or database. Otherwise, the client may reconnect to a different server than the one it started the session with, which will cause problems when the new server does not have access to the stored state of the old one.
Server side web sessions
Server-side sessions are handy and efficient, but can become difficult to handle in conjunction with load-balancing/high-availability systems and are not usable at all in embedded systems with no storage. The load-balancing problem can be solved by using shared storage or by applying forced peering between each client and a single server in the cluster, although this can compromise system efficiency and load distribution.
A method of using server-side sessions in systems without mass-storage is to reserve a portion of RAM for storage of session data. This method is applicable for servers with a limited number of clients (e.g. router or access point with infrequent or disallowed access to more than one client at a time).
In the two scenarios above, using client-side sessions could provide advantages over server-side sessions: in the first case by removing the limitations applied to load-balancing algorithms (which usually translates to load distribution optimisation), and in the second case by allowing the use of sessions in web applications when server disk space or RAM is not available or sufficient for this storage.
Client side web sessions
Client-side sessions use cookies and cryptographic techniques to maintain state without storing as much data on the server. When presenting a dynamic web page, the server sends the current state data to the client (web browser) in the form of a cookie. The client saves the cookie in
67
memory or on disk. With each successive request, the client sends the cookie back to the server, and the server uses the data to "remember" the state of the application for that specific client and generate an appropriate response.
This mechanism may work well in some contexts; however, data stored on the client is vulnerable to tampering by the user or by software that has access to the client computer. To use client-side sessions where confidentiality and integrity are required, the following must be guaranteed:
1. Confidentiality: Nothing apart from the server should be able to interpret session data.2. Data integrity: Nothing apart from the server should manipulate session data
(accidentally or maliciously).
3. Authenticity: Nothing apart from the server should be able to initiate valid sessions.
To accomplish this, the server needs to encrypt the session data before sending it to the client, and modification of such information by any other party should be prevented via cryptographic means.
Transmitting state back and forth with every request is only practical when the size of the cookie is small. In essence, client-side sessions trade server disk space for the extra bandwidth that each web request will require. Moreover, web browsers limit the number and size of cookies that may be stored by a web site. To improve efficiency and allow for more session data, the server may compress the data before creating the cookie, decompressing it later when the cookie is returned by the client.
HTTP session token
A session token is a unique identifier (usually in the form of a hash generated by a hash function) that is generated and sent from a server to a client to identify the current interaction session. The client usually stores and sends the token as an HTTP cookie and/or sends it as a parameter in GET or POST queries. The reason to use session tokens is that the client only has to handle the identifier - all session data is stored on the server (usually in a database, to which the client does not have direct access) linked to that identifier. Examples of the names that some programming languages use when naming their HTTP cookie include JSESSIONID (JSP), PHPSESSID (PHP), and ASPSESSIONID (ASP).
