Date post: | 31-Mar-2015 |
Category: |
Documents |
Upload: | edgardo-meakin |
View: | 214 times |
Download: | 0 times |
Ingres 10 Key FeaturesGIUA 2010, Würzburg September 30
Sarkaut Mohn
Ingres 10 Key Features
New Ingres 10 Features Derived by Ingres ISV Partners– Multi Version Concurrency Control (MVCC)– Scalar Subqueries– Batch Query Execution– Data Encryption– Rename Objects– Long Identifiers– Bulk Load Performance
Confidential — © 2010 Ingres Corporation Slide 2
MVCCWhy We Need It?
Confidential — © 2010 Ingres Corporation Slide 3
With MVCC, readers are no longer blocked by writers Likewise, writers are no longer blocked by readers
Update TransactionTime
• Updates row A in table X
Read Transaction with Traditional Locking
• Attempts to read row A in table X
Read Transaction with MVCC Locking
• Successfully reads row A from table X
• Commits transaction• Read from table X
finally succeeds after update has committed
• Successfully reads row B from table Y
• Updates row B in table Y
No applicationhangs
Better performance!
BLOCKED because read cannot acquire lock held by update!
Data read is pre-update, so no locking required
MVCCWhy We Need It?
Existing applications originally written for DBMSs like Oracle that support MVCC depend on the high concurrency it provides
Avoids performance problems and application hangs that Ingres encounters with traditional (ANSI Isolation Level based) locking
Applications can avoid having to read inconsistent data– Ugly workaround of running with dirty read isolation
will no longer be necessary Readers see consitent transaction / statement
snapshot of data based on isolation level
Confidential — © 2010 Ingres Corporation Slide 4
Scalar Subqueries
Confidential — © 2010 Ingres Corporation Slide 5
Ingres currently only allows scalar subqueries to be used in limited contexts
SELECT * FROM t1 WHERE t1.col = (SELECT min(col) from t2 where t2.key = t1.key);
DELETE FROM t WHERE col = (SELECT max(col) FROM t);
Ingres 10 will allow scalar subqueries to be used in more instances
Rewriting existing applications to avoid subqueries is not easily scriptable
Scalar SubqueriesExamples
Confidential — © 2010 Ingres Corporation Slide 6
Usage Example
Projection List of SELECT
SELECT emp_name, dept_name, (SELECT MAX(salary) FROM emp) AS highest_salaryFROM emp;
VALUES clause in INSERT
INSERT INTO credit(name, max_credit) VALUES(‘XYZ’, (SELECT MAX(credit) FROM credit_table WHERE name = ‘XYZ’));
SET clause in UPDATE
UPDATE tab1 t1 SET col = (SELECT sum(col) FROM tab2 t2 WHERE t2.key = t1.key);
CASE expression SELECT vars, CASE (SELECT expr FROM t2 WHERE t2.key = t1.key) WHEN 1 THEN ‘A’ ELSE ‘B’ END FROM t1;
Procedure parameter
EXECUTE PROCEDURE proc (count_param = (SELECT count(*) FROM t));
Batch Query Execution
Any SQL statement can appear in a batch, except:– SELECTs– Row producing procedures– Transaction Statements
Individual statements in the batch can succeed while others can fail, provided the failures aren’t severe
Confidential — © 2010 Ingres Corporation Slide 7
Client Application
Create Table
Insert into Table
Execute Procedure
Ingres DBMS
Creates Table
Inserts into Table
ExecutesProcedure
ok
ok
ok
Non-Batch Execution
JDBC Application
Create TableInsert into TableExecute Procedure
Ingres 10 DBMS
Creates TableInserts into TableExecutesProcedure
okokok
Batch Execution
Fewer round trip messages
Better performance!
Batch Query ExecutionCurrent Supported Interfaces
Java– Use addBatch and executeBatch methods– Unchanged from previous releases
• But now recognized by the DBMS rather than translated in JDBC
• Massive improvement in performance OpenAPI
– New IIapi_batch() function No other interfaces currently supported
– ODBC may come soon
Confidential — © 2010 Ingres Corporation Slide 8
Batch Query ExecutionAdditional Feature Details
Ingres JDBC driver already supports the API interfaces necessary for batch execution– addBatch– executeBatch
Existing JDBC implementation, however, executes the same as non-batch; i.e., no reduction in client-server communication
Batch execution also allows a single statement to be executed with multiple sets of parameters
This form of execution has been optimized for better load performance
Performance of ETL tools that use batch execution in this way will therefore benefit
Confidential — © 2010 Ingres Corporation Slide 9
Batch Query Execution Special Optimization for Insert
Confidential — © 2010 Ingres Corporation Slide 10
JDBC Application
• Prepare INSERT statement
Ingres 10 DBMS
• Load “copy” buffer with each parameter set
• Execute Copy Statement to insert data
• Return results
ok…ok
• Execute “prepare”
• Add “Execute INSERT with parameter set 1” to batch
• Add parameter set 2 to batch
• …• Add parameter set n to
batch• Execute batch
ok
Batch Query ExecutionRelated Parameters
JDBC– System Property: ingres.jdbc.batch.enabled
• Determines whether executeBatch() will translate into batch execution in the DBMS or revert to the pre-10.0 behavior
DBMS– Config Value: ii.<node>.*.batch_copy_optim
• Values “ON” (use the copy optimization), “OFF” (do not use the copy optimization, insert statements are not converted to a copy)
– SQL “SET [NO]BATCH_COPY_OPTIM”• As above for current session only
All parameters are for debug purposes only
Confidential — © 2010 Ingres Corporation Slide 11
Data Encryption
Required for– PCI-DSS 1.2 compliance - credit card information storage
(https://www.pcisecuritystandards.org/)– HIPAA compliance - health information storage
Applies only to data “at rest” Specified at the column level By default, a “salt” and “verification hash” will be
embedded in the encrypted data– Salt – Random bits that further obfuscate the encrypted
data– Verification hash – Collision-resistant hash that allows the
integrity of the decryption to be verified
Confidential — © 2010 Ingres Corporation Slide 12
Data EncryptionSecuring the Encryption Keys
Tables with encrypted columns must specify:– Whether to use AES-128, AES-192, or AES-256 encryption– An encryption passphrase
Passphrase is never stored on disk, only the encrypted table key
Confidential — © 2010 Ingres Corporation Slide 13
Random KeyKey Derived
From Passphrase
AES-256 Encryption
Encrypted Table Key
Encrypted Table Key
Unencrypted Table Key
AES-256 Decryption
To access encrypted columns within a table, encryption passphrase associated with the table must be enabled at server startup
Unencrypted table key can then be used to encrypt/decrypt column data
Encrypted Table Key
Encrypted Table Key
Key Derived From
Passphrase
Rename ObjectsIntroduction
Rename table/column:
Ability to rename a table or a column using new SQL syntax:
No need to drop and recreate the table. No need to drop and add the column. No unload/reload of data.
Rename ObjectsNew SQL Syntax
Table rename
Column rename
[EXEC SQL] RENAME [TABLE] [schema.]<table_name> TO <new_table_name>
[EXEC SQL] ALTER TABLE [schema.]<table_name> RENAME TO <new_table_name>
[EXEC SQL] ALTER TABLE [schema.]<table_name> RENAME [COLUMN] <column_name> TO <new_column_name>
You must own the tables (or whose column) you are renaming.
You cannot rename a system catalog, extended system catalog or any tables owned by super user ‘$ingres’.
You cannot rename columns of such catalogs
Rename ObjectsAccess Restrictions
The name of the new table/column should conform to naming rules for tables.– See SQL Reference Guide, chapter 2.
There should be no table / column already existing in the database with same name and owner– The new table / column name will get the same reltid
or attribute id.
Rename ObjectsNaming Restrictions
Long Identifiers
Ingres object names are currently limited to 32 characters Current limit is problematic for applications like Hibernate that
auto-generate long names Ingres 10 increases most object names to 256 characters
– Objects affected include: tables, columns, procedures, parameters, rules, sequences
– Objects excluded include: databases, users Enhancements made in DBMS to avoid additional memory
overhead if applications do not need long identifiers Pre-existing databases will be converted to support long
identifiers during database upgrade
Confidential — © 2010 Ingres Corporation Slide 18
Bulk Load Performance
CSV (comma separated values) and SSV (semi-colon separated values) delimiters allow COPY to READ / WRITE files
COPY from text file up to 2x faster COPY FROM loading partitioned table bulk loads
into partitions
Confidential — © 2010 Ingres Corporation Slide 19
Other Ingres 10 Features
Confidential — © 2010 Ingres Corporation Slide 20
Application Enablement Features– New BOOLEAN Data Type (TRUE, FALSE, NULL)– JDBC 4.0 Compliance
Additional remote data access performance improvements New Functions – GREATEST, LEAST, NVL, NVL2 Hash Join and Hash Aggregation Improvements Recovery Server Error Handling Ingres for 64-bit Windows 64-bit ODBC Driver
Full Ingres 10 Features
Confidential — © 2010 Ingres Corporation Slide 21
http://community.ingres.com/wiki/Ingres_Roadmap/10
Potential Features Post Ingres 10
Improved Data Partitioning Query Caching Improvements Improved Memory
Management Improved Top N Performance Improved Performance
Monitoring Generate data histograms
within the DBMS Improved UTF-8 Performance Installer Enhancements Improved GeoSpatial Support
Scrollable Cursors for Large Objects
UTF-8 Transliteration Reuse Indexes for Constraints Allow NULLs in Unique
Columns Expanded Usernames Stored Procedure
Enhancements ALTER TABLE Enhancements Implicit Commits for DDL …
Confidential — © 2010 Ingres Corporation Slide 22