Date post: | 15-Jan-2015 |
Category: |
Documents |
Upload: | yuki-morishita |
View: | 9,994 times |
Download: | 6 times |
©2012 DataStax
CQL3 in depth
Yuki MorishitaSoftware Engineer@DataStax / Apache Cassandra Committer
1
Cassandra Conference in Tokyo, 11/29/2012
©2012 DataStax2
• Why CQL3?• CQL3 walkthrough• Defining Schema• Querying / Mutating Data• New features
• Related topics• Native transport
Agenda!
©2012 DataStax
Why CQL3?
3
©2012 DataStax
Cassandra Storage
4
create column family profileswith key_validation_class = UTF8Typeand comparator = UTF8Typeand column_metadata = [ {column_name: first_name, validation_class: UTF8Type}, {column_name: last_name, validation_class: UTF8Type}, {column_name: year, validation_class: IntegerType}];
nobu first_name Nobunaga
last_name Oda
year 1582
row key columns
columns are sortedin comparator order
values are validated by validation_class
©2012 DataStax
Thrift API
• Low level: get, get_slice, mutate...• Directly exposes internal storage
structure• Hard to change the signature of API
5
©2012 DataStax6
Column col = new Column(ByteBuffer.wrap("name".getBytes()));col.setValue(ByteBuffer.wrap("value".getBytes()));col.setTimestamp(System.currentTimeMillis());
ColumnOrSuperColumn cosc = new ColumnOrSuperColumn();cosc.setColumn(col);
Mutation mutation = new Mutation();mutation.setColumn_or_supercolumn(cosc);
List<Mutation> mutations = new ArrayList<Mutation>();mutations.add(mutation);
Map<String, List<Mutation>> cf = new HashMap<String, List<Mutation>>();cf.put("Standard1", mutations);
Map<ByteBuffer, Map<String, List<Mutation>>> records = new HashMap<ByteBuffer, Map<String, List<Mutation>>>();records.put(ByteBuffer.wrap("key".getBytes()), cf);
client.batch_mutate(records, consistencyLevel);
Inserting data with Thrift
©2012 DataStax
... with Cassandra Query Language
7
INSERT INTO “Standard1” (key, name)VALUES (“key”, “value”);
• Introduced in 0.8(CQL), updated in 1.0(CQL2)
• Syntax similar to SQL• More extensible than Thrift API
©2012 DataStax
CQL2 Problems
• Almost 1 to 1 mapping to Thrift API, so not compose with the row-oriented parts of SQL
• No support for CompositeType
8
©2012 DataStax
CQL3
• Maps storage to a more natural rows-and-columns representation using CompositeType• Wide rows are “transposed” and unpacked
into named columns• beta in 1.1, default in 1.2• New features• Collection support
9
©2012 DataStax
CQL3 walkthrough
10
©2012 DataStax
Defining Keyspace
• Syntax is changed from CQL2
11
CREATE KEYSPACE my_keyspace WITH replication = { 'class': 'SimpleStrategy', 'replication_factor': 2};
©2012 DataStax
Defining Static Column Family
12
• “Strict” schema definition (and it’s good thing)• You cannot add column arbitrary• You need ALTER TABLE ... ADD column
first• Columns are defined and sorted using
CompositeType comparator
©2012 DataStax
Defining Static Column Family
13
CREATE TABLE profiles ( user_id text PRIMARY KEY, first_name text, last_name text, year int)
nobu
first_name: Nobunaga
last_name: Oda
year: 1582
user_id
columns are sortedin comparator order
values are validated by type definition
CompositeType(UTF8Type)
user_id | first_name | last_name | year---------+------------+-----------+------ nobu | Nobunaga | Oda | 1582
:
©2012 DataStax
Defining Dynamic Column Family
• Then, how can we add columns dynamically to our time series data like we did before?• Use compound key
14
©2012 DataStax
Compound key
15
CREATE TABLE comments ( article_id uuid, posted_at timestamp, author text, content text, PRIMARY KEY (article_id, posted_at))
550e8400-..
1350499616:author yukim
1350499616:content blah, blah, blah
1368499616:author
article_id
columns are sortedin comparator order,first by date, and thencolumn name
values are validated by type definition
CompositeType(DateType, UTF8Type)
1368499616:content well, well, well...
yukim
1350499616:
1368499616:
©2012 DataStax
Compound key
16
cqlsh:ks> SELECT * FROM comments;
article_id | posted_at | author | content--------------+--------------------------+--------+------------------ 550e8400-... | 1970-01-17 00:08:19+0900 | yukim | blah, blah, blah 550e8400-... | 1970-01-17 05:08:19+0900 | yukim | well, well, well
cqlsh:ks> SELECT * FROM comments WHERE posted_at >= '1970-01-17 05:08:19+0900';
article_id | posted_at | author | content--------------+--------------------------+--------+------------------ 550e8400-... | 1970-01-17 05:08:19+0900 | yukim | well, well, well
©2012 DataStax
Changes worth noting
• Identifiers (keyspace/table/columns names) are always case insensitive by default• Use double quote(“) to force case
• Compaction setting is now map type
17
CREATE TABLE test ( ...) WITH COMPACTION = { 'class': 'SizeTieredCompactionStrategy', 'min_threshold': 2, 'max_threshold': 4
};
©2012 DataStax
Changes worth noting
• system.schema_*• All schema information are stored in system
Keyspace• schema_keyspaces, schema_columnfamilies,
schema_columns• system tables themselves are CQL3 schema
• CQL3 schema are not visible through cassandra-cli’s ‘describe’ command.• use cqlsh’s ‘describe columnfamily’
18
©2012 DataStax
More on CQL3 schema
• Thrift to CQL3 migration• http://www.datastax.com/dev/blog/thrift-to-cql3
• For better understanding• http://www.datastax.com/dev/blog/whats-new-in-cql-3-0• http://www.datastax.com/dev/blog/cql3-evolutions• http://www.datastax.com/dev/blog/cql3-for-cassandra-experts
19
©2012 DataStax
Mutating Data
20
INSERT INTO example (id, name) VALUES (...)
UPDATE example SET f = ‘foo’ WHERE ...
DELETE FROM example WHERE ...
• No more USING CONSISTENCY• Consistency level setting is moved to protocol
level
©2012 DataStax
Batch Mutate
• Batches are atomic by default from 1.2• does not mean mutations are isolated
(mutation within a row is isolated from 1.1)• some performance penalty because of batch
log process
21
BEGIN BATCH INSERT INTO aaa (id, col) VALUES (...) UPDATE bbb SET col1 = ‘val1’ WHERE ... ...APPLY BATCH;
©2012 DataStax
Batch Mutate• Use non atomic batch if you need
performance, not atomicity
• More on dev blog• http://www.datastax.com/dev/blog/atomic-batches-in-cassandra-1-2
22
BEGIN UNLOGGED BATCH ...APPLY BATCH;
©2012 DataStax
Querying Data
23
SELECT article_id, posted_at, authorFROM commentsWHERE article_id >= ‘...’ORDER BY posted_at DESCLIMIT 100;
©2012 DataStax
Querying Data
• TTL/WRITETIME• You can query TTL or write time of the column.
24
cqlsh:ks> SELECT WRITETIME(author) FROM comments;
writetime(author)------------------- 1354146105288000
©2012 DataStax
Collection support
• Collection• Set• Unordered, no duplicates
• List• Ordered, allow duplicates
• Map• Keys and associated values
25
©2012 DataStax
Collection support
• Collections are typed, but cannot be nested(no list<list<text>>)
• No secondary index on collections26
CREATE TABLE example ( id uuid PRIMARY KEY, tags set<text>, points list<int>, attributes map<text, text>);
©2012 DataStax
Collection support
27
INSERT INTO example (id, tags, points, attributes)VALUES ( ‘62c36092-82a1-3a00-93d1-46196ee77204’, {‘foo’, ‘bar’, ‘baz’}, // set [100, 20, 93], // list {‘abc’: ‘def’} // map);
©2012 DataStax
Collection support
28
UPDATE example SET tags = tags + {‘qux’} WHERE ...UPDATE example SET tags = tags - {‘foo’} WHERE ...
UPDATE example SET points = points + [20, 30] WHERE ...UPDATE example SET points = points - [100] WHERE ...
UPDATE example SET attributes[‘ghi’] = ‘jkl’ WHERE ...DELETE attributes[‘abc’] FROM example WHERE ...
• Set
• Map
• List
©2012 DataStax
Collection support
• You cannot retrieve item in collection individually
29
SELECT tags, points, attributes FROM example;
tags | points | attributes-----------------+---------------+-------------- {baz, foo, bar} | [100, 20, 93] | {abc: def}
©2012 DataStax
Collection support
• Each element in collection is internally stored as one Cassandra column
• More on dev blog• http://www.datastax.com/dev/blog/cql3_collections
30
©2012 DataStax31
Related topics
©2012 DataStax
Native Transport
• CQL3 still goes through Thrift’s execute_cql3_query API
• Native Transport support introduces Cassandra’s original binary protocol• Async IO, server event push, ...• http://www.datastax.com/dev/blog/binary-protocol
• Try DataStax Java native driver with C* 1.2 beta today!• https://github.com/datastax/java-driver
32
©2012 DataStax33
Question ?
Or contact me later if you have one [email protected] yukim (IRC, twitter) Now
Hiringtalented engineers from all
over the world!