SlideShare a Scribd company logo
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
N1QL New Features
New features in the upcoming release
October, 2020
Keshav Murthy | VP, Couchbase R&D
Confidential and Proprietary.
Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
Agenda
01/
02/
03/
04/
05/
06/
N1QL So far… (short version)
Collections for N1QL
Language Features
Optimizer Features
Indexing Features
07/
Search Features
Transactions
08/ Wrap
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 3
N1QL So Far...
1
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
Give developers and
enterprises an expressive,
powerful, and complete
language for querying,
transforming, and
manipulating JSON data.
N1QL
SELECT breweries.name AS brewery,
COUNT(*) AS bwcount
FROM beers INNER JOIN breweries
ON beer.brewery_id = breweries.id
GROUP BY breweries.name;
INSERT INTO beers VALUES("paleale:12",
{"name":"Irish pale ale", "":"", "abv": 0.05});
UPDATE beers SET abv = 0.08
WHERE name = "Irish pale ale";
DELETE FROM breweries WHERE country = "Denmark";
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
N1QL = Productivity
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
Data
Service
Data
Service
Query
Service
Index
Service
FullText
Search
Couchbase Cluster
SDK
(KV, FTS,
Query)
SDK
(KV, FTS,
Query)
Web
Console
Analytics
Service
Eventing
Service
N1QL
7
SQL Features
Triggers
Stored Procedures in PL-SQL,
T-SQL, languages
JSON, XML
SQL is English
for Relational
Database
SQL Invented by Don
Chamberlin &
Raymond Boyce at
IBM
N1QL, based on
SQL, is English for
JSON
N1QL was invented by
Gerald Sangudi at
Couchbase
SQL
Instance
Database
Schema
Table
Row
Column
N1QL
Cluster
Bucket
Scope
Collection
Document
Attribute
SQL
Input and Output: Set(s) of
Tuples
N1QL STMT
CREATE BUCKET
CREATE INDEX
None
SELECT
INSERT
UPDATE
DELETE
MERGE
Subqueries
JOIN
GROUP BY
ORDER BY
OFFSET, LIMIT
EXPLAIN
PREPARE
EXECUTE
GRANT ROLE
REVOKE ROLE
INFER
PREPARE
EXECUTE
ADVISE (6.6)
ADVISOR
UPDATE-STATISTICS
CREATE FUNCTION
BEGIN WORK
COMMIT WORK
SAVEPOINT
ROLLBACK
EXECUTE
FUNCTION
Set of
JSON
N1QL Model
Set of
JSON
SQL
Tooling
ODBC, JDBC, .NET
Hibernate
BI Tools
erwin
TOAD
SQL Indexes
Primary Key
Secondary Key
Composite
Range Partitioned
Expression
Functional
Spatial
Search
SQL Logic
3 valued logic
TRUE, FALSE, NULL/UNKNOWN
N1QL
Tooling
ODBC,JDBC, .NET
Couchbase SDKs
Spring
BI Tools
erwin
SQL
TRANSACTIONS
ACID
Multi-Statement
Multi-table
Mullti-Database
Savepoints
Commit
Rollback [savepoint]
2 phase locking
Multi Isolation Levels
Single node
N1QL
TRANSACTIONS
ACID
Multi-Statement
Mullti-collection
Multi-Bucket
Savepoints
Commit
Rollback [savepoint]
OCC (Optimistic)
READ COMMITTED
Distributed tx
SQL Datatypes
Numeric
Decimal
Character
Date Time
Timezone
BLOB
Spatial
JSON
N1QL
Datatype
Numeric
Boolean
Character
Array
Object
Null
JSON
Conversion Functions
SQL
Optimizer
Rule Based
Cost Based
Index Selection
Join reordering
Query Rewrites
NL, Hash, Merge join
N1QL
Optimizer
Rule Based
Cost Based
Index Selection
Join as specified
Query Rewrites(Ltd)
NL, Hash join
N1QL
Engine
(SCALE UP &
SCALE OUT)
SQL STMT
CREATE TABLE
CREATE INDEX
ALTER TABLE
SELECT
INSERT
UPDATE
DELETE
MERGE
Subqueries
JOIN
GROUP BY
ORDER BY
OFFSET, LIMIT
EXPLAIN PLAN
PREPARE
EXECUTE
GRANT
REVOKE
DESCRIBE
PREPARE
EXECUTE
TRUNCATE
N1QL
Input and Output:
Set(s) of JSON
Tuples
SQL Model
Set of
Tuples
SQL Engine
(SCALE UP)
N1QL Features
Eventing Service
UDFs (N1QL, Javascript)
Defacto JSON
N1QL Logic
4 valued logic
TRUE, FALSE,
NULL/UNKNOWN, MISSING
N1QL
Indexes
Primary
Secondary
Composite
Partial
Expression
Functional
Array Index
Search
Flex Index(6.6)
8
SQL Features
Triggers
Stored Procedures in PL-SQL,
T-SQL, languages
JSON, XML
SQL is English
for Relational
Database
SQL Invented by Don
Chamberlin &
Raymond Boyce at
IBM
N1QL, based on
SQL, is English for
JSON
N1QL was invented by
Gerald Sangudi at
Couchbase
SQL
Instance
Database
Schema
Table
Row
Column
N1QL
Cluster
Bucket
Scope
Collection
Document
Attribute
SQL
Input and Output: Set(s) of
Tuples
N1QL STMT
CREATE BUCKET
CREATE INDEX
None
SELECT
INSERT
UPDATE
DELETE
MERGE
Subqueries
JOIN
GROUP BY
ORDER BY
OFFSET, LIMIT
EXPLAIN
PREPARE
EXECUTE
GRANT ROLE
REVOKE ROLE
INFER
PREPARE
EXECUTE
ADVISE (6.6)
ADVISOR
UPDATE-STATISTIC
S
CREATE FUNCTION
BEGIN WORK
COMMIT WORK
SAVEPOINT
ROLLBACK
EXECUTE FUNCTION
Set of
JSON
N1QL Model
Set of
JSON
SQL Tooling
ODBC, JDBC, .NET
Hibernate
++
SQL Indexes
Primary Key
Secondary Key
Composite
Range Partitioned
Expression
Functional
Spatial
Search
SQL Logic
3 valued logic
TRUE, FALSE, NULL/UNKNOWN
N1QL Tooling
ODBC,JDBC, .NET
Couchbase SDKs
Spring
SQL
TRANSACTIONS
ACID
Multi-Statement
Multi-table
Mullti-Database
Savepoints
Commit
Rollback [savepoint]
2 phase locking
Multi Isolation Levels
Single node
N1QL
TRANSACTIONS
ACID
Multi-Statement
Mullti-collection
Multi-Bucket
Savepoints
Commit
Rollback [savepoint]
OCC (Optimistic)
READ COMMITTED
Distributed tx
SQL Datatypes
Numeric
Decimal
Character
Date Time
Timezone
BLOB
Spatial
JSON
N1QL
Datatype
Numeric
Boolean
Character
Array
Object
Null
JSON
Conversion Functions
SQL
Optimizer
Rule Based
Cost Based
Index Selection
Join reordering
Query Rewrites
NL, Hash, Merge join
N1QL
Optimizer
Rule Based
Cost Based
Index Selection
Join as specified
Query Rewrites(Ltd)
NL, Hash join
N1QL
Engine
(SCALE UP &
SCALE OUT)
SQL STMT
CREATE TABLE
CREATE INDEX
ALTER TABLE
SELECT
INSERT
UPDATE
DELETE
MERGE
Subqueries
JOIN
GROUP BY
ORDER BY
OFFSET, LIMIT
EXPLAIN PLAN
PREPARE
EXECUTE
GRANT
REVOKE
DESCRIBE
PREPARE
EXECUTE
TRUNCATE
N1QL
Input and Output:
Set(s) of JSON
Tuples
SQL Model
Set of
Tuples
SQL Engine
(SCALE UP)
N1QL Features
Eventing Service
UDFs (N1QL, Javascript)
Defacto JSON
N1QL Logic
4 valued logic
TRUE, FALSE,
NULL/UNKNOWN, MISSING
N1QL
Indexes
Primary
Secondary
Composite
Partial
Expression
Functional
Array Index
Search
Flex Index(6.6)
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
N1QL - New Features in the next release
● INDEXES
○ Indexes per collections
○ All the existing Index features on collections
○ Supports up to 10,000 GSI indexes in a cluster
○ Improved performance for indexes on collections
● FULL TEXT SEARCH
○ Search indexing & querying on collections
○ Index on a single collection or multiple collections
○ Flex index to power N1QL (6.6)
● TRANSACTIONS
○ Multi-document,scope,collection,bucket
○ Multi-Statement
○ BEGIN TRANSACTION
○ COMMIT
○ SAVEPOINT
○ ROLLBACK
● Data Model Abstractions
○ SCOPE
○ COLLECTIONS
● LANGUAGE
○ ADVISE, ADVISOR()
○ User Defined Functions
■ N1QL
■ Javascript
● OPTIMIZER
○ Cost Based Optimizer
○ UPDATE STATISTICS
○ Index Advisor
■ ADVISE
■ ADVISOR()
■ Enhanced for Cost Based Optimizer
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 10
Collections for N1QL
2
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
N1QL: Data Logical Containment
UDF foo()
UDF foo2()
_default
Couchbase Cluster (e.g.
Bucket (cxprof)
Scope (usa)
Collection
(profille)
Collection
(loginfo)
● Bucket contains one or more logical scopes
● Each scope contains collections & UDFs.
● Cluster->Bucket->Scope->Collection
● Cluster->Bucket->Scope->UDFs
● In N1QL, use the collection as a keyspace reference
● Use query_context to for shorter reference
● Fully Backward Compatible
○ All your bucket operations still work
○ Uses _default scope & collection
Scope (uk)
Scope (eu)
Scope
(_default)
SELECT l.profname AS name,
l.uspszip AS postalcode
FROM cxprof.usa.loginfo l where l.type = "login";
CREATE INDEX i1 ON
cxprof.usa.loginfo(profname, uspszip)
PARTITION BY HASH(META().id)
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
N1QL
● Extend all N1QL DML SELECT, INSERT, UPDATE,
UPSERT, DELETE, MERGE to work with Couchbase
Collections
● New DDLs for creating and dropping
scopes, collections and UDFs
● Support fully qualified path
bucket.scope.collection or relative path
with query_context
● RBAC GRANT, REVOKE support at bucket,
scope and collection level
● Indexes can be created on specific
collections. Index names have to be unique
per collection
● Supports up to 1,000 collections and 10,000
indexes in a cluster
CREATE SCOPE cxprof.uk;
CREATE COLLECTION cxprof.uk.users;
CREATE COLLECTION cxprof.uk.royals;
CREATE INDEX i1 ON cxprof.uk.users(name, postal);
GRANT SELECT ON cx.uk.users TO manchester;
CREATE INDEX i1 ON cxprof.uk.royals(name,county);
GRANT SELECT ON cx.uk.royals TO london;
CREATE FUNCTION default:cx.uk.getusers(p)
{( SELECT name
FROM cx.uk.users
WHERE postal = p)}
SELECT u.name
FROM default:cxprof.uk.getusers("NW8 8Q") AS u;
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 13
Language Features
3
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
N1QL: UDFs in N1QL Language
● New Statements
○ CREATE FUNCTION
○ EXECUTE FUNCTION
○ DROP FUNCTION
● Use it in ANY DML query: SELECT, INSERT,
UPDATE, UPSERT, DELETE, MERGE
● Allows zero or more parameters and return values.
● Returns can be single value or an array of values.
● Each function is still executed on the same node as
the statement
● All the functions are to be created under a scope
○ UDFs and Collections are at the same level,
hierarchically speaking
● Unsupported for CREATE INDEX
○ CREATE INDEX i1 on f(bk.sc.foo(a))
CREATE FUNCTION default:cx.uk.getusers(p)
{( SELECT name
FROM cxprof.uk.users
WHERE postal = p)};
SELECT u.name
FROM default:cxprof.uk.getusers("NW8 8Q") AS u;
CREATE FUNCTION default:cxprof.uk.volume(l, w, b)
{(l * w * b)}
SELECT default:cxprof.uk.volume(9, 23, 43);
INSERT INTO cxprof.uk.users(KEY UUID(), VALUE v)
SELECT default:cxprof.uk.volume(48, 4, 23)}) AS v
SELECT * FROM cxprof.uk.shapes
WHERE default:cxprof.uk.volume(x, y, z) > 1000;
EXECUTE FUNCTION default:cxprof.uk.volume(3, 2,
4);
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
N1QL User Defined Functions using Javascript
● Add the javascript into the library
● Add the javascript function into the lilbray.
● CREATE FUNCTION in your scope
● This function can be used in any DML
● Larger functions can be registered from a file.
curl -v -X POST
http://localhost:8093/functions/v1/libraries/math/
functions/add -H 'content-type: application/json'
-d '{"name": "add", "code": "function add(a, b) {
let data = a + b; return data; }"}' -u
Administrator:password
CREATE FUNCTION default:cxprof.uk.jsadd(x, y)
LANGUAGE JAVASCRIPT AS "add" at "math";
EXECUTE FUNCTION default:cxprof.uk.jsadd(99,232);
/* Error : Mismatch number of parameters*/
EXECUTE FUNCTION default:cxprof.uk.jsadd(1, 2, 3);
[
{
"code": 10104,
"msg": "Incorrect number of arguments supplied
to function jsadd - cause: jsadd"
}
]
● Each query node comes with V8 javascript engine
● Each invocation of the javascript has about 8ms
overhead.
○ That’s each invocation overhead.
● Do something substantial in the UDF.
● The number of parameters during CREATE
FUNCTION and invocation should match
● Function name should be UNIQUE within the scope
● Create up to 500 functions in a cluster.
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
N1QL User Defined Functions using Javascript
● You can have the code in a file and register
● Everything else is the same.
● Globals aren’t supported
● The javascript functions are for compute
○ They can’t invoke N1QL statements from within
● They can invoke the usual Javascript library
File: add3.js
{"name": "add3", "code": "function add3(a, b, c) {
let res = a + b + c; return res; }"}
curl -v -X POST
http://localhost:8093/functions/v1/libraries/math/
functions/add3 -H 'content-type: application/json'
-d @./add3.js -u Administrator:password
CREATE FUNCTION default:cxprof.uk.jsadd3(p,q,r)
LANGUAGE JAVASCRIPT AS "add" at "math";
SELECT default:cxprof.uk.jsadd3(3, 4, 5)
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 17
Optimizer Features
4
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
N1QL: Cost Based Optimizer (CBO)
32 minutes
RULE BASED ROUTING COST BASED ROUTING
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
N1QL: Rule Based Optimizer (RBO)
N1QL Parser Semantic Analyzer
Rule Based
Optimizer
Query Execution
Tree
Executor
Query Results
Debug info (Query
Profile)
MetadataSystem metadata
Query Explain PlanN1QL Statement
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
N1QL: Cost Based Optimizer (CBO)
N1QL Parser Semantic Analyzer
Cost Based
Optimizer
Query Execution
Tree
Executor
Query Results
Debug info (Query
Profile)
STATISTICS &
metadataSystem metadata
Query Explain PlanN1QL Statement
P
atent
P
ending
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
N1QL: Cost Based Optimizer
● N1QL Optimizer Decisions
○ Query Rewrites
○ Index selection
■ Predicate pushdown
■ Pagination pushdowns (LIMIT, OFFSET)
○ Join type and sides selection
■ Nested loop (NL) or hash join (HJ)
■ NL: decide outer and inner sides
■ HJ: decide build and probe sides
● Create the plan and query execution tree.
● CBO: Decisions will be statistics
○ BETTER DECISIONS
● Collect statistics using UPDATE STATISTICS
● Best practices for collecting statistics are documented
● ADVISE recommends UPDATE STATISTICS as well
● When statistics is absent, rule based is used.
UPDATE STATISTICS FOR `travel-sample`
(type, destinationairport, city, faa, city, country)
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
N1QL: Cost Based Optimizer
● Use EXPLAIN as usual to see the CBO decisions
● EXPLAIN will give you the estimated cardinality & cost
● Profile output gives you actual cardinalities as well
● USE INDEX, USE HASH will continued to be
supported to override the default decisions
"#stats": {
"#itemsOut": 168168,
"#phaseSwitches": 672677,
"execTime": "526.643124ms",
"kernTime": "227.247789ms",
"servTime": "70.269443ms"
},
"as": "route",
"cardinality": 128068.90036900375,
"cost": 70181.75740221406,
"covers": [
"cover ((`route`.`sourceairport`))",
"cover
((`route`.`destinationairport`))",
"cover ((distinct (array (`v`.`day`)
for `v` in (`route`.`schedule`) end)))",
"cover ((meta(`route`).`id`))"
],
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
N1QL: Index Advisor
● ADVISE on a single statement
○ Preview in 6.5, GA in 6.6.
○ Enhance for CBO in the upcoming release
● ADVISOR() on a workload
○ Preview in 6.5, GA in the upcoming release
● ADVISE uses the rules for creating indexes and gives
non-covering and covering indexes.
○ For CBO, it also advises on statistics
● ADVISOR() can be invoked manually with set of
queries or as a result of queries from completed
requests
● ADVISOR Input
○ Workload
○ Result of a query
○ Static set of queries
○ It merges the index recommendations.
○ It is 1.0. Give us feedback.
ADVISE SELECT DISTINCT route.destinationairport
FROM `travel-sample` airport JOIN `travel-sample`
route
ON airport.faa = route.sourceairport
AND route.type = "route"
WHERE airport.type = "airport"
SELECT ADVISOR({"action": "start", "response": "0s",
"duration": "1h"}) AS Collect;
SELECT ADVISOR({"action": "get", "session":
"8c41a3c6-2252-437e-ab47-0b28f29f47fb"}) AS Get;
SELECT ADVISOR((
SELECT RAW statement
FROM system:completed_requests
WHERE LOWER(statement) LIKE "select%"
AND LOWER(statement) NOT LIKE "%advis%"))
INDEX-ADVISO
R.CO
UCHB
ASE.COM
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 24
Indexing Features
5
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
N1QL: Data Logical Containment
Couchbase Cluster
Bucket (cxprof)
Scope (usa)
Collection
(profille)
● Indexes per collection
○ Improved performance & resource usage
● Index names have to be unique PER collection
● All the index types are still supported
○ Partial, partitioned, expression, etc
○ Partial indexes on collections with multi doc types
● Supports large number of indexes
● Tools, query, explain are all enhanced to support it
SELECT profname AS name,
uspszip AS postalcode
FROM cxprof.usa.loginfo l where l.type = "login";
CREATE INDEX i1 ON
cxprof.usa.loginfo(profname, uspszip)
PARTITION BY HASH(META().id)
GSI Indexes
on profile
GSI Indexes
on loginfo
Collection
(loginfo)
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 26
Search Features
6
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
FTS: Searching on Collections
Couchbase Cluster
Bucket (cxprof)
Scope (usa)
Collection
(profille)
● FTS indexes on a single collection
○ Familiar indexing and queries
○ Support for SEARCH() from N1QL & Flex indexing
● Single FTS index on data residing in multiple collections
○ Direct query from FTS
○ Simpler search
FTS Index
on profile
Collection
(loginfo)
FTS index on
documents in
gameinfo & loginfo
Collection
(gameinfo)
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
● Improve query performance for ad hoc queries, and provide a predictable query performance in
applications where search patterns are not predetermined. This is available in Couchbase 6.6.
● Allow applications to leverage the unique capability of Keyword analyzer in search index to support
complex queries involving combinations of AND/OR and multiple array predicates.
● Can greatly reduce the number of indexes and resource needed to support complex queries.
SELECT * FROM crm a USE INDEX (USING FTS)
WHERE a.type='activity'
AND ( a.dept = 'iA88'
OR a.region > '59416' )
AND a.priority = 'High'
AND ( a.act_date BETWEEN
'2018-01-01' AND '2018-08-31'
OR a.event.location = 'Moscone Center' )
AND ( a.account.id = 'acc100'
OR a.owner.name = 'Amanda Morrison')
Flex Index allows query service to leverage search capabilities, using standard
N1QL
N1QL: FLEX INDEX
P
atent
P
ending
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 29
DISTRIBUTED N1QL TRANSACTIONS
7
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
Six thousand years ago,
the Sumerians invented writing for
TRANSACTION PROCESSING
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
● Multi-Everything: documents, statements,
collections, scope, buckets, APIs, tools
● Ensure data integrity for application
transaction that consists of multiple
operations where multiple concurrent
activities can occur
● Support all or nothing for multi documents
N1QL DML operations
● No central coordinator for scale out
● Optimistic Concurrency Control with RYOW
● Available in via SDKs, CBQ shell, REST API
● ACID details in the next slide.
N1QL Transactions ensures database consistency when data modifications
are performed with multiple documents in a single or multiple N1QL statements
BEGIN TRANSACTION;
UPDATE customer SET balance = balance + 100 WHERE cid =
4872;
UPDATE customer SET balance = balance - 100 WHERE cid =
1924;
SELECT cid, name, balance FROM customer
WHERE cid IN [4872, 1924];
COMMIT ;
N1QL: Transactions
Patent
Pending
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
ACID
● ATOMIC
○ ALL or Nothing for each statement and the whole
transaction
● CONSISTENT
○ Modifications are done on the qualified set
○ Only constraint is document key unique constraint
○ Handles halloween problem via index snapshots
● ISOLATED
○ READ COMMITTED isolation
○ Monotonic atomic reads
○ Scan consistency: Request_plus by default
■ Tunable with Unbounded
○ No dirty reads
● DURABLE
○ Majority - default
○ majorityAndPersistActive
○ persistToMajority
Couchbase Cluster
Query
Query
Query
Apps
GSI
GSI
FTS
Key
Valu
e
Apps
Apps
sdk
sdk
sdk
Query
CBQ Shell
Query
Workbench
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
ACIDic Statements
● BEGIN TRANSACTION;
○ request to any of the query nodes. It returns txid
○ Send subsequent tx statements to same node
○ Send each statements with txid
● All DMLs are allowed. non-DMLs raise error
○ Supported: SELECT, INSERT, UPDATE,
DELETE, UPSERT, MERGE
○ Failure within a statement simply fails and rolls
back the statement. Transaction continues
● SAVEPOINT savepointname
○ Sets and names a save point
● ROLLBACK TO SAVEPOINT savepointname
○ Rolls back the changes to a named savepoint.
● ROLLBACK TRANSACTION;
○ Rollback the entire transaction
● COMMIT
○ Commits all the modifications into Couchbase
○ The whole transaction is rolled back in case of
Couchbase Cluster
Query
Query
Query
Apps
GSI
GSI
FTS
Key
Valu
e
Apps
Apps
sdk
sdk
sdk
Query
CBQ Shell
Query
Workbench
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
ACID for single statement
/* Simply set tximplicit to True */
UPDATE customer
SET balance = balance + 100
WHERE dept = 100;
INSERT INTO customer(KEY cid, VALUE v)
SELECT meta().id AS cid, newcust AS v
FROM newcust;
● N1QL uses optimistic concurrency control
● Caches the version of the modific documents
○ Caching the changes until commit takes memory!
○ Suitable for large number of small transactions
● When you’re trying to modify large number of docs
○ Issue the statement like before
○ Set the tximplicit parameter to True.
● When tximplicit is set to True
○ N1QL starts a new transaction
○ Each write is staged in this transaction
○ If there’s any failure, the whole operation is rolled
back
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
ACID Support in SDK
try {
transactions.run((ctx) -> {
// starts the transaction
TransactionGetResult cxobj = ctx.get(customer, cxid);
ctx.replace(cxobj, newaddress);
ctx.query("UPDATE customer SET balance = balance + 100 WHERE cid = 4872");
ctx.query("UPDATE customer SET balance = balance - 100 WHERE cid = 1924;");
ctx.query("SELECT cid, name, balance FROM customer WHERE cid IN [4872, 1924]");
// This call is optional - if you leave it off, the transaction
// will be committed anyway.
ctx.commit();
});
} catch (TransactionFailed e) {
// Just log the error
logger.warn("Transaction did not reach commit:");
for (LogDefer log: e.result().log().logs()) {
logger.warn(log.toString());
}
}
● The Java SDK Lambda Example
● Supports all the transactional config
● Mix and match KV and query
● More SDKs on the roadmap
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
● Documents qualified are done by scanning the index using the scan consistency
● RYOW: Transaction sees it’s own changes in all subsequent queries by caching the changes
○ Limit the number of updates in a multi-statement transaction
○ Use singleton transaction when necessary
● Updates are all optimistic. Avoid the hot row problem with best practices in application design
● Don’t do non-transactional writes on the documents participating in transactions
● Multi-statement transactions are lock free and coordination free. Designed for scale out
N1QL: Best Practice
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 37
WRAP
8
38
SQL Features
Triggers
Stored Procedures in PL-SQL,
T-SQL, languages
JSON, XML
SQL is English
for Relational
Database
SQL Invented by Don
Chamberlin &
Raymond Boyce at
IBM
N1QL, based on
SQL, is English for
JSON
N1QL was invented by
Gerald Sangudi at
Couchbase
SQL
Instance
Database
Schema
Table
Row
Column
N1QL
Cluster
Bucket
Scope
Collection
Document
Attribute
SQL
Input and Output: Set(s) of
Tuples
N1QL STMT
CREATE BUCKET
CREATE INDEX
None
SELECT
INSERT
UPDATE
DELETE
MERGE
Subqueries
JOIN
GROUP BY
ORDER BY
OFFSET, LIMIT
EXPLAIN
PREPARE
EXECUTE
GRANT ROLE
REVOKE ROLE
INFER
PREPARE
EXECUTE
ADVISE (6.6)
ADVISOR
UPDATE-STATISTICS
CREATE FUNCTION
BEGIN WORK
COMMIT WORK
SAVEPOINT
ROLLBACK
EXECUTE
FUNCTION
Set of
JSON
N1QL Model
Set of
JSON
SQL
Tooling
ODBC, JDBC, .NET
Hibernate
BI Tools
erwin
TOAD
SQL Indexes
Primary Key
Secondary Key
Composite
Range Partitioned
Expression
Functional
Spatial
Search
SQL Logic
3 valued logic
TRUE, FALSE, NULL/UNKNOWN
N1QL
Tooling
ODBC,JDBC, .NET
Couchbase SDKs
Spring
BI Tools
erwin
SQL
TRANSACTIONS
ACID
Multi-Statement
Multi-table
Mullti-Database
Savepoints
Commit
Rollback [savepoint]
2 phase locking
Multi Isolation Levels
Single node
N1QL
TRANSACTIONS
ACID
Multi-Statement
Mullti-collection
Multi-Bucket
Savepoints
Commit
Rollback [savepoint]
OCC (Optimistic)
READ COMMITTED
Distributed tx
SQL Datatypes
Numeric
Decimal
Character
Date Time
Timezone
BLOB
Spatial
JSON
N1QL
Datatype
Numeric
Boolean
Character
Array
Object
Null
JSON
Conversion Functions
SQL
Optimizer
Rule Based
Cost Based
Index Selection
Join reordering
Query Rewrites
NL, Hash, Merge join
N1QL
Optimizer
Rule Based
Cost Based
Index Selection
Join as specified
Query Rewrites(Ltd)
NL, Hash join
N1QL
Engine
(SCALE UP &
SCALE OUT)
SQL STMT
CREATE TABLE
CREATE INDEX
ALTER TABLE
SELECT
INSERT
UPDATE
DELETE
MERGE
Subqueries
JOIN
GROUP BY
ORDER BY
OFFSET, LIMIT
EXPLAIN PLAN
PREPARE
EXECUTE
GRANT
REVOKE
DESCRIBE
PREPARE
EXECUTE
TRUNCATE
N1QL
Input and Output:
Set(s) of JSON
Tuples
SQL Model
Set of
Tuples
SQL Engine
(SCALE UP)
N1QL Features
Eventing Service
UDFs (N1QL, Javascript)
Defacto JSON
N1QL Logic
4 valued logic
TRUE, FALSE,
NULL/UNKNOWN, MISSING
N1QL
Indexes
Primary
Secondary
Composite
Partial
Expression
Functional
Array Index
Search
Flex Index(6.6)
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
THANK YOU

More Related Content

PPTX
Couchbase Tutorial: Big data Open Source Systems: VLDB2018
Keshav Murthy
 
PPTX
Couchbase N1QL: Language & Architecture Overview.
Keshav Murthy
 
PPTX
N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5
Keshav Murthy
 
PDF
Data Processing and Aggregation with MongoDB
MongoDB
 
PDF
All you need to know about CREATE STATISTICS
EDB
 
PPTX
OrientDB vs Neo4j - Comparison of query/speed/functionality
Curtis Mosters
 
PPSX
JSON in 18c and 19c
stewashton
 
PPTX
MongoDB World 2016 : Advanced Aggregation
Joe Drumgoole
 
Couchbase Tutorial: Big data Open Source Systems: VLDB2018
Keshav Murthy
 
Couchbase N1QL: Language & Architecture Overview.
Keshav Murthy
 
N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5
Keshav Murthy
 
Data Processing and Aggregation with MongoDB
MongoDB
 
All you need to know about CREATE STATISTICS
EDB
 
OrientDB vs Neo4j - Comparison of query/speed/functionality
Curtis Mosters
 
JSON in 18c and 19c
stewashton
 
MongoDB World 2016 : Advanced Aggregation
Joe Drumgoole
 

What's hot (20)

PPTX
From SQL to NoSQL: Structured Querying for JSON
Keshav Murthy
 
PDF
SQL on everything, in memory
Julian Hyde
 
PDF
Trivadis TechEvent 2016 Polybase challenges Hive relational access to non-rel...
Trivadis
 
PDF
Webscale PostgreSQL - JSONB and Horizontal Scaling Strategies
Jonathan Katz
 
PDF
Trivadis TechEvent 2016 Big Data Cassandra, wieso brauche ich das? by Jan Ott
Trivadis
 
PDF
What and Why and How: Apache Drill ! - Tugdual Grall
distributed matters
 
PPT
The NoSQL Way in Postgres
EDB
 
PDF
NoSQL meets Microservices - Michael Hackstein
distributed matters
 
PPTX
Using Spark to Load Oracle Data into Cassandra
Jim Hatcher
 
PDF
MongoDB Aggregation Framework
Caserta
 
PDF
MongoDB .local Paris 2020: Adéo @MongoDB : MongoDB Atlas & Leroy Merlin : et ...
MongoDB
 
PDF
Webinar: Data Processing and Aggregation Options
MongoDB
 
PDF
Data day texas: Cassandra and the Cloud
jbellis
 
PPTX
OrientDB vs Neo4j - and an introduction to NoSQL databases
Curtis Mosters
 
PDF
Bucketing 2.0: Improve Spark SQL Performance by Removing Shuffle
Databricks
 
PPTX
Cost-based query optimization in Apache Hive
Julian Hyde
 
PPTX
MongoDB - Aggregation Pipeline
Jason Terpko
 
PDF
Full metal mongo
Israel Gutiérrez
 
KEY
MongoDB Aggregation Framework
Tyler Brock
 
PPTX
Discover the Power of the NoSQL + SQL with MySQL
Dave Stokes
 
From SQL to NoSQL: Structured Querying for JSON
Keshav Murthy
 
SQL on everything, in memory
Julian Hyde
 
Trivadis TechEvent 2016 Polybase challenges Hive relational access to non-rel...
Trivadis
 
Webscale PostgreSQL - JSONB and Horizontal Scaling Strategies
Jonathan Katz
 
Trivadis TechEvent 2016 Big Data Cassandra, wieso brauche ich das? by Jan Ott
Trivadis
 
What and Why and How: Apache Drill ! - Tugdual Grall
distributed matters
 
The NoSQL Way in Postgres
EDB
 
NoSQL meets Microservices - Michael Hackstein
distributed matters
 
Using Spark to Load Oracle Data into Cassandra
Jim Hatcher
 
MongoDB Aggregation Framework
Caserta
 
MongoDB .local Paris 2020: Adéo @MongoDB : MongoDB Atlas & Leroy Merlin : et ...
MongoDB
 
Webinar: Data Processing and Aggregation Options
MongoDB
 
Data day texas: Cassandra and the Cloud
jbellis
 
OrientDB vs Neo4j - and an introduction to NoSQL databases
Curtis Mosters
 
Bucketing 2.0: Improve Spark SQL Performance by Removing Shuffle
Databricks
 
Cost-based query optimization in Apache Hive
Julian Hyde
 
MongoDB - Aggregation Pipeline
Jason Terpko
 
Full metal mongo
Israel Gutiérrez
 
MongoDB Aggregation Framework
Tyler Brock
 
Discover the Power of the NoSQL + SQL with MySQL
Dave Stokes
 
Ad

Similar to N1QL New Features in couchbase 7.0 (20)

PDF
Virtual training intro to InfluxDB - June 2021
InfluxData
 
PPTX
2018: State of the Dolphin, MySQL Keynote at Percona Live Europe 2018, Frankf...
Geir Høydalsvik
 
PPTX
Cb15 presentation-yingyi
Yingyi Bu
 
PDF
Restlet: Building a multi-tenant API PaaS with DataStax Enterprise Search
DataStax Academy
 
PDF
ClickHouse new features and development roadmap, by Aleksei Milovidov
Altinity Ltd
 
PDF
Node.js and the MySQL Document Store
Rui Quelhas
 
ODP
Couchbase training advanced
Knoldus Inc.
 
PDF
MySQL 8.0 Optimizer Guide
Morgan Tocker
 
PPTX
Enterprise Architect's view of Couchbase 4.0 with N1QL
Keshav Murthy
 
PDF
Fast federated SQL with Apache Calcite
Chris Baynes
 
PDF
Cassandra Summit 2015 - Building a multi-tenant API PaaS with DataStax Enterp...
Restlet
 
PDF
MySQL New Features -- Sunshine PHP 2020 Presentation
Dave Stokes
 
ODP
Lokijs
Joe Minichino
 
PDF
Openshift service broker and catalog ocp-meetup july 2018
Michael Calizo
 
PPTX
Couchbase Data Platform | Big Data Demystified
Omid Vahdaty
 
PDF
SQL for Analytics.pdfSQL for Analytics.pdf
namtunguyen6
 
PPTX
The openCypher Project - An Open Graph Query Language
Neo4j
 
PPTX
Azure Data Lake and U-SQL
Michael Rys
 
PPT
What's New for Developers in SQL Server 2008?
ukdpe
 
PPT
SQL Server 2008 Overview
Eric Nelson
 
Virtual training intro to InfluxDB - June 2021
InfluxData
 
2018: State of the Dolphin, MySQL Keynote at Percona Live Europe 2018, Frankf...
Geir Høydalsvik
 
Cb15 presentation-yingyi
Yingyi Bu
 
Restlet: Building a multi-tenant API PaaS with DataStax Enterprise Search
DataStax Academy
 
ClickHouse new features and development roadmap, by Aleksei Milovidov
Altinity Ltd
 
Node.js and the MySQL Document Store
Rui Quelhas
 
Couchbase training advanced
Knoldus Inc.
 
MySQL 8.0 Optimizer Guide
Morgan Tocker
 
Enterprise Architect's view of Couchbase 4.0 with N1QL
Keshav Murthy
 
Fast federated SQL with Apache Calcite
Chris Baynes
 
Cassandra Summit 2015 - Building a multi-tenant API PaaS with DataStax Enterp...
Restlet
 
MySQL New Features -- Sunshine PHP 2020 Presentation
Dave Stokes
 
Openshift service broker and catalog ocp-meetup july 2018
Michael Calizo
 
Couchbase Data Platform | Big Data Demystified
Omid Vahdaty
 
SQL for Analytics.pdfSQL for Analytics.pdf
namtunguyen6
 
The openCypher Project - An Open Graph Query Language
Neo4j
 
Azure Data Lake and U-SQL
Michael Rys
 
What's New for Developers in SQL Server 2008?
ukdpe
 
SQL Server 2008 Overview
Eric Nelson
 
Ad

More from Keshav Murthy (20)

PPTX
XLDB Lightning Talk: Databases for an Engaged World: Requirements and Design...
Keshav Murthy
 
PDF
Couchbase 5.5: N1QL and Indexing features
Keshav Murthy
 
PPTX
N1QL: Query Optimizer Improvements in Couchbase 5.0. By, Sitaram Vemulapalli
Keshav Murthy
 
PPTX
Couchbase Query Workbench Enhancements By Eben Haber
Keshav Murthy
 
PPTX
Mindmap: Oracle to Couchbase for developers
Keshav Murthy
 
PPTX
Couchbase N1QL: Index Advisor
Keshav Murthy
 
PPTX
N1QL: What's new in Couchbase 5.0
Keshav Murthy
 
PPTX
Tuning for Performance: indexes & Queries
Keshav Murthy
 
PPTX
Understanding N1QL Optimizer to Tune Queries
Keshav Murthy
 
PPTX
Utilizing Arrays: Modeling, Querying and Indexing
Keshav Murthy
 
PPTX
Extended JOIN in Couchbase Server 4.5
Keshav Murthy
 
PPTX
Bringing SQL to NoSQL: Rich, Declarative Query for NoSQL
Keshav Murthy
 
PPTX
Query in Couchbase. N1QL: SQL for JSON
Keshav Murthy
 
PPTX
SQL for JSON: Rich, Declarative Querying for NoSQL Databases and Applications 
Keshav Murthy
 
PPTX
Introducing N1QL: New SQL Based Query Language for JSON
Keshav Murthy
 
PPTX
Deep dive into N1QL: SQL for JSON: Internals and power features.
Keshav Murthy
 
PPTX
N1QL workshop: Indexing & Query turning.
Keshav Murthy
 
PPTX
Drilling on JSON
Keshav Murthy
 
PPTX
Accelerating analytics on the Sensor and IoT Data.
Keshav Murthy
 
PPTX
You know what iMEAN? Using MEAN stack for application dev on Informix
Keshav Murthy
 
XLDB Lightning Talk: Databases for an Engaged World: Requirements and Design...
Keshav Murthy
 
Couchbase 5.5: N1QL and Indexing features
Keshav Murthy
 
N1QL: Query Optimizer Improvements in Couchbase 5.0. By, Sitaram Vemulapalli
Keshav Murthy
 
Couchbase Query Workbench Enhancements By Eben Haber
Keshav Murthy
 
Mindmap: Oracle to Couchbase for developers
Keshav Murthy
 
Couchbase N1QL: Index Advisor
Keshav Murthy
 
N1QL: What's new in Couchbase 5.0
Keshav Murthy
 
Tuning for Performance: indexes & Queries
Keshav Murthy
 
Understanding N1QL Optimizer to Tune Queries
Keshav Murthy
 
Utilizing Arrays: Modeling, Querying and Indexing
Keshav Murthy
 
Extended JOIN in Couchbase Server 4.5
Keshav Murthy
 
Bringing SQL to NoSQL: Rich, Declarative Query for NoSQL
Keshav Murthy
 
Query in Couchbase. N1QL: SQL for JSON
Keshav Murthy
 
SQL for JSON: Rich, Declarative Querying for NoSQL Databases and Applications 
Keshav Murthy
 
Introducing N1QL: New SQL Based Query Language for JSON
Keshav Murthy
 
Deep dive into N1QL: SQL for JSON: Internals and power features.
Keshav Murthy
 
N1QL workshop: Indexing & Query turning.
Keshav Murthy
 
Drilling on JSON
Keshav Murthy
 
Accelerating analytics on the Sensor and IoT Data.
Keshav Murthy
 
You know what iMEAN? Using MEAN stack for application dev on Informix
Keshav Murthy
 

Recently uploaded (20)

PDF
Enhancing Healthcare RPM Platforms with Contextual AI Integration
Cadabra Studio
 
PPTX
Role Of Python In Programing Language.pptx
jaykoshti048
 
PDF
advancepresentationskillshdhdhhdhdhdhhfhf
jasmenrojas249
 
PPTX
TRAVEL APIs | WHITE LABEL TRAVEL API | TOP TRAVEL APIs
philipnathen82
 
PDF
WatchTraderHub - Watch Dealer software with inventory management and multi-ch...
WatchDealer Pavel
 
PDF
Bandai Playdia The Book - David Glotz
BluePanther6
 
PDF
Adobe Illustrator Crack Full Download (Latest Version 2025) Pre-Activated
imang66g
 
PDF
Generating Union types w/ Static Analysis
K. Matthew Dupree
 
PDF
Summary Of Odoo 18.1 to 18.4 : The Way For Odoo 19
CandidRoot Solutions Private Limited
 
PDF
Salesforce Implementation Services Provider.pdf
VALiNTRY360
 
PDF
MiniTool Power Data Recovery Crack New Pre Activated Version Latest 2025
imang66g
 
PPTX
The-Dawn-of-AI-Reshaping-Our-World.pptxx
parthbhanushali307
 
PPTX
AI-Ready Handoff: Auto-Summaries & Draft Emails from MQL to Slack in One Flow
bbedford2
 
PDF
New Download MiniTool Partition Wizard Crack Latest Version 2025
imang66g
 
PPTX
ConcordeApp: Engineering Global Impact & Unlocking Billions in Event ROI with AI
chastechaste14
 
PDF
49785682629390197565_LRN3014_Migrating_the_Beast.pdf
Abilash868456
 
PPTX
Odoo Integration Services by Candidroot Solutions
CandidRoot Solutions Private Limited
 
PPTX
Web Testing.pptx528278vshbuqffqhhqiwnwuq
studylike474
 
PDF
lesson-2-rules-of-netiquette.pdf.bshhsjdj
jasmenrojas249
 
PPTX
Can You Build Dashboards Using Open Source Visualization Tool.pptx
Varsha Nayak
 
Enhancing Healthcare RPM Platforms with Contextual AI Integration
Cadabra Studio
 
Role Of Python In Programing Language.pptx
jaykoshti048
 
advancepresentationskillshdhdhhdhdhdhhfhf
jasmenrojas249
 
TRAVEL APIs | WHITE LABEL TRAVEL API | TOP TRAVEL APIs
philipnathen82
 
WatchTraderHub - Watch Dealer software with inventory management and multi-ch...
WatchDealer Pavel
 
Bandai Playdia The Book - David Glotz
BluePanther6
 
Adobe Illustrator Crack Full Download (Latest Version 2025) Pre-Activated
imang66g
 
Generating Union types w/ Static Analysis
K. Matthew Dupree
 
Summary Of Odoo 18.1 to 18.4 : The Way For Odoo 19
CandidRoot Solutions Private Limited
 
Salesforce Implementation Services Provider.pdf
VALiNTRY360
 
MiniTool Power Data Recovery Crack New Pre Activated Version Latest 2025
imang66g
 
The-Dawn-of-AI-Reshaping-Our-World.pptxx
parthbhanushali307
 
AI-Ready Handoff: Auto-Summaries & Draft Emails from MQL to Slack in One Flow
bbedford2
 
New Download MiniTool Partition Wizard Crack Latest Version 2025
imang66g
 
ConcordeApp: Engineering Global Impact & Unlocking Billions in Event ROI with AI
chastechaste14
 
49785682629390197565_LRN3014_Migrating_the_Beast.pdf
Abilash868456
 
Odoo Integration Services by Candidroot Solutions
CandidRoot Solutions Private Limited
 
Web Testing.pptx528278vshbuqffqhhqiwnwuq
studylike474
 
lesson-2-rules-of-netiquette.pdf.bshhsjdj
jasmenrojas249
 
Can You Build Dashboards Using Open Source Visualization Tool.pptx
Varsha Nayak
 

N1QL New Features in couchbase 7.0

  • 1. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. N1QL New Features New features in the upcoming release October, 2020 Keshav Murthy | VP, Couchbase R&D
  • 2. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. Agenda 01/ 02/ 03/ 04/ 05/ 06/ N1QL So far… (short version) Collections for N1QL Language Features Optimizer Features Indexing Features 07/ Search Features Transactions 08/ Wrap
  • 3. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 3 N1QL So Far... 1
  • 4. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. Give developers and enterprises an expressive, powerful, and complete language for querying, transforming, and manipulating JSON data. N1QL SELECT breweries.name AS brewery, COUNT(*) AS bwcount FROM beers INNER JOIN breweries ON beer.brewery_id = breweries.id GROUP BY breweries.name; INSERT INTO beers VALUES("paleale:12", {"name":"Irish pale ale", "":"", "abv": 0.05}); UPDATE beers SET abv = 0.08 WHERE name = "Irish pale ale"; DELETE FROM breweries WHERE country = "Denmark";
  • 5. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. N1QL = Productivity
  • 6. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. Data Service Data Service Query Service Index Service FullText Search Couchbase Cluster SDK (KV, FTS, Query) SDK (KV, FTS, Query) Web Console Analytics Service Eventing Service N1QL
  • 7. 7 SQL Features Triggers Stored Procedures in PL-SQL, T-SQL, languages JSON, XML SQL is English for Relational Database SQL Invented by Don Chamberlin & Raymond Boyce at IBM N1QL, based on SQL, is English for JSON N1QL was invented by Gerald Sangudi at Couchbase SQL Instance Database Schema Table Row Column N1QL Cluster Bucket Scope Collection Document Attribute SQL Input and Output: Set(s) of Tuples N1QL STMT CREATE BUCKET CREATE INDEX None SELECT INSERT UPDATE DELETE MERGE Subqueries JOIN GROUP BY ORDER BY OFFSET, LIMIT EXPLAIN PREPARE EXECUTE GRANT ROLE REVOKE ROLE INFER PREPARE EXECUTE ADVISE (6.6) ADVISOR UPDATE-STATISTICS CREATE FUNCTION BEGIN WORK COMMIT WORK SAVEPOINT ROLLBACK EXECUTE FUNCTION Set of JSON N1QL Model Set of JSON SQL Tooling ODBC, JDBC, .NET Hibernate BI Tools erwin TOAD SQL Indexes Primary Key Secondary Key Composite Range Partitioned Expression Functional Spatial Search SQL Logic 3 valued logic TRUE, FALSE, NULL/UNKNOWN N1QL Tooling ODBC,JDBC, .NET Couchbase SDKs Spring BI Tools erwin SQL TRANSACTIONS ACID Multi-Statement Multi-table Mullti-Database Savepoints Commit Rollback [savepoint] 2 phase locking Multi Isolation Levels Single node N1QL TRANSACTIONS ACID Multi-Statement Mullti-collection Multi-Bucket Savepoints Commit Rollback [savepoint] OCC (Optimistic) READ COMMITTED Distributed tx SQL Datatypes Numeric Decimal Character Date Time Timezone BLOB Spatial JSON N1QL Datatype Numeric Boolean Character Array Object Null JSON Conversion Functions SQL Optimizer Rule Based Cost Based Index Selection Join reordering Query Rewrites NL, Hash, Merge join N1QL Optimizer Rule Based Cost Based Index Selection Join as specified Query Rewrites(Ltd) NL, Hash join N1QL Engine (SCALE UP & SCALE OUT) SQL STMT CREATE TABLE CREATE INDEX ALTER TABLE SELECT INSERT UPDATE DELETE MERGE Subqueries JOIN GROUP BY ORDER BY OFFSET, LIMIT EXPLAIN PLAN PREPARE EXECUTE GRANT REVOKE DESCRIBE PREPARE EXECUTE TRUNCATE N1QL Input and Output: Set(s) of JSON Tuples SQL Model Set of Tuples SQL Engine (SCALE UP) N1QL Features Eventing Service UDFs (N1QL, Javascript) Defacto JSON N1QL Logic 4 valued logic TRUE, FALSE, NULL/UNKNOWN, MISSING N1QL Indexes Primary Secondary Composite Partial Expression Functional Array Index Search Flex Index(6.6)
  • 8. 8 SQL Features Triggers Stored Procedures in PL-SQL, T-SQL, languages JSON, XML SQL is English for Relational Database SQL Invented by Don Chamberlin & Raymond Boyce at IBM N1QL, based on SQL, is English for JSON N1QL was invented by Gerald Sangudi at Couchbase SQL Instance Database Schema Table Row Column N1QL Cluster Bucket Scope Collection Document Attribute SQL Input and Output: Set(s) of Tuples N1QL STMT CREATE BUCKET CREATE INDEX None SELECT INSERT UPDATE DELETE MERGE Subqueries JOIN GROUP BY ORDER BY OFFSET, LIMIT EXPLAIN PREPARE EXECUTE GRANT ROLE REVOKE ROLE INFER PREPARE EXECUTE ADVISE (6.6) ADVISOR UPDATE-STATISTIC S CREATE FUNCTION BEGIN WORK COMMIT WORK SAVEPOINT ROLLBACK EXECUTE FUNCTION Set of JSON N1QL Model Set of JSON SQL Tooling ODBC, JDBC, .NET Hibernate ++ SQL Indexes Primary Key Secondary Key Composite Range Partitioned Expression Functional Spatial Search SQL Logic 3 valued logic TRUE, FALSE, NULL/UNKNOWN N1QL Tooling ODBC,JDBC, .NET Couchbase SDKs Spring SQL TRANSACTIONS ACID Multi-Statement Multi-table Mullti-Database Savepoints Commit Rollback [savepoint] 2 phase locking Multi Isolation Levels Single node N1QL TRANSACTIONS ACID Multi-Statement Mullti-collection Multi-Bucket Savepoints Commit Rollback [savepoint] OCC (Optimistic) READ COMMITTED Distributed tx SQL Datatypes Numeric Decimal Character Date Time Timezone BLOB Spatial JSON N1QL Datatype Numeric Boolean Character Array Object Null JSON Conversion Functions SQL Optimizer Rule Based Cost Based Index Selection Join reordering Query Rewrites NL, Hash, Merge join N1QL Optimizer Rule Based Cost Based Index Selection Join as specified Query Rewrites(Ltd) NL, Hash join N1QL Engine (SCALE UP & SCALE OUT) SQL STMT CREATE TABLE CREATE INDEX ALTER TABLE SELECT INSERT UPDATE DELETE MERGE Subqueries JOIN GROUP BY ORDER BY OFFSET, LIMIT EXPLAIN PLAN PREPARE EXECUTE GRANT REVOKE DESCRIBE PREPARE EXECUTE TRUNCATE N1QL Input and Output: Set(s) of JSON Tuples SQL Model Set of Tuples SQL Engine (SCALE UP) N1QL Features Eventing Service UDFs (N1QL, Javascript) Defacto JSON N1QL Logic 4 valued logic TRUE, FALSE, NULL/UNKNOWN, MISSING N1QL Indexes Primary Secondary Composite Partial Expression Functional Array Index Search Flex Index(6.6)
  • 9. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. N1QL - New Features in the next release ● INDEXES ○ Indexes per collections ○ All the existing Index features on collections ○ Supports up to 10,000 GSI indexes in a cluster ○ Improved performance for indexes on collections ● FULL TEXT SEARCH ○ Search indexing & querying on collections ○ Index on a single collection or multiple collections ○ Flex index to power N1QL (6.6) ● TRANSACTIONS ○ Multi-document,scope,collection,bucket ○ Multi-Statement ○ BEGIN TRANSACTION ○ COMMIT ○ SAVEPOINT ○ ROLLBACK ● Data Model Abstractions ○ SCOPE ○ COLLECTIONS ● LANGUAGE ○ ADVISE, ADVISOR() ○ User Defined Functions ■ N1QL ■ Javascript ● OPTIMIZER ○ Cost Based Optimizer ○ UPDATE STATISTICS ○ Index Advisor ■ ADVISE ■ ADVISOR() ■ Enhanced for Cost Based Optimizer
  • 10. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 10 Collections for N1QL 2
  • 11. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. N1QL: Data Logical Containment UDF foo() UDF foo2() _default Couchbase Cluster (e.g. Bucket (cxprof) Scope (usa) Collection (profille) Collection (loginfo) ● Bucket contains one or more logical scopes ● Each scope contains collections & UDFs. ● Cluster->Bucket->Scope->Collection ● Cluster->Bucket->Scope->UDFs ● In N1QL, use the collection as a keyspace reference ● Use query_context to for shorter reference ● Fully Backward Compatible ○ All your bucket operations still work ○ Uses _default scope & collection Scope (uk) Scope (eu) Scope (_default) SELECT l.profname AS name, l.uspszip AS postalcode FROM cxprof.usa.loginfo l where l.type = "login"; CREATE INDEX i1 ON cxprof.usa.loginfo(profname, uspszip) PARTITION BY HASH(META().id)
  • 12. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. N1QL ● Extend all N1QL DML SELECT, INSERT, UPDATE, UPSERT, DELETE, MERGE to work with Couchbase Collections ● New DDLs for creating and dropping scopes, collections and UDFs ● Support fully qualified path bucket.scope.collection or relative path with query_context ● RBAC GRANT, REVOKE support at bucket, scope and collection level ● Indexes can be created on specific collections. Index names have to be unique per collection ● Supports up to 1,000 collections and 10,000 indexes in a cluster CREATE SCOPE cxprof.uk; CREATE COLLECTION cxprof.uk.users; CREATE COLLECTION cxprof.uk.royals; CREATE INDEX i1 ON cxprof.uk.users(name, postal); GRANT SELECT ON cx.uk.users TO manchester; CREATE INDEX i1 ON cxprof.uk.royals(name,county); GRANT SELECT ON cx.uk.royals TO london; CREATE FUNCTION default:cx.uk.getusers(p) {( SELECT name FROM cx.uk.users WHERE postal = p)} SELECT u.name FROM default:cxprof.uk.getusers("NW8 8Q") AS u;
  • 13. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 13 Language Features 3
  • 14. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. N1QL: UDFs in N1QL Language ● New Statements ○ CREATE FUNCTION ○ EXECUTE FUNCTION ○ DROP FUNCTION ● Use it in ANY DML query: SELECT, INSERT, UPDATE, UPSERT, DELETE, MERGE ● Allows zero or more parameters and return values. ● Returns can be single value or an array of values. ● Each function is still executed on the same node as the statement ● All the functions are to be created under a scope ○ UDFs and Collections are at the same level, hierarchically speaking ● Unsupported for CREATE INDEX ○ CREATE INDEX i1 on f(bk.sc.foo(a)) CREATE FUNCTION default:cx.uk.getusers(p) {( SELECT name FROM cxprof.uk.users WHERE postal = p)}; SELECT u.name FROM default:cxprof.uk.getusers("NW8 8Q") AS u; CREATE FUNCTION default:cxprof.uk.volume(l, w, b) {(l * w * b)} SELECT default:cxprof.uk.volume(9, 23, 43); INSERT INTO cxprof.uk.users(KEY UUID(), VALUE v) SELECT default:cxprof.uk.volume(48, 4, 23)}) AS v SELECT * FROM cxprof.uk.shapes WHERE default:cxprof.uk.volume(x, y, z) > 1000; EXECUTE FUNCTION default:cxprof.uk.volume(3, 2, 4);
  • 15. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. N1QL User Defined Functions using Javascript ● Add the javascript into the library ● Add the javascript function into the lilbray. ● CREATE FUNCTION in your scope ● This function can be used in any DML ● Larger functions can be registered from a file. curl -v -X POST http://localhost:8093/functions/v1/libraries/math/ functions/add -H 'content-type: application/json' -d '{"name": "add", "code": "function add(a, b) { let data = a + b; return data; }"}' -u Administrator:password CREATE FUNCTION default:cxprof.uk.jsadd(x, y) LANGUAGE JAVASCRIPT AS "add" at "math"; EXECUTE FUNCTION default:cxprof.uk.jsadd(99,232); /* Error : Mismatch number of parameters*/ EXECUTE FUNCTION default:cxprof.uk.jsadd(1, 2, 3); [ { "code": 10104, "msg": "Incorrect number of arguments supplied to function jsadd - cause: jsadd" } ] ● Each query node comes with V8 javascript engine ● Each invocation of the javascript has about 8ms overhead. ○ That’s each invocation overhead. ● Do something substantial in the UDF. ● The number of parameters during CREATE FUNCTION and invocation should match ● Function name should be UNIQUE within the scope ● Create up to 500 functions in a cluster.
  • 16. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. N1QL User Defined Functions using Javascript ● You can have the code in a file and register ● Everything else is the same. ● Globals aren’t supported ● The javascript functions are for compute ○ They can’t invoke N1QL statements from within ● They can invoke the usual Javascript library File: add3.js {"name": "add3", "code": "function add3(a, b, c) { let res = a + b + c; return res; }"} curl -v -X POST http://localhost:8093/functions/v1/libraries/math/ functions/add3 -H 'content-type: application/json' -d @./add3.js -u Administrator:password CREATE FUNCTION default:cxprof.uk.jsadd3(p,q,r) LANGUAGE JAVASCRIPT AS "add" at "math"; SELECT default:cxprof.uk.jsadd3(3, 4, 5)
  • 17. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 17 Optimizer Features 4
  • 18. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. N1QL: Cost Based Optimizer (CBO) 32 minutes RULE BASED ROUTING COST BASED ROUTING
  • 19. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. N1QL: Rule Based Optimizer (RBO) N1QL Parser Semantic Analyzer Rule Based Optimizer Query Execution Tree Executor Query Results Debug info (Query Profile) MetadataSystem metadata Query Explain PlanN1QL Statement
  • 20. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. N1QL: Cost Based Optimizer (CBO) N1QL Parser Semantic Analyzer Cost Based Optimizer Query Execution Tree Executor Query Results Debug info (Query Profile) STATISTICS & metadataSystem metadata Query Explain PlanN1QL Statement P atent P ending
  • 21. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. N1QL: Cost Based Optimizer ● N1QL Optimizer Decisions ○ Query Rewrites ○ Index selection ■ Predicate pushdown ■ Pagination pushdowns (LIMIT, OFFSET) ○ Join type and sides selection ■ Nested loop (NL) or hash join (HJ) ■ NL: decide outer and inner sides ■ HJ: decide build and probe sides ● Create the plan and query execution tree. ● CBO: Decisions will be statistics ○ BETTER DECISIONS ● Collect statistics using UPDATE STATISTICS ● Best practices for collecting statistics are documented ● ADVISE recommends UPDATE STATISTICS as well ● When statistics is absent, rule based is used. UPDATE STATISTICS FOR `travel-sample` (type, destinationairport, city, faa, city, country)
  • 22. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. N1QL: Cost Based Optimizer ● Use EXPLAIN as usual to see the CBO decisions ● EXPLAIN will give you the estimated cardinality & cost ● Profile output gives you actual cardinalities as well ● USE INDEX, USE HASH will continued to be supported to override the default decisions "#stats": { "#itemsOut": 168168, "#phaseSwitches": 672677, "execTime": "526.643124ms", "kernTime": "227.247789ms", "servTime": "70.269443ms" }, "as": "route", "cardinality": 128068.90036900375, "cost": 70181.75740221406, "covers": [ "cover ((`route`.`sourceairport`))", "cover ((`route`.`destinationairport`))", "cover ((distinct (array (`v`.`day`) for `v` in (`route`.`schedule`) end)))", "cover ((meta(`route`).`id`))" ],
  • 23. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. N1QL: Index Advisor ● ADVISE on a single statement ○ Preview in 6.5, GA in 6.6. ○ Enhance for CBO in the upcoming release ● ADVISOR() on a workload ○ Preview in 6.5, GA in the upcoming release ● ADVISE uses the rules for creating indexes and gives non-covering and covering indexes. ○ For CBO, it also advises on statistics ● ADVISOR() can be invoked manually with set of queries or as a result of queries from completed requests ● ADVISOR Input ○ Workload ○ Result of a query ○ Static set of queries ○ It merges the index recommendations. ○ It is 1.0. Give us feedback. ADVISE SELECT DISTINCT route.destinationairport FROM `travel-sample` airport JOIN `travel-sample` route ON airport.faa = route.sourceairport AND route.type = "route" WHERE airport.type = "airport" SELECT ADVISOR({"action": "start", "response": "0s", "duration": "1h"}) AS Collect; SELECT ADVISOR({"action": "get", "session": "8c41a3c6-2252-437e-ab47-0b28f29f47fb"}) AS Get; SELECT ADVISOR(( SELECT RAW statement FROM system:completed_requests WHERE LOWER(statement) LIKE "select%" AND LOWER(statement) NOT LIKE "%advis%")) INDEX-ADVISO R.CO UCHB ASE.COM
  • 24. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 24 Indexing Features 5
  • 25. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. N1QL: Data Logical Containment Couchbase Cluster Bucket (cxprof) Scope (usa) Collection (profille) ● Indexes per collection ○ Improved performance & resource usage ● Index names have to be unique PER collection ● All the index types are still supported ○ Partial, partitioned, expression, etc ○ Partial indexes on collections with multi doc types ● Supports large number of indexes ● Tools, query, explain are all enhanced to support it SELECT profname AS name, uspszip AS postalcode FROM cxprof.usa.loginfo l where l.type = "login"; CREATE INDEX i1 ON cxprof.usa.loginfo(profname, uspszip) PARTITION BY HASH(META().id) GSI Indexes on profile GSI Indexes on loginfo Collection (loginfo)
  • 26. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 26 Search Features 6
  • 27. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. FTS: Searching on Collections Couchbase Cluster Bucket (cxprof) Scope (usa) Collection (profille) ● FTS indexes on a single collection ○ Familiar indexing and queries ○ Support for SEARCH() from N1QL & Flex indexing ● Single FTS index on data residing in multiple collections ○ Direct query from FTS ○ Simpler search FTS Index on profile Collection (loginfo) FTS index on documents in gameinfo & loginfo Collection (gameinfo)
  • 28. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. ● Improve query performance for ad hoc queries, and provide a predictable query performance in applications where search patterns are not predetermined. This is available in Couchbase 6.6. ● Allow applications to leverage the unique capability of Keyword analyzer in search index to support complex queries involving combinations of AND/OR and multiple array predicates. ● Can greatly reduce the number of indexes and resource needed to support complex queries. SELECT * FROM crm a USE INDEX (USING FTS) WHERE a.type='activity' AND ( a.dept = 'iA88' OR a.region > '59416' ) AND a.priority = 'High' AND ( a.act_date BETWEEN '2018-01-01' AND '2018-08-31' OR a.event.location = 'Moscone Center' ) AND ( a.account.id = 'acc100' OR a.owner.name = 'Amanda Morrison') Flex Index allows query service to leverage search capabilities, using standard N1QL N1QL: FLEX INDEX P atent P ending
  • 29. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 29 DISTRIBUTED N1QL TRANSACTIONS 7
  • 30. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. Six thousand years ago, the Sumerians invented writing for TRANSACTION PROCESSING
  • 31. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. ● Multi-Everything: documents, statements, collections, scope, buckets, APIs, tools ● Ensure data integrity for application transaction that consists of multiple operations where multiple concurrent activities can occur ● Support all or nothing for multi documents N1QL DML operations ● No central coordinator for scale out ● Optimistic Concurrency Control with RYOW ● Available in via SDKs, CBQ shell, REST API ● ACID details in the next slide. N1QL Transactions ensures database consistency when data modifications are performed with multiple documents in a single or multiple N1QL statements BEGIN TRANSACTION; UPDATE customer SET balance = balance + 100 WHERE cid = 4872; UPDATE customer SET balance = balance - 100 WHERE cid = 1924; SELECT cid, name, balance FROM customer WHERE cid IN [4872, 1924]; COMMIT ; N1QL: Transactions Patent Pending
  • 32. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. ACID ● ATOMIC ○ ALL or Nothing for each statement and the whole transaction ● CONSISTENT ○ Modifications are done on the qualified set ○ Only constraint is document key unique constraint ○ Handles halloween problem via index snapshots ● ISOLATED ○ READ COMMITTED isolation ○ Monotonic atomic reads ○ Scan consistency: Request_plus by default ■ Tunable with Unbounded ○ No dirty reads ● DURABLE ○ Majority - default ○ majorityAndPersistActive ○ persistToMajority Couchbase Cluster Query Query Query Apps GSI GSI FTS Key Valu e Apps Apps sdk sdk sdk Query CBQ Shell Query Workbench
  • 33. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. ACIDic Statements ● BEGIN TRANSACTION; ○ request to any of the query nodes. It returns txid ○ Send subsequent tx statements to same node ○ Send each statements with txid ● All DMLs are allowed. non-DMLs raise error ○ Supported: SELECT, INSERT, UPDATE, DELETE, UPSERT, MERGE ○ Failure within a statement simply fails and rolls back the statement. Transaction continues ● SAVEPOINT savepointname ○ Sets and names a save point ● ROLLBACK TO SAVEPOINT savepointname ○ Rolls back the changes to a named savepoint. ● ROLLBACK TRANSACTION; ○ Rollback the entire transaction ● COMMIT ○ Commits all the modifications into Couchbase ○ The whole transaction is rolled back in case of Couchbase Cluster Query Query Query Apps GSI GSI FTS Key Valu e Apps Apps sdk sdk sdk Query CBQ Shell Query Workbench
  • 34. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. ACID for single statement /* Simply set tximplicit to True */ UPDATE customer SET balance = balance + 100 WHERE dept = 100; INSERT INTO customer(KEY cid, VALUE v) SELECT meta().id AS cid, newcust AS v FROM newcust; ● N1QL uses optimistic concurrency control ● Caches the version of the modific documents ○ Caching the changes until commit takes memory! ○ Suitable for large number of small transactions ● When you’re trying to modify large number of docs ○ Issue the statement like before ○ Set the tximplicit parameter to True. ● When tximplicit is set to True ○ N1QL starts a new transaction ○ Each write is staged in this transaction ○ If there’s any failure, the whole operation is rolled back
  • 35. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. ACID Support in SDK try { transactions.run((ctx) -> { // starts the transaction TransactionGetResult cxobj = ctx.get(customer, cxid); ctx.replace(cxobj, newaddress); ctx.query("UPDATE customer SET balance = balance + 100 WHERE cid = 4872"); ctx.query("UPDATE customer SET balance = balance - 100 WHERE cid = 1924;"); ctx.query("SELECT cid, name, balance FROM customer WHERE cid IN [4872, 1924]"); // This call is optional - if you leave it off, the transaction // will be committed anyway. ctx.commit(); }); } catch (TransactionFailed e) { // Just log the error logger.warn("Transaction did not reach commit:"); for (LogDefer log: e.result().log().logs()) { logger.warn(log.toString()); } } ● The Java SDK Lambda Example ● Supports all the transactional config ● Mix and match KV and query ● More SDKs on the roadmap
  • 36. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. ● Documents qualified are done by scanning the index using the scan consistency ● RYOW: Transaction sees it’s own changes in all subsequent queries by caching the changes ○ Limit the number of updates in a multi-statement transaction ○ Use singleton transaction when necessary ● Updates are all optimistic. Avoid the hot row problem with best practices in application design ● Don’t do non-transactional writes on the documents participating in transactions ● Multi-statement transactions are lock free and coordination free. Designed for scale out N1QL: Best Practice
  • 37. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 37 WRAP 8
  • 38. 38 SQL Features Triggers Stored Procedures in PL-SQL, T-SQL, languages JSON, XML SQL is English for Relational Database SQL Invented by Don Chamberlin & Raymond Boyce at IBM N1QL, based on SQL, is English for JSON N1QL was invented by Gerald Sangudi at Couchbase SQL Instance Database Schema Table Row Column N1QL Cluster Bucket Scope Collection Document Attribute SQL Input and Output: Set(s) of Tuples N1QL STMT CREATE BUCKET CREATE INDEX None SELECT INSERT UPDATE DELETE MERGE Subqueries JOIN GROUP BY ORDER BY OFFSET, LIMIT EXPLAIN PREPARE EXECUTE GRANT ROLE REVOKE ROLE INFER PREPARE EXECUTE ADVISE (6.6) ADVISOR UPDATE-STATISTICS CREATE FUNCTION BEGIN WORK COMMIT WORK SAVEPOINT ROLLBACK EXECUTE FUNCTION Set of JSON N1QL Model Set of JSON SQL Tooling ODBC, JDBC, .NET Hibernate BI Tools erwin TOAD SQL Indexes Primary Key Secondary Key Composite Range Partitioned Expression Functional Spatial Search SQL Logic 3 valued logic TRUE, FALSE, NULL/UNKNOWN N1QL Tooling ODBC,JDBC, .NET Couchbase SDKs Spring BI Tools erwin SQL TRANSACTIONS ACID Multi-Statement Multi-table Mullti-Database Savepoints Commit Rollback [savepoint] 2 phase locking Multi Isolation Levels Single node N1QL TRANSACTIONS ACID Multi-Statement Mullti-collection Multi-Bucket Savepoints Commit Rollback [savepoint] OCC (Optimistic) READ COMMITTED Distributed tx SQL Datatypes Numeric Decimal Character Date Time Timezone BLOB Spatial JSON N1QL Datatype Numeric Boolean Character Array Object Null JSON Conversion Functions SQL Optimizer Rule Based Cost Based Index Selection Join reordering Query Rewrites NL, Hash, Merge join N1QL Optimizer Rule Based Cost Based Index Selection Join as specified Query Rewrites(Ltd) NL, Hash join N1QL Engine (SCALE UP & SCALE OUT) SQL STMT CREATE TABLE CREATE INDEX ALTER TABLE SELECT INSERT UPDATE DELETE MERGE Subqueries JOIN GROUP BY ORDER BY OFFSET, LIMIT EXPLAIN PLAN PREPARE EXECUTE GRANT REVOKE DESCRIBE PREPARE EXECUTE TRUNCATE N1QL Input and Output: Set(s) of JSON Tuples SQL Model Set of Tuples SQL Engine (SCALE UP) N1QL Features Eventing Service UDFs (N1QL, Javascript) Defacto JSON N1QL Logic 4 valued logic TRUE, FALSE, NULL/UNKNOWN, MISSING N1QL Indexes Primary Secondary Composite Partial Expression Functional Array Index Search Flex Index(6.6)
  • 39. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. THANK YOU