An introduction to SQL Server in-memory OLTP Engine
Download as PPTX, PDF0 likes1,076 views
The document provides an overview of SQL Server's in-memory OLTP engine introduced in SQL Server 2014, aiming for a significant speedup of transaction processing through memory optimization. It highlights key architectural changes including lock-free in-memory indexes, integration with the SQL Server, and the use of multi-version concurrency control for efficient data handling. The in-memory OLTP engine supports various design patterns and operational features while phasing out traditional locking mechanisms.
![SQL Server 2014 In-memory OLTP Engine
Optimize for main memory
1. No 8KB pages for tables; only stream of bytes
2. New lock and latch free in-memory indexes
1. Hash index (equality)
2. Bw index (range) [“Buzz Word” index]
3. Using MVCC (Muti-Version Concurrency
control); transactions see a snapshot of data
4. Log, data changes to transaction log
5. No logging for index changes; recreates when
server restart](https://image.slidesharecdn.com/anintroductiontosqlserverin-memoryoltp-140929002246-phpapp01/85/An-introduction-to-SQL-Server-in-memory-OLTP-Engine-7-320.jpg)
An introduction to SQL Server in-memory OLTP Engine
- 1. An introduction to SQL Server In-Memory OLTP Krishnakumar S
- 2. Traditional OLTP Architecture •Disk is cheap – Data lives in disk •Memory is fast – Data modified in memory (Buffer) •Disk to Memory – I/O utilization •Query processing – CPU & Memory utilization •ACID – I/O utilization Write ahead log, Transaction Log CPU utilization (Locking, latching) •Memory to Disk – I/O utilization (Dirty pages to disk – Checkpoint)
- 3. Traditional OLTP Architecture High volume data modifications Contentions – “Competition for resources” •Lock •Latch •Disk I/O •Allocation Airlines Sensor data Application logging Conversations etc…
- 4. Changing scenario….. CPU vs. Memory vs. I/O “The Free lunch is over…” – Herb Sutter, Dr. Dobb’s Journal, March 2005 CPU Still hold Moor’s law Becoming faster Multicore Memory Getting cheaper Becoming faster Hold more data I/O Lagging far behind CPU and Memory “Tape is dead Disk is Tape Flash is Disk…” – Jim Gray, Microsoft, Dec 2006
- 5. SQL Server 2014 In-memory OLTP Engine Goal was to achieve 100x speedup for certain TP operations Hekaton a Greek word for 100 Microsoft spend nearly seven years on Hekaton A major change in SQL Server engine from version 7.0
- 6. SQL Server 2014 In-memory OLTP Engine Four architectural pillars 1. Optimize for main memory data access 2. Accelerate business logic processing 3. Provide frictionless scale-up 4. Built-in to SQL Server
- 7. SQL Server 2014 In-memory OLTP Engine Optimize for main memory 1. No 8KB pages for tables; only stream of bytes 2. New lock and latch free in-memory indexes 1. Hash index (equality) 2. Bw index (range) [“Buzz Word” index] 3. Using MVCC (Muti-Version Concurrency control); transactions see a snapshot of data 4. Log, data changes to transaction log 5. No logging for index changes; recreates when server restart
- 8. SQL Server 2014 In-memory OLTP Engine Accelerate business logic processing 1. Table and index metadata compiled to DLL 1. Table row format preserved in DLL 2. Traditional query engine can be leveraged to access in-memory tables 2. Stored Procedure compiled to DLL
- 9. SQL Server 2014 In-memory OLTP Engine Frictionless scale-up 1. Multi-version optimistic concurrency control with ACID support 2. Lock free algorithms for in-memory engine 3. No lock manager, latches or spin lock
- 10. SQL Server 2014 In-memory OLTP Engine Built-in to SQL Server 1. In-memory engine is integrated in to SQL Server 2. Both memory optimized and traditional tables can be created for a database 3. No lock manager, latches or spin lock
- 11. In-memory OLTP Engine – Behind the scenes
- 12. In-memory OLTP Engine – Behind the scenes Rows • Row Header – BeginTS • TX TS that created the row – EndTS • TX TS that deleted the row • Infinity – StmtID • Statement that created the row – IdxLinkCount • Number of indexes that reference this row • Payload = actual data TxID global DB counter Reset on restart Incremented on TX start TxTimeStamp global DB counter Not reset on restart Incremented on TX end
- 13. In-memory OLTP Engine – Behind the scenes Rows – Hash index Hash index on Name f(John) Transaction 99: SELECT City WHERE Name = ‘John’ Simple hash lookup returns direct pointer to ‘John’ row 50, 100 50, ∞ John Paris 100, ∞ John Prague 90, ∞ Susan Bogota Timestamps Chain ptrs Name City Transaction 100: UPDATE City = ‘Prague’ where Name = ‘John’ No locks of any kind, no interference with transaction 99
- 14. In-memory OLTP Engine – Behind the scenes Rows – Non clustered index
- 15. In-memory OLTP Engine – Behind the scenes Rows – Non clustered index Page Mapping Table Addr. 0 1 2 3 4 … Δ Deleted record 48 Delta record Δ Inserted Record 50 Page P Delta record PID 0 1 2 3 4 … 4.21
- 16. In-memory OLTP Engine – Summary • T-SQL Features – 2 ways to access in-memory tables • T-SQL interop (usually slower) • Natively compiled stored procedures (fast) • T-SQL non supported features – TRUNCATE Table – MERGE (when in-memory table is the target) – Linked servers – Locking hints: TABLOCK, XLOCK, PAGLOCK, etc. • NOLOCK is supported, but is quietly ignored
- 17. In-memory OLTP Engine – Summary • Stored Procedures – TRY, CATCH, error functions, IF, WHILE – Memory-optimized table types and table variables – Math, Date, String functions – SCOPE_IDENTITY, ISNULL, NEWID, NEWSEQUENTIALID
- 18. In-memory OLTP Engine – Summary • Common Design patterns – In-Memory OLTP – High-performance OLTP – High data input or ”Shock Absorber” – Relational cache – Read-scale – ETL target and staging/temp tables
- 19. Demo
- 20. Questions?
- 21. Thank You