Oracle Database In-Memory Option for ILOUG
2 likes1,340 views
The document provides an overview of the Oracle database's In-Memory Option and its features, such as allowing data storage in both row and column formats to optimize analytic queries and improve performance. It highlights the benefits, limitations, and configuration details of in-memory and column store databases, along with specific SQL commands for managing in-memory data. The discussion is intended for database professionals, emphasizing the potential for significantly faster queries with proper implementation.
Oracle Database In-Memory Option for ILOUG
- 1. Zohar Elkayam CTO, Brillix [email protected] Twitter: @realmgic Oracle Database In Memory Option
- 2. Agenda •About in memory stores •About column stores •Introduction to Database In Memory –Oracle In Memory Option •How to use DBIM •Short demo 2 http://brillix.co.il
- 3. Who am I? •Zohar Elkayam, CTO at Brillix •Oracle ACE Associate •DBA, team leader, instructor and senior consultant for over 16 years •Editor (and manager) of ilDBA –Israel Database Community •Blogger –www.realdbamagic.com 3 http://brillix.co.il
- 4. What are In Memory Databases and Column Stores? 4 http://brillix.co.il
- 5. What is a In Memory Database? In memory databases are management systems that keeps the data in a non-persistent storage (RAM) for faster access Examples: •MemcacheDB •Oracle TimesTen 5 http://brillix.co.il
- 6. What is a Column Store Database? •Column Store databases are management systems that uses data managed in a columnar structure format for better analysis of single column data (i.e. aggregation). Data is saved and handled as columns instead of rows. Examples: •Vertica •GreenPlum •HBase 6 http://brillix.co.il
- 7. How Records are Organized? •This is a logical table in RDBMS systems •Its physical organization is just like the logical one: column by column, row by row Row 1 Row 2 Row 3 Row 4 Col 1 Col 2 Col 3 Col 4 7 http://brillix.co.il
- 8. Query Data •When we query data, records are read at the order they are organized in the physical structure •Even when we query a single column, we still need to read the entire table and extract the column Row 1 Row 2 Row 3 Row 4 Col 1 Col 2 Col 3 Col 4 Select Col2 From MyTable Select * From MyTable 8 http://brillix.co.il
- 9. How Does Column Store Save Data Organization in row store Organization in column store 10 http://brillix.co.il
- 10. Row Format vs. Column Format 11 http://brillix.co.il
- 11. Column Store Limitations •Most Column stores avoid or limit online transactions •Most Column stores avoid data changes (updates) and implement it as insert/delete •Columnar organization is very good for large quantities of data but is not very efficient when storing smaller amounts of data •Column store SQL might be somewhat different from ANSI SQL 12 http://brillix.co.il
- 12. In Memory Option 13 http://brillix.co.il
- 13. In Memory Option Breakthrough •In memory option introduces a dual format database •Tables can be accessed as row format and column format at the same time –the Optimizer is aware to the new format so: •OLTP continue using the old row format •Analytic queries start using the column format 14 http://brillix.co.il
- 14. In-Memory Option •Column data is pure in memory format and therefor is non- persistent and require no logging, archiving or backup •Data changes are simultaneously changed in both formats so data is consistent •Since the data is column based, it has better compression ratio and more data can be stored in memory •There are no changes to the application required –just turn on and start using 15 http://brillix.co.il
- 15. Order of Magnitude Faster •Customer report analytic queries run 10 to 1000 times faster •Since less analytic Indexes are needed, OLTP run faster too •Data processing is done using SIMD Vector Instruction so billions of records can be handled by a single CPU 16 http://brillix.co.il
- 16. In Memory Option –Good To Know •Oracle 12.1.0.2 feature –additional license required •It is NotIn Memory Database –it’s an accelerator to our current database •It is NotColumn Store Database –it allows keeping some of our data in column store which is non-persistent •It has nothing to do with Times-Ten or Oracle Coherence 17 http://brillix.co.il
- 17. More Good to Know •In memory option does not work on an Active Data Guard standby database but roadmap does contains this feature in the future. It does work on logical standby database •Optimization of the execution plan in regards of using the DBIM might (and probably will) change in future versions •Known bugs: no functionality bugs known at this time but there is a known bug around the license auditing of this feature 18 http://brillix.co.il
- 18. How To Use DBIM? 19 http://brillix.co.il
- 19. How to Configure •Configure memory capacity •The amount of memory allocated must be larger than the amount of data loaded (after compression). We should also allow some spare memory for maintenance. •Configure tablespaces, tables, partitions, sub-partitions or columns to be in memory: •Optional: drop unused OLTP indexes –those who used for analytics. inmemory_size= xxx GB alter table | partition ...inmemory; 20 http://brillix.co.il
- 20. Startup SQL> startup ORACLE instance started. Total System Global Area 5368709120 bytes Fixed Size 3056960 bytes Variable Size 620759744 bytes Database Buffers 1509949440 bytes Redo Buffers 13717504 bytes In-Memory Area 3221225472 bytes Database mounted. Database opened. 21 http://brillix.co.il
- 21. Parameters Related to In Memory Option SQL> show parameter inmememory NAME TYPE VALUE ----------------------------------------------------------------------------- inmemory_clause_defaultstring inmemory_forcestring DEFAULT inmemory_max_populate_serversinteger 2 inmemory_querystring ENABLE inmemory_sizebig integer 3G inmemory_trickle_repopulate_servers_ integer 1 percent optimizer_inmemory_awarebooleanTRUE 22 http://brillix.co.il
- 22. Where Does the In Memory Resides? •In Memory is part of the SGA –adding this buffer will require changing the SGA size Shared pool Buffercache Redo logbuffer Streams pool Large pool Java pool System Global Area (SGA) KEEP buffer pool RECYCLE buffer pool nKbuffer cache Column Store In Memory Buffer 23 http://brillix.co.il
- 23. Checking Table Settings SQL> r 1 SELECT table_name, 2 inmemory, 3 inmemory_priority, 4 inmemory_distribute, 5 inmemory_compression, 6 inmemory_duplicate 7 FROM user_tables 8* ORDER BY table_name TABLE_NAME INMEMORY INMEMORYINMEMORY_DISTRI INMEMORY_COMPRESS INMEMORY_DUPL --------------------------------------------------------------------------------- PEOPLE ENABLED HIGH AUTO FOR QUERY LOW NO DUPLICATE PEOPLE2 DISABLED TOWNS DISABLED 24 http://brillix.co.il
- 24. Managing In Memory at the Column Level •Setting a table to In Memory automatically set all the column to be loaded to the memory •If we want to load some of the columns we explicitly need to state which columns are NOT loaded to the memory: CREATE TABLE im_col_tab( id NUMBER, col1 NUMBER, col2 NUMBER, col3 NUMBER, col4 NUMBER ) INMEMORY INMEMORY MEMCOMPRESS FOR QUERY HIGH(col1, col2) INMEMORY MEMCOMPRESS FOR CAPACITY HIGH (col3) NO INMEMORY (id, col4); 25 http://brillix.co.il
- 25. Table Types That are Not Supported •There are tables and column types which are not supported: •IOT •Clustered tables •Objects owned by SYS, SYSTEM or stored in the SYSTEM and SYSAUX tablespaces •LONG type columns •Out of line LOB 26 http://brillix.co.il
- 26. How Data is Populated in the Memory •By default, data is being populated in the in memory cache while first reading the table •Objects could be configured to be loaded as the instance starts and prioritized according to the application needs •There are 5 level of prioritization: None, Low, Medium, High and Critical. None means no pre-loading, critical means before all others. 27 http://brillix.co.il
- 27. Checking Memory Population SQL> r 1 select segment_name, 2 inmemory_size, 3 bytes_not_populated, 4 populate_status, 5 inmemory_compression, 6 bytes / inmemory_sizecomp_ratio 7* from v$im_segments SEGMENT_NAME INMEMORY_SIZE BYTES_NOT_POPULATED POPULATE_ INMEMORY_COMPRESS COMP_RATIO -------------------------------------------------------------------------------------------------- PEOPLE 853540864 0 COMPLETED FOR QUERY LOW 2.43734644 28 http://brillix.co.il
- 28. How to Control In Memory Compression 29 http://brillix.co.il
- 29. Explain Plan, No In Memory SQL> select avg(salary), max(p.id_town_work), count(salary) from people p; Elapsed: 00:00:23.56 Execution Plan ---------------------------------------------------------- Plan hash value: 470504681 ----------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | ----------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1 | 26 | 68771 (1)| 00:00:23 | | 1 | SORT AGGREGATE | | 1 | 26 | | | | 2 | TABLE ACCESS FULL| PEOPLE | 160M| 3960M| 68771 (1)| 00:00:23 | ----------------------------------------------------------------------------- 30 http://brillix.co.il
- 30. Query Statistics –No In Memory Statistics ---------------------------------------------------------- 6 recursive calls 0 dbblock gets 2583630 consistent gets 2501640 physical reads 0 redo size 726 bytes sent via SQL*Net to client 552 bytes received via SQL*Net from client 2 SQL*Net roundtrips to/from client 0 sorts (memory) 0 sorts (disk) 1 rows processed 31 http://brillix.co.il
- 31. Explain Plan SQL> select avg(salary), max(p.id_town_work), count(salary) from people p; Elapsed: 00:00:01.10 Execution Plan ---------------------------------------------------------- Plan hash value: 470504681 -------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | -------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1 | 26 | 2733 (6)| 00:00:01 | | 1 | SORT AGGREGATE | | 1 | 26 | | | | 2 | TABLE ACCESS INMEMORYFULL| PEOPLE | 160M| 3960M| 2733 (6)| 00:00:01 | -------------------------------------------------------------------------------------- 32 http://brillix.co.il
- 32. Query Statistics –In Memory Statistics ---------------------------------------------------------- 0 recursive calls 0 dbblock gets 9 consistent gets 0 physical reads 0 redo size 726 bytes sent via SQL*Net to client 552 bytes received via SQL*Net from client 2 SQL*Net roundtrips to/from client 0 sorts (memory) 0 sorts (disk) 1 rows processed 33 http://brillix.co.il
- 33. Disabling In Memory Option •Disabling/Enabling in memory query at the session/system level at the optimizer level: •Disabling object maintenance: •Removing In Memory Option (requires restart): ALTER SESSION|SYSTEM SET INMEMORY_QUERY=DISABLE|ENABLE; ALTER SYSTEM SET INMEMORY_FORCE=OFF|DEFAULT; ALTER SYSTEM RESET INMEMORY_SIZE SCOPE=SPFILE ; SHUTDOWN IMMEDIATE ; STARTUP ; 34 http://brillix.co.il
- 35. What Did We Not Talk About? •Working with DBIM with RAC, Engineered systems and in multitenant environments •What happens when there is not enough memory? •In memory storage indexes and In memory joins •Read consistency, IMCU staleness and trickle repopulate 36 http://brillix.co.il
- 36. Conclusion •The DBIM is a very interesting feature which can make analytic queries run much faster •Working with in memory option requires understanding of the database model and relevant queries –memory is not infinite. •No significant bugs found yet but it’s very early •Some behaviors may (and should) change in the future 37 http://brillix.co.il
- 37. Thank YouZohar [email protected] 38 http://brillix.co.il