Is your SQL Exadata-aware?
2 likes•996 views
The document discusses Oracle Exadata, highlighting its unique software capabilities and hardware features that enhance database performance. It emphasizes the importance of optimizing SQL queries to fully utilize Exadata's capabilities, such as smart scans and efficient joins. Additionally, it addresses common pitfalls in SQL usage that can prevent effective leverage of Exadata’s powerful features.
![V$SQLFN_METADATA
• List of functions that can [not] be offloaded
– Increase version after version
• Should be familiar when writing SQL
• If SQL includes a not offloadable then
– Rewrite if possible using offloadable function
– If not, add super-set with offloadable function
• Specific functions can be turned off
1/29/17 32](https://image.slidesharecdn.com/exafriendlysql-170129154841/85/Is-your-SQL-Exadata-aware-32-320.jpg)
![Drop-in surprise #1
• Segment(s) too small for Smart Scan
– Legacy heavy partitioning to extreme of tiny parts
– Can also happen as consequence of compression
• Confirmed by nsmtio trace
SqlId = 9rxuuptqp3ru2, phv = 2502846130, Part# = 1
NSMTIO: kcbism: islarge 1 next 0 nblks 915656 ...
…
NSMTIO: qertbFetch:[MTT < OBJECT_SIZE < VLOT]:…
NSMTIO: kcbdpc:DirectRead
SqlId = 1vrwjrfzxmknd, phv = 2502846130, Part# = 2
NSMTIO: kcbism: islarge 0 next 0 nblks 836 …
…
NSMTIO: qertbFetch:NoDirectRead:[- STT < OBJECT_SIZE < MTT]
1/29/17 41](https://image.slidesharecdn.com/exafriendlysql-170129154841/85/Is-your-SQL-Exadata-aware-41-320.jpg)
Is your SQL Exadata-aware?
- 2. What is Exadata? • Complete solution to run Oracle RDBMS • Lots of top shelf hardware – That you can buy anywhere... • Specific software capabilities – That you can’t buy anywhere else J 1/29/17 2
- 3. Very partial list of Exa-specific features • Some are more advertised – Smart Scan (since 11.1) – Hybrid Columnar Compression (11.2.1.2) – Smart Flash Cache (11.2.1.2) – Smart Flash Log (11.2.2.4) • Some are less – Columnar Flash Caching (12.1.2.1.0) – Smart Block Transfer (12.1.2.2.0) 1/29/17 3
- 4. Exadata is a F1 car • ”Smart” features solve many old issues – Very large I/O bandwidth • Lots of CPU and memory available • Exadata is basically a supercar • But do you drive it at its full potential? – Do your SQLs fully leverage Exadata? 1/29/17 4
- 6. How many people drive Exadata • Many migrations are drop-in – Nothing special to make SQLs more Exa-friendly • SQLs often run much faster – Hardware is very powerful – Exa-specific SW features enabled by default – Many features require almost no configuration 1/29/17 6
- 7. Driving lesson #1 - Memo • Smart Scan is your friend J – Opens the door to extremely powerful scan – Requires Full Scan and Direct Path Reads • That you spent 20y trying to avoid!!! – Doesn’t work for objects mostly in memory • IMPQ, ABTC, Full DB Caching – Doesn’t work for every object • Cluster tables, IOT, orascn/rowdependencies, etc (*) 1/29/17 7
- 8. Driving lesson #2 - Memo • CBO has no specific intelligence for Exadata – Same formulas, algorithms and params as before • That you spent 20y tweaking to avoid full scans – Won’t favor Exadata features explicitly (*) • Might need a little push to get more Exa-friendly – Exadata system statistics help a bit 1/29/17 8
- 9. Example of simple drop-in Analytical SQL on large dataset (Exa SW ON/OFF) select cust_gender, calendar_week_number, sum(amount_sold), sum(quantity_sold) from countries, -- from SH schema sales_big, -- ~2.5B rows customers, -- from SH schema times -- from SH schema where sales_big.cust_id = customers.cust_id and sales_big.time_id = times.time_id and customers.country_id = countries.country_id and countries.country_name = 'Italy' and calendar_quarter_desc = '2001-02' group by cust_gender, calendar_week_number; 1/29/17 9
- 10. Exadata, Smart Scan OFF Exadata X3 11.2.0.3 Exadata X2 12.1.0.2 1/29/17 10 Oracle 12c generated a better performing plan than 11g, regardless of Exadata
- 13. Exadata HW, Smart Scan ON Exadata X3 11.2.0.3 Exadata X2 12.1.0.2 1/29/17 13
- 16. Cool, I got to do nothing! • Partially true – Many features require almost no attention – They just work and give you decent gains • Partially false – Better use of some features if you know them – Using them right makes significant difference 1/29/17 16
- 17. 1/29/17 17
- 19. 1/29/17 19
- 21. Help the joins • Smart Scan reads / filters data efficiently – But it doesn’t help with joins • Is there a way to write a join as a filter? – Kind of, using Bloom filters to reduce data before join – Less data returned to compute nodes to join • Trick was hint to encourage diff join order -> 2 BF – One BF per dimension, similar to VT idea 1/29/17 21
- 22. Does it always work? • Of course not J • Efficiency depends on – Selectivity of filters from the dimensions – Amount of false positives – Number of Bloom Filters applied on large table • 1 BF on many cols vs many BFs on 1 col (*) 1/29/17 22
- 23. Another drop-in Large table filtered by lookup table(s) select sum(amount_sold), sum(quantity_sold) from sales_big where sales_big.cust_id in (select cust_id from customers, countries where customers.country_id = countries.country_id and countries.country_name = 'Italy' and cust_credit_limit = 15000) 1/29/17 23
- 26. Remove the subquery • Subquery produces low NDV • Can be pre-computed and used as filter • Hint /*+ PRECOMPUTE_SUBQUERY */ • Doesn’t always work – If the subquery alone takes long to execute – If high NDV then Smart Scan disabled (uses DPR) 1/29/17 26
- 27. Drop-in #3 Simple SQL using complex predicate SELECT COUNT(*) FROM martin.bigtab WHERE v1 = DECODE( CASE WHEN ( CASE WHEN 'ED' IN ('ED','AIS') THEN 1 WHEN ('NA'='P' OR 'ED'='SAQ') AND v1 <> 'Y' THEN 1 ELSE 0 END = 1) THEN 'X' ELSE 'N' END ,'X','0001161826', 'Y') 1/29/17 27 '0001161826'
- 28. 12c, drop-in vs Exa-friendly Drop-in Exa-friendly 1/29/17 28 cell pIO bytes eligible for pred offload 83886137344 cell pIO bytes saved by storage index 0 cell pIO inter bytes ret by smart scan 14740392 cell pIO bytes eligible for pred offload 83886137344 cell pIO bytes saved by storage index 31303966720 cell pIO inter bytes ret by smart scan 9241000
- 29. Drop-in #4 Simple SQL using less complex predicate SELECT COUNT(*) FROM sales_big WHERE TO_CHAR(time_id,'YYYY-MM-DD HH24') = '1997-10-09 10' 1/29/17 29
- 30. 12c, drop-in vs Exa-friendly Drop-in Exa-friendly 1/29/17 30 cell pIO bytes eligible for pred offload 103534297088 cell pIO bytes saved by storage index 0 cell pIO inter bytes ret by smart scan 5985839720 cell pIO bytes eligible for pred offload 103534297088 cell pIO bytes saved by storage index 81859526656 cell pIO inter bytes ret by smart scan 950385176
- 31. Complex filters • CBO attempts to simplify them if possible – No logic to simplify ALL of them • Storage Index works with simple operators – Complex filters won’t use SI • Simplify logic when possible • Use super-set filter predicates – Can be offloaded and use storage index – Reduce num of rows to apply complex pred on 1/29/17 31
- 32. V$SQLFN_METADATA • List of functions that can [not] be offloaded – Increase version after version • Should be familiar when writing SQL • If SQL includes a not offloadable then – Rewrite if possible using offloadable function – If not, add super-set with offloadable function • Specific functions can be turned off 1/29/17 32
- 33. So far we learned • Clean your CBO environment – Let the CBO do its job J • Filter sooner to process less later – Help the CBO if needed • Applies to – Joins -> Bloom Filters – Subqueries -> Unnest+BF or precompute subq – Complex pred -> simplify or super-set pred • Old concept, indexes serve same purpose 1/29/17 33
- 35. What if I get an engine problem? • Previous examples increase offload efficiency – It’s like going faster with a functioning car • What is the car doesn’t start? – Several conditions disable / impact smart scan – Learning them to • Avoid unexpected surprises • Identify way to alleviate impact 1/29/17 35
- 38. Smart Scan party poopers • Requirements not met – No FullScan+DPR, mostly cached, etc • Not available – Cluster, IOT, rowscn, etc • Rows need resolution in compute node – Chained rows with rowpiece in different AU – Consistent reads not helped by minscn cache – Saturated cells shipping data to balance CPU 1/29/17 38
- 41. Drop-in surprise #1 • Segment(s) too small for Smart Scan – Legacy heavy partitioning to extreme of tiny parts – Can also happen as consequence of compression • Confirmed by nsmtio trace SqlId = 9rxuuptqp3ru2, phv = 2502846130, Part# = 1 NSMTIO: kcbism: islarge 1 next 0 nblks 915656 ... … NSMTIO: qertbFetch:[MTT < OBJECT_SIZE < VLOT]:… NSMTIO: kcbdpc:DirectRead SqlId = 1vrwjrfzxmknd, phv = 2502846130, Part# = 2 NSMTIO: kcbism: islarge 0 next 0 nblks 836 … … NSMTIO: qertbFetch:NoDirectRead:[- STT < OBJECT_SIZE < MTT] 1/29/17 41
- 43. Drop-in surprise #2 • Usually because of – Large UPD causing chained rows – Tables with > 255 cols in different AU • Chained rows are processed – In the cell if next piece available (less likely) • chained rows processed by cell – In the compute node if not (more likely) • chained rows skipped by cell • Some fixes (e.g. 9373758) try to minimize impact 1/29/17 43
- 44. Drop-in surprise #3 CR needed, returning data to compute node ------------------------------------------------------------------------------------------------------- SID @INST, USERNAME , TYPE, STATISTIC , DELTA ------------------------------------------------------------------------------------------------------- 1497 @1, EO00 , STAT, cell physical IO interconnect bytes , 650748816 1497 @1, EO00 , STAT, cell physical IO bytes eligible for predicate offload , 612196352 1497 @1, EO00 , STAT, cell physical IO interconnect bytes returned by smart scan, 613436648 1497 @1, EO00 , STAT, cell blocks processed by cache layer , 76033 1497 @1, EO00 , STAT, cell commit cache queries , 76033 1/29/17 44
- 45. Drop-in surprise #3 • Uncommited changes – Smart Scan returns data needs CR to compute • Not Exadata specific – Just push work back to compute • For commited changes MINSCN cache helps – cell blocks helped by minscn optimization 1/29/17 45
- 46. Conclusion • Drop-in migration often sub-optimally use Exadata • Legacy SQLs may need adjustments: – To fully leverage Exadata features – To improve areas where Exadata doesn’t automatically help • Most of the adjustments are simple: – Clean CBO environment – Streamline filter predicates • Sometimes CBO needs a little push • Get familiar with Smart Scan disablers – To predict behaviors and avoid surprises 1/29/17 46
- 47. 47
- 48. References • Oracle® Exadata Storage Server Software User's Guide 12c Release 1 (12.1) • Oracle® Exadata Database Machine System Overview 12c Release 1 (12.1) • Expert Oracle Exadata • Google.com J 48
- 49. Contact Information • http://mauro-pagano.com – Email • [email protected] – Download(s) • SQLd360 vYYMM (date) • TUNAs360 vYYMM (date) • Pathfinder vYYMM (date) 49