Indexing in Exadata

Indexing In Exadata
Richard Foote
Oracle Corporation

Richard Foote
• Working in IT for 25+ years (scary stuff)
• Working with Oracle Database for 15+ years
• Spent 19 years employed in Australian Federal Government in various IT roles,
last 9 years as a senior DBA with the Australian Federal Police
• Responsible for many large scale, mission critical, “life-dependant” classified
Oracle systems
• Previously a DBA Instructor with Oracle Corporation (1996 and 2002)
• Since Nov 2011, re-joined Oracle Corporation as a Technical Solutions
Consultant based in sunny Canberra
• Spent much of this time playing with Exadata (one of the reasons I rejoined) 
• Oracle OakTable Member since 2002 and Oracle ACE Director since 2008 (now
Oracle Employee Ace)
• Richard Foote’s Oracle Blog: http://richardfoote.wordpress.com/
• Spend as much free time as possible listening to the music of David Bowie !!
2 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Exadata Innovations
Exadata Storage Server Software
 Intelligent storage • Hybrid Columnar Compression
– Smart Scan query offload – 10x compression for warehouses
– Scale-out storage – 15x compression for archives

Data
+ + + remains
compressed Uncompressed
for scans
• Smart Flash Cache and in Flash
– Accelerates random I/O up to 30x primary backup
– Doubles data scan rate Benefits test
Multiply standby dev’t
Compressed


Things can run so much faster in Exadata …
With Non-Exadata …
SQL> select * from dwh_bowie where album_id = 42 and artist_id between 42 and 4200;
266176 rows selected.
Elapsed: 00:04:43.38
-------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 266K| 23M| 2348K (1)| 07:49:45 |
|* 1 | TABLE ACCESS FULL| DWH_BOWIE | 266K| 23M| 2348K (1)| 07:49:45 |
-------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - filter("ALBUM_ID"=42 AND "ARTIST_ID"<=4200 AND "ARTIST_ID">=42)
Statistics
----------------------------------------------------------
1 recursive calls
0 db block gets
8644512 consistent gets
8625479 physical reads
0 redo size
12279634 bytes sent via SQL*Net to client
195719 bytes received via SQL*Net from client
17747 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
266176 rows processed


Things can run so much faster in Exadata …
With Exadata, what previously took minutes can take seconds …
Elapsed: 00:00:03.97
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
|* 1 | TABLE ACCESS STORAGE FULL| DWH_BOWIE | 266K| 23M| 2348K (1)| 07:49:45 |
---------------------------------------------------------------------------------------
---------------------------------------------------
1 - storage("ALBUM_ID"=42 AND "ARTIST_ID"<=4200 AND "ARTIST_ID">=42)
filter("ALBUM_ID"=42 AND "ARTIST_ID"<=4200 AND "ARTIST_ID">=42)
Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
0 redo size
0 sorts (memory)
0 sorts (disk)


Indexing In Exadata …

Are indexes needed anymore ???


Worse - Indexes can get in the way …
SQL> create bitmap index dwh_album_artist_i on dwh_bowie(artist_id, album_id);
Index created.
Elapsed: 00:00:23.11
---------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 266K| 23M| 78267 (1)| 00:15:40 |
| 1 | TABLE ACCESS BY INDEX ROWID | DWH_BOWIE | 266K| 23M| 78267 (1)| 00:15:40 |
| 2 | BITMAP CONVERSION TO ROWIDS| | | | | |
|* 3 | BITMAP INDEX RANGE SCAN | DWH_ALBUM_ARTIST_I | | | | |
---------------------------------------------------------------------------------------------------
---------------------------------------------------
3 - access("ARTIST_ID">=42 AND "ALBUM_ID"=42 AND "ARTIST_ID"<=4200)
filter("ARTIST_ID">=42 AND "ARTIST_ID"<=4200 AND "ALBUM_ID"=42)
Statistics
----------------------------------------------------------
1 recursive calls
0 db block gets
29728 redo size
0 sorts (memory)
0 sorts (disk)


Indexing In Exadata …
Are indexes evil? Should they just be dropped?

Mr. Index, or Dr. Evil !!


What to do? Important Decision
• Many suggest simply drop indexes once you move to Exadata
• Indexes take up much storage (important consideration, especially with ¼
Exadata racks where storage is more limited)
• A redundant index wastes space and resources to maintain
• Indexes critical to performance (or they certainly used to be)
• But can “get in the way” in Exadata
• Once dropped, tricky to keep track of what indexes previously existed
• As usual, the decision to drop or keep indexes not a straightforward one …


Let’s Go Back To The Basics
Cost of using an Index:
Basic Cost = blevel +
ceil(effective index selectivity x leaf_blocks) +
ceil(effective table selectivity x clustering_factor) +
a bit of CPU …

In Exadata, this is basically the same for uncompressed tables.
With EHCC tables, there’s some additional CPU costs required
to decompress rows.
Therefore actual cost of using an Index can increase in Exadata


Let’s Go Back To The Basics

Simplistically, cost of Full Table Scan:

Cost = (((No. of table blocks / effective multi block read count) x
multi block read time) +
a bit of CPU) /
single block read time

The costing is still calculated the same in Exadata, however this
may no longer represent the true costs…


Exadata Intelligent Storage Grid
 Exadata cells implement smart scans to greatly reduce the data that
needs to be processed by database
– Only return relevant rows and columns to database
– Offload predicate evaluation
 Data reduction can be very large
– Column and row reduction often decrease data to be returned to the
database by 10x


Exadata Smart Scans
 Full Scans (Tables/Indexes/MVs/Partitions)
 Direct Path Reads (By-Passing Buffer Cache)
 Smart Scan Optimizations, some of which include:
– Column Projection
– Predicate Filtering
– Simple Joins
– Function Off-Loading
– Virtual Column Evaluation
– HCC Decompression
– Decryption
 See MOS Notes 793845.1 and 50415.1 for more info on Direct Path Reads

Exadata Smart Scans - Benefits
 CPU utilization on database nodes improves due to offload
 Less data shipped to database nodes (generally …)
 Query fully encrypted database by moving decryption from database to
storage cell hardware
 Full Table Scan performance can dramatically improve
 Costs associated with Full Table Scan operation can therefore
significantly decrease, especially with Storage Indexes kicking in
 With index costs perhaps increasing and FTS costs perhaps
decreasing, indexes in Exadata become less viable …
 Not looking good for indexes !!!


Non-Exadata Chart: Indexes vs. FTS

Cost

Rows Returned
Full Table Scan
Index Scan


Non-Exadata Chart: Indexes vs. FTS (DWH)

xx x x
xx xx x
xx xx xx
xx
Cost x x xx
x x
x
x
x
x x
xx
Rows Returned
Full Table Scan
x Queries using indexes
Index Scan


Non-Exadata Chart: Indexes vs. FTS (OLTP)

x x
Cost x
x

x x
x xx
xx x
x
x x xx x
Rows Returned
Full Table Scan
Index Scan x Queries using indexes


Exadata Chart: Indexes vs. FTS

Cost

Rows Returned
Full Table Scan
Index Scan (on HCC tables)


Exadata Chart: Indexes vs. FTS (DWH)

x
xx
x x
x
xx xxxx x x
xx x x
Cost x xx
x
x x
x
x
x
x
xx
Rows Returned
Full Table Scan


Exadata Chart: Indexes vs. FTS (OLTP)

x x
Cost x
x

x x
x xxx
xx x
xx
x x xx
Rows Returned
Full Table Scan


Exadata Indexes vs. FTS (CBO Costs)

x
xx xx
xx x
x
xx xx xx
xx
Cost x x xx
x
x x
x
x
x x
xx
Rows Returned
Full Table Scan
Index Scan
CBO Costs


Exadata Indexes vs. FTS (CBO Costs)

x x
x
xx x
x
x xxxx x x x
x x
Cost x x xxx
x
x x
x
x x
x
xx
Full Table Scan
Rows Returned
Index Scan
FTS CBO Cost


Without Exadata Smart Scans
Going back to the first example …
Elapsed: 00:04:43.38
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
|* 1 | TABLE ACCESS FULL| DWH_BOWIE | 266K| 23M| 2348K (1)| 07:49:45 |
-------------------------------------------------------------------------------
---------------------------------------------------
Statistics
----------------------------------------------------------
1 recursive calls
0 db block gets
0 redo size
0 sorts (memory)
0 sorts (disk)


Smart Scan Example
Elapsed: 00:00:03.97

---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------

---------------------------------------------------

1 - storage("ALBUM_ID"=42 AND "ARTIST_ID"<=4200 AND "ARTIST_ID">=42)
filter("ALBUM_ID"=42 AND "ARTIST_ID"<=4200 AND "ARTIST_ID">=42)

Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
0 redo size
0 sorts (memory)
0 sorts (disk)


Smart Scan – How can you tell ?
SQL> select
sql_text, io_cell_offload_eligible_bytes, io_interconnect_bytes,
io_cell_uncompressed_bytes, io_cell_offload_returned_bytes
from v$sql where sql_id = 'admdhphbh1a1c';

SQL_TEXT
--------------------------------------------------------------------------------
IO_CELL_OFFLOAD_ELIGIBLE_BYTES IO_INTERCONNECT_BYTES IO_CELL_UNCOMPRESSED_BYTES
------------------------------ --------------------- --------------------------
IO_CELL_OFFLOAD_RETURNED_BYTES
------------------------------
select * from dwh_bowie where album_id = 42 and artist_id between 42 and 4200
1.4132E+11 55217792 1.6718E+10
55217792

SQL> select name , value/1024/1024 MB from v$statname n, v$mystat s where
n.statistic# = s.statistic# and n.name in ('cell physical IO interconnect bytes
returned by smart scan', 'cell physical IO bytes saved by storage index');

NAME MB
---------------------------------------------------------------- ----------
cell physical IO bytes saved by storage index 59414.9453
cell physical IO interconnect bytes returned by smart scan 26.329895


Non-Smart Scan – CBO Costs The Same
SQL> alter session set cell_offload_processing=false;
Session altered.
Elapsed: 00:03:03.38
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------

---------------------------------------------------

Statistics
----------------------------------------------------------
1 recursive calls
0 db block gets
0 redo size
0 sorts (memory)
0 sorts (disk)

CBO is not aware of smart scan benefits and FTS costs remain the same …

Less Efficient Index Chosen By CBO

Elapsed: 00:00:23.11
---------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------
| 1 | TABLE ACCESS BY INDEX ROWID | DWH_BOWIE | 266K| 23M| 78267 (1)| 00:15:40 |
| 2 | BITMAP CONVERSION TO ROWIDS| | | | | |
|* 3 | BITMAP INDEX RANGE SCAN | DWH_ALBUM_ARTIST_I | | | | |
---------------------------------------------------------------------------------------------------
---------------------------------------------------
3 - access("ARTIST_ID">=42 AND "ALBUM_ID"=42 AND "ARTIST_ID"<=4200)
filter("ARTIST_ID">=42 AND "ARTIST_ID"<=4200 AND "ALBUM_ID"=42)
Statistics
----------------------------------------------------------
1 recursive calls
0 db block gets
29728 redo size
0 sorts (memory)
0 sorts (disk)

Therefore existing indexes although maybe slower, appear much cheaper…

CBO and Smart Scans
 The decision to go for Direct Reads and use a Smart Scan is not made by the CBO
 The CBO is not aware of potential efficiencies provided by Smart Scans and costs
Full Scans similar to Non-Exadata
 Bug 10248538: OPTIMIZER COST MODEL NOT EXADATA-AWARE
 Therefore, Full Scans are generally over-costed in Exadata, favoring Index Scan
related execution paths
 Simply dropping all indexes however is not the answer (to be discussed …)
 Consider setting MBRC system stat to 128, assuming 8K block size to compensate
(1MB default storage region size in storage cell)
 Consider v$sql_plan, dba_hist_sql_plan searching for index plans with high
cardinality values
 Storage Indexes can make cost differentials even worse …


Storage Index – Basic Example
Table Index • Exadata Storage Indexes maintain summary
information about table data in memory
A B C D
– Store MIN and MAX values of columns
1 – Typically index entry for every MB of disk

3 Min B = 1 • Eliminates disk I/Os if MIN and MAX can never
Max B = 5 match “where” clause of a query
5
5 • Completely automatic and transparent
8 Min B = 3 • Example: Select count(*) from table where B = 1;
Max B =8
3 • FTS no longer really a FTS …


Storage Indexes – Demo Setup
SQL> create table big_bowie (id number not null, album_id number not null, artist_id number not null, format_id
number, release_date date, total_sales number, description varchar2(100));
Table created.

SQL> create sequence bowie_seq order;
Sequence created.

SQL> create or replace procedure pop_big_bowie as
2 begin
3 for v_album_id in 1..100 loop
4 for v_artist_id in 1..100000 loop
5 insert into big_bowie values (bowie_seq.nextval, v_album_id, v_artist_id,
ceil(dbms_random.value(0,5)) * 2,
6 trunc(sysdate-ceil(dbms_random.value(0,10000))), ceil(dbms_random.value(0,500000)),
'THE RISE AND FALL OF ZIGGY STARDUST AND THE SPIDERS FROM MARS');
7 end loop;
8 commit;
9 end loop;
10 commit;
11 end;
12 /
Procedure created.

SQL> exec pop_big_bowie
PL/SQL procedure successfully completed.


Storage Indexes – Demo Setup
SQL> update big_bowie set format_id = 3 where mod(id,10000)=0;
1000 rows updated.
SQL> commit;
Commit complete.
SQL> update big_bowie set format_id = 5 where id between 424242 and 425241;
1000 rows updated.
SQL> commit;
Commit complete.
SQL> exec dbms_stats.gather_table_stats(ownname=>user, tabname=>'BIG_BOWIE‘, method_opt=>'FOR ALL
COLUMNS SIZE 1');
SQL> select blocks from dba_tables where table_name = 'BIG_BOWIE';
BLOCKS
----------
134809

This equates to approx. 1054 MB (a “typical” OLTP table)
The DWH_BOWIE table is simply 64 times the BIG_BOWIE table


Storage Indexes – “Clustering Factor”
ALBUM_ID column data is well clustered …
SQL> select * from big_bowie where album_id = 42;
Elapsed: 00:00:00.27
Execution Plan
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 100K| 8984K| 36663 (1)| 00:07:20 |
|* 1 | TABLE ACCESS STORAGE FULL| BIG_BOWIE | 100K| 8984K| 36663 (1)| 00:07:20 |
---------------------------------------------------------------------------------------
---------------------------------------------------
1 - storage("ALBUM_ID"=42)
filter("ALBUM_ID"=42)
Statistics
----------------------------------------------------------
1 recursive calls
0 db block gets
0 redo size
0 sorts (memory)
0 sorts (disk)


SQL> select name , value/1024/1024 MB from v$statname n, v$mystat s where n.statistic# = s.statistic# and
n.name in ('cell physical IO interconnect bytes returned by smart scan', 'cell physical IO bytes saved by
storage index');

NAME MB
---------------------------------------------------------------- ----------

The Storage Index has managed to avoid having to read
approx. 99% of the table.

Having data well clustered can result in Partition-Pruning
like efficiencies when using an associated Storage Index …


Note: Conventional database indexes love well clustered data as well …
Elapsed: 00:00:01.07
Execution Plan
----------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------------------
| 1 | TABLE ACCESS BY INDEX ROWID| BIG_BOWIE | 100K| 8984K| 1550 (1)| 00:00:19 |
|* 2 | INDEX RANGE SCAN | BIG_BOWIE_ALBUM_ID_I | 100K| | 199 (1)| 00:00:03 |
----------------------------------------------------------------------------------------------------
---------------------------------------------------
2 - access("ALBUM_ID"=42)
Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
1550 physical reads
0 redo size
0 sorts (memory)
0 sorts (disk)

But is taking twice as long as FTS, despite “reported” physical reads much reduced…

Even when all the data is cached …
Elapsed: 00:00:00.27
Execution Plan
----------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------------------
| 1 | TABLE ACCESS BY INDEX ROWID| BIG_BOWIE | 100K| 8984K| 1550 (1)| 00:00:19 |
----------------------------------------------------------------------------------------------------
---------------------------------------------------
2 - access("ALBUM_ID"=42)
Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
0 physical reads
0 redo size
0 sorts (memory)
0 sorts (disk)

Index is no more efficient than FTS, even though the index is extremely efficient …

The TOTAL_SALES column however is not well clustered …
SQL> select album_id, artist_id from big_bowie where total_sales between 42 and 142;
2009 rows selected.
Elapsed: 00:00:01.25
Execution Plan
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 2040 | 26520 | 36700 (1)| 00:07:21 |
|* 1 | TABLE ACCESS STORAGE FULL| BIG_BOWIE | 2040 | 26520 | 36700 (1)| 00:07:21 |
---------------------------------------------------------------------------------------
---------------------------------------------------
1 - storage("TOTAL_SALES"<=142 AND "TOTAL_SALES">=42)
filter("TOTAL_SALES"<=142 AND "TOTAL_SALES">=42)
Statistics
----------------------------------------------------------
1 recursive calls
0 db block gets
0 redo size
0 sorts (memory)
0 sorts (disk)
2009 rows processed


SQL> select name , value/1024/1024 MB from v$statname n, v$mystat s where n.statistic# = s.statistic# and
n.name in ('cell physical IO interconnect bytes returned by smart scan', 'cell physical IO bytes saved by
storage index');

NAME MB
---------------------------------------------------------------- ----------
cell physical IO interconnect bytes returned by smart scan .383415222

The Storage Index has only managed to avoid having to read approx. 7%
of the table.
The clustering of the data can make a huge impact to the effectiveness of
Storage Indexes.
Even though this query is only reading a fraction of the data as the
previous query, it’s taking twice as long.
The smart scan though is only returning a small fraction of the overall
data back to the database.

Note: Conventional indexes also hate poorly clustered tables …
2009 rows selected.
Elapsed: 00:00:01.45
Execution Plan
-------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |Cost (%CPU)| Time |
-------------------------------------------------------------------------------------------------------
| 1 | TABLE ACCESS BY INDEX ROWID| BIG_BOWIE | 2040 | 26520 | 2048 (1)| 00:00:25 |
|* 2 | INDEX RANGE SCAN | BIG_BOWIE_TOTAL_SALES_I | 2040 | | 7 (0)| 00:00:01 |
-------------------------------------------------------------------------------------------------------
---------------------------------------------------
2 - access("TOTAL_SALES">=42 AND "TOTAL_SALES"<=142)
Statistics
----------------------------------------------------------
1 recursive calls
0 db block gets
2005 physical reads
0 redo size
0 sorts (memory)
0 sorts (disk)
2009 rows processed

This index is taking longer than the equivalent Exadata FTS …

But on subsequent re-runs, when most of the data has been cached …
2009 rows selected.
Elapsed: 00:00:00.02
Execution Plan
------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------
---------------------------------------------------
Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
0 physical reads
0 redo size
0 sorts (memory)
0 sorts (disk)
2009 rows processed

The index is now significantly faster than the FTS which performs consistently …

Database vs. Storage Indexes
Have much in common:
• Both can eliminate reading unnecessary data within
a table
• Efficiencies of both are directly impacted by the
clustering of the data within the table
• Both may need a period of “warming” up before
optimal use
• Both can combine multiple indexes to further
increase efficiencies

Storage Indexes – Warm up
On the first execution of this query since table creation:
SQL> select artist_id, total_sales from big_bowie where album_id = 42;
Elapsed: 00:00:00.89

Execution Plan
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------

---------------------------------------------------


SQL> select name , value/1024/1024 MB from v$statname n, v$mystat s where n.statistic# =
s.statistic# and n.name in ('cell physical IO interconnect bytes returned by smart scan',
'cell physical IO bytes saved by storage index');

NAME MB
---------------------------------------------------------------- ----------


Storage Indexes – Warm up
Subsequent executions, many more IO bytes saved by “warm” storage index
SQL> select artist_id, total_sales from big_bowie where album_id = 42;
Elapsed: 00:00:00.27
Execution Plan
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------
NAME MB
---------------------------------------------------------------- ----------


Database Indexes – Cache Sensitive
2009 rows selected.
Elapsed: 00:00:01.45
Execution Plan
-------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------
---------------------------------------------------
Statistics
----------------------------------------------------------
1 recursive calls
0 db block gets
2005 physical reads
0 redo size
0 sorts (memory)
0 sorts (disk)
2009 rows processed


Database Indexes – Cache Sensitive
But on subsequent re-runs, when most of the data has been cached …
2009 rows selected.
Elapsed: 00:00:00.02
Execution Plan
-------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------
---------------------------------------------------
Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
0 physical reads
0 redo size
0 sorts (memory)
0 sorts (disk)
2009 rows processed

The index is now significantly faster than previously

Exadata Smart Flash Cache
 Improves Database Index performance
 Random reads against tables and indexes are likely to have
subsequent reads and normally will be cached and have their data
subsequently delivered from the flash cache
 10x -100x better performance than disk
 Both table and index blocks cached


Storage Indexes – Combine
First, we run a query using a storage index on the ALBUM_ID column
SQL> select * from big_bowie where album_id <= 10;
Elapsed: 00:00:06.41
Execution Plan
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
|* 1 | TABLE ACCESS STORAGE FULL| BIG_BOWIE | 1009K| 88M| 36667 (1)| 00:07:21 |
---------------------------------------------------------------------------------------
---------------------------------------------------
1 - storage("ALBUM_ID"<=10)
filter("ALBUM_ID"<=10)
NAME MB
---------------------------------------------------------------- ----------


Next, we run a query using a storage index on the ARTIST_ID column
SQL> select * from big_bowie where artist_id <= 10000;
Elapsed: 00:00:06.85
Execution Plan
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
|* 1 | TABLE ACCESS STORAGE FULL| BIG_BOWIE | 1000K| 87M| 36676 (1)| 00:07:21 |
---------------------------------------------------------------------------------------
---------------------------------------------------
1 - storage("ARTIST_ID"<=10000)
filter("ARTIST_ID"<=10000)
NAME MB
---------------------------------------------------------------- ----------


Finally, we run a query using storage indexes on both columns
SQL> select * from big_bowie where album_id <= 10 and artist_id <= 10000;
Elapsed: 00:00:00.52
Execution Plan
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------
1 - storage("ARTIST_ID"<=10000 AND "ALBUM_ID"<=10)
filter("ARTIST_ID"<=10000 AND "ALBUM_ID"<=10)

NAME MB
---------------------------------------------------------------- ----------

IO bytes saved by storage indexes is now greater than previous examples

Storage Index Advantages
 Are created automatically and transparently
 Used automatically if present during Smart Scans
 Do not take up any physical storage (memory only)
 Have negligible impact on DML statements
 Effective in processing IS NULL predicates
 Might make corresponding Database indexes redundant
 As they’re used during Smart Scans, when used can return
just the rows/columns of interest to the database


Storage Index Disadvantages
 Limited to a max of 8 columns per table
 May not exist on a particular column at a point in time
 Not as “focused” (1MB storage region default vs. 1 data block)
 Has limitations, can only use Min/Max as referenced data
 Database indexes determines specific location of data while a SI can only
determine where data can’t exist
 Therefore SI can still access unnecessarily storage of no interest
 Can not cache data in database buffer cache for general re-usability
 The CBO is not aware of Storage Indexes savings
 Not as general purpose as database indexes (can’t police constraints,
avoid sorts, provide CBO statistics, etc.)

Storage Indexes: 8 Column Limit
SQL> create table radiohead (id number, col1 number, col2 number, col3 number, col4 number, col5 number, col6 number, col7
number, col8 number, col9 number, col10 number, col11 number, col12 number, some_text varchar2(50));
Table created.
SQL> insert into radiohead select rownum, mod(rownum,10), mod(rownum,100), mod(rownum,1000), mod(rownum,10000),
mod(rownum,100000), mod(rownum,1000000), ceil(dbms_random.value(0,10)), ceil(dbms_random.value(0,100)),
ceil(dbms_random.value(0,1000)), ceil(dbms_random.value(0,10000)), ceil(dbms_random.value(0,100000)),
ceil(dbms_random.value(0,1000000)), 'OK COMPUTER' from dual connect by level <=2000000;
2000000 rows created.
SQL> commit;
Commit complete.
SQL> insert/*+ append */ into radiohead select * from radiohead;
SQL> commit;
Commit complete.
SQL> commit;
Commit complete.
SQL> commit;
Commit complete.
SQL> exec dbms_stats.gather_table_stats(ownname=>user, tabname=>'RADIOHEAD', estimate_percent=>null, method_opt=> 'FOR ALL
COLUMNS SIZE 1');


SQL> select * from radiohead where id = 42;
8 rows selected.
Elapsed: 00:00:00.05

Execution Plan
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
|* 1 | TABLE ACCESS STORAGE FULL| RADIOHEAD | 8 | 416 | 42425 (1)| 00:08:30 |
---------------------------------------------------------------------------------------
---------------------------------------------------
1 - storage("ID"=121212)
filter("ID"=121212)


NAME MB
---------------------------------------------------------------- ----------

Storage index created on the ID column …


Repeat with predicates based on each of the table columns
SQL> select * from radiohead where col1=42;
no rows selected
Elapsed: 00:00:00.01
Execution Plan
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------
1 - storage("COL1"=42)
filter("COL1"=42)
NAME MB
---------------------------------------------------------------- ----------

The storage index on the COL1 column is used as well (can be useful in situations
when the required value doesn’t exist)…


A Storage index is also used on COL4 although because of the distribution of data,
the saved I/Os is limited
1600 rows selected.
Elapsed: 00:00:00.68
Execution Plan
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------
filter("COL4"=42)
NAME MB
---------------------------------------------------------------- ----------
cell physical IO bytes saved by storage index 2612.64063 <= + 66MB


A Storage index is also used on, COL5, COL6, COL7, Col8 and COL9 making a
total of 8 storage indexes defined on this table
no rows selected
Elapsed: 00:00:00.02
Execution Plan
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 15984 | 811K| 42561 (1)| 00:08:31 |
|* 1 | TABLE ACCESS STORAGE FULL| RADIOHEAD | 15984 | 811K| 42561 (1)| 00:08:31 |
---------------------------------------------------------------------------------------
---------------------------------------------------
filter("COL9"=0)
NAME MB
---------------------------------------------------------------- ----------


However although a smart scan is performed, a viable storage index is not now
created on COL10 column with 8 existing storage indexes
1536 rows selected.
Elapsed: 00:00:00.73
Execution Plan
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------
filter("COL10"=42)
NAME MB
---------------------------------------------------------------- ----------
cell physical IO bytes saved by storage index 8792.94531  No change


On this much more selective query on COL12, a new storage index is created
8 rows selected.
Elapsed: 00:00:00.39

Execution Plan
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------

---------------------------------------------------

filter("COL12"=42)


NAME MB
---------------------------------------------------------------- ----------


If we go back to the first query on the ID column, the storage index is still used …
SQL> select * from radiohead where id=42;
8 rows selected.
Elapsed: 00:00:00.05
Execution Plan
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------
1 - storage("ID"=42)
filter("ID"=42)
NAME MB
---------------------------------------------------------------- ----------


But if we go back to the not so selective query on the COL4 column, the storage
index is no longer used as it was previously …
SQL> select * from radiohead where col4 = 4242;
1600 rows selected.
Elapsed: 00:00:00.73
Execution Plan
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------
filter("COL4"=4242)
NAME MB
---------------------------------------------------------------- ----------
cell physical IO bytes saved by storage index 10726.9844  No Change !!

The Storage Index on COL4 is replaced by the Storage Index on COL12 …

“Hidden” Values of Interest
Returning to the BIG_BOWIE table, the FORMAT_ID column has values 2 – 10, but
only a handful with a value of 3 and 5 …
SQL> update big_bowie set format_id = 3 where mod(id,10000)=0;
1000 rows updated.

SQL> commit;
Commit complete.

SQL> update big_bowie set format_id = 5 where id between 424242 and 425241;
1000 rows updated.

SQL> commit;
Commit complete.

SQL> create index big_bowie_format_id_i on big_bowie(format_id);
Index created.

SQL> exec dbms_stats.gather_table_stats(ownname=>user, tabname=>'BIG_BOWIE',
method_opt=> 'FOR COLUMNS FORMAT_ID SIZE 10');



With a histogram in place, the CBO uses an index very effectively …
SQL> select album_id, total_sales from big_bowie where format_id = 3;
1000 rows selected.
Elapsed: 00:00:00.01
Execution Plan
-----------------------------------------------------------------------------------------------------
-----------------------------------------------------------------------------------------------------
|* 2 | INDEX RANGE SCAN | BIG_BOWIE_FORMAT_ID_I | 1000 | | 4 (0)| 00:00:01 |
-----------------------------------------------------------------------------------------------------
---------------------------------------------------
2 - access("FORMAT_ID"=3)
Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
0 physical reads
0 redo size
0 sorts (memory)
0 sorts (disk)
1000 rows processed


Whereas value 3 was littered throughout the table, values 5 are all clustered
together, making the index even more effective …
1000 rows selected.
Elapsed: 00:00:00.00
Execution Plan
-----------------------------------------------------------------------------------------------------
-----------------------------------------------------------------------------------------------------
|* 2 | INDEX RANGE SCAN | BIG_BOWIE_FORMAT_ID_I | 1000 | | 4 (0)| 00:00:01 |
-----------------------------------------------------------------------------------------------------
---------------------------------------------------
2 - access("FORMAT_ID"=5)
Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
154 consistent gets
0 physical reads
0 redo size
0 sorts (memory)
0 sorts (disk)
1000 rows processed


Smart Scan performed but a storage index is totally ineffective as the required values
are “hidden” between the min/max (2 – 10) range found in most 1MB storage regions
SQL> alter index big_bowie_format_id_i invisible;
Index altered.
1000 rows selected.
Elapsed: 00:00:00.80
Execution Plan
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------
1 - storage("FORMAT_ID"=3)
filter("FORMAT_ID"=3)
NAME MB
---------------------------------------------------------------- ----------
cell physical IO bytes saved by storage index 0


In both cases, Storage Index is not used with Database index considerably faster …
1000 rows selected.
Elapsed: 00:00:00.89

Execution Plan
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------
1 - storage("FORMAT_ID"=5)
filter("FORMAT_ID"=5)

NAME MB
---------------------------------------------------------------- ----------


OLTP Environments
 In OLTP environments, most queries and processes are
based on highly selective data where database indexes are
more viable
 Indexes used for more than just “selecting” data, for example:
– Police Primary and Unique Key constraints
– Reduce locking implications with Foreign Keys
– Reduce sorting requirements
– Reduce function-based processing
– Provide additional selectivity data to the CBO


Storage Indexes - Unique Scan
With a storage index in place, a FTS is no longer really a FTS …
SQL> select album_id, artist_id from big_bowie where id = 42;
Elapsed: 00:00:00.02
Execution Plan
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------
1 - storage("ID"=42)
filter("ID"=42)
Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
0 redo size
0 sorts (memory)
0 sorts (disk)
1 rows processed
NAME MB
---------------------------------------------------------------- ----------


Unique Scan
However, an index is even faster and potentially far more scalable …
SQL> select album_id, artist_id from big_bowie where id = 42;
Elapsed: 00:00:00.00
Execution Plan
--------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------
|* 2 | INDEX UNIQUE SCAN | BIG_BOWIE_PK | 1 | | 2 (0)| 00:00:01 |
--------------------------------------------------------------------------------------------
---------------------------------------------------
2 - access("ID"=42)
Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
4 consistent gets
0 physical reads
0 redo size
0 sorts (memory)
0 sorts (disk)
1 rows processed


Primary and Unique Keys
Then there’s the issue of removing an index that enforces either a Primary or Unique
Key constraint …
SQL> drop index big_bowie_pk;
drop index big_bowie_pk
*
ERROR at line 1:
ORA-02429: cannot drop index used for enforcement of unique/primary key

SQL> alter table big_bowie drop primary key;

Table altered

A Storage Index can not be used to enforce constraints.


Foreign Key Constraints
Indexes can be critical in preventing locking issues with regard to Foreign Key
Constraints…
SQL> create table albums (id number, name varchar2(50));
Table created.
SQL> insert into albums select rownum, table_name from dba_tables where rownum <=101;
101 rows created.
SQL> commit;
Commit complete.
SQL> alter table albums add constraint albums_pk primary key (id);
Table altered.
SQL> exec dbms_stats.gather_table_stats(ownname=>user, tabname=>'ALBUMS', estimate_percent=>null,
method_opt=> 'FOR ALL COLUMNS SIZE 1');
SQL> alter table big_bowie add constraint albums_fk foreign key (album_id) referencing albums(id);
Table altered.


In Session 1 (no commit)
SQL> insert into big_bowie values (10000001, 1, 1, 1, sysdate, 1, 'TEST');

1 row created.

In Session 2 (hangs !!)
SQL> delete albums where id = 101;

In Session 3 (hangs !!)
SQL> insert into big_bowie values (10000002, 1,1,1,sysdate,1,'TEST2');

A Storage Index can not prevent Foreign Key locking related issues …


In Session 1 (no commit)
SQL> create index big_bowie_album_id_i on big_bowie(album_id);

Index created.

SQL> insert into big_bowie values (10000001, 1, 1, 1, sysdate, 1, 'TEST');

1 row created.

In Session 2 (no longer hangs)
SQL> delete albums where id = 101;

1 row deleted.

In Session 3 (no longer hangs)
SQL> insert into big_bowie values (10000002, 1,1,1,sysdate,1,'TEST2');

1 row created.


Database Indexes and Sorts
Database indexes return data in a sorted manner and so can avoid expensive sort
operations …
SQL> select * from big_bowie where album_id between 1 and 50 order by album_id;
Elapsed: 00:00:13.08
Execution Plan
----------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------------------
| 1 | TABLE ACCESS BY INDEX ROWID| BIG_BOWIE | 5049K| 443M| 79536 (1)| 00:15:55 |
----------------------------------------------------------------------------------------------------
---------------------------------------------------
2 - access("ALBUM_ID">=1 AND "ALBUM_ID"<=50)
Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
0 physical reads
0 redo size
0 sorts (memory)
0 sorts (disk)


Database Indexes and Sorts
SQL> alter index big_bowie_album_id_i invisible;
Index altered.
SQL> select * from big_bowie where album_id between 1 and 50 order by album_id;
Elapsed: 00:00:18.89
Execution Plan
------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time |
------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 5049K| 443M| | 143K (1)| 00:28:48 |
| 1 | SORT ORDER BY | | 5049K| 443M| 547M| 143K (1)| 00:28:48 |
|* 2 | TABLE ACCESS STORAGE FULL| BIG_BOWIE | 5049K| 443M| | 36855 (1)| 00:07:23 |
------------------------------------------------------------------------------------------------
---------------------------------------------------
2 - storage("ALBUM_ID"<=50 AND "ALBUM_ID">=1)
filter("ALBUM_ID"<=50 AND "ALBUM_ID">=1)
Statistics
----------------------------------------------------------
501 recursive calls
10 db block gets
0 redo size
0 sorts (memory)
1 sorts (disk)
NAME MB
---------------------------------------------------------------- ----------


NULLs and Storage Indexes
Storage Indexes include NULL flag to determine which storage regions contain NULLs
SQL> select count(*) from big_bowie where total_sales is null;
COUNT(*)
----------
0
Elapsed: 00:00:00.04
Execution Plan
----------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------
| 1 | SORT AGGREGATE | | 1 | 5 | | |
----------------------------------------------------------------------------------------
---------------------------------------------------
2 - storage("TOTAL_SALES" IS NULL)
filter("TOTAL_SALES" IS NULL)
NAME MB
---------------------------------------------------------------- ----------
cell physical IO interconnect bytes returned by smart scan 0


Offloading Functions
SQL> select id, album_id, mod(artist_id, 10) from dwh_bowie where album_id = 42;
Elapsed: 00:00:11.39
Execution Plan
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------
Statistics
----------------------------------------------------------
1 recursive calls
0 db block gets
0 redo size
0 sorts (memory)
0 sorts (disk)
NAME MB
---------------------------------------------------------------- ----------


Offloading Functions
Processing of many function calls can also be offloaded to the
storage cells.
To see the full list, simple run :
SQL> select name from v$sqlfn_metadata where offloadable = 'YES' order by name;

However, you can not base a storage index on a function;
Function-Based Storage Indexes are not supported …

Additionally, many functions can not currently be offloaded …


Function-Based Indexes
SQL> select id, album_id, mod(artist_id, 10) from dwh_bowie where trunc(album_id)=42;
Elapsed: 00:00:28.71
Execution Plan
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------
1 - storage(TRUNC("ALBUM_ID")=42)
filter(TRUNC("ALBUM_ID")=42)
Statistics
----------------------------------------------------------
1 recursive calls
0 db block gets
0 redo size
0 sorts (memory)
0 sorts (disk)
NAME MB
---------------------------------------------------------------- ----------
cell physical IO bytes saved by storage index 44460.3828 <= no change !!


Min/Mix Example
SQL> select min(id) from dwh_bowie;
MIN(ID)
----------
1
Elapsed: 00:00:41.31
Execution Plan
----------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 6 | 2345K (1)| 07:49:12 |
| 1 | SORT AGGREGATE | | 1 | 6 | | |
| 2 | TABLE ACCESS STORAGE FULL| DWH_BOWIE | 640M| 3662M| 2345K (1)| 07:49:12 |
----------------------------------------------------------------------------------------
Statist5ics
----------------------------------------------------------
0 recursive calls
0 db block gets
0 redo size
0 sorts (memory)
0 sorts (disk)
1 rows processed
NAME MB
---------------------------------------------------------------- ----------


Min/Mix Example
SQL> create index dwh_bowie_id_i on dwh_bowie(id);
Index created.
SQL> select min(id) from dwh_bowie;
MIN(ID)
----------
1
Elapsed: 00:00:00.01
Execution Plan
---------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------
| 1 | SORT AGGREGATE | | 1 | 6 | | |
| 2 | INDEX FULL SCAN (MIN/MAX)| DWH_BOWIE_ID_I | 1 | 6 | 4 (0)| 00:00:01 |
---------------------------------------------------------------------------------------------
Statistics
----------------------------------------------------------
1 recursive calls
0 db block gets
4 consistent gets
3 physical reads
0 redo size
0 sorts (memory)
0 sorts (disk)
1 rows processed


Smart Scans and Serial DML
SQL> update dwh_bowie set description = 'ZIGGY' where total_sales = 42;
1472 rows updated.
Elapsed: 00:15:00.38
Execution Plan
----------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------
| 0 | UPDATE STATEMENT | | 1264 | 84688 | 2350K (1)| 07:50:07 |
| 1 | UPDATE | DWH_BOWIE | | | | |
|* 2 | TABLE ACCESS STORAGE FULL| DWH_BOWIE | 1264 | 84688 | 2350K (1)| 07:50:07 |
----------------------------------------------------------------------------------------
---------------------------------------------------
2 - storage("TOTAL_SALES"=42)
filter("TOTAL_SALES"=42)
Statistics
----------------------------------------------------------
8 recursive calls
1525 db block gets
485364 redo size
2 sorts (memory)
0 sorts (disk)
1472 rows processed
NAME MB
---------------------------------------------------------------- ----------


Smart Scans and Parallel DML
SQL> update /*+ parallel */ dwh_bowie set description = 'ZIGGY' where total_sales = 42;
1472 rows updated.
Elapsed: 00:00:18.26
Execution Plan
------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | TQ |IN-OUT| PQ Distrib |
------------------------------------------------------------------------------------------------------------------------
| 0 | UPDATE STATEMENT | | 1264 | 84688 | 27171 (1)| 00:00:55 | | | |
| 1 | UPDATE | DWH_BOWIE | | | | | | | |
| 2 | PX COORDINATOR | | | | | | | | |
| 3 | PX SEND QC (RANDOM) | :TQ10000 | 1264 | 84688 | 27171 (1)| 00:00:55 | Q1,00 | P->S | QC (RAND) |
| 4 | PX BLOCK ITERATOR | | 1264 | 84688 | 27171 (1)| 00:00:55 | Q1,00 | PCWC | |
|* 5 | TABLE ACCESS STORAGE FULL| DWH_BOWIE | 1264 | 84688 | 27171 (1)| 00:00:55 | Q1,00 | PCWP | |
------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------
5 - storage("TOTAL_SALES"=42)
filter("TOTAL_SALES"=42)
Statistics
----------------------------------------------------------
336 recursive calls
1525 db block gets
567008 redo size
1 sorts (memory)
0 sorts (disk)
1472 rows processed
NAME MB
---------------------------------------------------------------- ----------


Indexes and DML
An Index can still be the best and most scalable option with selective DML …
SQL> create index dwh_bowie_total_sales_i on dwh_bowie(total_sales);
Index created.
SQL> update dwh_bowie set description = 'ZIGGY' where total_sales = 42;
1472 rows updated.
Elapsed: 00:00:00.92
Execution Plan
---------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------
| 0 | UPDATE STATEMENT | | 1264 | 84688 | 1270 (0)| 00:00:16 |
| 1 | UPDATE | DWH_BOWIE | | | | |
|* 2 | INDEX RANGE SCAN| DWH_BOWIE_TOTAL_SALES_I | 1264 | 84688 | 6 (0)| 00:00:01 |
---------------------------------------------------------------------------------------------
---------------------------------------------------
2 - access("TOTAL_SALES"=42)
Statistics
----------------------------------------------------------
0 recursive calls
1523 db block gets
7 consistent gets
0 physical reads
508060 redo size
1 sorts (memory)
0 sorts (disk)
1472 rows processed


Data Warehouse Environments
 Constraint management may not be such an issue
 Typically, queries can be large, expensive affairs
 Use of Full Table/Index Scans more prevalent
 Smart Scans and Storage Indexes more viable
 Good opportunity to drop unnecessary indexes
 However, InfiniBand is not Infinite-Band
 However, overall load and scalability still a consideration
 However, database indexes can still be effective


DWH Join Example – Serial Execution
SQL> select * from dwh_bowie d, albums a, formats f where d.album_id = a.id and d.format_id=f.id
and a.name in ('TAB$', 'COL$') and f.name = 'I_USER#';
Elapsed: 00:02:50.05
Execution Plan
------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)|Time |
------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1600K| 177M| 2352K (1)|07:50:33 |
|* 1 | HASH JOIN | | 1600K| 177M| 2352K (1)|07:50:33 |
| 2 | MERGE JOIN CARTESIAN | | 2 | 48 | 36 (0)|00:00:01 |
|* 3 | TABLE ACCESS STORAGE FULL | FORMATS | 1 | 11 | 18 (0)|00:00:01 |
| 4 | BUFFER SORT | | 2 | 26 | 18 (0)|00:00:01 |
|* 5 | TABLE ACCESS STORAGE FULL| ALBUMS | 2 | 26 | 18 (0)|00:00:01 |
| 6 | TABLE ACCESS STORAGE FULL | DWH_BOWIE | 640M| 54G| 2349K (1)|07:49:56 |
------------------------------------------------------------------------------------------
---------------------------------------------------
1 - access("D"."ALBUM_ID"="A"."ID" AND "D"."FORMAT_ID"="F"."ID")
3 - storage("F"."NAME"='I_USER#')
filter("F"."NAME"='I_USER#')
5 - storage("A"."NAME"='COL$' OR "A"."NAME"='TAB$')
filter("A"."NAME"='COL$' OR "A"."NAME"='TAB$')
NAME MB
---------------------------------------------------------------- ----------


Join Bloom Filter
 A bloom filter is a bitmap like structure (bit vector) that determines
whether a value belongs to a given set or not
 Used since Oracle10g R2 in parallel join processing and since
Oracle11g R1 in join-filter processing
 On Exadata the bloom filter is passed as an additional predicate so it
can be overloaded to the storage cells making bloom filtering very
efficient
 Following Parallel Query example highlights the potential efficiencies
with the use of bloom filtering
 For an excellent article on Bloom Filters, see Christian Antognini’s
article:
– http://www.antognini.ch/papers/BloomFilters20080620.pdf


DWH Example – Parallel Execution
SQL> select /*+ parallel */ * from dwh_bowie d, albums a, formats f where d.album_id = a.id and d.format_id=f.id and a.name in
('TAB$', 'COL$') and f.name = 'I_USER#';
Elapsed: 00:00:03.34
------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | TQ |IN-OUT| PQ Distrib |
------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1600K| 177M| 27198 (1)| 00:00:55 | | | |
| 1 | PX COORDINATOR | | | | | | | | |
| 2 | PX SEND QC (RANDOM) | :TQ10002 | 1600K| 177M| 27198 (1)| 00:00:55 | Q1,02 | P->S | QC (RAND) |
|* 3 | HASH JOIN | | 1600K| 177M| 27198 (1)| 00:00:55 | Q1,02 | PCWP | |
| 4 | PX RECEIVE | | 2 | 48 | 5 (0)| 00:00:01 | Q1,02 | PCWP | |
| 5 | PX SEND BROADCAST | :TQ10001 | 2 | 48 | 5 (0)| 00:00:01 | Q1,01 | P->P | BROADCAST |
| 6 | MERGE JOIN CARTESIAN | | 2 | 48 | 5 (0)| 00:00:01 | Q1,01 | PCWP | |
| 7 | BUFFER SORT | | | | | | Q1,01 | PCWC | |
| 8 | PX RECEIVE | | 1 | 11 | 2 (0)| 00:00:01 | Q1,01 | PCWP | |
| 9 | PX SEND BROADCAST | :TQ10000 | 1 | 11 | 2 (0)| 00:00:01 | Q1,00 | P->P | BROADCAST |
|* 11 | TABLE ACCESS STORAGE FULL| FORMATS | 1 | 11 | 2 (0)| 00:00:01 | Q1,00 | PCWP | |
| 12 | BUFFER SORT | | 2 | 26 | 3 (0)| 00:00:01 | Q1,01 | PCWP | |
|* 14 | TABLE ACCESS STORAGE FULL | ALBUMS | 2 | 26 | 2 (0)| 00:00:01 | Q1,01 | PCWP | |
| 15 | PX BLOCK ITERATOR | | 640M| 54G| 27161 (1)| 00:00:55 | Q1,02 | PCWC | |
|* 16 | TABLE ACCESS STORAGE FULL | DWH_BOWIE | 640M| 54G| 27161 (1)| 00:00:55 | Q1,02 | PCWP | |
------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------
3 - access("D"."ALBUM_ID"="A"."ID" AND "D"."FORMAT_ID"="F"."ID")
11 - storage("F"."NAME"='I_USER#')
filter("F"."NAME"='I_USER#')
14 - storage("A"."NAME"='COL$' OR "A"."NAME"='TAB$')
filter("A"."NAME"='COL$' OR "A"."NAME"='TAB$')
16 - storage(SYS_OP_BLOOM_FILTER(:BF0000,"D"."ALBUM_ID","D"."FORMAT_ID"))
filter(SYS_OP_BLOOM_FILTER(:BF0000,"D"."ALBUM_ID","D"."FORMAT_ID"))
NAME MB
---------------------------------------------------------------- ----------


DWH Example – Star Transformation
If we now make relevant Bitmap Indexes visible again
SQL> alter index dwh_bowie_album_id_i visible;
Index altered.

SQL> alter index dwh_bowie_format_i visible;
Index altered.

SQL> select * from dwh_bowie d, albums a, formats f where d.album_id = a.id and
d.format_id=f.id and a.name in ('TAB$', 'COL$') and f.name = 'I_USER#';


Elapsed: 00:00:00.34

In some scenarios, indexes still have a place in DWH environments as well. It all
depends …


DWH Example – Star Transformation
Execution Plan
---------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes| Cost (%CPU)| Time |
---------------------------------------------------------------------------------------------------------
|* 1 | HASH JOIN | | 198K| 28M| 319K (1)| 01:04:00 |
| 2 | MERGE JOIN CARTESIAN | | 2 | 48| 36 (0)| 00:00:01 |
|* 3 | TABLE ACCESS STORAGE FULL | FORMATS | 1 | 11| 18 (0)| 00:00:01 |
| 4 | BUFFER SORT | | 2 | 26| 18 (0)| 00:00:01 |
|* 5 | TABLE ACCESS STORAGE FULL | ALBUMS | 2 | 26| 18 (0)| 00:00:01 |
| 6 | VIEW | VW_ST_4B85F3B9 | 1584K| 190M| 319K (1)| 01:03:59 |
| 7 | NESTED LOOPS | | 1584K| 172M| 319K (1)| 01:03:59 |
| 8 | BITMAP CONVERSION TO ROWIDS | | 1584K| 27M| 275 (2)| 00:00:04 |
| 9 | BITMAP AND | | | | | |
| 10 | BITMAP MERGE | | | | | |
| 11 | BITMAP KEY ITERATION | | | | | |
|* 12 | TABLE ACCESS STORAGE FULL| ALBUMS | 2 | 26| 18 (0)| 00:00:01 |
|* 13 | BITMAP INDEX RANGE SCAN | DWH_BOWIE_ALBUM_ID_I | | | | |
| 14 | BITMAP MERGE | | | | | |
| 15 | BITMAP KEY ITERATION | | | | | |
|* 16 | TABLE ACCESS STORAGE FULL| FORMATS | 1 | 11| 18 (0)| 00:00:01 |
|* 17 | BITMAP INDEX RANGE SCAN | DWH_BOWIE_FORMAT_I | | | | |
| 18 | TABLE ACCESS BY USER ROWID | DWH_BOWIE | 1 | 96| 319K (1)| 01:03:56 |
---------------------------------------------------------------------------------------------------------
Note
-----
- star transformation used for this statement
Statistics
----------------------------------------------------------
1 recursive calls
0 db block gets
0 physical reads
476 redo size
1 sorts (memory)
0 sorts (disk)


Indexes – Skinny Tables
SQL> select blocks from dba_tables where table_name = 'DWH_BOWIE';
BLOCKS
----------
8634533
SQL> select count(*) from dwh_bowie;
COUNT(*)
----------
640000128
Elapsed: 00:00:40.31
Execution Plan
--------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Cost (%CPU)| Time |
--------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 2345K (1)| 07:49:12 |
| 1 | SORT AGGREGATE | | 1 | | |
| 2 | TABLE ACCESS STORAGE FULL| DWH_BOWIE | 640M| 2345K (1)| 07:49:12 |
--------------------------------------------------------------------------------
Statistics
----------------------------------------------------------
63 recursive calls
0 db block gets
0 redo size
6 sorts (memory)
0 sorts (disk)
1 rows processed
NAME MB
---------------------------------------------------------------- ----------


Indexes – Skinny Tables
SQL> create index dwh_bowie_album_id_i on dwh_bowie(album_id);
Index created.
SQL> select leaf_blocks from user_indexes where index_name = 'DWH_BOWIE_ALBUM_ID_I';
LEAF_BLOCKS
-----------
1250001
Elapsed: 00:00:20.30
Execution Plan
----------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 342K (2)| 01:08:32 |
| 2 | INDEX STORAGE FAST FULL SCAN| DWH_BOWIE_ALBUM_ID_I | 640M| 342K (2)| 01:08:32 |
----------------------------------------------------------------------------------------------
Statistics
----------------------------------------------------------
1 recursive calls
0 db block gets
0 redo size
0 sorts (memory)
0 sorts (disk)
1 rows processed
NAME MB
---------------------------------------------------------------- ----------


Bitmap Indexes – Really Skinny Tables
SQL> create bitmap index dwh_bowie_album_id_i on dwh_bowie(album_id);
Index created.
SQL> select leaf_blocks from user_indexes where index_name = 'DWH_BOWIE_ALBUM_ID_I';
LEAF_BLOCKS
-----------
16086
Elapsed: 00:00:00.22
Execution Plan
------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 14482 (1)| 00:02:54 |
| 2 | BITMAP CONVERSION COUNT | | 640M| 14482 (1)| 00:02:54 |
| 3 | BITMAP INDEX STORAGE FAST FULL SCAN| DWH_BOWIE_ALBUM_ID_I | | | |
------------------------------------------------------------------------------------------------------
Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
0 physical reads
0 redo size
0 sorts (memory)
0 sorts (disk)
1 rows processed
NAME MB
---------------------------------------------------------------- ----------


Hybrid Columnar Compression (HCC)
SQL> create table tiny_dwh_bowie compress for query high as select * from dwh_bowie where 1=2;
Table created.
SQL> insert /*+ append */ into tiny_dwh_bowie select * from dwh_bowie;
SQL> commit;
Commit complete.
SQL> exec dbms_stats.gather_table_stats(ownname=>user, tabname=>'TINY_DWH_BOWIE', estimate_percent=>null,
method_opt=> 'FOR ALL COLUMNS SIZE 1');
SQL> create table tiny_dwh_bowie2 compress for archive high as select * from dwh_bowie where 1=2;
Table created.
SQL> insert /*+ append */ into tiny_dwh_bowie2 select * from tiny_dwh_bowie;
SQL> commit;
Commit complete.
SQL> select table_name, blocks, round((blocks*8)/1024) MB, compress_for from user_tables where table_name like
'%DWH_BOWIE%';
TABLE_NAME BLOCKS MB COMPRESS_FOR
------------------------------ ---------- ---------- ------------
DWH_BOWIE 8634533 67457
TINY_DWH_BOWIE 521645 4075 QUERY HIGH
TINY_DWH_BOWIE2 403379 3151 ARCHIVE HIGH


HCC and Smart Scans
SQL> select * from dwh_bowie where album_id = 42;
Elapsed: 00:00:46.61
Execution Plan
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
SQL> select * from tiny_dwh_bowie where album_id = 42;
Elapsed: 00:00:33.54
Execution Plan
--------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------
|* 1 | TABLE ACCESS STORAGE FULL| TINY_DWH_BOWIE | 6400K| 561M| 148K (5)| 00:29:38 |
--------------------------------------------------------------------------------------------
SQL> select * from tiny_dwh_bowie2 where album_id = 42;
Elapsed: 00:00:41.04
Execution Plan
---------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------
|* 1 | TABLE ACCESS STORAGE FULL| TINY_DWH_BOWIE2 | 6400K| 561M| 116K (6)| 00:23:13 |
---------------------------------------------------------------------------------------------


HCC and Indexes
If we create indexes in all three tables …
SQL> create index dwh_bowie_album_i on dwh_bowie(album_id);
Index created.
SQL> create index tiny_bowie_album_i on tiny_dwh_bowie(album_id);
Index created.
SQL> create index tiny_bowie2_album_i on tiny_dwh_bowie2(album_id);
Index created.
SQL> select index_name, leaf_blocks, clustering_factor from user_indexes where index_name
like '%BOWIE%ALBUM%';
INDEX_NAME LEAF_BLOCKS CLUSTERING_FACTOR
------------------------------ ----------- -----------------
DWH_BOWIE_ALBUM_I 1250001 8876879
TINY_BOWIE_ALBUM_I 1250001 224887
TINY_BOWIE2_ALBUM_I 1250001 84549

All are the same size but have vastly differing Clustering Factors


HCC and Indexes
If we create indexes in all three tables and re-run with fully cached data …
SQL> select * from dwh_bowie where album_id = 42;
Elapsed: 00:00:16.96
Execution Plan
-------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------
| 1 | TABLE ACCESS BY INDEX ROWID| DWH_BOWIE | 6400K| 561M| 101K (1)| 00:20:18 |
|* 2 | INDEX RANGE SCAN | DWH_BOWIE_ALBUM_I | 6400K| | 12569 (1)| 00:02:31 |
-------------------------------------------------------------------------------------------------
SQL> select * from tiny_dwh_bowie where album_id = 42;
Elapsed: 00:00:21.16
Execution Plan
--------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost(%CPU)| Time |
--------------------------------------------------------------------------------------------------
| 1 | TABLE ACCESS BY INDEX ROWID| TINY_DWH_BOWIE | 6400K| 561M| 14900 (1)| 00:02:59 |
|* 2 | INDEX RANGE SCAN | TINY_BOWIE_ALBUM_I | 6400K| | 12569 (1)| 00:02:31 |
--------------------------------------------------------------------------------------------------
SQL> select * from tiny_dwh_bowie2 where album_id = 42;
Elapsed: 00:01:08.38
Execution Plan
---------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------
| 1 | TABLE ACCESS BY INDEX ROWID| TINY_DWH_BOWIE2 | 6400K| 561M| 13497 (2)| 00:02:42 |
|* 2 | INDEX RANGE SCAN | TINY_BOWIE2_ALBUM_I | 6400K| | 12569 (1)| 00:02:31 |
---------------------------------------------------------------------------------------------------


HCC and Indexes
 Index CBO costs can decrease due to decrease in CF
 Actual costs increase however when accessing HCC tables as
the required data needs to be reconstructed/decompressed
 Especially so for COMPRESS FOR ARCHIVE
 This extra work burns significant CPU
 The CBO however does not consider these additional CPU
costs in its calculations
 Indexes therefore more likely to be inappropriately selected by
CBO with HCC

Exadata Smart Scans
Operations that don’t benefit from Smart Scans include:
 Index range scans.
 Scans of IOTs or Clustered Tables.
 Access to a compressed index.
 Access to a reverse key index.


Index Organized Tables: No Smart Scan
SQL> create table big_bowie_iot (id number constraint big_bowie_iot_pk primary key, album_id number, artist_id
number, format_id number, release_date date, total_sales number, description varchar2(100)) organization index;
Table created.
SQL> insert into big_bowie_iot select * from big_bowie;
SQL> commit;
Commit complete.
SQL> select * from big_bowie_iot where total_sales = 42;
23 rows selected.
Elapsed: 00:00:16.72
Execution Plan
-----------------------------------------------------------------------------------------
-----------------------------------------------------------------------------------------
|* 1 | INDEX FAST FULL SCAN| BIG_BOWIE_IOT_PK | 20 | 1840 | 36698 (1)| 00:07:21 |
-----------------------------------------------------------------------------------------
---------------------------------------------------
1 - filter("TOTAL_SALES"=42)
Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
0 redo size
0 sorts (memory)
0 sorts (disk)
23 rows processed


General Conclusions
 With Exadata, things run faster, FTS especially so
 Some indexes can become redundant as a result
 CBO not fully aware of reduced FTS costs
 CBO not fully aware of increased index costs associated with HCC
 With indexes in place, faster smart scans operations may be dismissed
 However, using an index should generally be no worse than in Non-
Exadata environments (although HCC an exception)
 Database indexes still of value in numerous scenarios, especially in
OLTP environments
 Need to understand benefits/advantages of smart scans and storage
indexes and where there may have shortfalls

General Recommendations
 Easy to create an index, much harder and riskier to subsequently drop them
 When moving to Exadata, take the opportunity to review current indexes,
especially in DWH environments which are often “over indexed”
 Use information in v$sql_plan, dba_hist_sql_plan, AWR reports to
determine potentially problematic indexes
 But do not blindly drop indexes in the hope Exadata will just cope
regardless …
 Use Invisible Indexes to test impact of potentially dropping an index
 Real Application Testing (RAT) of enormous use when migrating to Exadata


richardfoote.wordpress.com
An Oracle Indexing blog might still have some value yet !!!


Indexing in Exadata

More Related Content

What's hot (20)

Similar to Indexing in Exadata (20)

More from Enkitec (20)

Recently uploaded (20)

Indexing in Exadata