ILOUG 2019 - Autonomous, what does it mean for DBAs

1/29/2019
1
Copyright © 2018, Oracle and/or its affiliates. All rights reserved.
Connor McDonald
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
2
1
2

1/29/2019
2
3
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Stuff
youtube bit.ly/youtube-connor
blog bit.ly/blog-connor
twitter bit.ly/twitter-connor
400+ posts mainly on database & development
250 technical videos, new uploads every week
rants and raves on tech and the world :-)
3
4

1/29/2019
3
Copyright © 2018, Oracle and/or its affiliates. All rights reserved.
etc...
facebook bit.ly/facebook-connor
linkedin bit.ly/linkedin-connor
instagram bit.ly/instagram-connor
slideshare bit.ly/slideshare-connor
https://asktom.oracle.com
5
6

1/29/2019
4
7https://asktom.oracle.com/officehours
150 hours of free access (so far)
8
7
8

1/29/2019
5
18c - The Autonomous Database
Connor McDonald
Developer Advocate
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 10
18c - The Autonomous Database
9
10

1/29/2019
6
"18c - the autonomous database"
11
12c, 13, 14, 15, 16, 17, 18c
12
?
11
12

1/29/2019
7
18c = 12.2.0.2
13
MOS 2285040.1
No “FIRST” and “SECOND” releases anymore!
13
14

1/29/2019
8
Fewer One-Off Patches!
3 Number Format
Year.Update.Revision
15
16

1/29/2019
9
Version number reflects patch level!
A Release Update or Revision is only a patch,
not a database upgrade!
17
18

1/29/2019
10
Patch: Oracle 18.2.1 ⇢ Oracle 18.6.0
opatch
Upgrade: Oracle 18.2.1 ⇢ Oracle 19.1.0
dbupgrade and DBUA
18.1.0
18.2.0
First
Update
18.1.1
First
Revision
security,
regression,
no functionality changes
no optimizer changes
security,
regression,
functional changes
optimizer changes
19
20

1/29/2019
11
"18c - the autonomous database"
21
autonomous is a cloud offering
22
21
22

1/29/2019
12
Autonomous
Database
= + +
so what is the big deal ?
24
23
24

1/29/2019
13
trip down memory lane #1
25
who remembers these numbers ?
26
25
26

1/29/2019
14
121
249
505
27
ORA-1631: max # extents 505 reached in table SCOTT.EMP
121 = 2k block size
249 = 4k block size
27
28

1/29/2019
15
who thinks about extents anymore?
29
7.3 = unlimited extents
30
semi-automatic
uet$, fet$
29
30

1/29/2019
16
8.1 = locally managed tablespaces
31
fully automatic
bitmap extents
32
31
32

1/29/2019
17
create table SALES (
...
...
...
)
freelists 4
freelist groups 1
ASSM
34
33
34

1/29/2019
18
automatic segment space management
35
automation =
36
"we know the implementation, and
it is being performed by the software
based on our rules"
35
36

1/29/2019
19
automation ... towards autonomy
37
38
37
38

1/29/2019
20
who remembers these parameters ?
39
ROLLBACK_SEGMENTS = (r0,r1,r2,r3,r4,r5,r6)
TRANSACTIONS_PER_ROLLBACK_SEGMENT = 20
CLEANUP_ROLLBACK_ENTRIES = 10
SYSTEM non-SYSTEM
39
40

1/29/2019
21
who thinks about undo anymore?
41
9.0 = automatic undo
42
undo_tablespace =
41
42

1/29/2019
22
but it's more than automation
SQL> select count(*) from dba_rollback_segs;
COUNT(*)
----------
????
43
44

1/29/2019
23
segment count growth / reduction ...
... decided by the database
45
46

1/29/2019
24
autonomy =
47
"we do not know the implementation,
we do not set the rules,
decisions by the software"
Oracle has been doing this for a while :-)
47
48

1/29/2019
25
Oracle Database 10g and below
• Automatic Storage Management (ASM)
• Automatic Memory Management
• Automatic DB Diagnostic Monitor (ADDM)
• Automatic Workload Repository (AWR)
• Automatic Undo tablespaces
• Automatic Segment Space Management
• Automatic Statistics Gathering
• Automatic Standby Management (Broker)
• Automatic Query Rewrite
Oracle Database 11g, 12c
• Automatic SQL Tuning
• Automatic Workload Replay
• Automatic Capture of SQL Monitor
• Automatic Data Optimization
• Automatic Storage Indexes
• Automatic Columnar Cache
• Automatic Diagnostic Framework
• Automatic Refresh of Database Cloning
• Autonomous Health Framework
49
50

1/29/2019
26
well ... sort of
databases ... are hard
transactional,
instantaneous response
standard reports,
fast response
end of day/week/month batch,
longer but capped window
ad hoc query,
arbitrary criteria/duration
51
52

1/29/2019
27
all at the same time
transactional,
instantaneous response
standard reports,
fast response
end of day/week/month batch,
longer but capped window
ad hoc query,
arbitrary criteria/duration
so we started with "easy" :-)
53
54

1/29/2019
28
data warehousing
usage commonality
flexible response times
Oracle Autonomous Database
Enterprise
OLTP
Data
Warehousing
Q1 2018 C4 2018
Data
Warehousing
55
56

1/29/2019
29
autonomous objectives
1) easy
storage
quotas
indexing
partitioning
parallelism
compression
availability
etc
etc
load data
run queries
57
58

1/29/2019
30
2) fast or faster
Exadata infrastructure
Enterprise edition
3) elastic
59
60

1/29/2019
31
3) truly elastic
scale CPU & storage up, no outage
scale CPU down to save $
4) fully managed
backup / recovery
patching
upgrades
HA (99.95)
61
62

1/29/2019
32
walkthrough
"5 easy steps"
63
64

1/29/2019
33
1
creating your database
66
name
region
CPUs
storage
65
66

1/29/2019
34
you're done!
67
admin user password
"demo"
68
67
68

1/29/2019
35
69
70

1/29/2019
36
71
72

1/29/2019
37
73
74

1/29/2019
38
2
connecting to your database
76
75
76

1/29/2019
39
emphasis on security
77
SSL only
wallet
single download
78
service console
77
78

1/29/2019
40
"demo"
79
79
80

1/29/2019
41
81
82

1/29/2019
42
SQL Developer
84
native support
83
84

1/29/2019
43
85
86

1/29/2019
44
most tools
87
standard connectivty
C:temp>unzip -l credential.zip
Archive: credential.zip
Length Date Time Name
--------- ---------- ----- ----
0 04/06/2018 08:37
7045 04/06/2018 08:22 cwallet.sso
0 04/06/2018 08:37 cwallet.sso.lck
7000 04/06/2018 08:22 ewallet.p12
0 04/06/2018 08:37 ewallet.p12.lck
3182 04/06/2018 08:22 keystore.jks
132 11/06/2018 11:04 sqlnet.ora
334 04/06/2018 08:22 tnsnames.ora
3797 04/06/2018 08:22 truststore.jks
--------- -------
21490 9 files
87
88

1/29/2019
45
sqlnet.ora
WALLET_LOCATION = (SOURCE = (METHOD = file)
(METHOD_DATA = (DIRECTORY="...")))
SSL_SERVER_DN_MATCH=yes
tnsnames.ora
DW = (description=
(address=(protocol=tcps)(port=1522)(host=...))
(connect_data=(service_name=...))
(security=(ssl_server_cert_dn="CN=..."))
)
JDBC
90
more complicated
89
90

1/29/2019
46
props.setProperty("oracle.net.ssl_server_dn_match" ,"true");
props.setProperty("javax.net.ssl.trustStore" ,"truststore.jks" );
props.setProperty("javax.net.ssl.trustStorePassword","blahblah" );
props.setProperty("javax.net.ssl.trustStoreType" ,"JKS" );
props.setProperty("javax.net.ssl.keyStore" ,"keystore.jks" );
props.setProperty("javax.net.ssl.keyStoreType" ,"JKS" );
props.setProperty("javax.net.ssl.keyStorePassword" ,"blahblah");
con =
DriverManager.getConnection(
"jdbc:oracle:thin:myuser/mypassword@tnsdetails",props);
use latest drivers!
database services
92
predefined
91
92

1/29/2019
47
No of concurrent queries Max idle time CPU shares
HIGH 3 5 mins 4
MEDIUM 20 5 mins 2
LOW 32 1 hour 1
MY_DBNAME_
MY_DBNAME_
MY_DBNAME_
3
93
94

1/29/2019
48
users for your database
95
SQL> create user MY_DATA identified by MY_PASSWORD;
SQL> grant DWROLE to MY_DATA;
You're done!
95
96

1/29/2019
49
other users as per normal
97
4
97
98

1/29/2019
50
loading your database
99
ADW
Object
Storage
SQL*Net
99
100

1/29/2019
51
DBMS_CLOUD
101
focus on "1) easy"
external tables are .... "fun"
102
101
102

1/29/2019
52
CREATE TABLE emp_ext (
EMPNO NUMBER(4),
ENAME VARCHAR2(10),
JOB VARCHAR2(9),
MGR NUMBER(4),
HIREDATE DATE,
SAL NUMBER(7,2),
COMM NUMBER(7,2),
DEPTNO NUMBER(2)
)
ORGANIZATION EXTERNAL
(
TYPE ORACLE_LOADER
DEFAULT DIRECTORY NFS_DIR
ACCESS PARAMETERS
...
( RECORDS DELIMITED BY NEWLINE
DNFS_ENABLE
DNFS_READBUFFERS 10
FIELDS TERMINATED BY ','
MISSING FIELD VALUES ARE NULL
(
empno,
ename,
job,
mgr,
hiredate CHAR(21) DATE_FORMAT MASK "DD-MON-YYYY HH24:MI:SS",
sal,
comm,
deptno
) ) LOCATION ('EMP.dat')
)
PARALLEL 1
REJECT LIMIT UNLIMITED;
SQL> select * from emp_ext;
ORA-12801: error signaled in parallel query server P002
ORA-29913: error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
ORA-06512: at "SYS.ORACLE_LOADER", line 19
?
loading from object store
104
103
104

1/29/2019
53
1) credentials
105
begin
dbms_cloud.create_credential(
credential_name => 'OBJ_STORE_CRED',
username => 'connor',
password => 'password'
);
end;
/
105
106

1/29/2019
54
2) load/transform to table
107
begin
dbms_cloud.copy_data(
table_name=>'CHANNELS',
credential_name=>'OBJ_STORE_CRED',
file_uri_list=>
'https://objectstorage..../DEMO_DATA/chan_v3.dat',
format=>
json_object('ignoremissingcolumns' value 'true',
'removequotes' value 'true')
);
end;
107
108

1/29/2019
55
3) do the usual :-)
109
SQL> select table_name,
2 status,
3 rows_loaded,
4 logfile_table,
5 badfile_table
6 from user_load_operations;
TABLE_NAME STATUS ROWS_LOADED LOGFILE_TABLE BADFILE_TABLE
------------ ------- ------------ ------------- ---------------
CHANNELS FAILED COPY$1_LOG COPY$1_BAD
CHANNELS COMPLETED 5 COPY$2_LOG COPY$2_BAD
109
110

1/29/2019
56
data pump option
111
begin
dbms_cloud.create_credential(
credential_name => 'OBJ_STORE_CRED',
username => 'connor',
password => 'password'
);
end;
/
impdp sh/sh@salesdb_low
directory=data_pump_dir
credential=OBJ_STORE_CRED
dumpfile=https://objectstorage.../dump/sh.dmp
111
112

1/29/2019
57
5
querying your database
114
113
114

1/29/2019
58
SQL> select *
2 from MY_TABLE
3 ...
"Well...duh"
but you can do more
116
115
116

1/29/2019
59
ADWC
Object
Storage
SQL*Net
query before you load
118
117
118

1/29/2019
60
external table on object store
119
begin
dbms_cloud.create_external_table(
table_name =>'CHANNELS_EXT',
credential_name =>'OBJ_STORE_CRED',
file_uri_list => 'https://...chan_v3.dat',
format =>
json_object('ignoremissingcolumns' value 'true',
'removequotes' value 'true'),
column_list => 'CHANNEL_ID NUMBER,
CHANNEL_DESC VARCHAR2(20),
CHANNEL_CLASS VARCHAR2(20),
CHANNEL_CLASS_ID NUMBER,
CHANNEL_TOTAL VARCHAR2(13),
CHANNEL_TOTAL_ID NUMBER'
);
end;
select count(*) from channels_ext;
119
120

1/29/2019
61
recap
121
creation/connection
users
loaded
queried
We're done !!!
122
121
122

1/29/2019
62
"Fast, unlimited data access is the
crack cocaine for data scientists"
- James Morle
123
"3) elasticity"
124
123
124

1/29/2019
63
CPU increase ... anytime, no outage
125
125
126

1/29/2019
64
127
128

1/29/2019
65
SQL> show parameter cpu_count
NAME TYPE VALUE
------------------------------------ ----------- ---------------
cpu_count integer 1
SQL> show parameter cpu_count
NAME TYPE VALUE
------------------------------------ ----------- ---------------
cpu_count integer 2
SQL>
129
130

1/29/2019
66
"Yeah...I know what that means"
131
132

1/29/2019
67
You can scale to zero
133
service console
REST API
134
load yesterday's data
cpu = 2
133
134

1/29/2019
68
135
unleash the dogs :-)
cpu = 8
all other times = zero CPU cost
136
135
136

1/29/2019
69
"4) fully managed"
137
FAQ
137
138

1/29/2019
70
OS access
139
nope
SYSDBA access
140
nope
139
140

1/29/2019
71
patching
141
automated
online* (AppCont)
gold image
encryption
142
mandatory
141
142

1/29/2019
72
init.ora
143
all automatic
nls...
approx...
plsql...
tablespaces
144
nope
143
144

1/29/2019
73
indexes
145
nope*
materialized views
146
nope
145
146

1/29/2019
74
constraints
147
nope ... "yes"
compression
148
nope (HCC)
147
148

1/29/2019
75
SQL> create table t
2 as select a.* from all_objects a,
3 ( select 1 from dual connect by level <= 100 );
Table created.
CREATE TABLE "ADMIN"."T"
( "OWNER" VARCHAR2(128) NOT NULL ENABLE,
"OBJECT_NAME" VARCHAR2(128) NOT NULL ENABLE,
...
...
"MODIFIED_VSNID" NUMBER
)
PCTFREE 1 PCTUSED 40
COLUMN STORE COMPRESS FOR QUERY HIGH
ROW LEVEL LOCKING
TABLESPACE "DATA"
partitioning
150
nope*
149
150

1/29/2019
76
sidebar: Maybe you're doing it wrong :-)
151
statistics
152
advanced direct mode
automatic or explicit
151
152

1/29/2019
77
recall: 12c, direct load statistics
153
SQL> insert /*+ APPEND */ into MY_EMPTY_TABLE
2 select ...
10000 rows inserted.
SQL> select num_rows
2 from user_tables
3 where table_name = '...';
NUM_ROWS
----------
10000
SQL> insert /*+ APPEND */ into ...
10000 rows inserted.
SQL> select num_rows
2 from user_tables
3 where table_name = '...';
NUM_ROWS
----------
20000
153
154

1/29/2019
78
histograms too!
155
caching
156
result_cache
everything!
155
156

1/29/2019
79
in-memory
157
yes*
SQL> select *
2 from v$sgastat
3 where pool like 'in%';
POOL NAME BYTES
-------------- -------------------------- ----------
in-memory pool data area 536870912
in-memory pool metadata area 536870912
157
158

1/29/2019
80
hints
159
can be enabled
ignored by default
backup / recovery
160
daily incremental
weekly full
159
160

1/29/2019
81
backup / recovery
161
point-in-time nomination
(process is automatic)
(flashback possible)
60 days
161
162

1/29/2019
82
graph, text, spatial
163
coming later
migration
164
focus on easy
data pump
(tde, compression, etc)
163
164

1/29/2019
83
SQL> create table T ( x int) tablespace system;
Table created.
SQL> insert into t values (1);
1 row created.
SQL> select tablespace_name
2 from dba_segments
3 where segment_name = 'T';
TABLESPACE_NAME
------------------------
DATA
monitoring
166
real time SQL monitoring
service console
(and REST)
165
166

1/29/2019
84
167
168

1/29/2019
85
169
170

1/29/2019
86
from "easy" to "not so easy"
171
Oracle Autonomous Database
Enterprise
OLTP
Data
Warehousing
Q1 2018 CY 2018
Data
Warehousing
171
172

1/29/2019
87
173
174

1/29/2019
88
just a normal database?
176
175
176

1/29/2019
89
based on 19c
177
critical features
automatic indexes
177
178

1/29/2019
90
we follow best practice
Identify index
candidates based
on column usage
(SQL workload)
Create indexes in
unusable and
invisible mode
Compile
statements hard
parsed in current
cycle/iteration
Rebuild indexes
that optimizer
selected (status
as invisible)
Mark indexes as
visible if the perf
of the statements
improved
Repeated every
15 minutes
Validation continues
for the entire
workload. The index
status changes to
complete or revoked
179
180

1/29/2019
91
SQL> select ...
2 from EMP e,
3 DEPT d
4 where e.JOB = 'SALES'
5 and d.LOC = 'NORTH';
SQL> select REGION, min(AMOUNT)
2 from EMP e,
3 SALES_TRANSACTIONS s
4 where e.JOB = 'SALES'
5 and s.EMPNO = e.EMPNO
6 and s.REGION = 'CA'
7 and s.TAX_AMT > 10
8 or s.SUBSIDY > 0
9 group by REGION
181
182

1/29/2019
92
which brings us to ...
183
•Will my job go away?
•Will my job change?
•What is in it for me?
183
184

1/29/2019
93
% of presentations that
have a pie graph
% of presentations that
need a pie graph
72%
http://www.computerworld.com/article/2486278/it-management/how-to-balance-maintenance-and-it-innovation.html
185
186

1/29/2019
94
think back to automatic undo ...
187
... did you lose your job ?
188
or just get more databases :-)
187
188

1/29/2019
95
it is about business success
189
Data Modeling
Architecture,
Design
189
190

1/29/2019
96
End-to-End Service
Level Management
Full stack availability
Data Security
Data classification,
Data life-cycle management
191
192

1/29/2019
97
Application Tuning
SQL tuning,
Connection management
stop focussing on the negatives
193
194

1/29/2019
98
•Will my job go away?
•Will my job change?
•What is in it for me?
195
196

1/29/2019
99
security patching
after the CVE was published
- DB Maestro
Breaches
85%
zugzwang for DBAs
197
198

1/29/2019
100
"A player who has no move that
does not worsen his position is
said to be in 'zugzwang'"
2) problem detection
199
200

1/29/2019
101
ORAchk
Cluster Verification Utility
Cluster Health Monitor
Cluster Trace File Analyzer
etc
Autonomous Health Framework
201
202

1/29/2019
102
3) control ...
... is a terrible thing :-)
203
204

1/29/2019
103
205
206

1/29/2019
104
wrap up
the appeal of "set and forget"
208
207
208

1/29/2019
105
the appeal of bypassing this!
209
Request for Provisioning of Resources
2TB for 3500 compute hours
210
https://cloud.oracle.com/try-autonomous-database
209
210

1/29/2019
106
Enjoy the day!
youtube bit.ly/youtube-connor
blog bit.ly/blog-connor
twitter bit.ly/twitter-connor
211

ILOUG 2019 - Autonomous, what does it mean for DBAs

More Related Content

Similar to ILOUG 2019 - Autonomous, what does it mean for DBAs (20)

More from Connor McDonald (20)

Recently uploaded (20)

ILOUG 2019 - Autonomous, what does it mean for DBAs