Automate your oracle cloud infrastructure operations v2.0 - OOW19
2 likes927 views
The document outlines the automation of Oracle Cloud Infrastructure (OCI) operations, presenting topics such as OCI concepts, CLI tools, and examples of automation techniques. It is tailored for DBAs interested in cloud and DevOps, with an introduction to various OCI services and CLI usage instructions. Additionally, it details how to create and manage resources within OCI through CLI commands and troubleshooting common issues.
![OCI CLI on Windows bash – installation 1/2
$ bash -c "$(curl -L https://raw.githubusercontent.com/oracle/oci- cli/master/scripts/install/install.sh)"
…
Downloading Oracle Cloud Infrastructure CLI install script …
######################################################################## 100.0%
Running install script.
python3 /tmp/oci_cli_install_tmp_QomF < /dev/tty
-- Verifying Python version.
-- Python version 3.5.2 okay.
-- Verifying native dependencies.
-- Executing: 'dpkg -s libssl-dev libffi-dev python3-dev build-essential'
-- One or more of the following native dependencies are not currently installed and may be required.
"sudo apt-get install -y libssl-dev libffi-dev python3-dev build-essential"
===> Missing native dependencies. Continue and install the following dependencies: libssl-dev,
libffi-dev, python3-dev, build-essential? (Y/n): Y
[sudo] password for ncalero:
Hit:1 http://archive.ubuntu.com/ubuntu xenial InRelease
…
24 © 2019 Pythian Services](https://image.slidesharecdn.com/oow19-automateyouroraclecloudinfrastructureoperationsv2-190922212132/85/Automate-your-oracle-cloud-infrastructure-operations-v2-0-OOW19-24-320.jpg)
![OCI CLI setup
ncalero@H6RMYZ1:~/ioug$ cat ~/.oci/config
[DEFAULT]
user=ocid1.user.oc1..aaaaaaaaxotqff7cznh76yknkyqau2l2ysw72l65l4tb3ojto7rhfiyhrscq
fingerprint=a4:c2:1d:57:91:ba:a8:da:41:26:fd:08:f8:65:46:56
key_file=/home/ncalero/ioug/OCI_api_key.pem
tenancy=ocid1.tenancy.oc1..aaaaaaaalqia3yg7kmfvj7fvmgt4j4kokziowmjkjmymyo62bwkw5hodl4pa
region=us-ashburn-1
27 © 2019 Pythian Services
ncalero@H6RMYZ1:~$ oci db system list --compartment-id
ocid1.compartment.oc1..aaaaaaaanesmhx3kiiuxohpynihp4owu7tidov5wfqehmoubc5je6s76hlsa
ServiceError:
{
"code": " NotAuthenticated",
"message": "The required information to complete authentication was not provided or was
incorrect.",
"opc-request-id":
"C6A15A6E74B741C89704721C843E002A/00990538D26560F13A71509835BE3DF7/7CAC362441627E09812C81DCDD4491D5
",
"status": 401
}
Make sure to use the proper values:](https://image.slidesharecdn.com/oow19-automateyouroraclecloudinfrastructureoperationsv2-190922212132/85/Automate-your-oracle-cloud-infrastructure-operations-v2-0-OOW19-27-320.jpg)
![OCI CLI examples (1/2)
ncalero@H6RMYZ1:~/ioug$ oci db system list -c $COMPID --output table --query 'data
[*].{ClusterName:"cluster-name", shape:shape}'
+-------------+----------------+
| ClusterName | shape |
+-------------+----------------+
| None | VM.Standard1.1 |
| None | VM.Standard1.1 |
+-------------+----------------+
28 © 2019 Pythian Services
List Database Systems
List Compute instances
ncalero@H6RMYZ1:~/ioug$ oci compute instance list -c $COMPID --output table --query 'data
[*].{AD:"availability-domain", Name:"display-name", shape:"shape", region:region}'
+----------------------+----------+--------+----------------+
| AD | Name | region | shape |
+----------------------+----------+--------+----------------+
| HDGG:US-ASHBURN-AD-1 | oem13c | iad | VM.DenseIO1.4 |
| HDGG:US-ASHBURN-AD-1 | ora122vm | iad | VM.Standard2.1 |
+----------------------+----------+--------+----------------+](https://image.slidesharecdn.com/oow19-automateyouroraclecloudinfrastructureoperationsv2-190922212132/85/Automate-your-oracle-cloud-infrastructure-operations-v2-0-OOW19-28-320.jpg)
![OCI CLI examples (2/2)
ncalero@H6RMYZ1:~/ioug$ oci network subnet list -c $COMPID --vcn-id $VCNID --output table
--query "data [?contains("display-name",'ashb')].{Name:"display-name",
AD:"availability-domain", CIDR:"cidr-block"}"
+----------------------+-------------+------------------------------------+
| AD | CIDR | Name |
+----------------------+-------------+------------------------------------+
| HDGG:US-ASHBURN-AD-3 | 10.0.2.0/24 | Public Subnet HDGG:US-ASHBURN-AD-3 |
| HDGG:US-ASHBURN-AD-2 | 10.0.1.0/24 | Public Subnet HDGG:US-ASHBURN-AD-2 |
| HDGG:US-ASHBURN-AD-1 | 10.0.0.0/24 | Public Subnet HDGG:US-ASHBURN-AD-1 |
+----------------------+-------------+------------------------------------+
29 © 2019 Pythian Services
Filtering results using --query "data[expr] ...". http://jmespath.org/proposals/filter-expressions.html
Getting a JSON template to use for creation commands
ncalero@H6RMYZ1:~/ioug$ $ oci db system launch --generate-full-command-json-input
{
"adminPassword": "string",
"availabilityDomain": "string",
"backupSubnetId": "string",
"characterSet": "string",
…](https://image.slidesharecdn.com/oow19-automateyouroraclecloudinfrastructureoperationsv2-190922212132/85/Automate-your-oracle-cloud-infrastructure-operations-v2-0-OOW19-29-320.jpg)
![Example: creating a Database System with CLI
35 © 2019 Pythian Services
Fixing previously reported problems:
1) Request is rejected as port 22 is not enabled in the security list for subnet
- The default security list for the subnet should have an ingress rule of type TCP for the subnet
CIDR for all ports as source and target port 22. MOS note 2433870.1 helps.
2) InvalidParameter, backupSubnetId : is not supported with current shape
- Double check parameters required for the DB we want to create.
Note “oci db system get“ reports some attributes not needed to launch
3) LimitExceeded, You have reached your service limit of 8 Virtual Machine CPU
Cores in this Availability Domain
- Validate the DBs already created in all your compartments
$ oci iam compartment list --output table --query 'data [*].{id:id}'
$ oci network security-list get --security-list-id $SECLISTID](https://image.slidesharecdn.com/oow19-automateyouroraclecloudinfrastructureoperationsv2-190922212132/85/Automate-your-oracle-cloud-infrastructure-operations-v2-0-OOW19-35-320.jpg)
![Example: creating a Database System with CLI
36 © 2019 Pythian Services
ncalero@H6RMYZ1:~/ioug$ cat db-ioug-ad1-2n.json
{
"adminPassword": "xxx",
"characterSet": "WE8MSWIN1252",
"dbWorkload": "OLTP",
"dbName": "CIOUG",
"dbVersion": "12.1.0.2.181016",
"ncharacterSet": "AL16UTF16",
"pdbName": "PIOUG",
"waitForState": "PROVISIONING",
"waitIntervalSeconds": 30,
"version": "12.2.0.1.181016",
"availabilityDomain": "HDGG:US-ASHBURN-AD-1",
"clusterName": "IOUG",
"compartmentId": "ocid1.compartment.oc1..aaaaaaaazh5fxaasdfa",
"cpuCoreCount": 8,
"dataStoragePercentage": 80,
"databaseEdition": "ENTERPRISE_EDITION_EXTREME_PERFORMANCE",
"diskRedundancy": "HIGH",
"diskRedundancy": "HIGH",
"displayName": "IOUG",
"domain": "sub1234.vcn1234.oraclevcn.com",
"faultDomains": [
"FAULT-DOMAIN-1"
],
"freeformTags": {},
"hostname": "iougdb0",
"initialDataStorageSizeInGb": 256,
"licenseModel": "LICENSE_INCLUDED",
"nodeCount": 2,
"shape": "VM.Standard1.2",
"sparseDiskgroup": null,
"sshAuthorizedKeysFile": "./ssh-keys-ioug.txt",
"subnetId": "ocid1.subnet.oc1.iad.aaaaaaaabcd",
"timeZone": "UTC"
}](https://image.slidesharecdn.com/oow19-automateyouroraclecloudinfrastructureoperationsv2-190922212132/85/Automate-your-oracle-cloud-infrastructure-operations-v2-0-OOW19-36-320.jpg)
![Example: creating a Database System with CLI
37 © 2019 Pythian Services
[opc@iougdb0 ~]$ sudo su -
[root@iougdb0 ~]# dbcli describe-component
System Version
---------------
18.3.3.0.0
Component Installed Version Available Version
---------------------------------------- -------------------- --------------------
GI 12.2.0.1.181016 12.2.0.1.190115
DB 12.1.0.2.181016 12.1.0.2.190115
[root@iougdb0 ~]# dbcli describe-latestpatch
componentType availableVersion
--------------- --------------------
gi 12.2.0.1.190115
gi 12.1.0.2.190115
gi 18.5.0.0.190115
db 11.2.0.4.190115
db 12.2.0.1.190115
db 12.1.0.2.190115
db 18.5.0.0.190115](https://image.slidesharecdn.com/oow19-automateyouroraclecloudinfrastructureoperationsv2-190922212132/85/Automate-your-oracle-cloud-infrastructure-operations-v2-0-OOW19-37-320.jpg)
![ncalero@H6RMYZ1:/mnt/d/GitHub/terraform-OCI/demo$ cat provider.tf
provider "oci" {
tenancy_ocid = "${var.tenancy_ocid}"
user_ocid = "${var.user_ocid}"
fingerprint = "${var.fingerprint}"
private_key_path = "${var.private_key_path}“
region = "${var.region}"
}
ncalero@H6RMYZ1:/mnt/d/GitHub/terraform-OCI$ cat compute.tf
resource "oci_core_instance" "DemoInstance" {
availability_domain =
"${lookup(data.oci_identity_availability_domains.ADs.availability_domains[0],"name")}"
compartment_id = "${var.compartment_ocid}"
display_name = "Demo-Instance"
image = "${lookup(data.oci_core_images.OLImageOCID.images[0], "id")}"
shape = "${var.InstanceShape}"
subnet_id = "${oci_core_subnet.SN-DemoSubnetAD1.id}"
metadata {
ssh_authorized_keys = "${var.ssh_public_key}"
user_data = "${base64encode(file(var.InstanceBootStrap))}"
}
}
41 © 2019 Pythian Services](https://image.slidesharecdn.com/oow19-automateyouroraclecloudinfrastructureoperationsv2-190922212132/85/Automate-your-oracle-cloud-infrastructure-operations-v2-0-OOW19-41-320.jpg)
![PS D:GitHubterraform-OCIdemo> d:softterraform apply
Terraform will perform the following actions:
…
Plan: 9 to add, 1 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
oci_core_volume.DemoBlock0: Creating...
…
oci_core_instance.DemoInstance: Creation complete after 1m9s (ID:
ocid1.instance.oc1.iad.abuwcljtxf33kdni...toxpdtljex6zq5gl2vsccmgffvlfcsswom5usa)
…
Apply complete! Resources: 9 added, 0 changed, 0 destroyed.
Outputs:
InstancePublicIP = [
129.213.48.178
]
45 © 2019 Pythian Services](https://image.slidesharecdn.com/oow19-automateyouroraclecloudinfrastructureoperationsv2-190922212132/85/Automate-your-oracle-cloud-infrastructure-operations-v2-0-OOW19-45-320.jpg)
Automate your oracle cloud infrastructure operations v2.0 - OOW19
- 1. Automate the operation of your Oracle Cloud Infrastructure 2.0 Nelson Calero Oracle Open World 2019 DEV1798 - Moscone South R310
- 2. Today’s topics • Oracle cloud concepts (quick review) • CLI tools (focus on OCI) • Examples • Automation 2 © 2019 Pythian Services Intended audience: DBAs interested in Cloud and DevOps
- 3. • Principal Consultant at Pythian – several roles since 2014 • Working with Oracle tools and Linux environments since 1996 • DBA Oracle (2001) & MySQL (2005) • Co-founder and President of the Oracle user Group of Uruguay (2009) • LAOUC Director of events (2013) • Computer Engineer (1998) • Oracle ACE (2014), Oracle ACE Director (2017) • Oracle Certified Professional 10g/11g/12c, OCE, Cloud DB & Infra • Amazon Solutions Architect – Associate (2016) • Google Cloud Architect (2017), Google Cloud Data Engineer (2017) • Oracle University Instructor (2011) • Blogger and speaker: Oracle Open World, Collaborate, OTN Tour, Regional conferences About me 3 © 2019 Pythian Services http://www.linkedin.com/in/ncalero @ncalerouy
- 4. Cloud Automation/DevOps Machine Learning and MLOps Data Lakes / Platforms / DataOps BI - Analytics - Visualizations Consulting - Strategy & Data Management Data Warehouse Migration/Modernization Database Troubleshooting Database Strategy & Estate Planning Database Migration Operational Data & Cloud Infrastructure Analytics & Cloud Data Solutions Plan Deploy ManageDatabase Mgt
- 6. 3 Membership Tiers • Oracle ACE Director • Oracle ACE • Oracle ACE Associate bit.ly/OracleACEProgram 500+ Technical Experts Helping Peers Globally Connect: @oracleace Facebook.com/oracleaces [email protected]
- 7. Oracle Cloud offering 7 https://blogs.oracle.com/futurestate/when-cloud-meets-on-premise:-a-story-of-two-applications © 2019 Pythian Services
- 8. Oracle Cloud - IaaS OCI (new gen. - 2017) •Audit •Compute •Container Engine for Kubernetes •Container Registry •Load Balancing •Object Storage •Oracle Cloud Infrastructure Database •Networking •Ravello •Wercker 8 OCI Classic (v1 - 2014) •Compute Classic •Container Classic •Container Registry Classic •Dedicated Compute Classic - SPARC Model 300 •DIVA (media assets) •Load Balancing Classic •Messaging •Storage Classic •Storage Software Appliance https://docs.cloud.oracle.com/iaas/Content/home.htm https://docs.oracle.com/en/cloud/iaas/index.html © 2019 Pythian Services
- 9. Oracle Cloud - IaaS OCI (new gen. - 2017) •Audit •Compute •Container Engine for Kubernetes •Container Registry •Load Balancing •Object Storage •Oracle Cloud Infrastructure Database •Networking •Ravello •Wercker 9 OCI Classic (v1 - 2014) •Compute Classic •Container Classic •Container Registry Classic •Dedicated Compute Classic - SPARC Model 300 •DIVA (media assets) •Load Balancing Classic •Messaging •Storage Classic •Storage Software Appliance https://docs.cloud.oracle.com/iaas/Content/home.htm https://docs.oracle.com/en/cloud/iaas/index.html © 2019 Pythian Services
- 13. Instance shapes 13 OCI Classic OCI Not in all regions
- 14. 14 http://cloud.oracle.com Connecting to the Web console
- 15. © 2019 Pythian Services15
- 16. © 2019 Pythian Services16
- 17. Oracle Cloud dashboard – includes Classic 17 © 2019 Pythian Services
- 18. Other ways to access the Oracle Cloud • REST API - https://docs.cloud.oracle.com/iaas/api/ • SDK - Java, Python, Ruby, Go • CLI tool – one for all services (not the case for OCI-C) • Specific for particular services: https://docs.cloud.oracle.com/iaas/Content/API/Concepts/devopstools.htm – HDFS Connector for Object storage Storage Gateway (NFS) – Terraform provider – Ansible Modules – Chef Knife Plug-in – Compute Jenkis Plug-in – Grafana Plug-in Note: OCI Object storage has an Amazon S3 compatible API - https://docs.cloud.oracle.com/iaas/Content/Object/Tasks/s3compatibleapi.htm 18 © 2019 Pythian Services
- 19. Cloud CLI tools (classic and OCI) 19 ORACLE CLOUD INFRASTRUCTURE Database System Bare Metal Compute Container dbaascli raccli dbpatchm bkup_api opc (classic) oci oracle-dbcs-cli (classic) ftmcli.jar (classic) bmcs Virtual Machine Load Balancer Object Storage REST SDK © 2019 Pythian Services Download tools from OTN: http://www.oracle.com/technetwork/topics/cloud/downloads/index.html
- 20. Cloud CLI tools (classic and OCI) 20 ORACLE CLOUD INFRASTRUCTURE Database System Bare Metal Compute Container dbaascli raccli dbpatchm bkup_api opc (classic) oci oracle-dbcs-cli (classic) ftmcli.jar (classic) bmcs Virtual Machine Load Balancer Object Storage REST SDK © 2019 Pythian Services Download tools from OTN: http://www.oracle.com/technetwork/topics/cloud/downloads/index.html
- 21. OCI CLI tools 21 ORACLE CLOUD INFRASTRUCTURE Database System Bare Metal Compute Container dbaascli (ExaCS) dbcli exacli bkup_api oci Virtual Machine Load Balancer Object Storage REST API SDK © 2019 Pythian Services
- 22. Required account details for CLI OCI Classic • REST endpoint • user • password • identity domain 22 © 2019 Pythian Services OCI • tenancy OCID • compartment OCID • user OCID • public key fingerprint • region OCI: https://docs.cloud.oracle.com/iaas/Content/API/Concepts/apisigningkey.htm OCI-C: https://docs.oracle.com/en/cloud/iaas/compute-iaas-cloud/stopc/preparing-use-cli.html
- 23. OCI CLI installation in unix $ bash -c "$(curl -L https://raw.githubusercontent.com/oracle/oci- cli/master/scripts/install/install.sh)" 23 © 2019 Pythian Services https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/cliinstall.htm • dependencies could require update of other packages (ex: openssl in in OEL 6.9) • It should work in Windows having admin privileges Follows an example of installing it under Ubuntu 16.04.3 LTS (WLS in Windows 10)
- 24. OCI CLI on Windows bash – installation 1/2 $ bash -c "$(curl -L https://raw.githubusercontent.com/oracle/oci- cli/master/scripts/install/install.sh)" … Downloading Oracle Cloud Infrastructure CLI install script … ######################################################################## 100.0% Running install script. python3 /tmp/oci_cli_install_tmp_QomF < /dev/tty -- Verifying Python version. -- Python version 3.5.2 okay. -- Verifying native dependencies. -- Executing: 'dpkg -s libssl-dev libffi-dev python3-dev build-essential' -- One or more of the following native dependencies are not currently installed and may be required. "sudo apt-get install -y libssl-dev libffi-dev python3-dev build-essential" ===> Missing native dependencies. Continue and install the following dependencies: libssl-dev, libffi-dev, python3-dev, build-essential? (Y/n): Y [sudo] password for ncalero: Hit:1 http://archive.ubuntu.com/ubuntu xenial InRelease … 24 © 2019 Pythian Services
- 25. OCI CLI on Windows bash – installation 2/2 … Successfully installed PyYAML-3.13 arrow-0.10.0 asn1crypto-0.24.0 certifi-2019.3.9 cffi-1.12.2 click-6.7 configparser-3.5.0 cryptography-2.4.2 cx-Oracle-7.0.0 httpsig-cffi-15.0.0 idna-2.6 jmespath-0.9.3 oci-2.2.3 oci-cli-2.5.4 pyOpenSSL-18.0.0 pycparser-2.19 python-dateutil-2.7.3 pytz-2016.10 retrying-1.3.3 six-1.11.0 terminaltables-3.1.0 ===> Modify profile to update your $PATH and enable shell/tab completion now? (Y/n): y ===> Enter a path to an rc file to update (leave blank to use '/home/ncalero/.bashrc'): -- Backed up '/home/ncalero/.bashrc' to '/home/ncalero/.bashrc.backup' -- Tab completion set up complete. -- If tab completion is not activated, verify that '/home/ncalero/.bashrc' is sourced by your shell. -- -- ** Run `exec -l $SHELL` to restart your shell. ** -- -- Installation successful. -- Run the CLI with /home/ncalero/bin/oci --help ncalero@H6RMYZ1:~$ ncalero@H6RMYZ1:~$ 25 © 2019 Pythian Services
- 26. OCI CLI setup $ openssl genrsa -out OCI_api_key.pem 2048 $ chmod go-rwx OCI_api_key.pem $ openssl rsa -pubout -in OCI_api_key.pem -out OCI_api_key-pub.pem 26 © 2019 Pythian Services 1) Generate RSA public key https://docs.cloud.oracle.com/Content/API/Concepts/apisigningkey.htm 2) Upload public key using OCI web console It should be a local user in your tenant It will report the fingerprint of the public key uploaded 3) Setup OCI CLI Create $HOME/.oci/config file, manually or using the wizard $ oci setup config Fingerprint: 8c:c7:11:87:b8:aa:d5:a7:5c:d4:9f:ff:48:48:5d:e3
- 27. OCI CLI setup ncalero@H6RMYZ1:~/ioug$ cat ~/.oci/config [DEFAULT] user=ocid1.user.oc1..aaaaaaaaxotqff7cznh76yknkyqau2l2ysw72l65l4tb3ojto7rhfiyhrscq fingerprint=a4:c2:1d:57:91:ba:a8:da:41:26:fd:08:f8:65:46:56 key_file=/home/ncalero/ioug/OCI_api_key.pem tenancy=ocid1.tenancy.oc1..aaaaaaaalqia3yg7kmfvj7fvmgt4j4kokziowmjkjmymyo62bwkw5hodl4pa region=us-ashburn-1 27 © 2019 Pythian Services ncalero@H6RMYZ1:~$ oci db system list --compartment-id ocid1.compartment.oc1..aaaaaaaanesmhx3kiiuxohpynihp4owu7tidov5wfqehmoubc5je6s76hlsa ServiceError: { "code": " NotAuthenticated", "message": "The required information to complete authentication was not provided or was incorrect.", "opc-request-id": "C6A15A6E74B741C89704721C843E002A/00990538D26560F13A71509835BE3DF7/7CAC362441627E09812C81DCDD4491D5 ", "status": 401 } Make sure to use the proper values:
- 28. OCI CLI examples (1/2) ncalero@H6RMYZ1:~/ioug$ oci db system list -c $COMPID --output table --query 'data [*].{ClusterName:"cluster-name", shape:shape}' +-------------+----------------+ | ClusterName | shape | +-------------+----------------+ | None | VM.Standard1.1 | | None | VM.Standard1.1 | +-------------+----------------+ 28 © 2019 Pythian Services List Database Systems List Compute instances ncalero@H6RMYZ1:~/ioug$ oci compute instance list -c $COMPID --output table --query 'data [*].{AD:"availability-domain", Name:"display-name", shape:"shape", region:region}' +----------------------+----------+--------+----------------+ | AD | Name | region | shape | +----------------------+----------+--------+----------------+ | HDGG:US-ASHBURN-AD-1 | oem13c | iad | VM.DenseIO1.4 | | HDGG:US-ASHBURN-AD-1 | ora122vm | iad | VM.Standard2.1 | +----------------------+----------+--------+----------------+
- 29. OCI CLI examples (2/2) ncalero@H6RMYZ1:~/ioug$ oci network subnet list -c $COMPID --vcn-id $VCNID --output table --query "data [?contains("display-name",'ashb')].{Name:"display-name", AD:"availability-domain", CIDR:"cidr-block"}" +----------------------+-------------+------------------------------------+ | AD | CIDR | Name | +----------------------+-------------+------------------------------------+ | HDGG:US-ASHBURN-AD-3 | 10.0.2.0/24 | Public Subnet HDGG:US-ASHBURN-AD-3 | | HDGG:US-ASHBURN-AD-2 | 10.0.1.0/24 | Public Subnet HDGG:US-ASHBURN-AD-2 | | HDGG:US-ASHBURN-AD-1 | 10.0.0.0/24 | Public Subnet HDGG:US-ASHBURN-AD-1 | +----------------------+-------------+------------------------------------+ 29 © 2019 Pythian Services Filtering results using --query "data[expr] ...". http://jmespath.org/proposals/filter-expressions.html Getting a JSON template to use for creation commands ncalero@H6RMYZ1:~/ioug$ $ oci db system launch --generate-full-command-json-input { "adminPassword": "string", "availabilityDomain": "string", "backupSubnetId": "string", "characterSet": "string", …
- 30. Options to create OCI Instances and DBs • Using web console • Using API / CLI / SDK – No wizard, each component used must be already created • Terraform provider - OCI Creates components if they don’t exists • Ansible modules • CLI examples provided by Oracle: https://github.com/oracle/oci-cli/blob/master/scripts/examples 30 © 2019 Pythian Services
- 31. We need to use several components from OCI: • Compartment • Subnet – route table, security list, security rules 31 © 2019 Pythian Services - dbname - characterSet - dbWorkload (OLTP/DW) - dbVersion - pdbName - databaseEdition - ssh keys We need to specify: • AD, faultDomains • Type (Exadata, BM, VM) • Shape • licenseModel • nodeCount • version • hostname Example: creating a Database System with CLI
- 32. Example: creating a Database System with CLI 32 © 2019 Pythian Services $ oci db system launch --from-json file://db-ioug-ad1-2n.json It returns a long JSON including ID when it completes Or returns error if it cannot be created Action completed. Waiting until the resource has entered state: PROVISIONING { "data": { "availability-domain": "HDGG:US-ASHBURN-AD-1", ... "id": "ocid1.dbsystem.oc1.iad.abuwcljt2ws772ch6xlw2zadecuvlgven4hslmupiyx6y3add3na5kra6xgq", ... ncalero@H6RMYZ1:~/ioug$ oci db system launch --from-json file://db-ioug.json ServiceError: { "code": "LimitExceeded", "message": "You have reached your service limit of 8 Virtual Machine CPU Cores in this Availability Domain. Please try launching the instance in a different Availability Domain or Region, or try using a different shape. If you have reached all Service limits, please contact Oracle support to request a limit increase.", "opc-request-id": "9360A40AFADE4A4EB9A316AF0EB57015/0D5F255C1E15084A13B3D/ABD24B413BE4BA2B9414E9108CF02", "status": 400 }
- 33. Example: creating a Database System with CLI 33 © 2019 Pythian Services $ oci db system launch --from-json file://db-ioug-ad1-2n.json It returns a long JSON including ID when it completes Or returns error if it cannot be created Action completed. Waiting until the resource has entered state: PROVISIONING { "data": { "availability-domain": "HDGG:US-ASHBURN-AD-1", ... "id": "ocid1.dbsystem.oc1.iad.abuwcljt2ws772ch6xlw2zadecuvlgven4hslmupiyx6y3add3na5kra6xgq", ... ncalero@H6RMYZ1:~/ioug$ oci db system launch --from-json file://db-ioug.json ServiceError: { "code": "LimitExceeded", "message": "You have reached your service limit of 8 Virtual Machine CPU Cores in this Availability Domain. Please try launching the instance in a different Availability Domain or Region, or try using a different shape. If you have reached all Service limits, please contact Oracle support to request a limit increase.", "opc-request-id": "9360A40AFADE4A4EB9A316AF0EB57015/0D5F255C1E15084A13B3D/ABD24B413BE4BA2B9414E9108CF02", "status": 400 } ncalero@H6RMYZ1:~/ioug$ oci db system launch --from-json file://db-ioug.json ServiceError: { "code": "InvalidParameter", "message": "backupSubnetId : is not supported with current shape.", "opc-request-id": "2A1EC170EFDC4B8385A92FA8CED88A07/A6255B952C6932028509361640F6…", "status": 400 }
- 34. Example: creating a Database System with CLI 34 © 2019 Pythian Services $ oci db system launch --from-json file://db-ioug-ad1-2n.json It returns a long JSON including ID when it completes Or returns error if it cannot be created Action completed. Waiting until the resource has entered state: PROVISIONING { "data": { "availability-domain": "HDGG:US-ASHBURN-AD-1", ... "id": "ocid1.dbsystem.oc1.iad.abuwcljt2ws772ch6xlw2zadecuvlgven4hslmupiyx6y3add3na5kra6xgq", ... ncalero@H6RMYZ1:~/ioug$ oci db system launch --from-json file://db-ioug.json ServiceError: { "code": "LimitExceeded", "message": "You have reached your service limit of 8 Virtual Machine CPU Cores in this Availability Domain. Please try launching the instance in a different Availability Domain or Region, or try using a different shape. If you have reached all Service limits, please contact Oracle support to request a limit increase.", "opc-request-id": "9360A40AFADE4A4EB9A316AF0EB57015/0D5F255C1E15084A13B3D/ABD24B413BE4BA2B9414E9108CF02", "status": 400 } ncalero@H6RMYZ1:~/ioug$ oci db system launch --from-json file://db-ioug.json ServiceError: { "code": "InvalidParameter", "message": "backupSubnetId : is not supported with current shape.", "opc-request-id": "2A1EC170EFDC4B8385A92FA8CED88A07/A6255B952C6932028509361640F6…", "status": 400 } ServiceError: { "code": "InvalidParameter", "message": "Request is rejected as port 22 is not enabled in the security list for subnet ocid1.subnet.oc1.iad.aaaaaaaahzydgowtl3tim7usrixkwhlbbzexowa", "opc-request-id": "7DE3DBB7637A4E9887B6913C40D3A227C33329E82/25664AE40C8E17A36F709A042656496B", "status": 400 }
- 35. Example: creating a Database System with CLI 35 © 2019 Pythian Services Fixing previously reported problems: 1) Request is rejected as port 22 is not enabled in the security list for subnet - The default security list for the subnet should have an ingress rule of type TCP for the subnet CIDR for all ports as source and target port 22. MOS note 2433870.1 helps. 2) InvalidParameter, backupSubnetId : is not supported with current shape - Double check parameters required for the DB we want to create. Note “oci db system get“ reports some attributes not needed to launch 3) LimitExceeded, You have reached your service limit of 8 Virtual Machine CPU Cores in this Availability Domain - Validate the DBs already created in all your compartments $ oci iam compartment list --output table --query 'data [*].{id:id}' $ oci network security-list get --security-list-id $SECLISTID
- 36. Example: creating a Database System with CLI 36 © 2019 Pythian Services ncalero@H6RMYZ1:~/ioug$ cat db-ioug-ad1-2n.json { "adminPassword": "xxx", "characterSet": "WE8MSWIN1252", "dbWorkload": "OLTP", "dbName": "CIOUG", "dbVersion": "12.1.0.2.181016", "ncharacterSet": "AL16UTF16", "pdbName": "PIOUG", "waitForState": "PROVISIONING", "waitIntervalSeconds": 30, "version": "12.2.0.1.181016", "availabilityDomain": "HDGG:US-ASHBURN-AD-1", "clusterName": "IOUG", "compartmentId": "ocid1.compartment.oc1..aaaaaaaazh5fxaasdfa", "cpuCoreCount": 8, "dataStoragePercentage": 80, "databaseEdition": "ENTERPRISE_EDITION_EXTREME_PERFORMANCE", "diskRedundancy": "HIGH", "diskRedundancy": "HIGH", "displayName": "IOUG", "domain": "sub1234.vcn1234.oraclevcn.com", "faultDomains": [ "FAULT-DOMAIN-1" ], "freeformTags": {}, "hostname": "iougdb0", "initialDataStorageSizeInGb": 256, "licenseModel": "LICENSE_INCLUDED", "nodeCount": 2, "shape": "VM.Standard1.2", "sparseDiskgroup": null, "sshAuthorizedKeysFile": "./ssh-keys-ioug.txt", "subnetId": "ocid1.subnet.oc1.iad.aaaaaaaabcd", "timeZone": "UTC" }
- 37. Example: creating a Database System with CLI 37 © 2019 Pythian Services [opc@iougdb0 ~]$ sudo su - [root@iougdb0 ~]# dbcli describe-component System Version --------------- 18.3.3.0.0 Component Installed Version Available Version ---------------------------------------- -------------------- -------------------- GI 12.2.0.1.181016 12.2.0.1.190115 DB 12.1.0.2.181016 12.1.0.2.190115 [root@iougdb0 ~]# dbcli describe-latestpatch componentType availableVersion --------------- -------------------- gi 12.2.0.1.190115 gi 12.1.0.2.190115 gi 18.5.0.0.190115 db 11.2.0.4.190115 db 12.2.0.1.190115 db 12.1.0.2.190115 db 18.5.0.0.190115
- 38. OCI: creating a Compute Instance We need to use several components from OCI: • Compartment • Virtual Cloud Network – Subnet, Internet GW, route table, security list, security rules • OS Image • Shape We need to provide: • ssh keys 38 © 2019 Pythian Services
- 39. Example: creating an OCI Instance with Terraform • Download Terraform binary (using v0.11.3 today) – https://www.terraform.io/downloads.html • OCI-provider – plugin in v3.0 automatically installed – https://github.com/oracle/terraform-provider-oci/releases • Create .tf HCL Project (based on examples): – https://github.com/oracle/terraform-provider-oci/tree/master/docs/examples • Configure your account Access (env-vars.ps1) – Tenancy and user OCIDs, fingerprint, private key, region • Terraform Plan • Terraform Apply • Step by step guide: https://community.oracle.com/docs/DOC-1019936 39 © 2019 Pythian Services
- 40. ncalero@H6RMYZ1:/mnt/d/GitHub/terraform-OCI/$ ls block.tf compute.tf env-vars.ps1 outputs.tf userdata datasources.tf network.tf provider.tf remote-exec.tf variables.tf ncalero@H6RMYZ1:/mnt/d/GitHub/terraform-OCI$ cat /mnt/c/users/calero/AppData/Roaming/terraform.rc providers { oci = "d:softterraform-provider-oci_v2.1.0.exe" } 40 © 2019 Pythian Services PS D:GitHubterraform-OCI> cat ./env-vars.ps1 $env:TF_VAR_tenancy_ocid="ocid1.tenancy.oc1..aaaaaaaaw5hodl4pajmymyo62bwkj4kokziowmjkvj7fvmgt4lqia3yg7km " $env:TF_VAR_user_ocid="ocid1.user.oc1..aaaaaaaaiyhrscq7rhfiyhrscql4tb3ojto2ysw72l656yknkyqau2lxotqf" $env:TF_VAR_compartment_ocid="ocid1.compartment.oc1.. aaaaaaaakdevdha7q7yjdheridcy63sgv2syefv7x46u67jk4hj" $env:TF_VAR_region="us-ashburn-1“ $env:TF_VAR_fingerprint="db:a3:b8:59:ce:05:ff:78:c0:fb:aa:ca:d7:09:77:ad" $env:TF_VAR_private_key_path="C:Userscalero.ocioci_api_key.pem" ### Public/private keys used on the instances $env:TF_VAR_ssh_public_key = Get-Content C:Userscalero.sshid_rsa.pub -Raw $env:TF_VAR_ssh_private_key = Get-Content C:Userscalero.sshid_rsa -Raw PS D:GitHubterraform-OCI> ./env-vars.ps1
- 41. ncalero@H6RMYZ1:/mnt/d/GitHub/terraform-OCI/demo$ cat provider.tf provider "oci" { tenancy_ocid = "${var.tenancy_ocid}" user_ocid = "${var.user_ocid}" fingerprint = "${var.fingerprint}" private_key_path = "${var.private_key_path}“ region = "${var.region}" } ncalero@H6RMYZ1:/mnt/d/GitHub/terraform-OCI$ cat compute.tf resource "oci_core_instance" "DemoInstance" { availability_domain = "${lookup(data.oci_identity_availability_domains.ADs.availability_domains[0],"name")}" compartment_id = "${var.compartment_ocid}" display_name = "Demo-Instance" image = "${lookup(data.oci_core_images.OLImageOCID.images[0], "id")}" shape = "${var.InstanceShape}" subnet_id = "${oci_core_subnet.SN-DemoSubnetAD1.id}" metadata { ssh_authorized_keys = "${var.ssh_public_key}" user_data = "${base64encode(file(var.InstanceBootStrap))}" } } 41 © 2019 Pythian Services
- 42. 42 © 2019 Pythian Services
- 43. 43 © 2019 Pythian Services
- 44. © 2019 Pythian Services44 PS D:GitHubterraform-OCI> d:softterraform graph | dot -Tpng -o demo-graph.png
- 45. PS D:GitHubterraform-OCIdemo> d:softterraform apply Terraform will perform the following actions: … Plan: 9 to add, 1 to change, 0 to destroy. Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: yes oci_core_volume.DemoBlock0: Creating... … oci_core_instance.DemoInstance: Creation complete after 1m9s (ID: ocid1.instance.oc1.iad.abuwcljtxf33kdni...toxpdtljex6zq5gl2vsccmgffvlfcsswom5usa) … Apply complete! Resources: 9 added, 0 changed, 0 destroyed. Outputs: InstancePublicIP = [ 129.213.48.178 ] 45 © 2019 Pythian Services
- 46. Terraform apply © 2019 Pythian Services46
- 47. Backup considerations: 47 © 2019 Pythian Services • "Backups that were configured using the Console may become unusable if you make changes using these commands. For backups configured using the Console, use these commands with support guidance only." dbcli getstatus-backup dbcli getstatus-backup -in CTEST2 -t Archivelog dbcli create-rmanbackupreport -w summary -rn test --dbname CTEST2 dbcli list-jobs dbcli describe-job -i 4340156e-9e72-48ce-9e6a-f1d34b343603 ls -lrt /opt/oracle/dcs/log/test2/rman/bckup/CTEST2_* | tail
- 48. Automation approach • Not the way you are used to as DBA: PaaS have built-in automation. DBaaS example: • Backup • Recovery • Patch • Failover • Standby creation (VM DB Systems only, same VCN for prim/stb, web console) • Extra tools provided by Oracle – OCI Terraform provider – Ansible modules: https://github.com/oracle/oci-ansible-modules – Chef Knife Plugin – EBS Cloud Admin Tool 48 © 2019 Pythian Services
- 49. © 2019 Pythian Services49 Built-in automation: DBCS patching (classic)
- 50. Built-in automation: DBCS patching © 2019 Pythian Services50 • Different in ExaCs and VM/BM DB Systems • Tools automatically deployed to server when created – manually update to new version before patching • PSU can be specified when creating the DB • Web console shows System (GI) and DB patches in different pages • CLI Example: apply the DBBP bundle patch to a database home: https://docs.cloud.oracle.com/iaas/Content/Database/References/dbacli.htm#updatedbhome
- 51. calero@H6RMYZ1:~$ oci db version list -c $COMPID --db-system-shape "VM.Standard2.4" --all --output table +-----------------------------+--------------+-----------------+ | is-latest-for-major-version | supports-pdb | version | +-----------------------------+--------------+-----------------+ | True | False | 11.2.0.4 | | False | False | 11.2.0.4.181016 | | False | False |11.2.0.4.190115 | | False | False |11.2.0.4.190416 | | True | True | 12.1.0.2 | | False | True | 12.1.0.2.181016 | | False | True |12.1.0.2.190115 | | False | True |12.1.0.2.190416 | | True | True | 12.2.0.1 | | False | True | 12.2.0.1.181016 | | False | True |12.2.0.1.190115 | | False | True |12.2.0.1.190416 | | True | True | 18.0.0.0 | | False | True | 18.4.0.0 | | False | True |18.5.0.0 | | False | True |18.6.0.0 | | True | True |19.0.0.0 | | False | True |19.3.0.0.190416 | +-----------------------------+--------------+-----------------+ DBCS versions available (August vs April 2019) +-----------------+ | version | +-----------------+ | 11.2.0.4 | | 11.2.0.4.180417 | | 11.2.0.4.180717 | | 11.2.0.4.181016 | | 12.1.0.2 | | 12.1.0.2.180417 | | 12.1.0.2.180717 | | 12.1.0.2.181016 | | 12.2.0.1 | | 12.2.0.1.180417 | | 12.2.0.1.180717 | | 12.2.0.1.181016 | | 18.0.0.0 | | 18.2.0.0 | | 18.3.0.0 | | 18.4.0.0 | +-----------------+
- 52. # patching cliadm update-dbcli dbcli list-jobs dbcli describe-component dbcli describe-latestpatch # patch GI dbcli update-server dbcli describe-job -i xxx # patch DB dbcli list-dbhomes dbcli update-dbhome -i xxx2 (from list-dbhomes) dbcli describe-job -i xxx3 (from update) # one-off Use opatch as usual 52 © 2019 Pythian Services DBCS patching from CLI – VM and BM https://docs.cloud.oracle.com/iaas/Content/Database/Tasks/patchingDB.htm
- 53. # list available patches dbaascli patch db list --oh exacs-node1:/u02/app/oracle/product/18.0.0.0/dbhome_1 # Validate patch prereqs dbaascli patch db prereq --patchid 12345678 --dbnames MYDB # Apply a patch # NOTE: you run the SQL part only on the last node dbaascli patch db apply --patchid 12345678 --dbnames MYDB --run_datasql 1 # rollback a patch dbaascli patch db switchback --patchid 12345678 --dbnames MYDB --run_datasql 1 --instance1 exacs-node1:/u02/app/oracle/product/18.0.0.0/dbhome_1 53 DBCS patching from CLI - Exadata CS https://docs.cloud.oracle.com/iaas/Content/Database/Tasks/exapatching.htm Note exadbcpatchmulti utility has been incorporated into dbaascli
- 54. Built-in automation: OCI DB backups 54 © 2019 Pythian Services • BM and VM: – Managed by Oracle: OCI to Object storage – unmanaged: RMAN or dbcli to local storage (BM only) https://docs.cloud.oracle.com/iaas/Content/Database/Tasks/backingup.htm • Exadata – Managed by Oracle: Object storage and local FRA disk https://docs.cloud.oracle.com/iaas/Content/Database/Tasks/exabackingup.htm – unmanaged: RMAN or dbcli to local storage • logs: /opt/oracle/dcs/log/<nodename>/rman/bkup/<db_unique_name>
- 55. What’s next? • Explore the APIs/CLI for all Oracle cloud services • Automate infrastructure changes with CI/CD pipelines • Take advantage of Oracle Management Cloud – more than just monitoring: https://cloud.oracle.com/management • Explore hybrid cloud deployments to expand your on-premises capacity for specific use cases 55 © 2019 Pythian Services
- 56. THANK YOU Questions? 56 [email protected] @ncalerouy http://www.linkedin.com/in/ncalero © 2019 Pythian Services
- 57. More terraform examples Creating a kubernetes cluster https://github.com/oracle/terraform-kubernetes-installer Same account configuration as previous OCI instance example, but also: • Policy at root compartment – allow service OKE to manage all-resources in tenancy • A group to which a policy grants appropriate permissions • command line tool kubectl https://docs.cloud.oracle.com/iaas/Content/ContEng/Concepts/contengprerequisites.htm NOTE: Only cluster setup – need an application to consume it 57 © 2019 Pythian Services
- 58. References - documentation Oracle Cloud tools: http://www.oracle.com/technetwork/topics/cloud/downloads/index.html OCI CLI examples: https://github.com/oracle/oci-cli/tree/master/scripts/examples Oracle Cloud API: https://docs.cloud.oracle.com/iaas/api/#/ Oracle Cloud Compute shapes: https://docs.cloud.oracle.com/iaas/Content/Compute/References/computeshapes.htm https://cloud.oracle.com/en_US/compute-classic/pricing EBS cloud admin tool https://blogs.oracle.com/ebsandoraclecloud/june-2018-ebs-cloud-admin-tool-updates-available Using Terraform with OCI: https://community.oracle.com/docs/DOC-1019936 OCI Examples used in this presentation: https://github.com/ncalero-uy/conferences/blob/master/oci-cli-demo-oow19.txt 58 © 2019 Pythian Services