Oracle RAC Presentation at Oracle Open World
Download as PPT, PDF3 likes1,549 views
The document details an evaluation and implementation project for Oracle on Linux at British Airways, highlighting various technical challenges and configuration issues encountered during the integration. It discusses database consolidation strategies, design patterns, storage solutions, and testing areas, alongside achievements such as creating reusable patterns and addressing key bugs. The project aims to improve resilience, reduce costs, and streamline infrastructure, with a roadmap for further consolidation and enhancements in database systems.
![Contact Details Imtiaz Mazhary Technical Architect, Infrastructure Design [email_address] Paul Marden Project Manager, Infrastructure Delivery [email_address]](https://image.slidesharecdn.com/20071114-oracle-rac-for-oow-rev-2-1229197984693981-1/85/Oracle-RAC-Presentation-at-Oracle-Open-World-25-320.jpg)
Oracle RAC Presentation at Oracle Open World
- 2. Oracle on Linux at BA From Evaluation To Implementation 15 th November 2007
- 3. Imtiaz Mazhary Technical Architect Paul Marden Project Manager
- 4. little teaser … SQL> create tablespace IMTEST_TS datafile '+DGTEST' size 10m; create tablespace IMTEST_TS datafile '+DGTEST' size 10m * ERROR at line 1: ORA-01119: error in creating database file '+DGTEST' ORA-17502: ksfdcre:4 Failed to create file +DGTEST ORA-15001: diskgroup "DGTEST" does not exist or is not mounted SQL> create diskgroup DGTEST 2 normal redundancy 3 failgroup DGTEST_BH 4 disk '/dev/raw/raw1001' name DGTEST_BH_DISK1001 5 failgroup DGTEST_CB 6 disk '/dev/raw/raw2001' name DGTEST_CB_DISK2001; Diskgroup created.
- 5. little teaser … SQL> create tablespace IMTEST_TS datafile '_DGTEST' size 10m; Tablespace created. SQL> !ls -lart $ORACLE_HOME/dbs | grep DGTEST -rw-rw---- 1 oracle oinstall 1056768 Sep 4 12:32 _DGTEST Bug 6391521 RAC DATABASE DATAFILE CREATION ALLOWS SPECIFICATION OF NON-SHARED STORAGE
- 6. Oracle on Linux Evaluation Proposition Cost reduction Improved resilience Simplification People Infrastructure Design & Delivery DBA Teams Linux Infrastructure Oracle RAC Pack, IBM, Red Hat, Scaleabilities
- 7. Oracle on Linux Evaluation Platform Design Stage Design Patterns Technology Stack Areas of Testing List of Issues Deliverables BA, Oracle and IBM Validated Configuration
- 8. Design Patterns – Database Tier SAN Storage DB01 Service X Basic Pattern Application X
- 9. Design Patterns – Database Tier SAN Storage Application X DB10R1 Service X DB10R2 Service X RAC Lite Pattern
- 10. Design Patterns – Database Tier SAN Storage DB10R1 Service X DB10R2 RAC Lite Pattern for Consolidation Application Y Application X Service Y Service X Service Y Service Y Service X
- 11. Design Patterns – Database Tier Application X DB20R1 Service X DB20R4 DB20R2 Service X DB20R3 Service X Service X Full RAC Pattern SAN Storage
- 12. Technology Stack IBM SDD v’s RHEL Device Mapper 3 rd Voting Disk on NFS
- 13. Areas of Testing Operating Environment RHEL 4.5 – CFS, LVM2, Device Mapper (and IBM SDD) SAN Attachment – QLogic vs RHEL QLogic driver RHEL 4.5 Cluster Suite BMC Patrol, EMC Legato Networker, IBM MQ Series, Control-M . . . . Oracle DB 10g R2 (142 tests) Serviceability – build, patching & configuration Clusterware resilience – interconnect, OCR & voting disks ASM configuration ASM resilience & recovery Stretched cluster performance Client connectivity & failover Backup & recovery, using EMC Legato Networker System Monitoring, using BMC Patrol EM Grid Control
- 14. List of key issues – all fixed in 10.2.0.3.0 ` Similar to the above 5621599.992 OCR synchronisation behaviour OCR storage too small when replacing the failed OCR disk 5698198.992 Failure to replace OCR device Bug – back port issue 5682851.993 DBVerify Segmentation Fault Internal RAID array firmware issue? 1768384 (RH) "read only filesystem" error Feature - to prevent OCR data loss 5593646.992 Loss of one OCR causes CW errors Bug - CSS error when processing stale member data requests 5594045.992 ASM & CW crashes/reboots on loss of one storage array Bug - Block corruptions when dropping & adding failure groups 5556890.993 ASM block corruption rootaddnode.sh using incorrect VIP 5449529.993 Problems adding additional node Bug – RHEL libaio driver 5473171.993 885818 (RH) Kernel panic caused by aio_put_req Use of NFS for 3 rd voting disk 5527179.993 CLSFMT error on CW install
- 15. Evaluation Project Deliverables Tested reusable patterns for use across the airline Supplier verification of our infrastructure - Oracle Validated Configuration undertaken by: IBM Oracle Red Hat OVC link: http://www.oracle.com/technology/tech/linux/validated-configurations/index.html
- 16. Database Consolidation Proposition reduce the size of Oracle DB estate cost reduction / simplification first RAC Lite and Basic Pattern – learning exercise Process LEAN - Runners / Repeaters pre-provisioning People - internal, RAC Pack IT - criteria, examples, experiences
- 17. FTD Batch (lxlhrl214) OAW Batch (lxlhrl213) Database Service - RAC Lite Pattern BH CB Batch/Feed Service (Linux HA) Node X ConMan Batch Node Y VIP X RACDBR1 VIP Y DGDATA_BH DGFRA_BH Public Network RAC Network Storage Network Database Consolidation - architecture DGDATA_CH DGFRA_CH ConMan Batch RACDBserviceFAST RACDBserviceB RACDBserviceC RACDBserviceD EXT3 RACDBR2 RACDBserviceC RACDBserviceD RACDBserviceFAST RACDBserviceB FAST Batch FAST Batch VIP X Database Service Basic Pattern DB1 DBservice_ConMan DBservice2 DBservice3 … . DB2 DBservice1 DBservice2 … . RACDBserviceFAST RACDBserviceB
- 18. Database Consolidation – Progress so far Current Production Environment: 30+ applications consolidated 3 on RAC Lite Capacity provisioned for 20 more Experiences Very few technical issues with infrastructure Mainly classic problems associated with migration
- 19. Database Consolidation - Criteria Any 4GB or less 2GB or less Memory (SGA + User) Any Greater than 3,600 per hour Less than 3,600 per hour Business transactions Any Less than 50 10 or less Concurrent clients (cpu constraint) Any Less than 50% Less than 25% Overall server capacity (cpu + memory constraint) No Yes or No Yes 3 rd party supplier contract does preclude database consolidation OLTP or Warehouse / DSS OLTP OLTP Transaction type IBM xSeries (standard BA standard model) or other Intel platform IBM xSeries (BA standard model) IBM xSeries (BA standard model) Hardware RHEL4 64 bit or other RH release RHEL4 64 bit RHEL4 64 bit Operating platform Full RAC pattern or other Basic or RAC Lite Pattern Basic or RAC Lite pattern Pattern type Any Greater than 60GB 60GB or less Data size (storage constraint) Oracle 10g RAC Oracle 10g DB or Oracle 10g RAC Oracle 10g DB or Oracle 10g RAC Database Stranger Repeater Runner
- 20. DW to Full RAC Pattern Proposition first Full RAC implementation – learning exercise important but not key business component Process - LEAN -- Strangers People - internal, NIIT, RAC Pack, Scaleabilities IT Where are we coming from? What does the cluster look like? Gotchas / lessons learnt?
- 21. Legacy platform Old SUN ES6500, 16*400MHz CPUs, 20GB RAM Solaris 8, Oracle 9.2 1.6 TB Database …. Operational Activity Warehouse & FTD 300+ batch feed jobs Oracle Forms 6! MQ Series licence costs for 16 CPUs Severe performance problems No HA
- 22. FTD Batch (lxlhrl214) OAW Batch (lxlhrl213) FICO DCS StaffAdmin Baggage … .. … .. WA21R Database - Full RAC Pattern VIP1 BH CB Batch/Feed Service (Linux HA) Node X OAW Batch (lxlhrl213) FTD Batch (A) Node Y FTD Batch (lxlhrl214) OAW Batch (A) VIP2 DGDATA_CB DGFRA_CB DGDATA_BH DGFRA_BH Public Network RAC Network Storage Network Target Platform WA21R1 WAPRD_BATCH WAPRD_APP FDPRD_APP WA21R2 FDPRD_APP WAPRD_APP WAPRD_BATCH VIP3 WA21R3 WAPRD_APP FDPRD_APP FDPRD_BATCH VIP4 WA21R4 FDPRD_BATCH FDPRD_APP WAPRD_APP
- 23. DW to Full RAC Pattern Proposition first Full RAC implementation – learning exercise important but not key business component Process - LEAN -- Strangers People - internal, NIIT, RAC Pack, Scaleabilities IT Where are we coming from? What does the cluster look like? Gotchas / lessons learnt?
- 24. Roadmap More consolidation on RAC 100 databases in FY0708 Key Business Systems move to RAC Orders Data Store Q1-2008 ba.com FY0809 SAP for BA Engineering ? Mainframe like availability More about people than technology Fast Recovery – Oracle Data Guard
- 25. Contact Details Imtiaz Mazhary Technical Architect, Infrastructure Design [email_address] Paul Marden Project Manager, Infrastructure Delivery [email_address]