![SQL query script
#!/bin/bash
SPOOL01=/tmp/spoolfile_01.lst
DISTRIBUTION=“myemail@someplace.com”
[ $SPOOL01 ] && rm –f ${SPOOL01}
sqlplus / as sysdba <<EOF
spool ${SPOOL01}
SELECT grantee
FROM dba_role_privs
WHERE granted_role = „DBA‟;
spool off
exit
EOF
mailx –s “DBA grantees” ${DISTRIBUTION} <${SPOOL01}
[ $SPOOL01 ] && rm –f ${SPOOL01}](https://image.slidesharecdn.com/2010smithscripting101-1272222800981-phpapp02/85/2010-Smith-Scripting101-26-320.jpg)
![Emphasize Visual Flow
for thisHOST in `cat ${HOSTLIST}`; do
if [ ${#thisHOST} -gt 5 ]; then
echo "BIG: ${thisHOST} is ${#thisHOST} characters"
else
if [ ${#thisHOST} -lt 3 ]; then
echo "LITTLE: ${thisHOST} is ${#thisHOST} characters"
fi
fi
done
for thisHOST in `cat ${HOSTLIST}`; do
if [ ${#thisHOST} -gt 5 ]; then
echo "BIG: ${thisHOST} name is long"
else
if [ ${#thisHOST} -lt 3 ]; then
echo "LITTLE: ${thisHOST} name is short"
fi
fi
done](https://image.slidesharecdn.com/2010smithscripting101-1272222800981-phpapp02/85/2010-Smith-Scripting101-35-320.jpg)
![Command Line Values
#!/bin/bash ${0} ${1} ${2}
thisSCRIPT=$0 > backup_db.sh silver hot
ORACLE_SID=$1
BU_TYPE=$2
echo “Executing ${thisSCRIPT} for ${thisSID}”
if [ ${BU_TYPE} == hot ]; then
echo “tRunning a hot backup”
TestForArchiveMode
else
RunColdBackup
fi](https://image.slidesharecdn.com/2010smithscripting101-1272222800981-phpapp02/85/2010-Smith-Scripting101-40-320.jpg)
![Command Line Illustration
#!/bin/bash
# file: input_variables.sh
if [ $1 ] ; then
echo "Amaze your $1"
else
echo “provide input, pathetic human!"
fi
>
> ./input_variables.sh friends
> Amaze your friends](https://image.slidesharecdn.com/2010smithscripting101-1272222800981-phpapp02/85/2010-Smith-Scripting101-41-320.jpg)
![Make the Machine do the Work
• Create everything you need, every time
– Fix permissions too
– Before and after pictures = Acts of Kindness
if [ ! -d $thisDIR ]; then
mkdir $thisDIR
fi
echo “Directory before:”
ls –l
chmod 775 $thisDIR
echo “Directory after:”
ls -l](https://image.slidesharecdn.com/2010smithscripting101-1272222800981-phpapp02/85/2010-Smith-Scripting101-43-320.jpg)
2010 Smith Scripting101
- 1. Linux Shell Scripting 101 Plumbing for DBAs Ray Smith Portland General Electric Copyright 2010 Raymond S. Smith
- 2. Agenda • Why Plumbing • Tools • Fittings and Parts • Procedures • Leaks and Repairs
- 3. The Plumbing Metaphor • Part of the Unix tradition – 40 year track record • Part of the lexicon – Pipes | – Redirects > – Input and Output – Buckets and spools
- 4. Part 1: Tools • Books and man pages • Spare parts • Stuff in the truck • Other plumbers
- 5. Most Important Tool: Nutshells
- 6. Handbooks Ken O. Burtch Anatole Olczak
- 7. Steal, Adapt, Repeat • If it works – Find out why it works – Test it for your problem – Apply it
- 8. Ask for Help • Hit the blog-o-sphere • Ask your SA – Have an example in hand • Ask other DBAs • Hit the man page David Korn (ksh)
- 9. Part 2 : Fittings and Parts • Input and Output • Pipes • Valves • Filters • Buckets
- 10. Input and Output • Unix Fundamental Rules – Simple, predictable interfaces – Text/string input expected – Text output anticipated – Something happens in between Text In Text Out
- 11. grep “string” grep –i grep –v cut –d: -f2 awk –F:„{print $2}‟
- 12. Redirect out example echo “Hello, world” > anyfile.txt cat anyfile.txt Hello, world echo “Collaborate OK?” >> anyfile.txt cat anyfile.txt Hello, world Collaborate OK?
- 13. Redirect ‘in’ example mailx –s “Collaborate” [email protected] < anyfile.txt
- 14. Subshell redirection sqlplus / nolog << EOF connect / as sysdba shutdown abort exit EOF • Subshell redirection – Opens a subshell to your shell script – Executes the commands – Exits when the matching ‘EOF’ is seen • EOF must be on left column of text file • Use this syntax for sqlplus, RMAN, etc
- 15. Shell scripting is pipe work echo “Hello, world” > anyfile.txt echo “Collaborate OK? >> anyfile.txt echo “Collaborate OK? >> anyfile.txt cat anyfile.txt | sort –u Collaborate OK? Hello, world cat anyfile.txt | grep OK Collaborate OK? cat anyfile.txt | grep OK | cut –f2 OK?
- 16. Part 3: Procedures • How to write a shell script – Define the output – Define the input – Fill in the gaps
- 17. oratab Only 10g hostname Skip comments Wanted: List of 10g databases Report
- 18. Sample oratab script #!/bin/bash WORKFILE=/tmp/workfile.lst REPORT=/tmp/oratab_report.rpt export HOSTNAME=`hostname` touch $REPORT cat /etc/oratab | grep “10.” | grep –v ^# >$WORKFILE for thisENTRY in `cat $WORKFILE`; do thisSID=`cat $thisENTRY | cut –d: –f1` echo “$thisSID is on $HOSTNAME” >>$REPORT done cat $REPORT
- 19. Sample oratab script #!/bin/bash WORKFILE=/tmp/workfile.lst •REPORT=/tmp/oratab_report.rpt Interpreter callout export HOSTNAME=`hostname` touch $REPORT cat /etc/oratab | grep “10.” | grep –v ^# >$WORKFILE for thisENTRY in `cat $WORKFILE`; do thisSID=`cat $thisENTRY | cut –d: –f1` echo “$thisSID is on $HOSTNAME” >>$REPORT done cat $REPORT
- 20. Sample oratab script #!/bin/bash WORKFILE=/tmp/workfile.lst REPORT=/tmp/oratab_report.rpt export HOSTNAME=`hostname` •touch $REPORT declaration Variables cat /etc/oratab | grepOnly forgrep –v ^# >$WORKFILE • No ‘export’ : “10.” | this script for thisENTRY ‘export’: Session variable • With in `cat $WORKFILE`; do thisSID=`cat $thisENTRY | cut –d: –f1` echo export is on $HOSTNAME” >>$REPORT “$thisSID WORKFILE=xyz.lst done cat $REPORT
- 21. Sample oratab script •#!/bin/bash ^# Grep -v WORKFILE=/tmp/workfile.lst • Dash v = ignore REPORT=/tmp/oratab_report.rpt export HOSTNAME=`hostname` • Caret = starts with touch $REPORT cat /etc/oratab | grep “10.” | grep –v ^# >$WORKFILE for thisENTRY in `cat $WORKFILE`; do #/etc/oratab thisSID=`cat $thisENTRY | cut –d: –f1` echo “$thisSID is on $HOSTNAME” >>$REPORT ##Production:Sales:18-6:10.2.0.4:Morgan done salesp:/oracle/product/10.2.0:Y cat##Production:Finance:24-7:11.1.0.7:Ames $REPORT finp:/oracle/product/11.2.0:Y
- 22. Do loops for thisENTRY in `cat $WORKFILE`; do thisSID=`cat $thisENTRY | cut –d: –f1` echo “$thisSID is on $HOSTNAME” >>$REPORT done • For something in something else • Do something • Quit when I say you’re done for thisENTRY in file1.txt file2.txt file3.txt; do thisSID=`cat $thisENTRY | cut –d: –f1` echo “$thisSID is on $HOSTNAME” >>$REPORT done
- 23. Manipulating strings w/ cut cat $thisENTRY | cut –d: –f1 • Cut the string using • Delimiter (-d) of colon • Return field (-f) -f1 salesp:/oracle/product/10.2.0:Y
- 24. Manipulating strings w/ awk cat $thisENTRY | awk –F: „{print $1} • Cut the string using • Delimiter (-F) of colon • Print first string ( print $1 ) ${1} salesp:/oracle/product/10.2.0:Y
- 25. Database No filter Email address Wanted: Oracle accounts with DBA role Mail
- 26. SQL query script #!/bin/bash SPOOL01=/tmp/spoolfile_01.lst DISTRIBUTION=“[email protected]” [ $SPOOL01 ] && rm –f ${SPOOL01} sqlplus / as sysdba <<EOF spool ${SPOOL01} SELECT grantee FROM dba_role_privs WHERE granted_role = „DBA‟; spool off exit EOF mailx –s “DBA grantees” ${DISTRIBUTION} <${SPOOL01} [ $SPOOL01 ] && rm –f ${SPOOL01}
- 27. RMAN script #!/bin/bash RMN_SCRIPT=/tmp/rman_script.rmn WORKFILE=/tmp/rman_workfile.lst DISTRIBUTION=“[email protected]” echo "connect target /" > ${RMN_SCRIPT} echo "connect catalog ${USER}/${PASS}@${RMANCAT}“ >>${RMN_SCRIPT} echo "resync catalog;" >>${RMN_SCRIPT} echo "exit;" >>${RMN_SCRIPT} rman @RMN_SCRIPT <<EOF | tee –a $WORKFILE exit EOF mailx –s “RMAN execution” ${DISTRIBUTION} <${SPOOL01}
- 28. Pipe to a Tee
- 29. OEM emcli script #!/bin/bash BO_NAME=scripted_blackout_${thisSID} echo "Creating blackout named '${BO_NAME}' ..." ${EMCLI} create_blackout -name="${BO_NAME}" -add_targets=${thisTARGET}:oracle_database -schedule="duration::360;tzinfo:specified;tzregion:America/Los_Angeles" -reason="Scripted blackout for maintenance or refresh" sleep 5 $ECHO "Getting blackout information for '${BO_NAME}' ..." ${EMCLI} get_blackout_details -name="${BO_NAME}"
- 30. Part 4: Leaks and Repairs Rule #1: SHELL SCRIPTS MUST WORK Rule #2: SCRIPTS MUST KEEP WORKING Design for maintenance – Clarity – Simplicity – Scalability
- 31. Predictable Layout • Consistent layout for all your scripts 1. Header block 2. Change log 3. Independent variables 4. Dependent variables 5. Functions 6. Run-time procedure • Take out the guesswork
- 32. Layout example #!/bin/bash #set -x ######################################################### # File : sample_script.sh # Parameter : Database name (optional) # Purpose : Amaze others ######################################################## #======================================================= # Independent variables #======================================================= export BASEDIR=/usr/lbin/orascripts #======================================================= # Dependent variables # Nothing to change below this line #======================================================= LOGDIR=$BASEDIR/logs WORKFILE=$LOGDIR/workfile.lst
- 33. Visual clarity
- 34. Just like a SQL statement … select distinct table_name,column_name from all_tab_columns where column_name like '%AGREE%' or column_name like '%ANN%„ or table_name like '%ANN%„ or table_name like „%STOR%„ order by table_name,column_name; SELECT DISTINCT table_name,column_name FROM all_tab_columns WHERE column_name LIKE '%AGREE%' OR column_name LIKE '%ANN%„ OR table_name LIKE '%ANN%' OR table_name LIKE '%STOR%' ORDER BY table_name,column_name;
- 35. Emphasize Visual Flow for thisHOST in `cat ${HOSTLIST}`; do if [ ${#thisHOST} -gt 5 ]; then echo "BIG: ${thisHOST} is ${#thisHOST} characters" else if [ ${#thisHOST} -lt 3 ]; then echo "LITTLE: ${thisHOST} is ${#thisHOST} characters" fi fi done for thisHOST in `cat ${HOSTLIST}`; do if [ ${#thisHOST} -gt 5 ]; then echo "BIG: ${thisHOST} name is long" else if [ ${#thisHOST} -lt 3 ]; then echo "LITTLE: ${thisHOST} name is short" fi fi done
- 36. The Penny Wise Quiz a. Shorter variable names = Less typing = Less work b. Obscure variable names = Reduced transparency = Poor quality • Save your cycles for decyphering the logic – Not the variable names
- 37. Shell script functions function SetPerms770 { echo “Setting permission to 770 for ${thisFILE}” chmod 770 ${thisFILE}/* } > for thisFILE in `cat ${FILELIST}`; do > SetPerms770 > done
- 38. function CopyFiles { cd $SOURCE_DIR ls -1 | grep -i $ORACLE_SID >$WORKFILE for thisFILE in `cat $WORKFILE`; do SOURCE_FILE=$SOURCE_DIR/$thisFILE TARGET_FILE=$TARGET_DIR/$thisFILE cp –f $SOURCE_FILE $TARGET_FILE done rm -f ${WORKFILE} } # ----------------------------------------------------------- # Run-time procedure # ------------------------------------------------------------ SOURCE_DIR=${GOLD_DIR}/rman TARGET_DIR=${LIVE_DIR}/rman CopyFiles
- 39. Scalability • Scalability goal #1: Never customize a script – Overuse variables – Never hardcode • Passwords • Host or database names • Paths – Use command-line input, ‘read’, or parameter files • Balance risk vs. maintenance cost
- 40. Command Line Values #!/bin/bash ${0} ${1} ${2} thisSCRIPT=$0 > backup_db.sh silver hot ORACLE_SID=$1 BU_TYPE=$2 echo “Executing ${thisSCRIPT} for ${thisSID}” if [ ${BU_TYPE} == hot ]; then echo “tRunning a hot backup” TestForArchiveMode else RunColdBackup fi
- 41. Command Line Illustration #!/bin/bash # file: input_variables.sh if [ $1 ] ; then echo "Amaze your $1" else echo “provide input, pathetic human!" fi > > ./input_variables.sh friends > Amaze your friends
- 42. No tools required • Nothing inside the tested script changes – No surprises – No customizations – One script across the enterprise • Securable – Does not require write permissions for others
- 43. Make the Machine do the Work • Create everything you need, every time – Fix permissions too – Before and after pictures = Acts of Kindness if [ ! -d $thisDIR ]; then mkdir $thisDIR fi echo “Directory before:” ls –l chmod 775 $thisDIR echo “Directory after:” ls -l