SlideShare a Scribd company logo
Step 1 (Create Stream Directory):
I recommend this be kept in a central location for all your jobs. It will make recovery and trouble
shooting much easier. For this Stream I created SHDRAD in /opt/scripts/JobStream

Step 2 (Create Sub-Directories):
bin = Where your launcher scripts are kept (required)
log = Where stdout and stderr are redirected (jobStream.sh can create this on the fly if
necessary)
run = Where the lock file for jobStream.sh is stored during its run this prevents the same job from
running multiple times if the first stream has not finished.
tmp = Where stdout and stderr are redirected during the run then moved to log
Only bin is requied to be created up front because you have to have a place to store your
launcher scripts, the others can be created by jobStream.sh if necessary, but is recommended to
create them up front to avoid failures.

james@localhost:/opt/scripts/JobStream> cd SHDRAD
james@localhost:/opt/scripts/JobStream/SHDRAD> ll
total 4
-rw-r--r-- 1 james users         231 Oct 11 15:49 SHDRAD.lst
drwxrwxr-x 2 james users            96 Oct 11 15:42 bin
-rw-r--r-- 1 james users          21 Oct 11 15:50 contacts.lst
drwxrwxr-x 2 james users            96 Oct 11 15:39 log
drwxrwxr-x 2 james users            96 Oct 11 15:39 run
drwxrwxrwx 2 james users             96 Oct 11 15:39 tmp

Step 3 (Create the Launcher scripts):
Launcher scripts can be simple commands or complex shell scripts. Here are the ones I created
for SHDRAD.
james@localhost:/opt/scripts/JobStream/SHDRAD> cd bin
james@localhost:/opt/scripts/JobStream/SHDRAD/bin> ll
total 4
-rwxrwx--- 1 james users           147 Oct 11 15:46 cleanLog.sh
-rwxrwx--- 1 james users            64 Oct 11 15:41 launchSHDRAD.sh

The obvious one is launchSHDRAD.sh, which launches the rc.startjob for executing the Baan job
SHDRAD.
james@localhost:/opt/scripts/JobStream/SHDRAD/bin> cat launchSHDRAD.sh
#!/bin/sh
BSE=/msl4/baan;
/opt/etc/rc.startjob SHDRAD;
exit $?;

Since jobStream.sh stores all log files from all runs I created a simple cleanup program for
SHDRAD, which will run if SHDRAD completes successfully, otherwise it will not run at all (I'll
explain that in a bit).
james@localhost:/opt/scripts/JobStream/SHDRAD/bin> cat cleanLog.sh
#!/bin/sh
MYLOG=/opt/script/JobStream/SHDRAD/log;
cd $MYLOG;
find . ( -f -ctime +30 ) -exec rm -f {} ;
ERR=$?;
cd -;
exit $ERR;
Step 4 (Create Job List):
I created the job list for SHDRAD directly under the main directory for the stream.
james@localhost:/opt/scripts/JobStream> cd SHDRAD
james@localhost:/opt/scripts/JobStream/SHDRAD> ll
total 4
-rw-r--r-- 1 james users           231 Oct 11 15:49 SHDRAD.lst
drwxrwxr-x 2 james users              96 Oct 11 15:42 bin
-rw-r--r-- 1 james users            21 Oct 11 15:50 contacts.lst
drwxrwxr-x 2 james users              96 Oct 11 15:39 log
drwxrwxr-x 2 james users              96 Oct 11 15:39 run
drwxrwxrwx 2 james users               96 Oct 11 15:39 tmp

It looks similar to a crontab file. It is a simple space delimited file with the following format:
Step#, Success#, Fail#, Script
Where:
Step# is a unique integer from 1-? that identifies each step (note two different steps can launch
the same program if desired)
Success# is the Step# to execute on a success condition. The number 0 means do nothing, and
whatever the last step is in the stream this value is ignored.
Fail# is just like Success# except it is for Failures. Failures are also handled differently in that a 0
in its Failure# will result in notification being sent out, whether via the log or via email is your
choice.
And finally Script is the full path to the launcher that you want to run.

For SHDRAD it looks like this:
james@localhost:/opt/scripts/JobStream/SHDRAD> cat SHDRAD.lst
# Step 1 executes a wrapper for rc.startjob SHDRAD
1 2 0 /opt/scripts/JobStream/SHDRAD/bin/launchSHDRAD.sh
# If successful execute logfile cleanup
2 0 0 /opt/scripts/JobStream/SHDRAD/bin/cleanLog.sh

What this says is that the launchSHDRAD.sh script will launch as Step 1, if it succeds it then runs
cleanLog.sh, otherwise it will send notification of its failure (coming up). Since CleanLog.sh is the
last step the success and failure conditions do not matter. If you wanted to ignore cleanLog's
failures you could implement a dummy script that always exited with a good status and place it in
this file as CleanLog's failure step.

Step 5 (Set up Contacts - optional):
The contact list is nothing more than a list of email addresses. For SHDRAD I created the
following:
james@localhost:/opt/scripts/JobStream/SHDRAD> cat contacts.lst
gjames@celestica.com

By default jobStream.sh ignores comments (lines beginning with #). This also applies the the Job
list file as well. This allows you to comment out lines without having to delete them, in case you
need them in the future.

Step 6 (Schedule the JobStream in cron):
To accomplish this step and to keep your crontab file as clean as possible I recommend wrapping
the execution of jobStream.sh and its parameters.

        Part 1 (Create file):
        In order to do this I created a directory in james's home directory called JobStreams
        james@localhost:/export/home/testuser> mkdir JobStreams
Then I created the file.
        james@localhost:/export/home/testuser> vi JobStreams/shdrad.sh
        "JobStreams/shdrad.sh" [New file]
        #!/bin/sh
        SDIR=/opt/scripts/JobStream/SHDRAD;
        EXEC=/opt/scripts/JobStream/jobStream.sh;
        lfil="$SDIR/SHDRAD.lst";
        cfil="$SDIR/contact.lst";
        $EXEC -l $lfil -n $cfil -p $SDIR -N SHDRAD_Stream;

        I used variables in the command line to keep things clean on the screen.

        Part 2 (Modify Cron):
        james@localhost:/export/home/testuser> crontab -l > crontab
        james@localhost:/export/home/testuser> vi crontab
        "crontab" 3 lines, 148 characters
        * 20 * * * /opt/scripts/compressadmn.sh
        # to compress and overwrite the previous days maxi log.
        #* 1 * * * /opt/log/compress crontmp7
        #0 7 * * * /export/home/testuser/JobStreams/shdrad.sh
        james@localhost:/export/home/testuser> crontab crontab

        I always follow these steps to insure that I don't mess up the crontab file. You'll not that
my entry for shdrad.sh is
        commented out till we are ready to start using it.

Misc.
The script jobStream.sh resides in /opt/scripts/JobStream:
james@localhost:/export/home/testuser> cd /opt/scripts/JobStream
james@localhost:/opt/scripts/JobStream> ll
total 48
drwxr-xr-x 6 james users            1024 Oct 11 15:50 SHDRAD
-rw-r----- 1 james    users         4974 Oct 11 11:24 README
drwxr-xr-x 6 james       users        1024 Oct 11 11:40 StreamTest
-rwxr-x--- 1 james     users        16435 Oct 11 15:56 jobStream.sh

The README has some documentation in it as well.

More Related Content

DOCX
Linux crontab
Teja Bheemanapally
 
PPTX
Job Automation using Linux
Jishnu Pradeep
 
ODP
Cron
Iryney Baran
 
ODP
Babitha.linux
banubabitha
 
ODP
Babitha.linux
banubabitha
 
PDF
Basic linux commands
Raghav Arora
 
PDF
Linux talk | scheduled tasks
YashwantVarma1
 
PDF
FunctionalJS - May 2014 - Streams
darach
 
Linux crontab
Teja Bheemanapally
 
Job Automation using Linux
Jishnu Pradeep
 
Babitha.linux
banubabitha
 
Babitha.linux
banubabitha
 
Basic linux commands
Raghav Arora
 
Linux talk | scheduled tasks
YashwantVarma1
 
FunctionalJS - May 2014 - Streams
darach
 

What's hot (13)

PDF
Linux fundamental - Chap 14 shell script
Kenny (netman)
 
ODP
Prabu linux
Prabu Cse
 
DOCX
lec4.docx
ismailaboshatra
 
PPT
Linux directory commands:more options on cd and ls command
bhatvijetha
 
PPTX
Code tacoma command_line
Andrea Urban
 
PPT
Linux commands part3
bhatvijetha
 
PPT
3.1.c apend scripting, crond, atd
Acácio Oliveira
 
PDF
A journey through the years of UNIX and Linux service management
Lubomir Rintel
 
PPTX
Linux commands part -2
bhatvijetha
 
DOCX
Samba 4 - debian instalacao
Eduardo Charquero
 
PDF
Vi Editor
Shiwang Kalkhanda
 
PPT
Basic command ppt
Rohit Kumar
 
DOCX
Basic linux commands
Dheeraj Nambiar
 
Linux fundamental - Chap 14 shell script
Kenny (netman)
 
Prabu linux
Prabu Cse
 
lec4.docx
ismailaboshatra
 
Linux directory commands:more options on cd and ls command
bhatvijetha
 
Code tacoma command_line
Andrea Urban
 
Linux commands part3
bhatvijetha
 
3.1.c apend scripting, crond, atd
Acácio Oliveira
 
A journey through the years of UNIX and Linux service management
Lubomir Rintel
 
Linux commands part -2
bhatvijetha
 
Samba 4 - debian instalacao
Eduardo Charquero
 
Basic command ppt
Rohit Kumar
 
Basic linux commands
Dheeraj Nambiar
 
Ad

Similar to Example Stream Setup (20)

PPT
101 apend. scripting, crond, atd
Acácio Oliveira
 
PDF
RHEL in Shell Scripting Advanced - RHCSA+.pdf
RHCSA Guru
 
PDF
Unix for developers
Mite Mitreski
 
PDF
Unit 10 investigating and managing
root_fibo
 
PDF
3.1.d manual bash script guide lsstv 2.0r11
Acácio Oliveira
 
PPTX
Linux automated tasks
yarden hanan
 
PDF
Course 102: Lecture 16: Process Management (Part 2)
Ahmed El-Arabawy
 
PDF
Course 102: Lecture 10: Learning About the Shell
Ahmed El-Arabawy
 
PDF
Shell scripting
Geeks Anonymes
 
PPTX
PASS 24HOP Linux Scripting Tips and Tricks
Kellyn Pot'Vin-Gorman
 
PPTX
unit_1_foss_2.pptxbfhfdhfgsdgtsdegtsdtetg
zmulani8
 
PPTX
LINUX_admin_commands.pptx
GuhanSenthil2
 
PPTX
Daemons
christina555
 
PPTX
Ordina Accelerator program 2019 - Linux
Bert Koorengevel
 
PPTX
how to create an auto-extractible shell script
Thierry Gayet
 
PDF
linux_Commads
tastedone
 
PDF
unix_ref_card.pdf
GiovaRossi
 
PDF
unix_ref_card.pdf
GiovaRossi
 
PDF
unix_ref_card.pdf
GiovaRossi
 
PPTX
2009 cluster user training
Chris Dwan
 
101 apend. scripting, crond, atd
Acácio Oliveira
 
RHEL in Shell Scripting Advanced - RHCSA+.pdf
RHCSA Guru
 
Unix for developers
Mite Mitreski
 
Unit 10 investigating and managing
root_fibo
 
3.1.d manual bash script guide lsstv 2.0r11
Acácio Oliveira
 
Linux automated tasks
yarden hanan
 
Course 102: Lecture 16: Process Management (Part 2)
Ahmed El-Arabawy
 
Course 102: Lecture 10: Learning About the Shell
Ahmed El-Arabawy
 
Shell scripting
Geeks Anonymes
 
PASS 24HOP Linux Scripting Tips and Tricks
Kellyn Pot'Vin-Gorman
 
unit_1_foss_2.pptxbfhfdhfgsdgtsdegtsdtetg
zmulani8
 
LINUX_admin_commands.pptx
GuhanSenthil2
 
Daemons
christina555
 
Ordina Accelerator program 2019 - Linux
Bert Koorengevel
 
how to create an auto-extractible shell script
Thierry Gayet
 
linux_Commads
tastedone
 
unix_ref_card.pdf
GiovaRossi
 
unix_ref_card.pdf
GiovaRossi
 
unix_ref_card.pdf
GiovaRossi
 
2009 cluster user training
Chris Dwan
 
Ad

More from cfministries (17)

PDF
Grade Statistics
cfministries
 
PDF
R P G Generator
cfministries
 
PDF
Py Die R E A D M E
cfministries
 
PDF
Dealing W Spam
cfministries
 
PDF
Python Menu
cfministries
 
PDF
Elemental Christianity
cfministries
 
PDF
Peering Through The Mercy Seat
cfministries
 
PDF
Apostolic Private Eye
cfministries
 
PDF
Do You Make Jesus Sick
cfministries
 
PDF
Why Wait Get It Now
cfministries
 
PDF
The Mind Of God ( Part 5)
cfministries
 
PDF
The Mind Of God ( Part 4)
cfministries
 
PDF
5 Questions
cfministries
 
PDF
The Mind Of God ( Part 3)
cfministries
 
PDF
The Mind Of God ( Part 2)
cfministries
 
PDF
The Mind Of God ( Part 1)
cfministries
 
Grade Statistics
cfministries
 
R P G Generator
cfministries
 
Py Die R E A D M E
cfministries
 
Dealing W Spam
cfministries
 
Python Menu
cfministries
 
Elemental Christianity
cfministries
 
Peering Through The Mercy Seat
cfministries
 
Apostolic Private Eye
cfministries
 
Do You Make Jesus Sick
cfministries
 
Why Wait Get It Now
cfministries
 
The Mind Of God ( Part 5)
cfministries
 
The Mind Of God ( Part 4)
cfministries
 
5 Questions
cfministries
 
The Mind Of God ( Part 3)
cfministries
 
The Mind Of God ( Part 2)
cfministries
 
The Mind Of God ( Part 1)
cfministries
 

Example Stream Setup

  • 1. Step 1 (Create Stream Directory): I recommend this be kept in a central location for all your jobs. It will make recovery and trouble shooting much easier. For this Stream I created SHDRAD in /opt/scripts/JobStream Step 2 (Create Sub-Directories): bin = Where your launcher scripts are kept (required) log = Where stdout and stderr are redirected (jobStream.sh can create this on the fly if necessary) run = Where the lock file for jobStream.sh is stored during its run this prevents the same job from running multiple times if the first stream has not finished. tmp = Where stdout and stderr are redirected during the run then moved to log Only bin is requied to be created up front because you have to have a place to store your launcher scripts, the others can be created by jobStream.sh if necessary, but is recommended to create them up front to avoid failures. james@localhost:/opt/scripts/JobStream> cd SHDRAD james@localhost:/opt/scripts/JobStream/SHDRAD> ll total 4 -rw-r--r-- 1 james users 231 Oct 11 15:49 SHDRAD.lst drwxrwxr-x 2 james users 96 Oct 11 15:42 bin -rw-r--r-- 1 james users 21 Oct 11 15:50 contacts.lst drwxrwxr-x 2 james users 96 Oct 11 15:39 log drwxrwxr-x 2 james users 96 Oct 11 15:39 run drwxrwxrwx 2 james users 96 Oct 11 15:39 tmp Step 3 (Create the Launcher scripts): Launcher scripts can be simple commands or complex shell scripts. Here are the ones I created for SHDRAD. james@localhost:/opt/scripts/JobStream/SHDRAD> cd bin james@localhost:/opt/scripts/JobStream/SHDRAD/bin> ll total 4 -rwxrwx--- 1 james users 147 Oct 11 15:46 cleanLog.sh -rwxrwx--- 1 james users 64 Oct 11 15:41 launchSHDRAD.sh The obvious one is launchSHDRAD.sh, which launches the rc.startjob for executing the Baan job SHDRAD. james@localhost:/opt/scripts/JobStream/SHDRAD/bin> cat launchSHDRAD.sh #!/bin/sh BSE=/msl4/baan; /opt/etc/rc.startjob SHDRAD; exit $?; Since jobStream.sh stores all log files from all runs I created a simple cleanup program for SHDRAD, which will run if SHDRAD completes successfully, otherwise it will not run at all (I'll explain that in a bit). james@localhost:/opt/scripts/JobStream/SHDRAD/bin> cat cleanLog.sh #!/bin/sh MYLOG=/opt/script/JobStream/SHDRAD/log; cd $MYLOG; find . ( -f -ctime +30 ) -exec rm -f {} ; ERR=$?; cd -; exit $ERR;
  • 2. Step 4 (Create Job List): I created the job list for SHDRAD directly under the main directory for the stream. james@localhost:/opt/scripts/JobStream> cd SHDRAD james@localhost:/opt/scripts/JobStream/SHDRAD> ll total 4 -rw-r--r-- 1 james users 231 Oct 11 15:49 SHDRAD.lst drwxrwxr-x 2 james users 96 Oct 11 15:42 bin -rw-r--r-- 1 james users 21 Oct 11 15:50 contacts.lst drwxrwxr-x 2 james users 96 Oct 11 15:39 log drwxrwxr-x 2 james users 96 Oct 11 15:39 run drwxrwxrwx 2 james users 96 Oct 11 15:39 tmp It looks similar to a crontab file. It is a simple space delimited file with the following format: Step#, Success#, Fail#, Script Where: Step# is a unique integer from 1-? that identifies each step (note two different steps can launch the same program if desired) Success# is the Step# to execute on a success condition. The number 0 means do nothing, and whatever the last step is in the stream this value is ignored. Fail# is just like Success# except it is for Failures. Failures are also handled differently in that a 0 in its Failure# will result in notification being sent out, whether via the log or via email is your choice. And finally Script is the full path to the launcher that you want to run. For SHDRAD it looks like this: james@localhost:/opt/scripts/JobStream/SHDRAD> cat SHDRAD.lst # Step 1 executes a wrapper for rc.startjob SHDRAD 1 2 0 /opt/scripts/JobStream/SHDRAD/bin/launchSHDRAD.sh # If successful execute logfile cleanup 2 0 0 /opt/scripts/JobStream/SHDRAD/bin/cleanLog.sh What this says is that the launchSHDRAD.sh script will launch as Step 1, if it succeds it then runs cleanLog.sh, otherwise it will send notification of its failure (coming up). Since CleanLog.sh is the last step the success and failure conditions do not matter. If you wanted to ignore cleanLog's failures you could implement a dummy script that always exited with a good status and place it in this file as CleanLog's failure step. Step 5 (Set up Contacts - optional): The contact list is nothing more than a list of email addresses. For SHDRAD I created the following: james@localhost:/opt/scripts/JobStream/SHDRAD> cat contacts.lst [email protected] By default jobStream.sh ignores comments (lines beginning with #). This also applies the the Job list file as well. This allows you to comment out lines without having to delete them, in case you need them in the future. Step 6 (Schedule the JobStream in cron): To accomplish this step and to keep your crontab file as clean as possible I recommend wrapping the execution of jobStream.sh and its parameters. Part 1 (Create file): In order to do this I created a directory in james's home directory called JobStreams james@localhost:/export/home/testuser> mkdir JobStreams
  • 3. Then I created the file. james@localhost:/export/home/testuser> vi JobStreams/shdrad.sh "JobStreams/shdrad.sh" [New file] #!/bin/sh SDIR=/opt/scripts/JobStream/SHDRAD; EXEC=/opt/scripts/JobStream/jobStream.sh; lfil="$SDIR/SHDRAD.lst"; cfil="$SDIR/contact.lst"; $EXEC -l $lfil -n $cfil -p $SDIR -N SHDRAD_Stream; I used variables in the command line to keep things clean on the screen. Part 2 (Modify Cron): james@localhost:/export/home/testuser> crontab -l > crontab james@localhost:/export/home/testuser> vi crontab "crontab" 3 lines, 148 characters * 20 * * * /opt/scripts/compressadmn.sh # to compress and overwrite the previous days maxi log. #* 1 * * * /opt/log/compress crontmp7 #0 7 * * * /export/home/testuser/JobStreams/shdrad.sh james@localhost:/export/home/testuser> crontab crontab I always follow these steps to insure that I don't mess up the crontab file. You'll not that my entry for shdrad.sh is commented out till we are ready to start using it. Misc. The script jobStream.sh resides in /opt/scripts/JobStream: james@localhost:/export/home/testuser> cd /opt/scripts/JobStream james@localhost:/opt/scripts/JobStream> ll total 48 drwxr-xr-x 6 james users 1024 Oct 11 15:50 SHDRAD -rw-r----- 1 james users 4974 Oct 11 11:24 README drwxr-xr-x 6 james users 1024 Oct 11 11:40 StreamTest -rwxr-x--- 1 james users 16435 Oct 11 15:56 jobStream.sh The README has some documentation in it as well.