oozie部署

OOZIE配置文档

一、     下载

1.      OOZIE

git clone https://github.com/apache/oozie.git

2.      编译安装

mvn  -DskipTests=true  -P hadoop-2  clean  package  assembly:single

cd  oozie/distrotarget

有生成的tar包oozie-4.0.1-distro.tar.gz

3.      OOZIE配置文件

1.oozie-4.0.1/conf/oozie-site.xml

连接hue所需配置项

<property>

       <name>oozie.service.ProxyUserService.proxyuser.hue.hosts</name>

       <value>*</value>

       <description>

           List of hosts the '#USER#' user is allowed to perform 'doAs'

           operations.

           The '#USER#' must be replaced with the username o the user who is

           allowed to perform 'doAs' operations.

           The value can be the '*' wildcard or a list of hostnames.

           For multiple users copy this property and replace the user name

           in the property name.

       </description>

   </property>

   <property>

       <name>oozie.service.ProxyUserService.proxyuser.hue.groups</name>

       <value>*</value>

       <description>

           List of groups the '#USER#' user is allowed to impersonate users

           from to perform 'doAs' operations.

           The '#USER#' must be replaced with the username o the user who is

           allowed to perform 'doAs' operations.

           The value can be the '*' wildcard or a list of groups.

           For multiple users copy this property and replace the user name

           in the property name.

       </description>

   </property>

2.hadoop集群core-site.xml需要加入下面话并重启

<property>

 <name>hadoop.proxyuser.hdp.hosts</name>

 <value>*</value>

</property>

<property>

 <name>hadoop.proxyuser.hdp.groups</name>

 <value>*</value>

</property>

3.oozie-4.0.1/conf/hadoop-conf

把hadoop相关配置文件拷贝到oozie-4.0.1/conf/hadoop-conf目录下面

4.      OOZIE启动前准备

1.在oozie安装目录建立libext 把对应版本的hadoopjars拷贝进去，并放入mysql驱动，以及ext-2.2.zip。

2.在oozie/bin

./oozie-setup.sh prepare-war -d ../libext

3. /ooziedb.sh  create -sqlfile  oozie.sql

4. /oozie-setup.sh db create  -run -sqlfile oozie.sql

5.      OOZIE 启动

1. oozied.sh start

2. oozied.sh run

查看网页

http://localhost:11000/

6.      查看OOZIE状态

oozie admin -ooziehttp://localhost:11000/oozie -status  正常启动会显示normal

7.      oozie运行Job

解压oozie-examples.tar.gz和oozie-sharelib-4.0.1.tar.gz

把oozie/example 目录以及oozie/share  put 到hdfs

修改examples/apps/map-reduce/job.properties

nameNode=hdfs://spark-jrdata-07.pekdc1.jdfin.local:9000

jobTracker=spark-jrdata-08.pekdc1.jdfin.local:8032   ---resourcemanager hostname:port

queueName=default

examplesRoot=examples

oozie job -ooziehttp://localhost:11000/oozie -config examples/apps/map-reduce/job.properties –run

查看状态

oozie job -ooziehttp://localhost:11000/oozie -info