hadoop3.1.3集群搭建(ha+yarn)_hadoop3.1.3 ha-CSDN博客

当前环境：

centos6.5,jdk8

准备工作：

1.服务器之间免密登录

$ ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
$ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
$ chmod 0600 ~/.ssh/authorized_keys

2.服务器之间时间同步

3.安装zookeeper集群

搭建步骤：

1.下载apache hadoop3.1.3并上传至服务器解压

https://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-3.1.3/hadoop-3.1.3.tar.gz

2.环境变量设置：

export BASE_DIR=/data/br
export JAVA_HOME=$BASE_DIR/base/jdk1.8.0_181
export HADOOP_HOME=$BASE_DIR/base/hadoop
export ZOOKEEPER_HOME=/data/br/base/zookeeper

export HDFS_JOURNALNODE_USER=root
export HDFS_ZKFC_USER=root
export HDFS_NAMENODE_USER=root
export HDFS_DATANODE_USER=root
export HDFS_SECONDARYNAMENODE_USER=root
export YARN_NODEMANAGER_USER=root
export YARN_RESOURCEMANAGER_USER=root

export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$ZOOKEEPER_HOME/bin:$KAFKA_HOME/bin

3.hadoop-env.sh设置

export JAVA_HOME=/data/br/base/jdk1.8.0_181

4.core-site.xml配置

<configuration>
<property>
    <name>fs.defaultFS</name>
    <value>hdfs://mycluster</value>
  </property>
  <property>
    <name>hadoop.t