1. CDH概述
1.1 创作背景
由于公司目前所用的自动化管理工具使用的是CDH的收费版本,为了刚好对cdh这类软件运维工具有深刻理解,所以自己搭建了一套基于免费开源CDH 6.3.2版本的集群;
1.2 CDH介绍
Cloudera
版本(Cloudera’s Distribution Including Apache Hadoop
,简称CDH
),基于Web
的用户界面,支持大多数Hadoop
组件,包括HDFS
、MapReduce
、Hive
、Pig
、HBase
、Zookeeper
、Sqoop
,简化了大数据平台的安装、使用简单。
Cloudera Manager
的功能:
- 管理:对集群进行管理,如添加、删除节点等操作。
- 监控:监控集群的健康情况,对设置的各种指标和系统运行情况进行全面监控。
- 诊断:对集群出现的问题进行诊断,对出现的问题给出建议解决方案。
- 集成:多组件进行整合。
1.3 展示成果
图中是我最终部署后的效果,由于规划时硬盘和内存给的都太小了导致上面出现告警的情况,需要在部署前规划好硬盘和内存;
1.4 注意事项
安装过程中一定会存在很多的问题,需要认真排查问题,我解决问题不下几十个,有时候是网络问题,有些事配置问题,需要通过报错一步一步解决,我会尽力将遇到的问题一一排除,让想了解cdh的能快速搭建好这样一个平台;
2. 准备工作
2.1 前置准备
前提条件 | 基本信息 | 配置 | 备注 |
CentOS | Linux release 7.9.2009 | centos-7-isos-x86_64安装包下载_开源镜像站-阿里云 | 最小安装 |
4台虚拟机 (最好5台) | 192.168.2.186 192.168.2.187 192.168.2.188 192.168.2.189 | 最低配置要求 内存:4G 硬盘:40G 注意:要不然后面会存在硬盘不够用,节点硬盘太小导致的不健康提示 | |
JDK | oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm | ||
Cloud Manager | cm6.3.1-redhat7 | 阿里云盘分享 | |
mysql | mysql57-community-release-el7-11.noarch.rpm |
由于资料很大,只能通过云盘分享:阿里云盘分享,部分资料由于太大了网盘设限了;如果有需要单独留言联系
2.2 网络配置
#所有节点
#关闭防火墙
systemctl stop firewalld
systemctl disable firewalld
#设置ip和域名
vi /etc/hosts
192.168.2.186 hadoop101 hadoop101
192.168.2.187 hadoop102 hadoop102
192.168.2.188 hadoop103 hadoop103
192.168.2.189 hadoop104 hadoop104
#固定ip,防止重启时变化
vi /etc/sysconfig/network-scripts/ifcfg-ens33
IPADDR=192.168.2.186
NETMASK=255.255.255.0
GATEWAY=192.168.2.2
DNS1=8.8.8.8
DNS2=114.114.114.114
#重启网络
systemctl restart network
2.3 各个节点之间免密登录
#所有节点都执行
#生成公钥和私钥(敲三个回车,就会生成两个文件id_rsa(私钥)、id_rsa.pub(公钥))
[root@hadoop101 ~]$ ssh-keygen -t rsa
#进入.ssh目录
[root@hadoop101 ~]$ cd .ssh
[root@hadoop101 .ssh]$ ls
[root@hadoop101 .ssh]$ id_rsa id_rsa.pub
#复制公钥到另外一台服务器
[root@hadoop101 .ssh]$ ssh-copy-id hadoop101
[root@hadoop101 .ssh]$ ssh-copy-id hadoop102
[root@hadoop101 .ssh]$ ssh-copy-id hadoop103
[root@hadoop101 .ssh]$ ssh-copy-id hadoop104
2.4 集群同步脚本 xsync
#在/root目录下创建bin目录,并在bin目录下创建文件xsync,文件内容如下:
[root@hadoop101 ~]$ mkdir bin
[root@hadoop101 ~]$ cd bin/
[root@hadoop101 bin]$ vi xsync
#在该文件中编写如下代码
#!/bin/bash
#1 获取输入参数个数,如果没有参数,直接退出
pcount=$#
if((pcount==0)); then
echo no args;
exit;
fi
#2 获取文件名称
p1=$1
fname=`basename $p1`
echo fname=$fname
#3 获取上级目录到绝对路径
pdir=`cd -P $(dirname $p1); pwd`
echo pdir=$pdir
#4 获取当前用户名称
user=`whoami`
#5 循环
for((host=102; host<106; host++)); do
echo ------------------- hadoop$host --------------
rsync -av $pdir/$fname $user@hadoop$host:$pdir
done
#退出后,修改脚本 xsync 具有执行权限
[root@hadoop101 bin]$ chmod 777 xsync
2.5 集群整体操作脚本 xcall.sh
#在/root/bin目录下创建脚本xcall.sh
[root@hadoop101 bin]$ vim xcall.sh
#在脚本中编写如下内容
#! /bin/bash
for i in hadoop101 hadoop102 hadoop103 hadoop104 hadoop105
do
echo --------- $i ----------
ssh $i "$*"
done
#退出后,修改脚本执行权限
[root@hadoop101 bin]$ chmod 777 xcall.sh
3. 安装
3.1 安装JDK
#需要使用官网提供的jdk oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm
#在hadoop101的/opt目录下创建module和software文件夹
[root@hadoop101 opt]# mkdir module
[root@hadoop101 opt]# mkdir software
#上传oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm并安装
[root@hadoop101 software]# rpm -ivh oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm
[root@hadoop101 software]# vim /etc/profile.d/my_env.sh
export JAVA_HOME=/usr/java/jdk1.8.0_181-cloudera
export CLASSPATH=.:$CLASSPATH:$JAVA_HOME/lib
export PATH=$PATH:$JAVA_HOME/bin
[root@hadoop101 software]# source /etc/profile.d/my_env.sh
[root@hadoop101 software]# java -version
java version "1.8.0_181"
#分发,并source
[root@hadoop101 software]# xsync /usr/java/
[root@hadoop101 software]# xsync /etc/profile.d/my_env.sh
[root@hadoop102 software]# source /etc/profile.d/my_env.sh
[root@hadoop103 software]# source /etc/profile.d/my_env.sh
[root@hadoop104 software]# source /etc/profile.d/my_env.sh
[root@hadoop101 software]# xcall.sh java -version
3.2 安装MySQL
#安装包准备
[root@hadoop101 ~]# rpm -qa|grep -i mysql
[root@hadoop101 ~]#
#如果安装了MySQL,就先卸载
[root@hadoop101 ~]# rpm -e --nodeps #mysql-version
#删除阿里云原有MySql依赖
[root@hadoop101 ~]# yum remove mysql-libs
#下载MySql依赖并安装
[root@hadoop101 ~]# yum install libaio
[root@hadoop101 ~]# yum -y install autoconf
[root@hadoop101 ~]# wget https://dev.mysql.com/get/mysql57-community-release-el7-11.noarch.rpm
[root@hadoop101 ~]# rpm -ivh mysql57-community-release-el7-11.noarch.rpm
#缺省会在/etc/yum.repos.d目录下生成两个配置文件
[root@hadoop101 ~]# ll /etc/yum.repos.d | grep mysql
#安装MySQL服务器
[root@hadoop101 ~]# rpm --import https://repo.mysql.com/RPM-GPG-KEY-mysql-2022
[root@hadoop101 ~]# yum -y install mysql-community-server
#启动mysql并查看状态
[root@hadoop101 ~]# systemctl start mysqld
[root@hadoop101 ~]# systemctl status mysqld
#获取临时密码
[root@hadoop101 ~]# grep "password" /var/log/mysqld.log
#设置root密码
[root@hadoop101 ~]# mysql_secure_installation
#设置新的root密码,缺省安全策略是MEDIUM,要求至少包含8个字符,其中至少有一个大小写字母,一个特殊#字符,一个数字,否则报 ... Failed! Error: Your password does not satisfy the current #policy requirements异常。
#设置内容:
Estimated strength of the password: 100
Do you wish to continue with the password provided?(Press y|Y for Yes, any other key for No) : y
By default, a MySQL installation has an anonymous user,
allowing anyone to log into MySQL without having to have
a user account created for them. This is intended only for
testing, and to make the installation go a bit smoother.
You should remove them before moving into a production
environment.
# 移除匿名用户
Remove anonymous users? (Press y|Y for Yes, any other key for No) : y
Success.
Normally, root should only be allowed to connect from
'localhost'. This ensures that someone cannot guess at
the root password from the network.
# 禁用远程root登录
Disallow root login remotely? (Press y|Y for Yes, any other key for No) : y
Success.
By default, MySQL comes with a database named 'test' that
anyone can access. This is also intended only for testing,
and should be removed before moving into a production
environment.
# 删除测试数据库及权限
Remove test database and access to it? (Press y|Y for Yes, any other key for No) : y
- Dropping test database...
Success.
- Removing privileges on test database...
Success.
Reloading the privilege tables will ensure that all changes
made so far will take effect immediately.
# 重新加载授权表
Reload privilege tables now? (Press y|Y for Yes, any other key for No) : y
Success.
All done!
#修改安全策略
[root@hadoop101 ~]# mysql -uroot -p
# 查看当前安全策略
mysql>show variables like 'validate_password%';
# 最低安全策略(0->LOW ,1->MEDIUM, 2->STRONG)
mysql>set global validate_password_policy=0;
# 最小密码字符长度是4
mysql>set global validate_password_length=4;
# 设置本机root密码
mysql>alter user 'root'@'localhost' identified by 'root';
# 远程授权,all表示所有ddl和dml操作权限,*.*表示针对所有数据库所有表,%表示本机和远程主机均能访问
mysql>grant all privileges on *.* to 'root'@'%' identified by 'root' with grant option;
# 刷新权限
mysql>flush privileges;
#退出
mysql>exit
# 查询数据库列表
show databases
# 切换数据库
use mysql
# 查询数据库表
show tables
# 查询用户信息
select * from user
#设置开机自启动
[root@hadoop101 ~]# systemctl enable mysqld
3.3 CM安装部署
3.3.1 MySQL中建库
#在MySQL中创建各组件需要的数据库
[root@hadoop101 ~]# mysql -uroot -proot
mysql> GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY 'root';
mysql> CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
mysql> CREATE DATABASE hive DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
mysql> CREATE DATABASE oozie DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
mysql> CREATE DATABASE hue DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
mysql> CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
3.3.2 CM安装
#将mysql-connector-java-5.1.27-bin.jar上传到/usr/share/java路径下,并重命名mysql-connector-java.jar,分发出去
[root@hadoop101 mysql-connector-java-5.1.27]# xsync /usr/share/java/
3.3.3 集群规划
节点 | hadoop101 | hadoop102 | hadoop103 | Hadoop104 |
服务 | cloudera-scm-server cloudera-scm-agent | cloudera-scm-agent | cloudera-scm-agent | cloudera-scm-agent |
3.3.4 安装流程
#创建cloudera-manager目录,存放cdh安装文件
[root@hadoop101 mysql-connector-java-5.1.27]# mkdir /opt/cloudera-manager
[root@hadoop101 mysql-connector-java-5.1.27]# cd /opt/software/
[root@hadoop101 software]# tar -zxvf cm6.3.1-redhat7.tar.gz
[root@hadoop101 software]# cd cm6.3.1/RPMS/x86_64/
[root@hadoop101 x86_64]# mv cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm /opt/cloudera-manager/
[root@hadoop101 x86_64]# mv cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm /opt/cloudera-manager/
[root@hadoop101 x86_64]# mv cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm /opt/cloudera-manager/
[root@hadoop101 x86_64]# cd /opt/cloudera-manager/
[root@hadoop101 cloudera-manager]# ll
#安装cloudera-manager-daemons,安装完毕后多出/opt/cloudera目录
[root@hadoop101 cloudera-manager]# rpm -ivh cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm
[root@hadoop101 cloudera-manager]# cd /opt/
cloudera/ cloudera-manager/
[root@hadoop101 cloudera-manager]# cd ..
[root@hadoop101 opt]# xsync /opt/cloudera-manager/
[root@hadoop102 ~]# cd /opt/cloudera-manager/
[root@hadoop102 cloudera-manager]# rpm -ivh cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm
[root@hadoop103 ~]# cd /opt/cloudera-manager/
[root@hadoop103 cloudera-manager]# rpm -ivh cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm
[root@hadoop104 ~]# cd /opt/cloudera-manager/
[root@hadoop104 cloudera-manager]# rpm -ivh cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm
[root@hadoop105 ~]# cd /opt/cloudera-manager/
#安装cloudera-manager-agent(hadoop101-hadoop104都要执行)
[root@hadoop101 cloudera-manager] yum install bind-utils psmisc cyrus-sasl-plain cyrus-sasl-gssapi fuse portmap fuse-libs /lib/lsb/init-functions httpd mod_ssl openssl-devel python-psycopg2 MySQL-python libxslt
#[root@hadoop101 cloudera-manager]# rpm -ivh cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm
#安装agent的server节点
[root@hadoop101 cloudera-manager]# vim /etc/cloudera-scm-agent/config.ini
server_host=hadoop101
[root@hadoop102 cloudera-manager]# vim /etc/cloudera-scm-agent/config.ini
server_host=hadoop101
[root@hadoop103 cloudera-manager]# vim /etc/cloudera-scm-agent/config.ini
server_host=hadoop101
[root@hadoop104 cloudera-manager]# vim /etc/cloudera-scm-agent/config.ini
server_host=hadoop101
#内容修改
server_host=hadoop101
#安装cloudera-manager-server
[root@hadoop101 cloudera-manager]# rpm -ivh cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm
#上传CDH包到parcel-repo
[root@hadoop101 ~]# cd /opt/software/
[root@hadoop101 software]# mv CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel* /opt/cloudera/parcel-repo
[root@hadoop100 software]# mv manifest.json /opt/cloudera/parcel-repo
[root@hadoop101 parcel-repo]# cd /opt/cloudera/parcel-repo
[root@hadoop101 parcel-repo]# mv CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha1 CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha
[root@hadoop101 parcel-repo]# ll
#修改server的db.properties
[root@hadoop101 parcel-repo]# vim /etc/cloudera-scm-server/db.properties
com.cloudera.cmf.db.type=mysql
com.cloudera.cmf.db.host=hadoop101:3306
com.cloudera.cmf.db.name=scm
com.cloudera.cmf.db.user=scm
com.cloudera.cmf.db.password=root
com.cloudera.cmf.db.setupType=EXTERNAL
#启动server服务
[root@hadoop101 log]# /opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm root
[root@hadoop101 software]# systemctl start cloudera-scm-server
[root@hadoop101 software]#tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log
#启动agent节点
[root@hadoop101 software]# systemctl start cloudera-scm-agent
[root@hadoop102 software]# systemctl start cloudera-scm-agent
[root@hadoop103 software]# systemctl start cloudera-scm-agent
[root@hadoop104 software]# systemctl start cloudera-scm-agent
4. 登录 并配置
地址:192.168.2.186:7180/cmf/login
用户名:admin
密码:admin
由于几个主机我已经弄好了,没有出现“当前管理的主机”的tab页,首次会出现,选中需要管理的主机,并打勾
修复问题
[root@hadoop101 software]# echo never > /sys/kernel/mm/transparent_hugepage/defrag
[root@hadoop101 software]# echo never > /sys/kernel/mm/transparent_hugepage/enabled
[root@hadoop102 software]# echo never > /sys/kernel/mm/transparent_hugepage/defrag
[root@hadoop102 software]# echo never > /sys/kernel/mm/transparent_hugepage/enabled
[root@hadoop103 software]# echo never > /sys/kernel/mm/transparent_hugepage/defrag
[root@hadoop103 software]# echo never > /sys/kernel/mm/transparent_hugepage/enabled
[root@hadoop104 software]# echo never > /sys/kernel/mm/transparent_hugepage/defrag
[root@hadoop104 software]# echo never > /sys/kernel/mm/transparent_hugepage/enabled