点击下方名片，设为星标！
回复“1024”获取2TB学习资源！
前面介绍了 HeartBeat 及其集群高可用部署、DRBD原理与实践、DRBD配置与管理、高可用集群架构 corosync+pacemaker、LVS+KeepAlived、haproxy+keepalived+nginx 实现 k8s 集群负载均衡等相关的知识点，今天我将详细的为大家介绍Pacemaker+Corosync 高可用架构实战相关知识，希望大家能够从中收获多多！如有帮助，请点在看、转发朋友圈支持一波！！！
Pacemaker+Corosync 实现高可用(双机热备)
环境的准备(server1、server2)
server1、server2之间设置ssh免密
关闭firewalld及selinux
软件的安装(server1、server2)
rhel中，系统镜像自带的资源有额外的高可用套件及存储套件，他们存放在镜像里的addons:
所以，需要配置yum仓库（server1、server2）：
接着进行安装（server1、server2）：
yum install pacemaker corosync -y
安装集群管理工具pcs及依赖性psmisc policycoreutils-python
集群管理工具pcs命令需要连接pcsd服务，所以，打开pcsd服务并设置为开机自启动：
[root@server1 ~]# systemctl enable pcsd --now
Created symlink from /etc/systemd/system/multi-user.target.wants/pcsd.service to /usr/lib/systemd/system/pcsd.service.集群的建立
设置建立认证的密码(server1、server2)。为建立认证的用户hacluster创建密码：
server1、server2建立认证
[root@server1 ~]# pcs cluster auth server1 server2
server1: Already authorized
server2: Already authorized将server1、server2加入到集群
pcs cluster setup --name mycluster server1 server2集群的启动
[root@server1 ~]# pcs cluster start --all
server1: Starting Cluster (corosync)...
server2: Starting Cluster (corosync)...
server1: Starting Cluster (pacemaker)...
server2: Starting Cluster (pacemaker)...集群状态的校验
[root@server1 ~]# corosync-cfgtool -s
Printing ring status.
Local node ID 1
RING ID 0
id = 172.25.5.1
status = ring 0 active with no faults
[root@server1 ~]# corosync-cfgtool -s
Printing ring status.
Local node ID 1
RING ID 0
id = 172.25.5.1
status = ring 0 active with no faults集群状态的查看
[root@server1 ~]# pcs cluster status
Cluster Status:
Stack: corosync
Current DC: server1 (version 1.1.19-8.el7-c3c624ea3d) - partition with quorum
Last updated: Thu Aug 6 12:00:02 2020
Last change: Wed Aug 5 09:10:47 2020 by root via cibadmin on server2
2 nodes configured
2 resources configured
PCSD Status:
server1: Online
server2: Online高可用集群建立成功！
资源的放置
pcs resource create vip ocf:heartbeat:IPaddr2 ip=172.25.5.99 op monitor interval=30s在集群中添加资源vip：172.168.5.99 j监控时间为30s ，ocf:heartbeat:IPaddr2为资源启动脚本。
[root@server1 ~]# pcs status
Cluster name: mycluster
Stack: corosync
Current DC: server1 (version 1.1.19-8.el7-c3c624ea3d) - partition with quorum
Last updated: Thu Aug 6 13:35:45 2020
Last change: Thu Aug 6 13:35:27 2020 by root via cibadmin on server1
2 nodes configured
1 resource configured
Online: [ server1 server2 ]
Full list of resources:
vip (ocf::heartbeat:IPaddr2): Started server2
Daemon Status:
corosync: active/disabled
pacemaker: active/disabled
pcsd: active/enabled可以看到，此时vip被添加到了server2上：
[root@server2 ~]# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 52:54:00:12:90:b5 brd ff:ff:ff:ff:ff:ff
inet 172.25.5.2/16 brd 172.25.255.255 scope global eth0
valid_lft forever preferred_lft forever
inet 172.25.5.99/16 brd 172.25.255.255 scope global secondary eth0
valid_lft forever preferred_lft forever
inet6 fe80::5054:ff:fe12:90b5/64 scope link
valid_lft forever preferred_lft forever此时停掉server2上的集群组件，server2会接管vip：
[root@server1 ~]# pcs cluster stop server2
server2: Stopping Cluster (pacemaker)...
server2: Stopping Cluster (corosync)...
[root@server1 ~]# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 52:54:00:49:4e:8f brd ff:ff:ff:ff:ff:ff
inet 172.25.5.1/16 brd 172.25.255.255 scope global eth0
valid_lft forever preferred_lft forever
inet 172.25.5.99/16 brd 172.25.255.255 scope global secondary eth0
valid_lft forever preferred_lft forever
inet6 fe80::5054:ff:fe49:4e8f/64 scope link
valid_lft forever preferred_lft forever[root@server1 ~]# pcs status
Cluster name: mycluster
Stack: corosync
Current DC: server1 (version 1.1.19-8.el7-c3c624ea3d) - partition with quorum
Last updated: Thu Aug 6 13:43:13 2020
Last change: Thu Aug 6 13:35:27 2020 by root via cibadmin on server1
2 nodes configured
1 resource configured
Online: [ server1 ]
OFFLINE: [ server2 ]
Full list of resources:
vip (ocf::heartbeat:IPaddr2): Started server1
Daemon Status:
corosync: active/disabled
pacemaker: active/disabled
pcsd: active/enabled此时server2上的vip已经没有了：
[root@server2 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 52:54:00:12:90:b5 brd ff:ff:ff:ff:ff:ff
inet 172.25.5.2/16 brd 172.25.255.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::5054:ff:fe12:90b5/64 scope link
valid_lft forever preferred_lft forever手动删除vip后，vip会自己创建：
[root@server1 ~]# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 52:54:00:49:4e:8f brd ff:ff:ff:ff:ff:ff
inet 172.25.5.1/16 brd 172.25.255.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::5054:ff:fe49:4e8f/64 scope link
valid_lft forever preferred_lft forever
[root@server1 ~]# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 52:54:00:49:4e:8f brd ff:ff:ff:ff:ff:ff
inet 172.25.5.1/16 brd 172.25.255.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::5054:ff:fe49:4e8f/64 scope link
[root@server1 ~]# ip addr del 172.25.5.99 dev eth0
Warning: Executing wildcard deletion to stay compatible with old scripts.
Explicitly specify the prefix length (172.25.5.99/32) to avoid this warning.
This special behaviour is likely to disappear in further releases,
fix your scripts!
[root@server1 ~]# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 52:54:00:49:4e:8f brd ff:ff:ff:ff:ff:ff
inet 172.25.5.1/16 brd 172.25.255.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::5054:ff:fe49:4e8f/64 scope link
valid_lft forever preferred_lft forever
[root@server1 ~]#
[root@server1 ~]# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 52:54:00:49:4e:8f brd ff:ff:ff:ff:ff:ff
inet 172.25.5.1/16 brd 172.25.255.255 scope global eth0
valid_lft forever preferred_lft forever
inet 172.25.5.99/16 brd 172.25.255.255 scope global secondary eth0
valid_lft forever preferred_lft forever
inet6 fe80::5054:ff:fe49:4e8f/64 scope link
valid_lft forever preferred_lft forever当系统发现vip没有时会调用资源启动脚本ocf:heartbeat:IPaddr2来创建vip。更多关于企业集群运维管理系列的学习文章，请参阅：玩转企业集群运维管理专栏，本系列持续更新中。
网络故障（关掉vip所在主机网卡）
ifdown eth0此时vip迁移到server2：
[root@server2 ~]# pcs status
Cluster name: mycluster
Stack: corosync
Current DC: server2 (version 1.1.19-8.el7-c3c624ea3d) - partition with quorum
Last updated: Thu Aug 6 14:01:20 2020
Last change: Thu Aug 6 13:35:27 2020 by root via cibadmin on server1
2 nodes configured
1 resource configured
Online: [ server2 ]
OFFLINE: [ server1 ]
Full list of resources:
vip (ocf::heartbeat:IPaddr2): Started server2
Daemon Status:
corosync: active/disabled
pacemaker: active/disabled
pcsd: active/enabled组资源的设置
如果需要的资源有httpd及vip，使用上述方式则有可能出现httpd和vip不在同一台服务器上。所以，此时应使用组资源。
单个服务的资源设置
资源调用的方式：
[root@server1 ~]# pcs resource standards
lsb
ocf
service
systemd当将apache作为资源时，就应该使用systemd的方式：
pcs resource create apache systemd:httpd op monitor interval=1min[root@server1 ~]# pcs status
Cluster name: mycluster
Stack: corosync
Current DC: server2 (version 1.1.19-8.el7-c3c624ea3d) - partition with quorum
Last updated: Thu Aug 6 14:31:20 2020
Last change: Thu Aug 6 14:31:06 2020 by root via cibadmin on server1
2 nodes configured
2 resources configured
Online: [ server1 server2 ]
Full list of resources:
vip (ocf::heartbeat:IPaddr2): Started server2
apache (systemd:httpd): Started server1
Daemon Status:
corosync: active/disabled
pacemaker: active/disabled
pcsd: active/enabled可以发现，apache和vip并没有绑定在同一服务器，不能实现通过vip对httpd的访问。
组资源的设置
pcs resource group add webgroup vip apache注意：vip 和 apache为按顺序启动！
[root@server1 ~]# pcs status
Cluster name: mycluster
Stack: corosync
Current DC: server2 (version 1.1.19-8.el7-c3c624ea3d) - partition with quorum
Last updated: Thu Aug 6 14:34:23 2020
Last change: Thu Aug 6 14:33:26 2020 by root via cibadmin on server1
2 nodes configured
2 resources configured
Online: [ server1 server2 ]
Full list of resources:
Resource Group: webgroup
vip (ocf::heartbeat:IPaddr2): Started server2
apache (systemd:httpd): Started server2
Daemon Status:
corosync: active/disabled
pacemaker: active/disabled
pcsd: active/enabled[root@server1 ~]# curl 172.25.5.99
server2此时停掉server2，vip及apache会整体迁到server1：
[root@server1 ~]# pcs cluster stop server2
server2: Stopping Cluster (pacemaker)...
^[[Aserver2: Stopping Cluster (corosync)...
[root@server1 ~]# pcs status
Cluster name: mycluster
Stack: corosync
Current DC: server1 (version 1.1.19-8.el7-c3c624ea3d) - partition with quorum
Last updated: Thu Aug 6 14:39:00 2020
Last change: Thu Aug 6 14:33:26 2020 by root via cibadmin on server1
2 nodes configured
2 resources configured
Online: [ server1 ]
OFFLINE: [ server2 ]
Full list of resources:
Resource Group: webgroup
vip (ocf::heartbeat:IPaddr2): Started server1
apache (systemd:httpd): Started server1
Daemon Status:
corosync: active/disabled
pacemaker: active/disabled
pcsd: active/enabled
[root@server1 ~]# curl 172.25.5.99
server1更多关于企业集群运维管理系列的学习文章，请参阅：玩转企业集群运维管理专栏，本系列持续更新中。
搭建 PostgreSQL 高可用集群（基于Pacemaker+Corosync）
此文以PostgreSQL 10版本为例！如未指定，下述命令在所有节点执行！
系统资源及组件规划
系统软件安装与设置
安装基本软件
yum -y install vim lrzsz bash-completion
设置名称解析
echo 192.168.0.11 pgsql1 >> /etc/hosts
echo 192.168.0.12 pgsql2 >> /etc/hosts
echo 192.168.0.13 pgsql3 >> /etc/hosts
设置NTP
yum -y install chrony
systemctl start chronyd
systemctl enable chronyd
systemctl status chronydchronyc sources
设置SELinux、防火墙
systemctl stop firewalld
systemctl disable firewalld
setenforce 0
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
搭建PostgreSQL高可用集群
部署Pacemaker和Corosync
安装Pacemaker和Corosync：
yum -y install pacemaker corosync pcs ipvsadm
启动pcsd，并设置自启动：
systemctl start pcsd
systemctl enable pcsd
systemctl status pcsd
设置hacluster用户密码：
echo hacluster | passwd hacluster --stdin
在任何节点上启用集群认证：
pcs cluster auth -u hacluster -p hacluster pgsql1 pgsql2 pgsql3
在任何节点上同步配置：
pcs cluster setup --last_man_standing=1 --name pgcluster pgsql1 pgsql2 pgsql3
在任何节点上启动集群：
pcs cluster start --all
设置Pacemaker和Corosync自启动：
systemctl enable pacemaker
systemctl enable corosync
部署PostgreSQL
查看Pacemaker支持的PostgreSQL版本：
cat /usr/lib/ocf/resource.d/heartbeat/pgsql | grep ocf_version_cmp
配置YUM源
参考地址：https://www.postgresql.org/download
yum -y install https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
安装PostgreSQL：
yum -y install postgresql10-server
配置环境变量：
su - postgres修改.bash_profile，添加如下内容：
export PATH=$PATH:/usr/pgsql-10/bin
在PGSQL1节点上初始化PostgreSQL：
/usr/pgsql-10/bin/postgresql-10-setup initdb
在PGSQL1节点上配置远程登录和复制权限：
修改/var/lib/pgsql/10/data/postgresql.conf：
listen_addresses = '*'
修改/var/lib/pgsql/10/data/pg_hba.conf，添加如下内容：
# IPv4 local connections:
host all all 0.0.0.0/0 md5
# replication privilege.
host replication repluser 192.168.0.0/24 md5
在PGSQL1节点上启动PostgreSQL：
su - postgres
pg_ctl start
pg_ctl status
在PGSQL1节点上修改数据库密码：
su - postgres
psql -U postgres
ALTER USER postgres WITH ENCRYPTED PASSWORD '111111';
\du
\q
在PGSQL1节点上创建复制用户：
su - postgres
psql
CREATE USER repluser WITH REPLICATION PASSWORD '111111';
\du
\q
在PGSQL2和PGSQL3节点上备份PGSQL1节点数据：
su - postgres
pg_basebackup -h pgsql1 -U repluser -D /var/lib/pgsql/10/data -P -v
在PGSQL1节点上停止PostgreSQL：
su - postgres
pg_ctl stop
pg_ctl status
配置PCS资源
在PGSQL1节点上配置PCS资源：
创建cib配置文件：
pcs cluster cib pgsql_cfg在Pacemaker级别忽略Quorum：
pcs -f pgsql_cfg property set no-quorum-policy=ignore禁用STONITH：
pcs -f pgsql_cfg property set stonith-enabled=false设置资源粘性，防止节点在故障恢复后迁移：
pcs -f pgsql_cfg resource defaults resource-stickiness=INFINITY
设置3次失败后迁移：
pcs -f pgsql_cfg resource defaults migration-threshold=3
设置Master节点虚IP：
pcs -f pgsql_cfg resource create vip-master IPaddr2 ip=192.168.0.10 cidr_netmask=24 op start timeout=60s interval=0s on-fail=restart op monitor timeout=60s interval=10s on-fail=restart op stop timeout=60s interval=0s on-fail=block设置Slave节点虚IP：
pcs -f pgsql_cfg resource create vip-slave IPaddr2 ip=192.168.0.20 cidr_netmask=24 op start timeout=60s interval=0s on-fail=restart op monitor timeout=60s interval=10s on-fail=restart op stop timeout=60s interval=0s on-fail=block设置pgsql集群资源：
pcs -f pgsql_cfg resource create pgsql pgsql pgctl=/usr/pgsql-10/bin/pg_ctl psql=/usr/pgsql-10/bin/psql pgdata=/var/lib/pgsql/10/data config=/var/lib/pgsql/10/data/postgresql.conf rep_mode=sync node_list="pgsql1 pgsql2 pgsql3" master_ip=192.168.0.10 repuser=repluser primary_conninfo_opt="password=111111 keepalives_idle=60 keepalives_interval=5 keepalives_count=5" restart_on_promote=true op start timeout=60s interval=0s on-fail=restart op monitor timeout=60s interval=4s on-fail=restart op monitor timeout=60s interval=3s on-fail=restart role=Master op promote timeout=60s interval=0s on-fail=restart op demote timeout=60s interval=0s on-fail=stop op stop timeout=60s interval=0s on-fail=block设置Master/Slave模式：
pcs -f pgsql_cfg resource master pgsql-cluster pgsql master-max=1 master-node-max=1 clone-max=3 clone-node-max=1 notify=true配置Master IP组：
pcs -f pgsql_cfg resource group add master-group vip-master配置Slave IP组：
pcs -f pgsql_cfg resource group add slave-group vip-slave配置Master IP组绑定Master节点：
pcs -f pgsql_cfg constraint colocation add master-group with master pgsql-cluster INFINITY配置启动Master节点：
pcs -f pgsql_cfg constraint order promote pgsql-cluster then start master-group symmetrical=false score=INFINITY配置停止Master节点：
pcs -f pgsql_cfg constraint order demote pgsql-cluster then stop master-group symmetrical=false score=0配置Slave IP组绑定Slave节点：
pcs -f pgsql_cfg constraint colocation add slave-group with slave pgsql-cluster INFINITY配置启动Slave节点：
pcs -f pgsql_cfg constraint order promote pgsql-cluster then start slave-group symmetrical=false score=INFINITY配置停止Slave节点：
pcs -f pgsql_cfg constraint order demote pgsql-cluster then stop slave-group symmetrical=false score=0把配置文件push到cib：
pcs cluster cib-push pgsql_cfg如果修改集群配置，执行如下命令实现：
cibadmin --query > tmp.xml
vim tmp.xml
cibadmin --replace --xml-file tmp.xml查看集群状态：
pcs status corosync
pcs status
PGSQL1节点为Master节点。更多关于企业集群运维管理系列的学习文章，请参阅：玩转企业集群运维管理专栏，本系列持续更新中。
PostgreSQL故障演示
通过虚IP连接数据库：
psql -U postgres -h 192.168.0.10创建数据库和表：
CREATE DATABASE db;\c db
CREATE TABLE tb (
id int NOT NULL,
name varchar(255) NULL,
PRIMARY KEY (id)
);插入数据：
INSERT INTO tb (id,name) VALUES (1,'MySQL');查看数据：
SELECT * FROM tb;
\q
关闭PGSQL1节点，模拟节点故障
在任意健康节点上查看集群状态：
pcs status corosync
pcs status
此时PGSQL3节点为Master节点
通过虚IP连接数据库：
psql -U postgres -h 192.168.0.10插入数据：
\c db
INSERT INTO tb (id,name) VALUES (2,'Redis');查看数据：
SELECT * FROM tb;
\q数据库读写正常。
原Master节点恢复后，出现如下问题，需要删除lock文件并清除资源状态与错误计数：
rm -rf /var/lib/pgsql/tmp/PGSQL.lock
pcs resource cleanup
以上就是今天给大家分享的基于 Pacemaker+Corosync 架构实现的高可用集群实战过程。更多关于企业集群运维管理系列的学习文章，请参阅：玩转企业集群运维管理专栏，本系列持续更新中。
参考链接：https://blog.csdn.net/qq_42564122/article
/details/107831918 https://blog.csdn.net/
mengshicheng1992/article/details/123612431
读者专属技术群
构建高质量的技术交流社群，欢迎从事后端开发、运维技术进群（备注岗位，已在技术交流群的请勿重复添加）。主要以技术交流、内推、行业探讨为主，请文明发言。广告人士勿入，切勿轻信私聊，防止被骗。
扫码加我好友，拉你进群
推荐阅读 点击标题可跳转
别再ifconfig了！这条命令足以让你摘掉小白工程师的帽子
原以为是Xshell、Mobaxterm的平替，没想到是个王者
PS：因为公众号平台更改了推送规则，如果不想错过内容，记得读完点一下“在看”，加个“星标”，这样每次新文章推送才会第一时间出现在你的订阅列表里。点“在看”支持我们吧!
扫一扫
1643
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
