目前打算在192.168.6.243和192.168.6.244这两台机器上安装Keepalived(首先需要先安装好MySQL,配置好Master-Master主主复制),虚拟IP(VIP)是192.168.6.252,在1台机器上的MySQL出现故障后,VIP自动地漂移到另外1台机器上。
查看下192.168.6.243的环境:
OS: Linuxmds-Master 2.6.18⑵38.el5 #1 SMP Sun Dec 19 14:22:44 EST 2010 x86_64 x86_64x86_64 GNU/Linux
CPU:8*Intel(R)Xeon(R) CPU E5506 @ 2.13GHz
Mem:32G
Keepalived的相干信息:
Version:1.2.1(注意:不要使用最新版本1.2.2,编译有毛病)
下载地址:http://www.keepalived.org/download.html
MySQL的相干信息
Version:5.1.47+XtraDB
MySQL_Install_Dictory: /usr/local/mysql
MySQL_Config_File:/home/mysql/mysq/my1.cnf
[mysql@mds-Master tool]$ cd /home/wangwei/tools
[mysql@mds-Master tool]$ tar xvzf keepalived⑴.2.1.tar.gz
[wangwei@mds-Masterkeepalived⑴.2.1]$./configure --prefix=/usr/local/keepalived --with-kernel-dir=/usr/src/kernels/2.6.18⑵38.el5-x86_64
编译出错
configure: error:
!!! OpenSSLis not properly installed on your system. !!!
!!! Can notinclude OpenSSL headers files. !!!
说明系统中没有安装OpenSSL
那就安装下OpenSS:L吧,出错
[mysql@mds-Master tool]# yum install openssl-devel
Loaded plugins: rhnplugin, security
This system is not registered with RHN
缘由是linux没有在红帽网络上注册,所以没法下载上面的软件包,替换方案可使用centos(使用中国科技大学的源 http://mirror.centos.org),具体如何配置我就不写了。
配置终了后,使用yum安装openssl-devel:
[root@mds-Master tools]# yum install openssl-devel
Dependency Installed:
e2fsprogs-devel.x86_64 0:1.39⑵3.el5_5.1 keyutils-libs-devel.x86_64 0:1.2⑴.el5 krb5-devel.x86_64 0:1.6.1⑸5.el5_6.2
libselinux-devel.x86_64 0:1.33.4⑸.7.el5 libsepol-devel.x86_64 0:1.15.2⑶.el5 zlib-devel.x86_64 0:1.2.3⑶
Complete
再次编译
[root@mds-Master keepalived⑴.2.1]#./configure --prefix=/usr/local/keepalived --with-kernel-dir=/usr/src/kernels/2.6.18⑵38.el5-x86_64
编译后的信息以下:
Keepalived configuration
------------------------
Keepalived version : 1.2.1
Compiler : gcc
Compiler flags : -g -O2 -DETHERTYPE_IPV6=0x86dd
Extra Lib : -lpopt -lssl -lcrypto
Use IPVS Framework : Yes
IPVS sync daemon support : Yes
Use VRRP Framework : Yes
Use Debug flags : No
Keepalived的安装路径为:/usr/local/keepalived,--with-kernel-dir表示指定Linux的内核位置,由于Keepalived是建立在LVS之上的(Linux2.4内核以后,已完全内置了LVS的各个功能模块,无需给内核打任何补钉,可以直接使用LVS提供的各种功能,关于LVS是甚么东东?可以百度之,也算是国人对Linux的1大贡献,LVS的作者是毕业国防科技大学的章文嵩博士,现就职于淘宝网)。
[root@mds-Master keepalived⑴.2.1]# make
Make complete
make[1]: Leaving directory`/home/wangwei/tools/keepalived⑴.2.1/genhash'
Make complete
[root@mds-Masterkeepalived⑴.2.1]# make install
make -C keepalived install
。。。。。。。。。。。。。。。。
install -m 644 ../doc/man/man1/genhash.1 /usr/local/keepalived/share/man/man1
make[1]: Leaving directory`/home/wangwei/tools/keepalived⑴.2.1/genhash'
这样的话,安装就告1段落了,在192.168.6.243上也安装一样版本的Keepalived。
[root@mds-Master ~]# ls -lh /usr/local/keepalived/
total 16K
drwxr-xr-x 2 root root 4.0K Jul 5 08:54 bin
drwxr-xr-x 5 root root 4.0K Jul 5 08:54 etc
drwxr-xr-x 2 root root 4.0K Jul 5 08:54 sbin
drwxr-xr-x 3 root root 4.0K Jul 5 08:54 share
[root@mds-Master ~]#/usr/local/keepalived/sbin/keepalived --help
Keepalived v1.2.1 (07/04,2011)
Usage:
/usr/local/keepalived/sbin/keepalived
/usr/local/keepalived/sbin/keepalived -n
/usr/local/keepalived/sbin/keepalived -f keepalived.conf
/usr/local/keepalived/sbin/keepalived -d
/usr/local/keepalived/sbin/keepalived -h
/usr/local/keepalived/sbin/keepalived-v
。。。。。。。。
可见版本是1.2.1,显示也很正常
[root@mds-Master ~]# more/etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
wei.wang@99bill.com
}
notification_email_from wei.wang@99bill.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id MySQL-HA
}
vrrp_instance VI_1 {
state Backup #两台配置此处均是Backup
interface eth0 #VIP绑定在那块网卡上
smtp_alert #开启邮件报警
virtual_router_id 51
priority 100 #优先级,另外一台改成90
advert_int1 #发送广播信息的间隔时间,单位秒
preempt #抢占,只在优先级高的机器上设置便可,优先级低的机器不设置
authentication {
auth_typePASS
auth_pass1111
}
virtual_ipaddress {
192.168.6.252 #VIP
}
}
virtual_server 192.168.6.252 4321 {
delay_loop 2 #每一个2秒检查1次real_server状态
lb_algo wrr #LVS算法
lb_kind DR #LVS模式
persistence_timeout 60 #会话保持时间
protocol TCP
real_server 192.168.6.243 4321 {
weight 3
notify_down"/etc/keepalived/shutdown_keepalived.sh" #检测到服务down后履行的脚本
TCP_CHECK {
connect_timeout 10 #连接超时时间
nb_get_retry 3 #重连次数
delay_before_retry #重连间隔时间
connect_port 4321 #健康检查端口
}
}
}
global_defs :全局定义,
notification_email:指定Keepalived在产生事件(如:自动切换),需要发送Email到的对象,可以为多个地址
notification_email_from:邮件发送者
smtp_server :发送邮件的邮件服务器,如果本地有开启sendmail的话,可以采取127.0.0.1
vrrp_instance VI_1 :VRRP实例配置,包括了主从状态、VIP、权重等
virtual_server: 虚拟主机,包括了虚拟主机的IP+Port,实际主机的IP+Port,和检查实际主机的IP+Port异常后的操作等
lb_algo:LVS中的负载均衡调度算法
lb_kind:LVS中的负载均衡转发规则,1般包括DR,NAT,TUN3种,DR是效力最好的1种规则,直接路由转发
[root@mds-Master ~]# /etc/init.d/sendmail status
sendmail (pid 31476) is running...
You have new mail in /var/spool/mail/root
[mysql@mds-Master tool]$ more/etc/keepalived/shutdown_keepalived.sh
#!/bin/sh
pkill keepalived
就是检查到MySQL宕机后,自动杀死Keepalived进程,那末另外1台机器上的Keepalived进程在接受广播信息超时后,自动接收VIP,进入到Master状态
上图是Keepalived自动发送的邮件,这个很好,可对Keepalived的状态变更自动监控
[root@mds-Master ~]#/usr/local/keepalived/sbin/keepalived -D
默许情况下,Keepalived的配置文件是/etc/keepalived/keepalived.conf
查看日志文件,Keepalived默许的日志文件是/var/log/messages
[root@mds-Master ~]# tail ⑴0 /var/log/messages
Jul 11 17:26:10 mds-Master Keepalived_vrrp:Registering Kernel netlink reflector
Jul 11 17:26:10 mds-Master Keepalived_vrrp:Registering Kernel netlink command channel
Jul 11 17:26:10 mds-Master Keepalived_vrrp:Registering gratutious ARP shared channel
Jul 11 17:26:10 mds-Master Keepalived_vrrp: Openingfile '/etc/keepalived/keepalived.conf'.
Jul 11 17:26:10 mds-Master Keepalived_vrrp:Configuration is using : 63222 Bytes
Jul 11 17:26:10 mds-Master Keepalived_vrrp: UsingLinkWatch kernel netlink reflector...
Jul 11 17:26:10 mds-Master Keepalived_vrrp:VRRP_Instance(VI_1) Entering Backup STATE
Jul 11 17:26:10 mds-Master Keepalived_vrrp: VRRPsockpool: [ifindex(2), proto(112), fd(11,12)]
Jul 11 17:26:11 mds-Master Keepalived_vrrp:VRRP_Instance(VI_1) forcing a new Master election
Jul 11 17:26:12 mds-Master Keepalived_vrrp:VRRP_Instance(VI_1) Transition to Master STATE
Jul 11 17:26:13 mds-Master Keepalived_vrrp:VRRP_Instance(VI_1) Entering Master STATE
Jul 11 17:26:13 mds-Master Keepalived_vrrp:VRRP_Instance(VI_1) setting protocol VIPs.
Jul 11 17:26:13 mds-Master Keepalived_vrrp:VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.6.252
至此,Keepalived的安装配置告1段落了。
Keepalived在RHEL 6.2中,不会自动的加载ipvs模块,可以通过下面命令来确认是不是已加载:
lsmod|grep ip_vs
所以依照上述方式安装后,日志文件(默许是:/var/log/messages)中出现以下报警信息,“Keepalived_healthcheckers:IPVS: Can't initialize ipvs: Protocol not available”,ipvs称之为IP 虚拟服务器(IP Virtual Server),主要是运行在LVS下的提供负载平衡功能的1种技术,而实际上,我们使用的keepalived并没有使用负载均衡的功能,只是使用虚拟IP自动漂移的部份,所以呢,即使是没有找到,keepalived也是正常运转的,不影响正常使用。
这样,我们就需要手工加载ipvs模块,命令以下:
modprobeip_vs
modprobeip_vs_wrr
再重启keepalived便可,即下面的两条命令:
Sh/etc/keepalived/shutdown_keepalived.sh --关闭
/usr/local/keepalived/sbin/keepalived -D --启动
再次视察日志文件,可以看到,上述的报警信息就再也没有出现了。
在启动192.168.6.243和192.168.6.244两台机器上的Keepalived后(正常与否,需要检查日志文件/var/log/messages),,利用MySQL的客户端连接之,看是不是可用登陆
下面我就在192.168.6.49这台机器上,远程登陆192.168.6.252:4321(虚拟IP+虚拟端口),看是不是可以正常登陆之
[root@mrs_test ~]# hostname -i
192.168.6.49
[root@mrs_test ~]# mysql -h192.168.6.252 -P4321-uwangwei -ptianma
Welcome to the MySQL monitor. Commands end with ; or g.
Your MySQL connection id is 295
Server version: 5.1.47-log Source distribution
Type 'help;' or 'h' for help. Type 'c' to clearthe buffer.
mysql> status
--------------
mysql Ver14.14 Distrib 5.1.33, for pc-linux-gnu (i686) using readline 5.1
Connection id: 295
Current database:
Current user: wangwei@192.168.6.49
SSL: Not in use
Current pager: stdout
Using outfile: ''
Using delimiter: ;
Server version: 5.1.47-log Source distribution
Protocol version: 10
Connection: 192.168.6.252 via TCP/IP
Server characterset: utf8
Db characterset: utf8
Client characterset: latin1
Conn. characterset: latin1
TCP port: 4321
Uptime: 10 min 32 sec
Threads: 2 Questions: 10 Slow queries:0 Opens: 14 Flush tables: 1 Open tables: 8 Queries per second avg: 0.15
--------------
mysql>
可见1切才正常,可以正常登陆之
下面我们摹拟MySQL挂掉后,看VIP是不是会自动地被另外1台机器接收。
[root@mds-Master ~]# tail ⑴0 /var/log/messages
Jul 12 10:26:24 mds-Master Keepalived_vrrp: RemoteSMTP server [127.0.0.1:25] connected.
Jul 12 10:26:24 mds-Master Keepalived_vrrp: SMTPalert successfully sent.
Jul 12 10:26:25 mds-Master Keepalived_vrrp:VRRP_Instance(VI_1) forcing a new Master election
Jul 12 10:26:26 mds-Master Keepalived_vrrp:VRRP_Instance(VI_1) Transition to Master STATE
Jul 12 10:26:27 mds-Master Keepalived_vrrp:VRRP_Instance(VI_1) Entering Master STATE
Jul 12 10:26:27 mds-Master Keepalived_vrrp:VRRP_Instance(VI_1) setting protocol VIPs.
Jul 12 10:26:27 mds-Master Keepalived_vrrp:VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.6.252
Jul 12 10:26:27 mds-Master Keepalived_vrrp: RemoteSMTP server [127.0.0.1:25] connected.
Jul 12 10:26:27 mds-Master Keepalived_vrrp: SMTPalert successfully sent.
Jul 12 10:26:32 mds-Master Keepalived_vrrp:VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.6.252
可见192.168.6.243上的Keepalived是Master状态
[root@mds-slave ~]# tail ⑴0 /var/log/messages
Jul 12 10:26:40 mds-slave avahi-daemon[4968]:Registering new address record for 192.168.6.252 on eth0.
Jul 12 10:26:40 mds-slave avahi-daemon[4968]:Registering new address record for 192.168.6.244 on eth0.
Jul 12 10:26:40 mds-slave avahi-daemon[4968]:Registering HINFO record with values 'X86_64'/'LINUX'.
Jul 12 10:26:56 mds-slave Keepalived_vrrp:VRRP_Instance(VI_1) Received higher prio advert
Jul 12 10:26:56 mds-slave Keepalived_vrrp:VRRP_Instance(VI_1) Entering Backup STATE
Jul 12 10:26:56 mds-slave Keepalived_vrrp:VRRP_Instance(VI_1) removing protocol VIPs.
Jul 12 10:26:56 mds-slave Keepalived_vrrp: Netlinkreflector reports IP 192.168.6.244 removed
Jul 12 10:26:56 mds-slave Keepalived_healthcheckers:Netlink reflector reports IP 192.168.6.244 removed
Jul 12 10:26:56 mds-slave avahi-daemon[4968]:Server startup complete. Host name is mds-slave⑸1.local. Local service cookieis 479334863.
Jul 12 10:26:56 mds-slave avahi-daemon[4968]:Withdrawing address record for 192.168.6.252 on eth0.
可见192.168.6.243上的Keepalived处于Slave状态
[mysql@mds-Master mysql]$ mysqladmin--defaults-file=my1.cnf -uroot -p99billshutdown
110712 10:50:16 mysqld_safe mysqld from pid file/home/mysql/mysql/mysql/mysqld.pid ended
[1]+ Done /usr/local/mysql/bin/mysqld_safe --defaults-file=my1.cnf
[root@mds-Master ~]# tail ⑴0 /var/log/messages
Jul 12 10:26:32 mds-Master Keepalived_vrrp:VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.6.252
Jul 12 10:50:09 mds-MasterKeepalived_healthcheckers: TCP connection to [192.168.6.243:4321] failed !!!
Jul 12 10:50:09 mds-MasterKeepalived_healthcheckers: Removing service [192.168.6.243:4321] from VS[192.168.6.252:4321]
Jul 12 10:50:09 mds-MasterKeepalived_healthcheckers: Executing [/etc/keepalived/shutdown_keepalived.sh]for service [192.168.6.243:4321] in VS [192.168.6.252:4321]
Jul 12 10:50:09 mds-MasterKeepalived_healthcheckers: Lost quorum 1-0=1 > 0 for VS [192.168.6.252:4321]
Jul 12 10:50:09 mds-MasterKeepalived_healthcheckers: Remote SMTP server [127.0.0.1:25] connected.
Jul 12 10:50:09 mds-Master Keepalived: Terminatingon signal
Jul 12 10:50:09 mds-Master Keepalived_vrrp:Terminating VRRP child process on signal
Jul 12 10:50:09 mds-Master Keepalived: StoppingKeepalived v1.2.1 (07/04,2011)
Jul 12 10:50:09 mds-MasterKeepalived_healthcheckers: Terminating Healthchecker child process on signal
可见192.168.6.243上的Keepalived对实际服务器进行了健康检查,失败之,触发脚本/etc/keepalived/shutdown_keepalived.sh的履行,发送报警邮件
3.2.5 查看192.168.6.243上的Keepalived进程
[root@mds-Master ~]# ps aux |grep keep
root 20370 0.0 0.0 61188 768 pts/4 S+ 10:54 0:00 grep keep
Keepalived进程已不存在
[root@mds-slave ~]# tail ⑴0 /var/log/messages
Jul 12 10:52:38 mds-slave avahi-daemon[4968]:Registering new address record for 192.168.6.252 on eth0.
Jul 12 10:52:38 mds-slave avahi-daemon[4968]:Registering new address record for 192.168.6.244 on eth0.
Jul 12 10:52:38 mds-slave avahi-daemon[4968]:Registering HINFO record with values 'X86_64'/'LINUX'.
Jul 12 10:52:58 mds-slave avahi-daemon[4968]:Withdrawing address record for fe80::baac:6fff:fe15:bf64 on eth0.
Jul 12 10:52:58 mds-slave avahi-daemon[4968]:Withdrawing address record for 192.168.6.244 on eth0.
Jul 12 10:52:58 mds-slave avahi-daemon[4968]:Registering new address record for fe80::baac:6fff:fe15:bf64 on eth0.
Jul 12 10:52:58 mds-slave avahi-daemon[4968]:Registering new address record for 192.168.6.252 on eth0.
Jul 12 10:52:58 mds-slave avahi-daemon[4968]:Registering new address record for 192.168.6.244 on eth0.
Jul 12 10:52:58 mds-slave avahi-daemon[4968]:Registering HINFO record with values 'X86_64'/'LINUX'.
可见192.168.6.243自动接收了192.168.6.252的管理
[root@mrs_test ~]# hostname -i
192.168.6.49
[root@mrs_test ~]# mysql -h192.168.6.252 -P4321-uwangwei -ptianma
Welcome to the MySQL monitor. Commands end with ; or g.
Your MySQL connection id is 293927
Server version: 5.1.47-log Source distribution
Type 'help;' or 'h' for help. Type 'c' to clearthe buffer.
mysql> status
--------------
mysql Ver14.14 Distrib 5.1.33, for pc-linux-gnu (i686) using readline 5.1
Connection id: 293927
Current database:
Current user: wangwei@192.168.6.49
SSL: Not in use
Current pager: stdout
Using outfile: ''
Using delimiter: ;
Server version: 5.1.47-log Source distribution
Protocol version: 10
Connection: 192.168.6.252 via TCP/IP
Server characterset: utf8
Db characterset: utf8
Client characterset: latin1
Conn. characterset: latin1
TCP port: 4321
Uptime: 6 days 19 hours 13 min 2 sec
Threads: 2 Questions: 1645 Slow queries:0 Opens: 175 Flush tables: 1 Open tables: 62 Queries per second avg: 0.2
--------------
mysql>
可见VIP+虚拟Port仍然可以正常访问,这时候候实际服务器是MySQL⑴92.168.6.243
[mysql@mds-Master mysql]$ /usr/local/mysql/bin/mysqld_safe --defaults-file=my1.cnf &
[1] 20374
[mysql@mds-Master mysql]$ 110712 10:55:06mysqld_safe Logging to '/home/mysql/mysql/mysql/mysqld.log'.
110712 10:55:06 mysqld_safe Starting mysqld daemonwith databases from /home/mysql/mysql/mysql
[root@mds-Master ~]#/usr/local/keepalived/sbin/keepalived -D
[root@mds-Master ~]# tail ⑴0 /var/log/messages
Jul 12 10:55:47 mds-Master Keepalived_vrrp: RemoteSMTP server [127.0.0.1:25] connected.
Jul 12 10:55:48 mds-Master Keepalived_vrrp: SMTPalert successfully sent.
Jul 12 10:55:48 mds-Master Keepalived_vrrp:VRRP_Instance(VI_1) forcing a new Master election
Jul 12 10:55:49 mds-Master Keepalived_vrrp:VRRP_Instance(VI_1) Transition to Master STATE
Jul 12 10:55:50 mds-Master Keepalived_vrrp:VRRP_Instance(VI_1) Entering Master STATE
Jul 12 10:55:50 mds-Master Keepalived_vrrp:VRRP_Instance(VI_1) setting protocol VIPs.
Jul 12 10:55:50 mds-Master Keepalived_vrrp:VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.6.252
Jul 12 10:55:50 mds-Master Keepalived_vrrp: RemoteSMTP server [127.0.0.1:25] connected.
Jul 12 10:55:50 mds-Master Keepalived_vrrp: SMTPalert successfully sent.
Jul 12 10:55:55 mds-Master Keepalived_vrrp: VRRP_Instance(VI_1)Sending gratuitous ARPs on eth0 for 192.168.6.252
可见192.168.6.243重新进入Master状态,这里需要注意的是:配置文件中我配置的是抢占式,如果是不抢占的话,192.168.6.243重启后会进入Slave状态,不会产生Master的抢占。
[root@mds-slave ~]# tail ⑴0 /var/log/messages
Jul 12 10:56:18 mds-slave avahi-daemon[4968]:Registering new address record for 192.168.6.252 on eth0.
Jul 12 10:56:18 mds-slave avahi-daemon[4968]:Registering new address record for 192.168.6.244 on eth0.
Jul 12 10:56:18 mds-slave avahi-daemon[4968]: RegisteringHINFO record with values 'X86_64'/'LINUX'.
Jul 12 10:56:19 mds-slave Keepalived_vrrp:VRRP_Instance(VI_1) Received higher prio advert
Jul 12 10:56:19 mds-slave Keepalived_vrrp:VRRP_Instance(VI_1) Entering Backup STATE
Jul 12 10:56:19 mds-slave Keepalived_vrrp:VRRP_Instance(VI_1) removing protocol VIPs.
Jul 12 10:56:19 mds-slave Keepalived_vrrp: Netlinkreflector reports IP 192.168.6.244 removed
Jul 12 10:56:19 mds-slave avahi-daemon[4968]:Withdrawing address record for 192.168.6.252 on eth0.
Jul 12 10:56:19 mds-slaveKeepalived_healthcheckers: Netlink reflector reports IP 192.168.6.244 removed
Jul 12 10:56:20 mds-slave avahi-daemon[4968]:Server startup complete. Host name is mds-slave⑻1.local. Local service cookieis 479334863.
可见192.168.6.243自动的进入Slave状态
其实3.2中已包括了3.3(摹拟Keepalived挂掉)这类的场景,这里就不再叙述,详见3.2