え、明日から10月・・・
Adachinですっ。
前回のロードバランサー構築ブログはNATの設定まででしたが、
今回はLVSを2台構成にして、一台落ちても問題ないように冗長化の設定をします。
つまり、Keepalivedを使って駆使するわけです。
レッツアンドゴー!
■構成図
VIPの外部は172.19.10.111に。内部は192.168.99.111にしました。
LVS02はLVS01と設定は同じです。
※注意点
全てVIPを使って通信したいため、現在WEB01,02のデフォルトゲートウェイが172.19.10.63を向いてるため、
keepalivedの設定が完了したらVIPの192.168.99.111に変更しましょう。
■Keepalivedの設定(LVS01,02)
・インストール
|
1 |
# yum install keepalived |
・keepalived構成
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
$ tree /etc/keepalived keepalived ├── conf.d │ ├── test-http.conf │ ├── test-ssh01.conf.org │ ├── test-ssh02.conf.org │ └── vrrp.conf ├── ip_list.txt ├── keepalived.conf ├── keepalived.confbk ├── vrrp_backup.sh ├── vrrp_master.sh └── vrrp_state.sh |
今回はDSRなので、いろいろと設定を分けています。
sshも現在ipvsadmで設定してるので、後でfirewalldの設定に変更しましょう。
ひとまずconfの設定から。
・keepalived.conf(includeします)
|
1 2 3 4 5 6 |
! Configuration File for keepalived global_defs { } include conf.d/*.conf |
・vrrp_master.sh(DSRのためのスクリプト)
|
1 2 3 4 5 6 7 8 9 10 11 |
#!/bin/sh /bin/echo 1 > /proc/sys/net/ipv4/conf/eth1/proxy_arp /bin/echo 1 ? /proc/sys/net/ipv4/ip_nonlocal_bind IPLST="/etc/keepalived/ip_list.txt" for ip in `cat $IPLST` ; do /sbin/ip route add $ip dev eth0 /bin/echo "/sbin/arping -q -U -I eth0 -c 2 $ip" | logger -t GARP /sbin/arping -q -U -I eth0 -c 2 $ip done |
・vrrp_backup.sh(DSRのためのスクリプト)
|
1 2 3 4 5 6 7 8 |
#!/bin/sh /bin/echo 0 > /proc/sys/net/ipv4/conf/eth1/proxy_arp IPLST="/etc/keepalived/ip_list.txt" for ip in `cat $IPLST` ; do /sbin/ip route del $ip dev eth0 done |
・vrrp_state.sh(DSRのためのスクリプト)
|
1 2 3 4 |
#!/bin/bash /bin/date > /var/run/vrrp_status /bin/echo "$@" > /var/run/vrrp_status |
・ip_list.txt(DSRのためのスクリプト/ファイル)
172.19.10.111
・vrrp.conf(VIPの設定)
| vrrp_sync_group VG { | |
| group { | |
| VI | |
| VE | |
| } | |
| notify_master /etc/keepalived/vrrp_master.sh | |
| notify_backup /etc/keepalived/vrrp_backup.sh | |
| notify_fault /etc/keepalived/vrrp_backup.sh | |
| notify /etc/keepalived/vrrp_state.sh | |
| } | |
| vrrp_instance VI { | |
| state BACKUP | |
| interface eth0 | |
| lvs_sync_daemon_interface eth1 | |
| garp_master_delay 5 | |
| virtual_router_id 101 #基本MASTERとBACKUPも同じにしてかぶらないよう気をつける | |
| priority 100 | |
| nopreempt | |
| advert_int 1 | |
| smtp_alert | |
| authentication { | |
| auth_type PASS | |
| auth_pass password | |
| } | |
| virtual_ipaddress { | |
| 172.19.10.111/24 dev eth0 | |
| } | |
| } | |
| vrrp_instance VE { | |
| state BACKUP | |
| interface eth1 | |
| garp_master_delay 5 | |
| virtual_router_id 102 | |
| priority 100 | |
| nopreempt | |
| advert_int 1 | |
| smtp_alert | |
| authentication { | |
| auth_type PASS | |
| auth_pass password | |
| } | |
| virtual_ipaddress { | |
| 192.168.99.111/24 dev eth1 | |
| } | |
| } |
・test-http.conf(HTTPの設定)
| virtual_server_group DOCKER_HTTP { | |
| 172.19.10.63 80 | |
| 172.19.10.111 80 | |
| } | |
| virtual_server group DOCKER_HTTP { | |
| delay_loop 3 | |
| lvs_sched wrr | |
| lvs_method NAT →Masp | |
| protocol TCP | |
| real_server 192.168.99.2 80 { | |
| weight 5 | |
| inhibit_on_failure | |
| HTTP_GET { | |
| url { | |
| path / | |
| status_code 200 | |
| } | |
| connect_port 80 | |
| connect_timeout 3 | |
| } | |
| } | |
| real_server 192.168.99.3 80 { | |
| weight 5 | |
| inhibit_on_failure | |
| HTTP_GET { | |
| url { | |
| path / | |
| status_code 200 | |
| } | |
| connect_port 80 | |
| connect_timeout 3 | |
| } | |
| } | |
| } |
・test-ssh.conf(SSHの設定)
10024のポートも同様に同じファイルを作ります。(firewalldの設定するので必要なし)
・keepalived起動
|
1 |
# systemctl start keepalived |
・keepalived status
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
# systemctl status keepalived ● keepalived.service - LVS and VRRP High Availability Monitor Loaded: loaded (/usr/lib/systemd/system/keepalived.service; enabled; vendor preset: disabled) Active: active (running) since 2016-xxxxxxxxxxxxxxxxx Process: 8451 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS) Main PID: 8452 (keepalived) CGroup: /system.slice/keepalived.service ├─8452 /usr/sbin/keepalived -D -S 2 --vrrp --check ├─8453 /usr/sbin/keepalived -D -S 2 --vrrp --check └─8454 /usr/sbin/keepalived -D -S 2 --vrrp --check 9月 29 18:19:33 xxxx Keepalived_vrrp[8454]: VRRP_Group(VG) Syncing instances to MASTER state 9月 29 18:19:33 xxxx Keepalived_vrrp[8454]: VRRP_Instance(VE) Transition to MASTER STATE 9月 29 18:19:33 xxxx Keepalived_vrrp[8454]: Opening script file /etc/keepalived/vrrp_master.sh 9月 29 18:19:33 xxxx Keepalived_vrrp[8454]: Opening script file /etc/keepalived/vrrp_state.sh 9月 29 18:19:34 xxxx Keepalived_vrrp[8454]: VRRP_Instance(VE) Entering MASTER STATE 9月 29 18:19:34 xxxx Keepalived_vrrp[8454]: VRRP_Instance(VE) setting protocol VIPs. 9月 29 18:19:34 xxxx Keepalived_vrrp[8454]: VRRP_Instance(VE) Sending gratuitous ARPs on eth1 for 192.168.99.111 9月 29 18:19:34 xxxx Keepalived_healthcheckers[8453]: Netlink reflector reports IP 192.168.99.111 added 9月 29 18:19:34 xxxx Keepalived_vrrp[8454]: VRRP_Instance(VI) Entering MASTER STATE 9月 29 18:19:39 xxxx Keepalived_vrrp[8454]: VRRP_Instance(VE) Sending gratuitous ARPs on eth1 for 192.168.99.111 |
VIPが振られてるのが分かります。
|
1 2 3 4 5 6 7 8 9 |
# ip a |grep eth0 2: eth0: >BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 inet 172.19.10.63/24 brd 172.19.10.255 scope global eth0 inet 172.19.10.111/24 scope global secondary eth0 # ip a |grep eth1 3: eth1: >BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 inet 192.168.99.1/24 brd 192.168.99.255 scope global eth1 inet 192.168.99.111/24 scope global secondary eth1 |
■DSRの設定
・sshの設定をfirewalldに振り分けする
|
1 2 3 4 |
# firewall-cmd --zone=external --add-forward-port=port=10023:proto=tcp:toaddr=192.168.99.2:toport=22 # firewall-cmd --zone=external --add-forward-port=port=10024:proto=tcp:toaddr=192.168.99.3:toport=22 # mv /etc/keepalived/conf.d/test-ssh01.conf /etc/keepalived/conf.d/test-ssh01.conf.org # mv /etc/keepalived/conf.d/test-ssh02.conf /etc/keepalived/conf.d/test-ssh02.conf.org |
・firewalld でVIP(172.19.10.111)に対してマスクセット
|
1 |
# firewall-cmd --permanent --direct --add-rule ipv4 mangle PREROUTING_direct 0 -d 172.19.10.111 -p tcp --dport 80 -j MARK --set-mark 1 |
・VIPの設定をコメントアウト
|
1 2 3 4 |
# vim /etc/keepalived/conf.d/vrrp.conf virtual_ipaddress { # 172.19.10.111/24 dev eth0 } |
・ルーティングの優先順位設定追加
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
# vi /etc/sysconfig/network-scripts/ifup-local #!/bin/sh DEVICE="$1" case "$DEVICE" in lo) /sbin/ip rule add prio 100 fwmark 1 table 100 /sbin/ip route add local 0/0 dev lo table 100 ;; esac # vi /etc/sysconfig/network-scripts/ifdown-local #!/bin/sh case "$DEVICE" in lo) /sbin/ip rule del prio 100 fwmark 1 table 100 /sbin/ip route del local 0/0 dev lo table 100 ;; esac |
・firewalld にVIPからのリクエストを リダイレクトするルール追加
|
1 |
# firewall-cmd --direct --add-rule ipv4 nat PREROUTING_direct 0 -d 172.19.10.111 -p tcp --dport 80 -j REDIRECT |
・keepalivedを止める(LVS01)
|
1 2 3 4 5 6 7 8 |
LVS01# cat /var/run/vrrp_status Thu Sep 29 18:19:33 JST 2016 GROUP VG MASTER 0 LVS02# cat /var/run/vrrp_status Thu Sep 29 18:19:27 JST 2016 GROUP VG BACKUP 0 |
|
1 2 3 4 5 6 |
LVS01# systemctl stop keepalived LVS02# cat /var/run/vrrp_status Thu Sep 29 18:19:27 JST 2016 GROUP VG MASTER 0 →切り替わってる |
■まとめ
これで冗長化はバッチグーですね。
haproxyと違ってLVSはパフォーマンスがいいですが、
設定がかなり難しいといったところ。
次回はベンチマークでお会いしましょう。

0件のコメント