え、明日から10月・・・
Adachinですっ。
前回のロードバランサー構築ブログはNATの設定まででしたが、
今回はLVSを２台構成にして、一台落ちても問題ないように冗長化の設定をします。
つまり、Keepalivedを使って駆使するわけです。
レッツアンドゴー!
■構成図
VIPの外部は172.19.10.111に。内部は192.168.99.111にしました。
LVS02はLVS01と設定は同じです。
※注意点
全てVIPを使って通信したいため、現在WEB01,02のデフォルトゲートウェイが172.19.10.63を向いてるため、
keepalivedの設定が完了したらVIPの192.168.99.111に変更しましょう。
■Keepalivedの設定(LVS01,02)
・インストール
|
1 |
# yum install keepalived |
・keepalived構成
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
$ tree /etc/keepalived keepalived ├── conf.d │ ├── test-http.conf │ ├── test-ssh01.conf.org │ ├── test-ssh02.conf.org │ └── vrrp.conf ├── ip_list.txt ├── keepalived.conf ├── keepalived.confbk ├── vrrp_backup.sh ├── vrrp_master.sh └── vrrp_state.sh |
今回はDSRなので、いろいろと設定を分けています。
sshも現在ipvsadmで設定してるので、後でfirewalldの設定に変更しましょう。
ひとまずconfの設定から。
・keepalived.conf(includeします)
|
1 2 3 4 5 6 |
! Configuration File for keepalived global_defs { } include conf.d/*.conf |
・vrrp_master.sh(DSRのためのスクリプト)
|
1 2 3 4 5 6 7 8 9 10 11 |
#!/bin/sh /bin/echo 1 > /proc/sys/net/ipv4/conf/eth1/proxy_arp /bin/echo 1 ? /proc/sys/net/ipv4/ip_nonlocal_bind IPLST="/etc/keepalived/ip_list.txt" for ip in `cat $IPLST` ; do /sbin/ip route add $ip dev eth0 /bin/echo "/sbin/arping -q -U -I eth0 -c 2 $ip" | logger -t GARP /sbin/arping -q -U -I eth0 -c 2 $ip done |
・vrrp_backup.sh(DSRのためのスクリプト)
|
1 2 3 4 5 6 7 8 |
#!/bin/sh /bin/echo 0 > /proc/sys/net/ipv4/conf/eth1/proxy_arp IPLST="/etc/keepalived/ip_list.txt" for ip in `cat $IPLST` ; do /sbin/ip route del $ip dev eth0 done |
・vrrp_state.sh(DSRのためのスクリプト)
|
1 2 3 4 |
#!/bin/bash /bin/date > /var/run/vrrp_status /bin/echo "$@" > /var/run/vrrp_status |
・ip_list.txt(DSRのためのスクリプト/ファイル)
172.19.10.111
・vrrp.conf(VIPの設定)
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| vrrp_sync_group VG { | |
| group { | |
| VI | |
| VE | |
| } | |
| notify_master /etc/keepalived/vrrp_master.sh | |
| notify_backup /etc/keepalived/vrrp_backup.sh | |
| notify_fault /etc/keepalived/vrrp_backup.sh | |
| notify /etc/keepalived/vrrp_state.sh | |
| } | |
| vrrp_instance VI { | |
| state BACKUP | |
| interface eth0 | |
| lvs_sync_daemon_interface eth1 | |
| garp_master_delay 5 | |
| virtual_router_id 101 #基本MASTERとBACKUPも同じにしてかぶらないよう気をつける | |
| priority 100 | |
| nopreempt | |
| advert_int 1 | |
| smtp_alert | |
| authentication { | |
| auth_type PASS | |
| auth_pass password | |
| } | |
| virtual_ipaddress { | |
| 172.19.10.111/24 dev eth0 | |
| } | |
| } | |
| vrrp_instance VE { | |
| state BACKUP | |
| interface eth1 | |
| garp_master_delay 5 | |
| virtual_router_id 102 | |
| priority 100 | |
| nopreempt | |
| advert_int 1 | |
| smtp_alert | |
| authentication { | |
| auth_type PASS | |
| auth_pass password | |
| } | |
| virtual_ipaddress { | |
| 192.168.99.111/24 dev eth1 | |
| } | |
| } |
・test-http.conf(HTTPの設定)
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| virtual_server_group DOCKER_HTTP { | |
| 172.19.10.63 80 | |
| 172.19.10.111 80 | |
| } | |
| virtual_server group DOCKER_HTTP { | |
| delay_loop 3 | |
| lvs_sched wrr | |
| lvs_method NAT →Masp | |
| protocol TCP | |
| real_server 192.168.99.2 80 { | |
| weight 5 | |
| inhibit_on_failure | |
| HTTP_GET { | |
| url { | |
| path / | |
| status_code 200 | |
| } | |
| connect_port 80 | |
| connect_timeout 3 | |
| } | |
| } | |
| real_server 192.168.99.3 80 { | |
| weight 5 | |
| inhibit_on_failure | |
| HTTP_GET { | |
| url { | |
| path / | |
| status_code 200 | |
| } | |
| connect_port 80 | |
| connect_timeout 3 | |
| } | |
| } | |
| } |
・test-ssh.conf(SSHの設定)
10024のポートも同様に同じファイルを作ります。(firewalldの設定するので必要なし)
https://gist.github.com/RVIRUS0817/3cfdb60457b89564689c1bde5ba1d4b1
・keepalived起動
|
1 |
# systemctl start keepalived |
・keepalived status
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
# systemctl status keepalived ● keepalived.service - LVS and VRRP High Availability Monitor Loaded: loaded (/usr/lib/systemd/system/keepalived.service; enabled; vendor preset: disabled) Active: active (running) since 2016-xxxxxxxxxxxxxxxxx Process: 8451 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS) Main PID: 8452 (keepalived) CGroup: /system.slice/keepalived.service ├─8452 /usr/sbin/keepalived -D -S 2 --vrrp --check ├─8453 /usr/sbin/keepalived -D -S 2 --vrrp --check └─8454 /usr/sbin/keepalived -D -S 2 --vrrp --check 9月 29 18:19:33 xxxx Keepalived_vrrp[8454]: VRRP_Group(VG) Syncing instances to MASTER state 9月 29 18:19:33 xxxx Keepalived_vrrp[8454]: VRRP_Instance(VE) Transition to MASTER STATE 9月 29 18:19:33 xxxx Keepalived_vrrp[8454]: Opening script file /etc/keepalived/vrrp_master.sh 9月 29 18:19:33 xxxx Keepalived_vrrp[8454]: Opening script file /etc/keepalived/vrrp_state.sh 9月 29 18:19:34 xxxx Keepalived_vrrp[8454]: VRRP_Instance(VE) Entering MASTER STATE 9月 29 18:19:34 xxxx Keepalived_vrrp[8454]: VRRP_Instance(VE) setting protocol VIPs. 9月 29 18:19:34 xxxx Keepalived_vrrp[8454]: VRRP_Instance(VE) Sending gratuitous ARPs on eth1 for 192.168.99.111 9月 29 18:19:34 xxxx Keepalived_healthcheckers[8453]: Netlink reflector reports IP 192.168.99.111 added 9月 29 18:19:34 xxxx Keepalived_vrrp[8454]: VRRP_Instance(VI) Entering MASTER STATE 9月 29 18:19:39 xxxx Keepalived_vrrp[8454]: VRRP_Instance(VE) Sending gratuitous ARPs on eth1 for 192.168.99.111 |
VIPが振られてるのが分かります。
|
1 2 3 4 5 6 7 8 9 |
# ip a |grep eth0 2: eth0: >BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 inet 172.19.10.63/24 brd 172.19.10.255 scope global eth0 inet 172.19.10.111/24 scope global secondary eth0 # ip a |grep eth1 3: eth1: >BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 inet 192.168.99.1/24 brd 192.168.99.255 scope global eth1 inet 192.168.99.111/24 scope global secondary eth1 |
■DSRの設定
・sshの設定をfirewalldに振り分けする
|
1 2 3 4 |
# firewall-cmd --zone=external --add-forward-port=port=10023:proto=tcp:toaddr=192.168.99.2:toport=22 # firewall-cmd --zone=external --add-forward-port=port=10024:proto=tcp:toaddr=192.168.99.3:toport=22 # mv /etc/keepalived/conf.d/test-ssh01.conf /etc/keepalived/conf.d/test-ssh01.conf.org # mv /etc/keepalived/conf.d/test-ssh02.conf /etc/keepalived/conf.d/test-ssh02.conf.org |
・firewalld でVIP（172.19.10.111）に対してマスクセット
|
1 |
# firewall-cmd --permanent --direct --add-rule ipv4 mangle PREROUTING_direct 0 -d 172.19.10.111 -p tcp --dport 80 -j MARK --set-mark 1 |
・VIPの設定をコメントアウト
|
1 2 3 4 |
# vim /etc/keepalived/conf.d/vrrp.conf virtual_ipaddress { # 172.19.10.111/24 dev eth0 } |
・ルーティングの優先順位設定追加
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
# vi /etc/sysconfig/network-scripts/ifup-local #!/bin/sh DEVICE="$1" case "$DEVICE" in lo) /sbin/ip rule add prio 100 fwmark 1 table 100 /sbin/ip route add local 0/0 dev lo table 100 ;; esac # vi /etc/sysconfig/network-scripts/ifdown-local #!/bin/sh case "$DEVICE" in lo) /sbin/ip rule del prio 100 fwmark 1 table 100 /sbin/ip route del local 0/0 dev lo table 100 ;; esac |
・firewalld にVIPからのリクエストを リダイレクトするルール追加
|
1 |
# firewall-cmd --direct --add-rule ipv4 nat PREROUTING_direct 0 -d 172.19.10.111 -p tcp --dport 80 -j REDIRECT |
・keepalivedを止める(LVS01)
|
1 2 3 4 5 6 7 8 |
LVS01# cat /var/run/vrrp_status Thu Sep 29 18:19:33 JST 2016 GROUP VG MASTER 0 LVS02# cat /var/run/vrrp_status Thu Sep 29 18:19:27 JST 2016 GROUP VG BACKUP 0 |
|
1 2 3 4 5 6 |
LVS01# systemctl stop keepalived LVS02# cat /var/run/vrrp_status Thu Sep 29 18:19:27 JST 2016 GROUP VG MASTER 0 →切り替わってる |
■まとめ
これで冗長化はバッチグーですね。
haproxyと違ってLVSはパフォーマンスがいいですが、
設定がかなり難しいといったところ。
次回はベンチマークでお会いしましょう。
0件のコメント