V tomto seriali, vam ukazeme niektore pekne vychytavky, ktore vam mozu pomoct pri administracii linuxu. V nasledujucom texte je popisane ako vyuzit linux na loadbalancing viacerych liniek do netu.
Prerequisities:
1. iproute2
2. kernel s podporou sietoveho multipathu
3. openvpn alebo iny tunnel software (ak netreba sifrovany tunel staci nam pouzit "ip tunnel ..." prikazy
4. (len v pripade per-packet balancingu) "-m nth" a "-j ROUTE" patch do iptables + jadra
MAPA:
intenet isp2
|
|gw2|
|
|tun1
|eth2 tun1
---------------tun0 tun0------------------
| HOME SERVER |eth1 chello modem internet isp 1 eth0| HOSTING SERVER |
| linux |-----------OOOOOOO-------+@#$%@#$TQGTDY$W%YQERGQ+------------| linux |
--------------- | | ------------------
|eth0 | |
| chello router hosting router
--------- gw1 gw
LAN
HOME SERVER
eth2: 172.16.3.2/24
eth1: 172.16.0.2/24
eth0: 192.168.0.110/24
gw1: 172.16.0.1 (chello router)
gw2: 172.16.3.1
tun0: 10.1.0.11
tun1: 10.1.0.10
HOSTING SERVER
eth0: 172.16.2.2/24
gw: 172.16.2.1 (hosting router)
tun0: 10.1.0.2
tun1: 10.1.0.1
LAN: 192.168.0.0/24
Najprv si pripravime routing. Tak aby packety odchadzajuce s adresou eth0 naozaj aj odisli cez eth0 a packety odchadzajuce s adresou eth1 odchadzali cez eth1. Za normalnych okolnosti by isli von interfejsom na ktorom je defaultroue. Cize ked by sme z internetu pingli eth1, tak icmp echo reply nepojde spat cez eth1 interface, ale cez eth0 na ktorom je zhodou okolnosti defaultrouta. Riesenie je policy routing, k tomu hned po priprave tunnelov.
nakonfigurujeme teda dva tunely, napriklad cez openvpn:
HOSTING SERVER openvpn config:
bash-2.05b# cat /etc/openvpn/tun1.conf
dev tun
ifconfig 10.1.0.1 10.1.0.10
up ./tun1.up
secret static.key
proto tcp-server
port 5000
user nobody
group nobody
ping 15
verb 3
comp-lzo
link-mtu 1500
bash-2.05b# cat /etc/openvpn/tun0.conf
dev tun
local 172.16.2.2
remote 72.16.0.2
ifconfig 10.1.0.2 10.1.0.11
up ./tun0.up
secret static.key
proto tcp-client
port 5001
user nobody
group nobody
ping 15
verb 3
link-mtu 1500
HOME SERVER openvpn config:
bash-2.05b# cat /etc/openvpn/tun1.conf
dev tun
local 172.16.3.2
remote 172.16.2.2
ifconfig 10.1.0.10 10.1.0.1
up ./tun1.up
secret static.key
proto tcp-client
port 5000
user nobody
group nobody
ping 15
verb 3
comp-lzo
link-mtu 1500
bash-2.05b# cat /etc/openvpn/tun0.conf
dev tun
ifconfig 10.1.0.11 10.1.0.2
up ./tun0.up
secret static.key
proto tcp-server
port 5001
user nobody
group nobody
ping 15
verb 3
comp-lzo
link-mtu 1500
poznamocka: tun0 mam doma ako "tcp-server", pretoze openvpn ma bug kedy tcp-client ignoruje adresu na ktoru sa ma bingnut, co sposobovalo, ze oba tunely mi komunikovali z adresou jedneho interfejsu a tym padom boli routovane len cez jednu linku.
takze teraz uz spominany policy routing:
do /etc/iproute2/rt_tables dopiseme toto:
10 openvpn
20 isp1
30 isp2
packety zo src adresami jednotlivych sietoviek budeme routovat aj fyzicky cez ne. dalej povieme domacemu kernelu ze pre packety z LANky ma pouzit routovaciu tabulku "openvpn" a zaroven nastavime v tejto routovacej tabulke multipath defaultroutu cez tunely:
bash-2.05b# ip route add default table isp1 via 172.16.0.1 dev eth1
bash-2.05b# ip route add default table isp2 via 172.16.3.1 dev eth2
bash-2.05b# ip rule add from 192.168.0.0/24 table openvpn
bash-2.05b# ip route add default table openvpn nexthop via 10.1.0.1 dev tun1 weight 1 nexthop via 10.1.0.2 dev tun0 weight 1
hosting kernelu tiez musime troska upravit routing:
bash-2.05b# ip route add 192.168.0.0/24 scope global nexthop via 10.1.0.10 dev tun1 weight 1 nexthop via 10.1.0.11 dev tun0 weight 1
v tomto momente mame chodivy per-destination balancing. to v praxi znamena, ze ked komunikujete s jednym servrom, tak cela komunikacia pojde cez jednu linku. pri komunikacii s dalsim servrom uz moze ist komunikacia druhou linkou, alebo opat tou istou (ako sa podari [podla istych pravidiel v kereli, na mojej nove 2.6.12 sa to da uz troska aj tweakovat])
este odporucam mat aspon takyto basic setup na servroch:
echo 1 > /proc/sys/net/ipv4/ip_forward
iptables -A FORWARD -m state --state RELATED, ESTABLISHED -j ACCEPT
iptables -A FORWARD -s 192.168.0.0/24 -j ACCEPT
tym povolime forwarding smerom z LANky, na HOSTING SERVERi este musime tomuto trafficu pridelit nejaku vonkajsiu adresu miesto privatnych 192.168.0.0/24:
iptables -A POSTROUTING -t nat -s 192.168.0.0/24 -j MASQUERADE
taaak, to by bola prva cast, idem si dat nejaky junk food a napisem ako spravit per packet balancing, kde sa kazdy packet trafficu rozklada podla nasich poziadaviek na jednotlive linky.
********************************************************************************************************
takze primitivny per-packet balancing mozme spravit pomocou patchu do jadra a iptables z bezneho iptables patch-o-matic-ng patchu. konkretne nam treba nth patch a ROUTE. po rozbaleni patch-o-matic-ng spustime ./runme extra a povieme co by sme radi. nasledne treba este raz prekompilovat iptables (ano patchomatic potrebuje zdrojaky iptables) a samozrejme aj kernel.
povieme kernelu na HOME SERVERi aby kazdy prvy a druhy packet posielal cez isp2 a kazdy treti cez isp1
bash-2.05b# iptables -A POSTROUTING -s 192.168.0.0/24 -d ! 192.168.0.0/24 -t mangle -m nth --counter 0 --every 3 --packet 0 -j ROUTE --oif tun1 --gw 10.1.0.1
bash-2.05b# iptables -A POSTROUTING -s 192.168.0.0/24 -d ! 192.168.0.0/24 -t mangle -m nth --counter 1 --every 3 --packet 1 -j ROUTE --oif tun1 --gw 10.1.0.1
bash-2.05b# iptables -A POSTROUTING -s 192.168.0.0/24 -d ! 192.168.0.0/24 -t mangle -m nth --counter 2 --every 3 --packet 2 -j ROUTE --oif tun0 --gw 10.1.0.2
a teraz to iste pre HOSTING SERVER:
iptables -A POSTROUTING -d 192.168.0.0/24 -t mangle -m nth --counter 0 --every 3 --packet 0 -j ROUTE --oif tun0 --gw 10.1.0.11
iptables -A POSTROUTING -d 192.168.0.0/24 -t mangle -m nth --counter 1 --every 3 --packet 1 -j ROUTE --oif tun1 --gw 10.1.0.10
iptables -A POSTROUTING -d 192.168.0.0/24 -t mangle -m nth --counter 2 --every 3 --packet 2 -j ROUTE --oif tun1 --gw 10.1.0.10
samozrejme vynechanim posledneho riadku na oboch servroch by sme docielili rozdelovanie trafficu 1:1, ale kedze ja mam jednu linku dvojnasobnej kapacity druhej tak som to rozdelil 1:2
neostava mi nic ine len popriat vela stastia dajte mi vediet ako vam to chodi, ja som bol trocha sklamany z overheadu (trocha pomohlo vyhodit lzo-comp z configu openvpn, ale aj tak..)
ak bude zase trocha casu tak skusim aj patchnutu zebru, vraj by mala tiez slusne balancovat..
takto nejak to vyzera v praxi na mojom HOME SERVERi (miesto eth2 mam eth0 spojenu s LANkou a inym gatewayom do internetu cez isp2 (bolo na to treba este vynat traffic HOMESERVRA do lokalky a na HOSTINGSERVER z balancingu pretoze, aj samotne tunnel packety ktore by mali chodit len cez isp2 by sa rozhadzovali aj na druhu linku:
iptables -A POSTROUTING -t mangle -s 192.168.0.110 -d 192.168.0.0/24 -j ACCEPT ; iptables -A POSTROUTING -t mangle -s 192.168.0.110 -d 172.16.2.2 -j ACCEPT:
Averages for the last 3014 msec
---------------------------------------------------------- | IF |Input |Output | ---------------------------------------------------------- | eth0 | 1544 kbps| 302 pk/s|| 2102 kbps| 328 pk/s| | eth1 | 706 kbps| 105 pk/s|| 101 kbps| 76 pk/s| | lo | 0 kbps| 0 pk/s|| 0 kbps| 0 pk/s| | tunl0 | 0 kbps| 0 pk/s|| 0 kbps| 0 pk/s| | gre0 | 0 kbps| 0 pk/s|| 0 kbps| 0 pk/s| | tun0 | 626 kbps| 84 pk/s|| 38 kbps| 61 pk/s| | tun1 | 1283 kbps| 173 pk/s|| 77 kbps| 122 pk/s|
----------------------------------------------------------
napisane povodne pre: kyberia.sk
