Skip to main content

linux - How to assign multiple public IP-Adresses for 2 KVM-Guests

itemprop="text">

I am new to this whole topic and I try
for days now to figure out how to assign multiple public ip-addresses to KVM-guests
through a KVM host. I found tons of examples howto get such a setup with 1 public IP
running.



Here is my setup:
The Server
has only one NIC/MAC and runs 2 KVM-Guests with apache(and other stuff). Both
guest-environments are ubuntu server 11.10 and must run in separate VMs. The 5 public
ip-addresses are used to handle SSL-certificates and other stuff. The first VM should
use 3 of the 5 addresses/certificates. The second VM gets the rest. The apache-stuff is
configured correctly.



I have tried a number of
differend ways via iptables to route the traffic from the hosts NIC to the guest-NICs.
In spite of the fact that one way was the right one but only wrong implemented, I leave
the details untold to leave you unprepossessed. The question is: Whats the ideal way it
should be done?



The following conditions should
be met:





  • Apache
    must get the original ip-address of the visitor

  • Apache
    must know, what public-ip address was ultilized to use the right
    ssl-vhost

  • The traffic must not be routed through a
    (reverse-)proxy on the host, since there are 2 other non-http-services, on other
    VM-guests, that should be accessible from public. And: Only sshd should listen directly
    on the host - nothing else

  • Each VM should be able to
    access the internet directly.

  • The network in the
    data-center is switched MAC-based. As I figured out, the only way to comunicate with the
    internet is through eth0 and its
    MAC-address.



If I would
discard all the virtualization-stuff, this would be perfectly easy, as apache get the
request directly from a specific
ip-address.




I am open for any working
solution.



src="https://i.stack.imgur.com/a30Ni.png" alt="Diagram">



Answer




Use a bridge on your dom0 (e.g. KVM Host) WAN interface. This requires
installing bridge-utils package. Since this is Debian-based
distro, you may configure it in
/etc/network/interfaces:



iface
eth0 inet manual

auto br_wan

iface br_wan inet
dhcp
# Assuming DHCP to get address, otherwise migrate all WAN connection
options here
#address 192.168.122.0
bridge_ports eth0
tap_guest1
bridge_stp off
bridge_maxwait 0
bridge_fd
0
pre-up ip tuntap add dev tap_guest1 user guest1 mode tap
# This
command is required if your ISP allocates static IPs depending on MAC address

# You shouldn't use this but might be handy some time

#pre-up
sysctl -q -w net/ipv4/conf/tap_guest1/proxy_arp=1
post-down ip tuntap del
tap_guest1 mode
tap


Pre-up commands
set up TAP interface to connect your KVM guest to a bridge. Note that this setup allows
to run kvm from non-privileged user guest1. Note that setting
net.ipv4.ip_forward = 1 with sysctl might be usefull as
well.



I have used ip
tuntap
command from iproute2 package. It's not yet
documented in the Debian package but soon will be available in upstream's manual page.
Since this package is installed on every Debian-based server, you won't need to install
uml-utilities or openvpn package to
just create these interfaces.



This approach
sure lacks some elegance to manage lots of tap interfaces, because you'll need to create
similar pre-up and post-down lines as for tap_guest1 interface.
This can be fixed by writing additional scripts in
/etc/network/pre-up.d and
/etc/network/post-down.d. It is also a problem if you want to
reconfigure br_wan interface with ifdown/ifup scripts while KVM
guests are still running — you'll need either to remove all interfaces except
eth0 from bridge configuration and detach them from bridge
manually (don't forget to attach them back after bridge reconfiguration then) or
shutdown all KVM instances running on a
bridge.




Another way, perhaps more
clean, is to write custom ifup script for KVM itself and use it in
script option for your NIC. You can get an example in
/etc/qemu-ifup. See href="http://linux.die.net/man/1/qemu-kvm" rel="nofollow noreferrer">kvm manual
page for details.



Then you can run
your KVM box like this:



kvm -net
nic,model=virtio,macaddr=12:34:56:78:9a:bc \
-net
tap,ifname=tap_guest1,script=no,downscript=no \
-boot c -nographic -display
none -daemonize \

guest1-drive.qcow2



Setting
several IP addresses on one interface for your KVM guest can be done manually with
command



ip address add
aaa.bbb.ccc.101/24 dev
eth0


Or permanently in
/etc/network/interfaces like
this:



auto eth0
eth0:1
iface eth0 inet static
address
aaa.bbb.ccc.100

network aaa.bbb.ccc.0
netmask
255.255.255.0
broadcast aaa.bbb.ccc.255
gateway
aaa.bbb.ccc.1

iface eth0:1 inet static
address
aaa.bbb.ccc.101
network aaa.bbb.ccc.0
netmask
255.255.255.0
broadcast aaa.bbb.ccc.255

gateway
aaa.bbb.ccc.1


Note
that if your datacenter/provider does not expect you to reveal additional boxes on the
same net he might not configure them and they will be unavailable. In this case you
might want to create internal bridge and use iptables to forward packets between your
WAN interface and this bridge using DNAT and SNAT. Assuming your local virtual bridge
network is 10.0.0.0/8, your guest1 is 10.0.0.2 you'll need
this:



iptables -t nat -A
PREROUTING --dst aaa.bbb.ccc.100 -p tcp --dport 80 -j DNAT --to-destination
10.0.0.2
iptables -t nat -A PREROUTING --dst aaa.bbb.ccc.101 -p tcp --dport 80
-j DNAT --to-destination 10.0.0.2
...
iptables -t nat -A POSTROUTING
-p tcp --dst 10.0.0.2 -j SNAT --to-source
aaa.bbb.ccc.100



Note
that you'll need as much DNAT commands as external IPs per KVM guest you have, but only
one SNAT rule to give access to the internet. Also you can allow only HTTP/HTTPS/SSH
traffic by allowing only desired ports. If you omit the --dport
statement then all ports will be forwarded. Your KVM guest should have static network
settings with KVM host as default gateway unless you're willing to host DHCP
server.


Comments

Popular posts from this blog

linux - iDRAC6 Virtual Media native library cannot be loaded

When attempting to mount Virtual Media on a iDRAC6 IP KVM session I get the following error: I'm using Ubuntu 9.04 and: $ javaws -version Java(TM) Web Start 1.6.0_16 $ uname -a Linux aud22419-linux 2.6.28-15-generic #51-Ubuntu SMP Mon Aug 31 13:39:06 UTC 2009 x86_64 GNU/Linux $ firefox -version Mozilla Firefox 3.0.14, Copyright (c) 1998 - 2009 mozilla.org On Windows + IE it (unsurprisingly) works. I've just gotten off the phone with the Dell tech support and I was told it is known to work on Linux + Firefox, albeit Ubuntu is not supported (by Dell, that is). Has anyone out there managed to mount virtual media in the same scenario?

hp proliant - Smart Array P822 with HBA Mode?

We get an HP DL360 G8 with an Smart Array P822 controller. On that controller will come a HP StorageWorks D2700 . Does anybody know, that it is possible to run the Smart Array P822 in HBA mode? I found only information about the P410i, who can run HBA. If this is not supported, what you think about the LSI 9207-8e controller? Will this fit good in that setup? The Hardware we get is used but all original from HP. The StorageWorks has 25 x 900 GB SAS 10K disks. Because the disks are not new I would like to use only 22 for raid6, and the rest for spare (I need to see if the disk count is optimal or not for zfs). It would be nice if I'm not stick to SAS in future. As OS I would like to install debian stretch with zfs 0.71 as file system and software raid. I have see that hp has an page for debian to. I would like to use hba mode because it is recommend, that zfs know at most as possible about the disk, and I'm independent from the raid controller. For us zfs have many benefits,

apache 2.2 - Server Potentially Compromised -- c99madshell

So, low and behold, a legacy site we've been hosting for a client had a version of FCKEditor that allowed someone to upload the dreaded c99madshell exploit onto our web host. I'm not a big security buff -- frankly I'm just a dev currently responsible for S/A duties due to a loss of personnel. Accordingly, I'd love any help you server-faulters could provide in assessing the damage from the exploit. To give you a bit of information: The file was uploaded into a directory within the webroot, "/_img/fck_uploads/File/". The Apache user and group are restricted such that they can't log in and don't have permissions outside of the directory from which we serve sites. All the files had 770 permissions (user rwx, group rwx, other none) -- something I wanted to fix but was told to hold off on as it wasn't "high priority" (hopefully this changes that). So it seems the hackers could've easily executed the script. Now I wasn't able