I'm trying to set up a minimal devstack that can launch nova instances, some which will have public addresses, and some which will need to open connections to the public network. I'd like to be able to assign floating ips to the instances, and have traffic originating from the instances with public addresses reach the public network.
Addressing
Devstack will be running on a single Ubuntu 14.04 box with two physical interfaces. The first interface eth0 is on 10.48.4.0/22, on which I own the address 10.48.6.232; this is the management connection to the box. The second interface eth1 is on 10.48.8.0/22 and owns the addresses 10.48.11.6 and 10.48.11.57-10.48.11.59. eth1 is configured to use the 10.48.11.6 address, leaving a small pool of addresses for the floating range.
auto eth1
iface eth1 inet static
address 10.48.11.6
netmask 255.255.252.0
I'd like to use the range 10.48.11.57-10.48.11.59 as the floating IP pool. This makes up the start of my local.conf
[[local|localrc]]
# Devstack host IP eth1 address
HOST_IP=10.48.11.6
# Private network
FIXED_RANGE=10.90.100.0/24
NETWORK_GATEWAY=10.90.100.1
# Public network
Q_FLOATING_ALLOCATION_POOL=start=10.48.11.57,end=10.48.11.59
FLOATING_RANGE=10.48.8.0/22
PUBLIC_NETWORK_GATEWAY=10.48.8.1
# Public network is eth1
PUBLIC_INTERFACE=eth1
ML2
The remainder of the relevant part of my local.conf is configuring neutron and ovs to use the public network. I've followed the instructions in the comments in neutron-legacy.
# Neutron
# -------
PUBLIC_BRIDGE=br-ex
Q_USE_PROVIDERNET_FOR_PUBLIC=True
PUBLIC_PHYSICAL_NETWORK=public
OVS_BRIDGE_MAPPINGS=public:br-ex
# Neutron Provider Network
ENABLE_TENANT_TUNNELS=True
PHYSICAL_NETWORK=public
OVS_PHYSICAL_BRIDGE=br-ex
# Use ml2 and openvswitch
Q_PLUGIN=ml2
Q_ML2_PLUGIN_MECHANISM_DRIVERS=openvswitch,logger
Q_AGENT=openvswitch
enable_service q-agt
# ml2 vxlan
Q_ML2_TENANT_NETWORK_TYPE=vxlan
Q_ML2_PLUGIN_VXLAN_TYPE_OPTIONS=(vni_ranges=1001:2000)
Q_AGENT_EXTRA_AGENT_OPTS=(tunnel_types=vxlan vxlan_udp_port=8472)
Q_USE_NAMESPACE=True
Q_USE_SECGROUP=True
Resulting network
I changed the default security policy for the demo project to be permissive.
The resulting network routes traffic between the devstack host and the private subnet, but not between the devstack host and the 10.48.8.0/22, between instances and the physical 10.48.8.0/22 or between the physical 10.48.8.0/22 network and the public 10.48.8.0/22 subnet.
\ destination gateway devstack router1 private
source \ 10.48.8.1 10.48.11.6 10.48.11.57 10.90.100.0/24
physical pings X X na
10.48.8.0/22
devstack X pings pings pings
10.48.11.6
private X pings pings pings
10.90.100.0/24
Traffic leaving the public network should reach the physical network. Traffic leaving the private network should be NATed onto the public network. Traffic entering from the physical network should reach the public network.
The resulting ovs bridges are
$sudo ovs-vsctl show
33ab25b5-f5d9-4f9f-b30e-20452d099f2c
Bridge br-ex
Port phy-br-ex
Interface phy-br-ex
type: patch
options: {peer=int-br-ex}
Port "eth1"
Interface "eth1"
Port br-ex
Interface br-ex
type: internal
Bridge br-int
fail_mode: secure
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
Port int-br-ex
Interface int-br-ex
type: patch
options: {peer=phy-br-ex}
Port "tapc5733ec7-e7"
tag: 1
Interface "tapc5733ec7-e7"
type: internal
Port "qvo280f2d3e-14"
tag: 1
Interface "qvo280f2d3e-14"
Port br-int
Interface br-int
type: internal
Port "qr-9a91aae3-7c"
tag: 1
Interface "qr-9a91aae3-7c"
type: internal
Port "qr-54611e0f-77"
tag: 1
Interface "qr-54611e0f-77"
type: internal
Port "qg-9a39ed65-f0"
tag: 2
Interface "qg-9a39ed65-f0"
type: internal
Bridge br-tun
fail_mode: secure
Port br-tun
Interface br-tun
type: internal
Port patch-int
Interface patch-int
type: patch
options: {peer=patch-tun}
ovs_version: "2.0.2"
The routing table on the devstack box is
$ip route
default via 10.48.4.1 dev eth0
10.48.4.0/22 dev eth0 proto kernel scope link src 10.48.6.232
10.48.8.0/22 dev br-ex proto kernel scope link src 10.48.11.6
10.90.100.0/24 via 10.48.11.57 dev br-ex
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1
The routing table of router1 is
$sudo ip netns exec qrouter-cf0137a4-49cc-45f9-bad8-5d71340b5462 ip route
default via 10.48.8.1 dev qg-9a39ed65-f0
10.48.8.0/22 dev qg-9a39ed65-f0 proto kernel scope link src 10.48.11.57
10.90.100.0/24 dev qr-9a91aae3-7c proto kernel scope link src 10.90.100.1
What's wrong? How can I set up a simple devstack that can host both public and private interfaces for nova instances?
