docker网络利用openvswitch对广播报文限速

话不多说，直接开整

1. 准备环境

CentOS7（1511）

Docker（1.11.2）

OpenVswitch（2.4.0-1）

Iperf（2.0.8-2）

2. 网络拓扑

3. 构建网络

3.1 创建容器

docker run -itd --name=magine1 --net=none 10.0.0.100:5000/centos /bin/bash
 ddfb7c3015f34f178d7efc112e6d9350b1553c07aaed4164a861c339346dba7e
 
 docker run -itd --name=magine2 --net=none 10.0.0.100:5000/centos /bin/bash
 67fe66ff23896010e5590dc8f841a82a957fd5b8694ef2d9b04b316a110854dc

3.2 创建虚拟网卡

容器ddfb7c30
 ip link add qbi-ddfb7c30 type veth peer name qbv-ddfb7c30
 ip link add qvb-ddfb7c30 type veth peer name qvo-ddfb7c30
 
 容器f52ebaa9
 ip link add qbi-f52ebaa9 type veth peer name qbv-f52ebaa9
 ip link add pvb-f52ebaa9 type veth peer name qvo-f52ebaa9

3.3 创建ovs拓扑

基础网桥
 ovs-vsctl add-br br-int
 ovs-vsctl add-br br-em1
 ovs-vsctl add-port br-em1 em1
 
 容器ddfb7c30
 ovs-vsctl add-br qbr-ddfb7c30
 ovs-vsctl add-port br-int qbi-ddfb7c30
 ovs-vsctl add-port qbr-ddfb7c30 qbv-ddfb7c30
 ovs-vsctl add-port qbr-ddfb7c30 qvb-ddfb7c30
 
 容器f52ebaa9
 ovs-vsctl add-br qbr-f52ebaa9
 ovs-vsctl add-port br-int qbi-f52ebaa9
 ovs-vsctl add-port qbr-f52ebaa9 qbv-f52ebaa9
 ovs-vsctl add-port qbr-f52ebaa9 qvb-f52ebaa9

3.4 构建容器NameSpace

容器ddfb7c30
 docker inspect -f  '{{ .State.Pid }}' ddfb7c3015f3
 32687
 
 关联系统namespace路径
 ln -s /proc/32687/ns/net /run/netns/ddfb7c3015f3
 
 查看容器ns
 ip netns
 ddfb7c3015f3
 
 容器分入网卡
 ip link set qvo-ddfb7c30 netns ddfb7c3015f3
 
 配置IP
 ip netns exec ddfb7c3015f3 ip addr add 10.1.1.2/24 dev qvo-ddfb7c30
 
 激活网卡
 ip netns exec ddfb7c3015f3 ip link set qvo-ddfb7c30 up
 ip link set qvb-ddfb7c30 up
 ip link set qbi-ddfb7c30 up
 ip link set qbv0ddfb7c30 up
 
 容器f52ebaa9
 以上操作

3.5 ovs创建队列和流表

容器ddfb7c30
 
 在qbv-ddfb7c30上创建1条qos，2条队列，其中队列0最高1Gbit，最低500Mbit，队列1最高1Mbit，最低1Mbit
 $ ovs-vsctl set port qbv-ddfb7c30 qos=@newqos -- --id=@newqos create qos type=linux-htb other-config:max-rate=1000000000 queues=0=@q0,1=@q1 -- --id=@q0 create queue other-config:min-rate=500000000 other-config:max-rate=1000000000 -- --id=@q1 create queue other-config:min-rate=1000000 other-config:max-rate=1000000
 
 c5c8d139-7443-49f7-81d5-d92c8754e5de
 ea91d271-618d-442f-b88c-6ab0bfc377ba
 b6f63f0d-193d-4e9d-95c8-2af7e3aac247
 
 查看qbr-ddfb7c30端口信息
 $ ovs-ofctl show qbr-ddfb7c30
 OFPT_FEATURES_REPLY (xid=0x2): dpid:0000aaee33131e41
 n_tables:254, n_buffers:256
 capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
 actions: output enqueue set_vlan_vid set_vlan_pcp strip_vlan mod_dl_src mod_dl_dst mod_nw_src mod_nw_dst mod_nw_tos mod_tp_src mod_tp_dst
 2(qvb-ddfb7c30): addr:96:60:70:41:70:0c
  config:     0
  state:      0
  current:    10GB-FD COPPER
  speed: 10000 Mbps now, 0 Mbps max
 3(qbv-ddfb7c30): addr:66:19:ab:4e:7e:09
  config:     0
  state:      0
  current:    10GB-FD COPPER
  speed: 10000 Mbps now, 0 Mbps max
 LOCAL(qbr-ddfb7c30): addr:aa:ee:33:13:1e:41
  config:     PORT_DOWN
  state:      LINK_DOWN
  speed: 0 Mbps now, 0 Mbps max
 OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0
 
 在qbr-ddfb7c30上创建两条流表，用于匹配广播地址和单播地址
 $ ovs-ofctl add-flow qbr-ddfb7c30 "table=1, in_port=2, dl_src=00:00:00:00:00:00/01:00:00:00:00:00, actions=enqueue:3:0"
 
 $ ovs-ofctl add-flow qbr-ddfb7c30 "table=0, in_port=2, dl_src=01:00:00:00:00:00/01:00:00:00:00:00, actions=enqueue:3:1"
 
 **查看流表**
 $ ovs-ofctl dump-flows qbr-ddfb7c30
 NXST_FLOW reply (xid=0x4):
 cookie=0x0, duration=264250.612s, table=0, n_packets=1465040, n_bytes=31895512642, idle_age=5938, hard_age=65534, priority=0 actions=NORMAL
 cookie=0x0, duration=2.382s, table=0, n_packets=0, n_bytes=0, idle_age=2, in_port=2,dl_src=01:00:00:00:00:00/01:00:00:00:00:00 actions=enqueue:3:1
 cookie=0x0, duration=11.006s, table=1, n_packets=0, n_bytes=0, idle_age=11, in_port=2,dl_src=00:00:00:00:00:00/01:00:00:00:00:00 actions=enqueue:3:0
 
 
 容器f52ebaa9
 以上操作

4. 测试

由于没有测试广播风暴的工具，就以iperf测试单播为准

4.1 iperf server

宿主机

iperf -s

Server listening on TCP port 5001

TCP window size: 85.3 KByte (default)

4.2 容器 iperf Client

不配置qos

[root@docker1 ~]# ip netns exec ddfb7c3015f3 iperf -c 10.1.1.1 -i 1 -t 10

[ 3] local 10.1.1.2 port 38143 connected with 10.1.1.1 port 5001

[ ID] Interval Transfer Bandwidth

[ 3] 0.0- 1.0 sec 3.05 GBytes 26.2 Gbits/sec

[ 3] 1.0- 2.0 sec 3.24 GBytes 27.9 Gbits/sec

[ 3] 2.0- 3.0 sec 3.24 GBytes 27.8 Gbits/sec

[ 3] 3.0- 4.0 sec 3.29 GBytes 28.3 Gbits/sec

[ 3] 4.0- 5.0 sec 3.35 GBytes 28.8 Gbits/sec

[ 3] 5.0- 6.0 sec 3.31 GBytes 28.4 Gbits/sec

[ 3] 6.0- 7.0 sec 3.29 GBytes 28.2 Gbits/sec

[ 3] 7.0- 8.0 sec 3.32 GBytes 28.5 Gbits/sec

[ 3] 8.0- 9.0 sec 3.28 GBytes 28.2 Gbits/sec

[ 3] 9.0-10.0 sec 3.26 GBytes 28.0 Gbits/sec

[ 3] 0.0-10.0 sec 32.6 GBytes 28.0 Gbits/sec

配置qos，单播走队列0（正常队列。限速1Gbit）

$ ovs-ofctl add-flow qbr-ddfb7c30 "table=0, in_port=2, dl_src=00:00:00:00:00:00/01:00:00:00:00:00, actions=enqueue:3:0"

[root@docker1 ~]# ip netns exec ddfb7c3015f3 iperf -c 10.1.1.1 -i 1 -t 10

[ 3] local 10.1.1.2 port 38157 connected with 10.1.1.1 port 5001

[ ID] Interval Transfer Bandwidth

[ 3] 0.0- 1.0 sec 117 MBytes 980 Mbits/sec

[ 3] 1.0- 2.0 sec 115 MBytes 965 Mbits/sec

[ 3] 2.0- 3.0 sec 114 MBytes 952 Mbits/sec

[ 3] 3.0- 4.0 sec 114 MBytes 955 Mbits/sec

[ 3] 4.0- 5.0 sec 114 MBytes 955 Mbits/sec

[ 3] 5.0- 6.0 sec 114 MBytes 954 Mbits/sec

[ 3] 6.0- 7.0 sec 114 MBytes 954 Mbits/sec

[ 3] 7.0- 8.0 sec 115 MBytes 967 Mbits/sec

[ 3] 8.0- 9.0 sec 114 MBytes 954 Mbits/sec

[ 3] 9.0-10.0 sec 114 MBytes 955 Mbits/sec

[ 3] 0.0-10.0 sec 1.12 GBytes 958 Mbits/sec

配置qos，单播走队列1（广播队列。限速1Mbit）

$ ovs-ofctl add-flow qbr-ddfb7c30 "table=0, in_port=2, dl_src=00:00:00:00:00:00/01:00:00:00:00:00, actions=enqueue:3:0"

[root@docker1 ~]# ip netns exec ddfb7c3015f3 iperf -c 10.1.1.1 -i 1 -t 10

[ 3] local 10.1.1.2 port 38177 connected with 10.1.1.1 port 5001

[ ID] Interval Transfer Bandwidth

[ 3] 0.0- 1.0 sec 83.4 KBytes 683 Kbits/sec

[ 3] 1.0- 2.0 sec 12.7 KBytes 104 Kbits/sec

[ 3] 2.0- 3.0 sec 276 KBytes 2.26 Mbits/sec

[ 3] 3.0- 4.0 sec 136 KBytes 1.11 Mbits/sec

[ 3] 4.0- 5.0 sec 127 KBytes 1.04 Mbits/sec

[ 3] 5.0- 6.0 sec 191 KBytes 1.56 Mbits/sec

[ 3] 6.0- 7.0 sec 191 KBytes 1.56 Mbits/sec

[ 3] 7.0- 8.0 sec 255 KBytes 2.09 Mbits/sec

[ 3] 8.0- 9.0 sec 382 KBytes 3.13 Mbits/sec

[ 3] 9.0-10.0 sec 382 KBytes 3.13 Mbits/sec

[ 3] 0.0-10.2 sec 1.99 MBytes 1.64 Mbits/sec

-------------------

结尾吐槽下，这个编辑器粘贴代码很蛋疼啊，显示不完整，求好心人帮助。