文章目录

• 版本
• 部署
• • kubeovn + kubevirt
  • dynamic-networks-controller
  • multus
• 使用
• • 创建虚拟机
  • KubeOVN 添加第二张网卡
  • virtctl 添加 interface
  • feature

版本

kubeovn 基于 feature 提交
kubevirt 基于 commit 3bee1de7b4f8bab059817cf44d636d2521dec8a9

部署

kubeovn + kubevirt

kubeovn 基于社区 install.sh 脚本部署，然后 kube-ovn-controller 镜像替换
kubevirt 可以使用 镜像 quay.io/kubevirt/virt-operator:20230320_3bee1de7b

kubevirt 需要开启 feature-gate

---
apiVersion: kubevirt.io/v1
kind: KubeVirt
metadata:name: kubevirtnamespace: kubevirt
spec:configuration:developerConfiguration: featureGates:- HotplugNICs

dynamic-networks-controller

git clone https://github.com/k8snetworkplumbingwg/multus-dynamic-networks-controller.git
cd multus-dynamic-networks-controller
kubectl apply -f manifests/dynamic-networks-controller.yaml

multus

git clone https://github.com/k8snetworkplumbingwg/multus-cni.git && cd multus-cni
cat ./deployments/multus-daemonset-thick.yml | kubectl apply -f -

apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:name: net1namespace: default
spec:config: '{"cniVersion": "0.3.1","name": "kube-ovn","plugins":[{"type":"kube-ovn","server_socket":"/run/openvswitch/kube-ovn-daemon.sock","provider": "net1.default.ovn"},{"type":"portmap","capabilities":{"portMappings":true}}]
}'

使用

创建虚拟机

apiVersion: kubevirt.io/v1
kind: VirtualMachineInstance
metadata:name: testvmi-nocloud
spec:terminationGracePeriodSeconds: 30domain:resources:requests:memory: 1024Mdevices:disks:- name: containerdiskdisk:bus: virtio- name: emptydiskdisk:bus: virtio- disk:bus: virtioname: cloudinitdiskvolumes:- name: containerdiskcontainerDisk:image: kubevirt/fedora-cloud-container-disk-demo:latest- name: emptydiskemptyDisk:capacity: "2Gi"- name: cloudinitdiskcloudInitNoCloud:userData: |-#cloud-configpassword: fedorachpasswd: { expire: False }

检查 pod

$ kubectl get pods -owide
virt-launcher-testvmi-nocloud-sl55d   2/2     Running   0          61s   10.16.0.49   node113              1/1$ kubectl exec -it virt-launcher-testvmi-nocloud-sl55d ip a
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
2: eth0:  mtu 1500 qdisc noop state DOWN group default link/ether 66:6d:26:df:ab:7c brd ff:ff:ff:ff:ff:ffinet 10.16.0.49/16 brd 10.16.255.255 scope global eth0valid_lft forever preferred_lft forever
3: k6t-eth0:  mtu 1400 qdisc noqueue state UP group default link/ether 3e:4c:d9:14:6f:1f brd ff:ff:ff:ff:ff:ffinet 169.254.75.10/32 scope global k6t-eth0valid_lft forever preferred_lft foreverinet6 fe80::3c4c:d9ff:fe14:6f1f/64 scope link valid_lft forever preferred_lft forever
4: tap0:  mtu 1400 qdisc fq_codel master k6t-eth0 state UP group default qlen 1000link/ether fa:5a:68:6c:06:50 brd ff:ff:ff:ff:ff:ffinet6 fe80::f85a:68ff:fe6c:650/64 scope link valid_lft forever preferred_lft forever
41: eth0-nic@if42:  mtu 1400 qdisc noqueue master k6t-eth0 state UP group default link/ether 3e:4c:d9:14:6f:1f brd ff:ff:ff:ff:ff:ff link-netnsid 0inet6 fe80::3c4c:d9ff:fe14:6f1f/64 scope link valid_lft forever preferred_lft forever

检查 虚拟机

$ virtctl console testvmi-nocloud
$ ip a
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
2: eth0:  mtu 1400 qdisc fq_codel state UP group default qlen 1000link/ether 00:00:00:3b:e5:7b brd ff:ff:ff:ff:ff:ffaltname enp1s0inet 10.16.0.49/16 brd 10.16.255.255 scope global dynamic noprefixroute eth0valid_lft 86313549sec preferred_lft 86313549secinet6 fe80::200:ff:fe3b:e57b/64 scope link valid_lft forever preferred_lft forever

KubeOVN 添加第二张网卡

添加第二张网卡信息，选择上面的 network-attachment-definition 和 subnet

$ kubectl edit pod  virt-launcher-testvmi-nocloud-sl55d
......
k8s.v1.cni.cncf.io/networks: '[{"interface":"net1","name":"net1","namespace":"default"}]'
net1.default.ovn.kubernetes.io/logical_switch: subnet-ipv4
......

检查，新增 net1 192.168.2.15/24

$ kubectl exec -it virt-launcher-testvmi-nocloud-sl55d ip a
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
2: eth0:  mtu 1500 qdisc noop state DOWN group default link/ether 66:6d:26:df:ab:7c brd ff:ff:ff:ff:ff:ffinet 10.16.0.49/16 brd 10.16.255.255 scope global eth0valid_lft forever preferred_lft forever
3: k6t-eth0:  mtu 1400 qdisc noqueue state UP group default link/ether 3e:4c:d9:14:6f:1f brd ff:ff:ff:ff:ff:ffinet 169.254.75.10/32 scope global k6t-eth0valid_lft forever preferred_lft foreverinet6 fe80::3c4c:d9ff:fe14:6f1f/64 scope link valid_lft forever preferred_lft forever
4: tap0:  mtu 1400 qdisc fq_codel master k6t-eth0 state UP group default qlen 1000link/ether fa:5a:68:6c:06:50 brd ff:ff:ff:ff:ff:ffinet6 fe80::f85a:68ff:fe6c:650/64 scope link valid_lft forever preferred_lft forever
41: eth0-nic@if42:  mtu 1400 qdisc noqueue master k6t-eth0 state UP group default link/ether 3e:4c:d9:14:6f:1f brd ff:ff:ff:ff:ff:ff link-netnsid 0inet6 fe80::3c4c:d9ff:fe14:6f1f/64 scope link valid_lft forever preferred_lft forever
43: net1@if44:  mtu 1400 qdisc noqueue state UP group default link/ether 00:00:00:d6:bb:bd brd ff:ff:ff:ff:ff:ff link-netnsid 0inet 192.168.2.15/24 brd 192.168.2.255 scope global net1valid_lft forever preferred_lft foreverinet6 fe80::200:ff:fed6:bbbd/64 scope link valid_lft forever preferred_lft forever

virtctl 添加 interface

添加到 虚拟机内，命名 eth1

$ virtctl addinterface testvmi-nocloud   --network-name default/net1 --iface-name eth1

检查 pod

$ kubectl exec -it virt-launcher-testvmi-nocloud-sl55d ip a
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
2: eth0:  mtu 1500 qdisc noop state DOWN group default link/ether 66:6d:26:df:ab:7c brd ff:ff:ff:ff:ff:ffinet 10.16.0.49/16 brd 10.16.255.255 scope global eth0valid_lft forever preferred_lft forever
3: k6t-eth0:  mtu 1400 qdisc noqueue state UP group default link/ether 3e:4c:d9:14:6f:1f brd ff:ff:ff:ff:ff:ffinet 169.254.75.10/32 scope global k6t-eth0valid_lft forever preferred_lft foreverinet6 fe80::3c4c:d9ff:fe14:6f1f/64 scope link valid_lft forever preferred_lft forever
4: tap0:  mtu 1400 qdisc fq_codel master k6t-eth0 state UP group default qlen 1000link/ether fa:5a:68:6c:06:50 brd ff:ff:ff:ff:ff:ffinet6 fe80::f85a:68ff:fe6c:650/64 scope link valid_lft forever preferred_lft forever
5: net1:  mtu 1500 qdisc noop state DOWN group default link/ether d6:d6:72:f1:12:5c brd ff:ff:ff:ff:ff:ffinet 192.168.2.15/24 brd 192.168.2.255 scope global net1valid_lft forever preferred_lft forever
6: k6t-net1:  mtu 1400 qdisc noqueue state UP group default link/ether 06:73:79:b3:c0:6e brd ff:ff:ff:ff:ff:ffinet 169.254.75.11/32 scope global k6t-net1valid_lft forever preferred_lft foreverinet6 fe80::473:79ff:feb3:c06e/64 scope link valid_lft forever preferred_lft forever
7: tap1:  mtu 1400 qdisc fq_codel master k6t-net1 state UP group default qlen 1000link/ether a2:7b:a6:7c:5a:81 brd ff:ff:ff:ff:ff:ffinet6 fe80::a07b:a6ff:fe7c:5a81/64 scope link valid_lft forever preferred_lft forever
41: eth0-nic@if42:  mtu 1400 qdisc noqueue master k6t-eth0 state UP group default link/ether 3e:4c:d9:14:6f:1f brd ff:ff:ff:ff:ff:ff link-netnsid 0inet6 fe80::3c4c:d9ff:fe14:6f1f/64 scope link valid_lft forever preferred_lft forever
43: net1-nic@if44:  mtu 1400 qdisc noqueue master k6t-net1 state UP group default link/ether 06:73:79:b3:c0:6e brd ff:ff:ff:ff:ff:ff link-netnsid 0inet6 fe80::473:79ff:feb3:c06e/64 scope link valid_lft forever preferred_lft forever

检查虚拟机，新增网卡

[fedora@testvmi-nocloud ~]$ ip a
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
2: eth0:  mtu 1400 qdisc fq_codel state UP group default qlen 1000link/ether 00:00:00:3b:e5:7b brd ff:ff:ff:ff:ff:ffaltname enp1s0inet 10.16.0.49/16 brd 10.16.255.255 scope global dynamic noprefixroute eth0valid_lft 86312937sec preferred_lft 86312937secinet6 fe80::200:ff:fe3b:e57b/64 scope link valid_lft forever preferred_lft forever
3: eth1:  mtu 1400 qdisc fq_codel state UP group default qlen 1000link/ether 00:00:00:d6:bb:bd brd ff:ff:ff:ff:ff:ffaltname enp8s0inet 192.168.2.15/24 brd 192.168.2.255 scope global dynamic noprefixroute eth1valid_lft 86313539sec preferred_lft 86313539secinet6 fe80::af9d:e259:1171:8b9/64 scope link noprefixroute valid_lft forever preferred_lft forever[fedora@testvmi-nocloud ~]$ ping 192.168.2.1
PING 192.168.2.1 (192.168.2.1) 56(84) bytes of data.
64 bytes from 192.168.2.1: icmp_seq=1 ttl=254 time=2.88 ms
64 bytes from 192.168.2.1: icmp_seq=2 ttl=254 time=1.34 ms

检查 libvirt xml，新增

......
......

检查 vmi，新增

......interfaces:......- bridge: {}name: eth1networks:......- multus:networkName: default/net1name: eth1
......

feature

热拔目前未实现，待 kubevirt 开发。