FreeBSD network performance on KVM/Qemu
Today I red an article that was comparing Fedora 29 and FreeBSD 11.2 network
performance in a KVM/Qemu environment.
Since I use KVM/Qemu and also Fedora and Freebsd I powered on a couple of vm
and did my own tests.
Results are quite interesting, I expected FreeBSD to be faster but it turns out
Fedora 29 actually is.
Host system configuration:
CPU: Ryzen 7 1700x @4 GHz
Motherboard: Gigabyte X370 K7 - BIOS F23d
RAM: 2x16 GB DDR4 @3133 MHz CAS 16
HDD: Some Samsung SSD
Operating systems
Host: Fedora 29 x86_64
Fedora VM0: Fedora 29 X86_64
Fedora VM1: Fedora 29 X86_64
FreeBSD VM0: FreeBSD 11.2 x86_64
FreeBSD VM1: FreeBSD 11.2 x86_64
Virtualization techonology: Qemu+KVM
Linux kernel version: 4.18.17-300.fc29.x86_64
What I did was setup 2 hidentical Fedora 29 virtual machines and 2 hidentical
FreeBSD 11.2 virtual machines, every one of them had iperf3 installed on it.
.:. Fedora —> Fedora
[root@fedora29-0 ~]# iperf3 -c 192.168.122.236
Connecting to host 192.168.122.236, port 5201
[ 5] local 192.168.122.28 port 36652 connected to 192.168.122.236 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 3.91 GBytes 33.6 Gbits/sec 0 3.02 MBytes
[ 5] 1.00-2.00 sec 3.81 GBytes 32.7 Gbits/sec 0 3.02 MBytes
[ 5] 2.00-3.00 sec 3.90 GBytes 33.5 Gbits/sec 0 3.02 MBytes
[ 5] 3.00-4.00 sec 3.85 GBytes 33.0 Gbits/sec 0 3.02 MBytes
[ 5] 4.00-5.00 sec 3.65 GBytes 31.4 Gbits/sec 0 3.02 MBytes
[ 5] 5.00-6.00 sec 3.87 GBytes 33.3 Gbits/sec 0 3.02 MBytes
[ 5] 6.00-7.00 sec 3.85 GBytes 33.0 Gbits/sec 0 3.02 MBytes
[ 5] 7.00-8.00 sec 3.68 GBytes 31.6 Gbits/sec 0 3.02 MBytes
[ 5] 8.00-9.00 sec 3.14 GBytes 27.0 Gbits/sec 0 3.02 MBytes
[ 5] 9.00-10.00 sec 3.11 GBytes 26.7 Gbits/sec 0 3.02 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 36.8 GBytes 31.6 Gbits/sec 0 sender
[ 5] 0.00-10.04 sec 36.8 GBytes 31.5 Gbits/sec receiver
iperf Done.
.:. FreeBSD —> FreeBSD
root@freebsd1:~ # iperf3 -c 192.168.101.211
Connecting to host 192.168.101.211, port 5201
[ 5] local 192.168.101.139 port 57782 connected to 192.168.101.211 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 2.45 GBytes 21.0 Gbits/sec 0 862 KBytes
[ 5] 1.00-2.00 sec 3.20 GBytes 27.5 Gbits/sec 0 1.62 MBytes
[ 5] 2.00-3.00 sec 3.17 GBytes 27.3 Gbits/sec 0 1.77 MBytes
[ 5] 3.00-4.00 sec 3.21 GBytes 27.6 Gbits/sec 0 1.77 MBytes
[ 5] 4.00-5.00 sec 3.03 GBytes 26.0 Gbits/sec 0 1.77 MBytes
[ 5] 5.00-6.00 sec 2.75 GBytes 23.6 Gbits/sec 0 1.77 MBytes
[ 5] 6.00-7.00 sec 2.73 GBytes 23.4 Gbits/sec 0 1.77 MBytes
[ 5] 7.00-8.00 sec 2.69 GBytes 23.1 Gbits/sec 0 1.77 MBytes
[ 5] 8.00-9.00 sec 2.75 GBytes 23.6 Gbits/sec 0 1.77 MBytes
[ 5] 9.00-10.00 sec 2.89 GBytes 24.8 Gbits/sec 0 1.77 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 28.9 GBytes 24.8 Gbits/sec 0 sender
[ 5] 0.00-10.01 sec 28.9 GBytes 24.8 Gbits/sec receiver
iperf Done.
.:. FreeBSD —> Fedora
root@freebsd1:~ # iperf3 -c 192.168.122.236
Connecting to host 192.168.122.236, port 5201
[ 5] local 192.168.122.134 port 10001 connected to 192.168.122.236 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 3.21 GBytes 27.6 Gbits/sec 0 3.00 MBytes
[ 5] 1.00-2.00 sec 3.37 GBytes 29.0 Gbits/sec 0 3.00 MBytes
[ 5] 2.00-3.00 sec 3.10 GBytes 26.7 Gbits/sec 0 3.00 MBytes
[ 5] 3.00-4.00 sec 3.48 GBytes 29.9 Gbits/sec 0 3.00 MBytes
[ 5] 4.00-5.00 sec 3.61 GBytes 31.0 Gbits/sec 0 3.00 MBytes
[ 5] 5.00-6.00 sec 3.64 GBytes 31.3 Gbits/sec 0 3.00 MBytes
[ 5] 6.00-7.00 sec 3.38 GBytes 29.0 Gbits/sec 0 3.00 MBytes
[ 5] 7.00-8.00 sec 3.72 GBytes 31.9 Gbits/sec 0 3.00 MBytes
[ 5] 8.00-9.00 sec 3.71 GBytes 31.9 Gbits/sec 0 3.00 MBytes
[ 5] 9.00-10.00 sec 3.58 GBytes 30.7 Gbits/sec 0 3.00 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 34.8 GBytes 29.9 Gbits/sec 0 sender
[ 5] 0.00-10.09 sec 34.8 GBytes 29.6 Gbits/sec receiver
iperf Done.
.:. Fedora —> FreeBSD
[root@fedora29-0 ~]# iperf3 -c 192.168.122.72
Connecting to host 192.168.122.72, port 5201
[ 5] local 192.168.122.28 port 50430 connected to 192.168.122.72 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 2.20 GBytes 18.9 Gbits/sec 0 867 KBytes
[ 5] 1.00-2.00 sec 2.89 GBytes 24.8 Gbits/sec 0 1.60 MBytes
[ 5] 2.00-3.00 sec 2.82 GBytes 24.2 Gbits/sec 0 1.77 MBytes
[ 5] 3.00-4.00 sec 2.89 GBytes 24.9 Gbits/sec 0 1.77 MBytes
[ 5] 4.00-5.00 sec 2.82 GBytes 24.2 Gbits/sec 0 1.77 MBytes
[ 5] 5.00-6.00 sec 2.96 GBytes 25.5 Gbits/sec 0 1.77 MBytes
[ 5] 6.00-7.00 sec 3.34 GBytes 28.7 Gbits/sec 0 1.77 MBytes
[ 5] 7.00-8.00 sec 3.39 GBytes 29.1 Gbits/sec 0 1.77 MBytes
[ 5] 8.00-9.00 sec 3.24 GBytes 27.8 Gbits/sec 0 1.77 MBytes
[ 5] 9.00-10.00 sec 2.85 GBytes 24.5 Gbits/sec 0 1.77 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 29.4 GBytes 25.3 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 29.4 GBytes 25.3 Gbits/sec receiver
iperf Done.
I honestly expected FreeBSD to actually be faster than Fedora, apparently this
isn’t the case though.
I also tried tweaking FreeBSD network stack, like disabling the mechanism that
uses the first two bytes of every TCP frame to build entropy, but without
registering any major impact on networking performance.
Keep in mind that this is just some quick and dirty comparison, I might have
missed some tweaks, virtio network driver might not be 100% supported by BSD or
something else.
.:. Useful links
FreeBSD network performance tuning
calomel FreeBSD networking tuning guide