FreeBSD network performance on KVM/Qemu · uwot.eu
another meaningless subtitle

FreeBSD network performance on KVM/Qemu

· by fabio · Read in about 5 min · (1018 Words)
Fedora FreeBSD qemu KVM

Today I red an article that was comparing Fedora 29 and FreeBSD 11.2 network performance in a KVM/Qemu environment.
Since I use KVM/Qemu and also Fedora and Freebsd I powered on a couple of vm and did my own tests.
Results are quite interesting, I expected FreeBSD to be faster but it turns out Fedora 29 actually is.
Host system configuration:

CPU:            Ryzen 7 1700x @4 GHz
Motherboard:    Gigabyte X370 K7 - BIOS F23d
RAM:            2x16 GB DDR4 @3133 MHz CAS 16
HDD:            Some Samsung SSD

Operating systems
Host:           Fedora 29 x86_64
Fedora VM0:     Fedora 29 X86_64
Fedora VM1:     Fedora 29 X86_64
FreeBSD VM0:    FreeBSD 11.2 x86_64
FreeBSD VM1:    FreeBSD 11.2 x86_64

Virtualization techonology: Qemu+KVM
Linux kernel version: 4.18.17-300.fc29.x86_64

What I did was setup 2 hidentical Fedora 29 virtual machines and 2 hidentical FreeBSD 11.2 virtual machines, every one of them had iperf3 installed on it.

.:. Fedora —> Fedora

[root@fedora29-0 ~]# iperf3 -c 192.168.122.236
Connecting to host 192.168.122.236, port 5201
[  5] local 192.168.122.28 port 36652 connected to 192.168.122.236 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  3.91 GBytes  33.6 Gbits/sec    0   3.02 MBytes       
[  5]   1.00-2.00   sec  3.81 GBytes  32.7 Gbits/sec    0   3.02 MBytes       
[  5]   2.00-3.00   sec  3.90 GBytes  33.5 Gbits/sec    0   3.02 MBytes       
[  5]   3.00-4.00   sec  3.85 GBytes  33.0 Gbits/sec    0   3.02 MBytes       
[  5]   4.00-5.00   sec  3.65 GBytes  31.4 Gbits/sec    0   3.02 MBytes       
[  5]   5.00-6.00   sec  3.87 GBytes  33.3 Gbits/sec    0   3.02 MBytes       
[  5]   6.00-7.00   sec  3.85 GBytes  33.0 Gbits/sec    0   3.02 MBytes       
[  5]   7.00-8.00   sec  3.68 GBytes  31.6 Gbits/sec    0   3.02 MBytes       
[  5]   8.00-9.00   sec  3.14 GBytes  27.0 Gbits/sec    0   3.02 MBytes       
[  5]   9.00-10.00  sec  3.11 GBytes  26.7 Gbits/sec    0   3.02 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  36.8 GBytes  31.6 Gbits/sec    0             sender
[  5]   0.00-10.04  sec  36.8 GBytes  31.5 Gbits/sec                  receiver

iperf Done.

.:. FreeBSD —> FreeBSD

root@freebsd1:~ # iperf3 -c 192.168.101.211
Connecting to host 192.168.101.211, port 5201
[  5] local 192.168.101.139 port 57782 connected to 192.168.101.211 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  2.45 GBytes  21.0 Gbits/sec    0    862 KBytes       
[  5]   1.00-2.00   sec  3.20 GBytes  27.5 Gbits/sec    0   1.62 MBytes       
[  5]   2.00-3.00   sec  3.17 GBytes  27.3 Gbits/sec    0   1.77 MBytes       
[  5]   3.00-4.00   sec  3.21 GBytes  27.6 Gbits/sec    0   1.77 MBytes       
[  5]   4.00-5.00   sec  3.03 GBytes  26.0 Gbits/sec    0   1.77 MBytes       
[  5]   5.00-6.00   sec  2.75 GBytes  23.6 Gbits/sec    0   1.77 MBytes       
[  5]   6.00-7.00   sec  2.73 GBytes  23.4 Gbits/sec    0   1.77 MBytes       
[  5]   7.00-8.00   sec  2.69 GBytes  23.1 Gbits/sec    0   1.77 MBytes       
[  5]   8.00-9.00   sec  2.75 GBytes  23.6 Gbits/sec    0   1.77 MBytes       
[  5]   9.00-10.00  sec  2.89 GBytes  24.8 Gbits/sec    0   1.77 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  28.9 GBytes  24.8 Gbits/sec    0             sender
[  5]   0.00-10.01  sec  28.9 GBytes  24.8 Gbits/sec                  receiver

iperf Done.

.:. FreeBSD —> Fedora

root@freebsd1:~ # iperf3 -c 192.168.122.236
Connecting to host 192.168.122.236, port 5201
[  5] local 192.168.122.134 port 10001 connected to 192.168.122.236 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  3.21 GBytes  27.6 Gbits/sec    0   3.00 MBytes       
[  5]   1.00-2.00   sec  3.37 GBytes  29.0 Gbits/sec    0   3.00 MBytes       
[  5]   2.00-3.00   sec  3.10 GBytes  26.7 Gbits/sec    0   3.00 MBytes       
[  5]   3.00-4.00   sec  3.48 GBytes  29.9 Gbits/sec    0   3.00 MBytes       
[  5]   4.00-5.00   sec  3.61 GBytes  31.0 Gbits/sec    0   3.00 MBytes       
[  5]   5.00-6.00   sec  3.64 GBytes  31.3 Gbits/sec    0   3.00 MBytes       
[  5]   6.00-7.00   sec  3.38 GBytes  29.0 Gbits/sec    0   3.00 MBytes       
[  5]   7.00-8.00   sec  3.72 GBytes  31.9 Gbits/sec    0   3.00 MBytes       
[  5]   8.00-9.00   sec  3.71 GBytes  31.9 Gbits/sec    0   3.00 MBytes       
[  5]   9.00-10.00  sec  3.58 GBytes  30.7 Gbits/sec    0   3.00 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  34.8 GBytes  29.9 Gbits/sec    0             sender
[  5]   0.00-10.09  sec  34.8 GBytes  29.6 Gbits/sec                  receiver

iperf Done.

.:. Fedora —> FreeBSD

[root@fedora29-0 ~]# iperf3 -c 192.168.122.72
Connecting to host 192.168.122.72, port 5201
[  5] local 192.168.122.28 port 50430 connected to 192.168.122.72 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  2.20 GBytes  18.9 Gbits/sec    0    867 KBytes       
[  5]   1.00-2.00   sec  2.89 GBytes  24.8 Gbits/sec    0   1.60 MBytes       
[  5]   2.00-3.00   sec  2.82 GBytes  24.2 Gbits/sec    0   1.77 MBytes       
[  5]   3.00-4.00   sec  2.89 GBytes  24.9 Gbits/sec    0   1.77 MBytes       
[  5]   4.00-5.00   sec  2.82 GBytes  24.2 Gbits/sec    0   1.77 MBytes       
[  5]   5.00-6.00   sec  2.96 GBytes  25.5 Gbits/sec    0   1.77 MBytes       
[  5]   6.00-7.00   sec  3.34 GBytes  28.7 Gbits/sec    0   1.77 MBytes       
[  5]   7.00-8.00   sec  3.39 GBytes  29.1 Gbits/sec    0   1.77 MBytes       
[  5]   8.00-9.00   sec  3.24 GBytes  27.8 Gbits/sec    0   1.77 MBytes       
[  5]   9.00-10.00  sec  2.85 GBytes  24.5 Gbits/sec    0   1.77 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  29.4 GBytes  25.3 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  29.4 GBytes  25.3 Gbits/sec                  receiver

iperf Done.

I honestly expected FreeBSD to actually be faster than Fedora, apparently this isn’t the case though.
I also tried tweaking FreeBSD network stack, like disabling the mechanism that uses the first two bytes of every TCP frame to build entropy, but without registering any major impact on networking performance.
Keep in mind that this is just some quick and dirty comparison, I might have missed some tweaks, virtio network driver might not be 100% supported by BSD or something else.

FreeBSD network performance tuning
calomel FreeBSD networking tuning guide