Virtualization / 2026-10-24
Gigabit Only Hits 350M: Two Ceilings, Don't Just Blame the Broadband
Full gigabit troubleshooting: Win10 on PVE can't saturate with an e1000 NIC, and a 2-core soft router CPU maxes out first. Two stacked bottlenecks.
Bandwidth was upgraded from 500 to 1000, but speed tests have long been stuck at 300–350M, and it's the same on different test nodes and at different times of day. The first reaction is usually that the ISP isn't delivering enough, but this kind of "flat across the whole range" curve looks more like a fixed ceiling somewhere in the middle of the path.
At the time I had two upstreams: a Windows VM on PVE, and a soft router running iStoreOS. Each had its own ceiling, and stacked together they landed right in that range.
First ceiling: the emulated NIC has no multiqueue
By default PVE attaches an e1000 to Windows guests — a purely software-emulated Intel gigabit card. Its key defect isn't throughput, it's the lack of RSS (receive-side multiqueue).
The measurements were very consistent:
- Single stream: about 170–180M
- 4 concurrent streams: about 181M
- 8 concurrent streams: still 181M
In other words, no matter how many concurrent streams you open, total throughput doesn't rise — all interrupts and packet receives are squeezed onto one core, and that core fills up first. This matches the behavior of a single-queue NIC; it's not a bandwidth problem.
Switching to virtio turns on multiqueue. In the virtual NIC settings, set the queue count to match the vCPU count (e.g. 4 cores → 4 queues); Windows will identify it as a Red Hat VirtIO Ethernet Adapter, and only once multiqueue takes effect do multiple streams actually run in parallel.
# Change the NIC model after shutdown, then power on (don't hot-change it)
qm set 101 --net0 virtio,bridge=vmbr0,queues=4
After the change, remember to verify inside the guest: Task Manager → Performance → Network, and check whether multiple queue curves appear.
Second ceiling: the soft router only has two cores
Once the VM problem was solved, I tested again and still couldn't reach 700M. So I turned my attention to the QEMU instance running iStoreOS: 2 vCPU / 1 GB RAM, and when the download saturates the CPU goes straight to 96–100%, mostly softirq.
The problem here isn't "the CPU is weak", it's the allocation ratio is wrong. All traffic in the house (including LAN SMB, backups, and streaming) goes through this soft router; its job is wire-speed forwarding, not just "getting online".
Three lessons:
- A soft router needs at least 4 cores. When cores are short, add cores first and then look at bandwidth — don't rush to swap hardware.
- 1 GB of RAM is tight. Forwarding itself doesn't eat memory, but the connection table, proxy rules, and DNS cache all take some; when memory fills up it triggers the OOM killer, which shows up as "the network drops for three seconds every now and then".
- When stress-testing the whole house, don't run backups or syncs at the same time. The soft router's CPU is a shared resource, and the test results get polluted.
Order of elimination
The value of this troubleshooting wasn't which spot got fixed, but confirming an order:
- Look at the curve shape first: completely flat → fixed ceiling; sawtooth → queues or interrupts; bad only at night → external congestion.
- Then look at the NIC model and queue count — the easiest item to overlook and the cheapest to change.
- Finally look at the CPU curve of the forwarding device, judged by the
si(softirq) share intop, not by total load.
After the fixes the whole house holds a steady 540–700M, peaking near 800M. Still short of gigabit, but that's a problem at another layer — uplink and the peer node's limits.
How to tell the two ceilings apart
When gigabit doesn't come through, first decide whether it's "link layer" or "host layer":
- Direct copy between two machines is also slow → host layer (NIC, CPU, protocol stack)
- Direct connection saturates but drops when going through the router → link layer (soft router, switch, cabling)
This one step can cut the search space in half.
Why e1000 can't saturate
PVE defaults to e1000 for Windows (an emulated Intel gigabit card); its single-stream throughput ceiling is roughly 300–400Mbps, with high CPU usage. Switch to virtio and the same link can exceed 900Mbps.
# View the VM's current NIC model
qm config 101 | grep net0
# Change to virtio (requires virtio drivers installed in the VM)
qm set 101 --net0 virtio,bridge=vmbr0
In Windows you have to install the virtio-win driver first and change it afterwards; do it in the reverse order and you'll lose the network outright.
The soft router's CPU bottleneck
One core maxed out with high ksoftirqd usage is the typical symptom of soft router forwarding:
top -H -p $(pgrep -f softirq)
If ksoftirqd/0 stays above 90% for a long time, that means single-stream load is all pinned to one core. The fix is to enable multiqueue (ethtool -L eth0 combined 4) or switch to a NIC that supports RSS.
One easily overlooked point
The speed test tool itself can become the bottleneck. iperf3 single-stream won't reach gigabit on a weak CPU; use -P 4 to test with multiple streams. A single stream running at 350M doesn't mean the link is only 350M.