Surprised no one mentioned HDD speeds yet, although I didn't read all the necroed posts so maybe it was there. Running the client and save data from SSD is almost a requirement for a dedicated server.
I'm running a server on a xeon 1225v3 (äquivalent to the i5-4570, 4 core, no HT, so from 2013!), previously ran with 16GB RAM (now 32GB, because even multiple servers for different games are running in parallel), with a Debian Linux with KVM as host (on old 128GB SSD), and the 7d2d server running from within a virtualized Win10 (old 5400rpm 500GB HDD). So much much weaker system as told here, and most likely with much more software overhead.
I had this running on A18 with a maximum of 9 players and we still just got little lags on bloodmoon (all default settings), however the server was at 100% load.
Current server (A19) is running on a Phenom II X6 1100T (6-core, from 2010, so even older) which is in overal performance on par with my xeon (but divided to 6 cores, so single thread performance is LESS). It also runs Linux KVM with 7d2d server from a virtualized Win10. It's a friends server so i don't have detailed information about what disks it uses and how the server load looks like... on this server we only played with 2 players, but doubled zombies per player (16) and still absolutely no performance issues.
Afaik 12-core CPUs that are weaker than those two don't even exist. Maybe their singlethread performance is a little less, or maybe the cpu doesn't have 12 cores, but only 6 cores with SMT. Whatever, such a CPU should be just fine. That's why i said, look at the load first. I guess, the issue is NOT the CPU.
Could also be a bandwith problem. I can't tell because both my servers have 50Mbit/s upload, so way enough, and the 9-player scenario was even with all 9 players in a LAN.