I had up to 4 players on my dedicated server about a year ago (game hasn't really changed since then), and I only noticed a bit of lag when everyone start exploring in different directions. So 3 players was optimal for mine. The server never once reached beyond 4GB usage total (including OS).
My current server is: Phenom II X4 965 @ 3.4GHz, 8GB ram, hdd. For the record, I plan on converting my current Ryzen system to the new server once AMD comes out with something even better.
So given that you have 2 additional cores over mine and possibly slightly better clock speed as well, I'd imagine your limit would be between 4 to 5 players needing just 8GB of ram. High CPU usage really only happens when ppl start exploring in different directions and it goes up a bit during horde night as well. So if you're playing co-op and ppl stick together, the CPU load is pretty low.
And as SylenThunder mentioned, graphics card makes no difference to the performance of the game server. I personally just ran mine off of onboard video and sold my graphics card.