I have a VPS machine with 8Core configuration 16G Ram but somehow 5 out of 10 people online are my server very lag.Even though the new VPS is 30% CPU and 40% Ram.
Read on some other forums posts before that you have to use SSDs instead of hard drives. Not sure if that's what's causing your FPS drop though since I have never personally tried it.
