Far as I understand, you can only receive a buff to your max your level allows. If you click Int in the perks menu, itll say like 1/4, it means you can gain 3 more int levels for your current xp level. I think. So if it's 4/4 or x/x, you wont receive the buff; if it's z/x, you'll get it.