It isn't just the forge. You can see this in the workbench with some things as well. If you leave it open, you get experience. If you close it, you do not. Probably also with the other workstations. I suppose it makes sense to avoid getting experience by just starting a bunch of projects in multiple workstations and leaving. Still, getting experience for doing something if you do it a certain way, but not getting it when doing it a different way isn't a good situation. Better to just remove the experience altogether.
Another thing that gives a lot of experience for no effort is creating stacks of things like stone or iron. You already got experience mining it. I don't think it makes sense to get experience by turning it into stacks. I haven't tried it, but I assume it would work to create a stack, unstack it, create a new stack, unstack it, etc. That seems like a significant exploit if it works that way. And I'm sure it must because I can't see the game tracking which pieces of ore were used the last times you created stacks.