I was having a similar issue some time ago - for me, I would start the session being able to access everything, then at some point (could be an hour, could be several - it was never the same), I would suddenly be unable to access my chests, forge, campfire, workbench, ect. I couldn't loot anything either, though I could still open doors and the drawbridge so I know it wasn't the button that was faulty.
To fix it, we had to shut down our session and clear cache. When I loaded back in, it worked fine. It hasn't been an issue for me in ages though.