The only problem I can see is if its a POI you've left stuff at you'd have no way to go get it before it disappears.
For example, I, like many others, will leave a chest outside a POI for dropping off loot while I clear it out. I don't always empty it out after.... I come back for it when I get around to it. If that POI then gets chosen for a quest, a reset will delete what I've left. Currently, I'd just empty out the chest before I start the quest. But with this proposal, I'd either have to abandon the quest or forfeit whatever I've left in the chest.
Granted, not a common occurrence, but still something that would annoy me.