Perhaps, this is the same mechanic as when you come too close to a POI, wake zombies inside, and then leave the area. When you come back later the zombies respawn in their original locations.
I've noticed that in larger POIs, if a group of zombies spawns and I don't kill them all, either...