With railings, it seems they bumb into the railing, decide they've hit an "invisible" block and start attacking it. Since they're standing on top of the block they collided with, they start "attacking down", ie digging. Heavy guesswork though, not exactly an AI expert here.
faatal, since you're here spilling some beans, do you have any hopes getting those "AI-invisible" blocks into a properly pathable condition? Walking on horisontal poles, knowing if you can fit between "half-blocks as a wall" etc? With the variety of shapes in game, it's a tall order, but it just allows so much cheese it's not even funny. Or well, it's still damn funny, but anyway