John Doherty was told by the woman not to come round, but he walked through the back door anyway