Plans for a new extension at an "eyesore" property have been rejected amid concerns it would be out of keeping with the area.