Plans to build a new home in Richmond have been refused by North Yorkshire Council after concerns were raised about road safety.