An appeal has been quashed for plans for five new homes in an Oxfordshire village, despite local support.