The owner of a tiny short term holiday flat in the attic of a former care home has lost their fight to keep it operating.