Landlords argue Health and Social Care Moray signed a letter agreeing to leave the building next month.