The family now have three years to renovate the property - or the town takes it back