It has also slapped the family with a planning enforcement notice