Frasier star Kelsey Grammer has been told he can knock down the 200-year-old cottage - with neighbours accusing him of 'arrogance' and a total 'disregard' for the historic building