Wyre Forest District Council has hit back at claims they bulldozed a historic building "unlawfully."