A wall that one person has described as “a large part of the history of Bradford City of Culture” can be partly demolished – planners have decided.