We have realised that the ban button can feel more powerful than freedom of speech itself as it eats away at the cultural fabric of society