“A symphony is not composed for everyone. It is something different and you cannot expect equality in it,” says the music director