Sir Keir Starmer said security concerns meant he could no longer sit in the stands at football matches.