Father Brown star John Burton has opened up about his concerns over the BBC One series following a huge cast shake-up, saying he was 'really worried' about the show