Tom Brittney has played the role of Reverend Will Davenport in Grantchester since 2019, but announced that he would be leaving the show last year and has now opened up on the decision