‘You’d have to speak to JK Rowling,’ British actor said