West Midlands Police said a 'small number' of people 'tried to tarnish' the massive sporting event