Victoria Police said officers were called to reports about 20 protesters were displaying a large banner near Parliament House about 6.55pm.