It says the scheme would "significantly harm" the health of the town centre.