Concerns have been raised after local groups said they were told trees will be planted over a wildflower meadow in Leeds despite objections.