'It may take time to see all the damage that is being done, but the damage is really quite substantial,' Dr. Tom Frieden said