Scientists say there is heightened vulnerability during prenatal development and early childhood