There are lots of different factors to consider, including genetics and your metabolism, when looking into the possible risks