Experts said it's "particularly worrying" that people approaching their retirement are not taking this step