Elderly motorists have been told to think about giving up the right to drive to boost their own safety amid fears they could be putting themselves at risk.