Nearly half of part-time and contract workers fear that taking time off could result in them being made redundant.