Tory leader criticised for using painful day for families to 'score political points'