The brothers and their father were out with their father on Christmas Eve