A family court judge said the injuries were either caused by the mum or dad