‘We just need a little time apart because we are sort of an allergy to one another,’ Phillips says