The family from Ramsbottom famously clashed with judges on the show