Millions of people are making a "costly mistake", Yorkshire Building Society warns.