One in five Australians are not attending specialist healthcare appointments due to cost concerns, a new survey has found.