The housing minister has said he is wary of predicting when house prices would moderate – despite his party leader predicting it would be within several years.