The party said this year marks the lowest number of new affordable homes built in recent years in Scotland.