Buy-to-let landlords are being hit by higher interest rates, higher costs and a less friendly tax regime.