Prince Harry and Meghan Markle’s neighbour has hinted at their "lonely" life in Montecito.