'It shouldn’t be that way. It wasn’t supposed to be that way,' prime minister said at Equal Voice Foundation dinner in Ottawa