No voters found the referendum scary, former Scottish Secretary Alastair Carmichael has said.