The council leader said he would like to spend more time on his ward councillor position and with his family