Forcing Government staff to work from their offices will not cut driving test waiting times, a transport minister has said.