‘It was all about my midriff,’ the actor said