Labor insiders are worried that severe flooding could prevent Prime Minister Anthony Albanese from leaving the disaster zone to call the poll.