John Swinney is “all over the place in terms of defence”, the Scottish Secretary has said.