No need to drag Speaker and Assembly into issue, says Shamseer