Broker pays $45mn while Blackstone and KKR also among those penalised by US regulator