The former prime minister’s lawyers claim the frequently-made criticism is “false and defamatory”