Understand tolerance as a social contract and not as a moral obligation. Tolerance is about piece within a society, not about right or wrong. If you do, there is no paradox, because the contract is broken and therefore does no longer apply. It also allows you to counter far right ideologies from whomever they come.