The law permitting sending-off defines violence as "excessive force" and as the stewards didn't seem to think it was excessive, I wonder why the referee thought so.
I believe the inclusion of "any other person" in the law is to prevent actions such as Cantona's infamous drop-kick, and it was never intended to cover the reasonable apprehension of a miscreant.
Yet again, a case of the man in the middle believing he is above the law instead of the servant of the law. Why is an ounce of common sense too much to ask for?