Aye its violent conduct even if its a spectator, official, player, manager, coach, physio etc
Correct. The referee would have failed his duties if he had not sent him off. Incidentally, the spectator is outside the juristiction of the referee. Only the police or FA can sanction the spectator