We had similar problems at Dundee United a few years ago when fans wanted the chairman (Jim McLean) out. It started off with protests outside the ground but these always seemed to descend into a rammy of drunken teenage idiots getting lifted and trying to look hard in front of their mates.
A supporters group was then formed called (you guessed it) United for Change. They had an elected secretary, spokesman, fundraiser etc and eventually managed to oust the chairman and buy shares in the club.
This is the kind of organisation required if you really want to make a difference - you're taking on a powerful guy and if you don't do things professionally, he'll walk all over you.
A word of warning though, Dundee United have gone right downhill ever since.