Not that it counts for all of the difference but haven't Blackburn and Fulham both only brought a few hundred people?
From someone who looks at it from the outside, I'd reckon it's mostly due to money. Of course there is an element of how crappy Ashley has been, especially when it comes to that moment where you have to decide if you want to spend the money, but the base of the problem is that the economy is pretty crap everywhere in the world right now and it's hard to justify spending money on a season ticket to your favorite team.