I was the same but it was more Pardew who ruined it for me as opposed to Ashley. I hated, and still hate, the corporate greed that football is all about nowadays.
Ashley started off my hatred for NUFC and I then, as you suggested, stopped enjoying most football without the emotional attachment. But then the Pardew era came along and it was a different level - he was on my tele box every Saturday telling me what I should be thinking and when I should be applauding/cheering, what I should expect from the club and it killed my love for the club/game. He was a woeful excuse of a football manager but it was more the off the field stuff that killed me off - he just didn't get this club or it's fans at all.
I never attended a single game for around 3 years until I surrendered on Boxing day against Everton. We got beat in the last seconds, I never got excited nor disappointed, and it pretty much vindicated my absence from SJP. There was nothing there - I didn't care that McClaren was hopeless and that we were clearly spiraling towards an inevitable relegation. But then I'd go out on a weekend, get pissed, and start talking football with my match going mates. I ended up realising that I do still care - I'm just trying my best not to.
I went to Rafas first game at Leicester, same shite but let's give it a chance I thought - he can't perform miracles over night. As the season came to its climax, I started sitting in the bar cheering every time we scored. I don't what it was but I'd went from saying "I hope we go down, the bastards over there deserve to be in the conference. I want numerous relegations" to hoping we'd stay up. I think this may have something to do with the fact that it was either us or Sunderland who would go down.
In the Summer, Rafa agreed to take on the job full time and there was a buzz, an excitement among my match going friends who couldn't wait for the season to start. I agreed to attend Fulham away, Rafas first game since taking on the full time role. I'd attended 3 games in over 3 years since the beginning of my sabbatical and we'd lost all the games 1-0. I've since been to Wolves at home in the cup and Norwich at home in the league. The Norwich game will stay in my memory for a long, long time. I definitely had the buzz back. I talked about the game in the pub, at work and to taxi drivers for a couple of weeks after. It was glorious.
I've admitted I can't let it go. I'm enjoying it at the minute. It feels like we're becoming a football club again - we have a goal that seems to be in relation to football once more and not for business interests (although I understand both go hand in hand). I'm not fully back on board - I've only been to three games this season but I've certainly enjoyed my return. In the next month or so I'm attending Hull away, Forest away and Burton Albion away. I'm really looking forward to standing at Burton Albion with a smallish away following - I imagine it'll be like falling in love with football all over again.
I kind of wish I could give it up, but I've accepted I can't. I've also accepted it'll likely go tits up again very soon with Ashley in charge - c'est la vie.