I think the main issue here is the fact that everyone does wear it, which makes not wearing one more of a political statement than freedom of choice. If it wasn't political it wouldn't have mattered for him to wear a shirt with it on even if he didn't care much for the reasoning. It's comparable to Ferdinand not wearing a Kick it Out shirt when everyone else did. It's freedom of choice, but it's still a statement as it's a larger effort to not wear it than to wear it.