I think 3 problems contribute to the fact that CVs keep getting consistently blamed for a lot of stuff
1) CVs tend to be more of a 'command type' ship than the other types of ships. So if the team loses, who do they blame? Naturally the guy in charge/at the top
2) The striking power of a CV can never be underestimated. Being able to drop torps or bombs on a target out of nowhere without the risk of getting shot back makes a CV a valuable (albit a slow) striking power.
3) Unless they played a CV themselves, other players will never understand how a CV player has to play (Work around AA bubbles, benefits vs risk, save 1 ship or save a group, what they can see)
With that much power and standing out it's no wonder everything a CV player does is scrutinised (and misunderstood) even more.