Because prejudice is no longer acceptable, instead prejudice is hidden behind support for non-prejudiced ideas with a disproportionate impact on a particular targeted group.
Immigration is probably the most obvious arena for this; individuals prejudiced against Mexicans insist we should tighten immigration laws.
But here's the thing: Tightening immigration laws, while repugnant to me, is not a racist cause. There are legitimate, entirely non-racial arguments for tightening immigration, and indeed the majority of people who want to tighten immigration are not racists.
But because they share a cause with racists, they're branded racists themselves. This clearly is not a legitimate branding; the fact that a racist supports a policy does not make the policy racist, nor does it make those who support the policy racist. Well-meaning individuals, looking at the racist basis, may treat the rational basis as a rationalization by the racists, invented by the racists, with no legitimate concerns. This does not make it so.
This is a serious problem, not least because the targets of the misbranding rapidly cease to treat "racism" as a valid complaint. It undermines broader efforts in eliminating prejudice.
This is why the race card is worn out; because individuals like Goldwater were and are branded with it; he had legitimate reasons for opposing the legislation he opposed; he didn't want the civil rights movement to go away, he wanted certain objectionable things removed from specific legislation, primarily the expansions of federal power at the expense of the states.