Your philosophical dilemma is valid, but I would point out that pointing fingers is the best choice we have in this matter. Say, indeed, the ones fucking shit up do have intellectual disabilities, they still continue doing it. Reprimanding them for it is perhaps society's way of trying to get them to stop. Sort of like in the case of a child I guess - children being considered in this argument as having not enough understanding of the consequences of their actions. Equate, if you will, 'goddamn it Billy, stop running in the street' to 'goddamnit grandma, stop voting for idiots'.

The problem then is that Billy may listen, but grandma is set in her ways and has the notion that age brings wisdom regardless, so she's less inclined to listen to what the equivalent of Billy for her has to say.

I see no alternatives for this (that is to say, an attempt to correct erroneous behaviour) in the context of the aspirations of modern society.

As a thought experiment, in the most extreme case, what would we do? Test everyone for lead and remove, for example, the right to vote across a certain threshold? That doesn't take into account the baseline intellectual ability of individual (which can vary across a population) and the degree to which said discovered lead levels would affect them. It's entirely possible that a lead-laden 'smart' brain still has more capability than a pure but idiotic one. Not sure how we'd ever assess that. Not to mention the system would be exploited as soon as possible to channel power.

We could of course, stop pointing fingers and forgive them, for they know not what they do, and it'd probably have about the same effect.