Weakness isn't factored in here consciously. A greasy guy with a knife would trigger a similar response. I think the linkage is that the same issues that cause someone to register as 'undesirable', as you put it, also reduce a person's social network, which compounds their issues with integrating into society.

I am sorry if this seems pedantic, it's not on purpose, but punching down implies a desire to increase social status by reducing the status of other members, implying they remain part of the tribe but subordinate. The impulse is existential rather than social - eliminating a threat as opposed to gaining social power.