Immigration is never or rather historically a rare problem. It's alway the scapegoat used by politicians, whose failed economic policies are what's the issue.
This is why neo-cons are a perfect counterpiece to neo-liberals. They want the same economic and fiscal policies, but the neo-cons will blame immigration every time to distract you from tax cuts given to the rich and large corporations.
It's two sides of the same shitty coin, except the other side likes to play racist... and I say "play racist", because it's just another manipulation tactic for some - and an easy one at that.
People are so poorly educated that they don't realise the biggest success of the west was the massive brain drain from authoritarian countries. That's not the issue, because those immigrants qualify to stay, because they are considered an asset.
But the issue becomes disingenuous when you only want "the good ones", because other immigrants fill sectors that western people don't want to work in. That's why you got US companies, luring in Mexicans to work for a month to two, and then have them deported without paying them.
Again, don't attribute to racism what you can attribute to greed and sociopathy.