I would argue that you are completely right. But, it would surprise me if there isn't a fairly strong correlation between the two (corelation!=majority). Living in a society where expected social interactions far exceed the comfort level can sure cause some anxiety.

All that said, if you are anxious to interact with another person with very fixed frames for the interaction (like a person working in a shop at the checkout counter (I don't know the name)), then you have social anxiety and unlikely just being introverted.