While it is true that "should of" etc. can easily originate from a confusion between "'ve'" and unstressed "of", which sound identical, the statement

"Should of" is incorrect

itself is at least a bit misleading and prescriptivist in its generality.

Interestingly, there seem to be at least some native English speakers who genuinely do say "should of" (with a stressed "of") sometimes. This paper for example argues that people who say "should of" really do use a grammatical construction of the form modal verb + of + past participle. One argument the author mentions is that this would also explain the words "woulda", "coulda" and "shoulda", since "of"->"a" is quite common in general (e.g. "kind of" -> "kinda"), but "'ve"->"a" basically doesn't occur elsewhere (e.g. no one says "I'a" or "you'a" instead of "I've" or "you've"). Another is that the reverse mistake, i.e. using "'ve'" in place of "of" (e.g. "kind've"), is much rarer, which is a clear difference to e.g. the situation with "they're"/"their"/"there", where people use these words in place of the others in all combinations frequently. I recommend this blog article for a much longer discussion.

Also, whether genuine mistake (which it almost certainly is in many cases, although probably not all) or different grammatical construction, YSK that "should of" etc. didn't just become popular recently, but have been used for centuries. E.g. John Keats wrote in a letter in 1814: "Had I known of your illness I should not of written in such fiery phrase in my first Letter.". Many more examples (some older as well) can be found e.g. here or here.

TL;DR: While in many cases "should of" etc. can well be a mistake, originating from the fact that it sounds identical to "should've" when unstressed, there is some interesting linguistic evidence that at least in some dialects of English native speakers really do say "should of" etc. (i.e. in those cases it is not a mistake, merely non-standard/dialectal).