i mean i'm pretty sure the nazis chose those colours specifically to benefit from being associated with socialism, the party's full name is literally the "national socialists" after all, the whole thing about nazism is taking something popular and using it for evil.

I more associate them with brown anyways, that's way more unique to fascism. EVERYONE uses red white and black, they're literally the first colours every language creates words for.