Never understood how they gained traction in the US

They gained traction specifically in 3 types of places:

1. Condo buildings with shared common elements where everyone in the building should share the financial burden of maintaining the roof, elevators, common areas.
2. Planned communities where farmland or other underdeveloped land was converted into a lot of houses, in a city or state unwilling to build or maintain the roads, power lines, sewers, and other infrastructure that makes it livable.
3. Communities with exclusive amenities, like private beach/lake access, private parks/playgrounds, golf courses, gate guards who keep out the uninvited non-residents, etc. There's a strain of historical practice here of basically keeping our non-white people from gated communities.

None of these 3 types of places need an HOA to accomplish this.

For that condo category, New York pioneered the use of co-ops that effectively accomplish the same thing. It's just that the co-op legal structure is a little bit more unwieldy and inefficient than a modern condominium owners association.

For the "the city won't pay for our infrastructure" category, it is always possible to persuade the city to actually take over those responsibilities, but it would probably slow down development, and put too much in favor of the incumbent residents over potential future residents. NIMBYism is bad enough, we don't need to take away a legal tool for overcoming it.

For the "let's keep out the poors" type of community, those are exactly the types of communities that actually love their HOAs. The HOAs are, in a sense, harmful to the people not within the community but upheld by the people who are in that community. Abolishing that is probably fine, although it would do nothing about the types of complaints that most people have about their HOAs.