“free speech absolutism” consistently leads to spaces becoming hostile to marginalized groups

It's not the free speech that causes it, it's that "free speech" is being used as a weapon to tolerate intolerance. You can tolerate Nazi insignias in a bar w/o tolerating Nazis, you throw people out who are intolerant, and let those remain who are respectful. In fact, I would love to go to a WW2-themed bar with a mix of historical symbols and whatnot from all sides of the war (Nazis, Japanese Imperialists, Allies, etc) where nobody tolerates actual Nazis.

I want a space where I can discuss things that are uncomfortable without fear of getting banned. That's what I'm after when I push for free speech.