There is not a single society without police that doesn't oppress it's children and women.

That's simply not true. Counterexamples:

• The CNT/FAI in 1930s Cathalonia
• Anarchist Ukraine after the1918 revolution
• The Zapatistas
• Many pre-colonial native American tribes, e.g. The Wendat
• Pretty much any immediate-return hunter gatherer people, e.g. the Hadza or the Pygmy

The only thing that is stopping them from enforcing their believes is that the country I live in decided that it is wrong to treat people that way and to enforce this believe they have laws and police.

So, if a country can "decide" this, why can't a community "decide" it? We both know that there are bucket loads of precedents of countries oppressing minorities both in the past and now. So obviously, a state doesn't guarantee that people aren't oppressed. I'd even claim that most countries still oppress their children (to raise them as "productive" workers).

I don't want to have to fit in to be free

You have to do so today. If you don't notice that, that's because you fit in.

and I don't want to be scared of my neighbours all the time.

Ever heard of racism in the police? And you claim that I don't acknowledge my privileges.

Anarchists just seem like a bunch of spoiled privileged people to me who's only concern is that someone doesn't allow them to consume drugs or whatever. I just wish they'd try living in a place without these structures in place for a while they privilege off but don't acknowledge.

Anarchists are acutely aware of the injustices that don't harm them directly. One rallying cry is "no one is free until we all are free" after all.