But it's true. These AI models are not some big database where every piece of information is stored and can just be removed whenever you desire.

Imagine you almost got hit by a car while crossing the road as a child. That memory influenced your decisions from there on out, you learnt to always look before crossing, and over time your brain literally got wired differently because of that incident. Suddenly 20 years later the law requires you to remove that memory from your brain because apparently it was private data. How do you do that? It's not a single data point that just hangs around in your brain. Even if you could remove that memory, it still has compound effects on who you are and what you do. There is no removing that memory in such a way that all its effects on your brain are completely gone. It's exactly the same for these AI models. The way this one private data point affected the model parameters cannot be reverted unless you retrain the entire thing.