The most natural authentication mechanism for humans is a key. That thing you carry with yourself. A physical key containing, well, the actual secret (shouldn't be retrievable, should be used for decrypting access request and signing the response) that, maybe combined with your password (another natural for humans authentication mechanism) or maybe, yes, TOTP, gives you access.

Like those "security keys" Imperial officers in Jedi Outcast carry with them. Maybe a bad example.

Phone numbers are used as identifiers because governments like it, nerds don't like it, and normies explicitly like what nerds don't like and also want everything to be insecure, they call it "having nothing to hide".

Also "normal and social" people have that idea that their social prowess is more elegant, smarter at ensuring their security that those dumb and boring nerd technical solutions. So them always choosing things logically opposite of sane, like social media instead of forums, and phone numbers instead of any other identifier, is literally a matter of principle. It's really not that hard to use something else. They do the stupidest possible thing technically to prove a point that you only have to do the smart thing socially. I mean, in Galileo Galilei's case the other side of the disagreement is generally considered right, but that's not an argument effective in society.

I should admit that I've been doing the opposite - the stupidest possible thing socially to prove a point that only technical sense matters, which is why nobody would send me encrypted mail except Facebook with its notifications, and nobody would write me in Tox, and nobody would even contact me via XMMP. Which is why I'm now using TG, VK, FB, WA and Signal for communication, of these Signal is secure, and WA is kinda better than the rest of them.