That's a couple more hundred thousand users that have moved on right there

That is still enough friction to prevent people from signing up. I didn't sign up for a long time because trying to choose a server felt overwhelming. It wasn't until they allowed more sign ups at mastodon.social that I gave it another try.

Therein lies the problem. A lot of people already get decision fatigue trying to choose a server and don't get past the sign up. Assuming that they are still willing to try and use a third party app, you get to layer finding a good app that works reliably on top of that and learning how to use the app which will be significantly different from the website. There comes a point where it's not tech illiteracy but a lack of time and interest spending hours how to use a service that is much harder to get into than its competitors.