Not quite sure I understand what you want...

Each client, the devices you ssh from, has a key pair. You take the public key part of that pair and put it in the authorized_keys file on the server, the device you ssh to. The cannonical paths for your keys and authorized_keys is ~/.ssh/

You say you cannot reach your pi from your phone? Find the key file named something .pub on your phone, add it to the authorized_keys (as a single line in that file) on your pi. That is it.

If you also want to drop the username you need to add something like this to your .ssh/config on your phone:

Host alias_for_pi
  Hostname pi_ipnumber
  User pi_user

Does this help?

Debian has a package ssh-import-id which you can then run ssh-import-id-gh $githubUsername and it will ingest all the public keys you’ve put in GitHub. Should be able to easily add it to the cloud-init.yaml but I just always install and run this first.

You can also just copy the keys to the install when you are imaging the SD card.

Or use Ansible.

I know just how you feel. I have two servers, a phone, and two laptops that all may need to intercommunicate at any given time. For this I use Warpgate: https://github.com/warp-tech/warpgate

There's a tiny bit of a learning curve I feel but after setting up a couple connections you'll get the hang of it. Teleport is a similar solution but I don't like to use tools I don't understand and it's a bit more complex.