They say that identifying the problem is half the battle. You're doing alright with that, self-aware, self-honest, reflective and articulate. I'm not seeing someone emotionally shallow in all that you've expressed here, more like a lack of experience.

Throwing out some ideas for you to mull over and experiment with:
Ask questions and listen to and reflect on different answers, as you've done here.
Talk therapy, if you have access to something affordable.
Journaling, which is like talk therapy with yourself and a notebook. Focus on exploring your feelings and areas you'd like to grow in. It won't be perfect. That's the point.
Observe others, find roll models (like the bold ones you mention above), but hold your observations lightly. It's easy to get it wrong. This is where fiction is very useful; any misassumptions are between you and the book and the author is a good guide.
Reading. Reading. Reading. Emotion-centered fiction. Action-centered fiction where the emotions are present but not well articulated. Non-fiction about emotions, including memoirs.
Visual stories too. Look for emotional arcs, how the characters affect each others' emotions, how your own are affected.
(Wee aside, Chinese and Korean dramas often go deeper into emotions than US, UK and European ones. There are a lot available for free on youtube and elsewhere.)
Seek out music that stirs your emotions.
Journal about your reactions to any of the above that you engage in.
Imagine. Make up your own stories. It can be fan fiction in your head but keep it coming back to how everyone feels, how they express that, how it influences others.
Imagine. What you would like to be, what your near-future self is like in a healthy relationship and friendships.
Bring your natural curiosity. This is a new field to explore and learn about.

Hopefully a few of those will resonate and work for you, or suggest things which will. You can do this :)