A word processor like MS Word or LibreOffice Writer will probably refuse to open it, giving some error such as "unsupported file type."

Depending on how much of a nerd you are, the plaintext editor your OS comes with may either also refuse to open it, or open it as if it were plaintext and you might see a few jumbles of letters and punctuation, or weird symbols if it interprets it as unicode. According to Vim, my mp3 copy of Glycerine by Bush is mostly @ symbols. I noticed that my Bash shell didn't want to autocomplete "Vim glycerine.mp3" but when typed manually it did it with minimal fuss.

If you open it in a hex editor, you might be surprised to see the first few lines are readable, they likely contain metadata that media player software like VLC can understand, like the track name, artist, year of release and such. Scroll down further and you'll start to see more gibberish where it's trying to interpret the individual bytes that make up the audio as ASCII characters. Funnily enough hexedit gave me a different looking bunch of gibberish than Vim did.