Solved, see below.
I recently reinstalled my home server and was unable to open my LUKS-encrypted hard drive. Neither my usual passphrase nor a newly created keyfile were working. I tested on different distros, initially on my new Proxmox installation, later on a the Arch ISO. I eventually tried the disk on my main system, on which it used to be and I still had an old keyfile on - et voilá. So I created keyfiles as suggested in the wiki and occasionally md5sum returned a different hash for the keyfile! Why is this happening? I find' this extremely concerning because this could potentially result in massive data loss due to a keyfile apparently randomly not working as I was experiencing it. What am I missing?
For reference because I don't know how to share what I exactly did.
Scenario #1:
A directory on a mounted hard drive on my desktop.
$ echo -n '$mypassphrase' > ./dir/keyfile
$ md5sum ./dir/keyfile
c6dd9329dbe030127ce5e19d85de4df9 ./dir/keyfile
# chown root:root ./dir/keyfile; chmod 400 ./dir/keyfile
# md5sum ./dir/keyfile
c6dd9329dbe030127ce5e19d85de4df9 ./dir/keyfile
Scenario #2:
My old keyfile in /etc on my desktop containing $mypassphrase.
# md5sum /etc/keyfile
a1c10c2d023c982259f6c945ebee664e /etc/keyfile
Scenario #3:
Booted from the Arch ISO on my server.
# echo -n '$mypassphrase' > keyfile
# chown root:root keyfile; chmod 400 keyfile
# md5sum keyfile
c6dd9329dbe030127ce5e19d85de4df9 keyfile
Scenario #4:
A directory in /home on my desktop.
$ echo -n '$mypassphrase' > ./keyfile
$ md5sum keyfile
a1c10c2d023c982259f6c945ebee664e keyfile
EDIT: I just moved the disk back into my server and tried echo'ing my passphrase into a keyfile which returned the hash starting with c6, whereas opening a file using nano and pasting the passphrase into the file returned the hash starting with a1.
EDIT: I moved the disk back into my server, reinstalled Proxmox and tried again. I was able to unlock the disk after I pasted the passphrase into a file and deleted all trailing spaces/newline. I also tried echo'ing the passphrase into a keyfile and that also did not work, no clue why but it seems to work on some systems on not on others.
Thanks for your answer. I'm using UTF-8 on my desktop running Arch Linux and my server running Proxmox.
$ echo -n 'myrandompassphrase123' > keyfile$ md5sum keyfile49b3cdce58e65935b8255438266dd87e keyfileWhereas editing the keyfile using nano and pasting 'myrandompassphrase123' returns 59be855b925013d35476e82763b7a80e.
This is the scenario that's the least concerning, because I get that echoing into a file and editing a file using nano could yield different results. Unfortunately (or luckily) I can't reproduce the 4th scenario, which concerned me.
I suspect you can't replicate scenario 4 because it was never a thing (Pretty natural to skip over something by accident). A trailing newline was my first thought, but scenario 4 ruled it out. If you can't replicate it then it's the likely answer.
Something like this would confirm
echo "$mypass" | md5sumVs
echo -e "$mypass\n" | md5sumI'll have a play tomorrow and see what I get.
I suspect opening up the two files in a hex editor will show the difference, assuming I can replicate your results.