The fungibility isn't what's at issue. The link between the money stream and the military is a system that can't be understood separately. Thinking of it as taxpayer funded doesn't make sense.

arguably the entire worth of the dollar as a currency comes from it being what taxpayers pay in, so yeah it kind of does come from taxpayers

I encourage you to read the book. It really changed my perspective on what value even is and what it means. I don't think you'll think this after reading it.