Yeah, that's kind of the nature of communication I think, and ultimately votes are a form of communication :/

If people understand them to mean agreement or disagreement, that's the message they'll recieve regardless of whether it was the message that you were trying to send.

I guess if I wanna convey I appreciate someone's contribution even though I don't share their perspective I can always leave a comment; that seems like something I'd appreciate seeing in my inbox so maybe it's worthwhile for me to make that a habit