I think it has the tendency to create a snowball effect. You see a comment with -50 points you are already subconsciously looking at it trying to analyze why everyone hates it. It essentially primes you into disagreeing with it. Sometimes it's obvious in the case of a troll or someone saying hate speech or something but other times it's someone sharing a genuine opinion that's relevant to the discussion but the snowball effect of the first few people downvoting it causes it to spiral downwards.

By itself it isn't a bad thing but when comments are ranked based on votes or downvoted comments past a certain threshold are hidden, it contributes to creating echo chambers.

Personally, I think it's like that Churchill quote. Democracy is trash and has a lot of problems. But still, it's the best thing we've come up with so far. It's got its issues but the transparent nature definitely helps if someone is consciously trying to read things with an open mind.