There isn't a lot of evidence that harsher punishments lead to less crime.

But harsher punishments intuitively seem like they reduce crime, to those already unlikely to commit crime. Because obviously getting into trouble with the justice system is the worst thing that can happen, therefore getting in more trouble is worse. Thus harsher punishments must make crime less likely.

What the people who think like this seem to miss, is that their experience of life is not the same as those that commit crime (as a general rule); these people are mapping their own experience onto all people rather than trying to understand the experience of the "other".