simple explanation: people get used to their monitors' frame rate.

if all you've been using is a 60Hz display, you won't notice a difference down to 30-40 fps as much as you would when you've been using a 144Hz display.

our brains notice differences much more easily than absolutes, so a larger difference in refresh rate produces a more negative experience.

think about it like this:

The refresh rate influences your cursor movements.

so if a game runs slower than you're used to, you'll miss more of your clicks, and you'll need to compensate by slowing down your movements until you get used to the new refresh rate.

this effect becomes very obvious at very low fps (>20fps). it's when people start doing super slow movements.

same thing happens when you go from 144Hz down to, say, 40Hz.

that's an immediately noticeable difference!