Any sort of media, including videos, I always go for the highest possible quality I can. I do have a number of 4k displays, so it makes sense to a certain extent, but a lot if it has to do with future-proofing.

Here's a good example: When personal video cameras were first starting to support 1080, I purchased a 1080i video camera. At the time, it looked great on my 1920x1080 (maybe 1024x768, not sure) monitor. Fast forward over 15 years later, and it the video I recorded back then looks like absolute garbage on my 4k TV.

I remember watching a TV show when 1080p video first became available, and I was blown away at the quality of what was probably a less-than-1GB file. Now watching the same file even on my phone has a noticeable drop is quality. I'm not surprised you saw little difference between a 670MB and a 570MB file, especially if it was animation, which contains large chunks of solid colors and is thus more easily compressed. The difference between two resolutions, though, can be staggering. At this point, I don't think you can easily find a 1080p TV; everything is 4k. 8k is still not widespread, but it will be one day. If you ever in your life think you'll buy a new TV, computer monitor, or mobile device, eventually you'll want higher quality video.

My recommendation would be to fill your media library with the highest-quality video you can possibly find. If you're going to re-encode the media to a lower resolution or bitrate, keep a backup of the original. You may find, though, that if you're re-encoding enough video, it makes more sense to save the time and storage space, and spend a bit of money on a dedicated video card for on-the-fly transcoding.

My solution was to install an RTX A1000 in my server and set it up with my Jellyfin instance. If I'm watching HDR content on a non-HDR screen, it will transcode and tone-map the video. If I'm taking a break at work and I want to watch a video from home, it will transcode it to a lower bitrate that I can stream over (slow) my home internet. Years from now, when I'm trying to stream 8k video over a 10Gb fiber link, I'll still be able to use most of the media I saved back in 2024 rather than try to find a copy that meets modern standards, if a copy even exists.

Edit: I wanted to point out that I realize not everyone has the time or financial resources to set up huge NAS with enterprise-grade drives. An old motherboard and a stack of cheap consumer-grade drives can still give you a fair amount of backup storage and will be fairly robust as long as the drive array is set up with a sufficient level of redundancy.