I ran HA on a pi 3 and then upgraded to a NUC with a Celeron using 4gb of RAM mostly because I used Plex at that time and needed better transcoding.
You can try running HAOS supervised on your Pi. Running supervised will let you run addons within HA such as pihole, Zigbee2mqtt, Mosquitto, Node-RED and others. After that, you can verify if you actually need to upgrade your hardware. You'll probably be fine with that amount of RAM.
I didn't, but because the pi already worked fine and didn't have many addons running. I have since introduced tailscale, diyhue, frigate without a tpu, etc and the nuc has been rock solid (even with 4gb of RAM and a Celeron J3455).
I think it comes down to what addons you run and what type of storage you use (if you're using an SD card on your pi, you might want to boot from an SSD instead). Also if your zigbee dongle is old like the Nortek one, you might see improvements in responsiveness when upgrading to a zigbee 3.0 stick.