FN is rarely used I have the function and media keys on that layer. It's deliberately there bc of the little use I make of it.

All my modifiers are at my thumbs, holding left space bar jumps to nav layer for hjkl as arrows and some other nav keys, holding right space jumps to my symbol/number layer.

Here's some similar layer setup on an ortho linear: https://ibb.co/SPqBvHM

It's not about real estate, if I would be devils advocate here, why have a shift key or why have ctrl key if you could have capital letters as separate keys? Or all of your symbols in extra keys to your numbers? It's all about low travel with your fingers, I rarely mistype bc I dont have to aim for a key that's far away, and I can customize the layout however I need it.

I'd encourage you to try some leaner layouts for a while, maybe you find out it's not stupid after all.