Storing it as a sparse graph should reduce the storage requirements drastically, since most edges wouldn’t exist.

you should consider moving to Facebook or Threads, maybe?

Not an option

As for the rest yeah those do seem like genuine obstacles. Partially think the reason I liked the algorithm is because it reminded me of the Web of Trust things like Scuttlebutt use to get relevant information to users but with a lower barrier to entry.

Also as I've said elsewhere it doesn't have to be this exact thing but since this is a new platform we have the chance to make algorithms that work for us and are transparent so I wanted to share examples that I thought were worthwhile.

Edit:

You’d also turn Lemmy into the strongest echo chamber you could possibly create.

PS. I don't think that's true. Big tech companies that have more advanced algorithms would probably be much better at creating echo chambers.

Instead of comparing every single individual users votes with every other one, you create clusters using data science techniques and bucket all users into those clusters, which are calculated on a nightly or weekly basis. By controlling the cluster size you can keep the number of comparisons managable, and still achieve OP's vision.