One problem with QC is that besting classical computers has been a moving target, improving exponentially for many years while QC was being researched. It's going to be a long, slow climb up the slope of enlightenment as it reveals its potential.

Inflated Expectations. Most people who are aware of them will still talk about how they're going to destroy crypto. We are very, very far off from the size of QC that could possibly do that. It may not even be feasible to do the quantum juggling act necessary to handle that many qbits. It primarily effects public key crypto, with relatively minor effects on block ciphers and hashes. Plus, we already have post-quantum crypto making its way into TLS and other cryptographic suites.

And don't get me started on the morons who think the NSA already has some super secret breakthrough QC that can already break all crypto. Often from the same sorts of people who (correctly) throw Russell's Teapot at creationists.

Meanwhile, there are far more interesting possibilities that don't need so many qbits. Things like improving logistics or molecular simulation.

Quantum computers are now where neural nets were in the 1980s.

I know they won't be something everyone has in their house

That's what they said about non-quantum computers 80 years ago.