For consumer software, yes, most is still being built with a baseline target instruction set from the early/mid-2000s. In 2019 there were reports of Apex Legends requiring SSE4.1, an instruction set from circa 2007. It will be be probably close to a couple decades before consumer software would start commonly requiring these instructions.

However, for more specialized environments, such as scientific and high-performance computing applications, its much more common that you will be using custom software designed for a specific task, and that it's normal to recompile the software when you get a new set of hardware. In those applications, these instructions can make a huge impact, as you know exactly which capabilities are supported by the hardware and can use everything available.

I believe there are also some (possibly limited) situations where a program can check what instructions a processor supports and use either the newer (higher-performance) version or the slower, more widely-supported version depending on that check. There may be limits on how often that can be done however.

This is why .NET code compiles to platform-independent binaries that get JIT translated to machine code and optimized for the target CPU. Developers don't need to do anything (the applications don't even need to be re-compiled), they will just get conditionally optimized when appropriate.

This is the only way really to move forward with ISA extensions.

Though, I think for this update we don't need to be too concerned. Since it changes the code in such an extensive way, compiler writers will be strongly incentivised to produce this duplicate path themselves. Instead of letting the burden of dispatching fall on the programmer like with AVX and friends

For an extension like this - unlike most prior extensions - you're best off with essentially an entirely separately compiled copy of the program/library. So IFUNC is a poor fit, even with peer optimization.