That is, instead of resorting to flatpaks and things of the like, why don’t we expose a way for libraries to express version information dynamically and just have those versions available and the dependant application can check if what it needs is available and just use that?



We actually do. Libraries are traditionally named something like
libfoo1.2.3.so. Then there are symlinks fromlibfoo1.soandlibfoo1.2.so. This way you can depend onlibfoo1.2.soand it works fine even if the user has libfoo 1.2.6.Now there are lots of problems here.
So yes, shared libraries and careful management is more elegant and efficient. But like so many things worse is better so “It works on your machine? We’re shipping your machine”.