That is, instead of resorting to flatpaks and things of the like, why don’t we expose a way for libraries to express version information dynamically and just have those versions available and the dependant application can check if what it needs is available and just use that?



If you take a look at it from the lens of distro maintainers, this would be an absolute nightmare. Distros package library versions, and currently tightly control the release of applications to given library versions. Allowing just any range of library versions for applications would loosen said requirement, but would require distros to then build and maintain a ton of versions of each library.
Flathub currently curbs this problem to a large extent by bundling explicit point release packages of common libraries, and expects the application to bundle uncommon ones themselves. This does mean that one cannot simply package an application without some thought put into the library versions used for release on Flathub still.
For distros, this style of packaging could somewhat be copied, but to little benefit. It would either be wasting time replicating congruence with an existing standard like Flathub’s library bundles or do their own thing and further complicate things for developers wanting to get their app published to a distro’s repos.
Overall, flatpaks make the process look a bit more seamless than things really are for developers and package maintainers. Flatpaks really only work as well as they do because everyone mostly agrees on adhering to the library packages published by Flathub. I don’t really think adding even the sliding scale of library ‘sets’ for various distros would any benefit beyond what is currently done.