[v9.x-backport] Backport #17198, #17738, #17841, #17736, #17881, #17879 and #18139
As per title, backports a whole bunch of inter-dependent PRs.
Note that #17198 used to be marked as semver-major
but several TSC members have approved it being downgraded as it's non-breaking and is a dependency for several subsequent changes.
There is at least one semver-minor
PR in here so this can't land until 9.7.0 at the earliest.
Checklist
-
make -j4 test
(UNIX), orvcbuild test
(Windows) passes -
tests and/or benchmarks are included -
documentation is changed or added -
commit message follows commit guidelines