The segmented caching request thing is… weird. I worked for a company that developed a caching proxy and it very much did not work that way. Like, random access in a caching system is usually kinda bad and you should try to avoid it. Like, our proxy manually controlled the disk (it wasn’t a mounted filesystem) so it could constantly sweep the head across the disk and cue up reads and writes optimally. This gets much harder when things are fragmented as fuck.
If the concern was about what would happen with multiple connections for the same cache miss, then the caching proxy should just combine the client-side connections into a single upstream one. You can still cache the first part of the response if your upstream connection gets terminated and then restart it from that point.
Nah, it’s really not necessary. I’m senior dev at a large software company you’ve absolutely heard of and I’m just as productive as my colleagues who use LLMs. My tasks usually take fewer PRs as well, since there are fewer bugs that need to be fixed.
I still don’t understand why people are foaming at the mouth about LLMs. They’re fucking awful at writing software.