Two HTTP requests to fetch index.jar

I have a private F-Droid repo at https://grobox.de/fdroid/ and noticed that each client is always doing (at least) two HTTP GET requests to fetch index.jar. I paste a log extract below to illustrate the behavior. Maybe there's a good reason for doing two requests, maybe not. I just want to report it, so you guys know about it.

46.166.xxx.xx - - [06/Oct/2016:21:26:16 +0200] "GET /fdroid/repo/index.jar?client_version=0.101 HTTP/1.1" 200 11897 "-" "Dalvik/2.1.0 (Linux; U; Android 6.0.1; Moto G? Build/MOB30Z)"
46.166.xxx.xx - - [06/Oct/2016:21:26:17 +0200] "GET /fdroid/repo/index.jar?client_version=0.101 HTTP/1.1" 304 332 "-" "Dalvik/2.1.0 (Linux; U; Android 6.0.1; Moto G? Build/MOB30Z)"
2.247.xxx.xxx - - [06/Oct/2016:21:27:55 +0200] "GET /fdroid/repo/index.jar?client_version=0.100.1 HTTP/1.1" 200 11897 "-" "Dalvik/2.1.0 (Linux; U; Android 6.0.1; Moto G 2014 Build/MOB30J)"
2.247.xxx.xxx - - [06/Oct/2016:21:27:57 +0200] "GET /fdroid/repo/index.jar?client_version=0.100.1 HTTP/1.1" 200 8256 "-" "Dalvik/2.1.0 (Linux; U; Android 6.0.1; Moto G 2014 Build/MOB30J)"
46.166.xxx.xx - - [06/Oct/2016:21:29:47 +0200] "GET /fdroid/repo/index.jar?client_version=0.101 HTTP/1.1" 200 11897 "-" "Dalvik/2.1.0 (Linux; U; Android 6.0.1; Moto G? Build/MOB30Z)"
46.166.xxx.xx - - [06/Oct/2016:21:29:48 +0200] "GET /fdroid/repo/index.jar?client_version=0.101 HTTP/1.1" 304 332 "-" "Dalvik/2.1.0 (Linux; U; Android 6.0.1; Moto G? Build/MOB30Z)"
46.166.xxx.xx - - [06/Oct/2016:21:31:38 +0200] "GET /fdroid/repo/index.jar?client_version=0.101 HTTP/1.1" 200 11897 "-" "Dalvik/2.1.0 (Linux; U; Android 6.0.1; Moto G? Build/MOB30Z)"
46.166.xxx.xx - - [06/Oct/2016:21:31:38 +0200] "GET /fdroid/repo/index.jar?client_version=0.101 HTTP/1.1" 304 332 "-" "Dalvik/2.1.0 (Linux; U; Android 6.0.1; Moto G? Build/MOB30Z)"
212.109.xx.xxx - - [06/Oct/2016:21:31:40 +0200] "GET /fdroid/repo/index.jar?client_version=0.101 HTTP/1.1" 200 11922 "-" "Dalvik/1.6.0 (Linux; U; Android 4.4.2; F5_15 Build/KVT49L)"
212.109.xx.xxx - - [06/Oct/2016:21:31:40 +0200] "GET /fdroid/repo/index.jar?client_version=0.101 HTTP/1.1" 304 197 "-" "Dalvik/1.6.0 (Linux; U; Android 4.4.2; F5_15 Build/KVT49L)"
78.214.xxx.xxx - - [06/Oct/2016:21:31:44 +0200] "GET /fdroid/repo/index.jar?client_version=0.101 HTTP/1.1" 200 11897 "-" "Dalvik/2.1.0 (Linux; U; Android 6.0.1; SM-G900F Build/MOB31E)"
78.214.xxx.xxx - - [06/Oct/2016:21:31:45 +0200] "GET /fdroid/repo/index.jar?client_version=0.101 HTTP/1.1" 304 332 "-" "Dalvik/2.1.0 (Linux; U; Android 6.0.1; SM-G900F Build/MOB31E)"
46.166.xxx.xx - - [06/Oct/2016:21:32:35 +0200] "GET /fdroid/repo/index.jar?client_version=0.101 HTTP/1.1" 200 11897 "-" "Dalvik/2.1.0 (Linux; U; Android 6.0.1; Moto G? Build/MOB30Z)"
46.166.xxx.xx - - [06/Oct/2016:21:32:36 +0200] "GET /fdroid/repo/index.jar?client_version=0.101 HTTP/1.1" 304 332 "-" "Dalvik/2.1.0 (Linux; U; Android 6.0.1; Moto G? Build/MOB30Z)"
176.10.xxx.xx - - [06/Oct/2016:21:37:56 +0200] "GET /fdroid/repo/index.jar?client_version=0.101 HTTP/1.1" 200 11897 "-" "Dalvik/2.1.0 (Linux; U; Android 6.0.1 Build/MTC20K)"
176.10.xxx.xx - - [06/Oct/2016:21:37:57 +0200] "GET /fdroid/repo/index.jar?client_version=0.101 HTTP/1.1" 304 332 "-" "Dalvik/2.1.0 (Linux; U; Android 6.0.1 Build/MTC20K)"
176.10.xxx.xx - - [06/Oct/2016:21:39:22 +0200] "GET /fdroid/repo/index.jar?client_version=0.101 HTTP/1.1" 200 11897 "-" "Dalvik/2.1.0 (Linux; U; Android 6.0.1 Build/MTC20K)"
176.10.xxx.xx - - [06/Oct/2016:21:39:24 +0200] "GET /fdroid/repo/index.jar?client_version=0.101 HTTP/1.1" 304 332 "-" "Dalvik/2.1.0 (Linux; U; Android 6.0.1 Build/MTC20K)"
176.10.xxx.xx - - [06/Oct/2016:21:40:14 +0200] "GET /fdroid/repo/index.jar?client_version=0.101 HTTP/1.1" 200 11897 "-" "Dalvik/2.1.0 (Linux; U; Android 6.0.1 Build/MTC20K)"
176.10.xxx.xx - - [06/Oct/2016:21:40:15 +0200] "GET /fdroid/repo/index.jar?client_version=0.101 HTTP/1.1" 304 332 "-" "Dalvik/2.1.0 (Linux; U; Android 6.0.1 Build/MTC20K)"
213.47.xx.xx - - [06/Oct/2016:21:43:45 +0200] "GET /fdroid/repo/index.jar?client_version=0.101 HTTP/1.1" 200 11922 "-" "Dalvik/1.6.0 (Linux; U; Android 4.4.2; GT-N7100 Build/KOT49H)"
213.47.xx.xx - - [06/Oct/2016:21:43:45 +0200] "GET /fdroid/repo/index.jar?client_version=0.101 HTTP/1.1" 304 197 "-" "Dalvik/1.6.0 (Linux; U; Android 4.4.2; GT-N7100 Build/KOT49H)"

Added bug help-wanted labels

My first guess is that this related to it checking whether it has the current index, HTTP 304 is "Not Modified" https://httpstatuses.com/304

Milestone changed to %0.102

Looks like this is to get the size of the download, for resumable downloads (but curiously not for the progress indicator purposes). Two thoughts come to mind:

• This should instead be done in the place where we get the downloaded size for progress purposes, so that a second request is not necessary.
• If this really needs to be done, it should use a HEAD request instead of a GET, to be more explicit about its intentions.

We'll need to do the HEAD request first anyhow for the #562 (closed) stuff.

There is something else going on here, it seems that something is making it two requests. Now that I switched the first check to HEAD, I sometimes see a GET before a HEAD that matches the cache etag. But I don't really have any ideas of why. Something to look out for, I guess.

Reassigned to @eighthave

Status changed to closed by commit ab1e869ebe49c13b182873868607db36dc971b8f

Status changed to closed by merge request !415 (merged)