Let gitlab-git-http-server handle archive downloads (!1548) · Merge requests · GitLab.org / GitLab FOSS · GitLab

Snippets Groups Projects

Do not update/delete: Banner broadcast message test data

Do not update/delete: Notification broadcast message test data

An error occurred while fetching the assigned milestone of the selected merge_request.

Merged username-removed-5302 requested to merge git-archive-golang into master 9 years ago

This change relies on changes in gitlab_git and gitlab-git-http-server.

fixes #2429 (closed)

Activity

username-removed-5302 mentioned in merge request gitlab_git!51 (merged) 9 years ago

mentioned in merge request gitlab_git!51 (merged)
username-removed-5302 @jacobvosmaer · 9 years ago

Author Contributor

gitlab_git MR: gitlab_git!51 (merged) . gitlab-git-http-server: gitlab-git-http-server!2 (merged)

Demo video:

golang_git_archive_download_demo.m4v

About the video: First click focusses window. Second click starts download, note there is no wait while the archive is created on disk. The archive is streamed and simultaneously cached on disk. The third click goes to the disk cache.
username-removed-5302 Milestone changed to 8.1 9 years ago

Milestone changed to 8.1
username-removed-5302 @jacobvosmaer · 9 years ago

Author Contributor

This implements #2824 (closed)
username-removed-5302 Reassigned to @DouweM 9 years ago

Reassigned to @DouweM
Douwe Maan mentioned in issue #2429 (closed) 9 years ago

mentioned in issue #2429 (closed)
Douwe Maan @DouweM started a thread on the diff 9 years ago

lib/support/nginx/gitlab

    proxy_pass http://gitlab;
+ }
  location ~ [-\/\w\.]+\.git\/ {
  location ~ ^/[\w\.-]+/[\w\.-]+/(info/refs|git-upload-pack|git-receive-pack)$ {
    # 'Error' 418 is a hack to re-use the @gitlab-git-http-server block

Douwe Maan @DouweM started a thread on an old version of the diff 9 years ago

Valery Sizov @vsizov · 9 years ago

Contributor

@jacobvosmaer What can we expect if archive is not built yet (not cached to disk) but we receive a new request? Will the request wait? I mean, do we have some mutex there?

username-removed-5302 @jacobvosmaer · 9 years ago

Author Contributor

@vsizov both requests will create an archive concurrently, each in a different tempfile. The last tempfile to finish will become the cached archive.

Valery Sizov @vsizov · 9 years ago

Contributor

Hm, If time needed to build archive is one second but we have two requests per second?

Valery Sizov @vsizov · 9 years ago

Contributor

Is it place for (D)DOS attack?

Edited 9 years ago by Valery Sizov

username-removed-5302 @jacobvosmaer · 9 years ago

Author Contributor

@vsizov this implementation does not put a limit on the number of archives being created concurrently.

username-removed-5302 Added 1 commit: 9 years ago

Added 1 commit:

549a840e - Fix NGINX API download regex

username-removed-5302 Added 1 commit: 9 years ago

Added 1 commit:

033a879c - Fix NGINX API download regex

username-removed-5302 @jacobvosmaer · 9 years ago

Author Contributor

The build is failing because I forgot to fix/delete some archive tests.

username-removed-5302 Added 2 commits: 9 years ago

Added 2 commits:

fc94b3b0 - Fix API archive specs
8f143326 - Remove RepositoryArchiveWorker specs

username-removed-5302 Added 2 commits: 9 years ago

Added 2 commits:

381ca79b - Remove archive file sending spec
a74915a4 - Always return HTML in git_not_found

username-removed-5302 Added 1 commit: 9 years ago

Added 1 commit:

83f04853 - Update gitlab_git to 7.2.19

Douwe Maan Added 221 commits: 9 years ago

Added 221 commits:

83f04853...123669a5 - 220 commits from branch master
c993481d - Merge branch 'master' into git-archive-golang

Douwe Maan Status changed to merged 9 years ago

Status changed to merged

Douwe Maan mentioned in commit 9 years ago

mentioned in commit 4b28f2d9

username-removed-5302 mentioned in issue #3063 (closed) 9 years ago

mentioned in issue #3063 (closed)

username-removed-5302 mentioned in issue omnibus-gitlab#875 (closed) 9 years ago

mentioned in issue omnibus-gitlab#875 (closed)

Please register or sign in to reply