Skip to content

WIP: Add option to buffer proxy requests

Jacob Vosmaer (GitLab) requested to merge buffer-requests into master

This change implements a simplified form of NGINX's proxy_request_buffering in gitlab-workhorse. It will allow us to turn off proxy_request_buffering in NGINX and use it selectively, buffering only requests that go to Unicorn.

Selective request buffering saves time and disk IO for requests that are handled by gitlab-workhorse. For example, a 1GB LFS upload will currently be buffered entirely by NGINX (on local disk) before being copied to LFS storage by gitlab-workhorse. With selective request buffering we would be able to write that file directly to LFS storage.

The implementation uses one fixed-size dynamically allocated memory buffer per request. If a request has an empty body we do not allocate a dynamic buffer. If the request buffer exceeds the dynamic buffer size we write it to a tempfile instead. The current implementation creates one tempfile per overflow request.

Merge request reports