Skip to content

http: add parser kOnStreamAlloc callback for faster uploads

The current parser OnStreamAlloc model uses a static 64KB memory buffer for the incoming upload data, which is then copied to a new JS buffer for every onBody event. This copies the data twice, and also creates a lot of small buffers that overwhelm GC.

This PR suggests the user to provide a callback function that allocates the buffers for receiving the data, which are then emitted to the http request, and once the user is done processing the data, it can recycle the buffer object by using a simple buffer pool.

This was observed while testing upload performance for https://github.com/noobaa/noobaa-core as every node process consumed a lot of GC and memcpy and could not exceed an upload throughput of 3 GB/sec. However with this suggestion, a significant boost of upload performance was observed, which could get almost up to 2x of the current performance.

Here are the results from the provided benchmark/http/server_upload.js running on Mac M1:

$ ./node benchmark/http/server_upload.js
http/server_upload.js useBufferPool=0 delay=-1 backpressure=0 upload=1048576 bufSize=65536 connections=200 duration=10 stats=0 benchmarker="wrk": 3,495.08
http/server_upload.js useBufferPool=1 delay=-1 backpressure=0 upload=1048576 bufSize=65536 connections=200 duration=10 stats=0 benchmarker="wrk": 5,855.73
http/server_upload.js useBufferPool=0 delay=0 backpressure=0 upload=1048576 bufSize=65536 connections=200 duration=10 stats=0 benchmarker="wrk": 3,031.98
http/server_upload.js useBufferPool=1 delay=0 backpressure=0 upload=1048576 bufSize=65536 connections=200 duration=10 stats=0 benchmarker="wrk": 4,831.23
http/server_upload.js useBufferPool=0 delay=1 backpressure=0 upload=1048576 bufSize=65536 connections=200 duration=10 stats=0 benchmarker="wrk": 3,107.22
http/server_upload.js useBufferPool=1 delay=1 backpressure=0 upload=1048576 bufSize=65536 connections=200 duration=10 stats=0 benchmarker="wrk": 4,926.17
http/server_upload.js useBufferPool=0 delay=5 backpressure=0 upload=1048576 bufSize=65536 connections=200 duration=10 stats=0 benchmarker="wrk": 3,065.13
http/server_upload.js useBufferPool=1 delay=5 backpressure=0 upload=1048576 bufSize=65536 connections=200 duration=10 stats=0 benchmarker="wrk": 5,012.74
http/server_upload.js useBufferPool=0 delay=-1 backpressure=1 upload=1048576 bufSize=65536 connections=200 duration=10 stats=0 benchmarker="wrk": 3,343.51
http/server_upload.js useBufferPool=1 delay=-1 backpressure=1 upload=1048576 bufSize=65536 connections=200 duration=10 stats=0 benchmarker="wrk": 5,095.58
http/server_upload.js useBufferPool=0 delay=0 backpressure=1 upload=1048576 bufSize=65536 connections=200 duration=10 stats=0 benchmarker="wrk": 2,981.24
http/server_upload.js useBufferPool=1 delay=0 backpressure=1 upload=1048576 bufSize=65536 connections=200 duration=10 stats=0 benchmarker="wrk": 4,491.13
http/server_upload.js useBufferPool=0 delay=1 backpressure=1 upload=1048576 bufSize=65536 connections=200 duration=10 stats=0 benchmarker="wrk": 2,972.44
http/server_upload.js useBufferPool=1 delay=1 backpressure=1 upload=1048576 bufSize=65536 connections=200 duration=10 stats=0 benchmarker="wrk": 4,412.07
http/server_upload.js useBufferPool=0 delay=5 backpressure=1 upload=1048576 bufSize=65536 connections=200 duration=10 stats=0 benchmarker="wrk": 1,802.44
http/server_upload.js useBufferPool=1 delay=5 backpressure=1 upload=1048576 bufSize=65536 connections=200 duration=10 stats=0 benchmarker="wrk": 2,105.78

I considered also adding an http server option to expose this capability, but didn't know if that would be acceptable as an API change.

Would be great to get your review! Thanks in advance.

Merge request reports

Loading