Skip to content

buffer: up to 2x times faster copy of buffers

Rodrigo Muino Tomonari requested to merge github/fork/reklatsmasters/patch/buffer-concat into master

%TypedArray%.prototype.set is able to copy data between ArrayBuffer a lot faster. This allows us to speed up Buffer.copy and Buffer.concat by 2x in some cases.

benchmarks

node@master vs node@pr
                                         confidence improvement accuracy (*)   (**)  (***)
 buffers/buffer-copy.js n=1024 size=10          ***    117.07 %       ±2.82% ±3.76% ±4.90%
 buffers/buffer-copy.js n=1024 size=1024        ***     94.86 %       ±3.36% ±4.47% ±5.83%
 buffers/buffer-copy.js n=1024 size=2048        ***     53.33 %       ±4.03% ±5.38% ±7.06%
 buffers/buffer-copy.js n=1024 size=4096        ***     30.17 %       ±2.49% ±3.33% ±4.36%
 buffers/buffer-copy.js n=1024 size=8192        ***     21.52 %       ±2.23% ±2.97% ±3.86%
                                                                           confidence improvement accuracy (*)    (**)   (***)
 buffers/buffer-concat.js n=1024 withTotalLength=0 pieceSize=16 pieces=1          ***      4.52 %       ±1.73%  ±2.31%  ±3.03%
 buffers/buffer-concat.js n=1024 withTotalLength=0 pieceSize=16 pieces=16         ***    144.18 %       ±1.97%  ±2.63%  ±3.46%
 buffers/buffer-concat.js n=1024 withTotalLength=0 pieceSize=16 pieces=4          ***    104.35 %       ±6.67%  ±8.97% ±11.88%
 buffers/buffer-concat.js n=1024 withTotalLength=0 pieceSize=1 pieces=1           ***      5.51 %       ±2.37%  ±3.18%  ±4.18%
 buffers/buffer-concat.js n=1024 withTotalLength=0 pieceSize=1 pieces=16          ***    143.85 %       ±2.21%  ±2.95%  ±3.86%
 buffers/buffer-concat.js n=1024 withTotalLength=0 pieceSize=1 pieces=4           ***    101.56 %       ±6.18%  ±8.31% ±10.99%
 buffers/buffer-concat.js n=1024 withTotalLength=0 pieceSize=256 pieces=1         ***      5.50 %       ±2.97%  ±3.99%  ±5.27%
 buffers/buffer-concat.js n=1024 withTotalLength=0 pieceSize=256 pieces=16        ***     45.37 %       ±2.16%  ±2.89%  ±3.81%
 buffers/buffer-concat.js n=1024 withTotalLength=0 pieceSize=256 pieces=4         ***     66.76 %       ±1.69%  ±2.25%  ±2.94%
 buffers/buffer-concat.js n=1024 withTotalLength=1 pieceSize=16 pieces=1          ***      5.70 %       ±2.81%  ±3.78%  ±4.99%
 buffers/buffer-concat.js n=1024 withTotalLength=1 pieceSize=16 pieces=16         ***    160.34 %       ±5.80%  ±7.81% ±10.34%
 buffers/buffer-concat.js n=1024 withTotalLength=1 pieceSize=16 pieces=4          ***    116.18 %       ±5.38%  ±7.23%  ±9.55%
 buffers/buffer-concat.js n=1024 withTotalLength=1 pieceSize=1 pieces=1           ***      5.19 %       ±1.82%  ±2.44%  ±3.20%
 buffers/buffer-concat.js n=1024 withTotalLength=1 pieceSize=1 pieces=16          ***    160.41 %       ±2.37%  ±3.17%  ±4.15%
 buffers/buffer-concat.js n=1024 withTotalLength=1 pieceSize=1 pieces=4           ***    108.86 %       ±7.55% ±10.16% ±13.47%
 buffers/buffer-concat.js n=1024 withTotalLength=1 pieceSize=256 pieces=1         ***      5.46 %       ±3.00%  ±4.02%  ±5.30%
 buffers/buffer-concat.js n=1024 withTotalLength=1 pieceSize=256 pieces=16        ***     47.44 %       ±1.57%  ±2.10%  ±2.75%
 buffers/buffer-concat.js n=1024 withTotalLength=1 pieceSize=256 pieces=4         ***     70.00 %       ±1.66%  ±2.22%  ±2.89%
node@v10 vs node@v10-pr
                                         confidence improvement accuracy (*)   (**)  (***)
 buffers/buffer-copy.js n=1024 size=10          ***    187.01 %       ±5.37% ±7.21% ±9.54%
 buffers/buffer-copy.js n=1024 size=1024        ***    129.87 %       ±3.41% ±4.57% ±6.01%
 buffers/buffer-copy.js n=1024 size=2048        ***     78.64 %       ±2.57% ±3.42% ±4.45%
 buffers/buffer-copy.js n=1024 size=4096        ***     59.96 %       ±4.00% ±5.34% ±7.00%
 buffers/buffer-copy.js n=1024 size=8192        ***     75.37 %       ±1.23% ±1.64% ±2.14%
                                                                          confidence improvement accuracy (*)   (**)  (***)
 buffers/buffer-concat.js n=1024 withTotalLength=0 pieceSize=16 pieces=1          ***     22.93 %       ±1.38% ±1.83% ±2.39%
 buffers/buffer-concat.js n=1024 withTotalLength=0 pieceSize=16 pieces=16         ***    203.86 %       ±3.10% ±4.15% ±5.45%
 buffers/buffer-concat.js n=1024 withTotalLength=0 pieceSize=16 pieces=4          ***    154.01 %       ±2.50% ±3.34% ±4.39%
 buffers/buffer-concat.js n=1024 withTotalLength=0 pieceSize=1 pieces=1           ***     26.54 %       ±1.39% ±1.86% ±2.42%
 buffers/buffer-concat.js n=1024 withTotalLength=0 pieceSize=1 pieces=16          ***    226.57 %       ±2.42% ±3.24% ±4.26%
 buffers/buffer-concat.js n=1024 withTotalLength=0 pieceSize=1 pieces=4           ***    153.34 %       ±2.78% ±3.74% ±4.94%
 buffers/buffer-concat.js n=1024 withTotalLength=0 pieceSize=256 pieces=1         ***     19.00 %       ±1.83% ±2.44% ±3.18%
 buffers/buffer-concat.js n=1024 withTotalLength=0 pieceSize=256 pieces=16        ***     72.48 %       ±1.56% ±2.08% ±2.72%
 buffers/buffer-concat.js n=1024 withTotalLength=0 pieceSize=256 pieces=4         ***     89.29 %       ±2.61% ±3.50% ±4.62%
 buffers/buffer-concat.js n=1024 withTotalLength=1 pieceSize=16 pieces=1          ***     22.45 %       ±1.68% ±2.24% ±2.92%
 buffers/buffer-concat.js n=1024 withTotalLength=1 pieceSize=16 pieces=16         ***    214.20 %       ±3.17% ±4.25% ±5.58%
 buffers/buffer-concat.js n=1024 withTotalLength=1 pieceSize=16 pieces=4          ***    155.20 %       ±3.64% ±4.88% ±6.43%
 buffers/buffer-concat.js n=1024 withTotalLength=1 pieceSize=1 pieces=1           ***     23.09 %       ±1.88% ±2.51% ±3.27%
 buffers/buffer-concat.js n=1024 withTotalLength=1 pieceSize=1 pieces=16          ***    232.81 %       ±2.64% ±3.53% ±4.63%
 buffers/buffer-concat.js n=1024 withTotalLength=1 pieceSize=1 pieces=4           ***    165.25 %       ±2.23% ±2.99% ±3.92%
 buffers/buffer-concat.js n=1024 withTotalLength=1 pieceSize=256 pieces=1         ***     21.09 %       ±1.66% ±2.22% ±2.88%
 buffers/buffer-concat.js n=1024 withTotalLength=1 pieceSize=256 pieces=16        ***     71.34 %       ±1.11% ±1.47% ±1.92%
 buffers/buffer-concat.js n=1024 withTotalLength=1 pieceSize=256 pieces=4         ***     91.33 %       ±2.32% ±3.10% ±4.07%

This PR is able to be backported to v10 branch. The main question is what to doing with error messages. Some of previous error messages are less informative.

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • commit message follows commit guidelines

Merge request reports