Skip to content

src: improve buffer.transcode performance

Rodrigo Muino Tomonari requested to merge github/fork/anonrig/improve-transcode-performance into main

Let's improve buffer.transcode performance for UTF8 to UTF16le and UTF16le to UTF8 encodings. I'm working on similar implementation on workerd at https://github.com/cloudflare/workerd/pull/2462.

Benchmark CI: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1591/

                                                                                          confidence improvement accuracy (*)   (**)  (***)
buffers/buffer-transcode.js n=100000 length=1 toEncoding='ascii' fromEncoding='latin1'                   -0.73 %       ±1.12% ±1.49% ±1.94%
buffers/buffer-transcode.js n=100000 length=1 toEncoding='ascii' fromEncoding='ucs2'             ***      2.23 %       ±1.18% ±1.57% ±2.05%
buffers/buffer-transcode.js n=100000 length=1 toEncoding='ascii' fromEncoding='utf8'                      0.50 %       ±1.16% ±1.55% ±2.01%
buffers/buffer-transcode.js n=100000 length=1 toEncoding='latin1' fromEncoding='ascii'                   -1.05 %       ±1.13% ±1.51% ±1.97%
buffers/buffer-transcode.js n=100000 length=1 toEncoding='latin1' fromEncoding='ucs2'                     0.45 %       ±0.76% ±1.01% ±1.31%
buffers/buffer-transcode.js n=100000 length=1 toEncoding='latin1' fromEncoding='utf8'                    -0.59 %       ±1.50% ±2.00% ±2.61%
buffers/buffer-transcode.js n=100000 length=1 toEncoding='ucs2' fromEncoding='ascii'             ***     26.86 %       ±1.08% ±1.44% ±1.87%
buffers/buffer-transcode.js n=100000 length=1 toEncoding='ucs2' fromEncoding='latin1'            ***     28.85 %       ±0.82% ±1.09% ±1.43%
buffers/buffer-transcode.js n=100000 length=1 toEncoding='ucs2' fromEncoding='utf8'                *      0.82 %       ±0.77% ±1.03% ±1.34%
buffers/buffer-transcode.js n=100000 length=1 toEncoding='utf8' fromEncoding='ascii'               *     -1.15 %       ±1.13% ±1.51% ±1.96%
buffers/buffer-transcode.js n=100000 length=1 toEncoding='utf8' fromEncoding='latin1'                    -0.80 %       ±1.20% ±1.60% ±2.08%
buffers/buffer-transcode.js n=100000 length=10 toEncoding='ascii' fromEncoding='latin1'                   0.01 %       ±1.21% ±1.61% ±2.10%
buffers/buffer-transcode.js n=100000 length=10 toEncoding='ascii' fromEncoding='ucs2'            ***      1.57 %       ±0.63% ±0.84% ±1.09%
buffers/buffer-transcode.js n=100000 length=10 toEncoding='ascii' fromEncoding='utf8'                    -0.12 %       ±1.04% ±1.39% ±1.81%
buffers/buffer-transcode.js n=100000 length=10 toEncoding='latin1' fromEncoding='ascii'                  -0.66 %       ±1.00% ±1.33% ±1.73%
buffers/buffer-transcode.js n=100000 length=10 toEncoding='latin1' fromEncoding='ucs2'                   -0.47 %       ±0.66% ±0.88% ±1.14%
buffers/buffer-transcode.js n=100000 length=10 toEncoding='latin1' fromEncoding='utf8'                    0.47 %       ±1.44% ±1.92% ±2.50%
buffers/buffer-transcode.js n=100000 length=10 toEncoding='ucs2' fromEncoding='ascii'            ***     27.27 %       ±0.86% ±1.15% ±1.50%
buffers/buffer-transcode.js n=100000 length=10 toEncoding='ucs2' fromEncoding='latin1'           ***     28.94 %       ±1.06% ±1.41% ±1.85%
buffers/buffer-transcode.js n=100000 length=10 toEncoding='ucs2' fromEncoding='utf8'                     -0.87 %       ±0.92% ±1.23% ±1.60%
buffers/buffer-transcode.js n=100000 length=10 toEncoding='utf8' fromEncoding='ascii'                    -0.23 %       ±1.08% ±1.44% ±1.87%
buffers/buffer-transcode.js n=100000 length=10 toEncoding='utf8' fromEncoding='latin1'             *     -1.51 %       ±1.19% ±1.58% ±2.06%
buffers/buffer-transcode.js n=100000 length=1000 toEncoding='ascii' fromEncoding='latin1'          *     -1.00 %       ±0.81% ±1.07% ±1.39%
buffers/buffer-transcode.js n=100000 length=1000 toEncoding='ascii' fromEncoding='ucs2'          ***      1.79 %       ±0.36% ±0.49% ±0.64%
buffers/buffer-transcode.js n=100000 length=1000 toEncoding='ascii' fromEncoding='utf8'                  -0.60 %       ±0.79% ±1.05% ±1.36%
buffers/buffer-transcode.js n=100000 length=1000 toEncoding='latin1' fromEncoding='ascii'                 0.04 %       ±0.65% ±0.86% ±1.12%
buffers/buffer-transcode.js n=100000 length=1000 toEncoding='latin1' fromEncoding='ucs2'                  0.02 %       ±0.38% ±0.51% ±0.68%
buffers/buffer-transcode.js n=100000 length=1000 toEncoding='latin1' fromEncoding='utf8'           *     -0.83 %       ±0.82% ±1.09% ±1.42%
buffers/buffer-transcode.js n=100000 length=1000 toEncoding='ucs2' fromEncoding='ascii'          ***     17.87 %       ±0.69% ±0.93% ±1.22%
buffers/buffer-transcode.js n=100000 length=1000 toEncoding='ucs2' fromEncoding='latin1'         ***      8.76 %       ±0.71% ±0.95% ±1.24%
buffers/buffer-transcode.js n=100000 length=1000 toEncoding='ucs2' fromEncoding='utf8'           ***     27.92 %       ±0.62% ±0.82% ±1.07%
buffers/buffer-transcode.js n=100000 length=1000 toEncoding='utf8' fromEncoding='ascii'                   0.26 %       ±0.34% ±0.46% ±0.60%
buffers/buffer-transcode.js n=100000 length=1000 toEncoding='utf8' fromEncoding='latin1'                 -0.01 %       ±0.46% ±0.61% ±0.79%

Merge request reports