Skip to content

buffer: add fast Buffer.encode() for simple string conversion

Rodrigo Muino Tomonari requested to merge github/fork/jasnell/buffer-encode into master
Checklist
  • tests and code linting passes
  • a test and/or benchmark is included
  • documentation is changed or added
  • the commit message follows commit guidelines
Affected core subsystem(s)

buffer

Description of change

/cc @trevnorris

Adds Buffer.encode(str, str_enc, target_enc) as a fast, simple replacement for the pattern Buffer.from(str, str_enc).toString(target_enc). The implementation is at least twice as efficient and doesn't require creating a new Buffer instance for straightforward conversions.

// Instead of:
Buffer.from('test').toString('base64');

// It's
Buffer.encode('test', 'base64');

Benchmark, test and doc included.

Benchmark numbers:

buffers/buffer-encode.js method=encode enc=utf8 size=16 millions=1: 3.14952
buffers/buffer-encode.js method=encode enc=utf8 size=512 millions=1: 1.72794
buffers/buffer-encode.js method=encode enc=utf8 size=1024 millions=1: 1.27162
buffers/buffer-encode.js method=encode enc=utf8 size=4096 millions=1: 0.42907
buffers/buffer-encode.js method=encode enc=utf16 size=16 millions=1: 3.01668
buffers/buffer-encode.js method=encode enc=utf16 size=512 millions=1: 1.66038
buffers/buffer-encode.js method=encode enc=utf16 size=1024 millions=1: 1.26316
buffers/buffer-encode.js method=encode enc=utf16 size=4096 millions=1: 0.48477
buffers/buffer-encode.js method=encode enc=hex size=16 millions=1: 3.11447
buffers/buffer-encode.js method=encode enc=hex size=512 millions=1: 1.68086
buffers/buffer-encode.js method=encode enc=hex size=1024 millions=1: 1.34287
buffers/buffer-encode.js method=encode enc=hex size=4096 millions=1: 0.47781
buffers/buffer-encode.js method=encode enc=base64 size=16 millions=1: 3.02440
buffers/buffer-encode.js method=encode enc=base64 size=512 millions=1: 1.71211
buffers/buffer-encode.js method=encode enc=base64 size=1024 millions=1: 1.27482
buffers/buffer-encode.js method=encode enc=base64 size=4096 millions=1: 0.47746
buffers/buffer-encode.js method=tostring enc=utf8 size=16 millions=1: 1.20082
buffers/buffer-encode.js method=tostring enc=utf8 size=512 millions=1: 0.76192
buffers/buffer-encode.js method=tostring enc=utf8 size=1024 millions=1: 0.51062
buffers/buffer-encode.js method=tostring enc=utf8 size=4096 millions=1: 0.26153
buffers/buffer-encode.js method=tostring enc=utf16 size=16 millions=1: 1.32567
buffers/buffer-encode.js method=tostring enc=utf16 size=512 millions=1: 0.75700
buffers/buffer-encode.js method=tostring enc=utf16 size=1024 millions=1: 0.57700
buffers/buffer-encode.js method=tostring enc=utf16 size=4096 millions=1: 0.26391
buffers/buffer-encode.js method=tostring enc=hex size=16 millions=1: 1.28990
buffers/buffer-encode.js method=tostring enc=hex size=512 millions=1: 0.74679
buffers/buffer-encode.js method=tostring enc=hex size=1024 millions=1: 0.55791
buffers/buffer-encode.js method=tostring enc=hex size=4096 millions=1: 0.26711
buffers/buffer-encode.js method=tostring enc=base64 size=16 millions=1: 1.32528
buffers/buffer-encode.js method=tostring enc=base64 size=512 millions=1: 0.75487
buffers/buffer-encode.js method=tostring enc=base64 size=1024 millions=1: 0.52509
buffers/buffer-encode.js method=tostring enc=base64 size=4096 millions=1: 0.26081

Perfectly ok if this doesn't land in core but the perf difference is compelling.

Merge request reports