Skip to content

benchmark: add secure-pair benchmark

Rodrigo Muino Tomonari requested to merge github/fork/alexfernandez/master into master
Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

Adds a benchmark to compare tls.createSecurePair() with new tls.TLSSocket(). The first option is deprecated, while the second is the recommended way forward for securing existing server sockets. A proxy server is placed between the client and the server to evidence any performance issues with socket events.

Results v8.11.1

tls.createSecurePair() with Node.js v8.11.1 is quite faster than new tls.TLSSocket():

 $ node benchmark/tls/secure-pair.js 
tls/secure-pair.js size=2 securing="SecurePair" dur=5: 15.94602188948082
tls/secure-pair.js size=1024 securing="SecurePair" dur=5: 2,060.951341694148
tls/secure-pair.js size=1048576 securing="SecurePair" dur=5: 3,234.913943348519
tls/secure-pair.js size=2 securing="TLSSocket" dur=5: 3.1348338992549807
tls/secure-pair.js size=1024 securing="TLSSocket" dur=5: 1,020.9719597316911
tls/secure-pair.js size=1048576 securing="TLSSocket" dur=5: 3,704.526017262844

Deprecation warnings have been removed from the output. Test machine is a dual-core Lenovo X1 Carbon with Ubuntu 17.10.

There is a performance regression using new tls.TLSSocket() for small packet sizes:

  • For 2 bytes throughput goes from 16 bytes per iteration to around 3.
  • For 1 KB throughput goes from 2 KB per iteration to 1 KB.
  • For 1 MB throughput actually increases, which is to be expected since there are less streams to pipe there.

The results point to an inefficiency in small packets, and experimentally it appears that while SecurePair buffers data, TLSSocket doesn't: the number of bytes per iteration is quite similar to the number of bytes per packet sent.

Results Using master

Also, a performance regression has been reported in https://github.com/nodejs/node/issues/20263 for v10.0.0, in which removes the old legacy code for tls.createSecurePair() using instead new tls.TLSSocket() underneath. So using the latest in master (node v11.0.0-pre):

 $ ./node benchmark/tls/secure-pair.js 
tls/secure-pair.js size=2 securing="SecurePair" dur=5: 3.612850011228876
tls/secure-pair.js size=1024 securing="SecurePair" dur=5: 1,073.6013989026228
tls/secure-pair.js size=1048576 securing="SecurePair" dur=5: 3,801.1694627588176
tls/secure-pair.js size=2 securing="TLSSocket" dur=5: 3.1985467121541213
tls/secure-pair.js size=1024 securing="TLSSocket" dur=5: 1,023.6285765524207
tls/secure-pair.js size=1048576 securing="TLSSocket" dur=5: 4,079.423755357186

In master there is no performance gap between SecurePair and TLSSocket; instead they are both in the low end of the range seen with v8.11.1.

As requested by @addaleax, this benchmark should work reliably on all systems since it does not vary with the number of cores or indeed with CPU load (within reason).

Results With #20287

As requested by @benjamingr, applying https://github.com/nodejs/node/pull/20287 by @apapirovski gives very similar numbers as without it. I don't have an explanation as to why it does not improve as expected.

 $ ./node benchmark/tls/secure-pair.js 
tls/secure-pair.js size=2 securing="SecurePair" dur=5: 3.4185875822957215
tls/secure-pair.js size=1024 securing="SecurePair" dur=5: 1,144.471090251163
tls/secure-pair.js size=1048576 securing="SecurePair" dur=5: 3,647.8361764068845
tls/secure-pair.js size=2 securing="TLSSocket" dur=5: 2.833589476787596
tls/secure-pair.js size=1024 securing="TLSSocket" dur=5: 1,026.9533974461256
tls/secure-pair.js size=1048576 securing="TLSSocket" dur=5: 3,879.9676141790346

Deployment Notes

PR should be safe to merge since it only adds a benchmark.

Merge request reports