Skip to content

benchmark: add undici websocket benchmark

Rodrigo Muino Tomonari requested to merge github/fork/Ch3nYuY/ws-bench into main

Refs: https://github.com/nodejs/performance/issues/114

I added a simple benchmark for the undici websocket. I modified the client in https://github.com/websockets/ws/blob/master/bench/speed.js to use undici instead.

Platform Linux benchmark-S2600WFT 5.15.0-41-generic #44-Ubuntu SMP Wed Jun 22 14:20:53 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

20.x vs 21.x

➜  websocket git: ✗ node-benchmark-compare compare-v20-v21.csv
                                                                   confidence improvement accuracy (*)    (**)   (***)
websocket/simple.js size=1048576 roundtrips=1 useBinary='false'           ***     -4.76 %       ±2.08%  ±3.06%  ±4.65%
websocket/simple.js size=1048576 roundtrips=1 useBinary='true'            ***     -3.67 %       ±0.89%  ±1.42%  ±2.50%
websocket/simple.js size=1048576 roundtrips=100 useBinary='false'         ***      3.80 %       ±1.05%  ±1.57%  ±2.47%
websocket/simple.js size=1048576 roundtrips=100 useBinary='true'                   2.15 %       ±2.42%  ±3.61%  ±5.65%
websocket/simple.js size=1048576 roundtrips=1000 useBinary='false'                 2.45 %       ±3.38%  ±5.38%  ±9.37%
websocket/simple.js size=1048576 roundtrips=1000 useBinary='true'         ***      2.92 %       ±0.96%  ±1.41%  ±2.15%
websocket/simple.js size=1048576 roundtrips=5000 useBinary='false'          *      3.18 %       ±3.04%  ±4.56%  ±7.20%
websocket/simple.js size=1048576 roundtrips=5000 useBinary='true'           *      2.19 %       ±2.03%  ±3.05%  ±4.82%
websocket/simple.js size=131072 roundtrips=1 useBinary='false'             **     -7.30 %       ±4.48%  ±6.73% ±10.66%
websocket/simple.js size=131072 roundtrips=1 useBinary='true'              **     -7.36 %       ±3.40%  ±4.99%  ±7.60%
websocket/simple.js size=131072 roundtrips=100 useBinary='false'          ***     10.01 %       ±3.04%  ±4.94%  ±8.92%
websocket/simple.js size=131072 roundtrips=100 useBinary='true'           ***     10.70 %       ±4.10%  ±6.30% ±10.37%
websocket/simple.js size=131072 roundtrips=1000 useBinary='false'                  1.93 %       ±1.97%  ±2.88%  ±4.35%
websocket/simple.js size=131072 roundtrips=1000 useBinary='true'            *      1.97 %       ±1.73%  ±2.58%  ±4.03%
websocket/simple.js size=131072 roundtrips=5000 useBinary='false'                  0.78 %       ±2.01%  ±2.94%  ±4.46%
websocket/simple.js size=131072 roundtrips=5000 useBinary='true'            *      1.79 %       ±1.67%  ±2.53%  ±4.09%
websocket/simple.js size=16384 roundtrips=1 useBinary='false'              **     -2.66 %       ±1.57%  ±2.29%  ±3.44%
websocket/simple.js size=16384 roundtrips=1 useBinary='true'                      -1.60 %       ±3.18%  ±5.06%  ±8.81%
websocket/simple.js size=16384 roundtrips=100 useBinary='false'            **     11.25 %       ±4.08%  ±6.69% ±12.23%
websocket/simple.js size=16384 roundtrips=100 useBinary='true'            ***     12.92 %       ±4.48%  ±6.53%  ±9.81%
websocket/simple.js size=16384 roundtrips=1000 useBinary='false'           **      6.52 %       ±3.31%  ±5.26%  ±9.16%
websocket/simple.js size=16384 roundtrips=1000 useBinary='true'           ***      6.92 %       ±0.66%  ±0.97%  ±1.47%
websocket/simple.js size=16384 roundtrips=5000 useBinary='false'                   2.79 %       ±4.68%  ±6.88% ±10.51%
websocket/simple.js size=16384 roundtrips=5000 useBinary='true'                    3.22 %       ±3.66%  ±6.05% ±11.24%
websocket/simple.js size=64 roundtrips=1 useBinary='false'                  *     -3.62 %       ±2.36%  ±3.78%  ±6.64%
websocket/simple.js size=64 roundtrips=1 useBinary='true'                   *     -3.05 %       ±2.39%  ±3.72%  ±6.28%
websocket/simple.js size=64 roundtrips=100 useBinary='false'              ***     12.20 %       ±5.13%  ±7.46% ±11.23%
websocket/simple.js size=64 roundtrips=100 useBinary='true'                **     10.84 %       ±4.84%  ±7.83% ±14.05%
websocket/simple.js size=64 roundtrips=1000 useBinary='false'             ***      9.51 %       ±0.92%  ±1.42%  ±2.32%
websocket/simple.js size=64 roundtrips=1000 useBinary='true'                *      7.52 %       ±6.22% ±10.26% ±19.04%
websocket/simple.js size=64 roundtrips=5000 useBinary='false'              **      1.62 %       ±0.90%  ±1.31%  ±1.97%
websocket/simple.js size=64 roundtrips=5000 useBinary='true'                *      1.33 %       ±1.06%  ±1.54%  ±2.34%
 
Be aware that when doing many comparisons the risk of a false-positive result increases.
In this case, there are 32 comparisons, you can thus expect the following amount of false-positive results:
  1.60 false positives, when considering a   5% risk acceptance (*, **, ***),
  0.32 false positives, when considering a   1% risk acceptance (**, ***),
  0.03 false positives, when considering a 0.1% risk acceptance (***)

Merge request reports