Skip to content

benchmark: add improvement and t-test in Node

Added the capability of calculate the improvement, the p.value and the confidence directly from node, for those that wants to check the performance impact of their changes without installing R.

This is not a replacement for the R scripts, still needed for the scatters and for having a more reliable results. But is a good way of avoid the R dependency when you only want to measure the improvement of your changes. Important note: the p.value calculated by R and the one calculated by the Student's test implementation in node are not equal, but pretty similar enough in scale, so the confidence should be the same. This is due to not knowing exactly how is implemented the Student's test in R, and also because the approximation of the gamma function.

The compare.js script works as always, but now you have a new parameter --stats. When this parameter is added, the csv lines are not showed in the console, and when all the jobs are processed the stats are calculated and shown in the console in the same format as the R script does.

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • commit message follows commit guidelines
Affected core subsystem(s)

Merge request reports

Loading