Skip to content

benchmark: ignore significance when using --runs 1

Rodrigo Muino Tomonari requested to merge github/fork/AndreasMadsen/benchmark-one-observation into master
Checklist
  • commit message follows commit guidelines
Affected core subsystem(s)

benchmark

Description of change

Because the standard deviation can't be calculated when there is only one observation the R scripts raises an error. However it may still be useful to run them for non-statistical purposes.

This changes the behaviour such when there is only one observation, the values that depends on the standard deviation becomes Not Applicable (NA).

Fixes: #8288 (closed)

/cc @mscdex

examples:

                                                               improvement significant   p.value
 http/check_is_http_token.js n=50000000 key="Content-Encoding"      2.42 %          NA        NA
 http/check_is_http_token.js n=50000000 key="Content-encoding"      1.79 %             0.8908776
 chunks length     rate confidence.interval
      0      4 12834.16            5850.699
      1      4 11944.62                  NA

Merge request reports