url: significantly improve the performance of the url module
See issues: https://github.com/iojs/io.js/issues/643, https://github.com/joyent/node/issues/6788
Original message:
I have rewritten the url parser module of node core as it was/is a serious bottleneck in some of the techempower benchmarks
Running node's urlparser benchmark using iojs it's still 16x faster when not retrieving properties and 11x faster when retrieving all properties (which are lazy getters in my implementation). In absolute terms the current iojs urlparser throughputs 25k parses per second vs 400k lazy/270k eager parses per second. format
and resolve
are also affected in similar magnitudes.
Testing this PR on my VM ubuntu I get
After
$ bleed benchmark/misc/url.js
misc/url.js parse(): 1.6529e+5
misc/url.js format(): 1.9114e+5
misc/url.js resolve("../foo/bar?baz=boom"): 24302
misc/url.js resolve("foo/bar"): 35696
misc/url.js resolve("http://nodejs.org"): 68003
misc/url.js resolve("./foo/bar?baz"): 29983
$ bleed benchmark/common.js url
url/url-parse.js
url/url-parse.js type=one n=250000: 1244025.74054
url/url-parse.js type=two n=250000: 2071492.21773
url/url-parse.js type=three n=250000: 1113916.06067
url/url-parse.js type=four n=250000: 3416688.31936
url/url-parse.js type=five n=250000: 1620766.16897
url/url-parse.js type=six n=250000: 1659564.59318
url/url-resolve.js
url/url-resolve.js type=one n=100000: 184353.89591
Before
$ git checkout v1.x && make -j4 && bleed benchmark/misc/url.js
misc/url.js parse(): 16510
misc/url.js format(): 18309
misc/url.js resolve("../foo/bar?baz=boom"): 8929.9
misc/url.js resolve("foo/bar"): 9919.5
misc/url.js resolve("http://nodejs.org"): 8177.6
misc/url.js resolve("./foo/bar?baz"): 8984.4
$ bleed benchmark/common.js url
url/url-parse.js
url/url-parse.js type=one n=250000: 82666.55318
url/url-parse.js type=two n=250000: 84669.54973
url/url-parse.js type=three n=250000: 80308.64406
url/url-parse.js type=four n=250000: 262017.65403
url/url-parse.js type=five n=250000: 203834.95043
url/url-parse.js type=six n=250000: 68911.18834
url/url-resolve.js
url/url-resolve.js type=one n=100000: 42331.42854
That's 10-24x faster depending on url.
Caveats
This implementation uses getters/setters on the prototype instead of own properties like current url. This is because often the properties are not needed and take time to calculate. Additionally @chrisdickinson introduced another reason to use setters at https://github.com/iojs/io.js/pull/893#issuecomment-75360607.
This patch is technically backward incompatible but to minimize any compatibility issues I made util._extend
to call toJSON
, because some core modules pass the Url instance directly to util._extend
. The toJSON
also means that serialized output of current url and the new url stay the same.
/cc @trevnorris @mikeal