Skip to content

url: significantly improve the performance of the url module

Rodrigo Muino Tomonari requested to merge github/fork/petkaantonov/faster-url-parser into v1.x

See issues: https://github.com/iojs/io.js/issues/643, https://github.com/joyent/node/issues/6788

Original message:

I have rewritten the url parser module of node core as it was/is a serious bottleneck in some of the techempower benchmarks

Running node's urlparser benchmark using iojs it's still 16x faster when not retrieving properties and 11x faster when retrieving all properties (which are lazy getters in my implementation). In absolute terms the current iojs urlparser throughputs 25k parses per second vs 400k lazy/270k eager parses per second. format and resolve are also affected in similar magnitudes.

Testing this PR on my VM ubuntu I get

After

$ bleed benchmark/misc/url.js
misc/url.js parse(): 1.6529e+5
misc/url.js format(): 1.9114e+5
misc/url.js resolve("../foo/bar?baz=boom"): 24302
misc/url.js resolve("foo/bar"): 35696
misc/url.js resolve("http://nodejs.org"): 68003
misc/url.js resolve("./foo/bar?baz"): 29983

$ bleed benchmark/common.js url
url/url-parse.js
url/url-parse.js type=one n=250000: 1244025.74054
url/url-parse.js type=two n=250000: 2071492.21773
url/url-parse.js type=three n=250000: 1113916.06067
url/url-parse.js type=four n=250000: 3416688.31936
url/url-parse.js type=five n=250000: 1620766.16897
url/url-parse.js type=six n=250000: 1659564.59318

url/url-resolve.js
url/url-resolve.js type=one n=100000: 184353.89591

Before

$ git checkout v1.x && make -j4 && bleed benchmark/misc/url.js
misc/url.js parse(): 16510
misc/url.js format(): 18309
misc/url.js resolve("../foo/bar?baz=boom"): 8929.9
misc/url.js resolve("foo/bar"): 9919.5
misc/url.js resolve("http://nodejs.org"): 8177.6
misc/url.js resolve("./foo/bar?baz"): 8984.4

$ bleed benchmark/common.js url
url/url-parse.js
url/url-parse.js type=one n=250000: 82666.55318
url/url-parse.js type=two n=250000: 84669.54973
url/url-parse.js type=three n=250000: 80308.64406
url/url-parse.js type=four n=250000: 262017.65403
url/url-parse.js type=five n=250000: 203834.95043
url/url-parse.js type=six n=250000: 68911.18834

url/url-resolve.js
url/url-resolve.js type=one n=100000: 42331.42854

That's 10-24x faster depending on url.

Caveats

This implementation uses getters/setters on the prototype instead of own properties like current url. This is because often the properties are not needed and take time to calculate. Additionally @chrisdickinson introduced another reason to use setters at https://github.com/iojs/io.js/pull/893#issuecomment-75360607.

This patch is technically backward incompatible but to minimize any compatibility issues I made util._extend to call toJSON, because some core modules pass the Url instance directly to util._extend. The toJSON also means that serialized output of current url and the new url stay the same.

/cc @trevnorris @mikeal

Merge request reports