Skip to content

http: added scheduling option to http agent

Rodrigo Muino Tomonari requested to merge github/fork/delvedor/lifo-agent into master

Hello everyone!

This pr introduces a new configuration option in the HTTP agent, named scheduling. By default, the agent iterates over the free sockets in a FIFO fashion, the newly added scheduling option allows using a LIFO instead. If nothing is configured, the behavior will be the same as before, so this change should not be considered as breaking.

Why this is useful?

I've encountered a case where the application was sending requests at a high rate (around 200 req/sec), with low response time. Every now and then, the response time increased massively for a few seconds and the agent started opening as many sockets as it could, later, the response time was low again, and only a few sockets were used at a given time. The issue is that the agent will not close the freeSockets, because since the default scheduling sequence is FIFO, every socket will be reused before it reaches its timeout. This situation was happening in a system with multiple instances of Node, and the receiver of the request was overwhelmed by the high number of open and never closed sockets.

By switching to a LIFO scheduling, I saw that the spikes were handled as expected, but as soon as the response time was low again, the freeSockets start decreasing, and only the minimum amount of sockets was used.

Given that both scheduling strategies have pros and cons, I've decided to expose them as an option, so the user can have more control based on their needs.

How does it work?

The user only need to configure the scheduling option, only fifo and lifo are accepted.

const agent = new http.Agent({
  keepAlive: true,
  maxSockets: 256,
  maxFreeSockets: 256,
  timeout: 30000,
  scheduling: 'lifo'
})

I didn't update the documentation yet, I'll do that once we settle on the feature and the implementation :) If possible, I would love to see this change landing both in Node.js v12 and v14.

Related: https://github.com/nodejs/node/pull/31526 cc @mcollina @ronag

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

Merge request reports