pipeline does not run callback when finished
Version
v14.16.0 (including latest version of node v14)
Platform
Darwin Kernel Version 20.6.0 (os x 11.6)
Subsystem
stream
What steps will reproduce the bug?
We have a custom protocol which operates similar to websockets. Using server.on('upgrade')
, we use the socket passed to the callback of the upgrade event in a call to pipeline similar to below:
server.on('upgrade', (req, socket) => {
verifyCredentials(...)
.then(authorizedClient => {
const protocol = new Protocol(authorizedClient);
pipeline(
socket,
new Parser(),
protocol,
socket,
err => {
if (err) {
// log error
}
protocol.markAsEnded()
}
);
How often does it reproduce? Is there a required condition?
Reproduceable by calling .end()
on the socket object. I have been stepping through pipeline.js in a debugger and it looks like none of the streams are destroyed unless an error has occurred which may be causing our issues.
It looks like pipeline.js has changed significantly between v10 and v14. As such I have no idea if this is a bug in pipeline.js or if it is a result of a breaking change for which I need to modify our code to support.
What is the expected behavior?
In node v10, this worked fine and the callback passed to pipeline
would run anytime the pipe breaks, an error occurs, or the connection is closed using .end()
or .destroy()
on the protocol
object or the socket
.
What do you see instead?
However, in node v14, the callback only runs if the connection is destroyed prematurely from inside the script (using protocol.destroy()
). The easiest way I have found to reproduce the bug has been to call .end()
on either the protocol stream object or the socket itself. This will probably work with a passthrough stream but I haven't tried that yet.
Additional information
Occasionally, connections in production break due to TCP timeouts or connection resets. Node v10 handled these fine. The pipeline callback would always run. However, in node v14, we initially noticed issues when testing out the latest version of node v14 in a production environment and saw that these errors were not getting logged and clients would appear to be online even though the underlying TCP connection had already been destroyed because the callback didn't run. It wasn't until we went to restart the service when the http server was given the signal to shutdown when all of sudden the pipeline callback ran for all the broekn connections and all the "Premature close" and "read ECONNRESET" error messages appeared in our logs.
Each connection is from a uniquely identified "person". If another client connects using credentials that match up to the same "person", then that "person" will not be able to connect until the existing connection matching them is removed. We rely on this to ensure data synchronization between the client and server for our use case.