websocket: proposal for a new core module
This is a proposal to add websockets as a new module for Node.js.
Standards Conformance
This implementation attempts to adhere to RFC 6455. If differences are identified I will supply the necessary changes. This implementation does not attempt to parse connection header values, such as extensions or sub-protocols values.
API
This proposal focuses exclusively on Node's APIs from its core modules and then supplies additional options as necessary to populate callbacks and RFC 6455 connection header values.
This proposal does not attempt to emulate any WHJATWG specification or execute with regards to HTTP. Node is not a web browser. The operating constraints are wildly different, therefore in the best interests of design a WebSocket implementation must adhere to the protocol standard closely, but otherwise isolated from other API concerns. All other API concerns represent additional constraints not necessary for the transport of messages over this protocol.
According to RFC 6455 section 1.7: The WebSocket Protocol is an independent TCP-based protocol. Its only relationship to HTTP is that its handshake is interpreted by HTTP servers as an Upgrade request. That means so long as the as the handshake completes according to the syntax provided HTTP remains completely unrelated. HTTP is a layer 7 (application) technology where WebSocket is defined as a TCP transport mechanism (layer 4). Executing WebSockets over HTTP then creates unnecessary overhead dramatically impacting maintenance, scale, and performance.
Performance
Some notes about performance.
Everybody that writes a websocket library wants to measure performance in terms of message sent speed, which seems to be a red herring. Message send speed is trivial compared to message receive speed.
Message send speed appears to be a memory bound operation. Using this approach to websockets I send at about 180,000 messages per second on my old desktop computer with slow DDR3 memory. On my newer laptop with faster DDR4 memory I can send at about 460,000 - 480,000 messages per second. This speed is largely irrelevant though because at around 460,000 messages garbage collection kicks in and message send speed slows to a crawl at around 5,000 per second.
Message receive speed is much slower and far more dependent upon the speed of the CPU. On my old desktop I can receive messages at a speed of around 18,000 messages per second while on my laptop its a bit slower at around 12,000-15,000 messages per second because the desktop has a more powerful CPU. The speed penalty on message reception appears to be due to analysis for message fragmentation. This approach accounts for 4 types of message fragmentation.
Origin
This implementation is based upon my own logic for my personal application. I have been experimenting with a faster native Node WebSocket implementation for more than 2 years. Origin code at https://github.com/prettydiff/share-file-systems/blob/master/lib/terminal/server/transmission/transmit_ws.ts
Test logic
A means to test and experiment with this proposal with minimal overhead.
const net = require("net");
const ws = require('./websocket.js');
const port = 567;
const host = '';
const secure = true;
const serverOpts = {
ca: `-----BEGIN CERTIFICATE-----
MIIF4jCCA8qgAwIBAgIJAODpjHdE+mQyMA0GCSqGSIb3DQEBDQUAMEQxGDAWBgNV
BAMMD3NoYXJlLWZpbGUtcm9vdDETMBEGA1UECgwKc2hhcmUtZmlsZTETMBEGA1UE
CwwKc2hhcmUtZmlsZTAgFw0yMzA3MTYyMzE3MTlaGA8yMDY4MDUyNDIzMTcxOVow
QjEWMBQGA1UEAwwNc2hhcmUtZmlsZS1jYTETMBEGA1UECgwKc2hhcmUtZmlsZTET
MBEGA1UECwwKc2hhcmUtZmlsZTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoC
ggIBAPMrUIbWSZpn6XeasO57WNF5lYmpOxEOb61GdYq/tawxiRGj6e7wHXmL2ToQ
62rQCGHQ0ylPr9yeUIxDDLwKL85H+kjzZSZzQg/+7shuiNRUBc6sumQuB34ldY/w
vbqv1FnYtLfxFn45WnVFbRmZe3wxHr18cSfWPzb5s7/oga586pY6lcYJO1Y+xGhy
3/jyEr+PAdqV3qSb/yU9qF+TNgaA2sAiDte4Bt5WYeQU7AQFgWflqdT6Dzr+NPin
cTRtXBDe2rIwrdS+gOsuwEqG7OUiac/Ex32hBuDbR3qgpwzVGWRCbW+w31YBr4oY
dPik1gg667ueqH4Jx5YN8YuisjoIouCc4/FJ2Vokh32udfY7Hdb3JSe6PyNvHjTt
qtNYcIzU8bq5rREDuFHNmLDoKD1L0ycZcbO1ftPz3VXXJ8W5R+8/WsUnJx5gZN8k
rnp3Ziq5KnUkCQwqgmqG7f4cQLB7h3Lq6znHjFrrwnauD1+ypHLa+Q7m6L1t4n5P
J2QDOYQSy9J5mfTNXYl99VMB5Xclc0GTak9ymKzKr45yZepeNGmJT9K4GVDNjbHE
GXt+1FX4uwncBlk4jSNKiqnXc9sEAPZ5wClSWOgboOdMTrN1Isi9ZPEzQ2owlUxX
p+QVnh5blot6OhgL77KtM6a+Fs6Dz3mR3cEqv+WZwd0mn7+XAgMBAAGjgdYwgdMw
EgYDVR0TAQH/BAgwBgEB/wIBATAdBgNVHQ4EFgQUqmfvp44Q4p0+BaQhXobyxEDb
Az8wRwYDVR0RBEAwPoIJbG9jYWxob3N0ghhsb2NhbGhvc3QucHJldHR5ZGlmZi5j
b22CEXNoYXJlZmlsZS5zeXN0ZW1zhwR/AAABMFUGA1UdHgROMEygSjALgglsb2Nh
bGhvc3QwGoIYbG9jYWxob3N0LnByZXR0eWRpZmYuY29tMBOCEXNoYXJlZmlsZS5z
eXN0ZW1zMAqHCH8AAAH/AAAAMA0GCSqGSIb3DQEBDQUAA4ICAQADH8+0kYtLBUaM
CET/WTw21AvQgVMNAQil+QbFt0rz567cSL5uM2EVTK2+ERCuXcMBxECTzMz+nYsK
z7nqrA5q9SE9M6q0weOULF/zPETXWAQLmRC2ej67pRIbRn10PoVTJIiyCTXKLEjR
zRgRIiaC1vAxgVq+U7udo6MzpCyVo6JdCIlJQpF7aA9XjQJYbtKSO1cDHYM2dJUc
fw3bzq0yYUodDscM7vD8SNJY+eynsApVFXi6h1TNGj283mVBvihnPq4PTOiRc4KK
iyFIJsT5/mJ3XD+pAHI3dPwe95zz6G1afdL2EGU3OAf7SGQ46LhLMV9L+izcE6zR
SnjaXzjk2rho1XtAityV82GJSUCE71Xv/ePQuEDNV4u0jFzF/uVHjIW40zzLFTYS
hjBf6SdsjwXxOHu/7RgDgRGuYJfabX/CcbpxlocFCLeJUNE0SHcrFpfW4piiFD/o
KPs2UNmuHSorUc7k6UTB8gQSqIjMSEeF6MHhVMcD5dgBCCEMhVWs5tRL/comCRTK
BcRlCOouZJZaBMrseWpAdTl3GTSqLNAsCtqMakvOyxFOTrk/kGhPz3jLs1BTZZQ8
b/boToH371317pu+k0eJ10CXr+W4+xUL9pBSMzaNw96/n5H7sCrWx9DUk3LUoXKN
rYmowESTH/PehBFm3ndXq+zyoX7UGg==
-----END CERTIFICATE-----`,
cert: `-----BEGIN CERTIFICATE-----
MIIF9TCCA92gAwIBAgIJAPAQEdC51VH5MA0GCSqGSIb3DQEBDQUAMEIxFjAUBgNV
BAMMDXNoYXJlLWZpbGUtY2ExEzARBgNVBAoMCnNoYXJlLWZpbGUxEzARBgNVBAsM
CnNoYXJlLWZpbGUwIBcNMjMwNzE2MjMxNzIyWhgPMjA2ODA1MjQyMzE3MjJaMD8x
EzARBgNVBAMMCnNoYXJlLWZpbGUxEzARBgNVBAoMCnNoYXJlLWZpbGUxEzARBgNV
BAsMCnNoYXJlLWZpbGUwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCf
nPdeUGUzUekm3heIyA1lPdqFHQZy5sipxwuJEu5ySV4RJExt+AcHwyj0gMS3olDZ
vMAQspJp05IACJGJ+boeYoTU3WqXHFCf2vxNNFKVZsfE4BuZwICpycB1/Wycj7zF
OJbaqeybnNaNIA9LeNexKL6AN8ngvWlQAlTW4YWpULsqSNz57wyoil6QV55G5WqF
vFCutL1IH8ccYihJnKk28kVvwK58taxOneGfQ8rCwQ/TIW2s+zsX2mzPoA1IXFPE
P2io3X9XjVp/ejotFfHIXsFBWI2wdDZozxObZegQkIQ9xaHnypPVyZO7uzMYNFC2
D8rC3+umZ1jqmiVaUF8Q31Lh38JaHr30Vaq87yMlzA0ouvEmz+3eW88MnOTcS3Pv
CyEW+Yfh2efh01+cfb7nDOGil4GbimAqJj7lPNq8eG84Lv4VBn2pN9ViGTsG1Z97
PItbg3rz+M46adNLKDmGON4UX2fdI3mJgBstuFokcvvNydnXXYLykSobdlzbuUgx
yXD1TfW7Kq5yQWdmeiYwjBhiO4LbszWnhNdOo13fqkYEFTe4Y8CjhF3I96ElZvqH
KBAHqY/VZQ5l0JLdULAGgtdXqHuKbkg5wKnC5TK9Hps/kXUyweNrBfBY5c4j/TWw
+F1IR3gsNY++YnfRzM7QdhmKMeagkbKvTs1CYm8u1QIDAQABo4HuMIHrMAkGA1Ud
EwQCMAAwHQYDVR0OBBYEFNB6IlDpbv18fv34YNsjk+mRv/ugMB8GA1UdIwQYMBaA
FKpn76eOEOKdPgWkIV6G8sRA2wM/MEcGA1UdEQRAMD6CCWxvY2FsaG9zdIIYbG9j
YWxob3N0LnByZXR0eWRpZmYuY29tghFzaGFyZWZpbGUuc3lzdGVtc4cEfwAAATBV
BgNVHR4ETjBMoEowC4IJbG9jYWxob3N0MBqCGGxvY2FsaG9zdC5wcmV0dHlkaWZm
LmNvbTATghFzaGFyZWZpbGUuc3lzdGVtczAKhwh/AAAB/wAAADANBgkqhkiG9w0B
AQ0FAAOCAgEAXMm7ADfL+GJTcj93zMIz8yUFr3IqDgzFRopgh4n9JEDlDmN5FC67
mRomGZ9GfTvkOoct0vdJc54MPmNaxb5ydCtM4/JeeA05zLcG83OWeh4CIGbUoDmf
dResaLeTHNI/Uf+0vuvliQvV/fhSAGU+Ai3EaC4ErYWW1lF4sq7uH06Ru2DoKZcR
ohdLWUAIxIqfYoEXrLQHjJnqXJ2UWQ7Fb1x2sryKsoR02sL/7FvhTAMNj2HdJxqf
thwrA011EzFw0Et14XrigRh136KkmWz2c/PC2UHobixN+cazaN5gWuXdySI43Vqv
GmBzyHV5kTorIS4ACUPweutn5pELKSCtaWcMDT8dlZnIN9nHety3TflAGL7w94Pp
R6GPaus0UP5UVGxy9HbD01EyvT90dO+eA4aYA3XlWKx0HZl78p4MvrHv1A7O2Arm
GYWkhJsz2uyq5HScxUQVTuBQ0yGVvy31eiGgIZWGIWBEVnAHJ8o9VRSqWcWnHrrS
fYKBStVx/kCCG/oY9NVPivIQ2YpYOWpL0fdbRaZTnEfb9TZhfFQtR/2tgBFq40eh
wFtdV2lk6MwjwENxci7w0kdi1YStBDBGHgyQALz3fhLNQy2JFTM6aFVmQoq1/iUB
/X/2uBo7Kl581WjgkBsAVGapwSiK+jSJY5i7wJ9CmOYPrmp0zRPDLus=
-----END CERTIFICATE-----`,
key: `-----BEGIN RSA PRIVATE KEY-----
MIIJKAIBAAKCAgEAn5z3XlBlM1HpJt4XiMgNZT3ahR0GcubIqccLiRLuckleESRM
bfgHB8Mo9IDEt6JQ2bzAELKSadOSAAiRifm6HmKE1N1qlxxQn9r8TTRSlWbHxOAb
mcCAqcnAdf1snI+8xTiW2qnsm5zWjSAPS3jXsSi+gDfJ4L1pUAJU1uGFqVC7Kkjc
+e8MqIpekFeeRuVqhbxQrrS9SB/HHGIoSZypNvJFb8CufLWsTp3hn0PKwsEP0yFt
rPs7F9psz6ANSFxTxD9oqN1/V41af3o6LRXxyF7BQViNsHQ2aM8Tm2XoEJCEPcWh
58qT1cmTu7szGDRQtg/Kwt/rpmdY6polWlBfEN9S4d/CWh699FWqvO8jJcwNKLrx
Js/t3lvPDJzk3Etz7wshFvmH4dnn4dNfnH2+5wzhopeBm4pgKiY+5TzavHhvOC7+
FQZ9qTfVYhk7BtWfezyLW4N68/jOOmnTSyg5hjjeFF9n3SN5iYAbLbhaJHL7zcnZ
112C8pEqG3Zc27lIMclw9U31uyquckFnZnomMIwYYjuC27M1p4TXTqNd36pGBBU3
uGPAo4RdyPehJWb6hygQB6mP1WUOZdCS3VCwBoLXV6h7im5IOcCpwuUyvR6bP5F1
MsHjawXwWOXOI/01sPhdSEd4LDWPvmJ30czO0HYZijHmoJGyr07NQmJvLtUCAwEA
AQKCAgAgFlcob7MYkRP1C1rh1Y3T145xijc8rCaU8v3frZ2f/h3aBlkTFnSbW+GE
3couPIRScX6PHMcQXUcRmKdhfIGtEBMyE90Uyc1vhX+JKcacYFAyxPbnfuqet39o
eOz3wHGrmEfDZ7u4QNxk/Jf2jTGXXOCHOC/ubUWZnw5dMHNFaYRm6MT7vdHmpAKE
tAiOqhozDnuN06nlsPW/QABnZAYklKne4HZzfbZJC7ZK5T8CzfsXb7Xzu4HStsd/
KebhsCXq4vBwWi76c+FIlVLSs4GqzVm+gEXjvkkd4ttHN0Ji6hqbrHpy9aeop+B6
MhUAfavoHd6eNJPUHRyj9R8jO9sQYP6eoGdYghk9HtnvG2RicgO7oGCfLj6ibUrm
rA2ZpWt7BVfTgufTCbzHSIkD245JGkSkMVvjX4fbL87bbtNPddJ0TW4VQKT9r0c2
4EH6LQgxyYZMaLpYJgPN2iUXHnPxght1kQQwVmSg5s1uQC9XAzbulYVOZu2g5/wz
Gd+EePV1SYoleEymDeQKUkzMxPwSE2cD8dL/HU4hQDltcwq0q/btzWlW5xvHampq
JGjsztRZKB1sznE2qqhUkYCzPqPfHkOecZW4Qwuif9xlHe+oAGU3ObcvgBNjO7t/
bPgADEsmKrSjijfVz8CN1JcymB+m9C6vsmbkQNbWumLLrgytYQKCAQEAymji6h8m
gv2RDnAeJ/cu79Ri601bQqdpbul9JDxK3w6U/SF74w5NIXPr33bf5JcPQp9Cj6fr
SdQYy1XxhTSirGm9STvymY0CKki67Kpd3zQoY40yLsRgPw0ZCwSWXnoDXl04/GA0
N6RvXOSJ76RCiQd0M/Lvz2yvOmISdne5LQRPk7SgHyJurmMk7ocl3yh4EJH0FHpf
ulodhLGi7ptRPIhMQ3m2bKtVmXWokQtoZCixpb08DbPNbVOpCrZSrcinwg+AtSnG
wE3XR8IpW7qGmFVvmqgs4nRQcM/goxazIHU+bO2dZx0sepI9li5AUtdpp08QrKn0
I4LOp5/oOWXmXQKCAQEAyd9f6yBCceIYi72fgpJZGCSIaGZEPSCDz4zINcDN1UzN
XDDTou7no9cseCYhVecvHtikvN2UcEi09+3OFhFaV+KyVeysrjj8nuSw0m3BBITN
A59ITZFFvjlytVpjf61JLp6hly/Q+eGrBjJnIPQTkW3asauuXaVBz7xWD3xlhDPo
IuYb0JCHIu4qmCnkvbeJHuCo7v772/+IjkoNKVxgK6U9ZibFqbJfTejfnz5fnLo+
0I9I5pvySRYExFwv3mYBROU0hmMyXJWvGPUuqVckoLa2Ww41eRgvI06Jd5lItnMu
jzCrEs9Pj1d5veh4IevC1cENmbSgT5Tx+xuXWcfy2QKCAQB9dD0Qt3X7Qoah2EQY
qVBiPdWB2lRyH6ltoTJ7PxN45WTa7+IFfVu5HExaGSf0WtyOgn+S4pUnEVq8zOwB
j/ozuuYjehCHs6pf4uxYu8+rBHz0FxO/gN/WtJuNBK7ep+lml4k2g7pZsoWDofMM
oVbL797KRAz3F3oUSaz/2HzhtgZMmmuUYJcRZ0oAvatvgXnJa21JNAAZVLlvAVrn
YUUcq635NHspJ5jKoO512Ag/7CkPfRa3t3XgCTaA+TiNlgzEby9rGhWiI50HUQSp
YhcCXBHsXchUI5uoEHA/JVapC4JBqZUh0Cc9YV7ispATyIgntw2ytzQmvnCv3KDm
0o3RAoIBAQCZ1q9LCGd6T/myrEvdfleFDXoiTSTdjGTGixub0xVI4mFxSwhNF1DR
S83ote4bf7UqBaDtCNLxCodWlRPDP3Agn3KWBmnFz0m8cLzLb7ZzEh0GEKFR8045
25+t0ncWumCVtW+hPmA7vRzO+SQcOcSbxCKv2Qxk8uYHQBg5buwR5liWF9PEig9h
sCwnj21womhNbpluoEQg8EgJXydOiMYFHMSAjzV8z6DPR5L60NaeIlRyLW85xkfK
KIxzc2lLS2LWNPFlJD0hzzQDif0IMY+JJhQrqdVYNfTeLCCYUujVmUs29bi4+eFA
dEIjVgAOoZL1wEv0AXFVlEUfvnQFiFlpAoIBAAxBPddDn/0KvDdctIE6L1XWPQkc
qfijyaQzrvdaEsrI9qZfl9fkub7q2DAZ5WBA+EDG972LtO8DiekOirHdGuMYI6/y
81Lk09z8pFL8GojrYAZgDMv6dA93xNVe+sMxGj40DakVXrAFDsMWCKFvfkdTNDHR
vlnznkTSibKhgVS+06j7vzglC1SqxY45TR4azVoSfDm+gkBSVouO70pR48lOREde
tjZ7ebCYzw7V6PDc13QfdIJfStWxgkPH2WrTcWJgUXm0dv+vmdDn+ERkA/XZZfkO
HfpjsSNvOKaKb9oaq5RH+DRBnexJAqholn8kTKYpxpfw2s4Bg7bZXIrMC+E=
-----END RSA PRIVATE KEY-----`
};
// change the value of variable 'secure' to switch between net and tls operation
// server logic, execute as:
// node ./tester.js server
if (process.argv.includes('server')) {
const server = ws.server({
callbackConnect: function(headerValues, socket, ready) {
//console.log("server callbackConnect, followed by header values");
//console.log(headerValues);
ready();
},
callbackListener: function(server) {
console.log("server callbackListener");
},
callbackOpen: function(err, socket) {
console.log("server callbackOpen");
},
messageHandler: function(message) {
console.log("message received at server");
console.log(message.toString());
},
listenerOptions: {
host: host,
port: port
},
secure: secure,
serverOptions: serverOpts
});
}
// client logic, execute as:
// node ./tester.js client
if (process.argv.includes('client')) {
// unused throwaway server to halt thread termination
const server = ws.server({
callbackConnect: function(headerValues, socket, ready) {
//console.log("server callbackConnect, followed by header values");
//console.log(headerValues);
// ready();
},
callbackListener: function(server) {
// console.log("server callbackListener");
},
callbackOpen: function(err, data) {
console.log("server callbackOpen, followed by http string");
console.log(data);
},
messageHandler: function(message) {
console.log("message received at server");
console.log(message.toString());
},
listenerOptions: {
host: host,
port: 777
},
secure: secure,
serverOptions: serverOpts
});
const client = ws.clientConnect({
authentication: 'auth-string',
callbackOpen: function(err, socket) {
console.log("client callbackOpen");
if (err === null) {
socket.messageSend("hello from client");
}
},
masking: false,
id: 'id-string',
messageHandler: function(message){
console.log("message received at client");
console.log(message.toString());
},
'proxy-authorization': 'proxy-auth',
subProtocol: 'sub-proto',
type: 'type-string',
secure: secure,
socketOptions: {
host: host,
port: port,
rejectUnauthorized: false
}
});
}