Static buffer for onData

[uasan]

A couple of years ago, the developers of the node added a reasonable way to read a socket from a static buffer. This reduces memory allocations for temporary buffer copies and reduces work GC.
nodejs/node#25436

Does this make sense, for read HTTP body?
If yes, I suggest adding an onRead handler, making the interface the same as in Node.js
https://nodejs.org/api/net.html#net_socket_connect_options_connectlistener

{
  ...,
  onread: {
    // Reuses a 4KiB Buffer for every read from the socket.
    buffer: Buffer.alloc(4 * 1024),
    callback: function(nread, buf) {
      // Received data is available in `buf` from 0 to `nread`.
      console.log(buf.toString('utf8', 0, nread));
    }
  }
});

This is not applicable for us, we already know how to do things correctly. And it still seems they don't really have any real idea what they are doing.

[uasan]

I agree that there is almost no sense for us.
But for example, when a file is uploaded, if the file size is 10Mb, and the transfer will be in 10 iterations of onData, then because of Buffer.concat we will spend 55 Mb (9 chunks copies = 45 Mb)

[e3dio]

@uasan But you are suggesting for your example to allocate at the start 10Mb buffer size for every request to your endpoint, request could cancel before completes, onData only uses the ram required at the time. Maybe for large sizes should write to disk instead buffer in ram

[uasan]

@e3dio No, I suggest for each connection (accept socket) to define a function buffer or static object (this is less flexible)
uWS always writes the contents of the socket to the buffer that I prepared in advance.

If we know the value HTTP header content-length, we can prepare a buffer of the required size and we will write to it, we do not need to copy

[uasan]

Resizable and Growable ArrayBuffer, will appear soon in V8, and then it will become even more convenient to use static ArrayBuffers for each socket
https://github.com/tc39/proposal-resizablearraybuffer

[uasan]

The buffer function gives flexibility, we can slice subarray from a static buffer, its sizes will depend on the request, for the upload of a file it is enough to slice by one chunk and save it to a file.

My main point is that we can reuse memory, for example, 1k RPS JSON (one request body 1Kb) creates 1MB of garbage in 1 second, this is not critical, but it makes no sense )

[e3dio]

1KB Json string only needs 1 chunk and is zero copy, JSON.parse(Buffer.from(ab).toString()) or JSON.parse(await req.body)

[uasan]

Yes this is an ideal case when the whole body is transferred in one call, but for a static buffer it will work just as well.

Http is a stream, there is no guarantee you'll know the total size of the body up front. That's why the interface is giving you chunks and whether or not the chunk is last.

What you do with these chunks is up to you, nobody forces you to use Buffer.concat. You can use whatever you think is best.

If you have a content-length header you can pre-allocate a buffer and fill it. That's already possible today.

[uasan]

What you do with these chunks is up to you, nobody forces you to use Buffer.concat. You can use whatever you think is best.

Yes, you are right
That's what I do, I have a single static buffer for small slices in pool and I reuse them.

I closed the ticket, but I am writing thoughts here, maybe someone will be useful.

My main point is that we can reuse memory, for example, 1k RPS JSON (one request body 1Kb) creates 1MB of garbage in 1 second, this is not critical, but it makes no sense )

This is not true. You are given an ArrayBuffer that is zero copy, a reference. There is no garbage collection in this library and your links to discussions in Node.js camp are not applicable. Inspiration and advice on performance is worth less than the dog shit under your shoe, if it comes from the Node.js people. These people have no vision, no idea about anything.

These aspects, memory management, are architectural and will not change. How it works now, is how it will remain.

[uasan]

Copies of buffers are made at the Node.js level of the application, I'm sure everything is fine on your part )
In the documentation and examples, Buffer.concat is always used in onData.

[e3dio]

Buffer.concat is always used in onData

No Buffer.concat is only used when needed or wanted. Single chunk posts like your 1KB Json example are zero copy JSON.parse(Buffer.from(ab).toString()). If you are using concat for that you are making unnecessary copy of data. If you want to pre-allocate buffer you can use Buffer.allocUnsafe() and Buffer.from(arrayBuffer).copy(allocatedBuffer,...) but for single chunk this is also unnecessary copy